Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 morluto
|
0992cb3ceb |
No files matched your search
@@ -1,113 +0,0 @@
|
||||
---
|
||||
name: rea-analysis-optimization
|
||||
description: Optimize REA providers and shared analysis workflows for agent context, latency, CPU, and memory while preserving correctness and Evidence. Use for analysis bottlenecks and cache or query design; CI/test-suite optimization belongs to rea-ci-tests-audit.
|
||||
---
|
||||
|
||||
# REA analysis optimization
|
||||
|
||||
Find a concrete source of unnecessary work, change it at its owner, and verify
|
||||
equivalent completed investigations. A useful outcome can also be a measured
|
||||
decision to leave an implementation alone.
|
||||
|
||||
Follow the checkout's [performance priorities](../../../AGENTS.md#performance),
|
||||
[tool design](../../../docs/tool-design.md), and
|
||||
[testing guide](../../../docs/testing.md). This skill supplies an optimization
|
||||
workflow, not additional permission to execute targets, publish, or merge.
|
||||
Use the current contracts, provider prerequisites, and user authorization.
|
||||
|
||||
Read [the Kuna case study](references/kuna-case-study.md) when selecting a
|
||||
reuse strategy or interpreting a benchmark. Its pinned observations illustrate
|
||||
tradeoffs; they are not current-provider guarantees or universal speedups.
|
||||
|
||||
## Pin the question and the correctness oracle
|
||||
|
||||
Describe the analyst's completed task, including the facts and relationships
|
||||
needed to answer it, requested detail, source access, and meaningful unknowns.
|
||||
Choose a representative real artifact and a small or empty case. Record the
|
||||
baseline revision, target digest, engine identity, options, host, and entrypoint.
|
||||
Use a separate checkout for changes without disturbing other work.
|
||||
|
||||
Establish an oracle independent of the changed implementation: source-built
|
||||
fixture expectations, independently mapped file bytes, or a native producer
|
||||
paired with checks of its interpretation. Comparing two wrappers around the
|
||||
same faulty decoder does not establish correctness. Preserve address widths,
|
||||
same-address alternatives, unresolved endpoints, coverage, and source linkage.
|
||||
An empty result proves absence only within the actually examined scope.
|
||||
|
||||
## Locate the earliest unnecessary work
|
||||
|
||||
Trace one public CLI or MCP task through shared application code to its producer.
|
||||
Measure relevant phases: admission and identity checks, native analysis,
|
||||
decoding, lookup, projection, serialization, Evidence retention, and cleanup.
|
||||
Include catalog discovery and extra calls when they affect the completed task.
|
||||
|
||||
Use the evidence to choose the smallest structural correction:
|
||||
|
||||
| Observed cost | Candidate correction | Condition to establish |
|
||||
| --------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------ |
|
||||
| Repeated scans of validated facts | Reuse existing order or lookup structure | The owner establishes ordering and identity; every matching alternative survives |
|
||||
| Repeated equivalent engine work | Share a validated result or in-flight producer | Inputs, freshness, lifetime, and per-caller outcomes remain explicit |
|
||||
| Full reports for selected facts | Push selection or projection to the earliest capable owner | Required facts and evidence remain reachable; examined coverage is reported |
|
||||
| Large temporary transport buffers | Decode incrementally or use bounded source reads | Validate the whole admitted document; retained facts and copies still consume memory |
|
||||
| Many repeated homogeneous reads | Share admission, decoding, or batch work | Preserve item identity, order where required, and individual failures |
|
||||
|
||||
Do not introduce an index, generic cache, batch endpoint, or orchestration layer
|
||||
without identifying the work it removes and its maintenance/resource cost.
|
||||
Projection after materializing a full report may reduce output while leaving
|
||||
analysis time and peak memory unchanged. Fix missing or incorrect facts before
|
||||
benchmarking an alternative composition against a better-covered workflow.
|
||||
|
||||
## Keep one interpretation and a defined reuse lifetime
|
||||
|
||||
Let the producer boundary validate and normalize facts once. Consumers query
|
||||
that representation through the shared CLI/MCP implementation; adapters should
|
||||
not rebuild relationships or independently reinterpret raw reports. Keep lookup
|
||||
identity separate from display formatting. Reuse contracts when their intent
|
||||
matches, honoring requested detail and complete Evidence access.
|
||||
|
||||
For every cache or shared producer, state its key, owner, invalidation, retained
|
||||
size, and lifetime. Account for result-affecting target bytes, engine contents,
|
||||
analysis options, architecture/load mapping, and parser/schema versions where
|
||||
they affect reuse. Validate live state before calling a cached observation
|
||||
current. Cache a refusal only when it is deterministic for that validated key;
|
||||
malformed, partial, transient, or cancelled attempts need explicit retry policy.
|
||||
|
||||
One cancelled waiter must not invalidate a successful peer. Preserve producer
|
||||
failure context and each caller's cancellation reason. Retain ownership and
|
||||
capacity charges until shared work and cleanup settle; preserve completed
|
||||
observations when cleanup fails. Exercise invalidation, concurrent consumers,
|
||||
last-waiter cancellation, close, and cleanup retry when those paths change.
|
||||
|
||||
Bound the process family and retained state, not just a parent heap or worker
|
||||
count. Account for decoding and serialization expansion before allocation.
|
||||
Choose CPU, memory, concurrency, and disk limits from the host and workload;
|
||||
stop before OOM and verify owned cleanup. A guard-stopped task is incomplete.
|
||||
Do not replace a transport buffer cap with an unbounded retained inventory or
|
||||
hide partial results behind a success status.
|
||||
|
||||
## Compare completed tasks and limit the claim
|
||||
|
||||
Run the affected real public workflow before and after with equivalent inputs,
|
||||
results, coverage, and Evidence semantics. Include small inputs so setup costs
|
||||
and regressions are visible. Distinguish fresh sessions, warm runtime caches,
|
||||
and OS file caches. Alternate or repeat runs when variance affects the decision.
|
||||
Choose meaningful boundary regressions for cases the actual engine cannot
|
||||
reliably reproduce; a fake engine does not qualify a real provider.
|
||||
|
||||
Report task outcome, relevant phase timings, total elapsed and CPU time, peak
|
||||
process-family memory and its accounting method, resource stops, and cleanup.
|
||||
Check the payload exit status as well as the profiler's. Keep failed attempts
|
||||
and identify unmeasured costs; a passing retry does not explain a failure.
|
||||
|
||||
For token claims, count schemas/discovery, arguments, replies, retries, and
|
||||
continuations needed to finish the task. Label tokenizer estimates separately
|
||||
from actual model usage and record the client/catalog behavior. Smaller JSON
|
||||
alone establishes neither fewer model tokens nor better investigation quality.
|
||||
Do not claim a complex-software advantage without completed representative
|
||||
investigations and comparable correctness/coverage.
|
||||
|
||||
Finish with the bottleneck, owning change, preserved semantics, executed
|
||||
checks, before/after measurements, and remaining limits. Keep any integration
|
||||
record current with its tested revision and distinguish merged production work
|
||||
from experiment-only code. Stop when the demonstrated bottleneck is resolved
|
||||
or further benefit remains speculative.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "REA analysis optimization"
|
||||
short_description: "Optimize analysis with preserved Evidence and measured costs"
|
||||
default_prompt: "Use $rea-analysis-optimization to find and verify a concrete analysis bottleneck while preserving correctness and Evidence."
|
||||
@@ -1,112 +0,0 @@
|
||||
# Kuna: reuse before adding machinery
|
||||
|
||||
Read this reference for concrete examples of the skill's decisions. The records
|
||||
below concern static Linux analysis with caller-supplied Kuna. They do not
|
||||
establish parity with another provider or a general advantage for Code Mode,
|
||||
atomic tools, or any particular orchestration design.
|
||||
|
||||
## Exact-address lookup: a measured, narrow improvement
|
||||
|
||||
[PR #4695](https://github.com/morluto/rea/pull/4695) reused the validated string
|
||||
inventory's numeric address order. Previously, each `list_strings(address)`
|
||||
filtered all cached facts. The owner now performs a binary lower-bound lookup
|
||||
and returns every encoding at that address: O(log n + matches), without a
|
||||
second retained lookup index.
|
||||
|
||||
The owning implementation is
|
||||
[`KunaStringInventory.ts`](../../../../src/kuna/KunaStringInventory.ts), consumed
|
||||
by [`KunaStringQueries.ts`](../../../../src/kuna/KunaStringQueries.ts). Inspect
|
||||
their current versions before adapting the example. Numeric u64 addresses,
|
||||
same-address encodings, duplicate rejection, source digests, caller selection,
|
||||
and the existing session lifetime were part of the contract. Full listings and
|
||||
literal/regex searches kept their semantics. An attempted removal of the
|
||||
duplicate-detection set was dropped: no stable memory benefit was established,
|
||||
and it delayed malformed-input rejection.
|
||||
|
||||
### Recorded workload and results
|
||||
|
||||
The benchmark used unmodified Kuna 1.833 and LLVM 16 on Linux x86-64, with
|
||||
602,544 string facts. Baseline was `be0b2d138`; the final feature was
|
||||
`5f564962da0e1b17f00a68a74a312539f0209a01`, subsequently integrated in
|
||||
`c6b62754b095a134d6d67877508fa34311a2eda2`. These are historical observations,
|
||||
not a benchmark of every subsequent main revision.
|
||||
|
||||
The same production stdio MCP task discovered the catalog, opened the binary,
|
||||
made one cold selected read and 28 warm exact reads, checked retained Evidence,
|
||||
and closed the session. Five selected addresses and two absent selectors were
|
||||
read four times. A native filtered scan selected the strings; an independent
|
||||
ELF load-header mapping checked the source bytes and terminators. Returned
|
||||
facts, source linkage, target digests, and cleanup were checked.
|
||||
|
||||
Three alternating fresh sessions per implementation used one CPU, a 4 GiB
|
||||
process-family hard limit, a 3 GiB early stop, and no swap. OS file caches were
|
||||
warm. Whole-harness costs include the independent native oracle, hashing,
|
||||
catalog/client work, and correctness checks.
|
||||
|
||||
| Median | Before | After |
|
||||
| ---------------------------------------- | ---------: | ---------: |
|
||||
| 28 warm exact MCP reads | 495.2 ms | 90.8 ms |
|
||||
| Cold selected MCP read | 11.30 s | 11.01 s |
|
||||
| Whole harness elapsed | 28.24 s | 27.43 s |
|
||||
| Whole process-family CPU | 27.71 s | 26.89 s |
|
||||
| Kernel family peak, including file cache | 2237.6 MiB | 2240.6 MiB |
|
||||
|
||||
Warm totals ranged from 491.7–544.3 ms before to 84.8–99.5 ms after: about
|
||||
5.5 times faster at this phase. Cold/whole-task timing and memory overlapped;
|
||||
no stable improvement in those costs was established. One small SQLite pair
|
||||
(7,985 facts) measured 20 warm reads at 88.6 → 80.5 ms and the whole harness at
|
||||
13.68 → 13.94 s. The large-inventory lookup benefit does not generalize to a
|
||||
whole-task speedup on small inputs.
|
||||
|
||||
Catalog hashes, call counts, and normalized facts were unchanged. A
|
||||
`cl100k_base` estimate over one pass of raw catalog JSON, arguments, and replies
|
||||
was about 846,000 tokens in both lanes, with about 802,562 from the catalog.
|
||||
This was transport serialization, not rendered model context or actual usage;
|
||||
client discovery, prompts, replay, and protocol framing were unmeasured.
|
||||
**No token reduction or improved agent investigation outcome was established.**
|
||||
|
||||
Pinned target SHA-256:
|
||||
`f62d254b7f2bf42df8c8b07d46ee3bb4c2cafeca436b2e6bc6ccbe4581f58f40`.
|
||||
Pinned engine SHA-256:
|
||||
`d858918b0766435cd96b999a77f2d5a31a0cc4f07256421c883b5b9dc4dd58e4`.
|
||||
All six normalized-result sequence digests were
|
||||
`4d2f90269da8568984090b20274163c4a7c2a0538aab63530bd2fe44840a0bba`.
|
||||
Selected byte agreement does not prove native discovery completeness; missing
|
||||
strings and relationships can remain unknown.
|
||||
|
||||
## Other transferable changes and their limits
|
||||
|
||||
- [Cached string inventory and search, #4095](https://github.com/morluto/rea/pull/4095):
|
||||
share validated facts across queries in a defined session. Audit key inputs
|
||||
and lifetime before extending reuse across targets or engine revisions.
|
||||
- [Shared single-function work, #4045](https://github.com/morluto/rea/pull/4045)
|
||||
and [repeat-request reuse, #4169](https://github.com/morluto/rea/pull/4169):
|
||||
avoid duplicate native work while retaining per-caller cancellation and
|
||||
cleanup ownership. A query count alone cannot reveal duplicate subprocesses.
|
||||
- [Streamed strings, #4318](https://github.com/morluto/rea/pull/4318)
|
||||
and [streamed call graph, #4329](https://github.com/morluto/rea/pull/4329):
|
||||
remove fixed aggregate transport-buffer failures through incremental parsing.
|
||||
Streaming transport does not bound every retained object or final reply.
|
||||
These links establish changes; no numerical speedup is asserted here.
|
||||
|
||||
Typed reference facts, native string-query projection, and shared native byte
|
||||
reads remained experimental at the 2026-10-17 checkpoint
|
||||
`65f2f9a22bc892f61d2b00f9a4611124f57d4bff` on
|
||||
`experiment/atomic-codemode-20261016`. Some experiments used a patched native
|
||||
engine. Check current integration and bring-your-own engine capabilities before
|
||||
proposing them; a working experiment is not a stock-provider guarantee.
|
||||
|
||||
## Applying the reasoning elsewhere
|
||||
|
||||
For another provider or format, inspect its actual representation and native
|
||||
capabilities before borrowing the implementation. Existing numeric ordering
|
||||
may support range reads; an on-disk native query may avoid materializing an
|
||||
inventory; an immutable parse may support several views. Measure the relevant
|
||||
alternative rather than requiring every provider to use Kuna's cached array.
|
||||
|
||||
For live observations, immutable artifact caching rules are insufficient:
|
||||
validate the process/session identity and freshness required by the caller.
|
||||
For compact replies, preserve access to source Evidence and report examined
|
||||
coverage; projection must not silently turn an unresolved edge into no edge.
|
||||
For batching, test mixed successes and failures and measure the entire task,
|
||||
including discovery, extra calls, native work, and serialization.
|
||||
@@ -1,134 +0,0 @@
|
||||
---
|
||||
name: rea-changelog-update
|
||||
description: Prepare or rewrite REA release changelogs and GitHub release notes from pinned Git history, with verified contributor thanks and Release Please synchronization.
|
||||
---
|
||||
|
||||
# REA changelog update
|
||||
|
||||
Use this for release-note preparation and contributor attribution. Read
|
||||
[docs/releasing.md](../../../docs/releasing.md) from the repository root for
|
||||
publication authority and the existing checkpoint checks. This skill prepares
|
||||
notes; it does not authorize merging, tagging, or publishing.
|
||||
|
||||
## Inventory before prose
|
||||
|
||||
Fetch complete history and tags. Select the published baseline tag from
|
||||
`.release-please-manifest.json` on the release PR's **base branch**, and pin that
|
||||
branch's exact SHA as the editorial target. REA tags are `rea-agents-VERSION`.
|
||||
Use the proposed version, including its prerelease suffix; do not change the
|
||||
`always-bump-minor` strategy to imitate OpenClaw's calendar versions.
|
||||
|
||||
```bash
|
||||
node scripts/release-notes.mjs inventory \
|
||||
--repository morluto/rea --base rea-agents-PREVIOUS \
|
||||
--target FULL_BASE_BRANCH_SHA --version VERSION \
|
||||
--manifest /tmp/rea-VERSION.contributions.json
|
||||
node scripts/release-notes.mjs record \
|
||||
--manifest /tmp/rea-VERSION.contributions.json \
|
||||
--summary \
|
||||
--output /tmp/rea-VERSION.record.md
|
||||
```
|
||||
|
||||
Read the inventory, not just the existing bot draft. It includes side-branch
|
||||
commits, GitHub-associated PRs whose merge commits are in the range, commit
|
||||
authors/co-authors, and confirmed closing issues with their reporters. GitHub
|
||||
connections are paginated. Network errors stop generation; rerun into a new
|
||||
output file after recovery. Existing output files are never overwritten.
|
||||
|
||||
Unresolved commit identities are retained as names/emails with `user: null`;
|
||||
never guess a handle or thank the merger as a substitute. The report preserves
|
||||
unknowns. This is a reviewed metadata snapshot, not independent proof that a
|
||||
human authored a patch. For unattributed identities, unusual merge strategies,
|
||||
backports, reversions, or references that GitHub did not associate, inspect
|
||||
the source PRs and diffs and report the limitation. The current collector uses
|
||||
confirmed PR-closing relationships for issue-reporter credit; unrelated issue
|
||||
mentions are not automatically credited. Do not claim that merely inventoried
|
||||
work still ships after a revert.
|
||||
|
||||
## Write for users and retain credit
|
||||
|
||||
Rewrite only the newest `## [VERSION]` section in `CHANGELOG.md`, preserving its
|
||||
version/date/link heading. Use single-line Markdown list items under:
|
||||
|
||||
- `### Highlights`: a few release-defining user outcomes; no quota for small releases.
|
||||
- `### Changes`: capabilities and meaningful behavior changes.
|
||||
- `### Fixes`: reliability, correctness, compatibility, and safety fixes.
|
||||
- `### ⚠ BREAKING CHANGES`: retain when applicable, with migrations and original
|
||||
PR/commit references required by the existing checkpoint validator.
|
||||
|
||||
Group related work by analyst workflow or user impact. Explain the resulting
|
||||
behavior in short, concrete entries. Highlights summarize the release; avoid
|
||||
repeating their full descriptions in Changes. Internal cleanup, tests, CI, and
|
||||
routine documentation generally belong only in the provenance file. Read ambiguous PRs/diffs before making
|
||||
claims. Distinguish simulated-provider checks from real-provider verification.
|
||||
|
||||
Credit each represented contribution once in the public notes with its link
|
||||
and `Thanks @...` for its verified human author, known co-authors, and confirmed
|
||||
closing-issue reporters. Grouping preserves all those credits in one line.
|
||||
When Highlights, detailed entries or migrations reference the same work,
|
||||
put its thanks in one of those entries, not all of them. Include issue links.
|
||||
Direct-commit bullets carry the commit link and known human credit.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor
|
||||
and Codex co-author trailers resolve to `@claude`, `@cursoragent` and `@codex`
|
||||
with GitHub type `User`; the credit policy excludes them while retaining their metadata.
|
||||
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
|
||||
authorship in the inventory. Other human contributors remain eligible.
|
||||
Use explicit GitHub links for cross-repository references to avoid ambiguous
|
||||
`#NNN` numbers. Unknown handles stay unknown; they are not fabricated credit.
|
||||
|
||||
Append the generated `record --summary` provenance link verbatim at the end
|
||||
of this release. The linked inventory accounts for every discovered in-range
|
||||
PR and direct contribution, including internal work omitted from the prose.
|
||||
Do not paste the full `record` output into public notes: it duplicates thanks
|
||||
and obscures the user-facing changes. Full records remain available for review.
|
||||
Do not hand-edit generated credits to make validation pass. Refresh the inventory
|
||||
when product history changes; a prose-only correction can reuse the snapshot.
|
||||
|
||||
## Finalize with Release Please
|
||||
|
||||
Before changing the live release PR, have authorization for that PR update.
|
||||
Local preparation and validation can proceed independently.
|
||||
|
||||
1. Add `rea:release-notes-finalized` to the selected open release PR **before**
|
||||
editing its branch/body. Main proposal pushes respect this label. This is
|
||||
our workflow guard, not Release Please's `autorelease: snooze` mechanism.
|
||||
A proposal already running must finish before editing. Do not dispatch
|
||||
manual preparation over finalized notes.
|
||||
2. On that release branch, save the reviewed inventory as
|
||||
`docs/releases/VERSION.contributions.json`. This is durable public release
|
||||
provenance, not a temporary planning file. Do not add credentials or local
|
||||
environment dumps. Keep the existing version artifacts aligned.
|
||||
3. Run the existing checkpoint validator and the new checker:
|
||||
|
||||
```bash
|
||||
node scripts/release-notes.mjs check \
|
||||
--manifest docs/releases/VERSION.contributions.json
|
||||
node scripts/release-notes.mjs render \
|
||||
--manifest docs/releases/VERSION.contributions.json \
|
||||
--output /tmp/rea-VERSION.notes.md
|
||||
```
|
||||
|
||||
The checker supports inline PR bodies, including their surrounding metadata,
|
||||
up to GitHub's 65,536-character limit. It fails on larger notes rather than
|
||||
truncating credit; Release Please's overflow-link bodies are not integrated.
|
||||
|
||||
4. Update the release PR body with the rendered section while preserving its
|
||||
Release Please header/footer and component/version structure. Use a body
|
||||
file for `gh pr edit --body-file`; changing only `CHANGELOG.md` does not
|
||||
change Release Please's release body. Push only authorized changes.
|
||||
5. Freshly fetch the PR body to a file and run `check --pr-body FILE`. Review
|
||||
the complete diff and wait for current CI. Publication checks the body
|
||||
again before release creation. Avoid further body edits while publishing.
|
||||
|
||||
The finalization label freezes bot refreshes, not main. New product changes
|
||||
after the inventory target require refreshing the notes before publication.
|
||||
The workflow checks target ancestry and allows only release metadata paths
|
||||
after that target. It also requires the inventory target to equal the reviewed
|
||||
merge's first parent, so intervening main commits require a fresh inventory.
|
||||
Review diffs within the allowed release metadata paths too: the path guard does
|
||||
not prove that package dependency/script edits are version-only changes.
|
||||
To resume automatic proposals, remove the finalization label and let the bot
|
||||
regenerate; expect editorial changes to be replaced.
|
||||
|
||||
Existing releases without a contribution inventory retain their current
|
||||
publication path. Never retag or republish a package to correct release prose.
|
||||
@@ -1,112 +0,0 @@
|
||||
---
|
||||
name: rea-ci-tests-audit
|
||||
description: Audit REA CI and test suites for correctness, fixture drift, false passes, and measured performance; use for suite or workflow investigations and efficiency changes, rather than ordinary feature testing or a single test failure.
|
||||
---
|
||||
|
||||
# REA CI and tests audit
|
||||
|
||||
Establish which checks still catch real regressions and which costs can be
|
||||
removed without losing that protection. The outcome may be a verified fix,
|
||||
an evidence-backed recommendation, or an explicit decision to stop.
|
||||
|
||||
Follow the checkout's `AGENTS.md`, [testing guide](../../../docs/testing.md),
|
||||
and [contribution rules](../../../CONTRIBUTING.md). Use their current commands
|
||||
and pinned toolchain, rather than commands remembered from an earlier audit.
|
||||
Preserve the requested scope: investigation alone does not authorize edits;
|
||||
this skill does not grant permission to publish issues, push, or merge. Existing
|
||||
user authorization still applies. Keep audit notes and raw results outside the
|
||||
tracked tree, normally in a task-specific `/tmp` directory.
|
||||
|
||||
## Establish the executed baseline
|
||||
|
||||
Fetch when the user asks for latest main. Pin the baseline SHA and inspected
|
||||
workflow revision; use an isolated worktree when changing or comparing code.
|
||||
Do not modify another task's checkout. Record dirty changes and the exact
|
||||
command, toolchain, host, selected tests, instrumentation, and prerequisites.
|
||||
An old run is evidence for its executed revision, even if main advances.
|
||||
|
||||
Trace workflow triggers, changed-path selection, job dependencies, reusable
|
||||
actions, package scripts, test discovery, artifact producers, and report gates
|
||||
through to the commands that actually execute. Include scripts, generated data,
|
||||
runtime registration, and subprocess boundaries missing from import graphs.
|
||||
Check positive and negative selection paths: required files must be discovered
|
||||
once per intended test configuration; deliberate platform/configuration repeats
|
||||
are separate coverage. A missing explicit test or missing required report must
|
||||
not turn green. Optional lanes may skip; preserve their stated prerequisites.
|
||||
|
||||
Use existing timing and failure artifacts to select suspicious families. Run
|
||||
relevant checks to establish a baseline; reserve a full local suite for broad
|
||||
CI diagnosis or uncertainty that focused checks cannot resolve. Do not rebuild
|
||||
or rerun an unchanged complete suite after every documentation edit.
|
||||
|
||||
## Prove the failure or protection
|
||||
|
||||
For each candidate, state the expected observable behavior and its owner,
|
||||
then construct the smallest reproduction against production behavior. Follow
|
||||
the producer and affected consumers before deciding where to fix it.
|
||||
|
||||
- For drift, compare fixtures with current admission requirements, protocol
|
||||
replies, named contracts, tool versions, and actual public entrypoints. A
|
||||
plausible-looking mock can fail before the behavior its test claims to cover.
|
||||
Search the whole affected fixture family, including other version strings.
|
||||
Preserve intentionally malformed fixtures and test their intended rejection.
|
||||
- For a suspicious green test, perturb the relevant behavior in an isolated
|
||||
copy or inject a controlled failure. Verify that the existing assertion or
|
||||
gate detects it. A test may merely repeat its mock, import abandoned code,
|
||||
or hide a subprocess/checker failure. Restore mutations after the experiment.
|
||||
- For shared setup or pruning, map distinct assertions to their retained
|
||||
scenarios. Preserve malformed inputs, cancellation, cleanup failure,
|
||||
permissions, capacity, identity, and meaningful concurrency interactions.
|
||||
A faster run that skips a case or weakens its oracle is not an improvement.
|
||||
- For provider coverage, separate fake-producer, boundary, and actual-engine
|
||||
evidence. Inspect named skips and prerequisites; fixtures do not establish
|
||||
real Ghidra, LLDB, browser, or other provider behavior.
|
||||
|
||||
Classify findings as reproduced defects, supported costs, hypotheses, or
|
||||
unavailable evidence. Record the baseline command, observed output/exit status,
|
||||
expected result, affected scope, and competing explanation. A failure on main
|
||||
may be pre-existing; that does not by itself make it irrelevant to a CI audit.
|
||||
Attribute PR failures using the executed revision and a clean baseline, not
|
||||
just their resemblance to a previous failure. Retry only to test a specific
|
||||
transient explanation; retain the original failure and stop when repetition
|
||||
adds no evidence. A passing retry does not establish a root cause or a fix.
|
||||
|
||||
## Assess efficiency without losing the gate
|
||||
|
||||
Read [measurement and CI checks](references/measurement-and-ci.md) when
|
||||
investigating slowness, caching, runner waits, shared artifacts, or log noise.
|
||||
Compare equivalent completed work before claiming a speedup. Distinguish
|
||||
setup/build time, test execution, coverage instrumentation, dependency waits,
|
||||
and runner scheduling; state which costs remain unmeasured.
|
||||
|
||||
Prefer removing proven duplicate preparation, reusing immutable setup, and
|
||||
reducing repeated subprocess work before shrinking correctness coverage.
|
||||
Preserve independent sessions where lifecycle or concurrency is the oracle.
|
||||
Keep live tests fresh; a cached previous pass cannot validate current host,
|
||||
installed provider contents, permissions, or external state.
|
||||
|
||||
## Fix, verify, and hand off within scope
|
||||
|
||||
If implementation is authorized, make the smallest structural change at the
|
||||
owning boundary and add a regression that fails on the baseline for the right
|
||||
reason. Retest affected consumers and meaningful negative paths; verify cache
|
||||
invalidation or discovery when those mechanisms change. Read the installed
|
||||
Turbo documentation before changing its configuration or commands.
|
||||
|
||||
Run the repository's applicable checks and inspect complete current-head CI
|
||||
results before an authorized merge. Preserve required coverage and discovery
|
||||
gates. Do not suppress a failure, add blanket retries, raise timeouts, or delete
|
||||
distinct assertions solely to make CI green. Changes to these controls need
|
||||
their own demonstrated reason and retained failure detection.
|
||||
|
||||
Report each confirmed issue with exact revision, reproduction, consequence,
|
||||
proposed correction, and validation limits. Report performance with workload
|
||||
and measurements, not just a percentage. Use English issue/PR records when
|
||||
requested; follow the user's assignment, commit identity, review, and merge
|
||||
preferences without assuming them for other users. Open ready-for-review PRs
|
||||
only when that lifecycle is authorized. Resolve necessary review findings and
|
||||
check final main against the tested tree after merging.
|
||||
|
||||
Stop when the authorized work is verified, remaining benefits are unproven,
|
||||
or further changes would trade correctness for speculative speed. List genuine
|
||||
uncertainties without turning them into additional optimization rounds.
|
||||
@@ -1,4 +0,0 @@
|
||||
interface:
|
||||
display_name: "REA CI and tests audit"
|
||||
short_description: "Audit CI/test correctness and measured performance"
|
||||
default_prompt: "Use $rea-ci-tests-audit to investigate CI and test correctness, drift, and performance before proposing changes."
|
||||
@@ -1,65 +0,0 @@
|
||||
# Measurement and CI checks
|
||||
|
||||
## Equivalent completed work
|
||||
|
||||
Record the source/workflow SHA, command, test identities and counts, skips,
|
||||
assertions or output contract, exit status, coverage scope and thresholds,
|
||||
toolchain/provider versions, host resources, and prepared artifacts. Inspect
|
||||
both the child result and its report: a profiling wrapper may exit successfully
|
||||
after the measured command failed. Focused tests can pass while global coverage
|
||||
thresholds fail; describe both rather than calling the command green.
|
||||
|
||||
Measure wall time, user/system CPU, and peak process-family memory for relevant
|
||||
completed tasks. Parent-only RSS misses workers and their children. On Linux,
|
||||
summed RSS can double-count shared pages; report RSS/PSS semantics, sampling
|
||||
interval, missing samples, and observed cleanup limits. Use equivalent host
|
||||
tools elsewhere. State unknown costs, including agent tokens; fewer log bytes
|
||||
alone prove neither fewer tokens nor faster tests.
|
||||
|
||||
Separate cold execution, warm build/file caches, and warm runtime caches.
|
||||
Use repeated or alternating runs when variance affects the decision, with
|
||||
enough observations to distinguish the proposed benefit from noise. Keep
|
||||
workload, instrumentation, and host conditions comparable; record interference.
|
||||
A run with fewer tests, early rejection, lost coverage, or a different runner
|
||||
does not prove suite acceleration. CI-wide claims need end-to-end measurements,
|
||||
not an extrapolation from one improved family.
|
||||
|
||||
## Workflow, caches, and shared artifacts
|
||||
|
||||
Trace each reused result from producer to consumers and identify its lifetime.
|
||||
Check key/input coverage against the actual file inventory and result-affecting
|
||||
configuration, environment, generated data, tool versions, and host state.
|
||||
Exercise an affected input change to prove invalidation; exercise an irrelevant
|
||||
change to support reuse where useful. Cache deterministic preparation and
|
||||
static analysis only when their inputs fully describe the result.
|
||||
|
||||
Verify downloaded artifacts match the expected revision, platform, runtime,
|
||||
and schema. Check restore misses, failed/partial producers, stale reports,
|
||||
removed output files, and consumers after producer failures. Shared writers
|
||||
need output ownership or locks; tests need isolated state, not an unnecessary
|
||||
global lock. Worker counts do not bound children spawned within each test.
|
||||
Preserve intentionally concurrent scenarios while bounding unrelated batches.
|
||||
|
||||
Check matrix expansion and all job pages before saying CI passed. Inspect
|
||||
required gates after failed, cancelled, and skipped dependencies. `always()`
|
||||
alone does not preserve failures; inspect the aggregation logic. Test/report
|
||||
uploads should retain available diagnostics after failure, while absent or
|
||||
malformed required results must fail closed. Do not demand an optional report
|
||||
from a lane intentionally excluded by the selector.
|
||||
|
||||
## Runner waits and console noise
|
||||
|
||||
Use per-job and per-step timestamps where available. Workflow creation to job
|
||||
start includes dependencies and other delays; it is not necessarily runner
|
||||
queue time. Identify when the job became eligible before attributing waits to
|
||||
runner capacity. If eligibility/queue timestamps are unavailable, report the
|
||||
combined delay and unknown split. Compare equivalent lanes and runner types
|
||||
across enough runs; a single busy runner is not a workflow regression.
|
||||
|
||||
For console reductions, retain checker exit codes, actionable errors, useful
|
||||
summary counts, and complete machine-readable artifacts. Verify authored code
|
||||
remains in the actual checker inventory when excluding generated outputs.
|
||||
Introduce an authored error as a control; it must still fail. An ignore pattern
|
||||
can hide real source as well as noise. Show that cached reports are restored,
|
||||
stale successful reports cannot mask current failures, and report parse errors
|
||||
remain failures. Measure readability or byte reduction separately from runtime.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
name: rea-tool-design
|
||||
description: Design or change REA investigation tools, CLI/MCP contracts, provider capabilities, and Evidence semantics.
|
||||
---
|
||||
|
||||
# REA tool design
|
||||
|
||||
Use the [tool-design guide](../../../docs/tool-design.md) for tool boundaries,
|
||||
contract semantics, provider ownership, and discoverability. Inspect the current
|
||||
contract and nearest existing tool relevant to the analyst question; source,
|
||||
documentation, and live catalogs may describe different versions.
|
||||
|
||||
For a design request, return the proposed boundary, input/result shape, Evidence
|
||||
semantics, nearest alternative, and verification approach. Implement when
|
||||
requested, preserving the user's scope and existing authorization.
|
||||
|
||||
For implementation or verification, consult [testing](../../../docs/testing.md)
|
||||
for the affected consumer and provider lanes, and
|
||||
[contributing](../../../CONTRIBUTING.md) for generated-file ownership. Read
|
||||
provider/workflow guides only as the task requires.
|
||||
@@ -1,410 +0,0 @@
|
||||
---
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "44"
|
||||
---
|
||||
|
||||
# REA
|
||||
|
||||
Use REA when a claim depends on a shipped binary or package, decompilation,
|
||||
passive application runtime evidence, or comparison with
|
||||
behavior not established by available source. For ordinary analysis of a
|
||||
complete source repository, use normal repository tools and do not run REA
|
||||
readiness or provider commands.
|
||||
|
||||
## Connect only when needed
|
||||
|
||||
Installing this skill supplies instructions; it does not register the REA MCP
|
||||
server, install analysis engines, or add tools to an already-running agent.
|
||||
|
||||
If REA tools are available and their registration is not known to be stale,
|
||||
proceed directly to the target. Do not run diagnostics or setup before every
|
||||
investigation. Use the connected server's actual tool list and input schemas;
|
||||
a skill installed from repository main may describe capabilities absent from
|
||||
an older npm release. Keep complete inline Evidence when that server does not
|
||||
advertise retained references.
|
||||
|
||||
If tools are absent, a call is blocked by the client, or a required provider is
|
||||
unavailable, read [connection and recovery](references/connection-and-recovery.md).
|
||||
Diagnose the observed blocker, continue useful supported work, and resume the
|
||||
original task after repair. Client approval, shell PATH, registration and engine
|
||||
availability have different remedies; avoid repeating setup for all of them.
|
||||
If the host times out without a REA result, do not repeat the unchanged call or
|
||||
assume work stopped. Read the recovery guide's deadline distinction before retrying.
|
||||
|
||||
1. Diagnose without changing REA or client configuration. `npx` may download
|
||||
the package and write to npm's cache. For Codex, run
|
||||
`npx -y rea-agents@latest doctor --client codex --json`. Substitute the current
|
||||
supported client: `claude_code`, `claude_desktop`, `codex`, `cursor`,
|
||||
`gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`, `copilot_cli`,
|
||||
`commandcode`, `vscode`, `grok_build`, `omp`, or `grok_bot`. `grok_bot` has no local
|
||||
file registration: doctor reports the Grok Bot chat step, and setup does not
|
||||
write `mcp.json`. If the client is unknown, use `doctor --json` and
|
||||
inspect its registration results before choosing a setup scope.
|
||||
2. Distinguish the reason. Missing, malformed, or stale registration needs a
|
||||
scoped configuration repair. An aligned registration with no tools in the
|
||||
active session needs a restart/reconnection; doctor cannot prove that the
|
||||
current agent has connected. A missing provider affects only tasks requiring
|
||||
that provider: static JavaScript inspection needs neither Hopper nor Ghidra,
|
||||
and Android inspection has separate bring-your-own JADX/Java prerequisites.
|
||||
3. For a configuration repair, prepare the read-only plan:
|
||||
`npx -y rea-agents@latest setup --client codex --dry-run --json`.
|
||||
Use the current client's ID, show its exact proposed paths, backups, and
|
||||
changes, and obtain approval before setup writes configuration or installs
|
||||
Hopper. Setup normally installs the matching bundled skill too; include that
|
||||
replacement in the reviewed plan. After approval, apply the same scope with
|
||||
`npx -y rea-agents@latest setup --client codex --yes`. Add `--install-hopper`
|
||||
only if that separate installation was needed and explicitly approved.
|
||||
4. Restart/reconnect the affected agent. Verify that REA tools actually appear
|
||||
in the session, then resume the original investigation. If they remain
|
||||
absent, inspect the client's MCP launch error rather than repeating setup.
|
||||
|
||||
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
|
||||
JADX, Kuna, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
|
||||
software to repair
|
||||
MCP registration. For an unsupported client, use manual stdio registration
|
||||
with a version-pinned `rea-agents` package or continue through the CLI.
|
||||
|
||||
A concrete CLI fallback for an operator-supplied JavaScript tree or ASAR is:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
```
|
||||
|
||||
No MCP registration or native engine is required. Read the returned Evidence,
|
||||
graph, limitations, and unknowns with the same care as an MCP result; the CLI
|
||||
returns the Evidence record directly. This fallback does not establish that
|
||||
MCP is configured. Native CLI tasks still require their selected engine.
|
||||
Prefer an existing `rea` CLI. When none is installed and REA package acquisition
|
||||
is within the authorized scope, use `npx -y rea-agents@latest` in its place.
|
||||
Configuration repair starts with a read-only setup plan; show paths, backups and
|
||||
skill replacement before applying authorized changes. Preserve existing approval
|
||||
and target-execution boundaries. REA setup never installs or upgrades unrelated
|
||||
software, including Node.js, npm, Homebrew, Java, Ghidra, IDA, JADX, Kuna, Binwalk or Unblob.
|
||||
|
||||
## Route the target first
|
||||
|
||||
Choose the first tool from the target the user supplied. Use `open_binary` for
|
||||
active-target native or archive workflows; target-free tools take their own
|
||||
explicit path or endpoint and do not need it.
|
||||
|
||||
- ASAR or extracted JavaScript/Electron tree:
|
||||
`analyze_javascript_application`.
|
||||
- Archive/package member inventory (ZIP/APK/IPA/MSIX/AppX, DMG, or a PyInstaller
|
||||
executable): `open_binary` with the supplied local path; use `inspect_artifact`
|
||||
when its graph and findings help answer the question. A PyInstaller CArchive
|
||||
is the application layer; the bootloader is only the native prefix. Start with
|
||||
`detail: "summary"` for large inventories, then use retained inventory views
|
||||
for selected member fields/pages. A complete small inventory can cost less
|
||||
than extra calls and schemas. Runtime options are metadata, not files.
|
||||
Compact delivery still inventories and hashes members. See the
|
||||
[Python packaged applications guide](https://github.com/morluto/rea/blob/main/docs/python-packaged-applications.md).
|
||||
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
|
||||
`open_bundle_resources`, then the selected resource tool. This does not
|
||||
require `CFBundleExecutable` or a deep provider. Resource sessions have no
|
||||
executable digest or binary snapshot; use `open_binary` for native code.
|
||||
- Android APK code, classes, methods, or incoming references:
|
||||
`inspect_android_package`, then focused Android tools when advertised.
|
||||
Archive member inventory still uses the archive route above.
|
||||
- Managed PE/CLI assembly: `inspect_managed_artifact`.
|
||||
- Firmware image: `inspect_firmware_regions` when advertised. Use
|
||||
`extract_firmware` when extraction is requested, with a caller-selected new
|
||||
absolute output directory. These tools use caller-supplied Binwalk/Unblob on
|
||||
Linux; see the [firmware guide](https://github.com/morluto/rea/blob/main/docs/firmware-analysis.md).
|
||||
- .NET NativeAOT PE/ELF: native Ghidra analysis. Native code can yield pseudocode;
|
||||
consult the NativeAOT workflow in `references/native-and-artifacts.md` for
|
||||
optional metadata recovery and its supported host/layout boundary.
|
||||
- User-owned browser page already open: `list_browser_targets`.
|
||||
- Browser action the agent must perform (click, type, navigate):
|
||||
`capture_browser_scenario` with declared actions. See
|
||||
[runtime observation](references/runtime-observation.md) for connect/launch
|
||||
and the focused follow-up browser tools.
|
||||
- Fiddler Everywhere traffic or a native `.saz` capture: read the
|
||||
[Fiddler workflow](references/runtime-observation.md#fiddler-everywhere-traffic-debugging).
|
||||
Inspect saved SAZ with `inspect_web_network_capture` and `format: "saz"`;
|
||||
connect Everywhere's separate MCP server for live traffic.
|
||||
- Retained HAR/SAZ/native mitmproxy/aioquic qlog capture: `inspect_web_network_capture`. Select
|
||||
original record ordinals when useful; inspect producer fields and byte/number
|
||||
sidecars without fetching recorded URLs or claiming live attribution. See
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
|
||||
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
|
||||
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
|
||||
or establish network loss. Prefer detail summary, then kind qlog trace/stream
|
||||
pages; select one trace, stream ID and direction for contribution/range pages.
|
||||
Facts retain all-recorded-events-before-selection scope. Other qlog layouts
|
||||
and PCAP correlation are unsupported.
|
||||
- User-owned Electron runtime already open: `list_electron_targets`. Before an electron-main Inspector attempt, check packaged fuses with `inspect_electron_fuses`; disabled `EnableNodeCliInspectArguments` means the app ignores `--inspect`.
|
||||
- Packet-field discovery: `inspect_packet_fields` searches installed protocols
|
||||
and field names/types before choosing fields for `inspect_packet_capture`.
|
||||
Query/page the relevant records; registrations do not establish observed traffic.
|
||||
- Recorded packet captures: `inspect_packet_capture` with selected TShark fields,
|
||||
display filters and decode-as rules. Requires Linux, util-linux prlimit and `REA_TSHARK_COMMAND`; see
|
||||
[capture guide](https://github.com/morluto/rea/blob/main/docs/packet-captures.md).
|
||||
Packet limits count input packets before filtering. Output defaults to 50 matching
|
||||
frames; use `match_offset` / `match_limit` for later pages, or explicit null
|
||||
limit for complete matching output. Preserve the source digest across pages.
|
||||
This does not perform live capture.
|
||||
For address/DNS/SNI/protocol summaries use `inspect_pcap_capture` with MCP
|
||||
detail summary, then retained kind pcap pages. Requires Wireshark 4.x tshark
|
||||
and capinfos, with Linux prlimit or Windows Job Objects; see
|
||||
[packet summaries](https://github.com/morluto/rea/blob/main/docs/pcap-analysis.md).
|
||||
Select an exact DNS question for address pages; page exhaustion does not prove
|
||||
network completeness. This reads recordings and does not acquire traffic.
|
||||
- Decoder discovery: `inspect_signal_decoders` searches installed decoder IDs
|
||||
or selected decoder help, including channels and options. Query and page the
|
||||
relevant details before choosing settings; this does not scan hardware.
|
||||
- Recorded logic/signal traces: `decode_signal_recording` with explicit decoder,
|
||||
channel and protocol settings. On Linux with util-linux prlimit, supply sigrok-cli 0.7.2 through
|
||||
`REA_SIGROK_COMMAND`; see [recording guide](https://github.com/morluto/rea/blob/main/docs/signal-recordings.md).
|
||||
Output defaults to 50 literal lines; select annotation classes and page with
|
||||
`line_offset` / `line_limit`, or explicit null for all remaining output.
|
||||
Window counts are display lines, not annotations or samples. Preserve source digests across pages.
|
||||
This reads existing files; it does not acquire hardware signals.
|
||||
- Explicit local WebAssembly artifact: `inspect_wasm_artifact`. Supply WABT 1.0.42
|
||||
through `REA_WABT_BIN_DIRECTORY`; retain exact byte/tool identity and decoded WAT.
|
||||
Select JavaScript glue and candidate paths explicitly and preserve URL/path
|
||||
ambiguity. No implicit fetch or target execution; WAT is not original source.
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
|
||||
- Explicit local EVM bytecode carrier: `inspect_evm_interface` with caller-selected
|
||||
`raw` or `hex` encoding. The owned worker deadline is `timeout_ms` (default
|
||||
30000 ms, range 1..2147483647, next call). It does not raise the fixed byte,
|
||||
address-space, or CPU allowances. Preserve carrier/decoded digests and treat selectors,
|
||||
argument strings and mutability as inferred candidates; this performs no chain
|
||||
lookup or target execution. For the complete inferred interface without hex or
|
||||
duplicate stdout, discover support and select `detail: "summary"`; complete
|
||||
Evidence stays retained under its original identity. Use the returned
|
||||
`parent_evidence_id` for exact function or `bytecode_hex`/stdout/stderr pages.
|
||||
Candidate coverage does not establish interface completeness. See the
|
||||
[EVM bytecode guide](https://github.com/morluto/rea/blob/main/docs/evm-bytecode.md).
|
||||
- Embedded Go compiler, module or build-setting metadata: `inspect_go_binary`
|
||||
with the explicit local file. This built-in target-free inspection requires no
|
||||
Go installation or disassembler. Preserve raw module bytes and source ranges;
|
||||
absent metadata does not prove a binary is not Go, and unparsed text records
|
||||
remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
|
||||
- Embedded CUDA/HIP code objects: `inspect_gpu_code_objects`; inventory targets
|
||||
first, then decode selected payloads and query retained GPU Evidence.
|
||||
- Bundled GGUF or safetensors model declarations: `inspect_model_artifact`
|
||||
with the explicit local file. Default header hashing leaves full-file identity
|
||||
unknown; `hash: "full"` additionally reads all weights for SHA-256. MCP returns
|
||||
retained summaries; optional `view` selects one page in the initial call. Use
|
||||
`inspect_analysis_view` kind `model-artifact` for
|
||||
batched metadata keys, tensor names or type totals, selected fields and pages.
|
||||
Preserve header-scoped indices and array preview omissions; do not infer model
|
||||
behavior from declared architecture or tensor types. See the
|
||||
[model file guide](https://github.com/morluto/rea/blob/main/docs/model-artifacts.md).
|
||||
- Unity IL2CPP names, member owners and string literals from `global-metadata.dat`:
|
||||
`inspect_il2cpp_metadata` with the explicit local file; MCP defaults to compact
|
||||
counts. Query sections, images, types, methods, fields, properties, events,
|
||||
parameters or string_literals through `inspect_analysis_view`
|
||||
with the returned `parent_evidence_id`, using exact filters, batch indices and
|
||||
selected fields. Parameters accept `method_indices`, avoiding a scan of all
|
||||
parameter pages. Missing collections in historical Evidence remain unknown.
|
||||
Versions 24, 27, 29, 31 and 39; no Unity, IL2CPP
|
||||
binary or disassembler is run. Unrecognized magic (possibly encrypted) and
|
||||
other versions are unsupported. Optional matching `binary_path` enables static native addresses and type references; query `method_addresses` with indices or addresses/states, preserving every shared-body method and both artifact identities. Without a linked binary native counts remain unknown. Byte-pattern fallback is inferred; no runtime behavior or same-build pairing is proved. See the
|
||||
[IL2CPP metadata guide](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
|
||||
- Embedded cargo-auditable Rust dependency metadata: `inspect_rust_binary`
|
||||
with the explicit local ELF, PE, or thin Mach-O file. This built-in
|
||||
target-free inspection requires no Rust installation or disassembler. Preserve
|
||||
package order, dependency indices, and the section file range. A missing
|
||||
`.dep-v0` section is null metadata, not proof the binary has no dependencies.
|
||||
A rustc commit hash is an inference from a read-only `/rustc/<40-hex>/` path.
|
||||
For selected crates, use `view: {collection: "packages", names: [...],
|
||||
fields: ["name", "version", "source"]}` on the first call; explicitly include
|
||||
`dependencies` for direct index references. Use `detail: "summary"` for counts
|
||||
and ranges, or retained `inspect_analysis_view` Rust package/edge pages for
|
||||
subsequent filters. Keep the complete parent Evidence accessible; CLI compact
|
||||
delivery requires `--evidence_output`.
|
||||
See the
|
||||
[Rust binary guide](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
- PE Authenticode certificate table: `inspect_pe_certificates` with the explicit
|
||||
local PE32/PE32+ file. This built-in target-free inspection requires no
|
||||
Windows SDK. Preserve WIN_CERTIFICATE ranges, types, payload SHA-256, and
|
||||
PKCS#7 signer/X.509 identity fields. A missing security directory is observed
|
||||
absence of that table, not proof the image is unsigned in every catalog sense.
|
||||
Signatures are not verified. See the
|
||||
[PE certificate guide](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
|
||||
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
|
||||
with Node.js 24.x >=24.15 or 26+ providing native SQLite limits. Read schema
|
||||
first (compact inventory by default), or select an exact ordinary table
|
||||
and `columns` when rows are needed. Defaults are selected-table schema and
|
||||
20 rows; request `schema_scope: "database"` for the complete stored schema. Preserve both database and existing sibling WAL identity; row samples
|
||||
report truncation and have unspecified order. Use a quiescent snapshot rather
|
||||
than assuming a live database/WAL pair was copied atomically. See the
|
||||
[SQLite guide](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
- Explicit Linux ELF file for offline layout, symbols, relocations or static
|
||||
mitigation evidence: `inspect_binary_layout`. This target-free operation uses
|
||||
caller-supplied pwntools without opening a disassembler database. Preserve its
|
||||
raw locations and inference/coverage limits; see the
|
||||
[offline binary guide](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
- Supplied Linux x86-64 ELF core: `inspect_recorded_crash`. Read every recorded
|
||||
thread, raw note and source-bound register without opening a live target.
|
||||
Historical PIDs do not select live processes. Request `include_debugger_context`
|
||||
only when core-only mapping candidates help; see the
|
||||
[recorded crash guide](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md).
|
||||
- PE resource type/name/language trees and static icon group references:
|
||||
`inspect_pe_resources` with an explicit path. This portable operation requires
|
||||
no active target or engine; opaque payloads retain file ranges and SHA-256.
|
||||
- Native executable, library, or analysis database: `open_binary`, then
|
||||
use focused analysis tools directly; call `binary_overview` when metadata or
|
||||
inventory context is useful and available from the selected provider.
|
||||
|
||||
For an existing IDA MCP configuration, use `open_binary` with the original
|
||||
input binary and `provider_id: "ida"`; do not pass an `.idb` or `.i64` database.
|
||||
The legacy attached profile binds the already-open GUI input by SHA-256 and
|
||||
leaves its database open. The modern headless profile creates and closes an
|
||||
owned private database without saving. Installation and registration reuse
|
||||
[mrexodia/ida-pro-mcp](https://github.com/mrexodia/ida-pro-mcp); see the
|
||||
[IDA provider guide](https://github.com/morluto/rea/blob/main/docs/ida-provider.md).
|
||||
Proceed directly to `analyze_function`, pseudocode, or function/string searches
|
||||
after opening; IDA does not supply `binary_overview`. Consult session availability
|
||||
when composing broader workflows. Modern direct callers and unsupported dossier
|
||||
facets remain unknown; live IDA results are not replayed from snapshots.
|
||||
|
||||
With `REA_KUNA_COMMAND` configured, `provider_id: "kuna"` answers procedure
|
||||
lists, lookups, pseudocode, assembly and `analyze_function` from Kuna's cached
|
||||
whole-program export. Callers/callees use a separate body-free graph shared by
|
||||
the session; `xrefs` returns exact-address incoming sites and filters forwarding
|
||||
aliases. These queries and inventory lookups do not trigger a C export. Indirect
|
||||
targets remain unknown and can make relationship coverage partial. Reference
|
||||
caches expire on close; a fresh CLI process repeats the native reference walk.
|
||||
Typed references, CFG, raw-memory tools and annotations remain unsupported. A cold first
|
||||
answer can come from a single-function run (its limitations say so); assembly
|
||||
waits for the export. For pseudocode and assembly, `raw_result.source` names the
|
||||
cached `.c`/`.asm` file and byte span, which you may search directly instead of
|
||||
calling per function. See the
|
||||
[Kuna provider guide](https://github.com/morluto/rea/blob/main/docs/kuna-provider.md).
|
||||
|
||||
If the user says only "my app" without a name, path, or target already selected
|
||||
for this task, ask which app before searching for artifacts. Paths in unrelated
|
||||
saved reports and nearby test fixtures do not select the user's target. Resolve
|
||||
an actual human-readable app name to one clear installed artifact when possible;
|
||||
ask when matches are ambiguous. Never choose an example app on the user's behalf.
|
||||
|
||||
In a target-free session, use `open_binary` to bind any archive/package or
|
||||
native target whose analysis tool operates on the active target. Do not call an
|
||||
unadvertised tool; inspect `binary_session` with `{}` and its
|
||||
`result.tool_availability` for availability reasons and remediation. Expand each
|
||||
group's `names` into entries, inheriting the inventory-level client requirements
|
||||
unless the group overrides them. In its closed effect and annotation vocabularies,
|
||||
listed flags are true and omitted known flags are false. The grouped name union
|
||||
is the complete tool inventory; use `tools/list` for canonical tool schemas.
|
||||
Provider candidate facts describe possible providers, while top-level session
|
||||
capabilities describe the current route and `analysis_provider_binding` names the
|
||||
selected provider. Do not treat those scopes as interchangeable.
|
||||
|
||||
For a Ghidra startup timeout on a large native target, follow
|
||||
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
|
||||
before abandoning deep analysis or repeating the same cold import. The default
|
||||
330 seconds is configurable; changing the server's environment requires a new
|
||||
REA process.
|
||||
|
||||
## Work summary-first
|
||||
|
||||
Start with the default result and use its inline Evidence and graph context.
|
||||
For a JavaScript application or ELF layout whose size is unknown or large,
|
||||
call `analyze_javascript_application` or `inspect_binary_layout` with
|
||||
`"detail": "summary"`, then inspect the retained `parent_evidence_id` with
|
||||
`inspect_analysis_view`.
|
||||
Page module identities and select the returned `node_id` before requesting an
|
||||
item; a single file path can represent several graph roles. Do not repeat an
|
||||
identical analysis or rejected call without a changed input or resolved blocker.
|
||||
Recheck session state when a lifecycle change makes that status useful.
|
||||
Make a focused follow-up only when the
|
||||
returned result leaves a specific question unanswered.
|
||||
|
||||
If MCP reports `resource_constraint` with `details.resource: "transport"`, call
|
||||
`inspect_analysis_view` with its reported same-session Evidence reference for a
|
||||
summary, one section/module, or a stable page. Use `trace_application_feature`
|
||||
once a seed is known, or call `export_evidence_bundle` to write the complete
|
||||
session to a selected path. The analysis remains complete in the ledger; avoid
|
||||
repeating it merely to request the same oversized response. Complete CLI JSON
|
||||
output also streams without a single MCP frame.
|
||||
|
||||
When the retained parent ID is unknown, use `get_evidence_bundle` with
|
||||
`detail: "summary"` and exact filters for operation, target SHA-256,
|
||||
analysis-profile digest or canonical procedure address. Select its returned
|
||||
Evidence ID for the next view instead of requesting the complete bundle.
|
||||
`retention: "complete-record"` means the whole record is stored; native
|
||||
value-flow truncation and unknown metadata remain separate. For an exported
|
||||
portable bundle, use `rea inspect-evidence-bundle` with JSON containing `path`,
|
||||
`detail: "summary"` and the same `filters`. Reading that file still requires
|
||||
validating its full contents within the local memory budget.
|
||||
|
||||
A client can also truncate a successful tool result without a REA transport
|
||||
error. Treat that preview as incomplete; use the retained parent ID for smaller
|
||||
views or export when the full record is needed. Do not parse the preview as
|
||||
canonical JSON or repeat the producer. For JavaScript, read the
|
||||
[module-view and trace recipes](references/javascript-applications.md) before
|
||||
drilling into a large graph; a broad bidirectional trace can expand unrelated
|
||||
containment relationships even after summary analysis.
|
||||
|
||||
Every conclusion must distinguish observations, inferences, and unknowns. Cite
|
||||
Evidence IDs, preserve limitations and incomplete coverage, and never imply
|
||||
that static analysis observed execution. Runtime requests execute the declared
|
||||
target and lifecycle; do not broaden the target or action beyond those fields.
|
||||
|
||||
Within the user's requested investigation, call available analysis tools
|
||||
directly. REA does not require permission grants or per-call approval flags.
|
||||
The agent client can independently require approval. Follow the declared request
|
||||
scope and the host's actual access requirements; use the recovery guide for an
|
||||
observed client-policy rejection.
|
||||
|
||||
## Plan broader investigations
|
||||
|
||||
For requests that span multiple features or subsystems, use a staged workflow:
|
||||
|
||||
1. Turn the request into a checklist of questions and the evidence each answer
|
||||
needs. Resolve target identity and constraints from the conversation and
|
||||
workspace before asking for information again.
|
||||
2. Inspect the current REA session, artifact identity, saved analysis database,
|
||||
bookmarks, and prior evidence. Reuse matching state; do not open duplicate
|
||||
sessions or repeat identical analysis.
|
||||
3. Start with the smallest useful overview or inventory. Follow each question
|
||||
from its entry point through relevant data and state changes to its result.
|
||||
Batch related operations around a specific hypothesis, then expand only when
|
||||
the returned evidence leaves a concrete gap.
|
||||
4. Inspect relevant packaged resources and configuration alongside code when
|
||||
they affect the question. Use format-aware inventory and parsers; do not
|
||||
infer behavior from filenames, strings, or layout alone.
|
||||
5. Corroborate a conclusion with the evidence type it requires. Use runtime
|
||||
observation only when static evidence cannot answer the question and the
|
||||
required host runtime and OS access are available. For JavaScript behavior,
|
||||
run probes against the actual app through the available browser, Electron,
|
||||
or process capture workflow.
|
||||
6. Decompose work into independent questions. When parallel workers are
|
||||
available and the questions do not depend on one another, assign distinct
|
||||
scopes, point workers to existing evidence, and ask them to return sources,
|
||||
conclusions, and unresolved gaps. Otherwise, work sequentially.
|
||||
7. Keep a concise finding ledger linking each conclusion to Evidence IDs,
|
||||
confidence/evidence type, search boundary, and remaining unknowns. Update
|
||||
the shared index or investigation report so later passes can reuse results.
|
||||
|
||||
Before finishing, revisit the original checklist. Mark each question as
|
||||
answered, partially answered, or unresolved based on its evidence; keep
|
||||
bounded negative searches bounded, and do not describe a broad investigation
|
||||
as complete while required questions remain open.
|
||||
|
||||
## Read only the relevant guide
|
||||
|
||||
- Native binaries, managed assemblies, archives, and extraction:
|
||||
[references/native-and-artifacts.md](references/native-and-artifacts.md)
|
||||
- ASARs, extracted JavaScript, feature tracing, and version comparison:
|
||||
[references/javascript-applications.md](references/javascript-applications.md)
|
||||
- Android APK declarations, classes, methods, and static references:
|
||||
[references/android-applications.md](references/android-applications.md)
|
||||
- Browser/Electron observation (passive and scenario) and static/runtime
|
||||
reconciliation:
|
||||
[references/runtime-observation.md](references/runtime-observation.md)
|
||||
- Evidence paging, comparisons, residual unknowns, and verification:
|
||||
[references/evidence-workflows.md](references/evidence-workflows.md)
|
||||
|
||||
## Finish the task
|
||||
|
||||
Explain findings in plain language and tie them to returned evidence. When the
|
||||
user asks to build something, use normal coding tools and separate observed
|
||||
behavior from design choices. Close an opened native session with
|
||||
`close_binary` when the investigation is complete.
|
||||
@@ -1,37 +0,0 @@
|
||||
# Android application artifacts
|
||||
|
||||
Use the connected server's advertised Android tools. Repository main and npm
|
||||
4.1.0 include this family. A skill installation alone cannot add it to an
|
||||
older server.
|
||||
|
||||
Start with `inspect_android_package` on the caller's APK path for package
|
||||
identity, manifest declarations, coverage, and inline Evidence. Use
|
||||
`search_android_classes` to find candidates, `inspect_android_class` for one
|
||||
class's members, `inspect_android_method` for decompilation, and
|
||||
`trace_android_references` for incoming static references. Each call takes its
|
||||
own APK path and explicit selectors; do not open a native session first.
|
||||
Choose an `overload_index` from the class inventory when methods are ambiguous
|
||||
instead of guessing.
|
||||
|
||||
The provider requires a separately supplied headless JADX MCP JAR 0.7.1 and
|
||||
Java 17 or newer. `REA_JADX_MCP_JAR` selects the absolute JAR path; `JAVA_HOME`
|
||||
can select Java. REA setup does not install or configure that JAR, and doctor
|
||||
provider scopes cover Hopper/Ghidra rather than JADX. Use existing tools and
|
||||
report missing prerequisites for the Android task specifically.
|
||||
|
||||
Linux and macOS use the POSIX owned-process boundary. Windows x64 requires the
|
||||
matching bundled native controls and uses an owned Job Object with bidirectional
|
||||
stdio. A full JDK, including `jdk.compiler`, is required on every host; Windows
|
||||
`JAVA_HOME` selects `bin\java.exe`. Consult the Android guide for actual host
|
||||
verification coverage. Calls use one worker and provider deadlines; optional
|
||||
`REA_JADX_HEAP_MIB` selects the JVM heap and `REA_JADX_OPERATION_TIMEOUT_MS`
|
||||
the per-operation deadline (integer milliseconds from 1 to 2147483647, default
|
||||
120000; a change applies on the next process start; the MCP client's tool
|
||||
timeout must also allow it). The APK is never launched,
|
||||
and no emulator is required.
|
||||
|
||||
Manifest entries and decompiled text are static observations, not execution or
|
||||
policy enforcement. Incoming references are recovered static relationships;
|
||||
reflection, dynamic loading, and unresolved calls remain unknown. Preserve
|
||||
artifact digests, locations, diagnostics, and incomplete coverage. For archive
|
||||
members rather than code analysis, use `open_binary` and `inspect_artifact`.
|
||||
@@ -1,168 +0,0 @@
|
||||
# Connection and task recovery
|
||||
|
||||
Read this guide only for an observed connection, command, approval or provider
|
||||
blocker. Keep the user's target and question in view. Explain what prevented the
|
||||
operation and the next useful step; after repair, continue the investigation.
|
||||
|
||||
Commands below use an installed `rea`. If none is available, use the package
|
||||
invocation in `SKILL.md` when acquiring REA is authorized. Follow the connected
|
||||
server's schemas and the installed CLI's help rather than assuming repository
|
||||
main and an older release expose the same arguments.
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
|
||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
||||
inspect its launch diagnostic. For example, Codex can refuse helper aliases when
|
||||
CODEX_HOME is under `/tmp`; use an authorized profile outside the OS temporary
|
||||
directory. REA setup cannot repair the client's executor. Preserve the sandbox
|
||||
instead of silently disabling it to make an analysis command work.
|
||||
|
||||
Static JavaScript and ordinary ZIP inventory need neither Hopper nor Ghidra.
|
||||
HAR inspection is offline and does not require a live browser. Native
|
||||
decompilation, Android analysis and firmware extraction have their own
|
||||
bring-your-own prerequisites. Use supported inventory/static evidence when it
|
||||
answers part of the question, and name the unavailable evidence separately.
|
||||
Do not substitute another target. Label observations from ordinary shell tools
|
||||
as such; only actual REA output supplies REA Evidence.
|
||||
|
||||
## Distinguish client and server deadlines
|
||||
|
||||
A host message such as `timed out awaiting tools/call` with no REA result means
|
||||
the client's deadline expired before a result arrived. It does not establish
|
||||
that REA finished, canceled successfully, or released all resources. Do not
|
||||
repeat the unchanged analysis. Inspect available session/provider state and
|
||||
retained Evidence only when useful; an idle `binary_session` is not proof that
|
||||
every target-free operation stopped. Never invent a missing Evidence reference.
|
||||
|
||||
Allow the host tool deadline to cover execution, cancellation/cleanup, and
|
||||
response delivery. In Codex, the setting is
|
||||
`mcp_servers.<server-id>.tool_timeout_sec` in **seconds**, not
|
||||
`startup_timeout_sec`. Select a value for the actual operation budget and
|
||||
headroom, not a universal timeout. See the
|
||||
[official MCP configuration reference](https://learn.chatgpt.com/docs/extend/mcp?surface=cli).
|
||||
Apply configuration changes only within the authorized scope, preserving
|
||||
unrelated entries and backups; reconnect/restart as required by the host.
|
||||
When configuration is not editable, report the setting needed or use an
|
||||
authorized CLI fallback. Do not silently broaden permissions or switch targets.
|
||||
|
||||
A returned REA `provider_timeout` instead means the **server/provider** reached
|
||||
its deadline. Follow its `remediation.action`, reuse accessible partial Evidence,
|
||||
and resolve reported incomplete cleanup before retrying. Raising only the
|
||||
client deadline does not change this server budget. For static JavaScript,
|
||||
`analysis_timeout_ms` (CLI `rea analyze-javascript-application <path>
|
||||
--analysis-timeout-ms <milliseconds>`) controls each source's
|
||||
startup, analysis and transfer deadline in **milliseconds**. Increase that
|
||||
budget or reduce semantic work / select exact inventoried `source_paths` when
|
||||
the task permits. Omit `source_paths` for all sources; `[]` is invalid.
|
||||
`detail: "summary"` reduces delivery, not analysis work. A per-source budget
|
||||
does not bound the elapsed time for an entire multi-source application.
|
||||
Partial-Evidence inspection can avoid recomputation; an unavailable reader is
|
||||
not itself a reason to block a corrected static-analysis retry.
|
||||
The shorthand `rea analyze <path>` does not expose this timeout flag; use the
|
||||
explicit JavaScript command when changing that budget.
|
||||
|
||||
For invalid arguments, use the connected tool's current input schema, or the
|
||||
installed CLI's command help. Refresh the catalog if stale; do not guess fields
|
||||
from an older skill or return the entire catalog to repair one field. Correct
|
||||
all reported fields before retrying.
|
||||
|
||||
## Repair registration when needed
|
||||
|
||||
For Codex:
|
||||
|
||||
```bash
|
||||
rea doctor --client codex --json
|
||||
rea setup --client codex --dry-run --json
|
||||
```
|
||||
|
||||
Use the current supported client's ID: `claude_code`, `claude_desktop`, `codex`,
|
||||
`cursor`, `gemini_cli`, `windsurf`, `devin`, `opencode`, `antigravity`,
|
||||
`copilot_cli`, `commandcode`, `qwen_code`, `vscode`, `trae`, `grok_build`, `omp`, `pi`,
|
||||
`hermes`, `qoder`, `droid`, `zcode`, `dsh`, or `grok_bot`. If unknown, inspect `doctor --json` before choosing a
|
||||
scope. `grok_bot` uses a chat registration step rather than local `mcp.json`.
|
||||
For unsupported clients, use version-pinned manual stdio registration or CLI.
|
||||
|
||||
Show the plan's exact paths, backups, changes and bundled-skill replacement.
|
||||
Obtain approval for writes when that concrete scope is not already authorized,
|
||||
then apply the same scope with `rea setup --client codex --yes`. Add
|
||||
`--install-hopper` only when separately needed and explicitly approved. Do not
|
||||
install or upgrade unrelated prerequisites. Restart/reconnect, verify tools
|
||||
actually appear, and resume the original task. If they remain absent, inspect
|
||||
the launch failure rather than looping through setup.
|
||||
|
||||
## Codex tool approval is separate from REA readiness
|
||||
|
||||
REA tools have no permission-grant handshake. Codex can still require approval
|
||||
because an operation creates owned resources, launches a helper or has cleanup
|
||||
effects. An advertised tool or successful doctor result does not override that
|
||||
client policy. `workspace-write` alone does not resolve an MCP approval
|
||||
rejection, and `approval_policy = "never"` does not mean every MCP tool is
|
||||
preapproved.
|
||||
|
||||
When interactive approval is available, use the client's approval flow for the
|
||||
requested operation. In an unattended run, continue through an authorized CLI
|
||||
workflow if the execution environment permits it, or report the blocked
|
||||
operation and the approval needed to proceed. Do not retry the same rejected
|
||||
call, silently broaden approvals, or rewrite MCP effects as read-only.
|
||||
|
||||
For an operator who explicitly wants to preapprove named operations in a
|
||||
disposable Codex profile, current Codex supports per-tool configuration. For
|
||||
example, opening and closing an analysis session can be scoped independently:
|
||||
|
||||
```toml
|
||||
[mcp_servers.rea.tools.open_binary]
|
||||
approval_mode = "approve"
|
||||
|
||||
[mcp_servers.rea.tools.close_binary]
|
||||
approval_mode = "approve"
|
||||
```
|
||||
|
||||
Use the actual server ID and only the operations whose effects were approved;
|
||||
this example does not approve inspection, extraction or runtime execution.
|
||||
Show the proposed config edit and preserve unrelated entries and backups before
|
||||
writing. Managed policy can constrain client settings. Consult the
|
||||
[Codex configuration reference](https://learn.chatgpt.com/docs/config-file/config-reference)
|
||||
for the installed client's supported settings. This is an optional recovery
|
||||
choice, not a prerequisite for every REA task.
|
||||
|
||||
## Continue through an installed CLI
|
||||
|
||||
Check the command environment only if the launcher fails:
|
||||
|
||||
```bash
|
||||
command -v rea
|
||||
command -v node
|
||||
node --version
|
||||
rea --version
|
||||
```
|
||||
|
||||
Use the public launcher at its known installation path with a supported existing
|
||||
Node on PATH when the child shell loses a prefix installation. Avoid internal
|
||||
`dist/cli.js` entrypoints or global toolchain changes. MCP may have its own valid
|
||||
launcher even when a shell cannot resolve `rea`.
|
||||
|
||||
For a user-supplied JavaScript tree or ASAR, select output files in the authorized
|
||||
workspace and save complete Evidence before reading a view:
|
||||
|
||||
```bash
|
||||
rea analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
|
||||
jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
|
||||
rea inspect-analysis-view app-view.json
|
||||
```
|
||||
|
||||
These commands do not register MCP or install an analysis engine. CLI JavaScript
|
||||
JSON is complete and can be very large; redirect it rather than loading it into
|
||||
context. Use an existing JSON processor if `jq` is absent, without installing
|
||||
one just for this recipe. Inspect a module page or item using the same inline
|
||||
source and the [JavaScript recipes](javascript-applications.md). Preserve graph
|
||||
coverage, limitations and unknowns. Native CLI analysis still requires its
|
||||
selected engine, and all CLI operations retain the host's execution permissions.
|
||||
@@ -1,102 +0,0 @@
|
||||
# Evidence, comparison, and verification workflows
|
||||
|
||||
Evidence-producing tools return the complete Evidence record directly in MCP
|
||||
`structuredContent` and CLI JSON. Read `normalized_result` for operation data,
|
||||
and cite `evidence_id` when another tool or your explanation needs a stable
|
||||
reference. Preserve authority, limitations, coverage, and residual unknowns. If a result
|
||||
is incomplete or paginated, continue only when the remaining data matters to
|
||||
the task; do not fetch a bundle or resource merely to read a result already
|
||||
returned inline. When complete `inspect_binary_layout` or
|
||||
`analyze_javascript_application` Evidence is already retained, call
|
||||
`inspect_analysis_view` for a summary, one section/symbol/module, or a stable
|
||||
page instead of repeating the producer.
|
||||
|
||||
Completed `inspect_artifact` inventory also supports a summary and
|
||||
`view: {kind: "inventory", collection: "occurrences", logical_paths: ["package.json", "main.js"]}`.
|
||||
Omit `logical_paths` to page all available occurrences; `[]` selects none.
|
||||
Other inventory collections are `nodes`, `edges` and
|
||||
`integrity_contradictions`. Keep `source_complete` separate from page exhaustion:
|
||||
a missing match in partial source Evidence does not establish absence. Use the
|
||||
returned parent Evidence ID for subsequent views and export before closing the
|
||||
session when the source must survive it.
|
||||
When `tools/list` advertises `inspect_artifact.detail`, request `"summary"` to
|
||||
retain its complete source and receive that view immediately. Without that
|
||||
advertised choice, keep the existing complete call. CLI inspection returns
|
||||
portable complete Evidence; save it before projecting a selected inline view.
|
||||
|
||||
For native procedure/name/string inventories, select `detail: "summary"` when
|
||||
the connected tool advertises it and the task needs counts or selected rows.
|
||||
Use its parent Evidence ID with
|
||||
`view: {kind: "native-inventory", offset: 0, limit: 32, fields: ["value"]}`.
|
||||
Search tools retain the existing pattern/mode/case/document selection. Omit
|
||||
`fields` for complete records; `record.address` and `source_index` remain on
|
||||
selected rows. Selecting `value` keeps its string/decoding qualifications.
|
||||
Parent Evidence ID plus source index preserves same-address
|
||||
alternatives. Follow `coverage.next_offset` only as far as the question needs;
|
||||
exhausting retained rows does not establish complete program discovery.
|
||||
|
||||
Use `record_unknown` to track unresolved questions in the user's investigation
|
||||
and name the authority or environment still required. Supply supporting and
|
||||
contradicting evidence IDs. It updates the session registry directly.
|
||||
Use `update_unknown` with the current revision; reread after a stale revision
|
||||
instead of retrying blindly. Only qualifying observed evidence can verify a
|
||||
resolution.
|
||||
|
||||
Use comparisons with complete, compatible page sets when claiming equivalence
|
||||
or absence:
|
||||
|
||||
- `compare_artifacts` compares inventory pages by occurrence path, content,
|
||||
metadata, and graph relations.
|
||||
- `compare_functions` compares explicit function Evidence, not fuzzy
|
||||
whole-binary matches.
|
||||
- `compare_bundles` compares canonical bundle membership and unknown history.
|
||||
- `find_changed_behavior` combines existing comparisons; static differences
|
||||
remain candidates, not causal proof.
|
||||
- `build_call_path` needs explicit function dossiers from one artifact/provider;
|
||||
an incomplete frontier makes absence unknown.
|
||||
- `correlate_static_and_runtime` uses explicit mappings; matching patterns are
|
||||
hypotheses, not causality.
|
||||
- `verify_reconstruction` evaluates a finite typed specification. A pass covers
|
||||
only declared comparable claims, not global equivalence.
|
||||
|
||||
Process captures are opt-in behavioral evidence, not a security sandbox. V3
|
||||
captures cannot be upgraded to V4; rerun the original scenario. Distinguish root
|
||||
exit from descendant settlement and require freshness when the task needs it.
|
||||
A timed-out target that must write a final report needs `finalization_ms`: REA
|
||||
sends SIGTERM to the captured root, keeps observing, and sends SIGKILL once
|
||||
only if the target has not exited by the end of the interval.
|
||||
`exit.finalization.signals` lists each attempt with its delivery result; the
|
||||
observed exit signal stays in `exit.signal`, `exit.code` is null for every
|
||||
deadline exit, and the code of a finalizer that exited on its own is in
|
||||
`exit.finalization.exit_code` (null on a signal exit or when no exit was
|
||||
observed).
|
||||
|
||||
Use `inspect_analysis_view` with `view: {kind: "api-surface"}` on retained
|
||||
`inspect_web_network_capture` HAR Evidence to group observed request lines by
|
||||
method, origin and path template. Format matches are inferences, not handler
|
||||
identity or schemas. Group pages cite parent record ordinals; consult record
|
||||
coverage and exclusions before claiming completeness. No requests are replayed.
|
||||
When the connected schema advertises `include_query_parameter_names`, select
|
||||
it in the view for distinct observed names with original record ordinals. Query
|
||||
values are omitted; the names establish neither required parameters nor a
|
||||
complete server schema. Keep the original parent Evidence for exact URLs.
|
||||
|
||||
Use `inspect_analysis_view` with `view: {kind: "environment-manifest"}` on the
|
||||
same retained HAR Evidence to project the primary origin's request lines into
|
||||
named tools with path templates, parameters, observed query names, value-free
|
||||
body shapes and record pointers for an agent environment. Opt into
|
||||
`template_rules` only when recorded identifiers need them; every template,
|
||||
parameter and merged sibling is an inference, no header, cookie or body value is
|
||||
copied, and the three stated unknowns stay unresolved.
|
||||
|
||||
Use `view: {kind: "environment-state"}` with the manifest's `template_rules` to
|
||||
extract the data those reads observed instead: entity types and identity
|
||||
fields, entities with recorded values, documents, and per-tool render plans
|
||||
checked against the recordings. The state comes only from reads before the
|
||||
first related write; types, identity and list membership are inferences.
|
||||
`rea environment-serve manifest.json --evidence capture.json` derives that
|
||||
state at startup, answers read tools from it and serves `data/reset`,
|
||||
`data/add` and `data/get` on `/agentenv`; write tools are not served. It
|
||||
replays the capture instead with `--replay-only` or when the state is
|
||||
unavailable (not derivable, or a catalog over the discovery budget), and its
|
||||
stopped document's `state` says which applied.
|
||||
@@ -1,226 +0,0 @@
|
||||
# JavaScript and Electron application artifacts
|
||||
|
||||
For captured or bundled JavaScript that needs readable source recovery, use
|
||||
`recover_javascript_sources` with an absolute `path` and absent
|
||||
`output_directory`. Supply related runtime/shared/lazy files in
|
||||
`additional_paths` when advertised, then analyze the returned `analysis_input`
|
||||
directory together. Each input retains its own snapshot, digest and byte-range
|
||||
provenance; inputs and recovered code are never executed. Recovery requires a
|
||||
caller-supplied Wakaru on Linux x64; see [the recovery guide](https://github.com/morluto/rea/blob/main/docs/javascript-recovery.md).
|
||||
|
||||
Use `analyze_javascript_application` directly on the operator-supplied ASAR or
|
||||
extracted tree. When the question concerns known source paths and the connected
|
||||
schema advertises `source_paths`, select exact inventoried paths such as
|
||||
`["package.json", "main.js", "preload.js"]`. Include supporting manifests or
|
||||
source maps explicitly; dependencies are not implicitly analyzed. REA retains
|
||||
the complete artifact inventory and identity but reads source text and analyzes
|
||||
only selected files. The result and summary record `source_selection`; files
|
||||
outside scope and their relationships remain unknown, so partial application
|
||||
coverage does not mean the selected source failed. Use repeated `--source-path`
|
||||
on either static application CLI route. Omit the selection when the question
|
||||
requires complete application analysis; `source_paths: []` is invalid, not an
|
||||
unfiltered request. `detail: "summary"` reduces delivery
|
||||
and does not by itself reduce analysis work.
|
||||
|
||||
ASAR header/index admission is fixed at 16 MiB / 16777216 bytes for this
|
||||
release and has no runtime override. `source_paths`, `detail`, and
|
||||
`max_heap_mb` do not bypass or raise it. Analyze an extracted directory, or
|
||||
supply an archive whose header fits.
|
||||
|
||||
Exact `#alias` specifiers resolve through the importer's
|
||||
enclosing package `imports` map to package-local files; nested `node_modules`
|
||||
packages use their own map. Wildcard, recursive, and external-package import
|
||||
targets stay unresolved with that limitation. When size is unknown or large and the connected schema advertises
|
||||
`detail`, request `detail: "summary"`. This returns an `inspect_analysis_view`
|
||||
Evidence record with a distinct ID and a `normalized_result.parent_evidence_id`
|
||||
for the complete retained application analysis. Source-map contents are part of
|
||||
that static analysis. Read findings, coverage, limitations and unknowns, then
|
||||
follow a specific unanswered question. Older servers without `detail` return
|
||||
the complete application Evidence inline when it fits.
|
||||
|
||||
When the connected schema advertises `max_heap_mb` and `analysis_timeout_ms`,
|
||||
these select the isolated source analyzer's old-space heap and per-source
|
||||
deadline. Defaults are 1024 MiB and 300000 ms. They do not change the MCP
|
||||
process heap or the client's tool deadline. For a host timeout without REA
|
||||
output, use the [deadline recovery guide](connection-and-recovery.md#distinguish-client-and-server-deadlines)
|
||||
instead of raising the server budget. A Bun-hosted REA does not enforce the heap choice; on Windows it
|
||||
instead limits the analyzer's committed memory to that choice plus a 512 MiB
|
||||
runtime reserve.
|
||||
Resource-limited analyses preserve completed static facts and
|
||||
report partial semantic coverage with specific unknown scopes; this does not
|
||||
mean the source is malformed. Inspect those limits before retrying a source.
|
||||
When expanded relationships exceed the parent's available memory, inspect the
|
||||
file's modules item view for retained `static_analysis` and
|
||||
`semantic_module_analysis`. Their facts remain available even when graph links
|
||||
are unknown. Cumulative result exhaustion can stop analysis before every file
|
||||
is examined; the inventory and remaining unknown scopes distinguish that case.
|
||||
Timeout or cancellation after source analysis begins retains partial Evidence
|
||||
in the same session. A returned `error.details.partial_observation` reference
|
||||
can be supplied directly to `inspect_analysis_view`. If the SDK cancels locally, retrieve its recorded ID from
|
||||
the Evidence bundle before inspecting the partial analysis.
|
||||
|
||||
## Read a module without repeating analysis
|
||||
|
||||
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
|
||||
The following IDs are placeholders to replace with returned values. When the
|
||||
module's node ID is not already known, page identities before selecting an item.
|
||||
Pages return only node IDs, paths and roles; choose a useful page size and follow
|
||||
`coverage.next_offset` when another page matters:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": { "kind": "page", "collection": "modules", "offset": 0, "limit": 16 }
|
||||
}
|
||||
```
|
||||
|
||||
Select the relevant page entry by its exact `node_id` and recorded role:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": {
|
||||
"kind": "item",
|
||||
"collection": "modules",
|
||||
"selector": { "node_id": "jag_node_ACTUAL_NODE_ID" }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Module pages include asset, bundled and source module roles, so one path can
|
||||
identify several entries. Use `selector.path` only when uniqueness is established;
|
||||
the returned node ID avoids needing a complete inventory to prove uniqueness. On ambiguity,
|
||||
select the matching node ID from the page or the error's candidates. Follow
|
||||
`next_offset` when another page matters. An item exposes that module node's
|
||||
recorded observations, identity, hashes, exports and source ranges; it does not
|
||||
include every child syntax observation, related edge or original source file.
|
||||
An absent item fact is not evidence of absence from the whole application.
|
||||
|
||||
For canonical identities and retained source-range counts, use a typed reference
|
||||
page instead of reading every full item:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_ACTUAL_PARENT_ID"
|
||||
},
|
||||
"view": {
|
||||
"kind": "javascript",
|
||||
"collection": "modules",
|
||||
"projection": "references",
|
||||
"where": { "path": "main.js", "kind": "javascript-module" },
|
||||
"offset": 0,
|
||||
"limit": 16
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Filters match exact retained strings; omit a filter to include all values, use
|
||||
`path: null` for unavailable paths, and `path: ""` for a recorded empty path.
|
||||
Rows preserve canonical `identity` and distinct observation support states in
|
||||
`basis`. `retained_source_range_count` counts retained source-range observations,
|
||||
not runtime imports or unique reference sites. Use a row's `node_id` with the item
|
||||
view above for exact locations and full supporting observations. Follow
|
||||
`coverage.next_offset`; page exhaustion describes matching retained rows only.
|
||||
Check `source_coverage` for source selection, partial/unknown graph coverage and
|
||||
unknown counts before drawing a negative conclusion.
|
||||
|
||||
## Trace only the remaining question
|
||||
|
||||
Use `trace_application_feature` on the parent application Evidence for one
|
||||
literal node ID, route, string, API, IPC channel, module, or native export.
|
||||
Select `incoming` for relationships leading to the seed, `outgoing` for those
|
||||
leaving it, and `both` when the question needs both. Trace traversal includes
|
||||
containment relationships, so even a narrow seed can expand many nodes; a
|
||||
bidirectional trace can reach unrelated sibling modules. A module item or page
|
||||
is preferable when its observations already answer the question. Do not treat
|
||||
all reached nodes as participants in the feature.
|
||||
|
||||
Reuse retained parent Evidence when the server advertises it, or supply complete
|
||||
Evidence inline. Include Hopper or Ghidra Evidence only when its artifact digest
|
||||
matches exactly.
|
||||
|
||||
## Recover an oversized display
|
||||
|
||||
An oversized response reports `resource_constraint` and
|
||||
`details.resource: "transport"`. Reuse
|
||||
`details.reported_limits.evidence_reference` with `inspect_analysis_view` for a
|
||||
summary, one module, or a stable page of module identities.
|
||||
`export_evidence_bundle` writes the complete canonical session to a
|
||||
caller-selected file. These workflows keep the original Evidence and coverage;
|
||||
a broad follow-up can also exceed framing.
|
||||
|
||||
A client can truncate a successful result inside its text envelope without a
|
||||
REA `resource_constraint`. Treat a truncation notice or incomplete JSON as a
|
||||
display limitation; do not claim the full record was examined. Reuse the known
|
||||
parent ID for smaller views, or export complete Evidence to a selected file
|
||||
while the session remains open. If the parent ID was lost from the preview,
|
||||
use the prior analysis summary or explicit ledger entries to recover it; do
|
||||
not invent an ID or rerun analysis just to reproduce the same display.
|
||||
|
||||
BrowserWindow preferences, preload and contextBridge surfaces, IPC
|
||||
registrations, utility processes, and native binding requests are static syntax
|
||||
observations. Only a unique exact literal IPC channel match supports an inferred
|
||||
pairing. Dynamic or ambiguous channels remain unresolved. A requested `.node`
|
||||
member is not a verified native export. Never claim runtime reachability,
|
||||
registration, defaults, or policy enforcement from static analysis.
|
||||
|
||||
For version comparison, analyze each version once, then call
|
||||
`compare_application_versions`. Accept only its digest, source-map, structural
|
||||
fingerprint, or non-module semantic matches. Module ordinals and minified names
|
||||
are not persistent identity. Report added or removed only with complete
|
||||
opposite-side coverage; otherwise report unknown.
|
||||
|
||||
When the question asks how one exact exported callable's returned object shape
|
||||
changed, analyze each version once and then call
|
||||
`compare_javascript_export_shapes` with explicit module paths and export names.
|
||||
Use the returned IDs on the same connection, or complete inline Evidence
|
||||
records, from both analysis calls. Accept variant
|
||||
pairing only through the tool's unique exact literal discriminant. Read
|
||||
`property_inventories` and each change's `presence` before treating `unknown` as
|
||||
"the name was not observed": inventories list observed property names even when
|
||||
values stay unknown, with a `source_range` for each paired or unpaired variant.
|
||||
Array holes and mutation-invalidated slots are not observed properties.
|
||||
Complete parent coverage reports presence-only
|
||||
`added`/`removed` independently of unresolved values. Cite the
|
||||
comparison Evidence and report JSON Pointer changes; dynamic values, ambiguous
|
||||
variants, and incomplete parent-property coverage stay unknown. This is static
|
||||
inference, not runtime behavior. When runtime semantics are needed, run
|
||||
behavioral probes against the relevant application versions and capture them
|
||||
through the available browser, Electron, or process workflows.
|
||||
|
||||
## Reusing application Evidence
|
||||
|
||||
When advertised by the connected server, application trace and compare tools
|
||||
accept complete inline Evidence or
|
||||
`{"kind":"retained-evidence","evidence_id":"ev_<64 lowercase hex characters>"}`
|
||||
for their application input (`application`, or `left`/`right`). `inspect_analysis_view`
|
||||
uses the same retained-reference form in `source`, or portable inline Evidence,
|
||||
to project a summary, module page, or one module with its recorded observations.
|
||||
This notation is
|
||||
a template: replace it with the actual returned ID. Versions before 4.1.0 accept only full
|
||||
inline Evidence. Use the exact ID
|
||||
returned by the producer on the same MCP connection. Resolution does not run
|
||||
analysis or select a provider; findings remain inline. Native observation
|
||||
arrays still take full Evidence. `close_binary` clears retained references;
|
||||
export a bundle before closing, or supply portable inline Evidence in another
|
||||
connection. A missing reference includes its ID and recovery guidance.
|
||||
|
||||
For exact file/native metadata or processing gaps, use
|
||||
`view: {kind: "javascript-facts", where: {paths: ["package.json"]}, property_keys: ["name", "operation", "analysis_status"], limit: 32}`.
|
||||
Rows retain canonical node/observation identity and original Evidence. Paths
|
||||
match Evidence artifact-path/source-range locations; missing properties remain
|
||||
omitted and source coverage stays separate from page exhaustion. Batch paths and
|
||||
select only needed property keys; complete Evidence remains accessible by export.
|
||||
If names are unknown, opt into `include_available_property_keys: true`; it lists
|
||||
the original keys per returned observation without unselected values. Keep each
|
||||
row's node/observation identity paired when composing facts.
|
||||
These rows select application-graph observations; semantic relationships and
|
||||
semantic unknown records remain in the complete parent Evidence and trace tools.
|
||||
@@ -1,433 +0,0 @@
|
||||
# Native, managed, and packaged artifacts
|
||||
|
||||
For release comparisons, export `list_procedures` Evidence from each target before
|
||||
closing it. Close clears session records: import both bundles after collecting the
|
||||
two targets, or supply portable inline Evidence. `match_native_functions` pairs unique exact names and, when supplied,
|
||||
complete observed body bytes. Reuse optional same-context `list_names`,
|
||||
`procedure_info`, `read_bytes` and existing `analyze_function` Evidence; avoid
|
||||
decompiling every function merely to match it. Its summary retains a parent for
|
||||
`inspect_analysis_view` `function-matches` collections `matches` and `functions`.
|
||||
Select columns, batch IDs and page individual facts; aliases remain unresolved.
|
||||
Unpaired observations do not prove additions/removals. Existing matched dossier
|
||||
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
|
||||
|
||||
## Apple app resources
|
||||
|
||||
Use `open_bundle_resources` with the selected app path, then `inspect_plist`,
|
||||
`decode_interface_builder`, `inspect_asset_catalog`, or `inspect_keyed_archive`.
|
||||
The resource session reads the app without resolving `CFBundleExecutable` or
|
||||
starting a deep provider. Preserve `resource_root` provenance, per-resource
|
||||
hashes where reported, and null executable identity; a directory path is not
|
||||
an executable digest. Export retained Evidence before closing or switching
|
||||
roots. Binary analysis snapshots do not apply. Use `open_binary` to switch to
|
||||
executable analysis, and `close_binary` to release either session kind.
|
||||
|
||||
The four matching CLI commands accept the app path directly. For an archive
|
||||
inside the app, use `inspect-keyed-archive APP_PATH RELATIVE_ARCHIVE_PATH`.
|
||||
Host requirements for plist and compiled asset/Interface Builder tooling still
|
||||
apply; a resource-only session does not establish provider availability.
|
||||
|
||||
## Desktop installation directories
|
||||
|
||||
When an installation directory is supplied and its runtime is unknown, use
|
||||
`inspect_desktop_application` with its absolute path. Default MCP summary
|
||||
delivery retains complete Evidence. Read `files`, `omissions`, `candidates`, `components` or
|
||||
`next_probes` with `inspect_analysis_view` kind `desktop-application`, selecting
|
||||
exact paths, batched file/omission IDs or families and only needed fields and pages.
|
||||
Summary `probe_issues` retains counts and bounded previews. Select files with
|
||||
`has_issues: true` and fields `path`, `read_status`, `issues` to retrieve exact
|
||||
failures directly; preview text is display data, while `file_id` selects full facts. Preserve source
|
||||
coverage, read-range identity and unknowns; directory snapshot identity is not
|
||||
a digest of every file's contents. Component path markers do not establish
|
||||
runtime presence. For compact CLI output, use
|
||||
`inspect-desktop-application PATH --detail summary --evidence_output BUNDLE`.
|
||||
See [desktop installations](https://github.com/morluto/rea/blob/main/docs/desktop-applications.md).
|
||||
|
||||
## Go build metadata
|
||||
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or thin Mach-O when the
|
||||
question concerns the embedded compiler version, module/dependency versions,
|
||||
local replacements or build settings. No `open_binary`, Go installation or
|
||||
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
|
||||
Preserve artifact SHA-256, original file ranges, framed module bytes and every
|
||||
unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## Model files
|
||||
|
||||
Use `inspect_model_artifact` (`inspect-model-artifact PATH`) for local GGUF v2/v3
|
||||
and safetensors declarations without a model runtime. Header-only mode does not
|
||||
read weights or establish full-file identity. `hash: "full"` explicitly streams
|
||||
all bytes. Retained `model-artifact` views page `metadata`, `tensors` and derived
|
||||
`types`, with exact batch filters and selected fields. Arrays retain typed counts
|
||||
and eight-element previews; omitted values, unknown types and runtime behavior
|
||||
stay explicit. See [model files](https://github.com/morluto/rea/blob/main/docs/model-artifacts.md).
|
||||
|
||||
## IL2CPP metadata
|
||||
|
||||
Use `inspect_il2cpp_metadata` for an explicit local Unity `global-metadata.dat`
|
||||
(versions 24, 27, 29, 31 and 39) when the question concerns assembly images,
|
||||
names, member owners, string literals, namespaces or index ranges. No `open_binary`, Unity installation or target
|
||||
execution is required. The CLI equivalent is `inspect-il2cpp-metadata PATH`.
|
||||
MCP defaults to counts with complete retained Evidence. Query `sections`,
|
||||
`images`, `types`, `methods`, `fields`, `properties`, `events`, `parameters` or
|
||||
`string_literals` through `inspect_analysis_view`; use exact names,
|
||||
batch indices and selected fields. Row indices are scoped by artifact SHA-256
|
||||
and collection; page offsets count filtered matches. Parameters support batch
|
||||
`method_indices` to read one method's parameters directly. Historical reports
|
||||
without member collections retain unknown counts rather than empty inventories. Preserve
|
||||
artifact SHA-256 and index widths. Unrecognized magic may mean encryption and
|
||||
is unsupported, not decrypted. Linking names to `GameAssembly` or
|
||||
`UnityFramework` code addresses remains a separate native-analysis question.
|
||||
See [IL2CPP metadata](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
|
||||
|
||||
## Rust dependency metadata
|
||||
|
||||
Use `inspect_rust_binary` for an explicit local ELF, PE, or thin Mach-O when
|
||||
the question concerns the cargo-auditable crate list embedded in `.dep-v0`
|
||||
(Mach-O: `__DATA,.dep-v0`). No `open_binary`, Rust installation, or target
|
||||
execution is required. The CLI equivalent is `inspect-rust-binary PATH`.
|
||||
Preserve artifact SHA-256, package order, dependency indices, and the section
|
||||
file range. A missing section is null metadata, not proof the binary has no
|
||||
dependencies. An omitted package kind stays null. A rustc commit hash is
|
||||
inferred from a read-only `/rustc/<40-hex>/` path and does not by itself
|
||||
identify the compiler. See [Rust binaries](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
|
||||
## PE Authenticode certificate tables
|
||||
|
||||
Use `inspect_pe_certificates` for an explicit local PE32/PE32+ image when the
|
||||
question concerns the Authenticode certificate table, WIN_CERTIFICATE types, or
|
||||
embedded PKCS#7 signer and X.509 identity fields. No `open_binary`, Windows SDK,
|
||||
or target execution is required. The CLI equivalent is
|
||||
`inspect-pe-certificates PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
payload hashes, and original validity-time spellings. A missing
|
||||
`IMAGE_DIRECTORY_ENTRY_SECURITY` directory is observed absence of that table,
|
||||
not proof the image is unsigned in every catalog or overlay sense. Signatures,
|
||||
chains, timestamps, and Authenticode digest coverage remain unobserved. See
|
||||
[PE Authenticode certificate tables](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
Use `inspect_sqlite_database` with an explicit local database snapshot for its
|
||||
schema and optional rows from one exact ordinary table. It requires Node.js
|
||||
24.x >=24.15 or 26+ with the native `DatabaseSync.limits` API. It also consumes the
|
||||
existing sibling `-wal`; retain both original digests when citing its result.
|
||||
Default output is a compact inventory; a selected table expands only its own
|
||||
schema and samples 20 rows. Select `columns` to avoid irrelevant or large cells,
|
||||
and request `schema_scope: "database"` only for complete schema details.
|
||||
Choose `row_limit` when a finite sample is useful and keep truncation and
|
||||
unspecified row order explicit. Virtual tables, shadow tables and views are
|
||||
schema facts, not row-read targets. REA inspects private copies and leaves
|
||||
the source files untouched; obtain a coherent quiescent database/WAL snapshot
|
||||
before inspection. See [SQLite snapshots](https://github.com/morluto/rea/blob/main/docs/sqlite-databases.md).
|
||||
|
||||
## Recorded Linux crashes
|
||||
|
||||
Use `inspect_recorded_crash` with an explicit Linux x86-64 ELF core path to
|
||||
inspect historical thread registers, signals, raw notes and original file
|
||||
ranges. It is independent of the active disassembler target. Source-bound
|
||||
register values and note bytes are observations; missing notes, signal-thread
|
||||
association and current executable/library identity remain unknown. PIDs are
|
||||
historical metadata and never authorize live attach or control.
|
||||
|
||||
The optional `include_debugger_context` facet adds mapping candidates through
|
||||
caller-supplied GDB/pwndbg in an owned core-only session. Display names do not
|
||||
establish current file identity; zero reported flags leave permissions unknown.
|
||||
Requested unavailable context fails with setup guidance. See
|
||||
[recorded crashes](https://github.com/morluto/rea/blob/main/docs/recorded-crashes.md)
|
||||
for exact upstream profiles, bounds and verification coverage.
|
||||
|
||||
## Offline ELF layout
|
||||
|
||||
Use `inspect_binary_layout` with an explicit local ELF path for file-backed
|
||||
sections, symbols, relocations, linkage names, and static mitigation inferences
|
||||
without opening a disassembler target. When that complete record is already
|
||||
retained, call `inspect_analysis_view` for a summary, mitigations or linkage
|
||||
facet, one section or symbol, or a stable page. Do not re-run the decoder to
|
||||
read a single object.
|
||||
|
||||
## Offline WASM
|
||||
|
||||
Use `inspect_wasm_artifact` with caller-supplied WABT for selected local bytes.
|
||||
For counts, validation and WAT identity, select `detail: "summary"` on the first
|
||||
MCP call. The complete original Evidence stays retained; use
|
||||
`normalized_result.parent_evidence_id` for exact exports or text pages. Omitted
|
||||
or complete detail keeps the existing complete result.
|
||||
For a completed result, use `inspect_analysis_view` with its reported retained
|
||||
reference for a summary or `view: {kind: "wasm", facet: "exports", offset: 0, limit: 32}`.
|
||||
Facets also include sections, imports, candidates, glue, WAT, headers, details
|
||||
and tool profile. Text offsets count UTF-16 code units; follow `next_offset`.
|
||||
The complete source remains retained after an oversized MCP reply. No target
|
||||
execution or network acquisition is implied, and WAT is decoded representation.
|
||||
|
||||
## Native targets
|
||||
|
||||
After `open_binary`, use focused search, procedure, or function tools directly.
|
||||
Use `binary_overview` when target metadata or inventory context helps answer the
|
||||
question. Prefer literal search, names, decompilation, callers, callees, and
|
||||
cross-references. Addresses and recovered pseudocode are analysis observations,
|
||||
not original source. Provider unavailability and unsupported metadata remain
|
||||
unknown rather than false.
|
||||
|
||||
For a completed `analyze_function` dossier, use `inspect_analysis_view` with
|
||||
`view: {kind: "native", facet: "value_flow_operations", offset: 0, limit: 32}`
|
||||
and the exact retained Evidence ID to read selected facts without reanalysis.
|
||||
Native facets also include procedure, pseudocode, assembly, basic blocks,
|
||||
comments, callers/callees, references, unresolved calls, referenced strings and
|
||||
names, the native API record, and value-flow summaries. Pseudocode offsets and limits count UTF-16
|
||||
code units; use the returned `next_offset` for continuation. Provider limitations
|
||||
and unavailable facts remain explicit; the complete parent Evidence is retained.
|
||||
|
||||
`inspect_macho` on a large framework returns megabytes of imports and exports.
|
||||
Select `detail: "summary"` for slice identity, UUID and per-collection counts,
|
||||
then page rows with `view: {kind: "macho", facet: "exports", offset: 0, limit: 64}`
|
||||
and `normalized_result.parent_evidence_id`. Facets are entrypoints,
|
||||
architectures, build_metadata, load_commands, dependencies, imports, exports,
|
||||
segments and provenance. `source_exhaustive: false` means unreturned symbols
|
||||
may exist.
|
||||
|
||||
To see which functions or Objective-C methods a Mach-O actually calls in one
|
||||
run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||
runs and may change files or use the network. Hardened-runtime targets need the
|
||||
`get-task-allow` entitlement.
|
||||
|
||||
Use `binary_session` with no arguments to check the open target, selected
|
||||
provider, and alignment. Its default `result.tool_availability` carries a
|
||||
complete inventory grouped by shared facts. Expand each group's `names` and
|
||||
inherit the inventory-level client requirements unless a group overrides them;
|
||||
availability, reasons, remediation, modes, and flags remain inline. The
|
||||
`effects` and `annotations` arrays name true flags from fixed vocabularies, so an
|
||||
omitted known flag is false. `tools/list` remains the canonical source for full
|
||||
tool schemas when target or provider state changes.
|
||||
|
||||
Provider capability groups carry provider identity and ordered limitations.
|
||||
Prepend each set's `shared_limitations` to a group's `limitations` to recover the
|
||||
operation's full ordered list. Candidate capabilities describe possible
|
||||
providers. Top-level session capabilities describe the current route, and
|
||||
`analysis_provider_binding` identifies the selected provider.
|
||||
|
||||
## Ghidra startup deadline recovery
|
||||
|
||||
`provider_timeout` during the first Ghidra query can cover import, auto-analysis,
|
||||
bridge startup, and readiness. The default 330,000 ms is configurable through
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS`; a smaller query still needs the same cold import.
|
||||
|
||||
Preserve useful Evidence before closing the session or restarting REA. Check
|
||||
`close_binary` and resolve reported cleanup failures. Set a larger startup budget
|
||||
in the environment that launches REA, then restart REA and reconnect the MCP
|
||||
client before reopening the same target. Closing and reopening a binary in the
|
||||
existing server keeps its original configuration and starts import again.
|
||||
Environment changes in an agent's shell do not update an already running MCP
|
||||
server. If its launch configuration cannot be changed, report that constraint
|
||||
and the exact retry setting instead of claiming a fixed technical limit.
|
||||
|
||||
The client's tool-call deadline must allow startup plus the requested query;
|
||||
server settings and progress notifications cannot override it. CLI analysis
|
||||
avoids the MCP client deadline, but retains the provider deadline. For example:
|
||||
|
||||
```sh
|
||||
REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000 rea analyze /absolute/path/to/app.exe --provider ghidra --json
|
||||
```
|
||||
|
||||
Thirty minutes is an example budget, not a completion guarantee. Keep host CPU
|
||||
and memory controls in place. Disable specific expensive analyzers only when
|
||||
their missing coverage is acceptable and retain the resulting limitations.
|
||||
Exported C and embedded resources remain useful independent evidence, but a
|
||||
timeout does not establish complete analysis or absence of code. See the
|
||||
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
|
||||
## Static memory and original file bytes
|
||||
|
||||
For Ghidra, `read_bytes` reads initialized analysis memory, including loader
|
||||
fixups; it does not read a running process. Check `returned_bytes` and `complete`
|
||||
before decoding a table. An incomplete read is missing coverage, not a zero-filled
|
||||
tail. Use `list_segments` to inspect the selected block's initialization and address-space
|
||||
facts, and retain unavailable facts as unknown.
|
||||
|
||||
Use `address_to_file_offset` to anchor one observed address to original file
|
||||
bytes. A mapping for the first address does not establish a contiguous mapping
|
||||
for an entire table, and loader fixups can make analysis bytes differ from source
|
||||
bytes. Keep source ranges and analysis-memory observations separate.
|
||||
|
||||
A pointer table can refer to writable slots populated by a constructor rather
|
||||
than directly to immutable resources. An uninitialized block or an observed zero
|
||||
does not establish the slot's later runtime value. Follow `xrefs` and the relevant
|
||||
writer's assembly or pseudocode; distinguish recorded assignments from inferred
|
||||
execution order. Do not replace unreadable bytes with zeros or conclude that a
|
||||
resource is absent without the necessary initialization evidence.
|
||||
|
||||
## DOS MZ with Ghidra
|
||||
|
||||
For an admitted DOS MZ target, use `inspect_native_load_image` to check measured
|
||||
loaded bytes, file mappings, relocations and the entry against the immutable
|
||||
snapshot. Keep `mismatch` and `unsupported` explicit. Verified import covers the
|
||||
reported static image, not runtime DOS or PC-98 hardware behavior.
|
||||
|
||||
For Ghidra decompilation, `procedure_pseudo_code` also accepts `procedure: "file-offset:0x..."`; CLI `rea decompile <path> --file-offset=0x... --provider ghidra` resolves the source byte offset internally. An unresolved 0x-prefixed numeric procedure selector also falls back to a unique file mapping. Existing symbols and analysis addresses win; multiple mappings fail as ambiguous. Preserve the returned file-offset and address locations and raw selection observations. ELF virtual addresses are distinct from file offsets; never assume an arbitrary offset plus image base is an analysis address.
|
||||
|
||||
Complete function body ranges use inclusive ends and can contain gaps. Do not
|
||||
treat the enclosing span as code.
|
||||
For packed targets, retain the original and separately derived artifact identities;
|
||||
decompiling an unpacking stub does not recover the unpacked program.
|
||||
|
||||
When reading Objective-C dispatch metadata, distinguish class implementation
|
||||
methods from protocol requirements: class methods report null requirement
|
||||
flags, while decoded protocol declarations retain required and optional facts.
|
||||
Property attributes remain raw name/value tokens; use the property's
|
||||
`atomicity` field for the decoded `N` marker or the default atomic behavior.
|
||||
Null class/meta flags, ivar counts, atomicity, or read-only values mean the
|
||||
selected provider did not establish that fact. Symbol names alone do not prove
|
||||
that a class is a root class or has no ivars.
|
||||
|
||||
## Managed PE/CLI
|
||||
|
||||
Start with `inspect_managed_artifact`. REA's canonical managed inspection is
|
||||
execution-free: do not claim it loaded, reflected, executed, or resolved the
|
||||
assembly. Keep managed/native boundaries and unavailable reconstruction facts
|
||||
explicit. A bring-your-own reconstruction oracle is separate from the canonical
|
||||
parser and must not become an implicit setup dependency.
|
||||
|
||||
For identity/classification questions, `inspect_managed_artifact` can explicitly
|
||||
select `detail: "summary"` on the first MCP call. It retains complete metadata
|
||||
and returns a linked overview with full classification/evidence, nullable
|
||||
module/MVID, frameworks, metadata coverage/issues/limitations and returned-row
|
||||
counts. Default complete requests remain unchanged. Read original rows with
|
||||
`inspect_analysis_view`, source `normalized_result.parent_evidence_id`, and
|
||||
`view: {kind: "managed", collection: "attributes", offset: 0, limit: 16}`.
|
||||
Collections are `references`, `resources` and `attributes`. Paging does not
|
||||
repeat parsing. Returned-row exhaustion
|
||||
does not prove complete metadata or absence under partial/unavailable coverage;
|
||||
preserve build-local tokens and their artifact/MVID scope.
|
||||
|
||||
`inspect_managed_members` always returns the assembly-wide `member_refs`, even
|
||||
for one selected type. For questions about a selected type, select
|
||||
`detail: "summary"` with `type_names`; it returns the selection (with unmatched
|
||||
names), coverage and per-collection counts. Page rows with
|
||||
`view: {kind: "managed-members", collection: "call_edges", offset: 0, limit: 64}`
|
||||
over `types`, `exported_types`, `fields`, `methods`, `member_refs`, `call_edges`,
|
||||
`field_accesses` or `user_strings`.
|
||||
|
||||
### .NET NativeAOT
|
||||
|
||||
NativeAOT binaries contain native machine code, so ordinary CIL decompilers
|
||||
cannot recover the original method bodies. When managed metadata/CIL is absent,
|
||||
classify that as a native-analysis route rather than claiming source recovery.
|
||||
Open the native image with the selected deep provider and inspect functions,
|
||||
strings, calls, and references; describe decompiler output as pseudocode and
|
||||
inference.
|
||||
|
||||
NativeAOT images can be PE/COFF, ELF, or Mach-O executables and shared
|
||||
libraries. Use file bytes and loader metadata rather than suffixes to identify
|
||||
them. For packaged apps, inventory/extract with REA's artifact tools and retain
|
||||
the exact contained image identity. PDB, ELF debug, or dSYM sidecars are
|
||||
additional evidence only when the provider accepts and matches them.
|
||||
|
||||
REA has a built-in read-only snapshot parser for x64 PE targets with .NET 8 RTR
|
||||
9.1 metadata. An optional headless Ghidra adapter adds database annotations and
|
||||
broader metadata recovery on Linux; it requires an explicitly configured
|
||||
NativeAOT JAR. These are separate routes. The built-in parser also runs on
|
||||
Windows Ghidra P0; read frozen literals from its report, because
|
||||
`search_strings` does not include them. See the
|
||||
[NativeAOT guide](https://github.com/morluto/rea/blob/main/docs/ghidra-nativeaot.md)
|
||||
for setup, supported layouts, and host restrictions. Preserve binary identity
|
||||
and distinguish derived metadata from Ghidra database observations. Recovered
|
||||
types and native pseudocode are not original C# source. Windows Ghidra P0 has
|
||||
no database mutation authority.
|
||||
|
||||
## Packages and extraction
|
||||
|
||||
Use `inspect_artifact` for application bundles, archives, ZIP/APK/IPA/MSIX/AppX,
|
||||
ASAR, or DMG inputs when the artifact graph and findings help answer the
|
||||
question. It returns the complete artifact graph inline. Cite graph manifest
|
||||
IDs when using them.
|
||||
|
||||
For an IPA or a macOS `.app`, ZIP, or DMG, pass the inventory Evidence to
|
||||
`project_apple_application_graph`. It reports application roots and nested
|
||||
bundles by path convention: app extensions, XPC services, frameworks, login
|
||||
items, system and driver extensions, and plug-ins. It also lists privileged
|
||||
helpers, launchd plists, symlinks, and each bundle's `info_plist_path` and
|
||||
executable candidates. Roles are path conventions, not parsed plists; read the
|
||||
listed plists with `inspect_plist`.
|
||||
|
||||
After `open_binary` on a `.app` or Mach-O, use `trace_dylib_resolution` to see
|
||||
which file each `@rpath`, `@loader_path` and `@executable_path` load reaches for
|
||||
every executable in the bundle. It also lists missing or weak loads and earlier
|
||||
`@rpath` candidates that are absent. Narrow large bundles with `roots` or
|
||||
`architecture`. System paths stay undetermined because the dyld shared cache
|
||||
provides them.
|
||||
|
||||
Use `extract_artifact` when materialized files from a supported format are needed.
|
||||
It extracts the active target and accepts optional `integrity_policy`; see the [MCP contracts](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md).
|
||||
It materializes all regular files into a fresh temporary directory chosen by
|
||||
REA. Symlinks and encrypted entries are inventory facts, not extractable files.
|
||||
The result reports the materialized directory; callers do not choose the
|
||||
destination. No separate permission grant is needed.
|
||||
|
||||
Native DMG traversal automatically uses a read-only mount on supported macOS
|
||||
hosts, with an owned temporary mount directory and cleanup. It needs no approval
|
||||
flag. Unsupported hosts or mount failures remain explicit limitations; do not
|
||||
claim child inventory when only root identity is available. DMG extraction is
|
||||
unavailable through `extract_artifact`. To access DMG files on macOS, create a
|
||||
read-only mount with your own tools, select files from that mount for analysis,
|
||||
and detach the mount you created when finished. REA's inspection mount is
|
||||
temporary, not a retained export of the image's contents.
|
||||
|
||||
For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
|
||||
`--target-format dos-com`). Inspect the load image before trusting function
|
||||
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
|
||||
|
||||
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
|
||||
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
|
||||
`arm`, `arm64`, `6502`, or `m68k`), and `entry_point` (`0x` hex). File offset 0 is mapped at
|
||||
that address. Hopper and IDA refuse this interpretation; Ghidra uses
|
||||
`BinaryLoader` with a generic language: big-endian for `m68k`, little-endian
|
||||
otherwise. Big-endian ARM, Thumb-only, and Cortex-M variants are unsupported.
|
||||
`6502` selects NMOS 6502; the entry and complete file mapping must fit within
|
||||
`0x0000..0xffff`. 65C02, 65816, cartridge containers, bank switching, and mapper
|
||||
semantics are unsupported. `m68k` requires an even entry and decodes the 68040
|
||||
superset of the 68000 family; Amiga/Atari containers and vector-table entry
|
||||
discovery are unsupported. Windows Ghidra P0 admits raw images only for `6502`. See [raw binary guide](https://github.com/morluto/rea/blob/main/docs/raw-binary.md).
|
||||
|
||||
### Function annotations in Ghidra
|
||||
|
||||
Use `annotate_native_function` for one function name and/or entry comments, with at least one explicit change. Review its annotation readback and refreshed dossier inline. Changes are atomic and session-scoped; empty comments clear them, omitted fields preserve them. Later MCP calls observe edits until close. CLI `annotate-native-function` returns the updated analysis before discarding the session. Original executable bytes are unchanged; edits invalidate immutable snapshots. Windows P0 does not admit database mutations. For naming only, `set_address_name` / `set_addresses_names` also work with Ghidra: they name function entries or data addresses (primary labels) and return compact per-address success booleans instead of a dossier.
|
||||
|
||||
## Offline PE resource inventory
|
||||
|
||||
Use `inspect_pe_resources` with an absolute `path`, or
|
||||
`rea inspect-pe-resources ./selected.exe --json`, for portable PE32/PE32+
|
||||
resource inspection without opening a native target. Preserve returned Evidence:
|
||||
type/name/language IDs and UTF-16 names, original directory/data/payload ranges,
|
||||
code pages, payload hashes and static icon-group references. Same-language icon
|
||||
resolution is explicit; missing references and size mismatch remain visible.
|
||||
Other payloads are opaque, and this operation does not decode images or simulate
|
||||
Windows resource loading. Consult the budgets and supported tree profile in
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
|
||||
For exact core-module import/export differences, use `compare_wasm_interfaces`
|
||||
with two complete inspection Evidence records (or current-session retained
|
||||
references) and explicit scope. Preserve unknown types/duplicate names; static
|
||||
interface equality is not runtime compatibility. CLI accepts complete inline
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`.
|
||||
|
||||
## Godot PCK
|
||||
|
||||
For a standalone PCK, `open_binary` then `inspect_artifact` with
|
||||
`detail: "summary"` retains the complete static inventory. Use the returned
|
||||
`parent_evidence_id` with `inspect_analysis_view` inventory occurrences filtered
|
||||
by exact `logical_paths` or `occurrence_ids`, and join `artifact_id` to nodes
|
||||
filtered by `artifact_ids`. Select only needed fields; identity links remain.
|
||||
Prefer `inspect-artifact --detail summary --evidence_output PATH` for CLI tasks;
|
||||
the same CLI discovers embedded packs via their EOF trailer without opening a
|
||||
native analysis provider. Normal native opening remains unchanged.
|
||||
|
||||
Distinguish declared MD5 from observed SHA-256 and graph/page completeness from
|
||||
`source_coverage`. Encrypted directories and sparse bundles have unknown entry
|
||||
coverage; encrypted members have unknown content identity. An empty filtered
|
||||
page cannot prove absence in unknown coverage. Retained views are historical,
|
||||
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
|
||||
decompilation or target execution is provided. See
|
||||
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
|
||||
@@ -1,84 +0,0 @@
|
||||
# Browser and Electron runtime observation
|
||||
|
||||
Supply the existing browser or Electron instance's literal-port loopback CDP
|
||||
endpoint in the request. No separate permission configuration or per-call
|
||||
approval flag is needed. Browser discovery lists eligible HTTP(S) pages;
|
||||
`allowed_origins` optionally filters them. An individual capture defaults to the
|
||||
selected page's current origin. Electron discovery lists eligible local
|
||||
`file://` targets without a configured file-root allowlist.
|
||||
|
||||
Start passive browser work with `list_browser_targets`; start Electron runtime work with
|
||||
`list_electron_targets`. Before an electron-main Inspector attempt, check
|
||||
packaged fuses with `inspect_electron_fuses`. Inspect the target selected for
|
||||
the user's task. Observation with these tools is passive: never claim REA
|
||||
clicked, navigated, evaluated page JavaScript, invoked IPC, or captured prior
|
||||
activity. REA does not contain the page's network.
|
||||
|
||||
Passive captures omit raw header collections, credential header values, storage
|
||||
values, and raw WebSocket/JSON bodies. Selected response metadata, including
|
||||
content encoding, policy values, and Link URLs, is retained. Local URLs retain ordinary query
|
||||
values and fragments; userinfo credentials are removed. Browser accessibility
|
||||
text, console primitives, value-free shapes, script sources, source maps, and
|
||||
storage key names have request-specific capture options. Electron script
|
||||
sources are also optional. Use `capture_web_screenshot` when pixels are needed.
|
||||
Treat scope filtering, truncation, and attach-window coverage as limitations.
|
||||
Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
|
||||
|
||||
## Who performs the action
|
||||
|
||||
- The user performs it: `list_browser_targets`, then `inspect_web_page` or
|
||||
`observe_web_session` on the selected target.
|
||||
- The agent must perform it: one `capture_browser_scenario` with declared
|
||||
actions. Use connect mode with the loopback CDP endpoint and `target_id` for
|
||||
an existing browser, or launch mode with a caller-selected executable for a
|
||||
temporary profile. Connect mode first navigates the selected page to
|
||||
`start_url`, which can clear unsaved form values and other temporary state.
|
||||
Recreate the state needed for the investigation in the declared actions.
|
||||
Connect mode leaves the external browser open; launch mode owns a temporary
|
||||
browser and profile. Check `browser.cleanup` for the outcome.
|
||||
Request only the DOM, accessibility, or network details needed for the question.
|
||||
The default final capture contains the URL; step outcomes are reported separately.
|
||||
Add a wait action for the expected page state before capturing it. Report that
|
||||
REA performed the declared actions.
|
||||
|
||||
Scenario network records have no request initiator stacks. For that evidence,
|
||||
use passive capture while the user performs the action. Listener source locations
|
||||
identify candidate handlers; they do not establish which script sent a particular
|
||||
request. Avoid overlapping passive capture and scenario actions on the same target.
|
||||
|
||||
Focused follow-up tools:
|
||||
|
||||
- `observe_web_execution`: execution coverage during a window; it resets
|
||||
counters and temporarily disables optimization.
|
||||
- `inspect_web_event_listeners`: source locations for listeners registered
|
||||
directly on the first CSS match, without dispatching an event.
|
||||
- `compare_web_captures`: two normalized passive captures or recorded scenarios.
|
||||
- `discover_webmcp_tools`: inspect page-declared WebMCP inventory; never invoke
|
||||
the discovered tools.
|
||||
|
||||
Use `reconcile_javascript_runtime` only after static application Evidence and
|
||||
passive browser/Electron Evidence exist. Prefer captured-source digest identity;
|
||||
report path/digest disagreement as a mismatch and preserve ambiguity. Explicit
|
||||
path mappings are inference inputs and never broaden filesystem or CDP authority.
|
||||
A bundle observed at runtime does not prove every contained module executed.
|
||||
|
||||
## Fiddler Everywhere traffic debugging
|
||||
|
||||
For large saved captures, request `detail: "summary"`, then page retained Evidence
|
||||
with `inspect_analysis_view` and `view: {kind: "capture", detail: "preview", offset: 0, limit: 20}`.
|
||||
Select complete record pages only for needed headers, metadata or payloads; views
|
||||
read the historical parent without repeating capture decoding.
|
||||
|
||||
For selected protobuf/gRPC bodies, keep raw bytes as the source and use
|
||||
`inspect_analysis_view` kind `protobuf`, collection `bodies`, `frames` or
|
||||
`fields`. Batch exact record ordinals and field numbers/IDs, and select
|
||||
`schema_value` or `text` only when needed. Supply a local descriptor set and
|
||||
message type for schema interpretation; compressed payloads remain unknown.
|
||||
|
||||
Inspect native `.saz` snapshots with `inspect_web_network_capture` and
|
||||
`format: "saz"`; encrypted archives are unsupported. For live
|
||||
traffic analysis, connect Everywhere's separate MCP server and use its
|
||||
advertised session tools with an explicit `sessionsSource` (`LiveTraffic` or
|
||||
`AgentCalls`). Preserve existing sessions and filters when inspecting a
|
||||
recorded run. See the [capture and MCP guide](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md)
|
||||
for the workflow and Telerik's public setup and traffic-debugging skills.
|
||||
@@ -1,8 +0,0 @@
|
||||
# Keep text checkouts consistent with the repository formatter on every platform.
|
||||
* text=auto eol=lf
|
||||
|
||||
# Windows command fixtures retain their native line endings.
|
||||
*.cmd text eol=crlf
|
||||
|
||||
# Preserve whitespace in original upstream licensing texts.
|
||||
licenses/npm-runtime-notices.txt whitespace=-blank-at-eol
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
name: Bug Report
|
||||
about: Report a bug in REA
|
||||
title: "[Bug] "
|
||||
labels: ["bug", "triage"]
|
||||
assignees: []
|
||||
---
|
||||
|
||||
## Bug Description
|
||||
|
||||
A clear and concise description of what the bug is.
|
||||
|
||||
## Steps to Reproduce
|
||||
|
||||
1.
|
||||
2.
|
||||
3.
|
||||
|
||||
## Expected Behavior
|
||||
|
||||
What you expected to happen.
|
||||
|
||||
## Actual Behavior
|
||||
|
||||
What actually happened, including any error messages or unexpected output.
|
||||
|
||||
## Environment
|
||||
|
||||
- REA version (`rea --version` or `npm list @morluto/rea`):
|
||||
- Node.js version (`node --version`):
|
||||
- macOS version:
|
||||
- Hopper Disassembler version:
|
||||
- Target binary format (Mach-O, ELF, PE, .hop):
|
||||
|
||||
## Additional Context
|
||||
|
||||
- MCP client and version (if applicable):
|
||||
- Any relevant logs or stack traces:
|
||||
- Does the issue reproduce with a different target binary?
|
||||
@@ -1,165 +0,0 @@
|
||||
name: Bug report
|
||||
description: Report a problem in REA with observations or source evidence
|
||||
title: "[Bug] "
|
||||
labels:
|
||||
- bug
|
||||
- triage
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Report one primary behavior per issue. Keep answers concise and grounded
|
||||
in available evidence. Distinguish observed behavior, source-derived
|
||||
reasoning, and suspected causes; do not claim a reproduction you did not run.
|
||||
If a narrative answer is not known, write `NOT_ENOUGH_INFO` and describe
|
||||
what is missing. Intermittent failures and clearly labeled source-only
|
||||
reports are welcome. Do not include credentials or sensitive target data.
|
||||
|
||||
- type: checkboxes
|
||||
id: before-submitting
|
||||
attributes:
|
||||
label: Before submitting
|
||||
options:
|
||||
- label: I searched the open and closed issues for a related report.
|
||||
required: true
|
||||
- label: I removed credentials, tokens, private paths, and sensitive target data.
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: area
|
||||
attributes:
|
||||
label: Area
|
||||
options:
|
||||
- Setup, packaging, or upgrade
|
||||
- Provider or capability discovery
|
||||
- Process capture or runtime observation
|
||||
- Native analysis or decompilation
|
||||
- Evidence, artifacts, or reconstruction
|
||||
- CLI
|
||||
- MCP or agent integration
|
||||
- Documentation
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: observed-behavior
|
||||
attributes:
|
||||
label: Observed behavior
|
||||
description: Describe the actual result, including the exact command or MCP request, relevant provider/target, and supporting evidence below. Label source-only findings instead of claiming execution.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: reproduction
|
||||
attributes:
|
||||
label: Reproduction
|
||||
description: Give the shortest steps you actually ran, with the relevant input or configuration. State how often they reproduced the problem when known. If reproduction is unavailable, say so and provide the observations or source reference you have.
|
||||
placeholder: |
|
||||
1. Configure or start ...
|
||||
2. Run ...
|
||||
3. Observe ...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: expected-behavior
|
||||
attributes:
|
||||
label: Expected behavior
|
||||
description: State the expected result, contract, or recovery action. Cite documentation, source, or a known-good version when available; mark an unverified expectation or unknown reference clearly.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: impact-scope
|
||||
attributes:
|
||||
label: Impact and scope
|
||||
description: Describe observed impact and frequency, including effects on evidence integrity, process isolation, compatibility, or recovery when known. Leave untested scope and severity unknown rather than guessing.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: rea-version
|
||||
attributes:
|
||||
label: REA version
|
||||
description: Use `rea --version` or provide the source revision.
|
||||
placeholder: Exact rea --version output or source commit SHA
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: runtime
|
||||
attributes:
|
||||
label: Runtime and platform
|
||||
description: Include Node.js, OS, architecture, and Hopper/Ghidra version when relevant.
|
||||
placeholder: Node 24.18; macOS 15 arm64; Hopper 5.15
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: target
|
||||
attributes:
|
||||
label: Target and artifact context
|
||||
description: Include binary format, app/container type, artifact kind, or workload details when relevant.
|
||||
placeholder: Mach-O arm64 app; Evidence v2 bundle; not applicable
|
||||
|
||||
- type: dropdown
|
||||
id: interface
|
||||
attributes:
|
||||
label: Interface
|
||||
options:
|
||||
- CLI
|
||||
- MCP
|
||||
- Python bridge
|
||||
- Package or setup
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: failure-stage
|
||||
attributes:
|
||||
label: Earliest failure stage
|
||||
description: Select where the incorrect behavior first becomes observable. Choose Unknown if this has not been established.
|
||||
options:
|
||||
- Request parsing or validation
|
||||
- Setup or provider admission
|
||||
- Capture or observation
|
||||
- Artifact or evidence registration
|
||||
- Native analysis or reconstruction
|
||||
- Transport or result projection
|
||||
- Cleanup, cancellation, or lifecycle
|
||||
- Unknown
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: relevant-evidence
|
||||
attributes:
|
||||
label: Relevant output or evidence
|
||||
description: Include the smallest useful output, MCP result, error, screenshot, artifact, or exact source reference. Say how it was obtained and which claim it supports. If unavailable, write NOT_ENOUGH_INFO and explain what is missing. Redact credentials and sensitive target data.
|
||||
placeholder: |
|
||||
Command or MCP request actually run:
|
||||
Relevant result, error, or artifact:
|
||||
For source-only findings, link the file/revision and relevant code:
|
||||
Missing evidence or remaining uncertainty:
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: typed-outcome
|
||||
attributes:
|
||||
label: Typed outcome or recovery expected
|
||||
description: Optional. What typed result, error, next tool, cleanup state, or limitation should the caller have received?
|
||||
|
||||
- type: textarea
|
||||
id: analysis
|
||||
attributes:
|
||||
label: Analysis or suspected cause
|
||||
description: Optional. Link supporting source and distinguish source-derived reasoning from a hypothesis. A root-cause analysis or proposed fix is not required.
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Add reduced fixtures, screenshots, or related issues. For observed regressions, include the last known good and first known bad versions when available. Label untested checks as proposals.
|
||||
@@ -1,8 +1,5 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Contribution scope and integration requirements
|
||||
url: https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope
|
||||
about: Check supported capabilities, excluded workflows, and the fail-closed integration rule before proposing a feature.
|
||||
- name: REA Documentation
|
||||
url: https://github.com/morluto/rea#readme
|
||||
about: Read the README and AGENTS.md for usage guides and configuration details.
|
||||
|
||||
@@ -1,111 +0,0 @@
|
||||
name: Design discussion
|
||||
description: Discuss an REA architecture, provider, contract, or scope decision
|
||||
title: "[Design] "
|
||||
labels:
|
||||
- enhancement
|
||||
- triage
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Use this for a substantial provider, public/persisted contract, or
|
||||
authority-boundary decision. One question and a short explanation are
|
||||
enough to start. Choose "Not sure yet" where needed; maintainers can help
|
||||
with technical details, alternatives, and verification.
|
||||
|
||||
Read the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
|
||||
Unresolved boundaries block integration, not opening a discussion.
|
||||
|
||||
- type: checkboxes
|
||||
id: before-submitting
|
||||
attributes:
|
||||
label: Before submitting
|
||||
options:
|
||||
- label: I searched the open and closed issues and pull requests for a related discussion.
|
||||
required: true
|
||||
- label: I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here.
|
||||
required: true
|
||||
- label: This discussion does not include credentials, private samples, or sensitive target data.
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: decision
|
||||
attributes:
|
||||
label: Decision needed
|
||||
description: What would you like maintainers to help decide? A short question is enough.
|
||||
placeholder: Should REA ... or ...?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: context-evidence
|
||||
attributes:
|
||||
label: Context and evidence
|
||||
description: Explain the problem or motivation and link relevant evidence when available. Distinguish observed behavior, source-derived reasoning, proposed behavior, and unknowns. A question without measurements or a finished design is welcome.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: target-interaction
|
||||
attributes:
|
||||
label: Expected interaction with targets
|
||||
description: >-
|
||||
Select all known effects, including upstream tool behavior. Add "Not sure
|
||||
yet" if anything is unclear. Use "No target interaction" only on its own.
|
||||
These selections help discussion and do not establish scope approval.
|
||||
multiple: true
|
||||
options:
|
||||
- Read existing files or captured data
|
||||
- Launch or attach to a process
|
||||
- Inject instrumentation or run scripts/code
|
||||
- Change process memory or registers
|
||||
- Write or change files/configuration
|
||||
- Contact network services
|
||||
- Recover protected code or work around protections
|
||||
- No target interaction (documentation or usability only)
|
||||
- Not sure yet
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: boundary-invariants
|
||||
attributes:
|
||||
label: Boundary details (optional)
|
||||
description: >-
|
||||
If known, clarify target selection, selected effects, provider boundaries,
|
||||
non-goals, or evidence invariants. Leave blank if you need help identifying
|
||||
them. Do not include private samples or proof-of-authorization documents.
|
||||
|
||||
- type: textarea
|
||||
id: options
|
||||
attributes:
|
||||
label: Options and tradeoffs
|
||||
description: Optional approaches or tradeoffs. One idea is enough; maintainers can help compare alternatives.
|
||||
|
||||
- type: dropdown
|
||||
id: verification-readiness
|
||||
attributes:
|
||||
label: Verification available so far
|
||||
description: Pick the closest match. An idea without samples or test results is welcome.
|
||||
options:
|
||||
- Idea only; I need help defining checks
|
||||
- I can suggest a sample or manual check
|
||||
- I have a prototype or observed results
|
||||
- Not sure yet
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: success-criteria
|
||||
attributes:
|
||||
label: Success criteria and open questions
|
||||
description: >-
|
||||
Optional expected output, sample, possible check, or unresolved question.
|
||||
A complete test plan is not required to start; maintainers and implementers
|
||||
will resolve material verification gaps before integration.
|
||||
|
||||
- type: textarea
|
||||
id: references
|
||||
attributes:
|
||||
label: References
|
||||
description: Link relevant contracts, upstream provider documentation, tests, measurements, or prior discussions.
|
||||
@@ -0,0 +1,24 @@
|
||||
---
|
||||
name: Documentation
|
||||
about: Suggest improvements to REA documentation
|
||||
title: "[Docs] "
|
||||
labels: ["documentation", "triage"]
|
||||
assignees: []
|
||||
---
|
||||
|
||||
## Documentation Area
|
||||
|
||||
Which documentation needs improvement? (README, AGENTS.md, API docs, inline JSDoc, skill files)
|
||||
|
||||
## Problem
|
||||
|
||||
What is unclear, missing, or incorrect in the current documentation?
|
||||
|
||||
## Suggested Improvement
|
||||
|
||||
Describe the change you would like to see.
|
||||
|
||||
## Additional Context
|
||||
|
||||
- Link to the relevant documentation file or section:
|
||||
- Any examples or references that would help:
|
||||
@@ -1,62 +0,0 @@
|
||||
name: Documentation
|
||||
description: Report unclear, missing, or incorrect REA documentation
|
||||
title: "[Docs] "
|
||||
labels:
|
||||
- documentation
|
||||
- triage
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Report one documentation problem with the exact location and the smallest
|
||||
useful supporting passage, example, or result. Keep answers concise.
|
||||
If something is unknown, write `NOT_ENOUGH_INFO` and explain what is missing.
|
||||
|
||||
- type: checkboxes
|
||||
id: before-submitting
|
||||
attributes:
|
||||
label: Before submitting
|
||||
options:
|
||||
- label: I searched the existing issues for a related documentation report.
|
||||
required: true
|
||||
- label: I removed credentials, tokens, private paths, and sensitive target data.
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: location
|
||||
attributes:
|
||||
label: Documentation location
|
||||
description: Link the page, section, command, API, skill, or inline comment.
|
||||
placeholder: README.md#quick-start or docs/mcp-contracts.md
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: problem
|
||||
attributes:
|
||||
label: Problem
|
||||
description: Quote or describe the current content, explain what is unclear, missing, outdated, or incorrect, and say how to observe the problem. State who is affected when known.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: expected-content
|
||||
attributes:
|
||||
label: Expected content
|
||||
description: Describe the accurate behavior or example that should be documented, with a reference when available. Label a proposed clarification or unknown behavior clearly.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: evidence
|
||||
attributes:
|
||||
label: Supporting evidence
|
||||
description: Include an exact passage, source reference, command output, provider documentation, or screenshot supporting the report. For missing or unclear guidance, point to the relevant section and explain the gap. If evidence is unavailable, write NOT_ENOUGH_INFO and explain what is missing.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Include screenshots, affected translations, or related issues.
|
||||
@@ -0,0 +1,29 @@
|
||||
---
|
||||
name: Feature Request
|
||||
about: Propose a new feature or enhancement for REA
|
||||
title: "[Feature] "
|
||||
labels: ["enhancement", "triage"]
|
||||
assignees: []
|
||||
---
|
||||
|
||||
## Problem Statement
|
||||
|
||||
What problem does this feature solve? Describe the use case and why existing tools are insufficient.
|
||||
|
||||
## Proposed Solution
|
||||
|
||||
Describe the feature or enhancement you would like to see. Include:
|
||||
|
||||
- Which REA tool(s) this affects (direct, enhanced, or session tools)
|
||||
- Expected inputs and outputs
|
||||
- How it integrates with the existing MCP workflow
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
Any alternative approaches or workarounds you have tried.
|
||||
|
||||
## Additional Context
|
||||
|
||||
- Relevant Hopper API methods or concepts:
|
||||
- Links to related issues or discussions:
|
||||
- Would this require changes to the bridge (`bridge/hopper_bridge.py`)?
|
||||
@@ -1,143 +0,0 @@
|
||||
name: Feature request
|
||||
description: Propose a focused improvement to REA
|
||||
title: "[Feature] "
|
||||
labels:
|
||||
- enhancement
|
||||
- triage
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Describe one problem and the outcome you would like. Short answers are
|
||||
welcome. Choose "Not sure yet" wherever needed; you do not need code, a
|
||||
finished design, or a complete test plan to propose an idea.
|
||||
Use concrete examples when available, distinguish current limitations
|
||||
from proposed behavior, and leave unmeasured impact or unknowns explicit.
|
||||
|
||||
Read the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
|
||||
Maintainers can help resolve scope and technical questions. Unresolved
|
||||
boundaries block integration, not opening a request for discussion.
|
||||
|
||||
- type: checkboxes
|
||||
id: before-submitting
|
||||
attributes:
|
||||
label: Before submitting
|
||||
options:
|
||||
- label: I searched the open and closed issues and pull requests for a related proposal.
|
||||
required: true
|
||||
- label: I understand the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope) must be reviewed before integration; open questions are welcome here.
|
||||
required: true
|
||||
- label: This request does not include credentials, private samples, or sensitive target data.
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: area
|
||||
attributes:
|
||||
label: Area
|
||||
options:
|
||||
- Setup, packaging, or upgrade
|
||||
- Provider or capability discovery
|
||||
- Process capture or runtime observation
|
||||
- Native analysis or decompilation
|
||||
- Evidence, artifacts, or reconstruction
|
||||
- CLI
|
||||
- MCP or agent integration
|
||||
- Documentation
|
||||
- Other
|
||||
- Not sure yet
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: current-limitation
|
||||
attributes:
|
||||
label: Current limitation
|
||||
description: What are you trying to understand or do, and what is difficult today? A short example is enough.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: use-cases
|
||||
attributes:
|
||||
label: Use cases
|
||||
description: Optional examples of affected workflows, targets, or callers and their practical impact. Include observed frequency or measurements when available; do not invent them. Leave blank if covered above.
|
||||
|
||||
- type: textarea
|
||||
id: desired-outcome
|
||||
attributes:
|
||||
label: Desired outcome and success criteria
|
||||
description: What would a useful result look like? Describe the output or behavior you want; implementation details are optional.
|
||||
placeholder: For a selected APK, show whether it is packed and include the detection evidence.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: target-interaction
|
||||
attributes:
|
||||
label: Expected interaction with targets
|
||||
description: >-
|
||||
Select all known effects, including upstream tool behavior. Add "Not sure
|
||||
yet" if anything is unclear. Use "No target interaction" only on its own.
|
||||
These selections help discussion and do not establish scope approval.
|
||||
multiple: true
|
||||
options:
|
||||
- Read existing files or captured data
|
||||
- Launch or attach to a process
|
||||
- Inject instrumentation or run scripts/code
|
||||
- Change process memory or registers
|
||||
- Write or change files/configuration
|
||||
- Contact network services
|
||||
- Recover protected code or work around protections
|
||||
- No target interaction (documentation or usability only)
|
||||
- Not sure yet
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: boundaries
|
||||
attributes:
|
||||
label: Target and scope details (optional)
|
||||
description: >-
|
||||
If known, name the target/platform, clarify selected effects, or list a
|
||||
non-goal or scope question. You can leave this blank; maintainers can help.
|
||||
Do not include private samples or proof-of-authorization documents.
|
||||
|
||||
- type: dropdown
|
||||
id: verification-readiness
|
||||
attributes:
|
||||
label: Verification available so far
|
||||
description: Pick the closest match. An idea without samples or test results is welcome.
|
||||
options:
|
||||
- Idea only; I need help defining checks
|
||||
- I can suggest a sample or manual check
|
||||
- I have a prototype or observed results
|
||||
- Not sure yet
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: verification
|
||||
attributes:
|
||||
label: Verification details (optional)
|
||||
description: >-
|
||||
Share an expected output, a possible check, or existing results if you have
|
||||
them. Self-owned or appropriately licensed samples can help, but are not
|
||||
required to open a request. Do not attach private or sensitive data.
|
||||
|
||||
- type: textarea
|
||||
id: alternatives
|
||||
attributes:
|
||||
label: Alternatives considered
|
||||
description: Describe existing REA operations, upstream tools, or workarounds considered and why they do not meet the need.
|
||||
|
||||
- type: textarea
|
||||
id: possible-direction
|
||||
attributes:
|
||||
label: Possible direction
|
||||
description: Optional implementation or integration idea. Treat it as a proposal, not a requirement.
|
||||
|
||||
- type: textarea
|
||||
id: references
|
||||
attributes:
|
||||
label: References
|
||||
description: Optional supporting examples, provider documentation, source, measurements, artifacts, issues, or prior art. Distinguish observed results from expected benefits; an idea without evidence is welcome.
|
||||
@@ -1,6 +0,0 @@
|
||||
self-hosted-runner:
|
||||
labels:
|
||||
- hopper
|
||||
- ghidra-12-1-2
|
||||
# Hosted label introduced after the pinned actionlint release.
|
||||
- macos-15-intel
|
||||
@@ -1,52 +0,0 @@
|
||||
name: Cache npm package downloads
|
||||
description: Restore npm's download cache. Trusted default-branch jobs that opt in may save it.
|
||||
inputs:
|
||||
save:
|
||||
description: Save the shared download cache when this event is allowed to write it.
|
||||
required: false
|
||||
default: "false"
|
||||
outputs:
|
||||
can-save:
|
||||
description: Whether this run may update the shared npm download cache.
|
||||
value: ${{ steps.context.outputs.can-save }}
|
||||
path:
|
||||
description: The npm download-cache directory.
|
||||
value: ${{ steps.context.outputs.path }}
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- id: context
|
||||
shell: bash
|
||||
env:
|
||||
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
LOCKFILES_HASH: ${{ hashFiles('package-lock.json', 'npm-shrinkwrap.json', '.npmrc') }}
|
||||
run: |
|
||||
cache_path="$(npm config get cache)"
|
||||
runtime_arch="$(node -p 'process.arch')"
|
||||
can_save=false
|
||||
if [[ -n "${DEFAULT_BRANCH}" && "${GITHUB_REF}" == "refs/heads/${DEFAULT_BRANCH}" && ( "${GITHUB_EVENT_NAME}" == "push" || "${GITHUB_EVENT_NAME}" == "schedule" || "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ) ]]; then
|
||||
can_save=true
|
||||
fi
|
||||
cache_key="npm-download-${RUNNER_OS}-${runtime_arch}-${LOCKFILES_HASH}"
|
||||
# Same platform only. A lockfile change can reuse tarballs from the previous key.
|
||||
restore_key="npm-download-${RUNNER_OS}-${runtime_arch}-"
|
||||
{
|
||||
printf 'can-save=%s\n' "${can_save}"
|
||||
printf 'path=%s\n' "${cache_path}"
|
||||
printf 'key=%s\n' "${cache_key}"
|
||||
printf 'restore-key=%s\n' "${restore_key}"
|
||||
} >> "${GITHUB_OUTPUT}"
|
||||
- name: Restore and save npm package downloads
|
||||
if: steps.context.outputs.can-save == 'true' && inputs.save == 'true'
|
||||
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: ${{ steps.context.outputs.path }}
|
||||
key: ${{ steps.context.outputs.key }}
|
||||
restore-keys: ${{ steps.context.outputs.restore-key }}
|
||||
- name: Restore npm package downloads without saving
|
||||
if: steps.context.outputs.can-save != 'true' || inputs.save != 'true'
|
||||
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: ${{ steps.context.outputs.path }}
|
||||
key: ${{ steps.context.outputs.key }}
|
||||
restore-keys: ${{ steps.context.outputs.restore-key }}
|
||||
@@ -1,41 +0,0 @@
|
||||
name: Portable build cache
|
||||
description: Restore portable Turbo build outputs from the current run or cross-run cache.
|
||||
inputs:
|
||||
save:
|
||||
description: Save newly built outputs after the job succeeds.
|
||||
default: "false"
|
||||
build_cache_artifact:
|
||||
description: Reuse portable Turbo outputs uploaded by the current workflow run.
|
||||
default: ""
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Download portable build outputs from this workflow run
|
||||
if: inputs.build_cache_artifact != ''
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: ${{ inputs.build_cache_artifact }}
|
||||
path: .turbo/cache/
|
||||
- name: Resolve portable build task hashes
|
||||
id: plan
|
||||
if: inputs.build_cache_artifact == ''
|
||||
shell: bash
|
||||
env:
|
||||
BUILD_CACHE_PREFIX: rea-portable-build-v2-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.npmrc', '.nvmrc', 'package-lock.json') }}
|
||||
run: |
|
||||
npm run --silent test:prepare -- --dry=json |
|
||||
node scripts/ci/portable-build-cache.mjs key >> "$GITHUB_OUTPUT"
|
||||
- name: Restore portable build outputs
|
||||
if: inputs.build_cache_artifact == '' && inputs.save != 'true'
|
||||
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: .turbo/cache/
|
||||
key: ${{ steps.plan.outputs.key }}
|
||||
restore-keys: ${{ steps.plan.outputs.prefix }}
|
||||
- name: Restore and save portable build outputs
|
||||
if: inputs.build_cache_artifact == '' && inputs.save == 'true'
|
||||
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
path: .turbo/cache/
|
||||
key: ${{ steps.plan.outputs.key }}
|
||||
restore-keys: ${{ steps.plan.outputs.prefix }}
|
||||
@@ -1,26 +0,0 @@
|
||||
name: Prepare qualified release inputs
|
||||
description: Restore and validate the exact qualified tarball and its verifier runtime
|
||||
outputs:
|
||||
version:
|
||||
description: Package version validated against the source and qualified manifest
|
||||
value: ${{ steps.package.outputs.version }}
|
||||
tarball:
|
||||
description: Absolute path to the validated qualified npm tarball
|
||||
value: ${{ steps.package.outputs.tarball }}
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Download the exact qualified package
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: verified-release-package
|
||||
path: .cache/verified-package/
|
||||
- name: Verify qualified package provenance and digest
|
||||
id: package
|
||||
shell: bash
|
||||
run: node scripts/ci/verified-package.mjs validate .cache/verified-package
|
||||
- name: Restore qualified verifier runtime
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: rea-test-runtime
|
||||
path: .
|
||||
@@ -1,41 +0,0 @@
|
||||
name: Publish MCP Registry metadata
|
||||
description: Validate and publish one released server.json, preserving an identical existing version
|
||||
inputs:
|
||||
directory:
|
||||
description: Directory containing the released server.json
|
||||
default: .
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- name: Install MCP Registry publisher
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.directory }}
|
||||
env:
|
||||
MCP_PUBLISHER_VERSION: v1.8.0
|
||||
MCP_PUBLISHER_SHA256_LINUX_AMD64: 1370446bbe74d562608e8005a6ccce02d146a661fbd78674e11cc70b9618d6cf
|
||||
run: |
|
||||
archive="mcp-publisher_linux_amd64.tar.gz"
|
||||
curl -fsSLo "${archive}" \
|
||||
"https://github.com/modelcontextprotocol/registry/releases/download/${MCP_PUBLISHER_VERSION}/${archive}"
|
||||
echo "${MCP_PUBLISHER_SHA256_LINUX_AMD64} ${archive}" | sha256sum --check --strict
|
||||
tar xzf "${archive}" mcp-publisher
|
||||
chmod +x mcp-publisher
|
||||
- name: Validate MCP Registry metadata
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.directory }}
|
||||
run: ./mcp-publisher validate
|
||||
- name: Check existing registry version
|
||||
id: registry
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.directory }}
|
||||
run: node "${GITHUB_ACTION_PATH}/registry-state.mjs"
|
||||
- name: Authenticate to MCP Registry
|
||||
if: steps.registry.outputs.exists != 'true'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.directory }}
|
||||
run: ./mcp-publisher login github-oidc
|
||||
- name: Publish MCP Registry metadata
|
||||
if: steps.registry.outputs.exists != 'true'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.directory }}
|
||||
run: ./mcp-publisher publish
|
||||
@@ -1,4 +0,0 @@
|
||||
export function registryPublicationState(
|
||||
server: Readonly<{ name: string; version: string } & Record<string, unknown>>,
|
||||
request?: (url: URL, init: RequestInit) => Promise<Response>,
|
||||
): Promise<"present" | "absent">;
|
||||
@@ -1,89 +0,0 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { appendFile, readFile } from "node:fs/promises";
|
||||
import { resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
|
||||
/** Exact-version routes currently return false 404s; enumerate the filtered listing. */
|
||||
export async function registryPublicationState(server, request = fetch) {
|
||||
assert.equal(typeof server?.name, "string");
|
||||
assert.equal(typeof server?.version, "string");
|
||||
const cursors = new Set();
|
||||
let cursor;
|
||||
const matches = [];
|
||||
do {
|
||||
const url = new URL(
|
||||
"https://registry.modelcontextprotocol.io/v0.1/servers",
|
||||
);
|
||||
url.search = new URLSearchParams({
|
||||
search: server.name,
|
||||
version: server.version,
|
||||
include_deleted: "true",
|
||||
limit: "100",
|
||||
}).toString();
|
||||
if (cursor !== undefined) url.searchParams.set("cursor", cursor);
|
||||
const response = await request(url, {
|
||||
signal: AbortSignal.timeout(30_000),
|
||||
});
|
||||
if (!response.ok)
|
||||
throw new Error(`Registry lookup failed: HTTP ${response.status}`);
|
||||
const page = await response.json();
|
||||
assert(Array.isArray(page?.servers), "Registry response omitted servers");
|
||||
assert(
|
||||
page.metadata && typeof page.metadata === "object",
|
||||
"Registry response omitted metadata",
|
||||
);
|
||||
for (const entry of page.servers) {
|
||||
assert(
|
||||
typeof entry?.server?.name === "string" &&
|
||||
typeof entry.server.version === "string",
|
||||
"Malformed registry server record",
|
||||
);
|
||||
if (
|
||||
entry.server.name === server.name &&
|
||||
entry.server.version === server.version
|
||||
)
|
||||
matches.push(entry);
|
||||
}
|
||||
cursor = page.metadata.nextCursor;
|
||||
if (cursor !== undefined) {
|
||||
assert(
|
||||
typeof cursor === "string" && cursor.length > 0 && !cursors.has(cursor),
|
||||
"Registry returned a non-advancing cursor",
|
||||
);
|
||||
cursors.add(cursor);
|
||||
}
|
||||
} while (cursor !== undefined);
|
||||
assert(
|
||||
matches.length <= 1,
|
||||
"Registry returned ambiguous exact-version records",
|
||||
);
|
||||
if (matches.length === 0) return "absent";
|
||||
assert.equal(
|
||||
matches[0]._meta?.["io.modelcontextprotocol.registry/official"]?.status,
|
||||
"active",
|
||||
"Existing registry version is not active",
|
||||
);
|
||||
assert.deepEqual(
|
||||
matches[0].server,
|
||||
server,
|
||||
"Existing registry version differs from released metadata",
|
||||
);
|
||||
return "present";
|
||||
}
|
||||
|
||||
if (
|
||||
process.argv[1] !== undefined &&
|
||||
resolve(process.argv[1]) === fileURLToPath(import.meta.url)
|
||||
) {
|
||||
const state = await registryPublicationState(
|
||||
JSON.parse(await readFile("server.json", "utf8")),
|
||||
);
|
||||
assert(process.env.GITHUB_OUTPUT, "GITHUB_OUTPUT is required");
|
||||
await appendFile(
|
||||
process.env.GITHUB_OUTPUT,
|
||||
`exists=${state === "present"}\n`,
|
||||
);
|
||||
process.stdout.write(
|
||||
`Registry version is ${state}; ${state === "present" ? "preserving identical published metadata" : "publication may proceed"}.\n`,
|
||||
);
|
||||
}
|
||||
@@ -1,126 +1,33 @@
|
||||
<!--
|
||||
PR title: type(optional-scope): imperative outcome
|
||||
The title becomes the squash-merge commit subject and generated changelog entry.
|
||||
-->
|
||||
## Description
|
||||
|
||||
## Summary
|
||||
<!-- What does this change do? Why is it needed? -->
|
||||
|
||||
<!-- What user, agent, or maintainer problem does this solve? Link the issue
|
||||
with "Fixes #123" when applicable. Keep the prior behavior, expected contract,
|
||||
and new behavior understandable without private context. -->
|
||||
## Type of Change
|
||||
|
||||
## Problem and expected behavior
|
||||
- [ ] Bug fix
|
||||
- [ ] New feature
|
||||
- [ ] Documentation update
|
||||
- [ ] Performance improvement
|
||||
- [ ] Refactoring (no functional change)
|
||||
- [ ] Tool contract change (MCP schema modification)
|
||||
|
||||
<!-- For fixes, give the smallest trigger and violated invariant. For features,
|
||||
describe the use case and observable outcome. -->
|
||||
## Testing Performed
|
||||
|
||||
## Change and scope
|
||||
<!-- Describe the testing you have done to verify this change. -->
|
||||
|
||||
<!-- Explain the chosen approach, why it fits REA's architecture, and what is
|
||||
intentionally not included. -->
|
||||
- [ ] Unit tests added or updated
|
||||
- [ ] TypeScript typecheck passes (`npm run typecheck`)
|
||||
- [ ] Linting passes (`npm run lint`)
|
||||
- [ ] Full test suite passes (`npm test`)
|
||||
- [ ] Real Hopper verification performed (`npm run verify:hopper`)
|
||||
- [ ] Linux real Hopper verification performed (`npm run verify:hopper:linux`) or unavailability explained
|
||||
- [ ] Package verification performed (`npm run verify:package`)
|
||||
|
||||
Follow the [contribution scope](https://github.com/morluto/rea/blob/main/CONTRIBUTING.md#contribution-scope).
|
||||
Unresolved scope, authority, side effects, or required verification block
|
||||
integration. Link the required maintainer scope decision for new providers or
|
||||
capabilities involving target execution, injection, mutation, or protection
|
||||
recovery; ordinary fixes within existing boundaries may state "not applicable".
|
||||
Draft PRs may leave the scope decision pending and the readiness and merge checklists
|
||||
unchecked. Maintainers can help resolve open questions.
|
||||
## Tool Contract Impact
|
||||
|
||||
## Contract and boundary impact
|
||||
<!-- If this PR adds, removes, or modifies MCP tools, list them here. -->
|
||||
<!-- The current inventory is 68 tools (33 direct + 10 enhanced + 5 native + 2 artifact + 18 session). -->
|
||||
|
||||
<!-- Short answers are welcome. Complete applicable lines; mark unresolved facts as "unknown".
|
||||
Use "none" or "not applicable" only when established. -->
|
||||
## Related Issues
|
||||
|
||||
- Semantic owner and earliest changed stage:
|
||||
- Contribution scope fit and linked maintainer decision:
|
||||
- CLI and MCP/tool-catalog contract:
|
||||
- Provider, bridge, target-format, or platform compatibility:
|
||||
- Evidence, artifact, provenance, or reconstruction contract:
|
||||
- Target selection, authority, upstream behavior, and cleanup details (if applicable):
|
||||
- Generated metadata (`docs/public/product-catalog.json`), package, or installation impact (CI generates and validates ignored catalogs and `skills/`; edit authored instructions in `.agents/skills/reverse-engineer-anything/` and never commit build outputs):
|
||||
|
||||
Expected effects (check known effects; use documentation-only on its own).
|
||||
Unchecked items do not establish absence; leave uncertainties visible.
|
||||
|
||||
- [ ] Read existing files or captured data.
|
||||
- [ ] Launch or attach to a process.
|
||||
- [ ] Inject instrumentation or run scripts/code.
|
||||
- [ ] Change process memory, registers, files, or configuration.
|
||||
- [ ] Contact network services.
|
||||
- [ ] Recover protected code or work around protections.
|
||||
- [ ] Documentation or usability only; no target interaction.
|
||||
- [ ] Not sure yet; investigation or maintainer guidance needed.
|
||||
|
||||
## Evidence and regression coverage
|
||||
|
||||
<!-- State whether evidence is executed, source-derived, or proposed. For fixes,
|
||||
show the original failure and explain how the regression would fail on the affected
|
||||
base revision when practical. Include the most useful proof inline; link long logs
|
||||
or artifacts, or put them in optional details. Documentation-only changes can use
|
||||
source, link, or rendered-document checks. -->
|
||||
|
||||
- Tests added or updated:
|
||||
- Base reproduction or other evidence:
|
||||
- User-visible CLI/MCP output (if applicable):
|
||||
- Remaining proof gaps:
|
||||
|
||||
For evidence-bearing changes:
|
||||
|
||||
- [ ] Observed, derived, and inferred claims remain distinguishable.
|
||||
- [ ] Artifact identity, source provenance, and failed attempts remain preserved.
|
||||
- [ ] Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.
|
||||
|
||||
## Validation performed
|
||||
|
||||
<!-- List only commands that actually ran, with observed results. Include the
|
||||
relevant real-provider or platform workflow, or explain why it was unavailable.
|
||||
Keep these results current after changes; a command without its result is incomplete. -->
|
||||
|
||||
- `command` — result
|
||||
|
||||
## Compatibility, safety, and release
|
||||
|
||||
<!-- Call out breaking changes, migrations, package effects, platform/provider
|
||||
limits, security/privacy/containment implications, and meaningful proof gaps. -->
|
||||
|
||||
- Breaking changes or migration steps:
|
||||
<!-- For ! / BREAKING CHANGE: show a previously valid call/configuration,
|
||||
its new behavior and migration, and why compatibility/deprecation cannot
|
||||
preserve it. Every release increments minor, including breaking changes;
|
||||
keep breaking markers and migration notes so callers can assess compatibility.
|
||||
See docs/releasing.md. -->
|
||||
- Real Hopper/Ghidra, browser, or OS coverage:
|
||||
- Package or release metadata impact:
|
||||
- Security, privacy, process, or containment review:
|
||||
|
||||
## Author readiness
|
||||
|
||||
<!-- The human owner completes these declarations before requesting review.
|
||||
The PR author is the owner by default. A different owner must explicitly accept
|
||||
ownership on this PR.
|
||||
AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA.
|
||||
Draft PRs may leave them unchecked. These declarations record initial readiness
|
||||
when opening the PR for review; later pushes do not require a new human sign-off.
|
||||
Keep the description and validation evidence current as the work changes.
|
||||
See CONTRIBUTING.md#pull-request-ownership-and-readiness. -->
|
||||
|
||||
Human owner: <!-- @GitHub-handle -->
|
||||
|
||||
- [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
|
||||
- [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
|
||||
- [ ] I declare this PR ready for initial maintainer review.
|
||||
|
||||
## Before merge
|
||||
|
||||
<!-- Draft PRs may leave pending items unchecked. Resolve material scope and
|
||||
authority questions before integration. -->
|
||||
|
||||
- [ ] This PR addresses a concrete problem or an agreed enhancement.
|
||||
- [ ] Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
|
||||
- [ ] The PR has one focused outcome and the title follows `type(scope): outcome`.
|
||||
- [ ] Related issue is linked, or the reason for not linking one is stated above.
|
||||
- [ ] Tests cover changed observable behavior and meaningful failure paths.
|
||||
- [ ] Owning docs, contracts, and generated metadata are updated where needed.
|
||||
- [ ] User-visible CLI/MCP changes include representative output.
|
||||
- [ ] I checked the final diff for secrets, unrelated cleanup, and unsupported claims.
|
||||
<!-- Link issues this PR closes or relates to. Use "Closes #123" syntax. -->
|
||||
@@ -1,64 +0,0 @@
|
||||
name: Cache retention
|
||||
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["CI"]
|
||||
types: [completed]
|
||||
|
||||
# Qualification calls ci.yml with actions: read. A nested workflow cannot raise
|
||||
# that token, so cache deletion stays in this workflow.
|
||||
permissions:
|
||||
contents: read
|
||||
actions: write
|
||||
|
||||
concurrency:
|
||||
group: cache-retention-${{ github.event.workflow_run.head_sha }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
retain:
|
||||
name: Retain Turbo snapshots and npm download caches
|
||||
# Restrict the triggering-source checkout to same-repository default-branch
|
||||
# CI events. The helper runs from this trusted workflow revision; triggering
|
||||
# source is used only for cache identity and is never executed. Static and
|
||||
# build save snapshots before later lanes finish, so failed, timed-out, or
|
||||
# cancelled runs still retain those snapshots.
|
||||
if: >-
|
||||
(github.event.workflow_run.conclusion == 'success' ||
|
||||
github.event.workflow_run.conclusion == 'failure' ||
|
||||
github.event.workflow_run.conclusion == 'timed_out' ||
|
||||
github.event.workflow_run.conclusion == 'cancelled') &&
|
||||
github.event.workflow_run.head_repository.full_name == github.repository &&
|
||||
github.event.workflow_run.head_branch == github.event.repository.default_branch &&
|
||||
(github.event.workflow_run.event == 'push' ||
|
||||
github.event.workflow_run.event == 'schedule' ||
|
||||
github.event.workflow_run.event == 'workflow_dispatch')
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
with:
|
||||
ref: ${{ github.workflow_sha }}
|
||||
path: trusted-retention
|
||||
persist-credentials: false
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
with:
|
||||
ref: ${{ github.event.workflow_run.head_sha }}
|
||||
path: triggering-source
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: trusted-retention/.nvmrc
|
||||
- name: Retain current Turbo snapshots and npm download caches
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
CACHE_PREFIX: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-
|
||||
CACHE_KEEP_KEY: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('triggering-source/.nvmrc', 'triggering-source/.npmrc', 'triggering-source/package-lock.json') }}-${{ github.event.workflow_run.head_sha }}
|
||||
CACHE_DEEP_PREFIX: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-
|
||||
CACHE_DEEP_KEEP_KEY: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('triggering-source/.nvmrc', 'triggering-source/.npmrc', 'triggering-source/package-lock.json') }}-${{ github.event.workflow_run.head_sha }}
|
||||
CACHE_KNIP_PREFIX: knip-meta-v1-${{ runner.os }}-
|
||||
CACHE_KNIP_KEEP_KEY: knip-meta-v1-${{ runner.os }}-${{ hashFiles('triggering-source/knip.json', 'triggering-source/tsconfig.json', 'triggering-source/package.json', 'triggering-source/package-lock.json') }}-${{ github.event.workflow_run.head_sha }}
|
||||
CACHE_PORTABLE_PREFIX: rea-portable-build-v2-${{ runner.os }}-${{ runner.arch }}-
|
||||
DEDUPE_NPM_DOWNLOADS: "true"
|
||||
run: node trusted-retention/scripts/ci/cache-retention.mjs
|
||||
@@ -1,18 +0,0 @@
|
||||
name: Full PR qualification
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main, "release/*"]
|
||||
types: [labeled]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
|
||||
jobs:
|
||||
qualification:
|
||||
name: Qualify labeled PR revision
|
||||
if: github.event.label.name == 'ci:full'
|
||||
uses: ./.github/workflows/ci.yml
|
||||
with:
|
||||
source_sha: ${{ github.sha }}
|
||||
@@ -1,84 +0,0 @@
|
||||
name: CI timings
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "17 * * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ci-timings-${{ github.repository }}
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
pull-requests: read
|
||||
|
||||
jobs:
|
||||
report:
|
||||
name: Collect recent completed workflow attempts
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
# Consume only API data and execute batch code from the trusted default branch.
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
with:
|
||||
ref: ${{ github.event.repository.default_branch }}
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
continue-on-error: true
|
||||
uses: ./.github/actions/npm-cache
|
||||
- name: Diagnose missing and startup-failed PR CI
|
||||
continue-on-error: true
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
CI_REPOSITORY_ID: ${{ github.repository_id }}
|
||||
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
run: node scripts/ci/pr-ci-health.mjs
|
||||
- name: Record queue, execution, and matched step timing
|
||||
continue-on-error: true
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
run: node scripts/ci/job-timings-batch.mjs
|
||||
- name: Upload CI timing artifacts
|
||||
if: always() && hashFiles('.cache/ci-job-timings/*') != ''
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: ci-job-timings-batch
|
||||
path: .cache/ci-job-timings/
|
||||
include-hidden-files: true
|
||||
if-no-files-found: error
|
||||
overwrite: true
|
||||
retention-days: 14
|
||||
- name: Install dependencies for static failure attribution
|
||||
id: failure-deps
|
||||
if: ${{ !cancelled() && hashFiles('.cache/ci-job-timings/batch-manifest.json') != '' }}
|
||||
continue-on-error: true
|
||||
run: npm ci
|
||||
- name: Attribute supported static failures against main
|
||||
if: ${{ !cancelled() && steps.failure-deps.outcome == 'success' && hashFiles('.cache/ci-job-timings/batch-manifest.json') != '' }}
|
||||
continue-on-error: true
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
CI_REPOSITORY: ${{ github.repository }}
|
||||
CI_DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
run: node scripts/ci/failure-report.mjs
|
||||
- name: Upload CI health and failure diagnostics
|
||||
if: ${{ !cancelled() }}
|
||||
continue-on-error: true
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: ci-health-diagnostics
|
||||
path: |
|
||||
.cache/ci-pr-health/
|
||||
.cache/ci-failures/
|
||||
include-hidden-files: true
|
||||
if-no-files-found: warn
|
||||
retention-days: 14
|
||||
+83
-1076
File diff suppressed because it is too large.
Load diff
@@ -1,48 +0,0 @@
|
||||
name: Ghidra bridge source compilation
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
ghidra-bridge-sources:
|
||||
name: Compile packaged Ghidra bridge sources
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
env:
|
||||
GHIDRA_RELEASE_URL: https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_12.1.4_build/ghidra_12.1.4_PUBLIC_20260921.zip
|
||||
GHIDRA_RELEASE_SHA256: ddac49f903da9d5bac833e5cc79395098b9c33cfd3279be5f31bd00387d2d4db
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Validate the SDK pin against the provider's verified release
|
||||
run: node scripts/verify-ghidra-bridge-sources.mjs --check-sdk-pin >> "$GITHUB_ENV"
|
||||
# Only the jars and application.properties are extracted: the check
|
||||
# compiles against them and never launches Ghidra or imports a target.
|
||||
- name: Acquire checksum-verified Ghidra jars in isolated runner storage
|
||||
shell: bash
|
||||
run: |
|
||||
ghidra_root="${RUNNER_TEMP}/rea-ghidra"
|
||||
mkdir -p "${ghidra_root}"
|
||||
curl -fsSL --retry 3 --write-out 'ghidra_download_bytes=%{size_download} ghidra_download_seconds=%{time_total}\n' -o "${ghidra_root}/ghidra.zip" "${GHIDRA_RELEASE_URL}"
|
||||
echo "${GHIDRA_RELEASE_SHA256} ${ghidra_root}/ghidra.zip" | sha256sum -c -
|
||||
unzip -q "${ghidra_root}/ghidra.zip" '*/Ghidra/*.jar' '*/Ghidra/application.properties' -d "${ghidra_root}"
|
||||
rm "${ghidra_root}/ghidra.zip"
|
||||
properties="$(find "${ghidra_root}" -path '*/Ghidra/application.properties' -print -quit)"
|
||||
test -n "${properties}"
|
||||
echo "GHIDRA_INSTALL_DIR=$(dirname "$(dirname "${properties}")")" >> "${GITHUB_ENV}"
|
||||
# JAVA_HOME_<major>_X64 comes from the runner image, not the env context.
|
||||
- name: Compile bridge sources with the verified runner JDK
|
||||
shell: bash
|
||||
run: |
|
||||
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
|
||||
test -n "${!java_home_variable}"
|
||||
JAVA_HOME="${!java_home_variable}" npm run verify:ghidra:bridge-sources -- --verified-sdk
|
||||
@@ -1,67 +0,0 @@
|
||||
name: Mutation sweep
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "47 4 * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: mutation-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
|
||||
jobs:
|
||||
mutation:
|
||||
name: Mutation sweep (${{ matrix.suite }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 60
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 2
|
||||
matrix:
|
||||
include:
|
||||
- suite: reverse-tools
|
||||
config: stryker.config.mjs
|
||||
- suite: reverse-tools-semantics
|
||||
config: stryker.semantics.config.mjs
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run test:prepare
|
||||
- name: Run the complete configuration
|
||||
env:
|
||||
MUTATION_CONFIG: ${{ matrix.config }}
|
||||
run: |
|
||||
node scripts/prepare-reverse-tools-mutation.mjs
|
||||
npx --no-install stryker run "${MUTATION_CONFIG}"
|
||||
- name: Require both reports from the completed sweep
|
||||
env:
|
||||
REPORT_BASE: reports/mutation/${{ matrix.suite }}
|
||||
run: |
|
||||
test -s "${REPORT_BASE}.json"
|
||||
test -s "${REPORT_BASE}.html"
|
||||
- name: Retain available mutation diagnostics
|
||||
if: ${{ !cancelled() }}
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: mutation-${{ matrix.suite }}-${{ github.run_attempt }}
|
||||
path: |
|
||||
reports/mutation/${{ matrix.suite }}.json
|
||||
reports/mutation/${{ matrix.suite }}.html
|
||||
if-no-files-found: warn
|
||||
retention-days: 7
|
||||
@@ -1,40 +0,0 @@
|
||||
name: Source-built NativeAOT fixtures
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
windows-fixture:
|
||||
name: Build Windows x64 NativeAOT fixture
|
||||
# .NET 8.0.22's bundled linker discovery supports VS 2019/2022.
|
||||
runs-on: windows-2022
|
||||
# Two bounded restore attempts and one compile can each take five minutes.
|
||||
timeout-minutes: 20
|
||||
env:
|
||||
DOTNET_CLI_TELEMETRY_OPTOUT: "1"
|
||||
DOTNET_PROCESSOR_COUNT: "1"
|
||||
COMPlus_GCHeapHardLimit: "30000000"
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- uses: actions/setup-dotnet@67a3573c9a986a3f9c594539f4ab511d57bb3ce9 # v4.3.1
|
||||
with:
|
||||
dotnet-version: "8.0.416"
|
||||
- name: Build benign fixture and independent byte oracle
|
||||
run: node scripts/generate-nativeaot-fixtures.mjs
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: nativeaot-win-x64-${{ github.event.pull_request.head.sha || github.sha }}
|
||||
path: |
|
||||
_reference/nativeaot-integration/generated/win-x64/symbols/NativeAotFixture.exe
|
||||
_reference/nativeaot-integration/generated/win-x64/NativeAotFixture.map
|
||||
_reference/nativeaot-integration/generated/win-x64/symbols.txt
|
||||
_reference/nativeaot-integration/generated/win-x64/oracle.json
|
||||
if-no-files-found: error
|
||||
retention-days: 7
|
||||
@@ -1,30 +0,0 @@
|
||||
name: Documentation build
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: documentation-build-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
build:
|
||||
name: Build documentation
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- run: npm run docs:build
|
||||
@@ -1,51 +0,0 @@
|
||||
name: PR readiness
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
branches: [main, "release/*"]
|
||||
types:
|
||||
[
|
||||
opened,
|
||||
edited,
|
||||
synchronize,
|
||||
reopened,
|
||||
ready_for_review,
|
||||
converted_to_draft,
|
||||
]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
pull_request_number:
|
||||
description: Open PR to recheck
|
||||
required: true
|
||||
type: string
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: pr-readiness-${{ github.event.pull_request.number || inputs.pull_request_number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
readiness:
|
||||
name: Validate author declaration
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 3
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
statuses: write
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ github.event.repository.default_branch }}
|
||||
persist-credentials: false
|
||||
- name: Check initial readiness
|
||||
uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9
|
||||
with:
|
||||
script: |
|
||||
const { pathToFileURL } = require("node:url");
|
||||
const entry = pathToFileURL(
|
||||
`${process.env.GITHUB_WORKSPACE}/scripts/ci/pr-readiness.mjs`,
|
||||
);
|
||||
const { runReadinessCheck } = await import(entry.href);
|
||||
await runReadinessCheck({ github, context, core });
|
||||
@@ -1,29 +0,0 @@
|
||||
name: PR title
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main, "release/*"]
|
||||
types: [opened, synchronize, reopened, edited]
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: pr-title-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
pr-title:
|
||||
name: Conventional PR title
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 2
|
||||
steps:
|
||||
- name: Validate title for Release Please
|
||||
env:
|
||||
PR_TITLE: ${{ github.event.pull_request.title }}
|
||||
run: >-
|
||||
node -e 'const title = process.env.PR_TITLE ?? "";
|
||||
const conventional = /^(?:feat|fix|docs|refactor|perf|test|build|ci|chore|revert)(?:\([a-z0-9._/-]+\))?!?: .+/u;
|
||||
if (!conventional.test(title)) {
|
||||
console.error("PR title must use Conventional Commits, for example: feat: add historical source import");
|
||||
process.exit(1);
|
||||
}'
|
||||
@@ -1,98 +0,0 @@
|
||||
name: Real agentenv environment bundle
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
PIP_DISABLE_PIP_VERSION_CHECK: "1"
|
||||
|
||||
jobs:
|
||||
real-agentenv:
|
||||
name: Export and run an environment bundle with the agentenv runner
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
# Public ECR caps anonymous pulls by data per source IP and hosted runners share IPs, so the
|
||||
# framework's image builds can fail here for hours. mirror.gcr.io serves the same Docker
|
||||
# official images under the same digests, so each base image the pinned framework builds from
|
||||
# is pulled there and named as its Dockerfile names it, as the framework's own CI does.
|
||||
# Digest-pinned names are set in containerd, whose images Docker reads only on the containerd
|
||||
# image store; switching stores restarts dockerd, so it happens before anything uses Docker.
|
||||
- name: Use the containerd image store
|
||||
run: |
|
||||
[ -s /etc/docker/daemon.json ] || echo '{}' | sudo tee /etc/docker/daemon.json >/dev/null
|
||||
jq '.features["containerd-snapshotter"] = true' /etc/docker/daemon.json > "$RUNNER_TEMP/daemon.json"
|
||||
sudo mv "$RUNNER_TEMP/daemon.json" /etc/docker/daemon.json
|
||||
sudo systemctl restart docker
|
||||
docker info --format '{{json .DriverStatus}}' | grep -q io.containerd.snapshotter
|
||||
- name: Acquire the pinned agentenv runner in isolated runner storage and check host Docker
|
||||
shell: bash
|
||||
run: |
|
||||
runner_root="${RUNNER_TEMP}/rea-agentenv-runner"
|
||||
python -m venv "${runner_root}"
|
||||
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1334'
|
||||
"${runner_root}/bin/agent-env" --version
|
||||
docker version --format '{{.Server.Version}}'
|
||||
echo "REA_AGENT_ENV_COMMAND=${runner_root}/bin/agent-env" >> "${GITHUB_ENV}"
|
||||
echo "REA_DOCKER_COMMAND=$(command -v docker)" >> "${GITHUB_ENV}"
|
||||
- name: Pull the pinned framework's public ECR base images from mirror.gcr.io
|
||||
shell: bash
|
||||
run: |
|
||||
package="$("${REA_AGENT_ENV_COMMAND%/agent-env}/python" -c 'import agent_env, pathlib; print(pathlib.Path(agent_env.__file__).parent)')"
|
||||
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}" | awk '{print $2}' | sort -u |
|
||||
while read -r image; do
|
||||
name=${image#public.ecr.aws/docker/library/}
|
||||
source=mirror.gcr.io/library/$name
|
||||
case $name in *@*) source=mirror.gcr.io/library/${name%%[:@]*}@${name#*@} ;; esac
|
||||
for attempt in 1 2 3; do
|
||||
docker pull --quiet "$source" && break
|
||||
if [ "$attempt" = 3 ]; then echo "::warning::couldn't pull $source; $image's build will pull it"; continue 2; fi
|
||||
sleep $((attempt * 10))
|
||||
done
|
||||
case ${name%@*} in *:*) docker tag "$source" "${image%@*}" ;; esac
|
||||
case $image in *@*) sudo ctr --namespace moby images tag "$source" "$image" >/dev/null ;; esac
|
||||
done
|
||||
- name: Export a bundle from the source-owned capture and run it with the agentenv runner
|
||||
run: npm run verify:agentenv:environment
|
||||
@@ -1,72 +0,0 @@
|
||||
name: Real offline binary layout
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
large_mcp:
|
||||
description: "Also verify the complete 65,281-section result over MCP (slow)"
|
||||
type: boolean
|
||||
default: false
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
REA_VERIFY_LARGE_ELF_MCP: ${{ inputs.large_mcp && '1' || '0' }}
|
||||
|
||||
jobs:
|
||||
real-binary-layout:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Acquire exact upstream libraries in an isolated runner environment
|
||||
shell: bash
|
||||
run: |
|
||||
layout_python_root="${RUNNER_TEMP}/rea-pwntools-layout"
|
||||
python3 -m venv "${layout_python_root}"
|
||||
"${layout_python_root}/bin/python" -m pip install --disable-pip-version-check \
|
||||
'pwntools==4.15.0' 'pyelftools==0.33' 'unicorn==2.1.2'
|
||||
command -v gcc
|
||||
command -v ld
|
||||
command -v strip
|
||||
command -v strace
|
||||
echo "REA_PWNTOOLS_PYTHON=${layout_python_root}/bin/python" >> "${GITHUB_ENV}"
|
||||
echo "REA_VERIFY_STRACE_COMMAND=$(command -v strace)" >> "${GITHUB_ENV}"
|
||||
- name: Verify source-owned fixtures through public CLI and MCP
|
||||
run: npm run verify:binary:layout
|
||||
@@ -1,54 +0,0 @@
|
||||
name: Real browser verification
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
real-browser:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: npm run verify:browser
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
@@ -1,84 +0,0 @@
|
||||
name: Real upstream Cutter verification
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
repository_dispatch:
|
||||
types: [real-cutter-upstream]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
|
||||
jobs:
|
||||
windows-native-artifact:
|
||||
name: Build Windows bridge security boundary
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Install the cross compiler used by the Windows native lane
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
|
||||
- run: npm run build:windows-native
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: cutter-windows-native
|
||||
path: native/windows/build/
|
||||
if-no-files-found: error
|
||||
|
||||
cutter-upstream:
|
||||
name: Cutter upstream smoke (${{ matrix.platform }})
|
||||
needs: windows-native-artifact
|
||||
runs-on: ${{ matrix.runner }}
|
||||
timeout-minutes: 20
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- platform: linux
|
||||
runner: [self-hosted, cutter-upstream, linux]
|
||||
- platform: macOS
|
||||
runner: [self-hosted, cutter-upstream, macOS]
|
||||
- platform: Windows
|
||||
runner: [self-hosted, cutter-upstream, Windows]
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- if: matrix.platform == 'Windows'
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: cutter-windows-native
|
||||
path: native/windows/build/
|
||||
- if: matrix.platform == 'Windows'
|
||||
run: npm run verify:windows-native:artifact
|
||||
- if: matrix.platform == 'Windows'
|
||||
run: npm run build:cached
|
||||
- if: matrix.platform == 'Windows'
|
||||
run: npm run verify:windows-native
|
||||
- name: Verify host and real Cutter bridge
|
||||
env:
|
||||
EXPECTED_PLATFORM: ${{ matrix.platform }}
|
||||
run: npm run verify:cutter
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
if: always()
|
||||
with:
|
||||
name: cutter-upstream-${{ matrix.platform }}
|
||||
path: cutter-smoke.json
|
||||
if-no-files-found: ignore
|
||||
retention-days: 30
|
||||
@@ -1,65 +0,0 @@
|
||||
name: Real offline EVM interface
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
|
||||
jobs:
|
||||
real-evm-interface:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Acquire unchanged fixture compiler in isolated runner storage
|
||||
shell: bash
|
||||
run: |
|
||||
evm_fixture_root="${RUNNER_TEMP}/rea-evm-fixture-compiler"
|
||||
mkdir -p "${evm_fixture_root}"
|
||||
npm install --prefix "${evm_fixture_root}" --ignore-scripts --no-audit --no-fund --save-exact solc@0.8.30
|
||||
command -v prlimit
|
||||
command -v strace
|
||||
echo "REA_VERIFY_STRACE_COMMAND=$(command -v strace)" >> "${GITHUB_ENV}"
|
||||
echo "REA_VERIFY_SOLC_MODULE=${evm_fixture_root}/node_modules/solc/index.js" >> "${GITHUB_ENV}"
|
||||
- name: Verify source-owned compiled bytecode through public CLI and MCP
|
||||
run: npm run verify:evm:interface
|
||||
- name: Verify content-hash-free and CBOR-free owned compiler artifacts
|
||||
run: npm run verify:solidity:metadata
|
||||
@@ -1,129 +0,0 @@
|
||||
name: Real Ghidra Windows E2E
|
||||
|
||||
on:
|
||||
repository_dispatch:
|
||||
types: [real-ghidra-windows]
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
native-artifact:
|
||||
name: Build matching Windows native artifact
|
||||
if: github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Prepare cross compiler
|
||||
run: |
|
||||
for source in /etc/apt/apt-mirrors.txt /etc/apt/sources.list /etc/apt/sources.list.d/ubuntu.sources; do
|
||||
if [ -f "$source" ]; then
|
||||
sudo sed -i 's|azure.archive.ubuntu.com|archive.ubuntu.com|g; s|http://archive.ubuntu.com|https://archive.ubuntu.com|g' "$source"
|
||||
fi
|
||||
done
|
||||
sudo apt-get update
|
||||
sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
|
||||
- run: npm run build:windows-native
|
||||
- run: npm run verify:windows-native:artifact
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: real-ghidra-windows-native
|
||||
path: native/windows/build/
|
||||
if-no-files-found: error
|
||||
# The boundary observations need a separate Windows executable, which the
|
||||
# native lane skips when it is absent. Cross-build it here, where MinGW is
|
||||
# already installed, so the proof run observes them too.
|
||||
- name: Build Windows process boundary fixture
|
||||
run: |
|
||||
mkdir -p .cache/windows-process-boundary
|
||||
x86_64-w64-mingw32-g++ -municode -std=c++17 -O2 -static -Wall -Wextra \
|
||||
-o .cache/windows-process-boundary/process-boundary.exe \
|
||||
tests/fixtures/windows/processBoundary.cc -ladvapi32
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: real-ghidra-windows-process-boundary
|
||||
path: .cache/windows-process-boundary/
|
||||
if-no-files-found: error
|
||||
|
||||
verify:
|
||||
name: Real Ghidra 12.1.4 Windows x64 PE proof
|
||||
if: github.ref == 'refs/heads/main'
|
||||
needs: native-artifact
|
||||
environment: real-ghidra-windows
|
||||
runs-on: [self-hosted, Windows, x64, ghidra-12-1-2]
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: real-ghidra-windows-native
|
||||
path: native/windows/build/
|
||||
- run: npm run verify:windows-native:artifact
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: real-ghidra-windows-process-boundary
|
||||
path: .cache/windows-process-boundary/
|
||||
- run: >-
|
||||
npm run verify:windows-native -- .
|
||||
"${{ github.workspace }}\.cache\windows-process-boundary\process-boundary.exe"
|
||||
- name: Verify all admitted Windows Ghidra operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows | Tee-Object -FilePath windows-ghidra-proof.log
|
||||
- name: Verify native x86 Windows Ghidra operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows -- --x86 | Tee-Object -FilePath windows-ghidra-x86-proof.log
|
||||
- name: Verify native x64 DLL Windows Ghidra operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows -- --dll | Tee-Object -FilePath windows-ghidra-dll-proof.log
|
||||
- name: Verify native x86 DLL Windows Ghidra operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows -- --x86 --dll | Tee-Object -FilePath windows-ghidra-x86-dll-proof.log
|
||||
- name: Verify ordinary-user CLI and MCP operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows:package | Tee-Object -FilePath windows-ghidra-package-proof.log
|
||||
- name: Verify ordinary-user x86 CLI and MCP operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows:package -- --x86 | Tee-Object -FilePath windows-ghidra-x86-package-proof.log
|
||||
- name: Verify ordinary-user x64 DLL CLI and MCP operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows:package -- --dll | Tee-Object -FilePath windows-ghidra-dll-package-proof.log
|
||||
- name: Verify ordinary-user x86 DLL CLI and MCP operations
|
||||
env:
|
||||
REA_ANALYSIS_PROVIDER: ghidra
|
||||
run: npm run verify:ghidra:windows:package -- --x86 --dll | Tee-Object -FilePath windows-ghidra-x86-dll-package-proof.log
|
||||
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
if: always()
|
||||
with:
|
||||
name: windows-ghidra-proof
|
||||
path: |
|
||||
windows-ghidra-proof.log
|
||||
windows-ghidra-package-proof.log
|
||||
windows-ghidra-x86-proof.log
|
||||
windows-ghidra-x86-package-proof.log
|
||||
windows-ghidra-dll-proof.log
|
||||
windows-ghidra-x86-dll-proof.log
|
||||
windows-ghidra-dll-package-proof.log
|
||||
windows-ghidra-x86-dll-package-proof.log
|
||||
retention-days: 30
|
||||
@@ -1,53 +0,0 @@
|
||||
name: Real Go binary metadata
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- ".github/workflows/real-go-binary.yml"
|
||||
- "src/go/**"
|
||||
- "src/domain/go/**"
|
||||
- "src/domain/dosMz.ts"
|
||||
- "src/application/go/**"
|
||||
- "src/contracts/go/**"
|
||||
- "src/composition/go.ts"
|
||||
- "src/cli/goCommands.ts"
|
||||
- "src/server/registerGoTools.ts"
|
||||
- "scripts/verify-go-binary.mjs"
|
||||
- "scripts/lib/mcp-verifier-results.mjs"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: real-go-binary-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
|
||||
jobs:
|
||||
real-go-binary:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- uses: actions/setup-go@40f1582b2485089dde7abd97c1529aa768e1baff # v5.6.0
|
||||
with:
|
||||
go-version: "1.26.1"
|
||||
cache: false
|
||||
- run: npm ci
|
||||
- name: Verify compiler-produced ELF, PE and Mach-O through public CLI and MCP
|
||||
run: npm run verify:go:binary
|
||||
@@ -2,64 +2,42 @@ name: Real Hopper Linux E2E
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
inputs:
|
||||
first_target:
|
||||
description: Absolute path to first Linux test binary
|
||||
required: true
|
||||
type: string
|
||||
second_target:
|
||||
description: Absolute path to a distinct Linux test binary
|
||||
required: true
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
clean-setup:
|
||||
name: Clean setup, CLI, and MCP
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 25
|
||||
verify:
|
||||
name: Installed package against real Linux Hopper
|
||||
runs-on: [self-hosted, Linux, hopper]
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- name: Pack CLI for setup verification
|
||||
- run: npm run rebuild:native
|
||||
- name: Install packed CLI in isolation
|
||||
shell: bash
|
||||
run: |
|
||||
tarball="$(npm pack --silent)"
|
||||
prefix="${RUNNER_TEMP}/rea-prefix"
|
||||
npm install --global --prefix "${prefix}" "./${tarball}"
|
||||
echo "REA_VERIFY_SERVER_COMMAND=${prefix}/bin/rea" >> "${GITHUB_ENV}"
|
||||
- name: Verify clean Linux setup through CLI and MCP
|
||||
run: sudo --preserve-env=REA_VERIFY_SERVER_COMMAND,PATH npm run verify:setup:linux
|
||||
|
||||
verify:
|
||||
name: Installed package against real Linux Hopper
|
||||
runs-on: [self-hosted, Linux, hopper]
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- name: Install packed CLI in isolation
|
||||
shell: bash
|
||||
run: |
|
||||
tarball="$(npm pack --silent)"
|
||||
prefix="${RUNNER_TEMP}/rea-prefix"
|
||||
npm install --global --ignore-scripts --prefix "${prefix}" "./${tarball}"
|
||||
echo "REA_VERIFY_SERVER_COMMAND=${prefix}/bin/rea" >> "${GITHUB_ENV}"
|
||||
echo 'REA_VERIFY_SERVER_ARGS_JSON=["mcp"]' >> "${GITHUB_ENV}"
|
||||
- name: Verify source-owned ELF fixtures through an alternate launcher symlink
|
||||
- name: Verify two binaries through the in-process bridge
|
||||
env:
|
||||
REA_HOPPER_CONFORMANCE_MANIFEST_PATH: ${{ github.workspace }}/build/conformance/manifest.json
|
||||
HOPPER_TARGET_PATH: ${{ inputs.first_target }}
|
||||
HOPPER_SECOND_TARGET_PATH: ${{ inputs.second_target }}
|
||||
run: npm run verify:hopper:linux
|
||||
@@ -2,14 +2,15 @@ name: Real Hopper E2E
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
inputs:
|
||||
first_target:
|
||||
description: Absolute path to first test binary on the runner
|
||||
required: true
|
||||
type: string
|
||||
second_target:
|
||||
description: Absolute path to distinct second test binary on the runner
|
||||
required: true
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
@@ -20,23 +21,23 @@ jobs:
|
||||
runs-on: [self-hosted, macOS, hopper]
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
cache: npm
|
||||
- run: npm ci
|
||||
- run: npm run rebuild:native
|
||||
- name: Install packed CLI in isolation
|
||||
shell: bash
|
||||
run: |
|
||||
tarball="$(npm pack --silent)"
|
||||
prefix="${RUNNER_TEMP}/rea-prefix"
|
||||
npm install --global --ignore-scripts --prefix "${prefix}" "./${tarball}"
|
||||
npm install --global --prefix "${prefix}" "./${tarball}"
|
||||
echo "REA_VERIFY_SERVER_COMMAND=${prefix}/bin/rea" >> "${GITHUB_ENV}"
|
||||
echo 'REA_VERIFY_SERVER_ARGS_JSON=["mcp"]' >> "${GITHUB_ENV}"
|
||||
- name: Verify source-owned fixtures with real Hopper
|
||||
- name: Verify two real binaries
|
||||
env:
|
||||
REA_HOPPER_CONFORMANCE_MANIFEST_PATH: ${{ github.workspace }}/build/conformance/manifest.json
|
||||
HOPPER_TARGET_PATH: ${{ inputs.first_target }}
|
||||
HOPPER_SECOND_TARGET_PATH: ${{ inputs.second_target }}
|
||||
run: npm run verify:hopper
|
||||
@@ -1,66 +0,0 @@
|
||||
name: Real JavaScript recovery
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
NODE_OPTIONS: "--max-old-space-size=768"
|
||||
RAYON_NUM_THREADS: "1"
|
||||
|
||||
jobs:
|
||||
real-javascript-recovery:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Fetch pinned Wakaru release into the job workspace
|
||||
shell: bash
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/rea-wakaru"
|
||||
curl --fail --location --retry 3 https://github.com/pionxzh/wakaru/releases/download/v1.14.0/wakaru-linux-x64.tar.gz -o "$RUNNER_TEMP/rea-wakaru/release.tar.gz"
|
||||
tar -xzf "$RUNNER_TEMP/rea-wakaru/release.tar.gz" -C "$RUNNER_TEMP/rea-wakaru" wakaru
|
||||
printf '%s %s\n' 664fa06782a61eda7ed15885134b52525fd565dd77607e642062ceb6cd927c38 "$RUNNER_TEMP/rea-wakaru/wakaru" | sha256sum --check
|
||||
- name: Install isolated fixture compilers
|
||||
run: npm install --prefix "$RUNNER_TEMP/rea-fixture-compilers" --ignore-scripts --no-audit --no-fund esbuild@0.25.10 webpack@5.101.3
|
||||
- run: npm run build:cached
|
||||
- run: npm run verify:javascript:recovery
|
||||
env:
|
||||
REA_WAKARU_COMMAND: ${{ runner.temp }}/rea-wakaru/wakaru
|
||||
REA_JAVASCRIPT_FIXTURE_TOOLS: ${{ runner.temp }}/rea-fixture-compilers
|
||||
@@ -1,56 +0,0 @@
|
||||
name: Real packet capture
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/captures/**"
|
||||
- "tests/boundary/cli/captureCleanup.test.ts"
|
||||
- "tests/boundary/mcp/packetCaptureMcp.test.ts"
|
||||
- "src/captures/**"
|
||||
- "src/pcap/**"
|
||||
- "src/domain/pcap/**"
|
||||
- "src/domain/analysisView/pcapCaptureView.ts"
|
||||
- "src/application/pcap/**"
|
||||
- "src/contracts/pcap/**"
|
||||
- "src/server/registerPcapTools.ts"
|
||||
- "tests/boundary/pcap/**"
|
||||
- "src/domain/captures/**"
|
||||
- "src/application/captures/**"
|
||||
- "src/contracts/captures/**"
|
||||
- "src/composition/captures.ts"
|
||||
- "src/cli/captureCommands.ts"
|
||||
- "src/server/registerCaptureTools.ts"
|
||||
- "scripts/verify-packet-capture.mjs"
|
||||
- ".github/workflows/real-packet-capture.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
tshark:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y tshark util-linux python3
|
||||
- run: npm ci
|
||||
- run: npm run build:cached
|
||||
- run: npm run test:focused -- tests/boundary/captures/fieldDiscovery.test.ts src/process/LinuxResourceLimits.test.ts tests/boundary/captures/captureProvider.test.ts tests/boundary/captures/captureDescriptorCleanup.test.ts tests/boundary/pcap/pcapProvider.test.ts tests/boundary/pcap/pcapProviderCrossPlatform.test.ts tests/boundary/captures/resourceLimits.test.ts tests/boundary/captures/packetOutput.test.ts tests/boundary/mcp/packetCaptureMcp.test.ts tests/boundary/cli/captureCleanup.test.ts
|
||||
- run: node scripts/verify-packet-capture.mjs
|
||||
env:
|
||||
REA_TSHARK_COMMAND: /usr/bin/tshark
|
||||
REA_CAPINFOS_COMMAND: /usr/bin/capinfos
|
||||
@@ -1,102 +0,0 @@
|
||||
name: Real recorded crash evidence
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
UV_CONCURRENT_BUILDS: "1"
|
||||
UV_CONCURRENT_INSTALLS: "1"
|
||||
UV_PYTHON_DOWNLOADS: never
|
||||
|
||||
jobs:
|
||||
real-recorded-crash:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
image: ["ubuntu:24.04", "debian:trixie-slim"]
|
||||
container: ${{ matrix.image }}
|
||||
steps:
|
||||
- name: Install fixture and debugger prerequisites in the disposable container
|
||||
shell: bash
|
||||
run: |
|
||||
apt-get update -qq
|
||||
apt-get install --yes --no-install-recommends \
|
||||
ca-certificates git python3-venv gcc libc6-dev strace gdb
|
||||
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Acquire exact unchanged upstreams in isolated runner storage
|
||||
shell: bash
|
||||
run: |
|
||||
command -v gcc
|
||||
command -v gdb
|
||||
command -v strace
|
||||
core_decoder_root="${RUNNER_TEMP}/rea-core-decoder"
|
||||
core_pwndbg_root="${RUNNER_TEMP}/rea-core-pwndbg"
|
||||
core_pwndbg_venv="${RUNNER_TEMP}/rea-core-pwndbg-venv"
|
||||
python3 -m venv "${core_decoder_root}"
|
||||
"${core_decoder_root}/bin/python" -m pip install --disable-pip-version-check \
|
||||
'pwntools==4.15.0' 'pyelftools==0.33' 'unicorn==2.1.2' 'uv==0.12.0'
|
||||
git init "${core_pwndbg_root}"
|
||||
git -C "${core_pwndbg_root}" remote add origin https://github.com/pwndbg/pwndbg.git
|
||||
git -C "${core_pwndbg_root}" fetch --depth=1 origin 55596aa676a2756efb8d67034f31675170eabe48
|
||||
git -C "${core_pwndbg_root}" checkout --detach FETCH_HEAD
|
||||
test "$(git -C "${core_pwndbg_root}" rev-parse HEAD)" = 55596aa676a2756efb8d67034f31675170eabe48
|
||||
UV_PROJECT_ENVIRONMENT="${core_pwndbg_venv}" \
|
||||
"${core_decoder_root}/bin/uv" sync --project "${core_pwndbg_root}" \
|
||||
--frozen --no-editable --no-default-groups --python /usr/bin/python3 --no-managed-python
|
||||
test -z "$(git -C "${core_pwndbg_root}" status --porcelain --untracked-files=no)"
|
||||
echo "REA_PWNTOOLS_PYTHON=${core_decoder_root}/bin/python" >> "${GITHUB_ENV}"
|
||||
echo "REA_PWNDBG_GDB=$(command -v gdb)" >> "${GITHUB_ENV}"
|
||||
echo "REA_PWNDBG_GDBINIT=${core_pwndbg_root}/gdbinit.py" >> "${GITHUB_ENV}"
|
||||
echo "REA_PWNDBG_VENV_PATH=${core_pwndbg_venv}" >> "${GITHUB_ENV}"
|
||||
echo "REA_VERIFY_STRACE_COMMAND=$(command -v strace)" >> "${GITHUB_ENV}"
|
||||
- name: Build and install the exact package in isolated runner storage
|
||||
shell: bash
|
||||
run: |
|
||||
npm run build:cached
|
||||
npm pack --ignore-scripts --json --pack-destination "${RUNNER_TEMP}" > "${RUNNER_TEMP}/rea-core-pack.json"
|
||||
core_package_name=$(node -p 'JSON.parse(require("node:fs").readFileSync(process.argv[1], "utf8"))[0].filename' "${RUNNER_TEMP}/rea-core-pack.json")
|
||||
npm install --prefix "${RUNNER_TEMP}/rea-core-installed" --ignore-scripts --no-fund --no-audit "${RUNNER_TEMP}/${core_package_name}"
|
||||
- name: Verify installed CLI and MCP against the source-owned recording
|
||||
shell: bash
|
||||
run: node scripts/verify-recorded-crash.mjs "${RUNNER_TEMP}/rea-core-installed/node_modules/rea-agents/scripts/rea.mjs"
|
||||
@@ -1,48 +0,0 @@
|
||||
name: Real signal recording
|
||||
on:
|
||||
workflow_dispatch:
|
||||
pull_request:
|
||||
paths:
|
||||
- "tests/boundary/signals/**"
|
||||
- "tests/boundary/cli/signalCleanup.test.ts"
|
||||
- "tests/boundary/mcp/signalRecordingMcp.test.ts"
|
||||
- "src/signals/**"
|
||||
- "src/domain/signals/**"
|
||||
- "src/application/signals/**"
|
||||
- "src/contracts/signals/**"
|
||||
- "src/composition/signals.ts"
|
||||
- "src/cli/signalCommands.ts"
|
||||
- "src/server/registerSignalTools.ts"
|
||||
- "scripts/verify-signal-recording.mjs"
|
||||
- ".github/workflows/real-signal-recording.yml"
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
concurrency:
|
||||
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
sigrok:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: sudo apt-get update && sudo apt-get install -y sigrok-cli util-linux python3
|
||||
- run: npm ci
|
||||
- run: npm run build:cached
|
||||
- run: npm run test:focused -- tests/boundary/signals/decoderDiscovery.test.ts src/process/LinuxResourceLimits.test.ts src/domain/signals/annotationTextWindow.test.ts tests/boundary/signals/recordingProvider.test.ts tests/boundary/signals/recordingDescriptorCleanup.test.ts tests/boundary/signals/resourceLimits.test.ts tests/boundary/mcp/signalRecordingMcp.test.ts tests/boundary/cli/signalCleanup.test.ts
|
||||
- run: node scripts/verify-signal-recording.mjs
|
||||
env:
|
||||
REA_SIGROK_COMMAND: /usr/bin/sigrok-cli
|
||||
@@ -1,62 +0,0 @@
|
||||
name: Real Termux browser verification
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: real-termux-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
|
||||
jobs:
|
||||
real-termux:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- name: Download the pinned Termux app and verify its release checksum
|
||||
shell: bash
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/rea-termux"
|
||||
cd "$RUNNER_TEMP/rea-termux"
|
||||
release=https://github.com/termux/termux-app/releases/download/v0.118.3
|
||||
apk=termux-app_v0.118.3+github-debug_x86_64.apk
|
||||
sums=termux-app_v0.118.3+github-debug_sha256sums
|
||||
curl --fail --location --retry 3 --output "$apk" "$release/$apk"
|
||||
curl --fail --location --retry 3 --output "$sums" "$release/$sums"
|
||||
awk -v apk="$apk" '$2 == apk || $2 == "*" apk' "$sums" > selected.sha256
|
||||
test "$(wc -l < selected.sha256)" -eq 1
|
||||
sha256sum --check selected.sha256
|
||||
- name: Enable emulator acceleration
|
||||
run: |
|
||||
echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
|
||||
sudo udevadm control --reload-rules
|
||||
sudo udevadm trigger --name-match=kvm
|
||||
- name: Build and capture a page inside Termux
|
||||
uses: reactivecircus/android-emulator-runner@a421e43855164a8197daf9d8d40fe71c6996bb0d # v2.38.0
|
||||
with:
|
||||
api-level: 30
|
||||
arch: x86_64
|
||||
target: google_apis
|
||||
ram-size: 4096M
|
||||
heap-size: 512M
|
||||
disk-size: 8192M
|
||||
disable-animations: true
|
||||
script: bash scripts/verify/termux/emulator.sh
|
||||
- name: Upload Termux and Android diagnostics
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
with:
|
||||
name: termux-browser-diagnostics
|
||||
path: ${{ runner.temp }}/rea-termux/diagnostics
|
||||
if-no-files-found: warn
|
||||
@@ -1,67 +0,0 @@
|
||||
name: Real historical web network captures
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
|
||||
jobs:
|
||||
real-web-network-captures:
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- name: Use pinned package manager
|
||||
run: npm install --global npm@11.16.0
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Acquire fixed upstream native profile in runner temporary storage
|
||||
shell: bash
|
||||
run: |
|
||||
capture_tool_root="${RUNNER_TEMP}/rea-mitmproxy-12.2.3"
|
||||
mkdir -p "${capture_tool_root}"
|
||||
curl --fail --location --retry 3 --max-time 180 \
|
||||
https://downloads.mitmproxy.org/12.2.3/mitmproxy-12.2.3-linux-x86_64.tar.gz \
|
||||
--output "${capture_tool_root}/mitmproxy.tar.gz"
|
||||
echo '2e95286b618fa6fd33e5e62a78c2e5112571d85f42ec2bac29b97ee242bdb5c5 mitmproxy.tar.gz' \
|
||||
| (cd "${capture_tool_root}" && sha256sum --check --strict)
|
||||
tar -xzf "${capture_tool_root}/mitmproxy.tar.gz" -C "${capture_tool_root}"
|
||||
echo "REA_MITMDUMP_COMMAND=${capture_tool_root}/mitmdump" >> "${GITHUB_ENV}"
|
||||
- name: Verify upstream producer fixtures through public CLI and MCP
|
||||
run: npm run verify:web:network-captures
|
||||
@@ -1,50 +0,0 @@
|
||||
name: Real website runtime attribution
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
jobs:
|
||||
real-web-runtime:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- run: env -u DBUS_SESSION_BUS_ADDRESS npm run verify:browser:runtime
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
@@ -1,53 +0,0 @@
|
||||
name: Real website source-map verification
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
jobs:
|
||||
real-web-source-map:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install the pinned compiler into an isolated fixture directory
|
||||
run: npm install --prefix "$RUNNER_TEMP/rea-web-source-map-compiler" --no-audit --no-fund --package-lock=false esbuild@0.25.10
|
||||
- run: npm run verify:browser:source-maps
|
||||
env:
|
||||
REA_BROWSER_EXECUTABLE: /usr/bin/google-chrome
|
||||
REA_BROWSER_NO_SANDBOX: "true"
|
||||
REA_WEB_SOURCE_MAP_COMPILER: ${{ runner.temp }}/rea-web-source-map-compiler/node_modules/esbuild/lib/main.js
|
||||
@@ -1,109 +0,0 @@
|
||||
name: Recover published release
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
version:
|
||||
description: Existing npm and GitHub release version
|
||||
required: true
|
||||
type: string
|
||||
release_run_id:
|
||||
description: Original Release run with successful exact-source qualification and npm publication
|
||||
required: true
|
||||
type: string
|
||||
|
||||
concurrency:
|
||||
group: release-recovery-${{ inputs.version }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
verify:
|
||||
name: Verify existing published package
|
||||
if: github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
outputs:
|
||||
source_sha: ${{ steps.release.outputs.source_sha }}
|
||||
version: ${{ steps.release.outputs.version }}
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
- run: npm ci
|
||||
- name: Validate original release authority
|
||||
id: release
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
RECOVERY_VERSION: ${{ inputs.version }}
|
||||
RECOVERY_RUN_ID: ${{ inputs.release_run_id }}
|
||||
run: node scripts/ci/release-recovery.mjs resolve
|
||||
- name: Check out released metadata
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ steps.release.outputs.source_sha }}
|
||||
path: .cache/release-source
|
||||
persist-credentials: false
|
||||
- name: Download original qualified package
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: verified-release-package
|
||||
run-id: ${{ steps.release.outputs.run_id }}
|
||||
github-token: ${{ github.token }}
|
||||
path: .cache/verified-package
|
||||
- name: Restore original verifier runtime
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: rea-test-runtime
|
||||
run-id: ${{ steps.release.outputs.run_id }}
|
||||
github-token: ${{ github.token }}
|
||||
path: .
|
||||
- name: Match published npm bytes to the qualified package
|
||||
env:
|
||||
RECOVERY_VERSION: ${{ steps.release.outputs.version }}
|
||||
RECOVERY_RUN_ID: ${{ steps.release.outputs.run_id }}
|
||||
RECOVERY_RUN_ATTEMPT: ${{ steps.release.outputs.run_attempt }}
|
||||
RECOVERY_SOURCE_SHA: ${{ steps.release.outputs.source_sha }}
|
||||
RECOVERY_SOURCE_DIRECTORY: .cache/release-source
|
||||
RECOVERY_ARTIFACT_DIRECTORY: .cache/verified-package
|
||||
run: node scripts/ci/release-recovery.mjs verify
|
||||
- name: Rehearse the original qualified tarball
|
||||
env:
|
||||
RECOVERY_VERSION: ${{ steps.release.outputs.version }}
|
||||
run: node scripts/verify-release-artifact.mjs .cache/verified-package/package.tgz "$RECOVERY_VERSION"
|
||||
- name: Verify published CLI, MCP and update
|
||||
env:
|
||||
RECOVERY_VERSION: ${{ steps.release.outputs.version }}
|
||||
run: node scripts/verify-published-package.mjs "$RECOVERY_VERSION"
|
||||
- name: Validate released registry metadata
|
||||
working-directory: .cache/release-source
|
||||
run: npm run mcp-registry:check
|
||||
|
||||
publish-mcp:
|
||||
name: Finish MCP Registry publication
|
||||
needs: verify
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Check out verified released metadata
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ needs.verify.outputs.source_sha }}
|
||||
path: .cache/release-source
|
||||
persist-credentials: false
|
||||
- uses: ./.github/actions/publish-mcp
|
||||
with:
|
||||
directory: .cache/release-source
|
||||
+28
-386
@@ -3,361 +3,34 @@ name: Release
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
types: [closed]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
release_branch:
|
||||
description: Frozen release branch (release/VERSION)
|
||||
required: true
|
||||
type: string
|
||||
phase:
|
||||
description: Prepare the release PR, or publish its reviewed merge
|
||||
required: true
|
||||
default: prepare
|
||||
type: choice
|
||||
options: [prepare, publish]
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# Remote cache stays local when either value is empty.
|
||||
# Writes only on the default branch for push, schedule, manual dispatch,
|
||||
# and workflow_call. Pull requests get an empty token and remote read-only mode.
|
||||
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
|
||||
|
||||
concurrency:
|
||||
group: release-${{ inputs.release_branch || github.event.pull_request.merge_commit_sha || github.ref_name }}
|
||||
cancel-in-progress: false
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
id-token: write
|
||||
|
||||
jobs:
|
||||
release-proposal:
|
||||
name: Update release proposal
|
||||
if: github.event_name == 'push'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Respect finalized release notes
|
||||
id: editorial
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
run: |
|
||||
finalized=$(gh api --paginate "repos/${GH_REPO}/pulls?state=open&base=main&per_page=100" \
|
||||
--jq '.[] | select(.head.repo.full_name == .base.repo.full_name) | select(.head.ref | startswith("release-please--branches--main--")) | select(any(.labels[]; .name == "rea:release-notes-finalized")) | .number')
|
||||
if [[ -n "${finalized}" ]]; then
|
||||
echo "Finalized release notes on PR ${finalized}; automatic proposal refresh is paused." >> "$GITHUB_STEP_SUMMARY"
|
||||
echo "frozen=true" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
if: steps.editorial.outputs.frozen != 'true'
|
||||
with:
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: main
|
||||
skip-github-release: true
|
||||
|
||||
merged-release:
|
||||
name: Release reviewed main merge
|
||||
if: >-
|
||||
github.event_name == 'pull_request' && github.event.pull_request.merged == true &&
|
||||
github.event.pull_request.head.repo.full_name == github.repository &&
|
||||
startsWith(github.event.pull_request.head.ref, 'release-please--branches--main--')
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
outputs:
|
||||
release_created: ${{ steps.release.outputs.release_created }}
|
||||
sha: ${{ steps.release.outputs.sha }}
|
||||
steps:
|
||||
- name: Check out reviewed release merge
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.merge_commit_sha }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
|
||||
- run: npm ci
|
||||
|
||||
- name: Validate reviewed merge before release creation
|
||||
env:
|
||||
SOURCE_SHA: ${{ github.event.pull_request.merge_commit_sha }}
|
||||
run: |
|
||||
if [[ "${SOURCE_SHA}" != "${GITHUB_SHA}" ]]; then
|
||||
echo "The reviewed merge differs from the workflow source; refusing mismatched publication provenance." >&2
|
||||
exit 1
|
||||
fi
|
||||
node scripts/verify-release-checkpoint.mjs \
|
||||
--phase publish --release-branch main --source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Validate finalized notes before release creation
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
RELEASE_PR: ${{ github.event.pull_request.number }}
|
||||
SOURCE_SHA: ${{ github.event.pull_request.merge_commit_sha }}
|
||||
run: |
|
||||
node scripts/release-notes.mjs publication --repository "$GH_REPO" \
|
||||
--source "$SOURCE_SHA" --pr "$RELEASE_PR"
|
||||
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
id: release
|
||||
with:
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: main
|
||||
skip-github-release: false
|
||||
skip-github-pull-request: true
|
||||
|
||||
- name: Bind publication to the reviewed merge
|
||||
env:
|
||||
RELEASE_CREATED: ${{ steps.release.outputs.release_created }}
|
||||
RELEASE_SHA: ${{ steps.release.outputs.sha }}
|
||||
SOURCE_SHA: ${{ github.event.pull_request.merge_commit_sha }}
|
||||
run: |
|
||||
if [[ "${RELEASE_CREATED}" != true ]]; then
|
||||
echo "Release Please did not create a release for the reviewed merge. Check its pending label and release metadata; for a partial publication, re-run failed jobs in the original run." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "${RELEASE_SHA}" != "${SOURCE_SHA}" ]]; then
|
||||
echo "The tagged release differs from the reviewed merge; refusing mismatched publication provenance. The GitHub release already exists; do not create another release as a retry." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
release-please:
|
||||
name: Release Please
|
||||
if: github.event_name == 'workflow_dispatch'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
outputs:
|
||||
release_created: ${{ steps.release.outputs.release_created }}
|
||||
sha: ${{ steps.release.outputs.sha }}
|
||||
tag_name: ${{ steps.release.outputs.tag_name }}
|
||||
steps:
|
||||
- name: Validate release selection
|
||||
shell: bash
|
||||
env:
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
RELEASE_PHASE: ${{ inputs.phase }}
|
||||
run: |
|
||||
if [[ "${RELEASE_PHASE}" != prepare && "${RELEASE_PHASE}" != publish ]]; then
|
||||
echo "phase must be prepare or publish." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "${RELEASE_PHASE}" == prepare && "${GITHUB_REF}" != "refs/heads/main" ]]; then
|
||||
echo "Run prepare from main; release_branch selects the frozen source." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "${RELEASE_PHASE}" == publish && "${GITHUB_REF}" != "refs/heads/${RELEASE_BRANCH}" ]]; then
|
||||
echo "Run publish from the selected frozen release branch so npm provenance records its source." >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ ! "${RELEASE_BRANCH}" =~ ^release/[^/]+$ ]]; then
|
||||
echo "release_branch must select a release/ branch; main is not a release checkpoint." >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! git check-ref-format "refs/heads/${RELEASE_BRANCH}"; then
|
||||
echo "Invalid release_branch: ${RELEASE_BRANCH}" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Resolve release branch tip
|
||||
id: branch
|
||||
shell: bash
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
run: |
|
||||
sha="$(gh api "repos/${GITHUB_REPOSITORY}/git/ref/heads/${RELEASE_BRANCH}" --jq '.object | select(.type == "commit") | .sha')"
|
||||
if [[ ! "${sha}" =~ ^[0-9a-f]{40}$ ]]; then
|
||||
echo "The release branch tip is not a commit." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "sha=${sha}" >> "${GITHUB_OUTPUT}"
|
||||
|
||||
- name: Require the publish dispatch to match the branch tip
|
||||
if: inputs.phase == 'publish'
|
||||
shell: bash
|
||||
env:
|
||||
BRANCH_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: |
|
||||
if [[ "${BRANCH_SHA}" != "${GITHUB_SHA}" ]]; then
|
||||
echo "The release branch tip differs from the dispatched checkpoint; refusing to create a release for a moved source." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Check out release controller and Git history
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ github.sha }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Node.js for checkpoint validation
|
||||
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
|
||||
- name: Install checkpoint validation dependencies
|
||||
run: npm ci
|
||||
|
||||
- name: Validate checkpoint version and ancestry before release creation
|
||||
env:
|
||||
RELEASE_PHASE: ${{ inputs.phase }}
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: >-
|
||||
node scripts/verify-release-checkpoint.mjs
|
||||
--phase "$RELEASE_PHASE" --release-branch "$RELEASE_BRANCH"
|
||||
--source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Validate manual finalized notes before release creation
|
||||
if: inputs.phase == 'publish'
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
GH_REPO: ${{ github.repository }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: |
|
||||
node scripts/release-notes.mjs publication --repository "$GH_REPO" \
|
||||
--source "$SOURCE_SHA"
|
||||
|
||||
- uses: googleapis/release-please-action@5c625bfb5d1ff62eadeeb3772007f7f66fdcf071 # v4.4.1
|
||||
- uses: googleapis/release-please-action@v4
|
||||
id: release
|
||||
with:
|
||||
# A dedicated credential starts PR CI automatically; the built-in
|
||||
# token creates approval-required PR runs for maintainers to approve.
|
||||
token: ${{ secrets.RELEASE_PLEASE_TOKEN || secrets.GITHUB_TOKEN }}
|
||||
target-branch: ${{ inputs.release_branch }}
|
||||
skip-github-release: ${{ inputs.phase == 'prepare' }}
|
||||
skip-github-pull-request: ${{ inputs.phase == 'publish' }}
|
||||
|
||||
- name: Require a merged release PR for publication
|
||||
if: inputs.phase == 'publish' && steps.release.outputs.release_created != 'true'
|
||||
run: |
|
||||
echo "No new release was created. Merge the reviewed release PR into its frozen branch before publishing. For a partial publication, re-run failed jobs in the original run." >&2
|
||||
exit 1
|
||||
|
||||
- name: Bind publication to the dispatched checkpoint
|
||||
if: inputs.phase == 'publish' && steps.release.outputs.release_created == 'true'
|
||||
shell: bash
|
||||
env:
|
||||
RELEASE_SHA: ${{ steps.release.outputs.sha }}
|
||||
run: |
|
||||
if [[ "${RELEASE_SHA}" != "${GITHUB_SHA}" ]]; then
|
||||
echo "The tagged release SHA differs from the dispatched checkpoint; refusing to publish mismatched source provenance. The GitHub release already exists; do not dispatch publish again." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Check out release pull request
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ fromJSON(steps.release.outputs.pr).headBranchName }}
|
||||
fetch-depth: 0
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Node.js for generated documentation
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
|
||||
- name: Install dependencies
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm ci
|
||||
|
||||
- name: Validate prepared release version and migration notes
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
env:
|
||||
RELEASE_BRANCH: ${{ inputs.release_branch }}
|
||||
SOURCE_SHA: ${{ steps.branch.outputs.sha }}
|
||||
run: >-
|
||||
node scripts/verify-release-checkpoint.mjs
|
||||
--phase prepare --stage candidate --release-branch "$RELEASE_BRANCH"
|
||||
--source-sha "$SOURCE_SHA"
|
||||
|
||||
- name: Regenerate release documentation
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm run docs:generate
|
||||
|
||||
- name: Validate generated release documentation
|
||||
if: inputs.phase == 'prepare' && steps.release.outputs.prs_created == 'true'
|
||||
run: npm run docs:check && git diff --exit-code
|
||||
|
||||
qualification:
|
||||
name: Qualify exact release revision
|
||||
needs: [merged-release, release-please]
|
||||
if: >-
|
||||
always() && !cancelled() &&
|
||||
((needs.merged-release.result == 'success' && needs.merged-release.outputs.release_created == 'true') ||
|
||||
(needs.release-please.result == 'success' && inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true'))
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
uses: ./.github/workflows/ci.yml
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
with:
|
||||
source_sha: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
|
||||
remote_cache_write: true
|
||||
|
||||
rehearsal:
|
||||
name: Rehearse qualified release on a clean runner
|
||||
needs: [merged-release, release-please, qualification]
|
||||
if: >-
|
||||
always() && !cancelled() && needs.qualification.result == 'success'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
|
||||
persist-credentials: false
|
||||
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/prepare-release-package
|
||||
id: package
|
||||
- name: Exercise the exact qualified tarball
|
||||
env:
|
||||
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
|
||||
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
|
||||
run: node scripts/verify-release-artifact.mjs "$VERIFIED_PACKAGE_TARBALL" "$VERIFIED_PACKAGE_VERSION"
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
publish:
|
||||
name: Publish to npm
|
||||
needs: [merged-release, release-please, qualification, rehearsal]
|
||||
if: >-
|
||||
always() && !cancelled() &&
|
||||
needs.qualification.result == 'success' && needs.rehearsal.result == 'success'
|
||||
needs: release-please
|
||||
if: needs.release-please.outputs.release_created == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
|
||||
persist-credentials: false
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
package-manager-cache: false
|
||||
@@ -365,55 +38,24 @@ jobs:
|
||||
|
||||
- run: npm ci
|
||||
|
||||
- uses: ./.github/actions/prepare-release-package
|
||||
id: package
|
||||
|
||||
- name: Inspect existing npm version
|
||||
id: registry
|
||||
env:
|
||||
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
|
||||
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
|
||||
run: node scripts/ci/npm-release.mjs inspect "$VERIFIED_PACKAGE_TARBALL" "$VERIFIED_PACKAGE_VERSION"
|
||||
- name: Verify build
|
||||
run: npm run build
|
||||
|
||||
- name: Publish
|
||||
if: steps.registry.outputs.published == 'false'
|
||||
shell: bash
|
||||
env:
|
||||
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
|
||||
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
|
||||
run: |
|
||||
tag="$(node scripts/release-npm-tag.mjs "$VERIFIED_PACKAGE_VERSION")"
|
||||
npm publish "${VERIFIED_PACKAGE_TARBALL}" --access public --tag "${tag}"
|
||||
|
||||
- name: Verify published npm bytes and dist-tag
|
||||
env:
|
||||
VERIFIED_PACKAGE_TARBALL: ${{ steps.package.outputs.tarball }}
|
||||
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
|
||||
run: node scripts/ci/npm-release.mjs verify "$VERIFIED_PACKAGE_TARBALL" "$VERIFIED_PACKAGE_VERSION"
|
||||
run: npm publish --access public
|
||||
|
||||
- name: Verify published CLI from npm
|
||||
env:
|
||||
VERIFIED_PACKAGE_VERSION: ${{ steps.package.outputs.version }}
|
||||
run: node scripts/verify-published-package.mjs "$VERIFIED_PACKAGE_VERSION"
|
||||
|
||||
publish-mcp:
|
||||
name: Publish to MCP Registry
|
||||
needs: [merged-release, release-please, publish]
|
||||
if: >-
|
||||
always() && !cancelled() && needs.publish.result == 'success' &&
|
||||
((needs.merged-release.result == 'success' && needs.merged-release.outputs.release_created == 'true') ||
|
||||
(needs.release-please.result == 'success' && inputs.phase == 'publish' && needs.release-please.outputs.release_created == 'true'))
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
|
||||
persist-credentials: false
|
||||
|
||||
- name: Verify package and registry metadata versions
|
||||
run: npm run mcp-registry:check
|
||||
|
||||
- uses: ./.github/actions/publish-mcp
|
||||
shell: bash
|
||||
run: |
|
||||
version="$(node --input-type=module -e 'import packageJson from "./package.json" with { type: "json" }; process.stdout.write(packageJson.version)')"
|
||||
for attempt in 1 2 3 4 5 6; do
|
||||
if npm view "rea-agents@${version}" version --json >/dev/null 2>&1; then
|
||||
npx --yes --package "rea-agents@${version}" rea --help
|
||||
npx --yes --package "rea-agents@${version}" rea --llms
|
||||
npm run verify:published -- "${version}"
|
||||
exit 0
|
||||
fi
|
||||
if [ "${attempt}" -lt 6 ]; then sleep 10; fi
|
||||
done
|
||||
echo "Published rea-agents@${version} was not available from npm" >&2
|
||||
exit 1
|
||||
@@ -1,39 +0,0 @@
|
||||
name: SQLite runtime compatibility
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
|
||||
jobs:
|
||||
sqlite:
|
||||
name: SQLite runtime compatibility (Node ${{ matrix.node }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
# Exercise unavailable-runtime diagnostics, the first supported release,
|
||||
# and new patches on both supported lines without hand-editing versions.
|
||||
# The repository-pinned Node 24 version also runs the complete test suite.
|
||||
node: ["22.x", "24.15.0", "24.x", "26.x"]
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version: ${{ matrix.node }}
|
||||
check-latest: true
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: rea-test-runtime
|
||||
path: .
|
||||
- name: Verify SQLite discovery, inspection and cleanup
|
||||
run: npm run test:sqlite:run
|
||||
@@ -1,30 +0,0 @@
|
||||
name: Website checks
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
check:
|
||||
name: Website checks
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Check out the repository
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- name: Set up Python for website assets
|
||||
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --no-install-recommends -y libcairo2 fonts-dejavu-core
|
||||
python3 -m pip install -r website/requirements.txt
|
||||
- name: Check shared HTML, links, assets and contributor workflows
|
||||
run: python3 scripts/website.py check
|
||||
@@ -1,178 +0,0 @@
|
||||
name: Publish REA website
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
target:
|
||||
description: Publish both hosts, or Pages while configuring Cloudflare
|
||||
type: choice
|
||||
required: true
|
||||
default: both
|
||||
options:
|
||||
- both
|
||||
- pages
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: website-pages
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
prepare:
|
||||
if: github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: read
|
||||
pages: read
|
||||
steps:
|
||||
- name: Validate the publication target
|
||||
env:
|
||||
PUBLISH_TARGET: ${{ inputs.target }}
|
||||
run: |
|
||||
case "$PUBLISH_TARGET" in
|
||||
both|pages) ;;
|
||||
*) echo "::error::Unknown publication target: $PUBLISH_TARGET"; exit 1 ;;
|
||||
esac
|
||||
- name: Require Cloudflare publishing credentials
|
||||
if: inputs.target == 'both'
|
||||
env:
|
||||
CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
run: |
|
||||
if [[ -z "$CLOUDFLARE_API_TOKEN" || -z "$CLOUDFLARE_ACCOUNT_ID" ]]; then
|
||||
echo "::error::Set CLOUDFLARE_API_TOKEN and CLOUDFLARE_ACCOUNT_ID in repository Actions secrets before publishing."
|
||||
exit 1
|
||||
fi
|
||||
- name: Check out the repository
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- name: Set up Python for website assets
|
||||
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
cache-dependency-path: website/requirements.txt
|
||||
- name: Install sharing-image dependencies
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --no-install-recommends -y libcairo2 fonts-dejavu-core
|
||||
python3 -m pip install -r website/requirements.txt
|
||||
- name: Check shared HTML, links, assets and contributor workflows
|
||||
run: python3 scripts/website.py check
|
||||
- name: Record the published commit
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
Path("website/_site/deployment.json").write_text(
|
||||
json.dumps({"commit": os.environ["GITHUB_SHA"]}) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
PY
|
||||
- name: Configure Pages
|
||||
uses: actions/configure-pages@983d7736d9b0ae728b81ab479565c72886d7745b # v5.0.0
|
||||
- name: Upload the website
|
||||
uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4.0.0
|
||||
with:
|
||||
path: website/_site
|
||||
|
||||
deploy:
|
||||
needs: prepare
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
actions: read
|
||||
pages: write
|
||||
id-token: write
|
||||
environment:
|
||||
name: github-pages
|
||||
url: ${{ steps.deployment.outputs.page_url }}
|
||||
steps:
|
||||
- name: Check out publishing configuration
|
||||
if: inputs.target == 'both'
|
||||
uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- name: Set up Node.js for Wrangler
|
||||
if: inputs.target == 'both'
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Download the verified website
|
||||
if: inputs.target == 'both'
|
||||
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
|
||||
with:
|
||||
name: github-pages
|
||||
path: ${{ runner.temp }}/website-archive
|
||||
- name: Extract the shared publishing artifact
|
||||
if: inputs.target == 'both'
|
||||
run: |
|
||||
mkdir -p "$RUNNER_TEMP/rea-website-public"
|
||||
tar -xf "$RUNNER_TEMP/website-archive/artifact.tar" -C "$RUNNER_TEMP/rea-website-public"
|
||||
- name: Publish the website to Cloudflare
|
||||
if: inputs.target == 'both'
|
||||
uses: cloudflare/wrangler-action@953926a2e2182532811c01a25e53647d93bf07c0 # v4.1.3
|
||||
with:
|
||||
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
|
||||
accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
|
||||
wranglerVersion: "4.149.0"
|
||||
packageManager: npm
|
||||
command: deploy --cwd website --env production --assets "${{ runner.temp }}/rea-website-public"
|
||||
- name: Publish to GitHub Pages
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4.0.5
|
||||
- name: Verify the selected published revisions
|
||||
env:
|
||||
PAGES_URL: ${{ steps.deployment.outputs.page_url }}
|
||||
PUBLISH_TARGET: ${{ inputs.target }}
|
||||
run: |
|
||||
python3 -u - <<'PY'
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from urllib.error import URLError
|
||||
from urllib.parse import urljoin
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
expected = os.environ["GITHUB_SHA"]
|
||||
pages_url = os.environ["PAGES_URL"]
|
||||
if not pages_url:
|
||||
sys.exit("GitHub Pages returned no published URL.")
|
||||
target = os.environ["PUBLISH_TARGET"]
|
||||
if target not in ("both", "pages"):
|
||||
sys.exit("Unknown publication target: " + target)
|
||||
hosts = {"GitHub Pages": pages_url}
|
||||
if target == "both":
|
||||
hosts["Cloudflare"] = "https://rea.tools/"
|
||||
pending = dict(hosts)
|
||||
for attempt in range(24):
|
||||
for name, base in list(pending.items()):
|
||||
url = urljoin(base, "deployment.json") + "?commit=" + expected
|
||||
try:
|
||||
request = Request(url, headers={
|
||||
"Cache-Control": "no-cache",
|
||||
"User-Agent": "REA-Website-Verifier/1.0 (+https://github.com/morluto/rea)",
|
||||
})
|
||||
with urlopen(request, timeout=10) as response:
|
||||
actual = json.load(response).get("commit")
|
||||
if actual == expected:
|
||||
print(f"{name} serves {expected}: {base}")
|
||||
del pending[name]
|
||||
else:
|
||||
print(f"Waiting for {name}: observed revision {actual}")
|
||||
except (URLError, TimeoutError, ValueError) as error:
|
||||
print(f"Waiting for {name}: {error}")
|
||||
if not pending:
|
||||
break
|
||||
if attempt < 23:
|
||||
time.sleep(5)
|
||||
if pending:
|
||||
sys.exit("Could not verify published revision on: " + ", ".join(pending))
|
||||
with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as summary:
|
||||
links = " and ".join(f"[{name}]({url})" for name, url in hosts.items())
|
||||
summary.write(f"Published commit `{expected}` to {links}.\n")
|
||||
PY
|
||||
+1
-41
@@ -6,7 +6,7 @@ __pycache__/
|
||||
# Distribution
|
||||
dist/
|
||||
build/
|
||||
node_modules
|
||||
node_modules/
|
||||
*.egg-info/
|
||||
*.egg
|
||||
|
||||
@@ -33,47 +33,7 @@ codedb.snapshot
|
||||
htmlcov/
|
||||
.coverage
|
||||
coverage/
|
||||
.stryker-tmp/
|
||||
reports/mutation/
|
||||
.vitest/
|
||||
.vitest-reports/
|
||||
test-results.xml
|
||||
test-timings.json
|
||||
.turbo/
|
||||
.cache/
|
||||
docs/.vitepress/cache/
|
||||
docs/.vitepress/dist/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
Thumbs.db
|
||||
|
||||
# Local reference sources and analysis artifacts
|
||||
/_reference/
|
||||
/experiments/analysis-efficiency/results/
|
||||
|
||||
# Generated website download
|
||||
/website/public/examples/notes-example.zip
|
||||
/website/public/deployment.json
|
||||
/website/public/sitemap.xml
|
||||
/website/public/assets/social-card.png
|
||||
/website/.venv/
|
||||
|
||||
# Local Cloudflare development state
|
||||
/website/.wrangler/
|
||||
/website/.dev.vars*
|
||||
/website/.env*
|
||||
!/website/.dev.vars.example
|
||||
!/website/.env*.example
|
||||
|
||||
# Derived catalogs, portable conformance projections, and packaged skill bundles
|
||||
# (regenerated by `npm run docs:generate`; never commit)
|
||||
/skills/
|
||||
/docs/public/product-catalog.json
|
||||
/docs/verification/managed-conformance-manifest.json
|
||||
/docs/verification/managed-conformance-ledger.json
|
||||
|
||||
# Complete static website output, including generated language editions.
|
||||
website/_site/
|
||||
# Temporary translation output prepared before replacing the complete site.
|
||||
website/.i18n-*/
|
||||
-15
@@ -1,15 +0,0 @@
|
||||
[submodule "third_party/jadx-headless-mcp"]
|
||||
path = third_party/jadx-headless-mcp
|
||||
url = https://github.com/1013503897/jadx-headless-mcp.git
|
||||
[submodule "third_party/binwalk"]
|
||||
path = third_party/binwalk
|
||||
url = https://github.com/ReFirmLabs/binwalk.git
|
||||
[submodule "third_party/unblob"]
|
||||
path = third_party/unblob
|
||||
url = https://github.com/onekey-sec/unblob.git
|
||||
[submodule "third_party/ghidra-nativeaot"]
|
||||
path = third_party/ghidra-nativeaot
|
||||
url = https://github.com/Washi1337/ghidra-nativeaot.git
|
||||
[submodule "third_party/wakaru"]
|
||||
path = third_party/wakaru
|
||||
url = https://github.com/pionxzh/wakaru.git
|
||||
+2
-1
@@ -1 +1,2 @@
|
||||
npx --no-install lint-staged
|
||||
npx lint-staged
|
||||
npm run typecheck
|
||||
@@ -1 +0,0 @@
|
||||
npm run check:fast
|
||||
+3
-7
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"threshold": 3.5,
|
||||
"threshold": 3,
|
||||
"reporters": ["consoleFull"],
|
||||
"ignore": [
|
||||
"node_modules/**",
|
||||
@@ -7,12 +7,8 @@
|
||||
"docs/**",
|
||||
".codex/**",
|
||||
".devin/**",
|
||||
"tests/fixtures/**",
|
||||
"third_party/**",
|
||||
"tests/conformance/**/*.c",
|
||||
"tests/conformance/**/*.m",
|
||||
"tests/conformance/**/*.swift",
|
||||
"tests/conformance/**/*.xib"
|
||||
"**/*.test.ts",
|
||||
"tests/fixtures/**"
|
||||
],
|
||||
"gitignore": true,
|
||||
"minLines": 5,
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
{
|
||||
"$schema": "./node_modules/oxfmt/configuration_schema.json",
|
||||
"printWidth": 80,
|
||||
"sortImports": false,
|
||||
"sortPackageJson": false,
|
||||
"ignorePatterns": [
|
||||
"dist/**",
|
||||
"build/**",
|
||||
"docs/api/**",
|
||||
"node_modules/**",
|
||||
"CHANGELOG.md",
|
||||
"package-lock.json",
|
||||
"_reference/**",
|
||||
"third_party/**"
|
||||
]
|
||||
}
|
||||
+7
-87
@@ -6,90 +6,22 @@
|
||||
"max-depth": ["warn", 4],
|
||||
"max-lines": [
|
||||
"warn",
|
||||
{
|
||||
"max": 500,
|
||||
"skipBlankLines": true,
|
||||
"skipComments": true
|
||||
}
|
||||
{ "max": 500, "skipBlankLines": true, "skipComments": true }
|
||||
],
|
||||
"max-lines-per-function": [
|
||||
"warn",
|
||||
{
|
||||
"max": 100,
|
||||
"skipBlankLines": true,
|
||||
"skipComments": true
|
||||
}
|
||||
{ "max": 100, "skipBlankLines": true, "skipComments": true }
|
||||
],
|
||||
"max-params": ["warn", 4],
|
||||
"no-var": "warn",
|
||||
"no-var-requires": "error",
|
||||
"typescript/no-import-type-side-effects": "error"
|
||||
"no-var-requires": "error"
|
||||
},
|
||||
"overrides": [
|
||||
{
|
||||
"files": [
|
||||
"scripts/electron-active-hook*.cjs",
|
||||
"tests/conformance/readiness/electron/*.js",
|
||||
"website/public/examples/notes-electron/*.js"
|
||||
],
|
||||
"files": ["tests/**/*.ts"],
|
||||
"rules": {
|
||||
"no-var-requires": "off"
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["tests/**/*.ts", "src/**/*.test.ts", "src/**/*.fixture.ts"],
|
||||
"rules": {
|
||||
"max-lines": "off",
|
||||
"max-lines-per-function": [
|
||||
"error",
|
||||
{
|
||||
"max": 150,
|
||||
"skipBlankLines": true,
|
||||
"skipComments": true
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["src/application/**/*.test.ts"],
|
||||
"rules": {
|
||||
"no-restricted-imports": [
|
||||
"error",
|
||||
{
|
||||
"patterns": [
|
||||
{
|
||||
"group": ["**/server/**", "**/cli.*", "**/main.*"],
|
||||
"message": "Service tests must use application ports rather than CLI or MCP entrypoints."
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"files": ["tests/composition/**/*.test.ts"],
|
||||
"rules": {
|
||||
"no-restricted-imports": [
|
||||
"error",
|
||||
{
|
||||
"patterns": [
|
||||
{
|
||||
"group": [
|
||||
"**/server/**",
|
||||
"**/cli.*",
|
||||
"**/main.*",
|
||||
"node:child_process",
|
||||
"node:net",
|
||||
"node:http",
|
||||
"node:https",
|
||||
"ws",
|
||||
"playwright"
|
||||
],
|
||||
"message": "Composition tests may wire provider-neutral sessions but must not invoke CLI, MCP, process, network, or browser boundaries."
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
"max-lines-per-function": "off",
|
||||
"max-lines": "off"
|
||||
}
|
||||
}
|
||||
],
|
||||
@@ -97,17 +29,5 @@
|
||||
"node": true,
|
||||
"es2023": true
|
||||
},
|
||||
"ignorePatterns": [
|
||||
"dist",
|
||||
"build",
|
||||
"node_modules",
|
||||
"docs/api",
|
||||
"coverage",
|
||||
".turbo",
|
||||
".cache",
|
||||
".codex",
|
||||
".devin",
|
||||
".factory",
|
||||
"third_party"
|
||||
]
|
||||
"ignorePatterns": ["dist", "node_modules", "docs/api", ".codex", ".devin"]
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
dist
|
||||
docs/api
|
||||
node_modules
|
||||
CHANGELOG.md
|
||||
package-lock.json
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "6.7.0"
|
||||
".": "1.0.0"
|
||||
}
|
||||
@@ -1,123 +1,74 @@
|
||||
# Repository Guidelines
|
||||
|
||||
## Product and Authority
|
||||
## Product Direction
|
||||
|
||||
REA is a local-only reverse-engineering tool with shared CLI/MCP workflows. Distinguish observations, derivations, inferences, and unknowns; preserve Evidence provenance and session semantics through both adapters. Provider support and setup live in [README.md](README.md#choosing-a-deep-analysis-provider) and its linked guides.
|
||||
REA exposes reverse-engineering tools through a CLI and MCP server. Hopper is its current binary-analysis provider; keep provider-specific code out of the domain and application layers.
|
||||
|
||||
Follow the [contribution scope and fail-closed integration rule](CONTRIBUTING.md#contribution-scope). Do not integrate capabilities while material scope, authority, side-effect, or required verification questions remain unresolved. For new providers or capabilities involving target execution, injection, mutation, or protection recovery, seek the required maintainer scope decision before substantial implementation for integration. Welcome early proposals, small exploratory prototypes, and draft PRs that help resolve questions; they do not authorize integration.
|
||||
Prioritize:
|
||||
|
||||
Ghidra uses ephemeral databases without modifying executable bytes or controlling a GUI; Windows Ghidra P0 has no mutation authority. IDA analysis is read-only; never save or close an attached GUI database.
|
||||
- tool results that distinguish observations, inferences, and unknowns;
|
||||
- equivalent behavior through the CLI and MCP;
|
||||
- additive, idempotent configuration with backups;
|
||||
- end-to-end tests for packaged artifacts and real Hopper claims.
|
||||
|
||||
Configuration changes must be additive and idempotent, with backups. Installers must not install or upgrade unrelated software, including Homebrew, Node.js, npm, Java, or Ghidra. Ghidra and IDA are bring-your-own. `rea setup` must show its planned changes and require approval before writing files or installing Hopper.
|
||||
Installers must not install or upgrade Homebrew, Node.js, npm, or other unrelated software. `rea setup` must print its planned changes and require approval before writing files or installing Hopper.
|
||||
|
||||
Preserve caller-selected inputs and local evidence, including output, URLs, paths, digests, and metadata. Redact transport authentication credentials and explicitly marked sensitive values; do not infer secrecy from names or text patterns. Do not persist the ambient environment merely because a child inherits it.
|
||||
## Project Structure & Module Organization
|
||||
|
||||
## Architecture and Cleanup
|
||||
REA is a layered ESM TypeScript application. Dependencies flow inward: `domain` ← `contracts` ← `hopper` ← `application` ← `server` ← adapters.
|
||||
|
||||
Dependencies flow inward: domain semantics and contracts, provider adapters, shared application workflows, then CLI/MCP adapters. Keep engine protocols and provider-specific code out of domain/application layers. See [docs/architecture.mermaid](docs/architecture.mermaid) when changing composition.
|
||||
See [docs/architecture.mermaid](docs/architecture.mermaid) for a visual architecture diagram.
|
||||
|
||||
Give each interpretation and resource a clear owner. Consumers must use that owner's parsed facts rather than reinterpret raw source. Preserve identity, omission, uncertainty, and source evidence across boundaries. Retain ownership of resources whose cleanup failed. Reuse `src/process/` supervision and identity primitives; a PID and executable path alone do not establish ownership after exit or reuse.
|
||||
- `scripts/rea.mjs`: executable dispatcher. Routes only bare `mcp` and `--mcp` to the production stdio server; Incur handles registration utilities and one-shot commands.
|
||||
- `src/main.ts`: MCP adapter. Parses config, wires the shared session runtime, starts stdio transport, and owns process-lifetime shutdown.
|
||||
- `src/cli.ts`: one-shot CLI adapter for setup, diagnostics, analysis, and decompilation.
|
||||
- `src/config.ts`: Zod-validated parsing of environment configuration into `AppConfig`.
|
||||
- `src/domain/`: pure, side-effect-free modules. `errors.ts` (tagged error algebra), `result.ts` (`Result`/`ok`/`err`), `hopperValues.ts` (boundary parsers for Hopper JSON), `symbolAnalysis.ts` (Swift/ObjC name parsing).
|
||||
- `src/contracts/`: caller-visible schemas for 33 direct, 10 enhanced, 5 native, 2 artifact, and 18 session tools; `enhancedInputs.ts` owns enhanced input parsing.
|
||||
- `src/hopper/`: Hopper launch and Unix-socket protocol mechanics. `BridgeLauncher.ts` spawns the Hopper app with the in-process bridge, `HopperClient.ts` correlates request/response over the socket with timeouts and cancellation, `protocol.ts` frames bridge messages.
|
||||
- `bridge/hopper_bridge.py`: runs inside Hopper and adapts declared operations to Hopper's public Python API. Hopper's bundled MCP server is not used.
|
||||
- `src/application/`: shared CLI/MCP session composition, setup and diagnostics, and enhanced workflows.
|
||||
- `src/server/`: MCP request translation. `createServer.ts` assembles the MCP server, `registerOfficialTools.ts`/`registerEnhancedTools.ts` register each tool set, `toolResult.ts` maps `Result` values to MCP content.
|
||||
- `tests/`: Vitest suite. `tests/fixtures/` holds the fake launcher and fake Hopper bridge used as production seams.
|
||||
- `scripts/verify-real-hopper.mjs`: real-Hopper end-to-end verifier.
|
||||
- `scripts/print-mcp-config.mjs`: prints an MCP server config with absolute paths filled in (`npm run config:print -- /path/to/binary`).
|
||||
|
||||
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
|
||||
## Build, Test, and Development Commands
|
||||
|
||||
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
|
||||
- `npm ci`: install exact lockfile dependencies.
|
||||
- `npm run build`: compile `src/` into `dist/`.
|
||||
- `npm test`: build, then run the Vitest suite once.
|
||||
- `npm run typecheck`: run strict TypeScript checks without emitting files.
|
||||
- `npm run lint`: apply oxlint rules (complexity, max-lines, unused vars, and TypeScript-specific checks).
|
||||
- `npm run lint:fix`: auto-fix oxlint violations where possible.
|
||||
- `npm run format:check`: verify Prettier formatting.
|
||||
- `npm run check`: run typecheck, lint, format:check, and tests.
|
||||
- `npm run knip`: detect unused files, dependencies, and exports.
|
||||
- `npm run jscpd`: detect duplicate code blocks.
|
||||
- `npm run scan:todos`: scan for TODO, FIXME, and HACK markers.
|
||||
- `npm run verify:hopper`: build and run the real-Hopper verifier with two distinct binaries.
|
||||
- `npm run verify:package`: pack and test the CLI, setup transaction, skill, and 68-tool target-free MCP server in an isolated environment.
|
||||
- `npm run docs:generate`: generate API documentation from JSDoc comments into `docs/api/` using TypeDoc.
|
||||
- `npm run config:print -- /path/to/binary`: print an MCP server config with absolute paths.
|
||||
- `HOPPER_TARGET_PATH=/path/to/binary npm start`: launch Hopper and run the built stdio MCP server.
|
||||
|
||||
Assess brittleness in touched code during implementation, debugging, review, and maintenance. Fix the producing representation or ownership rule before adding downstream repairs. Normalize once at the owning boundary using format-aware parsers. Remove superseded representations, aliases, wrappers, and redundant tests after verifying affected consumers. Introduce abstractions only when they remove observed complexity.
|
||||
Pre-commit hooks via Husky run `oxlint` on staged files before every commit. Use `npm run lint:fix` to auto-correct violations locally.
|
||||
|
||||
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
|
||||
## Configuration & Environment Variables
|
||||
|
||||
When addressing a review thread, inspect the producing boundary and affected consumers for brittleness. Complete directly related refactoring in the same change. Before resolving the thread, explain the root cause, structural fix, validation, and any remaining cleanup.
|
||||
- `HOPPER_TARGET_PATH` (optional): absolute initial binary or `.hop` target. Target-free MCP sessions use `open_binary` instead.
|
||||
- `HOPPER_LAUNCHER_PATH` (optional): override the Hopper executable path (defaults to `/Applications/Hopper Disassembler.app/Contents/MacOS/hopper`).
|
||||
- `HOPPER_TARGET_KIND` (optional, default `executable`): startup kind for `HOPPER_TARGET_PATH`; dynamic targets are classified from their paths and headers.
|
||||
- `HOPPER_LOADER_ARGS_JSON` (optional): JSON array overriding derived Hopper loader arguments for supported executable targets, e.g. `["-l","Mach-O","--aarch64"]`.
|
||||
|
||||
Parse unknown boundary values and model expected failures with the tagged error algebra and `Result`. Preserve partial facts and meaningful failure reasons. Keep identity and lookup values separate from display formatting; preserve source values and explicit unknowns when normalization loses information. Absolute paths, file URLs, and HTTP paths have different semantics.
|
||||
## Coding Style & Naming Conventions
|
||||
|
||||
Validate advertised JSON Schemas after SDK conversion against their declared dialect and actual clients. Bind handlers to named contracts with exact types; catalog ordering must never select an operation or schema.
|
||||
Use ESM TypeScript, two-space indentation, and Prettier defaults. Keep compiler strictness intact (`strict`, `exactOptionalPropertyTypes`, `noUncheckedIndexedAccess`, `verbatimModuleSyntax`). Use `camelCase` for values/functions, `PascalCase` for classes/types, and `UPPER_SNAKE_CASE` for constants. Parse unknown values at every MCP, environment, and subprocess boundary. Avoid `any`, unchecked casts, non-null assertions, import-time I/O, and floating promises. Exported APIs require concise JSDoc. Model expected failures with the tagged error algebra and `Result`, not broad exception wrappers.
|
||||
|
||||
## Performance
|
||||
## Testing Guidelines
|
||||
|
||||
Design analysis workflows and agent interfaces for efficiency from the first
|
||||
implementation. Correctness, useful answers, Evidence provenance, caller choices,
|
||||
compatibility and safe resource bounds remain required.
|
||||
Name tests `*.test.ts`. Use Vitest and production seams (`tests/fixtures/`) rather than module mocks. Domain tests assert pure behavior; adapter tests use the fake launcher/socket; MCP tests connect with the beta.3 client. Preserve the 33 direct, 10 enhanced, 5 native, 2 artifact, and 18 session tool inventory (68 total). Cover malformed input, cancellation, timeouts, process exit, concurrency, limits, and clean shutdown. Real Hopper claims cannot be replaced by mocks — use `npm run verify:hopper` with two distinct binaries for end-to-end verification.
|
||||
|
||||
Optimize in this order:
|
||||
## Commit & Pull Request Guidelines
|
||||
|
||||
1. Agent tokens and context. Return the smallest sufficient answer with
|
||||
necessary provenance, coverage and limitations. Use task selection,
|
||||
summaries, paging and retained Evidence where supported; honor requested
|
||||
detail and keep complete Evidence accessible. Count the whole task,
|
||||
including schemas, extra calls and retries.
|
||||
2. Analysis time. Reuse validated results and shared work. Cache keys must
|
||||
cover result-affecting inputs; define invalidation and lifetime. Validate
|
||||
freshness before reporting cached live state as current.
|
||||
3. CPU and memory. Avoid unnecessary work and copies; bound concurrency,
|
||||
retained state and serialization growth.
|
||||
|
||||
For changes affecting these costs, measure representative completed tasks and
|
||||
compare equivalent workflows before and after where available. Report token
|
||||
usage, elapsed time, CPU time and peak process-family memory. Separate cold
|
||||
and warm runs, label estimates, and state unmeasured costs. Fewer bytes alone
|
||||
do not establish fewer tokens or faster analysis.
|
||||
|
||||
## Development
|
||||
|
||||
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`.
|
||||
|
||||
- `npm ci`: install locked dependencies.
|
||||
- `npm run build:cached`: build the runtime and packaged skill.
|
||||
- `npm run test:local -- PATH...`: source tests without building.
|
||||
- `npm run test:focused -- PATH...`: exact tests with their required artifacts.
|
||||
- `npm run check:fast`: cached typecheck/lint and the pre-push check.
|
||||
- `npm run docs:check`: validate generated documentation.
|
||||
|
||||
Use relevant checks for the change. The complete local gate, `npm run check:pr`, is optional for broad changes or CI diagnosis. See [CONTRIBUTING.md](CONTRIBUTING.md) for development, generated-file ownership, and release conventions.
|
||||
|
||||
`docs/public/product-catalog.json`, `docs/verification/managed-conformance-*.json`, and `skills/` are ignored outputs. Edit source contracts and `.agents/skills/reverse-engineer-anything/`, then regenerate as needed. `.cache/mcp-tool-catalog.json` is test metadata; runtime builds and source checking do not consume it. Never commit binaries, provider project documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||
|
||||
## Tool and Test Changes
|
||||
|
||||
For tool design, follow [docs/tool-design.md](docs/tool-design.md): start from the analyst question, reuse existing contracts, preserve caller choices, and return useful evidence inline. Limits must follow real format, protocol, authority, or resource constraints. Account for representation expansion before allocation; derive completeness from examined coverage.
|
||||
|
||||
For test changes or provider claims, use [docs/testing.md](docs/testing.md). Prefer real public CLI/MCP workflows, then production-boundary integration, then producer goldens. Keep module tests for distinct semantics stronger workflows cannot reliably reproduce. Preserve malformed-input, cancellation, cleanup, permission, capacity, and target-identity coverage when pruning. Report unverified coverage explicitly; simulated providers do not establish real-provider behavior.
|
||||
|
||||
Use Conventional Commit subjects and PR titles. Mark breaking changes with `!` or a `BREAKING CHANGE:` footer. Describe behavior changes, relevant validation, and real-provider verification scope.
|
||||
|
||||
## Issue Reports
|
||||
|
||||
Follow [issue reporting guidance](CONTRIBUTING.md#reporting-issues) and the matching
|
||||
issue form. Keep reports concise and cite the smallest useful evidence.
|
||||
Distinguish observed results, source-derived reasoning, hypotheses, and proposed
|
||||
checks. Never claim runtime reproduction or measurements you did not perform.
|
||||
Use `NOT_ENOUGH_INFO` with an explanation for missing narrative answers or evidence.
|
||||
Incomplete reports, source-only findings, and early feature or design ideas are
|
||||
welcome when their limitations are explicit; a completed diagnosis or fix is not
|
||||
required to open an issue.
|
||||
|
||||
## Pull Request Readiness
|
||||
|
||||
Follow [human ownership and readiness](CONTRIBUTING.md#pull-request-ownership-and-readiness)
|
||||
when preparing a PR. AI-assisted contributions are welcome; a human must review
|
||||
and own the final diff, check the validation evidence and its limitations, and
|
||||
explicitly declare the PR ready for initial maintainer review.
|
||||
|
||||
Keep the PR body concise and current, with the concrete problem, observable
|
||||
outcome, approach, and evidence. Report only checks that actually ran as executed;
|
||||
distinguish source-derived conclusions, proposed checks, and unavailable coverage.
|
||||
Address actionable automated review findings and explain disagreements.
|
||||
|
||||
Leave the template's **Author readiness** declarations unchecked for the human
|
||||
owner to complete. Never attest to human review on their behalf. Keep the PR in
|
||||
draft until the human declares the PR initially ready. Later pushes do not
|
||||
invalidate that declaration or require another human sign-off. Keep the PR body
|
||||
and affected validation current as work changes. The readiness check validates
|
||||
the filled declaration; CI and automated review results do not establish that
|
||||
human review occurred, maintainer scope approval, or merge permission.
|
||||
|
||||
<!-- BEGIN:turborepo-agent-rules -->
|
||||
|
||||
# This is NOT the Turborepo you know
|
||||
|
||||
Turborepo configuration, task behavior, and CLI commands can vary between installed versions and may differ from your training data. Resolve the `turbo` package from this file's directory or relevant workspace; in monorepos, it may not be visible from the repository root. For example, run `node -p "require.resolve('turbo/package.json')"` from a workspace that depends on `turbo`.
|
||||
|
||||
Read `docs/README.md` inside that installed package first, then read the relevant pages from its `docs/` directory before changing Turborepo configuration or commands. Heed deprecation notices. These bundled docs match the installed package version and are available without network access.
|
||||
|
||||
This block is written and re-added by `turbo` before repository-scoped commands when an AI agent is detected. In the Turborepo source repository, its template is defined in `crates/turborepo-cli/src/cli/agent_guidance.rs`. Removing the managed block while updates are enabled means a later qualifying invocation will add it again. Set `"agentGuidance": false` in the root `turbo.json` or `turbo.jsonc` to opt out; this does not remove an existing block. Keep the block committed with your work to avoid an uncommitted change on the next agent invocation.
|
||||
<!-- END:turborepo-agent-rules -->
|
||||
Use Conventional Commit subjects because Release Please derives versions and changelogs from them. Examples: `feat: add historical source import`, `fix(process): stop timers after exit`, and `docs: update architecture`. Use `!` or a `BREAKING CHANGE:` footer for breaking changes. Pull request titles must follow the same format because squash merges use the title as the release commit. Pull requests should describe contract or behavior changes, list verification commands, link issues, and include sanitized MCP examples when schemas change. State whether real Hopper verification was performed. Never commit binaries, Hopper documents, credentials, `dist/`, `node_modules/`, or local planning artifacts (e.g. `.codex/`).
|
||||
-2494
File diff suppressed because it is too large.
Load diff
+52
-361
@@ -1,389 +1,80 @@
|
||||
# Contributing to REA
|
||||
|
||||
REA welcomes focused bug fixes, documentation improvements, tests, and reverse-engineering workflows. For large contract or architecture changes, explain the intended scope and link the relevant issue or design discussion.
|
||||
|
||||
## Choosing a contribution
|
||||
|
||||
Explain the concrete problem, keep related corrections together, and report
|
||||
verification and unavailable coverage. Contributions should justify their review
|
||||
and maintenance cost. Keep cosmetic polish with useful fixes; confusing or
|
||||
incorrect user instructions warrant standalone corrections. Collect other
|
||||
cosmetic-only suggestions in an issue rather than separate PRs.
|
||||
|
||||
For capability organization and provider composition, use the
|
||||
[architecture map](docs/architecture.mermaid). Production factories in
|
||||
`src/composition/` construct fresh providers without starting engines or acquiring
|
||||
targets; optional adapter failures must preserve successful peers. Run
|
||||
`npm run verify:test-discovery` after adding or moving tests. Resolved import
|
||||
boundaries are checked by `npm run verify:module-boundaries`, included in lint.
|
||||
|
||||
For analysis workflows and agent interfaces, follow the
|
||||
[performance priorities](AGENTS.md#performance). Explain the selected output
|
||||
scope, reuse/cache decisions, and relevant before/after measurements when these
|
||||
costs change.
|
||||
Use the [analysis optimization skill](.agents/skills/rea-analysis-optimization/SKILL.md)
|
||||
to locate provider and shared-workflow bottlenecks, preserve Evidence, and
|
||||
compare equivalent real tasks before claiming an improvement.
|
||||
|
||||
When adding or changing an MCP tool, follow the [tool design guide](docs/tool-design.md) and preserve the canonical contracts and generated catalog.
|
||||
|
||||
## Reporting issues
|
||||
|
||||
Use the issue form that fits the report and keep it concise. For bugs, include
|
||||
the actual result, the shortest reproduction you ran, the tested version and
|
||||
environment, and the smallest useful output, artifact, screenshot, or source
|
||||
reference. Ground expected behavior in documentation, source, or a known-good
|
||||
version when available. State observed impact and frequency; keep untested scope
|
||||
and suspected causes explicit.
|
||||
|
||||
Incomplete reports and intermittent failures are welcome. If a narrative answer
|
||||
or supporting evidence is unavailable, write `NOT_ENOUGH_INFO` and explain the
|
||||
gap. Clearly label source-only findings and proposed checks; do not claim runtime
|
||||
reproduction or results you did not observe. A root-cause analysis or fix is not
|
||||
required to report a problem. Documentation reports should identify the relevant
|
||||
passage or missing guidance and explain the correction or confusion.
|
||||
|
||||
Feature and design proposals can start with a problem and desired outcome.
|
||||
Examples, prior art, and measurements help when available; ideas without them
|
||||
are welcome. Maintainers can investigate reports and help resolve open questions.
|
||||
Human ownership and readiness declarations apply to PRs as described below.
|
||||
|
||||
## Pull request ownership and readiness
|
||||
|
||||
AI-assisted contributions are welcome. Every PR must have a human owner who
|
||||
takes responsibility for the complete change, including code, tests,
|
||||
documentation, and claims produced with AI assistance. The PR author is the
|
||||
owner unless another human is named in the PR and explicitly accepts ownership.
|
||||
These requirements also apply to maintainer-authored PRs.
|
||||
|
||||
Before requesting maintainer review, the human owner must:
|
||||
|
||||
- Review the final diff and relevant existing code. Be able to explain the
|
||||
problem, the chosen approach, how it fits REA, and meaningful edge cases.
|
||||
- Provide evidence that the change works. Summarize what was checked, the
|
||||
observed results, and meaningful gaps in the PR body. Use focused tests,
|
||||
representative CLI/MCP output, CI artifacts, screenshots, or other checks
|
||||
appropriate to the change. For fixes, show the original failure and why the
|
||||
regression coverage catches it when practical. Documentation-only changes
|
||||
can use source, link, or rendered-document checks. Label source-derived
|
||||
conclusions and proposed checks accurately; never report an unrun check as
|
||||
passing. Follow the [verification requirements](#development-feedback-and-pr-verification)
|
||||
for required checks and real-provider coverage.
|
||||
- Complete the **Author readiness** declaration in the PR template, explicitly
|
||||
stating that the PR is ready for initial review.
|
||||
|
||||
Keep the PR description concise and current. Explain the problem, observable
|
||||
outcome, approach, and useful evidence; link long logs or put them in optional
|
||||
details. Address actionable automated review findings and explain disagreements.
|
||||
Human ownership and review remain required when CI or an automated reviewer
|
||||
reports success. Readiness requests maintainer review; it does not grant scope
|
||||
approval or permission to merge.
|
||||
|
||||
Draft PRs may request early feedback with open questions and incomplete checks.
|
||||
Leave the readiness declaration unchecked until the human owner has completed
|
||||
it. The declaration records initial readiness; later pushes do not invalidate
|
||||
it or require another human sign-off. Keep the PR description and affected
|
||||
validation current as work changes. AI agents must leave human declarations to
|
||||
the human owner and must not attest to a human review on their behalf.
|
||||
|
||||
The `PR readiness` check reports pending for drafts and requires a named human
|
||||
owner and all three template declarations for non-drafts. It checks that the
|
||||
declaration is filled, not whether the code or evidence is correct. Existing PRs
|
||||
can copy the **Author readiness** section from the current template.
|
||||
Open PRs must have distinct head commits because GitHub shares commit statuses.
|
||||
|
||||
Maintainers enable merge enforcement after the workflow is merged: run
|
||||
`gh workflow run pr-readiness.yml -f pull_request_number=NUMBER` for existing
|
||||
open PRs, then add `PR readiness` from GitHub Actions to the branch's required
|
||||
status checks alongside its existing checks. Deploy the workflow to each
|
||||
protected target branch before requiring it there. The workflow reads PR
|
||||
metadata using trusted default-branch code and writes only a commit status.
|
||||
|
||||
Maintainers may return a PR to the author or close it when the problem is
|
||||
unsupported, validation is missing, or the owner cannot explain the change.
|
||||
Contributors are responsible for completing this work before requesting review.
|
||||
|
||||
## Contribution scope
|
||||
|
||||
REA focuses on software structure, runtime behavior, and evidence through
|
||||
general-purpose reverse-engineering workflows. Evaluate a proposal by its actual
|
||||
capabilities, side effects, defaults, and complete workflow. A tool name or a
|
||||
reference to vulnerability research, malware analysis, or CTFs does not determine
|
||||
whether it belongs in REA.
|
||||
|
||||
Supported contributions include format inspection, ordinary archive and standard
|
||||
compression extraction, disassembly, decompilation, cross-references, general
|
||||
static deobfuscation, vulnerability analysis, and evidence reporting. Static
|
||||
protection detection is distinct from defeating that protection. Security fixes
|
||||
and isolated regression tests remain in scope.
|
||||
|
||||
Runtime investigation needs explicit boundaries. Launching or attaching to a
|
||||
process, injecting instrumentation, running scripts, and changing memory or
|
||||
registers require review of target selection, execution and mutation authority,
|
||||
provenance, resource limits, cancellation, and cleanup. General Frida or debugger
|
||||
support is not inherently read-only, and existing support does not authorize
|
||||
every workflow built on it. General-purpose interfaces are not classified as
|
||||
dedicated bypass workflows solely because caller-provided code could be misused;
|
||||
their execution and mutation authority must still be reviewed.
|
||||
|
||||
REA does not integrate or officially distribute:
|
||||
|
||||
- Purpose-built workflows to defeat commercial packing or code-virtualization
|
||||
protections and automatically recover protected code.
|
||||
- Anti-debugging or anti-instrumentation evasion, or DRM, license, anti-cheat, or
|
||||
access-control bypass workflows.
|
||||
- Exploit or attack-payload automation, credential theft, covert persistence,
|
||||
unauthorized access, or bulk attack workflows.
|
||||
|
||||
These boundaries apply to first-party providers, adapters, plugins, bundled
|
||||
scripts, defaults, examples, and documentation. Moving a capability into an
|
||||
optional dependency or plugin does not exempt it. Review what the integration
|
||||
exposes, including upstream defaults and raw-command or script execution paths.
|
||||
Ordinary extraction and general static deobfuscation are not excluded merely
|
||||
because the input is packed or obfuscated; assess added bypass or execution
|
||||
behavior separately.
|
||||
|
||||
This is a product and maintenance boundary, not a judgment about a contributor's
|
||||
intentions or a declaration that all excluded research is unlawful. Contributors
|
||||
must have the rights and authorization needed for their work. A local-only design
|
||||
or an "authorized use only" disclaimer does not establish containment or settle
|
||||
legal and scope questions.
|
||||
|
||||
### Fail-closed integration
|
||||
|
||||
If scope fit, target or authorization boundaries, side effects, or the verification
|
||||
needed to establish those boundaries remain materially unresolved, **do not merge,
|
||||
enable, or include the capability in official packages or distributions**. Keep
|
||||
the proposal pending, narrow it, or decline it. Record unknowns explicitly; do not
|
||||
turn them into "none" or assumed safety. Passing CI, completed implementation,
|
||||
upstream popularity, or silence from maintainers is not scope approval.
|
||||
|
||||
For a new provider or a capability involving target execution, injection,
|
||||
mutation, or protection recovery, seek an explicit maintainer scope decision in
|
||||
the linked issue or design discussion before substantial implementation intended
|
||||
for integration. Small exploratory prototypes and draft PRs may help inform that
|
||||
decision; they do not authorize integration. The decision must cover the final
|
||||
proposed behavior; a scope expansion needs a new decision. Routine fixes and
|
||||
documentation changes within existing boundaries do not need a separate advance
|
||||
decision, but must not silently expand those boundaries.
|
||||
|
||||
### Starting a proposal
|
||||
|
||||
To open a feature request or design discussion, describe the problem and desired
|
||||
outcome or decision. Short answers and the forms' "Not sure yet" options are
|
||||
welcome. You do not need code, a complete design or test plan, legal analysis, or
|
||||
private proof-of-authorization documents to start a discussion. Share what you
|
||||
know and leave optional details blank when you need help.
|
||||
|
||||
Form selections guide triage; they are not proof of authorization or scope
|
||||
approval. Unselected effects must not be treated as established absence.
|
||||
Maintainers and implementers work together to resolve the following before
|
||||
integration:
|
||||
|
||||
- The concrete analyst question, requested inputs and outputs, and why existing
|
||||
REA operations are insufficient.
|
||||
- Why the workflow fits this scope, its non-goals, and any protection recovery,
|
||||
bypass, or attack automation it would expose.
|
||||
- The selected targets and every execution, injection, mutation, network, or
|
||||
persistence effect, including behavior inherited from upstream tools.
|
||||
- How the boundaries and outcome will be verified with self-owned or appropriately
|
||||
licensed samples; identify real CLI/MCP and provider coverage, cleanup and
|
||||
resource checks, and remaining unknowns. Do not publish private binaries,
|
||||
credentials, or sensitive target data; follow [SECURITY.md](SECURITY.md) for
|
||||
private vulnerability reports.
|
||||
|
||||
Mixed proposals should separate acceptable analysis capabilities from excluded
|
||||
workflows so each can be decided independently. An issue may discuss unresolved
|
||||
questions; opening an issue or PR does not authorize integration. This is a
|
||||
maintainer integration gate, not a per-call confirmation requirement or a sandbox
|
||||
guarantee.
|
||||
REA welcomes focused bug fixes, documentation improvements, tests, and reverse-engineering workflow enhancements. Open an issue before a large contract or architecture change so its scope can be agreed before implementation.
|
||||
|
||||
## Development setup
|
||||
|
||||
Use the development toolchain pinned in `.nvmrc` and
|
||||
`package.json#packageManager`. Run `nvm use` before installing dependencies;
|
||||
installed-package runtime support is documented in
|
||||
[installation](docs/installation.md#start-setup).
|
||||
REA development requires Node.js 24.18.x and npm 11.16.x. Real-Hopper verification additionally requires either macOS 12+ or an officially supported Linux host (Ubuntu 24.04+, Fedora 41+, or 64-bit Arch), an installed and activated Hopper application, and a desktop session. Run `nvm use` before installing dependencies.
|
||||
|
||||
```bash
|
||||
npm ci
|
||||
npm run check:fast
|
||||
npm run rebuild:native # only when the packaged PTY binary is incompatible
|
||||
npm run build
|
||||
npm test
|
||||
```
|
||||
|
||||
`npm ci` installs the exact dependencies and prepares the Husky hooks without
|
||||
building the project. Run `npm run build:cached` when you need the standalone
|
||||
CLI or MCP server; it compiles the runtime and bundles the authored skill.
|
||||
`npm run build` forces those tasks to rerun. Neither command generates test
|
||||
catalogs, documentation projections, or managed verification evidence. Turbo
|
||||
caches deterministic builds and static checks across Git worktrees. After a package, lockfile, or managed-skill version change, run
|
||||
`npm run metadata:generate` before building.
|
||||
Keep dependencies flowing inward through the existing domain, contracts, Hopper, application, server, and adapter layers. Parse unknown values at process and protocol boundaries, model expected failures with `Result`, and preserve the 33 direct, 10 enhanced, 5 native, 2 artifact, and 18 session tools (68 total) unless a deliberate contract change updates every verifier and snapshot.
|
||||
|
||||
Repository invariants live in [AGENTS.md](AGENTS.md). Keep provider-specific
|
||||
protocols in adapters and shared CLI/MCP workflows in the application layer.
|
||||
|
||||
### Windows source checkouts
|
||||
|
||||
The published npm package includes the Windows x64 native addon and its
|
||||
manifest. A source checkout does not: `native/windows/build/` is ignored by
|
||||
Git, and neither `npm ci` nor `npm run build:cached` (or `npm run build`)
|
||||
produces `rea-windows-x64.node` or `manifest.json` in that directory.
|
||||
Windows workflows and tests that require native controls need these artifacts;
|
||||
this includes `run-exclusive` commands, which use the native Job Object to keep
|
||||
the artifact lock until every command descendant has exited. Without the
|
||||
artifacts, those controls are unavailable even after a successful runtime
|
||||
build.
|
||||
|
||||
Follow the [Windows native build guide](native/windows/README.md#build) before
|
||||
running those workflows from source. The artifact build requires Linux,
|
||||
MinGW-w64, and `tar`; the build script does not run on Windows. See that guide
|
||||
for the build and artifact-verification commands and the separate real Windows
|
||||
verification prerequisites. Keep the generated addon and manifest uncommitted.
|
||||
|
||||
## Documentation website
|
||||
|
||||
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
|
||||
for live editing, `npm run docs:build` to check the production build and
|
||||
links, and `npm run docs:preview` to preview that build at `/rea/`.
|
||||
|
||||
Site navigation lives in `docs/.vitepress/config.ts`. Keep links to guides
|
||||
relative so they work on GitHub and the website; link to repository files
|
||||
outside `docs/` using their full GitHub URLs. Generated reference documents
|
||||
use `npm run docs:generate`; `docs:build` generates them before building the website.
|
||||
|
||||
Pull requests run `npm run docs:check`. `.github/workflows/pages.yml` is a
|
||||
manual VitePress build and does not publish. The public site at
|
||||
<https://rea.tools/> serves `website/public/` through Cloudflare Workers. See
|
||||
the [website README](website/README.md#cloudflare-workers) for previews and
|
||||
manual deployments.
|
||||
|
||||
The separate GitHub Pages host uses only the manual
|
||||
`.github/workflows/website-pages.yml` workflow on `main`. The repository's
|
||||
**Settings → Pages → Build and deployment → Source** must be set to
|
||||
**GitHub Actions** before deploying there.
|
||||
|
||||
### Translation corrections
|
||||
|
||||
README editions and website editions have separate translation sources:
|
||||
|
||||
- Find the language's `readme` path in the
|
||||
[language inventory](website/locales/languages.json) and edit that file.
|
||||
Locale codes and README suffixes can differ. Use the
|
||||
[English README](README.md) to check commands, requirements, and analysis claims.
|
||||
- Edit `msgstr` values in `website/locales/<language>/messages.po` for website
|
||||
text. Follow the [website translation guide](website/locales/README.md) for
|
||||
the Python environment, catalog validation, and local preview. Preserve source
|
||||
strings, contexts, placeholders, commands, paths, and recorded evidence.
|
||||
|
||||
Before submitting a website translation, run the catalog and website checks in
|
||||
that guide. Translated HTML in `website/_site/` is generated; edit the PO catalog
|
||||
rather than the generated pages.
|
||||
|
||||
## Development feedback and PR verification
|
||||
|
||||
For routine merges, wait for the required checks to pass for the current PR
|
||||
revision. Administrators retain an override for deliberate maintainer recovery.
|
||||
If `main` is broken, prioritize a focused repair before updating affected PRs.
|
||||
|
||||
Select checks for the behavior changed. For a source edit:
|
||||
Before submitting a pull request, run:
|
||||
|
||||
```bash
|
||||
npm run test:focused -- src/config.test.ts
|
||||
npm run check:fast
|
||||
npm run check
|
||||
npm run verify:package
|
||||
npm pack --dry-run
|
||||
```
|
||||
|
||||
`test:local` runs source tests without building. `check:changed` combines cached
|
||||
static checks with source tests affected since the merge base with `origin/main`.
|
||||
Import-graph selection can miss runtime registration, bridges, and generated data;
|
||||
select relevant boundary tests explicitly. See [testing](docs/testing.md#developer-commands)
|
||||
for commands, test placement, pruning, and real-provider prerequisites.
|
||||
`npm install` prepares the Husky pre-commit hook. Commits format and lint staged
|
||||
files with lint-staged, then typecheck the complete project. The normal test and
|
||||
CI coverage runs enforce the thresholds in `vitest.config.ts`; `npm run
|
||||
lint:dead` rejects unused files, exports, and dependencies.
|
||||
|
||||
Before requesting review, run relevant tests and `npm run check`; add
|
||||
`npm run docs:check` when contracts or generated metadata change. Record the
|
||||
checks and unavailable coverage. CI owns the full deterministic suite and
|
||||
coverage. `npm run check:pr` is an optional full local gate for broad changes or
|
||||
CI diagnosis. Packaging/setup/distribution changes additionally require
|
||||
`npm run verify:package` and `npm pack --dry-run`; provider changes require the
|
||||
matching real-provider lane. State which host/provider workflows ran.
|
||||
Set `REA_LOG_LEVEL` to `trace`, `debug`, `info`, `warn`, `error`, `fatal`, or
|
||||
`silent` to control structured JSON diagnostics. MCP mode defaults to `info` and
|
||||
always writes logs to stderr so the stdio protocol remains intact. One-shot CLI
|
||||
logging is opt-in and writes to stdout when a level is configured, preserving
|
||||
machine-readable command output by default. Request arguments, bridge
|
||||
authentication tokens, and environment data are redacted.
|
||||
|
||||
For CI/test correctness, drift, or performance investigations, use the
|
||||
[`rea-ci-tests-audit` contributor skill](.agents/skills/rea-ci-tests-audit/SKILL.md).
|
||||
It guides reproduction and comparable measurements before changing checks.
|
||||
Changes that claim real Hopper behavior must also be tested against two distinct binaries:
|
||||
|
||||
`check:pr` reuses build and static-check outputs while executing the test suite
|
||||
fresh, including the fast Python producer checks. `docs:check` also runs the
|
||||
onboarding example consistency suite for README-only changes.
|
||||
```bash
|
||||
HOPPER_TARGET_PATH=/path/to/target-a \
|
||||
HOPPER_SECOND_TARGET_PATH=/path/to/distinct-target-b \
|
||||
npm run verify:hopper
|
||||
```
|
||||
|
||||
Pre-commit formats and lints staged files; pre-push runs `check:fast`.
|
||||
On a self-hosted Linux runner with `DISPLAY` or `WAYLAND_DISPLAY`, use:
|
||||
|
||||
### Generated files and command ownership
|
||||
```bash
|
||||
HOPPER_TARGET_PATH=/path/to/target-a \
|
||||
HOPPER_SECOND_TARGET_PATH=/path/to/distinct-target-b \
|
||||
npm run verify:hopper:linux
|
||||
```
|
||||
|
||||
`docs:generate` prepares documentation and portable managed evidence;
|
||||
`evidence:generate` prepares only the managed commitment and its runtime/skill
|
||||
dependencies. The ignored outputs are `docs/public/product-catalog.json`,
|
||||
`docs/verification/managed-conformance-*.json`, and `skills/`. Edit authored
|
||||
skill instructions in `.agents/skills/reverse-engineer-anything/`, where Skills
|
||||
can discover the public bundle directly from a clean checkout. `verify:package`
|
||||
installs that bundle with Skills 1.7.2 using default repository-root discovery
|
||||
and checks every installed file, alongside the packaged setup checks.
|
||||
CI validates generated outputs and retains them as artifacts; do not commit
|
||||
them. Reviewed source metadata, including
|
||||
`src/generatedPackageMetadata.ts` and `docs/error-contract.schema.json`, remains
|
||||
tracked and checked for freshness.
|
||||
The macOS and Linux real-Hopper workflows remain separate so a successful mock or package test cannot be reported as platform-runtime proof. Pull requests changing setup, launch, bridge, or Hopper behavior must state which real workflows ran and why either workflow was unavailable.
|
||||
|
||||
The packaged skill contains instructions and inventory metadata, not runtime
|
||||
schema hashes. Its conformance commitment covers that exact bundle and portable
|
||||
verification; it does not attest optional real-provider runs. Schema-only changes
|
||||
should not alter that commitment. Doctor and `binary_session` expose runtime
|
||||
schema identity.
|
||||
|
||||
Build/generation commands hold a SQLite transaction around the entire Turbo
|
||||
graph, including cache restoration. `.cache/rea-command-locks/` databases identify
|
||||
locks, not stale files; never remove them while commands run. POSIX cancellation
|
||||
retains process-group ownership until settlement; Windows waits for completion.
|
||||
Real-provider execution is uncached.
|
||||
|
||||
### Temporary files and diagnostics
|
||||
|
||||
Tests use `createTestTempDirectory` from `tests/fixtures/temporaryDirectory.ts`
|
||||
for exact-path, awaited cleanup, including failures and timeouts.
|
||||
`npm run verify:test-temp-hygiene` runs the suite under a fresh `TMPDIR` and
|
||||
rejects remaining REA-owned paths. Never glob-clean shared `/tmp/rea-*` content.
|
||||
|
||||
`REA_LOG_LEVEL` selects `trace`, `debug`, `info`, `warn`, `error`, `fatal`, or
|
||||
`silent`. Logs go to stderr: MCP defaults to `info`, and one-shot CLI logging
|
||||
is opt-in, so stdout carries only protocol messages and command results.
|
||||
Request arguments, bridge tokens, and environment data are redacted from
|
||||
diagnostic logs.
|
||||
|
||||
## Real Hopper changes
|
||||
|
||||
Run `npm run verify:hopper` on a supported macOS host or
|
||||
`npm run verify:hopper:linux` on the supported Linux demo runner. These build
|
||||
native fixtures and verify the installed engine through CLI/MCP; package or
|
||||
simulated-provider checks do not establish engine behavior. See
|
||||
[real-toolchain verification](docs/testing.md#end-to-end-integration-and-golden-evidence)
|
||||
for prerequisites and scope, and [Hopper setup](docs/installation.md#hopper)
|
||||
for supported hosts and unattended startup.
|
||||
|
||||
Use the default `build/conformance/manifest.json` unless explicitly verifying
|
||||
another source-built manifest with `REA_HOPPER_CONFORMANCE_MANIFEST_PATH`.
|
||||
Generated fixtures and manifests remain ignored.
|
||||
Describe the behavior change and verification performed in the pull request. Never commit binaries, Hopper documents, credentials, `dist/`, `node_modules/`, or local planning artifacts.
|
||||
|
||||
## Maintainer release checklist
|
||||
|
||||
Use the [release guide](docs/releasing.md). Main pushes automatically refresh a
|
||||
Release Please PR with the next version and changelog. Review its migration
|
||||
notes, wait for the current CI checks to pass, and merge it into main to publish.
|
||||
Run the full checks, isolated package verifier, package dry run, and two-target real-Hopper verifier described above. Build a local tarball and exercise the executable through the package boundary:
|
||||
|
||||
The Release workflow validates and tags that merge; both publishers build its
|
||||
exact SHA. Ordinary main pushes cannot publish packages. An optional manual
|
||||
`release/VERSION` checkpoint retains explicit preparation and publication.
|
||||
```bash
|
||||
npm pack
|
||||
npm exec --yes --package ./morluto-rea-0.1.0.tgz -- rea --help
|
||||
```
|
||||
|
||||
For grouped release notes with verified contributor thanks, use the repository's
|
||||
`rea-changelog-update` skill in `.agents/skills/rea-changelog-update/`. Its inventory,
|
||||
credit checks, and finalization process are described in the release guide.
|
||||
Publish the public package:
|
||||
|
||||
Keep new implementation commits on main for the next release. The workflow
|
||||
owns packaged-artifact verification, npm publication, the published CLI/MCP
|
||||
canary, and MCP Registry publication. Sync manual checkpoint metadata back to
|
||||
main after publication; see the guide for recovery and verification.
|
||||
```bash
|
||||
npm publish --access public
|
||||
```
|
||||
|
||||
After npm registry propagation, verify the published CLI and connect a beta.3 MCP client to the published server to confirm all 68 tools:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents --help
|
||||
npx -y rea-agents doctor
|
||||
npx -y rea-agents setup --yes
|
||||
npx -y rea-agents mcp
|
||||
```
|
||||
-136
@@ -1,136 +0,0 @@
|
||||
# Disclaimer
|
||||
|
||||
## Responsible use
|
||||
|
||||
REA is built for lawful analysis: security research, malware and incident
|
||||
analysis, vulnerability research and coordinated disclosure, CTFs and education,
|
||||
interoperability, software preservation, and debugging or auditing software you
|
||||
own or are authorized to examine.
|
||||
|
||||
The REA project does not support, and will not help with:
|
||||
|
||||
- circumventing DRM, license checks, anti-cheat systems or other technical
|
||||
protection measures;
|
||||
- developing game cheats or tampering with online services;
|
||||
- bypassing authentication, bot protection or access controls on systems you are
|
||||
not authorized to test;
|
||||
- copying proprietary software or reproducing its protected code and assets;
|
||||
- evading the usage policies or safety measures of AI model providers;
|
||||
- reselling or sharing AI accounts or API access.
|
||||
|
||||
You are responsible for obtaining any required authorization and for complying
|
||||
with applicable laws, license agreements and terms of service. Reverse
|
||||
engineering may be restricted in your jurisdiction or by the license of the
|
||||
software you analyze.
|
||||
|
||||
Requests for help with the uses above will be closed without response in issues
|
||||
and discussions, and may result in removal from the REA community.
|
||||
|
||||
## User responsibility
|
||||
|
||||
Do not use REA for unauthorized access or exploitation, credential theft,
|
||||
malicious deployment of malware, unlawful collection or disclosure of data,
|
||||
unauthorized service disruption, or unlawful circumvention of access controls or
|
||||
licensing restrictions. Legitimate security research and malware analysis must
|
||||
remain within the rights and permissions applicable to the work.
|
||||
|
||||
You are responsible for your use of REA, including commands, configurations,
|
||||
scripts, and actions you direct an AI agent to perform. Establish the legal
|
||||
authority and scope required for the targets, accounts, devices, software, data,
|
||||
and operations involved. Comply with applicable laws, contracts,
|
||||
intellectual-property rights, privacy obligations, and tool licenses. Public
|
||||
availability of software, possession of a file, or access to a debugging
|
||||
endpoint does not by itself establish permission for every operation.
|
||||
|
||||
## Runtime, network, and device operations
|
||||
|
||||
Some REA workflows execute or observe programs, attach to runtime or browser
|
||||
endpoints, capture network traffic, interact with applications, or operate on
|
||||
connected devices. These actions can affect application state, files, accounts,
|
||||
devices, and services. Authorization to inspect an artifact does not necessarily
|
||||
authorize execution, interception, device changes, or interaction with another
|
||||
party's system.
|
||||
|
||||
Review the operations you request, including agent-generated plans and scripts,
|
||||
and use an appropriate environment for untrusted targets. You are responsible
|
||||
for protecting credentials, personal data, captured traffic, and investigation
|
||||
artifacts, and for obtaining any permission required to collect, retain,
|
||||
process, or share them. Local operation does not establish authorization or
|
||||
guarantee that executed targets and third-party tools have no external effects.
|
||||
|
||||
## Analysis results and generated output
|
||||
|
||||
Decompilation, reconstruction, heuristics, and AI-assisted analysis may be
|
||||
incomplete or incorrect. Results do not guarantee accuracy, completeness,
|
||||
security, compatibility, or legal compliance. Review the underlying evidence and
|
||||
verify consequential conclusions before relying on them.
|
||||
|
||||
Obtaining source-like output or an analysis report does not grant rights to the
|
||||
original program, its data, or the generated output. Check the target's rights
|
||||
and the producing tool's applicable terms before publishing, sharing, or using
|
||||
results in another product or service.
|
||||
|
||||
## Third-party tools and licenses
|
||||
|
||||
Third-party components and analysis tools retain their own licenses, copyright
|
||||
notices, and usage conditions. Consult
|
||||
[Third-party notices](THIRD_PARTY_NOTICES.md) for their roles, licensing
|
||||
records, and preserved upstream texts.
|
||||
|
||||
You must legally obtain IDA Pro and Hex-Rays decompilers, Hopper, JEB, and any
|
||||
other proprietary tools, together with the rights required for your use. REA
|
||||
provides no commercial-tool license, key, or activation entitlement. Permitted
|
||||
trials and demos remain subject to vendor terms. An open-source bridge or
|
||||
accessible API does not grant additional user, machine, feature, automation,
|
||||
hosting, or output-sharing rights. Cracked copies, stolen keys, unauthorized key
|
||||
sharing, and bypassing license restrictions are not authorized by REA.
|
||||
|
||||
## Disclaimer of warranty
|
||||
|
||||
REA is provided **AS IS**, without express or implied warranties, including
|
||||
merchantability, fitness for a particular purpose, and noninfringement, as set
|
||||
out in the [MIT License](LICENSE). You assume the risks of using the software
|
||||
and deciding whether it is suitable for your task. No project description,
|
||||
example, analysis result, or third-party integration guarantees a particular
|
||||
outcome or establishes that your intended use is lawful or safe.
|
||||
|
||||
## Limitation of liability
|
||||
|
||||
The warranty and liability provisions of the [MIT License](LICENSE) govern REA's
|
||||
authors and copyright holders, subject to applicable law. That license disclaims
|
||||
liability for claims, damages, and other liability arising from or in connection
|
||||
with the software or its use, whether in contract, tort, or otherwise. This
|
||||
document does not expand those provisions or override rights and obligations
|
||||
that cannot be excluded under applicable law.
|
||||
|
||||
## Disclaimer of endorsement
|
||||
|
||||
References to third-party projects, contributors, products, services, and
|
||||
trademarks identify their work and REA's integrations. Credits and thanks do not
|
||||
imply affiliation, sponsorship, approval, or endorsement by the named parties,
|
||||
including the National Security Agency and commercial tool vendors.
|
||||
Redistribution, bundling, or hosting by a third party does not by itself imply
|
||||
affiliation with or endorsement by the REA project.
|
||||
|
||||
## Relationship to software licenses
|
||||
|
||||
This document explains REA's intended use, user responsibilities, and community
|
||||
support boundaries. It does not amend the MIT license, impose additional
|
||||
copyright-license conditions, replace upstream licenses, or grant rights to
|
||||
third-party software or targets. The applicable software licenses and agreements
|
||||
retain their own terms.
|
||||
|
||||
## Versions, distribution channels, and platforms
|
||||
|
||||
The project's position on lawful use and non-endorsement of illegal or
|
||||
unauthorized activity covers all REA versions, distribution forms, and
|
||||
platforms. This includes historical releases and Git commits, npm packages,
|
||||
bundled or separately distributed REA skills, source checkouts, release
|
||||
archives, and redistributed copies, regardless of installation method, agent
|
||||
client, operating system, or platform. Using an older version or another
|
||||
distribution channel does not imply project endorsement or remove obligations
|
||||
under applicable law or to respect third-party rights.
|
||||
|
||||
Each copy remains subject to the licenses accompanying that copy. This statement
|
||||
does not retroactively amend existing license grants, add license conditions, or
|
||||
expand historical warranty or liability exclusions.
|
||||
@@ -1,416 +1,454 @@
|
||||
<div align="center">
|
||||
|
||||
**English** · [简体中文](docs/readme/README_zh.md) · [繁體中文](docs/readme/README_zh-TW.md) · [日本語](docs/readme/README_ja.md) · [한국어](docs/readme/README_ko.md) · [Türkçe](docs/readme/README_tr.md) · [Русский](docs/readme/README_ru.md) · [Tiếng Việt](docs/readme/README_vi.md) · [ไทย](docs/readme/README_th.md) · [Deutsch](docs/readme/README_de.md) · [Español](docs/readme/README_es.md) · [Français](docs/readme/README_fr.md) · [Українська](docs/readme/README_uk.md) · [Polski](docs/readme/README_pl.md) · [Português (Brasil)](docs/readme/README_pt-BR.md) · [العربية](docs/readme/README_ar.md) · [فارسی](docs/readme/README_fa.md) · [Bahasa Indonesia](docs/readme/README_id.md) · [Italiano](docs/readme/README_it.md)
|
||||
· [Ελληνικά](docs/readme/README_el.md) · [മലയാളം](docs/readme/README_ml.md)
|
||||
**English** · [简体中文](README_zh.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [العربية](README_ar.md)
|
||||
|
||||
# REA: Reverse Engineer Anything
|
||||
|
||||
### One MCP for reverse engineering across binaries, applications, and runtime behavior.
|
||||
### One CLI and MCP server for coding agents to reverse engineer anything
|
||||
|
||||
**See a feature you like. Understand how it works, down to the binary level.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](docs/mcp-contracts.md#generated-catalog)
|
||||
[](#what-you-can-analyze)
|
||||
[](https://skills.sh/morluto/rea/reverse-engineer-anything)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](#68-tools-for-investigation)
|
||||
[](https://nodejs.org/)
|
||||
[](LICENSE)
|
||||
[](https://discord.gg/GkcryMnJDM)
|
||||
|
||||
<a href="https://trendshift.io/repositories/82054?utm_source=repository-badge&utm_medium=badge&utm_campaign=badge-repository-82054" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/82054" alt="morluto%2Frea | Trendshift" width="250" height="55"/></a>
|
||||
|
||||
**[Website](https://rea.tools/) · [Guides](https://rea.tools/guides/) · [Showcases](https://rea.tools/showcase/)**
|
||||
|
||||
[Quick start](#quick-start) · [How REA works](#how-rea-works) · [What you can analyze](#what-you-can-analyze) · [Showcases](#showcases) · [FAQ](#faq) · [Documentation](#documentation)
|
||||
|
||||
<code>npx rea-agents setup</code>
|
||||
|
||||
</div>
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Get REA from [morluto/rea](https://github.com/morluto/rea),
|
||||
> [rea.tools](https://rea.tools/), or the [rea-agents npm package](https://www.npmjs.com/package/rea-agents).
|
||||
> Follow the installation and release links from these official sources. A similar
|
||||
> repository name or copied README does not establish that a download is official.
|
||||
|
||||
<div align="center">
|
||||
[Quick start](#quick-start) · [Current status](#current-status) · [Investigation model](#the-investigation-model) · [68 tools](#68-tools-for-investigation) · [Roadmap](#roadmap) · [How it works](#how-it-works)
|
||||
|
||||
<br />
|
||||
|
||||
<img src="docs/assets/rea-hopper-analysis.png" alt="REA launching its analysis bridge inside Hopper while inspecting a native binary" width="1200" />
|
||||
|
||||
<br /><br />
|
||||
|
||||
<table aria-label="REA community">
|
||||
<tr>
|
||||
<td align="center" width="360">
|
||||
<a href="https://discord.gg/GkcryMnJDM">
|
||||
<img src="docs/assets/discord.svg" height="42" alt="Discord" /><br />
|
||||
<strong>Join the Reverse Engineering Community</strong>
|
||||
</a><br />
|
||||
<sub>Discord · Q&A · Show and Tell</sub>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<br />
|
||||
<code>npm install --global rea-agents && rea setup</code>
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
See a feature in an app that you want in your own product? Ask your agent to investigate it with REA. It can inspect the app without its source code, explain how the feature works, show the evidence, and build a version for your project.
|
||||
See a feature in an app that you want in your own product? Give the app to your coding agent—even without its source code. With REA, the agent can investigate the feature, explain how it works, show its evidence, and build a version adapted to your stack and requirements.
|
||||
|
||||
REA connects your agent to tools for inspecting native binaries, JavaScript and Electron apps, .NET assemblies, and websites. You can also use the same tools from your terminal. Analysis runs locally, and results include the evidence and limitations behind each conclusion.
|
||||
REA gives agents one consistent way to investigate software. Today that includes deep native analysis through Hopper, complete function dossiers, reproducible Evidence v2 records, and controlled process capture. The longer-term toolkit extends the same agent workflow to packaged apps, JavaScript bundles, websites, APIs, protocols, mobile artifacts, firmware, runtime behavior, and differences between versions.
|
||||
|
||||
Setup registers REA with your agent and installs matching workflow instructions. Native analysis can use an existing Hopper, Ghidra, or IDA installation; setup can optionally install Hopper with approval. Static JavaScript analysis needs no native analysis engine.
|
||||
Reverse engineering normally makes the operator choose a tool, learn its API, move evidence between programs, and decide what to inspect next. REA gives that work to the agent through commands, skills, structured results, and repeatable investigation workflows.
|
||||
|
||||
> **[Visit the REA website](https://rea.tools/)** for setup instructions, illustrated guides, and real case studies.
|
||||
## Just ask your agent
|
||||
|
||||
Install the REA skill:
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
Then ask:
|
||||
|
||||
```text
|
||||
Use REA to understand how search works in the Notes app, show me the
|
||||
evidence, and build a similar feature for my project.
|
||||
```
|
||||
|
||||
Notes is only an example. Name any app you want to understand, or ask the agent to start with an overview.
|
||||
|
||||
## The investigation model
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td width="33%" valign="top">
|
||||
<strong>Decompile</strong><br /><br />
|
||||
Open an app and recover readable code, strings, names, and other clues about how it works.
|
||||
</td>
|
||||
<td width="33%" valign="top">
|
||||
<strong>Understand</strong><br /><br />
|
||||
Follow the code from one part of the app to another until the agent can explain how a feature actually works.
|
||||
</td>
|
||||
<td width="33%" valign="top">
|
||||
<strong>Recreate</strong><br /><br />
|
||||
Turn what the agent learned into a feature for your own product, adapted to your stack, interface, and requirements.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
REA shows how it reached its conclusions. It does not claim to recover original source code or automatically clone an application.
|
||||
|
||||
## Why REA
|
||||
|
||||
| | |
|
||||
| ------------------------ | ------------------------------------------------------------------------------------ |
|
||||
| **Built for agents** | Ask what an app does and let your agent inspect it instead of guessing. |
|
||||
| **CLI and MCP** | Run the same reverse-engineering capabilities from your terminal or coding agent. |
|
||||
| **Complexity handled** | REA installs and manages the reverse-engineering tools behind the scenes. |
|
||||
| **From insight to code** | Understand a feature, then build your own version in the same coding session. |
|
||||
| **Local by design** | Analysis runs on your Mac. REA does not upload the app to a hosted analysis service. |
|
||||
| **Keeps context** | Investigate several apps without starting over for every question. |
|
||||
|
||||
## Quick start
|
||||
|
||||
On Linux, Ghidra, JADX, firmware tool launches and static JavaScript/Electron
|
||||
analysis require a procps-compatible `ps` on REA's `PATH` for process ownership
|
||||
checks. Recovery steps are in `docs/installation.md` under
|
||||
"Linux owned-process prerequisite".
|
||||
|
||||
### Set up your agent
|
||||
|
||||
With Node.js and npm installed, run:
|
||||
### Install the CLI — recommended
|
||||
|
||||
```bash
|
||||
npx rea-agents setup
|
||||
npm install --global rea-agents
|
||||
rea setup
|
||||
```
|
||||
|
||||
Choose your agents, review the proposed changes, and approve them. Setup adds
|
||||
REA's MCP server and matching workflow instructions, with backups of existing
|
||||
configuration. Restart your agent afterward.
|
||||
Installing the CLI does not update Homebrew, Node.js, npm, Hopper, or coding-agent configuration. `rea setup` detects what is already present, prints every proposed change, and asks before applying it.
|
||||
|
||||
Setup supports Claude Code, Codex, Cursor, Gemini CLI, Grok Build and
|
||||
[other agents](docs/installation.md#supported-agents). See
|
||||
[installation and setup](docs/installation.md) for provider configuration and
|
||||
manual MCP registration.
|
||||
REA detects Claude Code, Claude Desktop, Codex, Cursor, Gemini CLI, Windsurf, and Devin. Registrations are additive, backup-first, and read back after writing. You can safely rerun setup.
|
||||
|
||||
MCP uses compact input schemas by default to reduce context overhead. Set
|
||||
`REA_MCP_INPUT_SCHEMA_PROFILE=full` in your MCP registration when you need the
|
||||
complete advertised field guidance and nested validation details. Both profiles
|
||||
keep the full tool set and canonical server validation. See
|
||||
"Compact input schema profile" in the MCP contracts guide.
|
||||
An optional curl wrapper installs the same CLI package and starts setup only when a terminal is available:
|
||||
|
||||
### Ask your agent
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash
|
||||
```
|
||||
|
||||
Pass installer options after `bash -s --`, for example `--dry-run`, `--no-setup`, or `--version 1.0.0`. The curl wrapper never installs prerequisites or configures integrations itself. See [Installation and setup](docs/installation.md) for its exact mutation boundary.
|
||||
|
||||
### With a coding agent — recommended
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
Ask your agent to set up REA. It will check your Mac, explain anything it needs to install, ask for approval, and guide you through system prompts. After setup, restart the agent if it asks you to load the full REA toolset.
|
||||
|
||||
Review the setup plan, approve it if appropriate, complete Hopper's one-time activation when prompted, then describe the app or feature you want to understand.
|
||||
|
||||
### From Terminal — no installation
|
||||
|
||||
```bash
|
||||
npx -y rea-agents setup
|
||||
npx -y rea-agents doctor
|
||||
npx -y rea-agents analyze /Applications/Notes.app
|
||||
```
|
||||
|
||||
Review the setup plan before confirming it. Restart a configured coding agent so it loads REA.
|
||||
|
||||
### From Terminal — install the `rea` command
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea setup
|
||||
rea doctor
|
||||
rea analyze /Applications/Notes.app
|
||||
```
|
||||
|
||||
Update that global installation in place:
|
||||
|
||||
```bash
|
||||
rea upgrade
|
||||
```
|
||||
|
||||
REA checks npm for the latest release and verifies that the running package is
|
||||
the global installation it will replace. Source, local, and `npx` copies report
|
||||
the manual `npm install --global rea-agents@latest` command instead of updating
|
||||
an unrelated global package.
|
||||
|
||||
Choose either the no-install commands or the global installation. You do not need both.
|
||||
|
||||
### Requirements
|
||||
|
||||
- macOS 12 or newer
|
||||
- Ubuntu 24.04+, Fedora 41+, or 64-bit Arch Linux
|
||||
- Node.js 22.19+ or 24.11+ (including newer releases)
|
||||
- npm; REA does not require or install a particular npm version
|
||||
|
||||
Deep binary analysis currently uses [Hopper](https://www.hopperapp.com/), a separate desktop application with its own license. Setup reuses an existing installation. If Hopper is missing, interactive setup proposes the official package and includes it in the confirmation plan. Unattended installation requires `rea setup --yes --install-hopper`.
|
||||
|
||||
If something is not working, run:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents doctor
|
||||
```
|
||||
|
||||
`rea doctor --json` is read-only and distinguishes unsupported hosts, missing dependencies, a missing local analysis engine, configuration drift, and healthy checks. Fresh-install activation is reported by setup because Hopper does not expose a reliable noninteractive activation probe.
|
||||
|
||||
### Linux installation and troubleshooting
|
||||
|
||||
On macOS, approved setup downloads Hopper's official DMG, verifies it, and installs the app into `~/Applications` without Homebrew or administrator privileges. It opens Hopper once for activation; no manual drag-and-drop is required.
|
||||
|
||||
On Ubuntu 24.04+, Fedora 41+, and 64-bit Arch Linux, approved setup downloads the matching official Hopper package, restricts downloads to Hopper's public origin, verifies the published size and checksum, and invokes `apt-get`, `dnf`, or `pacman`. When REA is not already running as root, `pkexec` presents the system authorization prompt. REA never invokes `sudo`.
|
||||
|
||||
The normal Linux launcher is `/opt/hopper/bin/Hopper`. If Hopper was installed elsewhere:
|
||||
|
||||
```bash
|
||||
export HOPPER_LAUNCHER_PATH=/absolute/path/to/Hopper
|
||||
rea doctor --json
|
||||
```
|
||||
|
||||
If doctor reports a missing analysis engine even though the file exists, inspect shared-library resolution with:
|
||||
|
||||
```bash
|
||||
ldd /opt/hopper/bin/Hopper | grep 'not found'
|
||||
```
|
||||
|
||||
Install the missing distribution packages and rerun `rea setup`. Hopper is a desktop application: real analysis requires an active `DISPLAY` or `WAYLAND_DISPLAY`, plus one-time license activation. The curl installer places the `rea` command in `~/.local/bin` on Linux; add that directory to future shell `PATH` values if it is not already present.
|
||||
|
||||
REA defaults `HOPPER_LAUNCHER_PATH` to `/Applications/Hopper Disassembler.app/Contents/MacOS/hopper` on macOS and `/opt/hopper/bin/Hopper` on Linux. Explicit configuration always takes precedence.
|
||||
|
||||
To remove only REA-owned MCP registrations and the managed skill:
|
||||
|
||||
```bash
|
||||
rea uninstall
|
||||
rea uninstall --purge-data # also removes only ~/.rea/cache and ~/.rea/state
|
||||
```
|
||||
|
||||
Uninstall preserves Hopper, Node.js, evidence, captures, external evidence roots, unrelated skills, and other MCP servers. It refuses malformed client configuration and never follows purge-data symlinks.
|
||||
|
||||
### CLI or coding agent?
|
||||
|
||||
| If you want to… | Use |
|
||||
| --------------------------------------------------------- | -------------------------------------------------------------- |
|
||||
| Ask an agent to investigate an app and build a feature | Install the skill, then talk to your agent |
|
||||
| Inspect or decompile one part of an app from the Terminal | `rea analyze` or `rea decompile` |
|
||||
| Validate, canonicalize, or compare Evidence v2 bundles | `rea evidence-import`, `rea evidence-export`, or `rea compare` |
|
||||
| Import source as historical reference | `rea import-reference-source` |
|
||||
|
||||
Filesystem evidence commands and MCP file tools are disabled until the operator approves absolute roots:
|
||||
|
||||
```bash
|
||||
export REA_EVIDENCE_ROOTS_JSON='["/absolute/path/to/evidence"]'
|
||||
rea evidence-import /absolute/path/to/evidence/bundle.json
|
||||
rea evidence-export /absolute/path/to/evidence/bundle.json /absolute/path/to/evidence/canonical.json
|
||||
rea compare /absolute/path/to/evidence/left.json /absolute/path/to/evidence/right.json
|
||||
```
|
||||
|
||||
Historical source import requires a separate allowlist and never treats source as current behavioral authority:
|
||||
|
||||
```bash
|
||||
export REA_REFERENCE_ROOTS_JSON='["/absolute/path/to/source"]'
|
||||
rea import-reference-source /absolute/path/to/source
|
||||
```
|
||||
|
||||
Exports never replace an existing file unless `--overwrite` is explicit. Imports are size/depth bounded, validate every Evidence v2 ID and manifest, and never execute bundle content.
|
||||
|
||||
## One prompt, a full investigation
|
||||
|
||||
```text
|
||||
Understand how search works in the Notes app, show me the evidence, and build a
|
||||
similar feature for my project.
|
||||
Reverse engineer the Notes app. Find how offline search works, explain it,
|
||||
and build a version for my project using TypeScript and SQLite.
|
||||
```
|
||||
|
||||
Replace Notes with your target app and the feature you want to understand.
|
||||
REA gives the agent a clear path from that request to working code:
|
||||
|
||||
### Use the terminal
|
||||
| Step | What the agent does | REA tools |
|
||||
| ---: | --------------------------------------- | ---------------------------------------------------------------- |
|
||||
| 1 | Opens and identifies the binary | `open_binary`, `binary_overview` |
|
||||
| 2 | Finds likely offline-search clues | `search_strings`, `search_procedures`, `list_names` |
|
||||
| 3 | Connects those clues to executable code | `find_xrefs_to_name`, `xrefs`, `procedure_callers` |
|
||||
| 4 | Reconstructs the relevant control flow | `get_call_graph`, `procedure_callees`, `procedure_info` |
|
||||
| 5 | Decompiles the relevant routines | `procedure_pseudo_code`, `procedure_assembly`, `batch_decompile` |
|
||||
| 6 | Builds the feature in your project | code adapted to your stack, product, and requirements |
|
||||
|
||||
Inspect an extracted JavaScript/Electron app directory or ASAR:
|
||||
REA handles the app analysis in steps 1–5. The agent performs step 6 with its normal file-editing and test tools, using what it learned about the app.
|
||||
|
||||
## What agents can do
|
||||
|
||||
- Investigate a feature you like and build a version tailored to your own product.
|
||||
- Explain how a feature works when its source code is unavailable.
|
||||
- Reconstruct an app's authentication, storage, update, or networking flow.
|
||||
- Recover enough structure to document an undocumented format or interface.
|
||||
- Trace a suspicious behavior from a string or symbol to the code that implements it.
|
||||
- Compare implementation paths across two app versions by switching targets in one session.
|
||||
- Turn recovered behavior into product features, tests, migration notes, ports, or interoperable replacements.
|
||||
- Analyze Swift and Objective-C metadata without manually untangling every mangled symbol.
|
||||
- Leave names, comments, and bookmarks in Hopper so human and agent analysis reinforce each other.
|
||||
|
||||
## 68 tools for investigation
|
||||
|
||||
| Tool family | Count | Examples |
|
||||
| ------------------------- | ----: | ----------------------------------------------------------------------------------------------------------------------- |
|
||||
| Native inspection | 33 | procedures, pseudocode, assembly, strings, names, segments, callers, callees, xrefs, annotations |
|
||||
| Investigation workflows | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `trace_feature`, call graphs, Swift and Objective-C discovery |
|
||||
| Native macOS utilities | 5 | Mach-O metadata, code signatures, plists, architectures, Swift demangling; Hopper-free and provenance-bearing |
|
||||
| Artifact graph | 2 | deterministic directory, ZIP/APK/IPA, and ASAR inventory; explicitly selected extraction into an absent owned tree |
|
||||
| Workspace and observation | 18 | target lifecycle, Evidence v2 bundles, process/artifact/function comparison, evidence-linked residual-unknown lifecycle |
|
||||
|
||||
The public interface describes what the agent is trying to learn. Providers decide how to answer. macOS utilities handle common semantic inspection without launching Hopper; Hopper handles deeper native analysis; the process harness implements controlled behavioral capture.
|
||||
|
||||
## Current status
|
||||
|
||||
REA is already useful for native application investigation on macOS:
|
||||
|
||||
- Open Mach-O, ELF, PE, `.app`, ZIP, APK, IPA, ASAR, plist, JavaScript, source-map, and Hopper database targets.
|
||||
- Traverse content-addressed artifact graphs without extraction; materialize only approved occurrences into absent output roots.
|
||||
- Build bounded function dossiers with pseudocode, assembly, CFG edges, comments, calls, references, strings, and names.
|
||||
- Search and trace features across symbols, strings, metadata, references, and call paths.
|
||||
- Record every successful result as deterministic Evidence v2 with artifact and provider identity, confidence, authority, limitations, and locations.
|
||||
- Export and import evidence bundles across sessions.
|
||||
- Capture approved PTY scenarios, child processes, filesystem changes, and loopback HTTP/WebSocket exchanges, then compare normalized captures.
|
||||
- Compare complete artifact inventories by stable path, content, metadata, and relations; incomplete evidence never implies equivalence.
|
||||
- Compare explicit function dossiers across text, calls, references, strings, and address-normalized CFG topology with per-facet unknowns.
|
||||
- Compare canonical Evidence bundles by exact membership, explicit observation pairs, and residual-unknown histories without turning omissions into behavioral absence.
|
||||
- Aggregate runtime comparisons into observed behavior changes while keeping static artifact/function differences labeled as candidates.
|
||||
- Build bounded, Evidence-cited direct call paths by exact address without treating missing dossiers as graph leaves.
|
||||
- Correlate exact static/runtime findings through explicit hypotheses without claiming causality from cochange.
|
||||
- Verify finite behavioral and structural reconstruction specifications with pass, fail, and unknown kept distinct.
|
||||
- Track residual unknowns through immutable CAS revisions, evidence-qualified resolution, contradictions, probes, and validated dependency relationships.
|
||||
- With explicit `unknown_registry_approved: true`, record bounded trace/capture residuals, typed provider unavailability, and capture disagreements automatically.
|
||||
|
||||
Hopper is the first provider, not the boundary of the project. Some current workflows still require Hopper and macOS; every evidence record identifies the provider and limitations behind its result.
|
||||
|
||||
## Roadmap
|
||||
|
||||
REA is growing into a toolkit for understanding software across static artifacts and observed behavior. The next capability families are:
|
||||
|
||||
1. **Artifact decomposition** — DMG, ASAR, ZIP, packages, universal-binary slices, application resources, embedded frameworks, mobile packages, and artifact graphs.
|
||||
2. **Web and Electron investigation** — Playwright/CDP capture of DOM, accessibility trees, screenshots, storage, console, IPC, HTTP, WebSocket, routes, and visual or structural differences.
|
||||
3. **Deterministic behavior harnesses** — stronger process-tree ownership, protocol fixtures, network policy, filesystem tracing, signals, reconnects, and cross-version comparison.
|
||||
4. **JavaScript and source recovery** — bundle indexing, AST/module reconstruction, source-map discovery, historical-source matching, and CodeDB-backed cross-references.
|
||||
5. **Runtime observation** — approval-gated LLDB, Frida, system logs, process and filesystem observers, and native API tracing.
|
||||
6. **More static-analysis providers** — native platform utilities first, followed by Ghidra, IDA/Hex-Rays, Binary Ninja, Rizin, LIEF, and other engines behind provider-neutral capabilities.
|
||||
7. **More targets and platforms** — Windows-native providers and ConPTY verification, Linux parity, websites and APIs, mobile artifacts, firmware, document formats, and other software-defined systems.
|
||||
8. **Differential reconstruction** — compare artifacts, functions, bundles, protocols, UIs, and process captures; track residual unknowns; verify a reconstruction against observed behavior.
|
||||
|
||||
Roadmap items describe direction, not shipped support. New providers must produce the same evidence and safety metadata as existing capabilities before they become part of the public workflow. Once REA has multiple optional toolchains, setup can become capability-selective; the consent rules for that future work are recorded in the [installation roadmap](docs/roadmap.md).
|
||||
|
||||
See the [static-analysis provider evaluation](docs/provider-evaluation.md) for the current research matrix and admission gate.
|
||||
|
||||
## Using REA with other coding agents
|
||||
|
||||
Setup currently configures Claude Desktop and Cursor automatically. Any coding agent that supports local MCP servers can use REA with the configuration below.
|
||||
|
||||
### Manual MCP configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## How it works
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Agent["Coding agent"] --> REA["REA<br/>CLI + MCP"]
|
||||
Terminal --> REA
|
||||
REA --> Workspace["Investigation workspace<br/>evidence + artifacts + captures"]
|
||||
Workspace --> Router["Capability router"]
|
||||
Router --> Hopper["Hopper provider"]
|
||||
Router --> Native["Native macOS provider"]
|
||||
Router --> Artifact["Artifact graph provider"]
|
||||
Router --> Process["Process capture provider"]
|
||||
Router -. roadmap .-> More["Browser, dynamic,<br/>and additional static providers"]
|
||||
Hopper --> Target["Target software"]
|
||||
Process --> Target
|
||||
Native --> Target
|
||||
Artifact --> Target
|
||||
```
|
||||
|
||||
The CLI and MCP server use the same application workflows and evidence contracts. A provider declares which capabilities it supports and the side effects those capabilities may have. Terminal commands are short-lived; an MCP session can retain an active target and evidence ledger across an investigation.
|
||||
|
||||
## CLI
|
||||
|
||||
The agent workflow above is the easiest way to use REA. For a one-off overview from the Terminal:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
|
||||
npx -y rea-agents analyze /Applications/Notes.app
|
||||
npx -y rea-agents inspect /Applications/Notes.app
|
||||
npx -y rea-agents inspect /Applications/Notes.app --detail detailed --limit 20
|
||||
npx -y rea-agents search /Applications/Notes.app "offline"
|
||||
npx -y rea-agents function /Applications/Notes.app 0x1000
|
||||
npx -y rea-agents xrefs /Applications/Notes.app 0x1000
|
||||
npx -y rea-agents trace /Applications/Notes.app "offline"
|
||||
npx -y rea-agents compare /absolute/path/to/left-evidence.json /absolute/path/to/right-evidence.json
|
||||
npx -y rea-agents capabilities
|
||||
npx -y rea-agents providers
|
||||
```
|
||||
|
||||
The result includes modules, imports, Electron boundaries and their evidence.
|
||||
Replace the path with your target, such as `"D:/apps/example"` on Windows.
|
||||
Run `npx -y rea-agents --help` for direct decompilation, bounded search and
|
||||
other options. `analyze` and `inspect` share the same overview workflow;
|
||||
`function`, `xrefs`, and `trace` return the same Evidence v2 envelopes as MCP.
|
||||
|
||||
To install the `rea` command for regular use:
|
||||
Or install the `rea` command globally:
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea --help
|
||||
rea upgrade
|
||||
rea mcp
|
||||
```
|
||||
|
||||
For native analysis, configure a provider first. See the
|
||||
[CLI and Evidence guide](docs/cli.md) for native commands, provider selection,
|
||||
snapshots and scripting.
|
||||
REA accepts a Mac `.app` folder directly. If an agent cannot find an app by name, tell it where the app is installed.
|
||||
|
||||
### Update REA
|
||||
## Current Hopper provider
|
||||
|
||||
REA changes quickly, and new releases include frequent bug fixes. Keep your
|
||||
installation up to date.
|
||||
REA starts Hopper when needed; Hopper does not need to be running first. Hopper's launcher internally activates the application, so opening a target may bring Hopper to the foreground. REA asks macOS to start Hopper hidden and in the background when possible, but cannot guarantee that it will remain behind the current application.
|
||||
|
||||
For a CLI installed with `npm install --global` or `bun add --global`:
|
||||
REA derives explicit format and architecture arguments to prevent common FAT and ARM selection dialogs. Other Hopper or macOS dialogs may still require a person. REA reports timeouts and remediation through CLI or MCP results instead of attempting to answer UI prompts.
|
||||
|
||||
```bash
|
||||
rea update
|
||||
```
|
||||
Closing a REA session shuts down its bridge and removes its private socket directory. It does not quit a Hopper application the user may be using.
|
||||
|
||||
`rea --update` is an alias for this verified workflow and accepts the same global
|
||||
output flags. Use `rea update` when combining it with other command controls.
|
||||
## Advanced process-capture setup
|
||||
|
||||
Updater commands retain at most 4 MiB of stdout and stderr combined. Oversized
|
||||
metadata replies are rejected after the command exits; verbose installs continue
|
||||
with bounded diagnostics. Human progress output respects terminal backpressure
|
||||
and waits for submitted writes to finish.
|
||||
Process capture is disabled by default. Enabling it requires
|
||||
`REA_PROCESS_CAPTURE_ENABLED=true`, approved executable and working roots in
|
||||
`REA_PROCESS_EXECUTABLE_ROOTS_JSON` and `REA_PROCESS_WORKING_ROOTS_JSON`, and an
|
||||
environment allowlist in `REA_PROCESS_ALLOWED_ENV_JSON`. Because the current PTY
|
||||
adapter uses host networking, it also requires
|
||||
`REA_PROCESS_ALLOW_EXTERNAL_NETWORK=true`.
|
||||
|
||||
Bun global updates require Bun 1.2.17 or later. With an older Bun release,
|
||||
upgrade Bun manually before running `rea update`.
|
||||
If the native PTY backend is unavailable, install Xcode command-line tools and
|
||||
run `npm run rebuild:native`. Linux source builds require Python, `make`, and a
|
||||
C++ toolchain. Compatible packaged binaries do not require this rebuild.
|
||||
|
||||
To refresh your agent registrations and skill, run the setup command printed
|
||||
by the update.
|
||||
## Security model
|
||||
|
||||
If you use `npx`, update your agent setup with:
|
||||
|
||||
```bash
|
||||
npx rea-agents@latest setup
|
||||
```
|
||||
|
||||
Review the setup changes and restart your agent. For one-off CLI commands,
|
||||
use `npx rea-agents@latest` followed by the command. With Bun, use
|
||||
`bunx rea-agents@latest` in place of `npx rea-agents@latest`.
|
||||
|
||||
## How REA works
|
||||
|
||||
Your agent calls REA through MCP to inspect the target and trace relevant code.
|
||||
REA returns findings with their evidence. The agent uses them to ask follow-up
|
||||
questions, explain the behavior, or write and test an implementation.
|
||||
CLI commands use the same workflows.
|
||||
|
||||
|
||||
|
||||
[Open the full-size figure](website/public/assets/figures/rea-investigation-flow.svg).
|
||||
|
||||
<a id="current-status"></a>
|
||||
|
||||
## What you can analyze
|
||||
|
||||
REA requires Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+, plus npm.
|
||||
Additional tools and host support depend on the target:
|
||||
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| GPU code objects | CUDA/HIP targets, selected kernel parameters and resources, retained PTX text | Built-in static ELF/PE/fatbin/CUBIN/PTX inspection; [GPU code objects](docs/gpu-code-objects.md) |
|
||||
| Model artifacts | GGUF / safetensors metadata, tensor declarations and derived totals | Built-in header inspection; optional full-file hash; [model files](docs/model-artifacts.md) |
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
|
||||
| PyInstaller executables | CArchive members, Python version, entry scripts and extracted bytecode | None (static only); [Python packaged applications](docs/python-packaged-applications.md) |
|
||||
|
||||
Static JavaScript and .NET inspection read the supplied files without running
|
||||
the application. Runtime capture runs or interacts with the selected target
|
||||
using your user permissions; each runtime guide describes its effects.
|
||||
|
||||
For offline native-binary inspection, REA also supports caller-supplied GNU
|
||||
objdump and Rizin. See the [reverse-engineering tools guide](docs/reverse-engineering-tools.md)
|
||||
for provider setup, CLI commands, and Evidence limitations. REA does not install
|
||||
or update either provider.
|
||||
|
||||
<a id="choosing-a-deep-analysis-provider"></a>
|
||||
|
||||
Native formats and host support vary by provider. See
|
||||
[Hopper and Ghidra setup](docs/installation.md#hopper), the
|
||||
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), the
|
||||
[Kuna decompiler guide](docs/kuna-provider.md), and
|
||||
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
|
||||
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
|
||||
For large binaries, raise the provider's startup deadline with
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS` or `REA_HOPPER_STARTUP_TIMEOUT_MS`.
|
||||
For provider selection, see the [CLI guide](docs/cli.md#choose-a-provider).
|
||||
Check [release availability](docs/installation.md#released-package-and-main)
|
||||
for features added since the latest npm release.
|
||||
|
||||
## Showcases
|
||||
|
||||
[](https://rea.tools/showcase/)
|
||||
|
||||
### DX-Ball: reconstruct a sound-pan calculation
|
||||
|
||||
Follow a sound call into its position-to-pan helper, inspect the instructions,
|
||||
and turn incomplete pseudocode into C. The reconstruction passes 3,205
|
||||
original-x86 cases and reproduces all 63 compiled function bytes.
|
||||
|
||||
[Read the case study](https://rea.tools/showcase/dx-ball/) ·
|
||||
[Reconstruction repository](https://github.com/N0zoM1z0/dx-ball)
|
||||
|
||||
### Notion: trace the Electron clipboard bridge
|
||||
|
||||
Find the renderer's clipboard API, follow it through preload and IPC into the
|
||||
main process, and inspect the rich clipboard format.
|
||||
|
||||
[Read the case study](https://rea.tools/showcase/notion/)
|
||||
|
||||
### TH04: recover a DOS bullet-ring calculation
|
||||
|
||||
Inspect the original PC-98 game's 16-bit instructions, recover the fixed and
|
||||
aimed angle calculations, and compare the reconstructed C++ with the
|
||||
historical compiler output.
|
||||
|
||||
[Read the case study](https://rea.tools/showcase/th04/) ·
|
||||
[Reconstruction repository](https://github.com/N0zoM1z0/th04)
|
||||
|
||||
If you've used REA on something interesting, we'd love to see it. Share your
|
||||
case in an [issue](https://github.com/morluto/rea/issues) or a
|
||||
[pull request](https://github.com/morluto/rea/pulls), including the target,
|
||||
your question, how REA helped, and what you found.
|
||||
REA does not provide a hosted analysis service. Hopper communication uses an authenticated private local socket. Dynamic capabilities are disabled by default and require both operator policy and explicit per-call approval. REA is not a security sandbox: providers and launched targets run with the current user's permissions, and each capability reports its side effects and limitations. Report vulnerabilities through the private process in [SECURITY.md](SECURITY.md).
|
||||
|
||||
## FAQ
|
||||
|
||||
<details>
|
||||
<summary><strong>Does Hopper need to be running before I start REA?</strong></summary>
|
||||
|
||||
No. REA starts Hopper when an operation needs it. An already-running Hopper application is also supported.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Why did Hopper appear in front of my other windows?</strong></summary>
|
||||
|
||||
Hopper's launcher internally activates the application. REA requests background startup, but macOS and Hopper may still bring a window or dialog forward. See [Hopper application behavior](#hopper-application-behavior).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Does REA include Hopper?</strong></summary>
|
||||
|
||||
No. Setup can install Hopper for you, but Hopper remains separate software with its own license. REA supplies the CLI, MCP server, and workflows that make it usable by agents.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Does REA upload the app?</strong></summary>
|
||||
|
||||
REA has no hosted analysis service. Current providers analyze artifacts and capture behavior locally. Your coding agent or model provider may have its own data policy, so review that separately.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Can REA recover the original source code?</strong></summary>
|
||||
|
||||
No decompiler can guarantee the original source. REA gives an agent pseudocode, assembly, symbols, strings, metadata, and relationships that it can use to explain or compatibly recreate observed behavior.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Which agents can use REA?</strong></summary>
|
||||
|
||||
Any agent that supports local MCP servers. Setup configures the
|
||||
[supported agents](docs/installation.md#supported-agents); other clients can use
|
||||
[manual MCP registration](docs/installation.md#mcp-registry).
|
||||
Any coding agent that can run a local MCP server can use the manual configuration. Setup currently detects and configures Claude Desktop and Cursor automatically.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Do I need Hopper, Ghidra or IDA?</strong></summary>
|
||||
## Development
|
||||
|
||||
Deep native analysis uses one of them. Static JavaScript and .NET inspection
|
||||
work without a native analysis engine. Setup can install Hopper after approval;
|
||||
Ghidra and IDA use your existing installations. See [provider setup](docs/installation.md#hopper).
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, architecture, tests, and release instructions. Generated API documentation is available under [`docs/api`](docs/api/index.html).
|
||||
|
||||
</details>
|
||||
## Project links
|
||||
|
||||
<details>
|
||||
<summary><strong>Do I need to start Hopper first?</strong></summary>
|
||||
|
||||
REA starts Hopper when an operation needs it. On macOS, a first-run dialog may
|
||||
ask you to choose demo mode or activate your license. Complete that prompt in the
|
||||
same user session before your first CLI/MCP operation or unattended job. Open
|
||||
Hopper manually if needed. See
|
||||
[Hopper startup and troubleshooting](docs/installation.md#launcher-paths-and-troubleshooting).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>What does installing the skill from skills.sh do?</strong></summary>
|
||||
|
||||
The skill supplies investigation instructions for your agent. Use `npx rea-agents setup`
|
||||
to register REA's MCP server and install the matching instructions, then restart
|
||||
your agent. See [skill-only installation](docs/installation.md#skill-only-installation).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>What code does REA return?</strong></summary>
|
||||
|
||||
Native analysis returns pseudocode and assembly. JavaScript/Electron analysis
|
||||
recovers modules and their relationships. Your agent uses these findings to
|
||||
write and test an implementation; the [showcases](#showcases) give worked examples.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Does REA upload my app?</strong></summary>
|
||||
|
||||
REA analyzes targets locally. Your agent receives the tool results, and its
|
||||
model provider has its own data policy.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>What should I do if I hit a bug?</strong></summary>
|
||||
|
||||
Update first; a recent release may already fix it.
|
||||
|
||||
For a CLI installed with `npm install --global` or `bun add --global`:
|
||||
|
||||
```bash
|
||||
rea update
|
||||
```
|
||||
|
||||
For agent setup through `npx`:
|
||||
|
||||
```bash
|
||||
npx rea-agents@latest setup
|
||||
```
|
||||
|
||||
If you're using an agent, complete the [setup refresh](#update-rea) and restart
|
||||
it. Retry the same task. If the problem persists, [open an issue](https://github.com/morluto/rea/issues)
|
||||
with your REA version, target type, steps to reproduce and error output.
|
||||
|
||||
</details>
|
||||
|
||||
## Documentation
|
||||
|
||||
Start with the website's [worked guides](https://rea.tools/guides/).
|
||||
For exact options, prerequisites and result contracts:
|
||||
|
||||
- [Installation and setup](docs/installation.md): agent registration, provider configuration, updates and uninstall.
|
||||
- [Readiness and troubleshooting](docs/installation.md#check-readiness-for-your-task): diagnose one agent or analysis engine.
|
||||
- [CLI and Evidence](docs/cli.md): commands, provider selection, snapshots, import/export and exit statuses.
|
||||
- [MCP contracts](docs/mcp-contracts.md) and [agent prompts](docs/mcp-prompts.md): tool results, sessions and guided investigations.
|
||||
- [Tool catalog](docs/mcp-contracts.md#generated-catalog): generated inventory of tools, providers and CLI commands.
|
||||
- [Roadmap](docs/roadmap.md): planned work and capability trackers.
|
||||
|
||||
Report vulnerabilities through [SECURITY.md](SECURITY.md).
|
||||
|
||||
## Star history
|
||||
|
||||
🎉 **300,000 GitHub stars — thank you!**
|
||||
|
||||
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
|
||||
|
||||
<a href="https://www.star-history.com/?repos=morluto%2Frea&type=date">
|
||||
<picture>
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/chart?repos=morluto/rea&type=date&theme=dark&legend=top-left" />
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/chart?repos=morluto/rea&type=date" />
|
||||
<img alt="REA GitHub star history" src="https://api.star-history.com/chart?repos=morluto/rea&type=date" />
|
||||
</picture>
|
||||
</a>
|
||||
|
||||
## Disclaimer
|
||||
|
||||
See [DISCLAIMER.md](DISCLAIMER.md) for responsible-use guidance.
|
||||
|
||||
The project's position on illegal or unauthorized use covers all REA versions,
|
||||
distribution channels, and platforms; each copy retains its applicable licenses.
|
||||
|
||||
REA is for lawful research, education, interoperability, software preservation,
|
||||
and analysis of software you own or are authorized to examine.
|
||||
|
||||
## Contributing
|
||||
|
||||
We'd love your help with REA! [Open an issue](https://github.com/morluto/rea/issues) to
|
||||
report a bug or suggest a feature, or [send a pull request](https://github.com/morluto/rea/pulls)
|
||||
to improve the code or docs.
|
||||
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and checks,
|
||||
[testing](docs/testing.md) for verification lanes, and the
|
||||
[architecture map](docs/architecture.mermaid) for the project structure.
|
||||
[npm](https://www.npmjs.com/package/rea-agents) · [Issues](https://github.com/morluto/rea/issues) · [Security](SECURITY.md) · [Contributing](CONTRIBUTING.md) · [Hopper](https://www.hopperapp.com/)
|
||||
|
||||
## License
|
||||
|
||||
[MIT](LICENSE)
|
||||
|
||||
REA-authored code is licensed under MIT. Third-party material retains its own
|
||||
licenses and copyright notices; see
|
||||
[Third-party notices](THIRD_PARTY_NOTICES.md) for the dependency inventory,
|
||||
preserved license texts, and external-tool terms.
|
||||
|
||||
**You must legally obtain and be authorized to use commercial tools**, including
|
||||
IDA Pro / Hex-Rays decompilers, Hopper, and JEB, under their applicable
|
||||
licenses. REA grants no tool license, key, or activation entitlement. Trials and
|
||||
demos remain subject to vendor terms, including user, feature, automation, and
|
||||
output sharing restrictions. Cracked copies, stolen keys, unauthorized key
|
||||
sharing, and bypassing license restrictions are not authorized. This guidance
|
||||
does not amend MIT or third-party licenses.
|
||||
+272
@@ -0,0 +1,272 @@
|
||||
<div align="center">
|
||||
|
||||
[English](README.md) · [简体中文](README_zh.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · **العربية**
|
||||
|
||||
# REA: هندسة أي شيء عكسيًا
|
||||
|
||||
### واجهة CLI وخادم MCP واحدان يمكّنان وكلاء البرمجة من إجراء هندسة عكسية لأي شيء
|
||||
|
||||
**اعثر على ميزة تعجبك. افهم آلية عملها. ابنها بالطريقة التي تريدها.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](#منصة-من-68-أداة)
|
||||
[](https://nodejs.org/)
|
||||
[](LICENSE)
|
||||
|
||||
[البدء السريع](#البدء-السريع) · [من الملف التنفيذي إلى السلوك](#من-الملف-التنفيذي-إلى-السلوك) · [68 أداة](#منصة-من-68-أداة) · [كيف يعمل](#كيف-يعمل) · [الأسئلة الشائعة](#الأسئلة-الشائعة)
|
||||
|
||||
<br />
|
||||
|
||||
<code>curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash</code>
|
||||
|
||||
</div>
|
||||
|
||||
هل وجدت في تطبيق ما ميزة تريدها في منتجك؟ أعط التطبيق إلى وكيل البرمجة حتى من دون شيفرته المصدرية. باستخدام REA، يستطيع الوكيل استقصاء الميزة وفهم آلية عملها ثم بناء نسخة ملائمة لتقنياتك وتصميمك ومتطلباتك.
|
||||
|
||||
يوفر REA هذه العملية عبر CLI وخادم MCP واحدين. يستطيع الوكيل قراءة شيفرة التطبيق وتتبع طريقة عمل الميزات ثم استخدام الأدلة مباشرة في عمله البرمجي المعتاد. ويتولى REA أدوات الهندسة العكسية المعقدة خلف واجهة واحدة.
|
||||
|
||||
## اطلب من وكيلك مباشرة
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
ثم اطلب:
|
||||
|
||||
```text
|
||||
أعد REA وأجر هندسة عكسية لتطبيق Notes. اشرح طريقة عمل البحث وكيف توصلت إلى ذلك،
|
||||
ثم ابنِ ميزة مشابهة لمشروعي.
|
||||
```
|
||||
|
||||
تطبيق Notes مجرد مثال. يمكنك تسمية أي تطبيق تريد فهمه أو طلب البدء بنظرة عامة.
|
||||
|
||||
## من الملف التنفيذي إلى السلوك
|
||||
|
||||
| فك الترجمة | الفهم | إعادة الإنشاء |
|
||||
| ----------------------------------------------------------------- | --------------------------------------------------------- | ------------------------------------------------------------- |
|
||||
| استخرج الشيفرة شبه المصدرية، والتعليمات، والسلاسل النصية، والرموز | تتبع تدفق التحكم والمراجع المتبادلة والاستدعاءات والأنواع | حوّل ما تعلمه الوكيل إلى ميزة تناسب تقنياتك وواجهتك ومتطلباتك |
|
||||
|
||||
يوضح REA كيف وصل إلى نتائجه. وهو لا يدّعي استعادة الشيفرة المصدرية الأصلية أو استنساخ التطبيق تلقائيًا.
|
||||
|
||||
## لماذا REA؟
|
||||
|
||||
| | |
|
||||
| -------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| **مصمم للوكلاء** | اسأل عما يفعله تطبيق مترجم ودع الوكيل يجمع الأدلة بدلًا من التخمين. |
|
||||
| **CLI وMCP** | استخدم قدرات الهندسة العكسية نفسها من الطرفية أو وكيل البرمجة. |
|
||||
| **يتولى التعقيد** | يدير REA إعداد الأدوات وفتح التطبيق واستمرار الاستقصاء والتنظيف بعد الانتهاء. |
|
||||
| **سير عمل كامل** | انتقل من النظرة الأولى إلى الشيفرة شبه المصدرية وعلاقات الاستدعاء والأنواع وأدلة التنفيذ. |
|
||||
| **محلي حسب التصميم** | يجري التحليل على جهاز Mac ولا يرفع REA الملف التنفيذي إلى خدمة تحليل مستضافة. |
|
||||
| **يحافظ على السياق** | استقصِ عدة ملفات تنفيذية من دون بدء التحليل من جديد عند كل سؤال. |
|
||||
|
||||
## البدء السريع
|
||||
|
||||
### باستخدام وكيل برمجة — موصى به
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
اطلب من الوكيل إعداد REA. سيفحص جهاز Mac ويشرح ما يحتاج إلى تثبيته ويطلب الموافقة ويرشدك خلال رسائل النظام. أعد تشغيل الوكيل بعد الإعداد إذا طلب ذلك لتحميل مجموعة الأدوات الكاملة.
|
||||
|
||||
### قبل البدء
|
||||
|
||||
- macOS 12 أو أحدث
|
||||
- Ubuntu 24.04+ أو Fedora 41+ أو Arch Linux 64 بت
|
||||
- Node.js 22.19+ أو 24.11+ وnpm المرفق مع Node
|
||||
|
||||
يعرض `rea setup` خطة التغييرات كاملة ويطلب التأكيد قبل تطبيقها. لا يثبت أو يحدّث Homebrew أو Node.js أو npm. إذا كان [Hopper](https://www.hopperapp.com/) مفقودًا، يقترح Setup الحزمة الرسمية. Hopper برنامج منفصل يحتاج إلى ترخيص خاص به.
|
||||
|
||||
#### التثبيت على Linux واستكشاف الأخطاء
|
||||
|
||||
على Ubuntu 24.04+ وFedora 41+ وArch Linux 64 بت، ينزّل REA حزمة DEB أو RPM أو Arch الرسمية، ويتحقق من الحجم وقيمة التحقق المنشورين، ثم يستخدم `apt-get` أو `dnf` أو `pacman` لحل الاعتماديات. عند التشغيل دون root يعرض `pkexec` طلب تفويض النظام. لا يستدعي REA الأمر `sudo`.
|
||||
|
||||
مسار المشغّل الافتراضي هو `/opt/hopper/bin/Hopper`. استخدم `HOPPER_LAUNCHER_PATH` للمسارات الأخرى. إذا أبلغ Doctor عن غياب محرك التحليل، شغّل `ldd /opt/hopper/bin/Hopper | grep 'not found'`، وثبّت المكتبات الناقصة ثم أعد `rea setup`. يتطلب التحليل الحقيقي `DISPLAY` أو `WAYLAND_DISPLAY` وترخيص Hopper مفعّلًا. أضف `~/.local/bin` إلى `PATH` أيضًا.
|
||||
|
||||
### 1. إعداد REA
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash
|
||||
npx -y rea-agents setup
|
||||
```
|
||||
|
||||
إذا طلب macOS أو برنامج التثبيت تأكيدًا، فأكمل الخطوة ثم شغّل الأمر نفسه مرة أخرى.
|
||||
|
||||
### 2. إعادة تشغيل وكيل البرمجة
|
||||
|
||||
يهيئ Setup تلقائيًا نسخ Claude Desktop وCursor التي يعثر عليها. أعد تشغيل التطبيق كي يحمّل REA.
|
||||
|
||||
### 3. اطلب من الوكيل
|
||||
|
||||
يمكنك ذكر التطبيق باسمه. سيعثر وكيل البرمجة عليه ويمرر ملف البرنامج الذي يحتاجه REA.
|
||||
|
||||
```text
|
||||
استخدم REA لإجراء هندسة عكسية لتطبيق Notes. اشرح طريقة عمل البحث واعرض الأدلة،
|
||||
ثم ابنِ ميزة مشابهة لمشروعي باستخدام SQLite.
|
||||
```
|
||||
|
||||
إذا واجهت مشكلة، فشغّل:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents doctor
|
||||
rea uninstall
|
||||
rea uninstall --purge-data
|
||||
```
|
||||
|
||||
## استقصاء كامل بطلب واحد
|
||||
|
||||
بعد الإعداد، يمكنك أن تطلب من وكيلك:
|
||||
|
||||
> افتح التطبيق، واكتشف آلية عمل البحث من دون اتصال، واشرح تدفق التحكم، ثم ابنِ نسخة لمشروعي باستخدام TypeScript وSQLite.
|
||||
|
||||
يمكن للوكيل تحويل ذلك إلى سير عمل قابل للتحقق:
|
||||
|
||||
```text
|
||||
فتح الهدف
|
||||
→ انتظار اكتمال تحليل Hopper
|
||||
→ جرد المعماريات والرموز والسلاسل النصية
|
||||
→ تحديد نقاط الدخول والدوال المهمة
|
||||
→ تتبع الاستدعاءات والمراجع المتبادلة وتدفق التحكم
|
||||
→ فك ترجمة التنفيذ ذي الصلة
|
||||
→ تلخيص السلوك والقيود والحالات الطرفية
|
||||
→ بناء الميزة بما يلائم تقنيات المشروع ومتطلباته
|
||||
```
|
||||
|
||||
## ما الذي يمكنك فعله؟
|
||||
|
||||
### فهم تطبيق لا تملك مصدره
|
||||
|
||||
اكشف نقاط الدخول، وأسماء الفئات، والسلاسل النصية، والأطر المستخدمة، ومسارات الميزات من الملف التنفيذي وحده.
|
||||
|
||||
### بناء ميزة تعجبك بطريقتك
|
||||
|
||||
استقصِ الميزة، وافهم سلوكها، ثم ابنِ نسخة ملائمة لمنتجك وتقنياتك ومتطلباتك.
|
||||
|
||||
### توثيق بروتوكول أو تنسيق غير موثق
|
||||
|
||||
تتبع الموزعات، والثوابت، والمقارنات، والقراءة والكتابة لفهم الرسائل وتنسيقات الملفات وآلات الحالات.
|
||||
|
||||
### إعادة إنشاء سلوك متوافق
|
||||
|
||||
حوّل منطقًا مفكوك الترجمة إلى مواصفات واضحة، ثم نفذ بديلًا نظيفًا تحكمه اختبارات السلوك الملحوظ.
|
||||
|
||||
### تدقيق منطق أمني حساس
|
||||
|
||||
افحص التحقق من الإدخال، والتشفير، والصلاحيات، والتخزين، ومسارات الأخطاء من دون رفع الملف التنفيذي إلى خدمة بعيدة.
|
||||
|
||||
## منصة من 68 أداة
|
||||
|
||||
| عائلة الأدوات | العدد | أمثلة |
|
||||
| --------------------- | ----: | ----------------------------------------------------------------------------------- |
|
||||
| فحص الملفات التنفيذية | 33 | الدوال، والشيفرة شبه المصدرية، والتعليمات، والسلاسل، والأسماء، والمراجع، والتعليقات |
|
||||
| التحليل المركب | 10 | النظرة العامة، وفك الترجمة الدفعي، ومخططات الاستدعاء، والمراجع، واكتشاف الأنواع |
|
||||
| أدوات macOS الأصلية | 5 | بيانات Mach-O والتوقيعات وملفات plist والمعماريات وفك رموز Swift دون تشغيل Hopper |
|
||||
| رسم القطع الأثرية | 2 | جرد حتمي للدلائل وZIP/APK/IPA وASAR واستخراج معاملاتي محدد صراحة |
|
||||
| جلسة الملف التنفيذي | 18 | فتح الأهداف وحزم الأدلة ومقارنة العمليات والقطع والدوال وسجل المجهولات المتبقية |
|
||||
|
||||
## استخدام REA مع وكلاء برمجة آخرين
|
||||
|
||||
يهيئ Setup حاليًا Claude Desktop وCursor تلقائيًا. يمكن لأي وكيل برمجة يدعم خوادم MCP المحلية استخدام REA بالإعداد التالي:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## كيف يعمل؟
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Human[المستخدم] --> Agent[وكيل البرمجة]
|
||||
Agent --> REA[REA]
|
||||
Human -->|CLI| REA
|
||||
REA --> Hopper[محرك التحليل]
|
||||
Hopper --> App[تطبيقك]
|
||||
```
|
||||
|
||||
يستخدم CLI وخادم MCP محرك التحليل نفسه. تغلق أوامر الطرفية التطبيق عند انتهائها، بينما تبقيه جلسة الوكيل مفتوحًا أثناء الاستقصاء.
|
||||
|
||||
## أوامر CLI
|
||||
|
||||
سير عمل الوكيل أعلاه هو أسهل طريقة لاستخدام REA. للحصول على نظرة سريعة على تطبيق من الطرفية:
|
||||
|
||||
```bash
|
||||
npx -y rea-agents analyze /Applications/Notes.app
|
||||
npx -y rea-agents compare /absolute/path/to/left-evidence.json /absolute/path/to/right-evidence.json
|
||||
```
|
||||
|
||||
شغّل `npx -y rea-agents --help` لمعرفة خيارات فك الترجمة المباشر والخيارات الأخرى.
|
||||
|
||||
يمكن لـ REA فتح مجلد `.app` على Mac مباشرة. إذا لم يعثر الوكيل على التطبيق، فأخبره بمكان تثبيته.
|
||||
|
||||
## سلوك Hopper
|
||||
|
||||
يستطيع REA تشغيل Hopper تلقائيًا؛ لا حاجة إلى افتراض أنه يعمل مسبقًا. لكنه تطبيق macOS رسومي، لذلك قد يطلب التركيز أو يعرض حوارًا عندما يفرض Hopper قرارًا تفاعليًا. لا يمكن لـ REA ضمان تشغيل غير مرئي تمامًا لأن هذه الواجهة يملكها Hopper وmacOS.
|
||||
|
||||
لتجنب مربع اختيار معمارية ملف شامل، حدد معطيات المحمل مسبقًا:
|
||||
|
||||
```bash
|
||||
export HOPPER_LOADER_ARGS_JSON='["-l", "Mach-O", "--aarch68"]'
|
||||
```
|
||||
|
||||
يدير REA دورة حياة Hopper ويحاول إغلاق العمليات التي بدأها عند انتهاء الأمر أو جلسة MCP.
|
||||
|
||||
## الأمان والخصوصية
|
||||
|
||||
- يبقى تحليل الملف التنفيذي محليًا بين REA وHopper.
|
||||
- لا يرفع REA الأهداف إلى خدمة مستضافة.
|
||||
- راجع الملفات التنفيذية غير الموثوقة واعزلها كما تفعل مع أي مدخل أصلي قد يكون ضارًا.
|
||||
- أبلغ عن الثغرات وفق [سياسة الأمان](SECURITY.md)، وليس عبر قضية عامة.
|
||||
|
||||
## الأسئلة الشائعة
|
||||
|
||||
<details>
|
||||
<summary><strong>هل يتضمن REA محرك فك ترجمة خاصًا به؟</strong></summary>
|
||||
|
||||
لا. يستطيع Setup تثبيت Hopper، لكنه يظل برنامجًا منفصلًا بترخيص خاص به. يوفر REA واجهة CLI وخادم MCP وسير العمل المخصص للوكلاء.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>هل يجب أن يكون Hopper مفتوحًا مسبقًا؟</strong></summary>
|
||||
|
||||
لا. يستطيع REA تشغيله. لكن Hopper قد يصبح مرئيًا أو يطلب إدخالًا عند الحاجة إلى قرار تفاعلي.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>هل يعمل REA على Linux أو Windows؟</strong></summary>
|
||||
|
||||
نعم، يعمل REA على macOS 12+ وUbuntu 24.04+ وFedora 41+ وArch Linux 64 بت؛ Windows غير مدعوم حاليًا. يعتمد التحليل العميق على تطبيق Hopper وواجهة Python الخاصة به.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>هل يمكنني تحليل قواعد Hopper المحفوظة؟</strong></summary>
|
||||
|
||||
نعم. يدعم وقت التشغيل الملفات التنفيذية وقواعد `.hop`.
|
||||
|
||||
</details>
|
||||
|
||||
## التطوير
|
||||
|
||||
راجع [CONTRIBUTING.md](CONTRIBUTING.md) لإعداد بيئة التطوير وبنية المشروع والاختبارات وإرشادات الإصدار.
|
||||
|
||||
## المشروع
|
||||
|
||||
- [حزمة npm](https://www.npmjs.com/package/rea-agents)
|
||||
- [GitHub](https://github.com/morluto/rea)
|
||||
- [القضايا والطلبات](https://github.com/morluto/rea/issues)
|
||||
- [Hopper Disassembler](https://www.hopperapp.com/)
|
||||
- [سياسة الأمان](SECURITY.md)
|
||||
|
||||
## الترخيص
|
||||
|
||||
[MIT](LICENSE)
|
||||
+247
@@ -0,0 +1,247 @@
|
||||
<div align="center">
|
||||
|
||||
[English](README.md) · [简体中文](README_zh.md) · **日本語** · [한국어](README_ko.md) · [العربية](README_ar.md)
|
||||
|
||||
# REA：あらゆるものをリバースエンジニアリング
|
||||
|
||||
### コーディングエージェントがあらゆるものをリバースエンジニアリングするための、ひとつの CLI / MCP サーバー
|
||||
|
||||
**気になる機能を見つけ、仕組みを理解し、自分の形で実装する。**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](#68-ツールのワークベンチ)
|
||||
[](https://nodejs.org/)
|
||||
[](LICENSE)
|
||||
|
||||
[クイックスタート](#クイックスタート) · [バイナリから動作へ](#バイナリから動作へ) · [68 ツール](#68-ツールのワークベンチ) · [仕組み](#仕組み) · [FAQ](#faq)
|
||||
|
||||
<br />
|
||||
|
||||
<code>curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash</code>
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
アプリの気になる機能を自分のプロダクトにも取り入れたいと思ったことはありませんか。ソースコードがなくても、そのアプリをコーディングエージェントに渡せます。REA を使えば、エージェントが機能を調査して仕組みを理解し、あなたの技術スタック、デザイン、要件に合わせた形で実装できます。
|
||||
|
||||
REA はこの流れをひとつの CLI / MCP サーバーで実現します。エージェントはコンパイル済みのアプリを調べ、機能の流れを追い、分かったことを通常のコーディング作業に活用できます。複雑なリバースエンジニアリングツールは REA がひとつのインターフェースの裏側で扱います。
|
||||
|
||||
## エージェントに頼むだけ
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
次のように依頼します：
|
||||
|
||||
```text
|
||||
REA をセットアップしてメモアプリをリバースエンジニアリングしてください。
|
||||
検索の仕組みと判断の理由を示し、私のプロジェクト向けに同様の機能を実装してください。
|
||||
```
|
||||
|
||||
メモアプリは例にすぎません。調べたいアプリを指定するか、まず概要から始めるよう依頼できます。
|
||||
|
||||
## バイナリから動作へ
|
||||
|
||||
| 逆コンパイル | 理解 | 再現 |
|
||||
| -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
|
||||
| ネイティブアプリや実行ファイルから、プロシージャ、疑似コード、アセンブリ、文字列、シンボル、セグメント、メタデータを復元します。 | 呼び出し元、呼び出し先、クロスリファレンス、コールグラフをたどり、機能やアルゴリズムの実際の動作を説明できる状態にします。 | エージェントが得た知見を、あなたの技術スタック、画面、要件に合うプロダクト機能へ変えます。 |
|
||||
|
||||
REA は調査をバイナリ上の根拠に結び付けます。元のソースコードを復元したり、アプリ全体を自動複製したりするとは主張しません。
|
||||
|
||||
## REA を選ぶ理由
|
||||
|
||||
| | |
|
||||
| ---------------------- | ------------------------------------------------------------------------------------- |
|
||||
| **エージェント向け** | コンパイル済みアプリについて質問し、推測ではなく根拠を集めさせることができます。 |
|
||||
| **CLI と MCP** | ターミナルとコーディングエージェントから同じリバースエンジニアリング機能を使えます。 |
|
||||
| **複雑さを処理** | ツール設定、アプリの読み込み、調査の維持、終了後のクリーンアップを REA が担います。 |
|
||||
| **一連の調査に対応** | 最初の概要から疑似コード、呼び出し関係、型、実装の手掛かりまで掘り下げられます。 |
|
||||
| **ローカルで解析** | 解析は Mac 上で実行され、REA がバイナリをホスト型解析サービスへ送ることはありません。 |
|
||||
| **コンテキストを維持** | 質問ごとに解析を最初からやり直さず、複数のバイナリを続けて調査できます。 |
|
||||
|
||||
## クイックスタート
|
||||
|
||||
### コーディングエージェントから（推奨）
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
エージェントに REA のセットアップを依頼してください。Mac を確認し、必要なインストールを説明して承認を求め、システムプロンプトを案内します。完全なツールを読み込むため再起動を求められた場合は、セットアップ後に再起動してください。
|
||||
|
||||
### 始める前に
|
||||
|
||||
- macOS 12 以降
|
||||
- Ubuntu 24.04+、Fedora 41+、または 64 ビット Arch Linux
|
||||
- Node.js 22.19+ または 24.11+ と、Node に付属する npm
|
||||
|
||||
`rea setup` は変更計画を表示し、確認後に適用します。Homebrew、Node.js、npm をインストールまたは更新しません。[Hopper](https://www.hopperapp.com/) がない場合は公式パッケージを提案します。Hopper は別製品で、ライセンスは別途必要です。
|
||||
|
||||
#### Linux のインストールとトラブルシューティング
|
||||
|
||||
Ubuntu 24.04+、Fedora 41+、64 ビット Arch Linux では、REA が公式の DEB、RPM、または Arch パッケージを取得し、公開されたサイズとチェックサムを検証してから `apt-get`、`dnf`、または `pacman` で依存関係を解決します。root 以外では `pkexec` がシステム承認を表示します。REA は `sudo` を呼び出しません。
|
||||
|
||||
既定のランチャーは `/opt/hopper/bin/Hopper` です。別の場所では `HOPPER_LAUNCHER_PATH` を設定してください。Doctor が解析エンジン不足を報告した場合は `ldd /opt/hopper/bin/Hopper | grep 'not found'` を実行し、不足ライブラリを入れて `rea setup` を再実行します。実解析には `DISPLAY` または `WAYLAND_DISPLAY` と有効化済み Hopper ライセンスが必要です。`~/.local/bin` も `PATH` に追加してください。
|
||||
|
||||
```bash
|
||||
# 1. REA をセットアップ
|
||||
curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash
|
||||
npx -y rea-agents setup
|
||||
```
|
||||
|
||||
macOS やインストーラーから確認を求められた場合は、その操作を完了してから同じコマンドをもう一度実行してください。
|
||||
|
||||
### 2. コーディングエージェントを再起動
|
||||
|
||||
Setup は検出した Claude Desktop と Cursor を自動設定します。REA を読み込むため、アプリを再起動してください。
|
||||
|
||||
### 3. エージェントに依頼
|
||||
|
||||
アプリは名前で指定できます。コーディングエージェントがアプリを探し、REA に必要なプログラムファイルを渡します。
|
||||
|
||||
```text
|
||||
REA でメモアプリをリバースエンジニアリングしてください。検索機能の仕組みと
|
||||
根拠を示し、SQLite を使って私のプロジェクト向けに同様の機能を実装してください。
|
||||
```
|
||||
|
||||
問題がある場合は次を実行します：
|
||||
|
||||
```bash
|
||||
npx -y rea-agents doctor
|
||||
rea uninstall
|
||||
rea uninstall --purge-data
|
||||
```
|
||||
|
||||
## ひとつのプロンプトで調査を完結
|
||||
|
||||
```text
|
||||
メモアプリをリバースエンジニアリングし、オフライン検索機能の仕組みを説明して、
|
||||
TypeScript と SQLite を使って私のプロジェクト向けに実装してください。
|
||||
```
|
||||
|
||||
| 手順 | エージェントの処理 | REA ツール |
|
||||
| ---: | ------------------------------ | ---------------------------------------------------------------- |
|
||||
| 1 | バイナリを開いて識別 | `open_binary`, `binary_overview` |
|
||||
| 2 | オフライン検索の手掛かりを探す | `search_strings`, `search_procedures`, `list_names` |
|
||||
| 3 | 手掛かりと実行コードを接続 | `find_xrefs_to_name`, `xrefs`, `procedure_callers` |
|
||||
| 4 | 制御フローを復元 | `get_call_graph`, `procedure_callees`, `procedure_info` |
|
||||
| 5 | 関連する処理を逆コンパイル | `procedure_pseudo_code`, `procedure_assembly`, `batch_decompile` |
|
||||
| 6 | プロジェクトに機能を実装する | 技術スタック、プロダクト、要件に合わせたコード |
|
||||
|
||||
REA は手順 1〜5 のバイナリ解析を処理し、手順 6 はエージェントの通常の編集・テストツールが行います。
|
||||
|
||||
## エージェントにできること
|
||||
|
||||
- ソースコードがない機能の仕組みを説明する。
|
||||
- アプリの認証、保存、更新、ネットワークフローを復元する。
|
||||
- 非公開の形式やインターフェースを文書化できる構造を回収する。
|
||||
- 文字列やシンボルから疑わしい動作の実装コードまで追跡する。
|
||||
- 同じセッションで 2 バージョンを切り替え、実装経路を比較する。
|
||||
- 気になる機能を調査し、自分のプロダクトに合わせた形で実装する。
|
||||
- 復元した動作をプロダクト機能、テスト、移行ノート、移植、相互運用できる代替実装へ変換する。
|
||||
- Swift / Objective-C のメタデータを解析する。
|
||||
- Hopper に名前、コメント、ブックマークを残し、人間とエージェントの調査を共有する。
|
||||
|
||||
## 68 ツールのワークベンチ
|
||||
|
||||
| ツール群 | 数 | 例 |
|
||||
| ---------------------- | --: | ------------------------------------------------------------------------------------------------------------------- |
|
||||
| バイナリ調査 | 33 | プロシージャ、疑似コード、アセンブリ、文字列、名前、セグメント、callers、callees、xrefs、注釈 |
|
||||
| 合成解析 | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, Swift / ObjC 検出 |
|
||||
| macOS ネイティブ | 5 | Mach-O メタデータ、署名、plist、アーキテクチャ、Swift デマングル。Hopper 起動不要 |
|
||||
| アーティファクトグラフ | 2 | ディレクトリ、ZIP/APK/IPA、ASAR の決定的一覧と明示選択されたトランザクション抽出 |
|
||||
| バイナリセッション | 18 | `open_binary`、`binary_session`、証拠バンドル、プロセス／成果物／関数比較、残余未知レジストリ |
|
||||
|
||||
## 他のコーディングエージェントで使う
|
||||
|
||||
Setup は現在 Claude Desktop と Cursor を自動設定します。ローカル MCP サーバーに対応するコーディングエージェントなら、次の設定で REA を利用できます。
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 仕組み
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Agent["コーディングエージェント"] --> REA["REA<br/>CLI + MCP"]
|
||||
Terminal["ターミナル"] --> REA
|
||||
REA --> Hopper["解析エンジン"]
|
||||
Hopper --> App["あなたのアプリ"]
|
||||
```
|
||||
|
||||
CLI と MCP サーバーは同じ解析エンジンを使います。ターミナルのコマンドは終了時にアプリを閉じ、エージェントのセッションは調査中アプリを開いたままにします。
|
||||
|
||||
## CLI
|
||||
|
||||
上のエージェントワークフローが、REA を使う最も簡単な方法です。ターミナルから一度だけアプリの概要を調べる場合は、次を実行します。
|
||||
|
||||
```bash
|
||||
npx -y rea-agents analyze /Applications/Notes.app
|
||||
```
|
||||
|
||||
直接デコンパイルする方法やその他のオプションは、`npx -y rea-agents --help` で確認できます。
|
||||
|
||||
グローバルな `rea` コマンドとしてもインストールできます。
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea --help
|
||||
rea mcp
|
||||
```
|
||||
|
||||
REA は Mac の `.app` フォルダーを直接開けます。エージェントがアプリを見つけられない場合は、インストール場所を伝えてください。
|
||||
|
||||
## Hopper アプリの動作
|
||||
|
||||
REA は必要なときに Hopper を起動します。Hopper のランチャーは内部でアプリをアクティブ化するため、ターゲットを開くと Hopper が前面に出る場合があります。REA はバックグラウンド起動を要求しますが、常に背面に留まる保証はありません。
|
||||
|
||||
明示的な形式・アーキテクチャ引数により一般的な FAT / ARM 選択ダイアログを避けますが、別の Hopper / macOS ダイアログは人の操作を必要とする場合があります。セッションを閉じるとブリッジとソケットを削除しますが、ユーザーが利用中の Hopper は終了しません。
|
||||
|
||||
## セキュリティモデル
|
||||
|
||||
各セッションはランダムな capability token と現在のユーザーだけが使える Unix ソケットを使用します。これはサンドボックスではなく、同じユーザー権限で動作する悪意あるプロセスを防御しません。信頼できないバイナリの解析は、現在の macOS ユーザー権限で Hopper に委譲されます。脆弱性は [SECURITY.md](SECURITY.md) の非公開手順で報告してください。
|
||||
|
||||
## FAQ
|
||||
|
||||
<details><summary><strong>Hopper を先に起動する必要がありますか？</strong></summary>
|
||||
|
||||
いいえ。REA が必要時に起動します。すでに起動している Hopper にも対応します。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA に Hopper は含まれますか？</strong></summary>
|
||||
|
||||
含まれません。Setup で Hopper をインストールできますが、Hopper は独自のライセンスを持つ別製品です。REA は CLI、MCP サーバー、エージェント向けワークフローを提供します。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>バイナリはアップロードされますか？</strong></summary>
|
||||
|
||||
REA にホスト型解析サービスはありません。ローカル Unix ソケット経由で Hopper を操作します。エージェントやモデル提供者のデータポリシーは別途確認してください。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>元のソースコードを復元できますか？</strong></summary>
|
||||
|
||||
保証できません。REA は疑似コード、アセンブリ、シンボル、文字列、メタデータ、関係を提供し、エージェントが観察した動作を説明または互換再現できるようにします。
|
||||
|
||||
</details>
|
||||
|
||||
## 開発
|
||||
|
||||
開発環境、アーキテクチャ、テスト、リリース手順は [CONTRIBUTING.md](CONTRIBUTING.md) を参照してください。
|
||||
|
||||
## ライセンス
|
||||
|
||||
[MIT](LICENSE)
|
||||
+247
@@ -0,0 +1,247 @@
|
||||
<div align="center">
|
||||
|
||||
[English](README.md) · [简体中文](README_zh.md) · [日本語](README_ja.md) · **한국어** · [العربية](README_ar.md)
|
||||
|
||||
# REA: 무엇이든 리버스 엔지니어링
|
||||
|
||||
### 코딩 에이전트가 무엇이든 리버스 엔지니어링할 수 있게 하는 하나의 CLI 및 MCP 서버
|
||||
|
||||
**마음에 드는 기능을 찾고, 작동 방식을 이해하고, 원하는 방식으로 구현하세요.**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](#68개-도구로-구성된-워크벤치)
|
||||
[](https://nodejs.org/)
|
||||
[](LICENSE)
|
||||
|
||||
[빠른 시작](#빠른-시작) · [바이너리에서 동작까지](#바이너리에서-동작까지) · [68개 도구](#68개-도구로-구성된-워크벤치) · [작동 방식](#작동-방식) · [FAQ](#faq)
|
||||
|
||||
<br />
|
||||
|
||||
<code>curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash</code>
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
어떤 앱에서 마음에 드는 기능을 자신의 제품에도 넣고 싶나요? 소스 코드가 없어도 앱을 코딩 에이전트에 전달할 수 있습니다. REA를 사용하면 에이전트가 기능을 조사하고 작동 방식을 이해한 뒤, 사용자의 기술 스택, 디자인, 요구 사항에 맞는 버전을 구현할 수 있습니다.
|
||||
|
||||
REA는 이 과정을 하나의 CLI 및 MCP 서버로 제공합니다. 에이전트는 컴파일된 앱을 조사하고 기능의 작동 방식을 추적한 뒤 알아낸 내용을 일반 코딩 작업에 활용할 수 있습니다. 복잡한 리버스 엔지니어링 도구는 REA가 하나의 인터페이스 뒤에서 처리합니다.
|
||||
|
||||
## 에이전트에게 바로 요청하기
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
그런 다음 요청하세요.
|
||||
|
||||
```text
|
||||
REA를 설정하고 메모 앱을 리버스 엔지니어링해 주세요. 검색 기능의 작동 방식과
|
||||
그렇게 판단한 이유를 보여 주고, 제 프로젝트에 비슷한 기능을 구현해 주세요.
|
||||
```
|
||||
|
||||
메모 앱은 예시일 뿐입니다. 이해하고 싶은 앱을 지정하거나 먼저 개요부터 살펴보라고 요청할 수 있습니다.
|
||||
|
||||
## 바이너리에서 동작까지
|
||||
|
||||
| 디컴파일 | 이해 | 재현 |
|
||||
| ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
||||
| 네이티브 앱이나 실행 파일에서 프로시저, 의사 코드, 어셈블리, 문자열, 심볼, 세그먼트, 메타데이터를 복구합니다. | 호출자, 피호출자, 상호 참조, 호출 그래프를 따라 기능이나 알고리즘의 실제 동작을 설명합니다. | 에이전트가 배운 내용을 사용자의 기술 스택, 화면, 요구 사항에 맞는 제품 기능으로 만듭니다. |
|
||||
|
||||
REA는 분석을 바이너리 증거에 근거하게 합니다. 원본 소스 코드를 복원하거나 앱 전체를 자동으로 복제한다고 주장하지 않습니다.
|
||||
|
||||
## REA를 사용하는 이유
|
||||
|
||||
| | |
|
||||
| ------------------ | ---------------------------------------------------------------------------------- |
|
||||
| **에이전트용** | 컴파일된 앱에 관해 질문하고 추측 대신 증거를 수집하게 합니다. |
|
||||
| **CLI와 MCP** | 터미널과 코딩 에이전트에서 동일한 리버스 엔지니어링 기능을 사용합니다. |
|
||||
| **복잡성 처리** | 도구 설정, 앱 열기, 조사 유지, 작업 후 정리를 REA가 처리합니다. |
|
||||
| **전체 조사 과정** | 첫 개요에서 의사 코드, 호출 관계, 타입, 구현 단서까지 이어서 조사합니다. |
|
||||
| **로컬 분석** | 분석은 Mac에서 실행되며 REA는 바이너리를 호스팅 분석 서비스에 업로드하지 않습니다. |
|
||||
| **컨텍스트 유지** | 질문마다 분석을 처음부터 시작하지 않고 여러 바이너리를 연속으로 조사합니다. |
|
||||
|
||||
## 빠른 시작
|
||||
|
||||
### 코딩 에이전트에서 시작하기(권장)
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
에이전트에게 REA 설정을 요청하세요. Mac을 확인하고 필요한 설치 내용을 설명한 뒤 승인을 요청하며 시스템 프롬프트를 안내합니다. 전체 도구를 불러오기 위해 재시작하라고 하면 설정 후 에이전트를 재시작하세요.
|
||||
|
||||
### 시작하기 전에
|
||||
|
||||
- macOS 12 이상
|
||||
- Ubuntu 24.04+, Fedora 41+ 또는 64비트 Arch Linux
|
||||
- Node.js 22.19+ 또는 24.11+ 및 Node와 함께 제공되는 npm
|
||||
|
||||
`rea setup`은 전체 변경 계획을 표시하고 확인 후 적용합니다. Homebrew, Node.js 또는 npm을 설치하거나 업데이트하지 않습니다. [Hopper](https://www.hopperapp.com/)가 없으면 공식 패키지 설치를 제안합니다. Hopper는 별도 라이선스가 필요한 독립 소프트웨어입니다.
|
||||
|
||||
#### Linux 설치 및 문제 해결
|
||||
|
||||
Ubuntu 24.04+, Fedora 41+, 64비트 Arch Linux에서 REA는 공식 DEB, RPM 또는 Arch 패키지를 내려받아 게시된 크기와 체크섬을 검증한 뒤 `apt-get`, `dnf` 또는 `pacman`으로 의존성을 설치합니다. root가 아니면 `pkexec`가 시스템 승인 창을 표시합니다. REA는 `sudo`를 호출하지 않습니다.
|
||||
|
||||
기본 실행 파일은 `/opt/hopper/bin/Hopper`입니다. 다른 위치에 설치했다면 `HOPPER_LAUNCHER_PATH`를 설정하세요. Doctor가 분석 엔진 누락을 보고하면 `ldd /opt/hopper/bin/Hopper | grep 'not found'`를 실행하고 누락된 라이브러리를 설치한 다음 `rea setup`을 다시 실행하세요. 실제 분석에는 `DISPLAY` 또는 `WAYLAND_DISPLAY`와 활성화된 Hopper 라이선스가 필요합니다. `~/.local/bin`도 `PATH`에 추가해야 합니다.
|
||||
|
||||
```bash
|
||||
# 1. REA 설정
|
||||
curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash
|
||||
npx -y rea-agents setup
|
||||
```
|
||||
|
||||
macOS나 설치 프로그램이 확인을 요청하면 해당 절차를 완료하고 같은 명령을 다시 실행하세요.
|
||||
|
||||
### 2. 코딩 에이전트 재시작
|
||||
|
||||
Setup은 감지된 Claude Desktop과 Cursor를 자동으로 구성합니다. REA를 불러오도록 앱을 재시작하세요.
|
||||
|
||||
### 3. 에이전트에게 요청
|
||||
|
||||
앱 이름만 말해도 됩니다. 코딩 에이전트가 앱을 찾아 REA에 필요한 프로그램 파일을 전달합니다.
|
||||
|
||||
```text
|
||||
REA로 메모 앱을 리버스 엔지니어링해 주세요. 검색 기능의 작동 방식과 증거를 보여 주고,
|
||||
SQLite를 사용해 제 프로젝트에 비슷한 기능을 구현해 주세요.
|
||||
```
|
||||
|
||||
문제가 있으면 다음을 실행하세요.
|
||||
|
||||
```bash
|
||||
npx -y rea-agents doctor
|
||||
rea uninstall
|
||||
rea uninstall --purge-data
|
||||
```
|
||||
|
||||
## 하나의 프롬프트로 전체 조사
|
||||
|
||||
```text
|
||||
메모 앱을 리버스 엔지니어링해 오프라인 검색 기능의 작동 방식을 설명한 다음,
|
||||
TypeScript와 SQLite를 사용해 제 프로젝트에 맞는 버전을 구현해 주세요.
|
||||
```
|
||||
|
||||
| 단계 | 에이전트 작업 | REA 도구 |
|
||||
| ---: | ----------------------- | ---------------------------------------------------------------- |
|
||||
| 1 | 바이너리 열기 및 식별 | `open_binary`, `binary_overview` |
|
||||
| 2 | 오프라인 검색 단서 찾기 | `search_strings`, `search_procedures`, `list_names` |
|
||||
| 3 | 단서를 실행 코드에 연결 | `find_xrefs_to_name`, `xrefs`, `procedure_callers` |
|
||||
| 4 | 제어 흐름 재구성 | `get_call_graph`, `procedure_callees`, `procedure_info` |
|
||||
| 5 | 관련 루틴 디컴파일 | `procedure_pseudo_code`, `procedure_assembly`, `batch_decompile` |
|
||||
| 6 | 프로젝트에 기능 구현 | 기술 스택, 제품, 요구 사항에 맞는 코드 |
|
||||
|
||||
REA는 1–5단계의 바이너리 분석을 처리합니다. 6단계는 에이전트가 일반 파일 편집 및 테스트 도구로 수행합니다.
|
||||
|
||||
## 에이전트가 할 수 있는 일
|
||||
|
||||
- 소스 코드가 없는 기능의 동작을 설명합니다.
|
||||
- 앱의 인증, 저장소, 업데이트 또는 네트워크 흐름을 재구성합니다.
|
||||
- 문서화되지 않은 형식이나 인터페이스의 구조를 복구합니다.
|
||||
- 문자열이나 심볼에서 의심스러운 동작을 구현한 코드까지 추적합니다.
|
||||
- 한 세션에서 두 앱 버전을 전환하며 구현 경로를 비교합니다.
|
||||
- 마음에 드는 기능을 조사하고 자신의 제품에 맞는 버전으로 구현합니다.
|
||||
- 복구한 동작을 제품 기능, 테스트, 마이그레이션 문서, 포팅 또는 상호 운용 가능한 대체품으로 변환합니다.
|
||||
- Swift 및 Objective-C 메타데이터를 분석합니다.
|
||||
- Hopper에 이름, 주석, 북마크를 남겨 사람과 에이전트의 분석을 연결합니다.
|
||||
|
||||
## 68개 도구로 구성된 워크벤치
|
||||
|
||||
| 도구 그룹 | 수 | 예시 |
|
||||
| ------------------- | --: | -------------------------------------------------------------------------------------------------------------------- |
|
||||
| 바이너리 검사 | 33 | 프로시저, 의사 코드, 어셈블리, 문자열, 이름, 세그먼트, callers, callees, xrefs, 주석 |
|
||||
| 합성 분석 | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`, Swift 및 ObjC 탐색 |
|
||||
| macOS 네이티브 도구 | 5 | Mach-O 메타데이터, 코드 서명, plist, 아키텍처, Swift 디맹글링. Hopper 실행 불필요 |
|
||||
| 아티팩트 그래프 | 2 | 디렉터리, ZIP/APK/IPA, ASAR 결정적 인벤토리와 명시적으로 선택한 트랜잭션 추출 |
|
||||
| 바이너리 세션 | 18 | `open_binary`, `binary_session`, 증거 번들, 프로세스·아티팩트·함수 비교, 잔여 미지수 레지스트리 |
|
||||
|
||||
## 다른 코딩 에이전트에서 사용하기
|
||||
|
||||
Setup은 현재 Claude Desktop과 Cursor를 자동으로 구성합니다. 로컬 MCP 서버를 지원하는 코딩 에이전트라면 다음 설정으로 REA를 사용할 수 있습니다.
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 작동 방식
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Agent["코딩 에이전트"] --> REA["REA<br/>CLI + MCP"]
|
||||
Terminal["터미널"] --> REA
|
||||
REA --> Hopper["분석 엔진"]
|
||||
Hopper --> App["사용자의 앱"]
|
||||
```
|
||||
|
||||
CLI와 MCP 서버는 같은 분석 엔진을 사용합니다. 터미널 명령은 끝날 때 앱을 닫고, 에이전트 세션은 조사하는 동안 앱을 열어 둡니다.
|
||||
|
||||
## CLI
|
||||
|
||||
위의 에이전트 워크플로가 REA를 사용하는 가장 쉬운 방법입니다. 터미널에서 앱을 한 번 빠르게 살펴보려면 다음을 실행하세요.
|
||||
|
||||
```bash
|
||||
npx -y rea-agents analyze /Applications/Notes.app
|
||||
```
|
||||
|
||||
직접 디컴파일하는 방법과 기타 옵션은 `npx -y rea-agents --help`에서 확인할 수 있습니다.
|
||||
|
||||
전역 `rea` 명령으로 설치할 수도 있습니다.
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea --help
|
||||
rea mcp
|
||||
```
|
||||
|
||||
REA는 Mac의 `.app` 폴더를 직접 열 수 있습니다. 에이전트가 앱을 찾지 못하면 설치 위치를 알려 주세요.
|
||||
|
||||
## Hopper 앱 동작
|
||||
|
||||
REA는 필요할 때 Hopper를 시작합니다. Hopper 런처는 내부적으로 앱을 활성화하므로 대상을 열 때 Hopper가 다른 창 앞으로 나타날 수 있습니다. REA는 백그라운드 시작을 요청하지만 항상 뒤에 머무르는 것을 보장할 수 없습니다.
|
||||
|
||||
명시적인 형식과 아키텍처 인자로 일반적인 FAT/ARM 선택 대화상자를 피하지만, 다른 Hopper 또는 macOS 대화상자에는 사람이 응답해야 할 수 있습니다. 세션 종료 시 브리지와 소켓 디렉터리를 제거하지만 사용자가 작업 중인 Hopper 앱은 종료하지 않습니다.
|
||||
|
||||
## 보안 모델
|
||||
|
||||
각 세션은 무작위 capability token과 현재 사용자 전용 Unix 소켓을 사용합니다. 이는 샌드박스가 아니며 같은 사용자로 실행되는 악성 프로세스를 방어하지 않습니다. 신뢰할 수 없는 바이너리를 열면 현재 macOS 사용자 권한으로 Hopper가 분석합니다. 취약점은 [SECURITY.md](SECURITY.md)의 비공개 절차로 신고하세요.
|
||||
|
||||
## FAQ
|
||||
|
||||
<details><summary><strong>Hopper를 미리 실행해야 하나요?</strong></summary>
|
||||
|
||||
아니요. REA가 필요할 때 시작하며 이미 실행 중인 Hopper도 지원합니다.
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA에 Hopper가 포함되나요?</strong></summary>
|
||||
|
||||
아니요. Setup이 Hopper를 설치할 수 있지만 Hopper는 자체 라이선스가 필요한 별도 소프트웨어입니다. REA는 CLI, MCP 서버, 에이전트용 워크플로를 제공합니다.
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>바이너리가 업로드되나요?</strong></summary>
|
||||
|
||||
REA에는 호스팅 분석 서비스가 없습니다. 로컬 Unix 소켓을 통해 Hopper를 조작합니다. 에이전트나 모델 공급자의 데이터 정책은 별도로 확인하세요.
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>원본 소스 코드를 복구할 수 있나요?</strong></summary>
|
||||
|
||||
보장할 수 없습니다. REA는 의사 코드, 어셈블리, 심볼, 문자열, 메타데이터 및 관계를 제공하여 에이전트가 관찰된 동작을 설명하거나 호환되게 재현하도록 돕습니다.
|
||||
|
||||
</details>
|
||||
|
||||
## 개발
|
||||
|
||||
개발 환경, 아키텍처, 테스트, 릴리스 지침은 [CONTRIBUTING.md](CONTRIBUTING.md)를 참고하세요.
|
||||
|
||||
## 라이선스
|
||||
|
||||
[MIT](LICENSE)
|
||||
+249
@@ -0,0 +1,249 @@
|
||||
<div align="center">
|
||||
|
||||
[English](README.md) · **简体中文** · [日本語](README_ja.md) · [한국어](README_ko.md) · [العربية](README_ar.md)
|
||||
|
||||
# REA：逆向工程一切
|
||||
|
||||
### 一个 CLI 与 MCP 服务器，让编程智能体逆向工程任何程序
|
||||
|
||||
**看到喜欢的功能。理解它的原理。按照你的方式实现。**
|
||||
|
||||
[](https://www.npmjs.com/package/rea-agents)
|
||||
[](https://github.com/morluto/rea/actions/workflows/ci.yml)
|
||||
[](#68-个工具组成的工作台)
|
||||
[](https://nodejs.org/)
|
||||
[](LICENSE)
|
||||
|
||||
[快速开始](#快速开始) · [从二进制到行为](#从二进制到行为) · [68 个工具](#68-个工具组成的工作台) · [工作原理](#工作原理) · [常见问题](#常见问题)
|
||||
|
||||
<br />
|
||||
|
||||
<code>curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash</code>
|
||||
|
||||
</div>
|
||||
|
||||
---
|
||||
|
||||
看到某个应用中想加入自己产品的功能？即使没有源代码，也可以把应用交给编程智能体。借助 REA，智能体能够调查该功能、理解其工作原理，并按照你的技术栈、设计和需求构建适合你产品的版本。
|
||||
|
||||
REA 通过一个 CLI 与 MCP 服务器实现这套流程。智能体可以检查编译后的应用、追踪功能的工作方式，并把了解到的内容用于日常编码。REA 在一个接口背后处理复杂的逆向工程工具。
|
||||
|
||||
## 直接询问智能体
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
然后说：
|
||||
|
||||
```text
|
||||
设置 REA 并逆向工程“备忘录”应用。解释搜索功能的工作方式，
|
||||
说明你是如何得出结论的，然后为我的项目构建类似功能。
|
||||
```
|
||||
|
||||
“备忘录”只是示例。你可以说出任何想了解的应用，也可以让智能体先从概览开始。
|
||||
|
||||
## 从二进制到行为
|
||||
|
||||
| 反编译 | 理解 | 重建 |
|
||||
| ---------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
|
||||
| 打开原生应用或可执行文件，恢复过程、伪代码、汇编、字符串、符号、段和元数据。 | 沿调用者、被调用者、交叉引用和调用图追踪，直到智能体能够解释功能或算法的实际工作方式。 | 将智能体学到的内容变成适合你的技术栈、界面和需求的产品功能。 |
|
||||
|
||||
REA 让调查始终以二进制证据为依据。它不会声称能恢复原始源代码，也不会自动克隆整个应用。
|
||||
|
||||
## 为什么选择 REA
|
||||
|
||||
| | |
|
||||
| ---------------- | -------------------------------------------------------------- |
|
||||
| **为智能体设计** | 直接询问编译后应用的行为，让智能体搜集证据，而不是猜测。 |
|
||||
| **CLI 与 MCP** | 在终端或编程智能体中使用同一套逆向工程能力。 |
|
||||
| **处理复杂流程** | REA 负责工具设置、打开应用、维持调查过程，并在完成后清理资源。 |
|
||||
| **完整工作流** | 从初步概览推进到伪代码、调用关系、类型和实现线索。 |
|
||||
| **本地运行** | 分析在你的 Mac 上运行；REA 不会把二进制上传到托管式分析服务。 |
|
||||
| **保留上下文** | 连续调查多个二进制文件，无需为每个问题重新开始整个分析过程。 |
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 使用编程智能体（推荐）
|
||||
|
||||
```bash
|
||||
npx skills add morluto/rea
|
||||
```
|
||||
|
||||
让智能体设置 REA。它会检查你的 Mac，说明需要安装的内容，先征得同意，并引导你处理系统提示。如果智能体要求重启以加载完整工具，请在设置后重启。
|
||||
|
||||
### 开始之前
|
||||
|
||||
- macOS 12 或更高版本
|
||||
- Ubuntu 24.04+、Fedora 41+ 或 64 位 Arch Linux
|
||||
- Node.js 22.19+ 或 24.11+，以及随 Node 安装的 npm
|
||||
|
||||
`rea setup` 会先显示完整变更计划并请求确认。它不会安装或更新 Homebrew、Node.js 或 npm。缺少 [Hopper](https://www.hopperapp.com/) 时，Setup 会提议安装官方软件包；Hopper 是需要单独许可证的独立软件。
|
||||
|
||||
#### Linux 安装与故障排除
|
||||
|
||||
在 Ubuntu 24.04+、Fedora 41+ 和 64 位 Arch Linux 上，REA 会下载 Hopper 官方的 DEB、RPM 或 Arch 软件包，验证发布方提供的大小和校验和，然后通过 `apt-get`、`dnf` 或 `pacman` 安装依赖。非 root 运行时，`pkexec` 会显示系统授权提示；REA 不调用 `sudo`。
|
||||
|
||||
默认启动器是 `/opt/hopper/bin/Hopper`。自定义安装可设置 `HOPPER_LAUNCHER_PATH`。如果 Doctor 报告缺少分析引擎，请运行 `ldd /opt/hopper/bin/Hopper | grep 'not found'`，安装缺少的系统库后重新运行 `rea setup`。真实分析需要 `DISPLAY` 或 `WAYLAND_DISPLAY` 以及已激活的 Hopper 许可证。Linux 上还应确保 `~/.local/bin` 位于 `PATH` 中。
|
||||
|
||||
```bash
|
||||
# 1. 安装和配置 REA
|
||||
curl -fsSL https://raw.githubusercontent.com/morluto/rea/main/install.sh | bash
|
||||
npx -y rea-agents setup
|
||||
```
|
||||
|
||||
如果 macOS 或安装程序要求确认，请完成提示，然后再次运行同一命令。
|
||||
|
||||
### 2. 重启编程智能体
|
||||
|
||||
Setup 会自动配置检测到的 Claude Desktop 和 Cursor。重启应用，让它加载 REA。
|
||||
|
||||
### 3. 直接询问智能体
|
||||
|
||||
你可以直接说出应用名称。编程智能体会找到应用，并把 REA 所需的程序文件交给它。
|
||||
|
||||
```text
|
||||
使用 REA 逆向工程“备忘录”应用。解释搜索功能的工作方式，展示证据，
|
||||
然后使用 SQLite 为我的项目构建一个类似功能。
|
||||
```
|
||||
|
||||
如果遇到问题，请运行：
|
||||
|
||||
```bash
|
||||
npx -y rea-agents doctor
|
||||
rea uninstall
|
||||
rea uninstall --purge-data
|
||||
```
|
||||
|
||||
## 一个提示词，完成一次完整调查
|
||||
|
||||
```text
|
||||
逆向工程“备忘录”应用，找到离线搜索功能的工作方式，解释其控制流，
|
||||
并使用 TypeScript 和 SQLite 为我的项目构建一个版本。
|
||||
```
|
||||
|
||||
| 步骤 | 智能体的操作 | REA 工具 |
|
||||
| ---: | ---------------------- | ---------------------------------------------------------------- |
|
||||
| 1 | 打开并识别二进制文件 | `open_binary`, `binary_overview` |
|
||||
| 2 | 搜索可能的离线搜索线索 | `search_strings`, `search_procedures`, `list_names` |
|
||||
| 3 | 将线索连接到可执行代码 | `find_xrefs_to_name`, `xrefs`, `procedure_callers` |
|
||||
| 4 | 重建相关控制流 | `get_call_graph`, `procedure_callees`, `procedure_info` |
|
||||
| 5 | 反编译相关程序 | `procedure_pseudo_code`, `procedure_assembly`, `batch_decompile` |
|
||||
| 6 | 在你的项目中构建该功能 | 适合你的技术栈、产品和需求的代码 |
|
||||
|
||||
REA 负责第 1–5 步中的二进制分析。第 6 步由智能体使用其常规文件编辑与测试工具完成。
|
||||
|
||||
## 智能体可以完成什么
|
||||
|
||||
- 在没有源代码时解释某项功能的实现方式。
|
||||
- 重建应用的身份验证、存储、更新或网络流程。
|
||||
- 恢复足够的结构，以记录未公开的格式或接口。
|
||||
- 从字符串或符号追踪到实现可疑行为的代码。
|
||||
- 在一个会话中切换两个应用版本并比较实现路径。
|
||||
- 调查你喜欢的功能，并为自己的产品构建量身定制的版本。
|
||||
- 将恢复的行为转换为产品功能、测试、迁移说明、移植代码或互操作替代品。
|
||||
- 分析 Swift 和 Objective-C 元数据。
|
||||
- 在 Hopper 中留下名称、注释与书签，使人与智能体的分析互相增强。
|
||||
|
||||
## 68 个工具组成的工作台
|
||||
|
||||
| 工具类别 | 数量 | 示例 |
|
||||
| -------------- | ---: | -------------------------------------------------------------------------------------------------------------------- |
|
||||
| 二进制检查 | 33 | 过程、伪代码、汇编、字符串、名称、段、调用者、被调用者、交叉引用、注释 |
|
||||
| 组合分析 | 10 | `binary_overview`, `analyze_function`, `batch_decompile`, `get_call_graph`, `find_xrefs_to_name`、Swift 与 ObjC 发现 |
|
||||
| macOS 原生工具 | 5 | Mach-O 元数据、代码签名、plist、架构与 Swift 符号还原，无需启动 Hopper |
|
||||
| 制品图谱 | 2 | 确定性目录、ZIP/APK/IPA 与 ASAR 清单，以及显式选择的事务式提取 |
|
||||
| 二进制会话 | 18 | `open_binary`、`binary_session`、证据包、进程、制品与函数比较、残余未知项注册表 |
|
||||
|
||||
## 与其他编程智能体一起使用
|
||||
|
||||
Setup 目前会自动配置 Claude Desktop 和 Cursor。任何支持本地 MCP 服务器的编程智能体都可以使用以下配置连接 REA。
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 工作原理
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Agent["编程智能体"] --> REA["REA<br/>CLI + MCP"]
|
||||
Terminal["终端"] --> REA
|
||||
REA --> Hopper["分析引擎"]
|
||||
Hopper --> App["你的应用"]
|
||||
```
|
||||
|
||||
CLI 与 MCP 服务器使用相同的分析引擎。终端命令完成后会关闭应用；智能体会话则会在调查期间保持应用打开。
|
||||
|
||||
## CLI
|
||||
|
||||
上面的智能体工作流是使用 REA 最简单的方式。如果只想在终端中快速了解一个应用：
|
||||
|
||||
```bash
|
||||
npx -y rea-agents analyze /Applications/Notes.app
|
||||
```
|
||||
|
||||
运行 `npx -y rea-agents --help` 查看直接反编译和其他选项。
|
||||
|
||||
也可以全局安装 `rea` 命令：
|
||||
|
||||
```bash
|
||||
npm install --global rea-agents
|
||||
rea --help
|
||||
rea mcp
|
||||
```
|
||||
|
||||
REA 可以直接打开 Mac 的 `.app` 文件夹。如果智能体找不到应用，请告诉它应用安装在哪里。
|
||||
|
||||
## Hopper 应用行为
|
||||
|
||||
REA 会在需要时启动 Hopper，无需预先运行。Hopper 启动器内部会激活应用，因此打开目标时 Hopper 可能出现在其他窗口前。REA 会请求 macOS 在后台启动 Hopper，但无法保证窗口始终位于后台。
|
||||
|
||||
REA 会推导明确的格式和架构参数，以避免常见的 FAT 与 ARM 选择对话框。其他 Hopper 或 macOS 对话框仍可能需要人工响应。关闭 REA 会话会终止桥并删除私有套接字目录，但不会退出用户正在使用的 Hopper 应用。
|
||||
|
||||
## 安全模型
|
||||
|
||||
每个桥会话都使用随机能力令牌和仅限当前用户的 Unix 套接字。协议消息有大小限制，面向调用者的错误不会暴露启动器 stderr 或内部异常原因。
|
||||
|
||||
这不是沙箱，也无法防御以同一 macOS 用户身份运行的恶意进程。打开不可信二进制文件会让 Hopper 以当前用户权限进行解析和分析。请按照 [SECURITY.md](SECURITY.md) 中的私密流程报告漏洞。
|
||||
|
||||
## 常见问题
|
||||
|
||||
<details><summary><strong>Hopper 是否需要提前运行？</strong></summary>
|
||||
|
||||
不需要。REA 会在操作需要时启动 Hopper，也支持已经运行的 Hopper。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA 是否包含 Hopper？</strong></summary>
|
||||
|
||||
不包含。Setup 可以为你安装 Hopper，但 Hopper 仍是需要单独授权的软件。REA 提供 CLI、MCP 服务器和面向智能体的工作流。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA 会上传我的二进制文件吗？</strong></summary>
|
||||
|
||||
REA 不提供托管分析服务，而是通过本地 Unix 套接字把操作交给 Hopper。你的智能体或模型服务商可能有自己的数据政策，请单独核查。
|
||||
|
||||
</details>
|
||||
|
||||
<details><summary><strong>REA 能恢复原始源代码吗？</strong></summary>
|
||||
|
||||
不能保证。REA 提供伪代码、汇编、符号、字符串、元数据和关系，智能体可据此解释或兼容地重建观察到的行为。
|
||||
|
||||
</details>
|
||||
|
||||
## 开发
|
||||
|
||||
开发环境、架构、测试和发布说明请参阅 [CONTRIBUTING.md](CONTRIBUTING.md)。
|
||||
|
||||
## 许可证
|
||||
|
||||
[MIT](LICENSE)
|
||||
+4
-33
@@ -6,45 +6,16 @@ Security fixes are provided for the latest published version of REA. Upgrade to
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Do not disclose vulnerabilities, proof-of-concept exploits, credentials,
|
||||
private binaries, Hopper documents, or private Ghidra projects in a public
|
||||
issue.
|
||||
Do not disclose vulnerabilities, proof-of-concept exploits, credentials, private binaries, or Hopper documents in a public issue.
|
||||
|
||||
Use **Report a vulnerability** in the repository's **Security** tab to submit the report privately. Do not open a public issue to request a private contact channel.
|
||||
|
||||
Include the affected REA version, operating system/distribution, selected
|
||||
provider and version, impact, reproduction conditions, and any suggested
|
||||
mitigation in the private report. Remove credentials, proprietary binaries,
|
||||
Hopper documents, Ghidra project contents, socket capability tokens, and
|
||||
unrelated local paths from logs or examples.
|
||||
Include the affected REA version, operating system/distribution, Hopper version, impact, reproduction conditions, and any suggested mitigation in the private report. Remove credentials, proprietary binaries, Hopper documents, socket capability tokens, and unrelated local paths from logs or examples.
|
||||
|
||||
The maintainer will acknowledge the report, assess its scope, and coordinate
|
||||
remediation and disclosure timing with the reporter. Please allow time for
|
||||
provider-dependent behavior to be reproduced safely.
|
||||
The maintainer will acknowledge the report, assess its scope, and coordinate remediation and disclosure timing with the reporter. Please allow time for Hopper-dependent behavior to be reproduced safely.
|
||||
|
||||
## Security boundary
|
||||
|
||||
REA authenticates its Hopper and Ghidra bridges with per-session random
|
||||
capability tokens. POSIX uses current-user Unix sockets; Windows Ghidra uses
|
||||
authenticated loopback TCP with a private NTFS runtime whose DACL admits only
|
||||
the current user and SYSTEM. See [Windows Ghidra P0](docs/windows-ghidra-p0.md#private-runtime-transport-and-lifecycle).
|
||||
Tokens are transferred through private session descriptors rather than process
|
||||
arguments or environment variables.
|
||||
This is not a sandbox and does not protect against malicious processes already
|
||||
running as the same operating-system user. Opening an untrusted binary
|
||||
delegates parsing and analysis to the selected local provider with that user's
|
||||
permissions.
|
||||
|
||||
Ghidra sessions use a packaged Java `HeadlessScript`, an isolated temporary
|
||||
project, and private home, cache, and runtime directories. REA imports only the
|
||||
requested target, bounds startup and protocol messages, requests bounded CPU
|
||||
and heap settings, and deletes the temporary project when the session closes.
|
||||
These controls limit accidental persistence and resource use; they do not make
|
||||
Ghidra a sandbox. REA does not open or modify user-owned Ghidra projects.
|
||||
|
||||
`GHIDRA_INSTALL_DIR` and optional `JAVA_HOME` identify user-managed
|
||||
installations. Setup can add those values to detected client configuration only
|
||||
after showing its plan and receiving approval. REA does not download, install,
|
||||
upgrade, or modify Ghidra or Java.
|
||||
REA authenticates each local bridge session with a random capability token and a current-user Unix socket. This is not a sandbox and does not protect against malicious processes already running as the same operating-system user. Opening an untrusted binary delegates parsing and analysis to Hopper with that user's permissions.
|
||||
|
||||
On Linux, REA accepts release metadata only from Hopper's HTTPS endpoint, restricts package URLs to Hopper's public download origin, bounds the package size, and compares the downloaded bytes with Hopper's published checksum before requesting installation. The published SHA-1 value is a corruption check, not a modern package signature; HTTPS origin validation remains part of the trust boundary. Dependency installation is delegated without shell evaluation to `apt-get`, `dnf`, or `pacman`, directly as root or through `pkexec`. REA never invokes `sudo`.
|
||||
@@ -1,135 +0,0 @@
|
||||
# Third-party notices
|
||||
|
||||
REA-authored code is licensed under [MIT](LICENSE). Third-party material retains
|
||||
its own copyright, license, attribution, and NOTICE requirements. REA's MIT
|
||||
license grants no rights to proprietary analysis tools. Project and product
|
||||
names identify integrations; they do not imply affiliation or endorsement.
|
||||
|
||||
This inventory covers the locked npm dependency graph, code or assets derived
|
||||
from upstream projects, and separately obtained engines used by REA. A listed
|
||||
integration is not a statement that its executable is bundled. Upstream
|
||||
components and caller-selected versions may have additional terms; retain the
|
||||
notices supplied with the software you actually obtain or redistribute.
|
||||
|
||||
## Distributed dependencies, derived code, and assets
|
||||
|
||||
| Material | Use and licensing records |
|
||||
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| npm dependencies | [Complete inventory](licenses/npm-dependencies.txt) of every entry in `package-lock.json`, including transitive, optional platform, development, and website dependencies. [Runtime notices](licenses/npm-runtime-notices.txt) preserve the copyright, license, and NOTICE documents from integrity-verified release archives, plus the MIT notice for Rolldown's generated runtime helpers. Development dependencies are inventoried but are not all distributed in REA's npm package. |
|
||||
| NativeAOT parser | REA's derived parser uses the MIT-licensed [ghidra-nativeaot](https://github.com/Washi1337/ghidra-nativeaot/tree/effeb734fc570c32650f88b159608979dc7b423e) work of Washi and the NativeAOT layout definitions in [dotnet/runtime](https://github.com/dotnet/runtime/tree/a3b2d40328be0be3f8dd43f0e5cc925b8827b044). Preserve [Washi's MIT notice](licenses/nativeaot-parser-MIT.txt) and the [.NET Foundation MIT notice](licenses/dotnet-runtime-MIT.txt). |
|
||||
| EVM analysis | `evmole@0.9.3`, MIT, copyright 2023 Maxim Andreev. The published npm archive omits its license document; REA retains the [original upstream license](third_party/evmole/LICENSE) and [provenance](third_party/evmole/README.md). It is also included in the runtime notices. |
|
||||
| Windows native addon | The release addon is compiled with MinGW-w64 and statically links GCC runtime libraries. Preserve the [MinGW-w64 runtime notices](licenses/MinGW-w64-runtime.txt), [GPLv3 text](licenses/GPL-3.0.txt), and [GCC Runtime Library Exception 3.1](licenses/GCC-Runtime-Library-Exception-3.1.txt). The exception governs eligible compiled combinations; it does not relicense the runtime itself under MIT. The Node-API build headers are from Node 22.19.0; its [original license and component notices](licenses/Node-22.19.0-LICENSE.txt) are retained. Node itself is supplied by the user. Snapshot provenance and release-toolchain review requirements are in [licenses/README.md](licenses/README.md). |
|
||||
| Dinosaur website example | Selected excerpts from [wayou/t-rex-runner](https://github.com/wayou/t-rex-runner/tree/5455bfa408ec6b707c7300ff194b7390733a766d), derived from Chromium's dinosaur game. The website retains the [original BSD-3-Clause notices](website/public/assets/licenses/dinosaur.txt), including Chromium authors and wayou. This asset is not part of the npm runtime. |
|
||||
|
||||
Two locked runtime releases, [`clean-git-ref@2.0.1`](https://github.com/TheSavior/clean-git-ref/tree/bf3c8afc8032bcdc6a53634c18b601dc6df478fb) (declared Apache-2.0) and
|
||||
[`minimisted@2.0.1`](https://github.com/kt3k/minimisted/tree/95fdfb8d6efb9546c1117854fe3a58105f6332ce) (declared MIT), omit a full license/copyright document in their
|
||||
published archives and inspected upstream release sources. Their runtime notice
|
||||
entries preserve the declared license and author metadata and include a clearly
|
||||
labeled standard license text. These are documented upstream notice gaps, not
|
||||
recovered copyright notices or a claim that attribution has been fully verified.
|
||||
|
||||
The optional `frida@17.23.2` npm archive and tagged Node binding source also
|
||||
omit a standalone license/copyright document. Its declared
|
||||
`LGPL-2.0 WITH WxWindows-exception-3.1` license is recorded with unchanged
|
||||
[Frida licensing text](licenses/frida-17.23.2-COPYING.txt) and the
|
||||
[GNU Library GPL v2](licenses/LGPL-2.0.txt). These are licensing supplements,
|
||||
not recovered Node binding or prebuild-component copyright notices.
|
||||
|
||||
## Open-source engine credits and licenses
|
||||
|
||||
We thank the maintainers and contributors of the projects below. Their
|
||||
decompilers, binary-analysis engines, debuggers, parsers, and capture tools
|
||||
provide capabilities used through REA's integrations. Each row identifies the
|
||||
upstream project, its own license, and its role in REA. In particular, JADX
|
||||
provides the Android decompiler, while jadx-headless-mcp provides the separate
|
||||
headless integration used by REA; both projects deserve credit.
|
||||
|
||||
These acknowledgments are voluntary thanks. Required upstream copyright and
|
||||
NOTICE texts remain in their original form and are not replaced by this
|
||||
summary. Credits do not imply endorsement or change any license.
|
||||
|
||||
REA's adapters do not grant, replace, or waive these licenses. The upstream
|
||||
executables and their dependency graphs are supplied separately. Git submodules
|
||||
pin inspection sources; they are not included in the npm package.
|
||||
|
||||
| Project | License and source | REA relationship |
|
||||
| ------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Ghidra | [Apache-2.0](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/LICENSE), with the government-work and component qualifications in its [NOTICE](https://github.com/NationalSecurityAgency/ghidra/blob/Ghidra_12.1.4_build/NOTICE) | Bring your own Ghidra 12.1.4. Copies of its [LICENSE](licenses/ghidra-12.1.4-LICENSE.txt), [NOTICE](licenses/ghidra-12.1.4-NOTICE.txt), and [DISCLAIMER](licenses/ghidra-12.1.4-DISCLAIMER.txt) are retained for reference. |
|
||||
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
|
||||
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
|
||||
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
|
||||
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.833/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.833/NOTICE) | Bring your own `kuna`; REA verified 1.833 and never downloads, installs or bundles it. REA invokes it as a separate process. |
|
||||
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
|
||||
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
|
||||
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
|
||||
| Wakaru | [Apache-2.0](https://github.com/pionxzh/wakaru/blob/8219bf5016a063d7fb215101a8c45d5959ec733f/LICENSE) | v1.14.0; pinned source and separately obtained JavaScript reconstruction tooling. |
|
||||
| WABT | [Apache-2.0](https://github.com/WebAssembly/wabt/blob/ff0ef7e0009402740c805a9744c09b05be063e48/LICENSE) | v1.0.42; external WebAssembly tools; [integration provenance](third_party/wabt/README.md). |
|
||||
| pwntools | [MIT, with GPL/BSD component qualifications](https://github.com/Gallopsled/pwntools/blob/3ad86ec5adab1abf9781ca7552a6f5d7cff7a33c/LICENSE-pwntools.txt) | 4.15.0; caller-supplied offline ELF analysis package. |
|
||||
| pyelftools | [Public-domain dedication and component notices](https://github.com/eliben/pyelftools/blob/v0.33/LICENSE) | 0.33; includes separately licensed bundled material, including Construct. |
|
||||
| Unicorn | [GPL-2.0 and component notices](https://github.com/unicorn-engine/unicorn/blob/2.1.2/COPYING) | 2.1.2; used by the caller-supplied ELF analysis profile for instruction emulation. |
|
||||
| pwndbg | [MIT](third_party/pwndbg/LICENSE.md) | 2026.09.15, commit `55596aa676a2756efb8d67034f31675170eabe48`; separately obtained debugger extension. |
|
||||
| GNU GDB | [GPL-3.0-or-later](https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=gdb/COPYING) | Caller-supplied debugger for the optional pwndbg core-file workflow. |
|
||||
| LLDB / LLVM | [Apache-2.0 with LLVM exceptions](https://github.com/llvm/llvm-project/blob/main/LICENSE.TXT) | Caller-supplied debugger; the installed distribution's component and historical notices also apply. |
|
||||
| mitmproxy | [MIT](https://github.com/mitmproxy/mitmproxy/blob/6c09d56e4c29a92f5ad01b03199977584b8ea14f/LICENSE) | 12.2.3; separately supplied network capture proxy. |
|
||||
| Android Debug Bridge (adb) | [Apache-2.0](https://android.googlesource.com/platform/packages/modules/adb/+/refs/heads/main/NOTICE) | User-supplied Android device tool. A complete Android SDK download has its own [SDK license terms](https://developer.android.com/studio/terms); adb's source license does not replace them. |
|
||||
| ILSpy / ICSharpCode.Decompiler | [MIT](https://github.com/icsharpcode/ILSpy/blob/master/ICSharpCode.Decompiler/Properties/AssemblyInfo.cs) | Optional separately installed .NET decompiler; not a redistributed REA dependency. |
|
||||
|
||||
Ghidra's original notice credits software developed at the National Security
|
||||
Agency. It explains that US-government-created portions are not subject to US
|
||||
copyright protection, while other portions retain their authors' rights. Its
|
||||
modules include additional third-party licenses, and standalone programs under
|
||||
`GPL/` have GPLv3 terms. The three retained top-level documents do not replace
|
||||
the module notices and license directories in a complete Ghidra distribution.
|
||||
REA is not affiliated with or endorsed by the NSA.
|
||||
|
||||
## Commercial tools and bridges
|
||||
|
||||
**You must legally obtain the tools and the rights needed for your use.** REA
|
||||
does not include an IDA Pro, Hex-Rays decompiler, Hopper, or JEB license, license
|
||||
key, or activation entitlement. A permitted trial/demo may be used only within
|
||||
its official terms. Cracked copies, stolen keys, unauthorized sharing, and
|
||||
bypassing activation or license limits are not authorized by REA.
|
||||
|
||||
| Project / product | Applicable terms and distinction |
|
||||
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| IDA Pro and Hex-Rays decompilers | Proprietary Hex-Rays software governed by the [EULA](https://hex-rays.com/eula), purchase agreement, and applicable license scheme. Obtain rights for the required product, user, machine, decompiler, and automation interfaces. Availability of an API or a bridge does not grant hosting, service, or additional-user rights. |
|
||||
| ida-pro-mcp | The bridge's [MIT license](https://github.com/mrexodia/ida-pro-mcp/blob/main/LICENSE) belongs to Duncan Ogilvie. It licenses the bridge code, not IDA or Hex-Rays products. Its [README](https://github.com/mrexodia/ida-pro-mcp#prerequisites) requires IDA Pro and explicitly states that IDA Free is unsupported. Follow the prerequisites for the version you actually install. The bridge is separately supplied, not bundled by REA. |
|
||||
| Hopper Disassembler | Proprietary Cryptic Apps software under its [license agreement](https://www.hopperapp.com/license_agreement.html). Personal and computer licenses have different permitted-user and installation rules; an official evaluation trial is available. REA setup may obtain the official installer after approval, but this grants no paid license or activation entitlement. |
|
||||
| JEB | Proprietary PNF Software product under its [EULA](https://www.pnfsoftware.com/jeb/LICENSE.TXT) and edition-specific agreement. Check user/seat/server rights and whether your edition permits automation, backend/service integration, and disclosure of generated output. The published EULA restricts public or third-party disclosure of decompiled code, reconstructed artifacts, and analysis reports unless expressly permitted by the license type. |
|
||||
|
||||
The vendor's applicable agreement controls. These summaries do not interpret
|
||||
your individual contract or expand the rights it grants. Verify permissions
|
||||
before publishing analysis output, sharing tool access, or operating a service.
|
||||
Vendor terms above were checked on 2026-10-12.
|
||||
|
||||
## Runtimes, development tooling, and fixtures
|
||||
|
||||
REA uses a caller-supplied Node.js runtime ([MIT and component notices](https://github.com/nodejs/node/blob/main/LICENSE)).
|
||||
Optional workflows may also use Python ([PSF and component licenses](https://docs.python.org/3/license.html)),
|
||||
OpenJDK ([GPLv2 with Classpath Exception and component notices](https://openjdk.org/legal/)),
|
||||
Electron ([MIT and component notices](https://github.com/electron/electron/blob/main/LICENSE)),
|
||||
or a Chromium-based browser ([BSD-3-Clause and third-party licenses](https://chromium.googlesource.com/chromium/src/+/main/LICENSE)).
|
||||
These external installations are not licensed or bundled by REA's MIT license.
|
||||
|
||||
Development, test, and website npm packages are included in the locked inventory.
|
||||
GitHub Actions, operating-system utilities, downloaded verification fixtures,
|
||||
and their runtimes retain the terms supplied by their publishers. Fixture
|
||||
identities and provenance are recorded under `scripts/fixtures/` and the
|
||||
existing `third_party/` records; downloaded executable fixtures are not tracked
|
||||
or included in the npm package. This notice does not claim to inventory every
|
||||
dependency of a caller's operating system, agent, or separately installed tool.
|
||||
|
||||
## Maintaining and redistributing notices
|
||||
|
||||
Refresh the dependency inventory and runtime notice collection when the lockfile
|
||||
changes. Review the actual versions and distributed contents, retain the
|
||||
upstream license and copyright texts, and account for optional platform
|
||||
packages. See [licenses/README.md](licenses/README.md) for source provenance,
|
||||
collection scope, and known upstream notice gaps.
|
||||
|
||||
The npm package includes this document, the `licenses/` directory, and the
|
||||
existing redistributed license records under `third_party/`. If you additionally
|
||||
bundle an external engine or modify upstream code, review that actual
|
||||
distribution and retain its full applicable licenses, copyright notices,
|
||||
NOTICE files, modification notices, and source obligations. REA's inventory is
|
||||
not a substitute for those obligations.
|
||||
@@ -1,512 +0,0 @@
|
||||
import com.atxx.jhmcp.*;
|
||||
import com.google.gson.*;
|
||||
import io.modelcontextprotocol.kotlin.sdk.server.*;
|
||||
import io.modelcontextprotocol.kotlin.sdk.types.*;
|
||||
import jadx.api.*;
|
||||
import jadx.api.plugins.input.data.ICodeReader;
|
||||
import jadx.api.plugins.input.data.IMethodRef;
|
||||
import jadx.api.plugins.input.insns.InsnIndexType;
|
||||
import jadx.api.plugins.input.insns.Opcode;
|
||||
import jadx.core.dex.info.MethodInfo;
|
||||
import jadx.core.dex.nodes.*;
|
||||
import java.io.PrintStream;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.util.*;
|
||||
import java.util.concurrent.CountDownLatch;
|
||||
import java.util.stream.Collectors;
|
||||
import kotlin.Unit;
|
||||
import kotlin.ResultKt;
|
||||
import kotlin.coroutines.Continuation;
|
||||
import kotlin.coroutines.CoroutineContext;
|
||||
import kotlin.coroutines.EmptyCoroutineContext;
|
||||
import kotlin.coroutines.intrinsics.IntrinsicsKt;
|
||||
import kotlinx.coroutines.BuildersKt;
|
||||
import kotlinx.io.CoreKt;
|
||||
import kotlinx.io.JvmCoreKt;
|
||||
|
||||
/** REA-owned metadata boundary for the unmodified jadx-headless-mcp 0.7.1 JAR. */
|
||||
class ReaJadxBridge {
|
||||
private static final Gson JSON = new Gson();
|
||||
private final SessionHolder holder;
|
||||
private JadxSession indexedSession;
|
||||
private Map<String, List<ClassSnapshot>> classes = Map.of();
|
||||
|
||||
private record MethodSnapshot(String name, String rawName, String shortId, List<String> bodyAbsenceMarkers,
|
||||
Map<String, Object> summary) {}
|
||||
private record ClassSnapshot(JavaClass node, List<MethodSnapshot> methods, Map<String, Object> summary) {}
|
||||
|
||||
private static final class InputFailure extends IllegalArgumentException {
|
||||
final String field;
|
||||
InputFailure(String field, String message) { super(message); this.field = field; }
|
||||
}
|
||||
|
||||
private ReaJadxBridge(SessionHolder holder) { this.holder = holder; }
|
||||
|
||||
private static CallToolResult text(String value, boolean failed) {
|
||||
return new CallToolResult(List.of(new TextContent(value, null, null)), failed, null, null);
|
||||
}
|
||||
|
||||
private static CallToolResult json(Object value) { return text(JSON.toJson(value), false); }
|
||||
|
||||
/** Translate the typed load outcome before upstream drops its exception identity. */
|
||||
static Object loadResult(Object result) {
|
||||
try { ResultKt.throwOnFailure(result); }
|
||||
catch (Throwable failure) {
|
||||
if (failure instanceof java.util.concurrent.CancellationException
|
||||
|| failure instanceof Error && !(failure instanceof OutOfMemoryError)) return result;
|
||||
if (failure instanceof OutOfMemoryError) {
|
||||
String reason = failure.getMessage();
|
||||
boolean heap = "Java heap space".equals(reason) || "GC overhead limit exceeded".equals(reason);
|
||||
Map<String, Object> reply = new LinkedHashMap<>();
|
||||
reply.put("kind", "jvm-memory-failure");
|
||||
reply.put("memory_area", heap ? "heap" : "unknown");
|
||||
reply.put("exception_type", "java.lang.OutOfMemoryError");
|
||||
if (reason != null) reply.put("message", reason);
|
||||
reply.put("max_heap_bytes", Runtime.getRuntime().maxMemory());
|
||||
return text(JSON.toJson(reply), true);
|
||||
}
|
||||
return text("load failed: " + failure.getMessage(), true);
|
||||
}
|
||||
SessionHolder.LoadResult loaded = (SessionHolder.LoadResult) result;
|
||||
Map<String, Object> reply = new LinkedHashMap<>();
|
||||
reply.put("state", "LOADED");
|
||||
reply.put("apk_path", loaded.getApkPath());
|
||||
reply.put("class_count", loaded.getClassCount());
|
||||
reply.put("raw_class_count", loaded.getRawClassCount());
|
||||
reply.put("resource_count", loaded.getResourceCount());
|
||||
reply.put("load_duration_ms", loaded.getLoadDurationMs());
|
||||
reply.put("decompile_timeout_ms", loaded.getDecompileTimeoutMs());
|
||||
reply.put("threads", loaded.getThreads());
|
||||
reply.put("code_cache_size", loaded.getCodeCacheSize());
|
||||
reply.put("resources", loaded.getResourceMode());
|
||||
if (!loaded.getIncludePackages().isEmpty()) reply.put("include_packages", loaded.getIncludePackages());
|
||||
if (!loaded.getExcludePackages().isEmpty()) reply.put("exclude_packages", loaded.getExcludePackages());
|
||||
return json(reply);
|
||||
}
|
||||
|
||||
private Object load(kotlinx.serialization.json.JsonObject arguments, SessionConfig defaults,
|
||||
Continuation<? super CallToolResult> continuation) {
|
||||
JsonObject args = JsonParser.parseString(arguments.toString()).getAsJsonObject();
|
||||
String path = string(args, "path");
|
||||
SessionConfig options = defaults.withLoadOverrides(
|
||||
optionalString(args, "include_packages"), optionalString(args, "exclude_packages"),
|
||||
args.has("threads") ? integer(args, "threads", 1) : null,
|
||||
args.has("code_cache_size") ? integer(args, "code_cache_size", 64) : null,
|
||||
optionalString(args, "resources"));
|
||||
return loadCall(next -> holder.load(path, options, next), continuation);
|
||||
}
|
||||
|
||||
static Object loadCall(kotlin.jvm.functions.Function1<Continuation<SessionHolder.LoadResult>, Object> load,
|
||||
Continuation<? super CallToolResult> continuation) {
|
||||
var forwarded = new java.util.concurrent.atomic.AtomicBoolean();
|
||||
Continuation<SessionHolder.LoadResult> resumed = new Continuation<>() {
|
||||
public CoroutineContext getContext() { return continuation.getContext(); }
|
||||
public void resumeWith(Object result) {
|
||||
if (!forwarded.compareAndSet(false, true)) throw new IllegalStateException("Load resumed twice");
|
||||
Object translated;
|
||||
try { translated = loadResult(result); }
|
||||
catch (Throwable failure) { translated = ResultKt.createFailure(failure); }
|
||||
// Keep the consumer outside the translation catch, including inline completion.
|
||||
continuation.resumeWith(translated);
|
||||
}
|
||||
};
|
||||
Object result;
|
||||
try { result = load.invoke(resumed); }
|
||||
catch (Throwable failure) {
|
||||
if (forwarded.get()) ResultKt.throwOnFailure(ResultKt.createFailure(failure));
|
||||
result = ResultKt.createFailure(failure);
|
||||
}
|
||||
if (result == IntrinsicsKt.getCOROUTINE_SUSPENDED()) return result;
|
||||
Object translated = loadResult(result);
|
||||
ResultKt.throwOnFailure(translated);
|
||||
return translated;
|
||||
}
|
||||
|
||||
private static String optionalString(JsonObject args, String name) {
|
||||
return args.has(name) ? string(args, name) : null;
|
||||
}
|
||||
|
||||
private static String string(JsonObject args, String name) {
|
||||
JsonElement value = args.get(name);
|
||||
if (value == null || !value.isJsonPrimitive() || !value.getAsJsonPrimitive().isString())
|
||||
throw new IllegalArgumentException(name + " must be a string");
|
||||
return value.getAsString();
|
||||
}
|
||||
|
||||
private static int integer(JsonObject args, String name, int fallback) {
|
||||
JsonElement value = args.get(name);
|
||||
if (value == null) return fallback;
|
||||
if (!value.isJsonPrimitive() || !value.getAsJsonPrimitive().isNumber())
|
||||
throw new IllegalArgumentException(name + " must be a nonnegative integer");
|
||||
try {
|
||||
int result = value.getAsBigDecimal().intValueExact();
|
||||
if (result < 0) throw new ArithmeticException();
|
||||
return result;
|
||||
} catch (ArithmeticException error) {
|
||||
throw new IllegalArgumentException(name + " must be a nonnegative integer", error);
|
||||
}
|
||||
}
|
||||
|
||||
private JadxSession session() {
|
||||
JadxSession current = holder.current();
|
||||
if (current == null) throw new IllegalArgumentException("No APK loaded");
|
||||
return current;
|
||||
}
|
||||
|
||||
private ClassSnapshot selected(JsonObject args) {
|
||||
JadxSession current = session();
|
||||
if (indexedSession != current) {
|
||||
Map<String, List<ClassSnapshot>> index = new LinkedHashMap<>();
|
||||
if (!current.getClasses().isEmpty()) {
|
||||
JadxDecompiler decompiler = current.getClasses().get(0).getClassNode().root().getDecompiler();
|
||||
// JavaClass.getMethods/getFields/getInnerClasses all load code. Raw nodes do not.
|
||||
// Snapshot every identity before code generation can rewrite aliases or members.
|
||||
for (JavaClass cls : decompiler.getClassesWithInners()) {
|
||||
ClassNode node = cls.getClassNode();
|
||||
List<MethodNode> ordered = new ArrayList<>(node.getMethods());
|
||||
ordered.sort(Comparator.comparing(MethodNode::getAlias));
|
||||
List<MethodSnapshot> methods = ordered.stream().map(method -> {
|
||||
String name = method.getAlias();
|
||||
String shortId = method.getMethodInfo().getShortId();
|
||||
String rawName = method.getMethodInfo().getName();
|
||||
if (!shortId.startsWith(rawName + "("))
|
||||
throw new IllegalStateException("JADX method ID has no raw-name prefix: " + shortId);
|
||||
String descriptor = shortId.substring(rawName.length());
|
||||
List<String> bodyAbsenceMarkers = new ArrayList<>();
|
||||
if (method.getAccessFlags().isNative()) bodyAbsenceMarkers.add("native_method");
|
||||
if (method.getAccessFlags().isAbstract()) bodyAbsenceMarkers.add("abstract_method");
|
||||
String signature = name + "(" + method.getArgTypes().stream()
|
||||
.map(Object::toString).collect(Collectors.joining(", "))
|
||||
+ "): " + method.getReturnType().toString();
|
||||
return new MethodSnapshot(name, rawName, shortId, List.copyOf(bodyAbsenceMarkers), Map.of(
|
||||
"name", name, "signature", signature,
|
||||
"dex_descriptor", descriptor,
|
||||
"is_constructor", method.isConstructor(), "def_pos", 0));
|
||||
}).toList();
|
||||
List<Map<String, Object>> fields = node.getFields().stream()
|
||||
.sorted(Comparator.comparing(FieldNode::getAlias))
|
||||
.map(field -> Map.<String, Object>of("name", field.getAlias(),
|
||||
"type", field.getType().toString(), "def_pos", 0)).toList();
|
||||
List<String> inner = node.getInnerClasses().stream().map(ClassNode::getFullName).toList();
|
||||
Map<String, Object> summary = Map.of(
|
||||
"full_name", cls.getFullName(), "name", cls.getName(),
|
||||
"method_count", methods.size(), "field_count", fields.size(),
|
||||
"inner_class_count", inner.size(), "inner_classes", inner,
|
||||
"methods", methods.stream().map(MethodSnapshot::summary).toList(), "fields", fields);
|
||||
index.computeIfAbsent(cls.getFullName(), ignored -> new ArrayList<>())
|
||||
.add(new ClassSnapshot(cls, methods, summary));
|
||||
}
|
||||
}
|
||||
classes = Collections.unmodifiableMap(index);
|
||||
indexedSession = current;
|
||||
}
|
||||
String name = string(args, "class_name");
|
||||
List<ClassSnapshot> candidates = classes.get(name);
|
||||
if (candidates == null) throw new InputFailure("class_name", "Exact class not found: " + name
|
||||
+ "; use search_android_classes to discover fully qualified names.");
|
||||
if (candidates.size() != 1) throw new InputFailure("class_name", "Ambiguous metadata class identity: " + name
|
||||
+ "; raw identities: " + candidates.stream().map(candidate -> candidate.node().getRawName()).toList());
|
||||
return candidates.get(0);
|
||||
}
|
||||
|
||||
private List<MethodSnapshot> methods(ClassSnapshot cls, JsonObject args) {
|
||||
String name = string(args, "method_name");
|
||||
List<MethodSnapshot> result = cls.methods().stream().filter(method -> method.name().equals(name)).toList();
|
||||
if (result.isEmpty()) throw new InputFailure("method_name", "Method not found: " + name
|
||||
+ "; use inspect_android_class to discover available methods.");
|
||||
return result;
|
||||
}
|
||||
|
||||
private JavaMethod method(ClassSnapshot cls, MethodSnapshot selected) {
|
||||
JavaMethod result = cls.node().searchMethodByShortId(selected.shortId());
|
||||
if (result == null) throw new IllegalArgumentException("Method identity no longer available: " + selected.shortId());
|
||||
return result;
|
||||
}
|
||||
|
||||
private JadxSession.SmartCode methodBody(JadxSession current, ClassSnapshot cls,
|
||||
MethodSnapshot selected, JavaMethod method) {
|
||||
// Access flags belong to the parsed member, before code generation rewrites it.
|
||||
// These methods have no Java body; source slicing can include unrelated members.
|
||||
if (!selected.bodyAbsenceMarkers().isEmpty())
|
||||
return new JadxSession.SmartCode("", "java", false, selected.bodyAbsenceMarkers());
|
||||
// Java code generation can fold all or part of <clinit> into field declarations.
|
||||
// Its method text cannot establish coverage, even when a static block remains.
|
||||
if (selected.shortId().equals("<clinit>()V"))
|
||||
return exactMethodSmali(current, cls, selected, List.of("static_initializer_disassembly"));
|
||||
String java = current.getMethodSource(method);
|
||||
if (java.startsWith("// ERROR:")) throw new IllegalArgumentException(java);
|
||||
// The upstream placeholder means only that Java code was omitted, including
|
||||
// methods merged into synthetic bridges. Access flags established absence above.
|
||||
if (java.isEmpty() || java.equals(
|
||||
"// method exists but has no decompiled body (native/abstract)"))
|
||||
return exactMethodSmali(current, cls, selected, List.of("java_body_unavailable"));
|
||||
List<String> markers = DecompileFailureKt.detectDecompileFailure(java);
|
||||
if (markers.stream().anyMatch(DecompileFailureKt.getSTRONG_FAILURE_MARKERS()::contains))
|
||||
return exactMethodSmali(current, cls, selected, markers);
|
||||
return new JadxSession.SmartCode(
|
||||
TextUtilKt.truncateToBytes(java, current.getMaxSourceBytes()), "java", false, markers);
|
||||
}
|
||||
|
||||
private JadxSession.SmartCode exactMethodSmali(JadxSession current, ClassSnapshot cls,
|
||||
MethodSnapshot selected, List<String> markers) {
|
||||
JadxSession.MethodSmali smali = current.getMethodSmali(cls.node(), selected.rawName());
|
||||
String block = selectMethodSmali(smali, selected.shortId());
|
||||
return new JadxSession.SmartCode(
|
||||
TextUtilKt.truncateToBytes(block, current.getMaxSourceBytes()), "smali", true, markers);
|
||||
}
|
||||
|
||||
private static String selectMethodSmali(JadxSession.MethodSmali smali, String shortId) {
|
||||
if (!smali.getFound())
|
||||
throw new IllegalArgumentException("JADX could not disassemble " + shortId
|
||||
+ (smali.getNote() == null ? "" : ": " + smali.getNote()));
|
||||
// Upstream returns every same-name overload, and can return an unterminated
|
||||
// trailing block. Match the full original identity before applying output limits.
|
||||
List<String> matches = smali.getBlocks().stream().filter(block -> {
|
||||
String header = block.lines().findFirst().orElse("").strip();
|
||||
return header.startsWith(".method ") && header.endsWith(" " + shortId);
|
||||
}).toList();
|
||||
if (matches.size() != 1)
|
||||
throw new IllegalArgumentException("Disassembly could not establish one selected method: " + shortId);
|
||||
String block = matches.get(0);
|
||||
if (!block.stripTrailing().endsWith("\n.end method"))
|
||||
throw new IllegalArgumentException("Selected method disassembly is incomplete: " + shortId);
|
||||
return block;
|
||||
}
|
||||
|
||||
// The local transport admits 8 MiB frames. Reserve 1 KiB for the fixed
|
||||
// JSON-RPC/CallToolResult envelope and generated numeric request id, and
|
||||
// account for JSON text being escaped again inside TextContent.
|
||||
private static final int MAX_XREF_TEXT_BYTES = 8 * 1024 * 1024 - 1024;
|
||||
private static int escapedJsonBytes(Object value) {
|
||||
return JSON.toJson(JSON.toJson(value)).getBytes(StandardCharsets.UTF_8).length;
|
||||
}
|
||||
private static final class InvokeBudget {
|
||||
long remaining;
|
||||
InvokeBudget(long remaining) { this.remaining = Math.max(0, remaining); }
|
||||
boolean admit(Map<String, Object> site) {
|
||||
// Check a conservative two-layer escape bound before serialization.
|
||||
long bound = 256;
|
||||
for (Object value : site.values())
|
||||
if (value instanceof String text) bound += 12L * text.length();
|
||||
if (bound > remaining) return false;
|
||||
long bytes = escapedJsonBytes(site) - 2L + 1L;
|
||||
if (bytes > remaining) return false;
|
||||
remaining -= bytes;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
private record DirectInvokes(List<Map<String, Object>> sites, boolean exhaustive) {}
|
||||
|
||||
/** Raw DEX invokes within one JADX using method; offsets are code units from its code start. */
|
||||
private static DirectInvokes directInvokes(JavaMethod source, MethodInfo target, InvokeBudget budget) {
|
||||
MethodNode node = source.getMethodNode();
|
||||
ICodeReader code = node.getCodeReader();
|
||||
if (code == null) return null;
|
||||
List<Map<String, Object>> sites = new ArrayList<>();
|
||||
boolean[] exhaustive = {true};
|
||||
code.visitInstructions(insn -> {
|
||||
insn.decode();
|
||||
if (insn.getIndexType() != InsnIndexType.METHOD_REF) return;
|
||||
Opcode opcode = insn.getOpcode();
|
||||
if (!opcode.name().startsWith("INVOKE_")) return;
|
||||
IMethodRef ref = insn.getIndexAsMethod();
|
||||
ref.load();
|
||||
if (!MethodInfo.fromRef(node.root(), ref).getRawFullId().equals(target.getRawFullId())) return;
|
||||
Map<String, Object> site = Map.of(
|
||||
"dex_entry", node.getInputFileName(),
|
||||
"source_raw_method_id", node.getMethodInfo().getRawFullId(),
|
||||
"dex_code_unit_offset", insn.getOffset(),
|
||||
"dex_method_index", insn.getIndex(),
|
||||
"opcode", opcode.name());
|
||||
if (budget.admit(site)) sites.add(site);
|
||||
else exhaustive[0] = false;
|
||||
});
|
||||
return new DirectInvokes(sites, exhaustive[0]);
|
||||
}
|
||||
|
||||
private CallToolResult handle(String name, kotlinx.serialization.json.JsonObject input) {
|
||||
JsonObject args = JsonParser.parseString(input.toString()).getAsJsonObject();
|
||||
if (name.equals("rea_jvm_status")) return json(Map.of(
|
||||
"engine_reported_version", BuildInfo.VERSION,
|
||||
"max_heap_bytes", Runtime.getRuntime().maxMemory(),
|
||||
"available_processors", Runtime.getRuntime().availableProcessors(),
|
||||
"java_version", System.getProperty("java.version"),
|
||||
"metadata_scope", "parsed_members"));
|
||||
JadxSession current = session();
|
||||
if (name.equals("list_classes")) {
|
||||
List<String> names = current.getClassFqns();
|
||||
String query = args.has("query") ? string(args, "query") : "";
|
||||
int offset = integer(args, "offset", 0);
|
||||
int limit = integer(args, "limit", 200);
|
||||
int start = Math.min(offset, names.size());
|
||||
int end = start;
|
||||
List<String> items;
|
||||
if (query.isEmpty()) {
|
||||
end = (int) Math.min((long) start + limit, names.size());
|
||||
items = names.subList(start, end);
|
||||
} else {
|
||||
items = new ArrayList<>();
|
||||
while (end < names.size() && items.size() < limit) {
|
||||
String nameInInventory = names.get(end++);
|
||||
if (nameInInventory.contains(query)) items.add(nameInInventory);
|
||||
}
|
||||
}
|
||||
return json(Map.of("total_classes", names.size(), "next_offset", end,
|
||||
"scope", "top_level", "offset", offset, "limit", limit,
|
||||
"items", items));
|
||||
}
|
||||
ClassSnapshot cls = selected(args);
|
||||
if (name.equals("get_class_summary")) return json(cls.summary());
|
||||
if (name.equals("get_method_body")) {
|
||||
List<MethodSnapshot> overloads = methods(cls, args);
|
||||
int index = integer(args, "overload_index", 0);
|
||||
if (index >= overloads.size()) throw new InputFailure("overload_index", "overload_index out of range: " + index);
|
||||
JavaMethod method = method(cls, overloads.get(index));
|
||||
JadxSession.SmartCode code = methodBody(current, cls, overloads.get(index), method);
|
||||
return json(Map.of("class_name", cls.summary().get("full_name"),
|
||||
"method_name", overloads.get(index).name(), "full_name", method.getFullName(),
|
||||
"dex_descriptor", overloads.get(index).summary().get("dex_descriptor"),
|
||||
"overload_index", index, "overload_count", overloads.size(),
|
||||
"mode", code.getKind(), "fell_back", code.getFellBack(),
|
||||
"markers", code.getMarkers(), "body", code.getText()));
|
||||
}
|
||||
List<JavaNode> uses;
|
||||
String target = string(args, "class_name");
|
||||
MethodSnapshot selectedMethod = null;
|
||||
int overloadIndex = 0;
|
||||
if (name.equals("get_xrefs_to_class")) uses = cls.node().getUseIn();
|
||||
else {
|
||||
List<MethodSnapshot> overloads = methods(cls, args);
|
||||
if (overloads.size() != 1 && !args.has("overload_index"))
|
||||
throw new InputFailure("overload_index", "Select one of " + overloads.size() + " method overloads");
|
||||
overloadIndex = integer(args, "overload_index", 0);
|
||||
if (overloadIndex >= overloads.size())
|
||||
throw new InputFailure("overload_index", "overload_index out of range: " + overloadIndex);
|
||||
selectedMethod = overloads.get(overloadIndex);
|
||||
target += "." + selectedMethod.name();
|
||||
uses = method(cls, selectedMethod).getUseIn();
|
||||
}
|
||||
// REA consumes all references without materializing source line information.
|
||||
MethodInfo targetInfo = selectedMethod == null ? null : method(cls, selectedMethod).getMethodNode().getMethodInfo();
|
||||
List<Map<String, Object>> items = uses.stream().map(node -> {
|
||||
Map<String, Object> item = new LinkedHashMap<>(current.describeUsage(node, false));
|
||||
if (targetInfo != null && node instanceof JavaMethod) {
|
||||
item.put("direct_invokes", List.of());
|
||||
item.put("direct_invokes_exhaustive", false);
|
||||
}
|
||||
return item;
|
||||
}).toList();
|
||||
Map<String, Object> result = new LinkedHashMap<>(Map.of("target", target,
|
||||
"total", uses.size(), "count", items.size(), "resolve_line", false, "items", items));
|
||||
if (selectedMethod != null) {
|
||||
result.put("overload_index", overloadIndex);
|
||||
result.put("reported_signature", selectedMethod.summary().get("signature"));
|
||||
}
|
||||
if (targetInfo != null) {
|
||||
InvokeBudget budget = new InvokeBudget(MAX_XREF_TEXT_BYTES - escapedJsonBytes(result));
|
||||
for (int index = 0; index < uses.size(); index++) {
|
||||
if (!(uses.get(index) instanceof JavaMethod source)) continue;
|
||||
DirectInvokes invokes = directInvokes(source, targetInfo, budget);
|
||||
Map<String, Object> item = items.get(index);
|
||||
if (invokes == null) {
|
||||
item.remove("direct_invokes");
|
||||
item.remove("direct_invokes_exhaustive");
|
||||
} else {
|
||||
item.put("direct_invokes", invokes.sites());
|
||||
item.put("direct_invokes_exhaustive", invokes.exhaustive());
|
||||
}
|
||||
}
|
||||
}
|
||||
return json(result);
|
||||
}
|
||||
|
||||
public static void main(String[] args) throws Exception {
|
||||
PrintStream protocolOut = System.out;
|
||||
System.setOut(System.err);
|
||||
SessionConfig defaults = new SessionConfig(
|
||||
1048576, 0, 90000L, 1, List.of(), List.of(), 64, ResourceMode.FULL);
|
||||
SessionHolder holder = new SessionHolder(defaults);
|
||||
ServerCapabilities capabilities = new ServerCapabilities(
|
||||
new ServerCapabilities.Tools(null), null, null, null, null, null, null, Map.of());
|
||||
Server server = new Server(new Implementation("rea-jadx-bridge", "4", null, null, null),
|
||||
new ServerOptions(capabilities), "", ignored -> Unit.INSTANCE);
|
||||
SessionToolsKt.registerSessionTools(server, holder);
|
||||
ManifestToolsKt.registerManifestTools(server, holder);
|
||||
ClassToolsKt.registerClassTools(server, holder);
|
||||
MethodToolsKt.registerMethodTools(server, holder);
|
||||
XrefToolsKt.registerXrefTools(server, holder);
|
||||
ReaJadxBridge bridge = new ReaJadxBridge(holder);
|
||||
Tool loadTool = server.getTools().get("load_apk").getTool();
|
||||
server.removeTool("load_apk");
|
||||
server.addTool(loadTool, (connection, request, continuation) ->
|
||||
bridge.load(request.getArguments(), defaults, continuation));
|
||||
Set<String> consumed = Set.of("load_apk", "get_app_info", "get_android_manifest",
|
||||
"list_classes", "get_class_summary", "get_method_body",
|
||||
"get_xrefs_to_class", "get_xrefs_to_method");
|
||||
for (String name : new ArrayList<>(server.getTools().keySet()))
|
||||
if (!consumed.contains(name)) server.removeTool(name);
|
||||
Tool diagnostics = new Tool("rea_jvm_status",
|
||||
new ToolSchema(null, new kotlinx.serialization.json.JsonObject(Map.of()), List.of(), null),
|
||||
"Observed worker JVM and metadata scope", null, null, null, null, null, null);
|
||||
server.addTool(diagnostics, (connection, request, continuation) -> bridge.handle("rea_jvm_status", request.getArguments()));
|
||||
for (String name : List.of("list_classes", "get_class_summary",
|
||||
"get_method_body", "get_xrefs_to_class", "get_xrefs_to_method")) {
|
||||
Tool tool = server.getTools().get(name).getTool();
|
||||
server.removeTool(name);
|
||||
List<String> fields = switch (name) {
|
||||
case "list_classes" -> List.of("offset", "limit");
|
||||
case "get_method_body" -> List.of("class_name", "method_name", "overload_index");
|
||||
case "get_xrefs_to_method" -> List.of("class_name", "method_name", "overload_index");
|
||||
default -> List.of("class_name");
|
||||
};
|
||||
Map<String, kotlinx.serialization.json.JsonElement> properties = new LinkedHashMap<>();
|
||||
for (String field : fields) properties.put(field, tool.getInputSchema().getProperties().get(field));
|
||||
if (name.equals("list_classes")) properties.put("query",
|
||||
new kotlinx.serialization.json.JsonObject(Map.of(
|
||||
"type", kotlinx.serialization.json.JsonElementKt.JsonPrimitive("string"))));
|
||||
String description = switch (name) {
|
||||
case "list_classes" -> "Pages of case-sensitive matching top-level class names; offset and next_offset are inventory positions, limit bounds matching rows. Omitted or empty query inventories all names.";
|
||||
case "get_class_summary" -> "Parsed member metadata for one exact class name; no source generation.";
|
||||
case "get_method_body" -> "Source for an exact class and stable same-name overload index; static initializers use smali to preserve folded field initialization.";
|
||||
default -> "Complete static using-node inventory for an exact class or selected method overload; one node may contain multiple invoke instructions, and source lines are unresolved.";
|
||||
};
|
||||
Tool narrowed = new Tool(name, new ToolSchema(null, new kotlinx.serialization.json.JsonObject(properties),
|
||||
tool.getInputSchema().getRequired(), null), description, null, null, null, null, null, null);
|
||||
server.addTool(narrowed, (connection, request, continuation) -> {
|
||||
try { return bridge.handle(name, request.getArguments()); }
|
||||
catch (InputFailure error) { return text(JSON.toJson(Map.of(
|
||||
"kind", "invalid-input", "field", error.field, "message", error.getMessage())), true); }
|
||||
catch (IllegalArgumentException error) { return text(error.getMessage(), true); }
|
||||
});
|
||||
}
|
||||
CountDownLatch done = new CountDownLatch(1);
|
||||
server.onClose(() -> { done.countDown(); return Unit.INSTANCE; });
|
||||
StdioServerTransport transport = new StdioServerTransport(
|
||||
CoreKt.buffered(JvmCoreKt.asSource(System.in)),
|
||||
CoreKt.buffered(JvmCoreKt.asSink(protocolOut)));
|
||||
transport.onClose(() -> { done.countDown(); return Unit.INSTANCE; });
|
||||
ServerSession connection = null;
|
||||
try {
|
||||
connection = BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE,
|
||||
(scope, continuation) -> server.createSession(transport, continuation));
|
||||
done.await();
|
||||
} finally {
|
||||
try {
|
||||
if (connection != null) {
|
||||
ServerSession connected = connection;
|
||||
// Unsubscribe the connection before the SDK joins its notification service.
|
||||
BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE,
|
||||
(scope, continuation) -> connected.close(continuation));
|
||||
}
|
||||
} finally {
|
||||
try {
|
||||
BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE,
|
||||
(scope, continuation) -> server.close(continuation));
|
||||
} finally {
|
||||
BuildersKt.runBlocking(EmptyCoroutineContext.INSTANCE,
|
||||
(scope, continuation) -> holder.unload(continuation));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,825 +0,0 @@
|
||||
"""REA's experimental local bridge plugin for Cutter.
|
||||
|
||||
Install this file in Cutter's user plugin directory. The bridge is REA-owned
|
||||
IPC, not an upstream Cutter transport. It binds to loopback and requires a
|
||||
per-process random token stored in a user-only descriptor file.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import hashlib
|
||||
import hmac
|
||||
import os
|
||||
import secrets
|
||||
import socket
|
||||
import stat
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
if os.name == "nt":
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
|
||||
import cutter
|
||||
|
||||
try:
|
||||
from PySide6.QtCore import QCoreApplication, QObject, Signal
|
||||
except ImportError:
|
||||
from PySide2.QtCore import QCoreApplication, QObject, Signal
|
||||
|
||||
|
||||
MAX_FRAME_BYTES = 16 * 1024 * 1024
|
||||
SOCKET_TIMEOUT_SECONDS = 20
|
||||
AUTH_FRAME_TIMEOUT_SECONDS = 1
|
||||
MAX_DESCRIPTOR_BYTES = 64 * 1024
|
||||
MAX_COMPLETION_RECEIPTS = 64
|
||||
|
||||
|
||||
def _windows_api() -> tuple[Any, Any]:
|
||||
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
|
||||
advapi32 = ctypes.WinDLL("advapi32", use_last_error=True)
|
||||
kernel32.GetCurrentProcess.restype = wintypes.HANDLE
|
||||
kernel32.LocalFree.argtypes = [wintypes.HLOCAL]
|
||||
kernel32.LocalFree.restype = wintypes.HLOCAL
|
||||
kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
|
||||
kernel32.CloseHandle.restype = wintypes.BOOL
|
||||
kernel32.GetFileAttributesW.argtypes = [wintypes.LPCWSTR]
|
||||
kernel32.GetFileAttributesW.restype = wintypes.DWORD
|
||||
kernel32.CreateDirectoryW.argtypes = [wintypes.LPCWSTR, wintypes.LPVOID]
|
||||
kernel32.CreateDirectoryW.restype = wintypes.BOOL
|
||||
kernel32.CreateFileW.argtypes = [wintypes.LPCWSTR, wintypes.DWORD, wintypes.DWORD,
|
||||
wintypes.LPVOID, wintypes.DWORD, wintypes.DWORD, wintypes.HANDLE]
|
||||
kernel32.CreateFileW.restype = wintypes.HANDLE
|
||||
kernel32.WriteFile.argtypes = [wintypes.HANDLE, wintypes.LPVOID, wintypes.DWORD,
|
||||
ctypes.POINTER(wintypes.DWORD), wintypes.LPVOID]
|
||||
kernel32.WriteFile.restype = wintypes.BOOL
|
||||
kernel32.FlushFileBuffers.argtypes = [wintypes.HANDLE]
|
||||
kernel32.FlushFileBuffers.restype = wintypes.BOOL
|
||||
kernel32.MoveFileExW.argtypes = [wintypes.LPCWSTR, wintypes.LPCWSTR, wintypes.DWORD]
|
||||
kernel32.MoveFileExW.restype = wintypes.BOOL
|
||||
kernel32.DeleteFileW.argtypes = [wintypes.LPCWSTR]
|
||||
kernel32.DeleteFileW.restype = wintypes.BOOL
|
||||
advapi32.OpenProcessToken.argtypes = [wintypes.HANDLE, wintypes.DWORD, ctypes.POINTER(wintypes.HANDLE)]
|
||||
advapi32.OpenProcessToken.restype = wintypes.BOOL
|
||||
advapi32.GetTokenInformation.argtypes = [wintypes.HANDLE, ctypes.c_int, wintypes.LPVOID,
|
||||
wintypes.DWORD, ctypes.POINTER(wintypes.DWORD)]
|
||||
advapi32.GetTokenInformation.restype = wintypes.BOOL
|
||||
advapi32.ConvertSidToStringSidW.argtypes = [wintypes.LPVOID, ctypes.POINTER(wintypes.LPWSTR)]
|
||||
advapi32.ConvertSidToStringSidW.restype = wintypes.BOOL
|
||||
advapi32.GetNamedSecurityInfoW.argtypes = [wintypes.LPWSTR, ctypes.c_int, wintypes.DWORD,
|
||||
ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.LPVOID),
|
||||
ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.LPVOID),
|
||||
ctypes.POINTER(wintypes.LPVOID)]
|
||||
advapi32.GetNamedSecurityInfoW.restype = wintypes.DWORD
|
||||
advapi32.GetSecurityDescriptorControl.argtypes = [wintypes.LPVOID, ctypes.POINTER(wintypes.WORD),
|
||||
ctypes.POINTER(wintypes.DWORD)]
|
||||
advapi32.GetSecurityDescriptorControl.restype = wintypes.BOOL
|
||||
advapi32.EqualSid.argtypes = [wintypes.LPVOID, wintypes.LPVOID]
|
||||
advapi32.EqualSid.restype = wintypes.BOOL
|
||||
advapi32.GetAclInformation.argtypes = [wintypes.LPVOID, wintypes.LPVOID, wintypes.DWORD, ctypes.c_int]
|
||||
advapi32.GetAclInformation.restype = wintypes.BOOL
|
||||
advapi32.GetAce.argtypes = [wintypes.LPVOID, wintypes.DWORD, ctypes.POINTER(wintypes.LPVOID)]
|
||||
advapi32.GetAce.restype = wintypes.BOOL
|
||||
advapi32.IsValidSid.argtypes = [wintypes.LPVOID]
|
||||
advapi32.IsValidSid.restype = wintypes.BOOL
|
||||
advapi32.GetLengthSid.argtypes = [wintypes.LPVOID]
|
||||
advapi32.GetLengthSid.restype = wintypes.DWORD
|
||||
advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW.argtypes = [
|
||||
wintypes.LPCWSTR, wintypes.DWORD, ctypes.POINTER(wintypes.LPVOID), ctypes.POINTER(wintypes.DWORD),
|
||||
]
|
||||
advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW.restype = wintypes.BOOL
|
||||
return kernel32, advapi32
|
||||
|
||||
|
||||
def _windows_user_sid_text() -> str:
|
||||
kernel32, advapi32 = _windows_api()
|
||||
token = wintypes.HANDLE()
|
||||
if not advapi32.OpenProcessToken(kernel32.GetCurrentProcess(), 0x0008, ctypes.byref(token)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
try:
|
||||
required = wintypes.DWORD()
|
||||
advapi32.GetTokenInformation(token, 1, None, 0, ctypes.byref(required))
|
||||
if required.value == 0:
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
buffer = ctypes.create_string_buffer(required.value)
|
||||
if not advapi32.GetTokenInformation(token, 1, buffer, required, ctypes.byref(required)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
|
||||
class SID_AND_ATTRIBUTES(ctypes.Structure):
|
||||
_fields_ = [("Sid", wintypes.LPVOID), ("Attributes", wintypes.DWORD)]
|
||||
|
||||
class TOKEN_USER(ctypes.Structure):
|
||||
_fields_ = [("User", SID_AND_ATTRIBUTES)]
|
||||
|
||||
sid = ctypes.cast(buffer, ctypes.POINTER(TOKEN_USER)).contents.User.Sid
|
||||
text = wintypes.LPWSTR()
|
||||
if not advapi32.ConvertSidToStringSidW(sid, ctypes.byref(text)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
try:
|
||||
return text.value
|
||||
finally:
|
||||
kernel32.LocalFree(text)
|
||||
finally:
|
||||
kernel32.CloseHandle(token)
|
||||
|
||||
|
||||
def _verify_windows_private_path(path: Path, directory: bool) -> None:
|
||||
kernel32, advapi32 = _windows_api()
|
||||
attributes = kernel32.GetFileAttributesW(str(path))
|
||||
if attributes == 0xFFFFFFFF:
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
if attributes & 0x400 or bool(attributes & 0x10) != directory:
|
||||
raise RuntimeError("Cutter bridge path is a reparse point or has the wrong object type")
|
||||
|
||||
class ACL_SIZE_INFORMATION(ctypes.Structure):
|
||||
_fields_ = [("AceCount", wintypes.DWORD), ("AclBytesInUse", wintypes.DWORD), ("AclBytesFree", wintypes.DWORD)]
|
||||
|
||||
class ACE_HEADER(ctypes.Structure):
|
||||
_fields_ = [("AceType", wintypes.BYTE), ("AceFlags", wintypes.BYTE), ("AceSize", wintypes.WORD)]
|
||||
|
||||
class SID_HEADER(ctypes.Structure):
|
||||
_fields_ = [("Revision", wintypes.BYTE), ("SubAuthorityCount", wintypes.BYTE),
|
||||
("IdentifierAuthority", wintypes.BYTE * 6)]
|
||||
|
||||
class ACCESS_ALLOWED_ACE(ctypes.Structure):
|
||||
_fields_ = [("Header", ACE_HEADER), ("Mask", wintypes.DWORD), ("SidStart", wintypes.DWORD)]
|
||||
|
||||
owner = wintypes.LPVOID()
|
||||
dacl = wintypes.LPVOID()
|
||||
descriptor = wintypes.LPVOID()
|
||||
result = advapi32.GetNamedSecurityInfoW(
|
||||
str(path), 1, 0x1 | 0x4, ctypes.byref(owner), None, ctypes.byref(dacl), None, ctypes.byref(descriptor),
|
||||
)
|
||||
if result != 0:
|
||||
raise ctypes.WinError(result)
|
||||
try:
|
||||
control = wintypes.WORD()
|
||||
revision = wintypes.DWORD()
|
||||
if not advapi32.GetSecurityDescriptorControl(descriptor, ctypes.byref(control), ctypes.byref(revision)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
if not control.value & 0x1000 or not dacl:
|
||||
raise RuntimeError("Cutter bridge ACL must be present and protected from inheritance")
|
||||
|
||||
token = wintypes.HANDLE()
|
||||
if not advapi32.OpenProcessToken(kernel32.GetCurrentProcess(), 0x0008, ctypes.byref(token)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
try:
|
||||
required = wintypes.DWORD()
|
||||
advapi32.GetTokenInformation(token, 1, None, 0, ctypes.byref(required))
|
||||
token_user = ctypes.create_string_buffer(required.value)
|
||||
if not advapi32.GetTokenInformation(token, 1, token_user, required, ctypes.byref(required)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
|
||||
class SID_AND_ATTRIBUTES(ctypes.Structure):
|
||||
_fields_ = [("Sid", wintypes.LPVOID), ("Attributes", wintypes.DWORD)]
|
||||
|
||||
class TOKEN_USER(ctypes.Structure):
|
||||
_fields_ = [("User", SID_AND_ATTRIBUTES)]
|
||||
|
||||
current_sid = ctypes.cast(token_user, ctypes.POINTER(TOKEN_USER)).contents.User.Sid
|
||||
if not advapi32.EqualSid(owner, current_sid):
|
||||
raise RuntimeError("Cutter bridge path owner does not match the current user")
|
||||
finally:
|
||||
kernel32.CloseHandle(token)
|
||||
|
||||
info = ACL_SIZE_INFORMATION()
|
||||
if not advapi32.GetAclInformation(dacl, ctypes.byref(info), ctypes.sizeof(info), 2) or info.AceCount != 2:
|
||||
raise RuntimeError("Cutter bridge ACL must allow only the current user and SYSTEM")
|
||||
expected_user = _windows_user_sid_text()
|
||||
user_allowed = False
|
||||
system_allowed = False
|
||||
for index in range(info.AceCount):
|
||||
ace_pointer = wintypes.LPVOID()
|
||||
if not advapi32.GetAce(dacl, index, ctypes.byref(ace_pointer)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
header = ctypes.cast(ace_pointer, ctypes.POINTER(ACE_HEADER)).contents
|
||||
if header.AceType != 0 or header.AceSize < ctypes.sizeof(ACE_HEADER) + ctypes.sizeof(wintypes.DWORD) + ctypes.sizeof(SID_HEADER):
|
||||
raise RuntimeError("Cutter bridge ACL contains a malformed or unsupported ACE")
|
||||
ace_address = ctypes.cast(ace_pointer, ctypes.c_void_p).value
|
||||
if ace_address is None:
|
||||
raise RuntimeError("Cutter bridge ACL returned a null ACE")
|
||||
sid_pointer = ctypes.c_void_p(ace_address + ctypes.sizeof(ACE_HEADER) + ctypes.sizeof(wintypes.DWORD))
|
||||
sid_header = ctypes.cast(sid_pointer, ctypes.POINTER(SID_HEADER)).contents
|
||||
sid_size = ctypes.sizeof(SID_HEADER) + sid_header.SubAuthorityCount * ctypes.sizeof(wintypes.DWORD)
|
||||
if sid_size > header.AceSize - ctypes.sizeof(ACE_HEADER) - ctypes.sizeof(wintypes.DWORD) or not advapi32.IsValidSid(sid_pointer) or advapi32.GetLengthSid(sid_pointer) != sid_size:
|
||||
raise RuntimeError("Cutter bridge ACL contains an invalid SID")
|
||||
ace = ctypes.cast(ace_pointer, ctypes.POINTER(ACCESS_ALLOWED_ACE)).contents
|
||||
sid_text = wintypes.LPWSTR()
|
||||
if ace.Mask != 0x001F01FF or ace.Header.AceFlags != 0x03:
|
||||
raise RuntimeError("Cutter bridge ACL contains an unsupported ACE")
|
||||
if not advapi32.ConvertSidToStringSidW(sid_pointer, ctypes.byref(sid_text)):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
try:
|
||||
if sid_text.value == expected_user:
|
||||
user_allowed = True
|
||||
elif sid_text.value == "S-1-5-18":
|
||||
system_allowed = True
|
||||
else:
|
||||
raise RuntimeError("Cutter bridge ACL allows an unrelated principal")
|
||||
finally:
|
||||
kernel32.LocalFree(sid_text)
|
||||
if not user_allowed or not system_allowed:
|
||||
raise RuntimeError("Cutter bridge ACL must allow only the current user and SYSTEM")
|
||||
finally:
|
||||
kernel32.LocalFree(descriptor)
|
||||
|
||||
|
||||
def _windows_security_attributes() -> tuple[Any, Any]:
|
||||
kernel32, advapi32 = _windows_api()
|
||||
sid = _windows_user_sid_text()
|
||||
sddl = f"O:{sid}G:{sid}D:P(A;OICI;FA;;;{sid})(A;OICI;FA;;;SY)"
|
||||
descriptor = wintypes.LPVOID()
|
||||
if not advapi32.ConvertStringSecurityDescriptorToSecurityDescriptorW(sddl, 1, ctypes.byref(descriptor), None):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
|
||||
class SECURITY_ATTRIBUTES(ctypes.Structure):
|
||||
_fields_ = [("nLength", wintypes.DWORD), ("lpSecurityDescriptor", wintypes.LPVOID), ("bInheritHandle", wintypes.BOOL)]
|
||||
|
||||
attributes = SECURITY_ATTRIBUTES(ctypes.sizeof(SECURITY_ATTRIBUTES), descriptor, False)
|
||||
return kernel32, (descriptor, attributes)
|
||||
|
||||
|
||||
def _ensure_windows_bridge_directory(directory: Path) -> None:
|
||||
directory.parent.mkdir(parents=True, exist_ok=True)
|
||||
if not directory.exists():
|
||||
kernel32, (descriptor, attributes) = _windows_security_attributes()
|
||||
try:
|
||||
if not kernel32.CreateDirectoryW(str(directory), ctypes.byref(attributes)):
|
||||
error = ctypes.get_last_error()
|
||||
if error != 183:
|
||||
raise ctypes.WinError(error)
|
||||
finally:
|
||||
kernel32.LocalFree(descriptor)
|
||||
_verify_windows_private_path(directory, True)
|
||||
|
||||
|
||||
def _ensure_posix_bridge_directory(directory: Path) -> None:
|
||||
directory.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
if not hasattr(os, "O_NOFOLLOW") or not hasattr(os, "O_DIRECTORY"):
|
||||
raise RuntimeError("Cutter bridge directory requires no-follow directory handles")
|
||||
descriptor = os.open(directory, os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW)
|
||||
try:
|
||||
directory_info = os.fstat(descriptor)
|
||||
if directory_info.st_uid != os.getuid():
|
||||
raise RuntimeError("Cutter bridge directory must be owned by the current user")
|
||||
os.fchmod(descriptor, 0o700)
|
||||
directory_info = os.fstat(descriptor)
|
||||
if stat.S_IMODE(directory_info.st_mode) != 0o700:
|
||||
raise RuntimeError("Cutter bridge directory must have mode 0700")
|
||||
finally:
|
||||
os.close(descriptor)
|
||||
|
||||
|
||||
def _write_windows_descriptor(directory: Path, destination: Path, payload: bytes) -> None:
|
||||
if len(payload) > MAX_DESCRIPTOR_BYTES:
|
||||
raise RuntimeError("Cutter bridge descriptor exceeds its byte limit")
|
||||
kernel32, (security_descriptor, attributes) = _windows_security_attributes()
|
||||
temporary = directory / f".{destination.name}.{uuid.uuid4().hex}.tmp"
|
||||
handle = wintypes.HANDLE()
|
||||
try:
|
||||
handle = kernel32.CreateFileW(str(temporary), 0x40000000 | 0x00020000 | 0x00040000,
|
||||
0x1 | 0x2 | 0x4, ctypes.byref(attributes), 1, 0x80, None)
|
||||
if handle == wintypes.HANDLE(-1).value:
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
_verify_windows_private_path(temporary, False)
|
||||
if payload:
|
||||
data = ctypes.create_string_buffer(payload)
|
||||
written = wintypes.DWORD()
|
||||
if not kernel32.WriteFile(handle, data, len(payload), ctypes.byref(written), None) or written.value != len(payload):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
if not kernel32.FlushFileBuffers(handle):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
kernel32.CloseHandle(handle)
|
||||
handle = wintypes.HANDLE()
|
||||
if not kernel32.MoveFileExW(str(temporary), str(destination), 0x1 | 0x8):
|
||||
raise ctypes.WinError(ctypes.get_last_error())
|
||||
finally:
|
||||
if handle and handle != wintypes.HANDLE(-1).value:
|
||||
kernel32.CloseHandle(handle)
|
||||
kernel32.DeleteFileW(str(temporary))
|
||||
kernel32.LocalFree(security_descriptor)
|
||||
|
||||
|
||||
def _bridge_directory() -> Path:
|
||||
configured = os.environ.get("REA_CUTTER_BRIDGE_DIR")
|
||||
if configured:
|
||||
return Path(configured).expanduser()
|
||||
local_app_data = os.environ.get("LOCALAPPDATA")
|
||||
if os.name == "nt" and local_app_data:
|
||||
return Path(local_app_data) / "REA" / "CutterBridge"
|
||||
return Path.home() / ".cache" / "rea" / "cutter-bridge"
|
||||
|
||||
|
||||
class _RequestDispatcher(QObject):
|
||||
dispatch = Signal(object, object)
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.dispatch.connect(self._handle)
|
||||
self.generation = 0
|
||||
self.completed_commands: dict[str, str] = {}
|
||||
self._receipt_lock = threading.Lock()
|
||||
self.current_file = self._read_current_file()
|
||||
try:
|
||||
value = QCoreApplication.applicationVersion()
|
||||
self.cutter_version = value if isinstance(value, str) and value else None
|
||||
except Exception:
|
||||
self.cutter_version = None
|
||||
|
||||
@staticmethod
|
||||
def _read_current_file() -> str | None:
|
||||
try:
|
||||
value = cutter.core().getConfig("file.path")
|
||||
return value if isinstance(value, str) and value else None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
def _handle(self, request: dict[str, Any], reply: dict[str, Any]) -> None:
|
||||
kind = request.get("kind")
|
||||
request_id = request.get("request_id")
|
||||
if kind in ("command", "ack"):
|
||||
try:
|
||||
valid_request_id = (
|
||||
isinstance(request_id, str)
|
||||
and str(uuid.UUID(request_id)) == request_id.lower()
|
||||
)
|
||||
except (ValueError, AttributeError):
|
||||
valid_request_id = False
|
||||
if not valid_request_id:
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": "invalid-command-request-id",
|
||||
"execution_state": "not_started",
|
||||
})
|
||||
request["_event"].set()
|
||||
return
|
||||
if kind == "ack":
|
||||
with self._receipt_lock:
|
||||
acknowledged = self.completed_commands.pop(request_id, None) is not None
|
||||
reply.update({"ok": True, "acknowledged": acknowledged})
|
||||
request["_event"].set()
|
||||
return
|
||||
if request["_cancelled"].is_set():
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": "request-cancelled-before-execution",
|
||||
"execution_state": "not_started",
|
||||
})
|
||||
if kind == "command":
|
||||
self._record_command_completion(request_id, "not_started")
|
||||
request["_event"].set()
|
||||
return
|
||||
self._refresh_document_identity()
|
||||
if kind == "status":
|
||||
with self._receipt_lock:
|
||||
completed_commands = [
|
||||
{"request_id": receipt_id, "execution_state": state}
|
||||
for receipt_id, state in self.completed_commands.items()
|
||||
]
|
||||
reply.update({
|
||||
"ok": True,
|
||||
"session_id": request.get("session_id"),
|
||||
"current_file": self.current_file,
|
||||
"document_generation": self.generation,
|
||||
"cutter_version": self.cutter_version,
|
||||
"identity_status": "partial",
|
||||
"completed_commands": completed_commands,
|
||||
})
|
||||
request["_event"].set()
|
||||
return
|
||||
if kind != "command":
|
||||
reply.update({"ok": False, "error": "unsupported-request-kind"})
|
||||
request["_event"].set()
|
||||
return
|
||||
if request.get("expected_generation") != self.generation:
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": "active-document-generation-changed",
|
||||
"execution_state": "not_started",
|
||||
})
|
||||
self._record_command_completion(request_id, "not_started")
|
||||
request["_event"].set()
|
||||
return
|
||||
command = request.get("command")
|
||||
if not isinstance(command, str) or not command.strip():
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": "invalid-command",
|
||||
"execution_state": "not_started",
|
||||
})
|
||||
self._record_command_completion(request_id, "not_started")
|
||||
request["_event"].set()
|
||||
return
|
||||
reply["execution_state"] = "running"
|
||||
try:
|
||||
output = cutter.cmdj(command) if request.get("json") is True else cutter.cmd(command)
|
||||
command_error = None
|
||||
except Exception as error:
|
||||
output = None
|
||||
command_error = str(error)
|
||||
self._refresh_document_identity()
|
||||
if command_error is None:
|
||||
reply.update({
|
||||
"ok": True,
|
||||
"output": output,
|
||||
"current_file": self.current_file,
|
||||
"document_generation": self.generation,
|
||||
"cutter_version": self.cutter_version,
|
||||
"identity_status": "partial",
|
||||
"execution_state": "complete",
|
||||
})
|
||||
self._record_command_completion(request_id, "complete")
|
||||
else:
|
||||
reply.update({
|
||||
"ok": False,
|
||||
"error": command_error,
|
||||
"current_file": self.current_file,
|
||||
"document_generation": self.generation,
|
||||
"cutter_version": self.cutter_version,
|
||||
"identity_status": "partial",
|
||||
"execution_state": "unknown",
|
||||
"message": "The command raised an exception; it may have produced partial effects. Do not retry automatically.",
|
||||
})
|
||||
self._record_command_completion(request_id, "failed")
|
||||
request["_event"].set()
|
||||
|
||||
def _record_command_completion(
|
||||
self, request_id: str, execution_state: str
|
||||
) -> None:
|
||||
with self._receipt_lock:
|
||||
if request_id in self.completed_commands or len(self.completed_commands) < MAX_COMPLETION_RECEIPTS:
|
||||
self.completed_commands[request_id] = execution_state
|
||||
|
||||
def can_accept_command(self) -> bool:
|
||||
with self._receipt_lock:
|
||||
return len(self.completed_commands) < MAX_COMPLETION_RECEIPTS
|
||||
|
||||
def _refresh_document_identity(self) -> None:
|
||||
current_file = self._read_current_file()
|
||||
if current_file != self.current_file:
|
||||
self.current_file = current_file
|
||||
self.generation += 1
|
||||
|
||||
|
||||
class _BridgeServer(threading.Thread):
|
||||
def __init__(self, dispatcher: _RequestDispatcher) -> None:
|
||||
super().__init__(name="rea-cutter-bridge", daemon=True)
|
||||
self.dispatcher = dispatcher
|
||||
self.server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
self.server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
self.server.bind(("127.0.0.1", 0))
|
||||
self.server.listen(8)
|
||||
self.server.settimeout(0.5)
|
||||
self.port = self.server.getsockname()[1]
|
||||
self.session_id = str(uuid.uuid4())
|
||||
self.token = secrets.token_urlsafe(32)
|
||||
self.poisoned = False
|
||||
self.connection_ordinal = 0
|
||||
self.directory = _bridge_directory()
|
||||
self.descriptor = self.directory / f"cutter-{os.getpid()}-{self.session_id}.json"
|
||||
|
||||
def publish(self) -> None:
|
||||
if os.name == "nt":
|
||||
_ensure_windows_bridge_directory(self.directory)
|
||||
else:
|
||||
_ensure_posix_bridge_directory(self.directory)
|
||||
if os.name == "nt" and (self.directory.is_symlink() or not self.directory.is_dir()):
|
||||
raise RuntimeError("Cutter bridge directory must be a real directory")
|
||||
payload = {
|
||||
"session_id": self.session_id,
|
||||
"pid": os.getpid(),
|
||||
"host": "127.0.0.1",
|
||||
"port": self.port,
|
||||
"token": self.token,
|
||||
"document_generation": self.dispatcher.generation,
|
||||
"current_file": self.dispatcher.current_file,
|
||||
"cutter_version": self.dispatcher.cutter_version,
|
||||
"identity_status": "partial",
|
||||
}
|
||||
if os.name == "nt":
|
||||
_write_windows_descriptor(self.directory, self.descriptor, json.dumps(payload).encode("utf-8"))
|
||||
return
|
||||
descriptor, temporary_path = tempfile.mkstemp(prefix=f".{self.descriptor.name}.", dir=self.directory)
|
||||
try:
|
||||
os.fchmod(descriptor, 0o600)
|
||||
descriptor_info = os.fstat(descriptor)
|
||||
if descriptor_info.st_uid != os.getuid() or stat.S_IMODE(descriptor_info.st_mode) & 0o077:
|
||||
raise RuntimeError("Cutter bridge descriptor must be owned by the current user with mode 0600")
|
||||
with os.fdopen(descriptor, "w", encoding="utf-8") as stream:
|
||||
stream.write(json.dumps(payload))
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
os.replace(temporary_path, self.descriptor)
|
||||
except Exception:
|
||||
try:
|
||||
os.close(descriptor)
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
os.unlink(temporary_path)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
def run(self) -> None:
|
||||
try:
|
||||
self.publish()
|
||||
except Exception:
|
||||
self.server.close()
|
||||
return
|
||||
while True:
|
||||
try:
|
||||
connection, _ = self.server.accept()
|
||||
except socket.timeout:
|
||||
if getattr(self, "stopping", False):
|
||||
break
|
||||
continue
|
||||
except OSError:
|
||||
break
|
||||
self.connection_ordinal += 1
|
||||
connection_ordinal = self.connection_ordinal
|
||||
with connection:
|
||||
connection.settimeout(AUTH_FRAME_TIMEOUT_SECONDS)
|
||||
authenticated = False
|
||||
try:
|
||||
hello = self._receive_frame(connection)
|
||||
if not isinstance(hello, dict) or hello.get("kind") != "rea-auth-v2" or not isinstance(hello.get("nonce"), str) or len(hello["nonce"]) != 64:
|
||||
self._send(connection, {"ok": False, "error": "authentication-failed"})
|
||||
continue
|
||||
client_nonce = hello["nonce"]
|
||||
server_nonce = f"{connection_ordinal:016x}{secrets.token_hex(24)}"
|
||||
server_proof = hmac.new(self.token.encode(), f"rea-cutter-server-v2:{self.session_id}:{client_nonce}:{server_nonce}".encode(), hashlib.sha256).hexdigest()
|
||||
self._send(connection, {"session_id": self.session_id, "nonce": server_nonce, "proof": server_proof})
|
||||
authenticated_frame = self._receive_frame(connection)
|
||||
request_bytes = authenticated_frame.get("request") if isinstance(authenticated_frame, dict) else None
|
||||
expected_proof = hmac.new(self.token.encode(), f"rea-cutter-client-v2:{self.session_id}:{client_nonce}:{server_nonce}:{request_bytes}".encode(), hashlib.sha256).hexdigest()
|
||||
if not isinstance(request_bytes, str) or not isinstance(authenticated_frame.get("proof"), str) or not hmac.compare_digest(authenticated_frame["proof"], expected_proof):
|
||||
self._send(connection, {"ok": False, "error": "authentication-failed"})
|
||||
continue
|
||||
request = json.loads(request_bytes)
|
||||
if not isinstance(request, dict) or request.get("session_id") != self.session_id:
|
||||
self._send(connection, {"ok": False, "error": "authentication-failed"})
|
||||
continue
|
||||
authenticated = True
|
||||
connection.settimeout(SOCKET_TIMEOUT_SECONDS)
|
||||
if self.poisoned and request.get("kind") not in ("status", "ack"):
|
||||
self._send_authenticated(connection, {
|
||||
"ok": False,
|
||||
"error": "session-poisoned-after-ui-timeout",
|
||||
"execution_state": "not_started",
|
||||
}, client_nonce, server_nonce)
|
||||
continue
|
||||
if (
|
||||
request.get("kind") == "command"
|
||||
and not self.dispatcher.can_accept_command()
|
||||
):
|
||||
self._send_authenticated(connection, {
|
||||
"ok": False,
|
||||
"error": "completion-receipt-capacity-exceeded",
|
||||
"execution_state": "not_started",
|
||||
}, client_nonce, server_nonce)
|
||||
continue
|
||||
request["session_id"] = self.session_id
|
||||
event = threading.Event()
|
||||
cancelled = threading.Event()
|
||||
reply: dict[str, Any] = {}
|
||||
request["_event"] = event
|
||||
request["_cancelled"] = cancelled
|
||||
self.dispatcher.dispatch.emit(request, reply)
|
||||
if not event.wait(SOCKET_TIMEOUT_SECONDS):
|
||||
cancelled.set()
|
||||
self.poisoned = True
|
||||
self._send_authenticated(connection, {
|
||||
"ok": False,
|
||||
"error": "ui-thread-request-timeout",
|
||||
"execution_state": "unknown",
|
||||
"message": "The UI operation may still complete after this timeout; restart Cutter before issuing further commands.",
|
||||
}, client_nonce, server_nonce)
|
||||
else:
|
||||
if request.get("kind") == "status" and reply.get("ok") is True:
|
||||
reply["completed_connection"] = f"{connection_ordinal - 1:016x}"
|
||||
self._send_authenticated(connection, reply, client_nonce, server_nonce)
|
||||
except socket.timeout:
|
||||
if authenticated:
|
||||
self._send_authenticated(connection, {
|
||||
"ok": False,
|
||||
"error": "request-timeout",
|
||||
"execution_state": "unknown",
|
||||
"message": "The command may have produced partial or persistent effects. Do not retry automatically.",
|
||||
}, client_nonce, server_nonce)
|
||||
except Exception as error:
|
||||
self._send(connection, {"ok": False, "error": str(error)})
|
||||
self.server.close()
|
||||
try:
|
||||
self.descriptor.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def _receive_frame(connection: socket.socket) -> dict[str, Any]:
|
||||
data = bytearray()
|
||||
deadline = time.monotonic() + AUTH_FRAME_TIMEOUT_SECONDS
|
||||
while b"\n" not in data:
|
||||
if len(data) >= MAX_FRAME_BYTES:
|
||||
raise ValueError("frame-limit-exceeded")
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0:
|
||||
raise socket.timeout("authentication frame timed out")
|
||||
connection.settimeout(remaining)
|
||||
part = connection.recv(min(65536, MAX_FRAME_BYTES + 1 - len(data)))
|
||||
if not part:
|
||||
raise ValueError("incomplete authentication frame")
|
||||
data.extend(part)
|
||||
line, remainder = bytes(data).split(b"\n", 1)
|
||||
if len(line) > MAX_FRAME_BYTES or remainder:
|
||||
raise ValueError("invalid framed request")
|
||||
value = json.loads(line)
|
||||
return value if isinstance(value, dict) else {}
|
||||
|
||||
def _send_authenticated(self, connection: socket.socket, value: dict[str, Any], client_nonce: str, server_nonce: str) -> None:
|
||||
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
|
||||
if encoded is None:
|
||||
value = self._truncated_response(value)
|
||||
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
|
||||
if encoded is None:
|
||||
# This fixed-size fallback is checked through the same signed framing path.
|
||||
value = {
|
||||
"ok": False,
|
||||
"error": "response-limit-exceeded",
|
||||
"execution_state": value.get("execution_state", "unknown"),
|
||||
"output": None,
|
||||
"output_truncated": True,
|
||||
"message": "The result exceeded the bridge frame limit; command completion state is preserved.",
|
||||
}
|
||||
encoded = self._encode_authenticated_response(value, client_nonce, server_nonce)
|
||||
if encoded is None:
|
||||
return
|
||||
try:
|
||||
connection.sendall(encoded)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def _encode_authenticated_response(self, value: dict[str, Any], client_nonce: str, server_nonce: str) -> bytes | None:
|
||||
if not _within_response_limit(value):
|
||||
return None
|
||||
prefix = b'{"response":"'
|
||||
suffix = b'","proof":"' + (b"0" * 64) + b'"}\n'
|
||||
frame_size = len(prefix) + len(suffix)
|
||||
response_parts: list[bytes] = []
|
||||
escaped_parts: list[bytes] = []
|
||||
for fragment in json.JSONEncoder(ensure_ascii=False, separators=(",", ":")).iterencode(value):
|
||||
raw = fragment.encode("utf-8")
|
||||
escaped = json.dumps(fragment, ensure_ascii=False)[1:-1].encode("utf-8")
|
||||
frame_size += len(escaped)
|
||||
if frame_size > MAX_FRAME_BYTES:
|
||||
return None
|
||||
response_parts.append(raw)
|
||||
escaped_parts.append(escaped)
|
||||
proof = hmac.new(self.token.encode(), digestmod=hashlib.sha256)
|
||||
proof.update(f"rea-cutter-response-v2:{self.session_id}:{client_nonce}:{server_nonce}:".encode())
|
||||
for part in response_parts:
|
||||
proof.update(part)
|
||||
return prefix + b"".join(escaped_parts) + b'","proof":"' + proof.hexdigest().encode() + b'"}\n'
|
||||
|
||||
@staticmethod
|
||||
def _truncated_response(value: dict[str, Any]) -> dict[str, Any]:
|
||||
response: dict[str, Any] = {
|
||||
"ok": value.get("ok") is True,
|
||||
"output": None,
|
||||
"execution_state": value.get("execution_state", "unknown"),
|
||||
"output_truncated": True,
|
||||
"message": "The command result exceeded the bridge frame limit; completion state is preserved and output was omitted.",
|
||||
}
|
||||
for key in ("current_file", "document_generation", "cutter_version", "identity_status"):
|
||||
if key in value:
|
||||
response[key] = value[key]
|
||||
if value.get("ok") is not True and isinstance(value.get("error"), str):
|
||||
response["error"] = value["error"]
|
||||
return response
|
||||
|
||||
@staticmethod
|
||||
def _send(connection: socket.socket, value: dict[str, Any]) -> None:
|
||||
if not _within_response_limit(value):
|
||||
value = {
|
||||
"ok": False,
|
||||
"error": "response-limit-exceeded",
|
||||
"execution_state": value.get("execution_state", "unknown"),
|
||||
"output_truncated": True,
|
||||
"message": "The command completed or may have completed, but its result exceeded the bridge response limit. Do not retry automatically.",
|
||||
}
|
||||
parts: list[bytes] = []
|
||||
encoded_size = 0
|
||||
for part in json.JSONEncoder(ensure_ascii=False).iterencode(value):
|
||||
encoded_part = part.encode("utf-8")
|
||||
encoded_size += len(encoded_part)
|
||||
if encoded_size > MAX_FRAME_BYTES:
|
||||
fallback = {
|
||||
"ok": False,
|
||||
"error": "response-limit-exceeded",
|
||||
"execution_state": value.get("execution_state", "unknown"),
|
||||
"output_truncated": True,
|
||||
"message": "The command completed or may have completed, but its result exceeded the bridge response limit. Do not retry automatically.",
|
||||
}
|
||||
parts = [json.dumps(fallback, separators=(",", ":")).encode("utf-8")]
|
||||
break
|
||||
parts.append(encoded_part)
|
||||
encoded = b"".join(parts)
|
||||
try:
|
||||
connection.sendall(encoded + b"\n")
|
||||
except OSError:
|
||||
# A disconnected caller must not terminate the bridge thread or
|
||||
# prevent later commands from being served.
|
||||
pass
|
||||
|
||||
def stop(self) -> None:
|
||||
self.stopping = True
|
||||
try:
|
||||
self.server.close()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
class ReaCutterPlugin(cutter.CutterPlugin):
|
||||
name = "REA Local Bridge"
|
||||
description = "Local REA analysis bridge for the active Cutter session"
|
||||
version = "1.0"
|
||||
author = "REA"
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self._dispatcher: _RequestDispatcher | None = None
|
||||
self._bridge: _BridgeServer | None = None
|
||||
|
||||
def setupPlugin(self) -> None:
|
||||
return
|
||||
|
||||
def setupInterface(self, main: Any) -> None:
|
||||
self._dispatcher = _RequestDispatcher()
|
||||
self._bridge = _BridgeServer(self._dispatcher)
|
||||
self._bridge.start()
|
||||
|
||||
def terminate(self) -> None:
|
||||
if self._bridge is not None:
|
||||
self._bridge.stop()
|
||||
self._bridge.join(timeout=2)
|
||||
self._bridge = None
|
||||
self._dispatcher = None
|
||||
|
||||
|
||||
def create_cutter_plugin() -> ReaCutterPlugin:
|
||||
return ReaCutterPlugin()
|
||||
|
||||
|
||||
def _within_response_limit(value: Any) -> bool:
|
||||
pending = [value]
|
||||
seen: set[int] = set()
|
||||
size = 0
|
||||
while pending:
|
||||
current = pending.pop()
|
||||
if isinstance(current, str):
|
||||
size += 2
|
||||
if len(current) > MAX_FRAME_BYTES - size:
|
||||
return False
|
||||
if current.isascii():
|
||||
size += len(current)
|
||||
else:
|
||||
# Count UTF-8 bytes without allocating a copy of the whole output.
|
||||
try:
|
||||
size += sum(
|
||||
len(current[start:start + 65536].encode("utf-8"))
|
||||
for start in range(0, len(current), 65536)
|
||||
)
|
||||
except UnicodeEncodeError:
|
||||
# Unencodable output also needs the signed omission fallback.
|
||||
return False
|
||||
size += current.count('"') + current.count("\\")
|
||||
size += sum(5 for character in current if ord(character) < 0x20)
|
||||
elif isinstance(current, dict):
|
||||
identity = id(current)
|
||||
if identity in seen:
|
||||
return False
|
||||
seen.add(identity)
|
||||
size += 2 + 4 * len(current)
|
||||
if size > MAX_FRAME_BYTES:
|
||||
return False
|
||||
pending.extend(current.keys())
|
||||
pending.extend(current.values())
|
||||
elif isinstance(current, (list, tuple)):
|
||||
identity = id(current)
|
||||
if identity in seen:
|
||||
return False
|
||||
seen.add(identity)
|
||||
size += 2 + 2 * len(current)
|
||||
if size > MAX_FRAME_BYTES:
|
||||
return False
|
||||
pending.extend(current)
|
||||
else:
|
||||
size += 16
|
||||
if size > MAX_FRAME_BYTES:
|
||||
return False
|
||||
return True
|
||||
@@ -1,38 +0,0 @@
|
||||
import java.util.Map;
|
||||
|
||||
import ghidra.app.plugin.core.analysis.AutoAnalysisManager;
|
||||
import ghidra.app.util.headless.HeadlessScript;
|
||||
|
||||
/** Disable the named auto-analyzers before default auto-analysis. */
|
||||
public final class ReaGhidraAnalysisOptions extends HeadlessScript {
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
if (arguments.length != 1 || arguments[0].isBlank()) {
|
||||
throw new IllegalArgumentException(
|
||||
"ReaGhidraAnalysisOptions requires one comma-separated analyzer-name argument");
|
||||
}
|
||||
String[] names = arguments[0].split(",");
|
||||
AutoAnalysisManager analysis = AutoAnalysisManager.getAnalysisManager(currentProgram);
|
||||
Map<String, String> options = getCurrentAnalysisOptionsAndValues(currentProgram);
|
||||
for (String requestedName : names) {
|
||||
monitor.checkCancelled();
|
||||
String name = requestedName.trim();
|
||||
if (analysis.getAnalyzer(name) == null || !options.containsKey(name)) {
|
||||
throw new IllegalArgumentException("Unknown Ghidra analyzer for this program: " + name);
|
||||
}
|
||||
}
|
||||
for (String requestedName : names) {
|
||||
monitor.checkCancelled();
|
||||
setAnalysisOption(currentProgram, requestedName.trim(), "false");
|
||||
}
|
||||
options = getCurrentAnalysisOptionsAndValues(currentProgram);
|
||||
for (String requestedName : names) {
|
||||
monitor.checkCancelled();
|
||||
String name = requestedName.trim();
|
||||
if (!"false".equals(options.get(name))) {
|
||||
throw new IllegalStateException("Ghidra analyzer remained enabled: " + name);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,151 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.address.AddressSpace;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.mem.MemoryBlock;
|
||||
import ghidra.program.model.symbol.SourceType;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.LinkOption;
|
||||
import java.nio.file.Path;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HexFormat;
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* Apply caller-declared analysis seeds before default auto-analysis.
|
||||
*
|
||||
* The single argument is REA's canonical seed snapshot: one
|
||||
* {@code 0x<hex>\t(function|code|label)[\tname]} line per seed. Code seeds are
|
||||
* applied after functions (highest address first), then labels. The outcome is stored as a
|
||||
* program property that the bridge reports during its handshake.
|
||||
*/
|
||||
public final class ReaGhidraApplySeeds extends GhidraScript {
|
||||
static final String REPORT_OPTION = "REA analysis seeds";
|
||||
|
||||
private record Seed(Address address, String kind, String name) {}
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] args = getScriptArgs();
|
||||
if (args.length != 1) {
|
||||
throw new IllegalArgumentException("REA seed script requires exactly one seed file");
|
||||
}
|
||||
Path path = Path.of(args[0]);
|
||||
if (!Files.isRegularFile(path, LinkOption.NOFOLLOW_LINKS)) {
|
||||
throw new IllegalArgumentException("REA seed file is not a regular file");
|
||||
}
|
||||
byte[] bytes = Files.readAllBytes(path);
|
||||
String sha256 = HexFormat.of().formatHex(
|
||||
MessageDigest.getInstance("SHA-256").digest(bytes)
|
||||
);
|
||||
AddressSpace space = currentProgram.getAddressFactory().getDefaultAddressSpace();
|
||||
List<Seed> seeds = new ArrayList<>();
|
||||
for (String line : new String(bytes, StandardCharsets.UTF_8).split("\n")) {
|
||||
if (line.isEmpty()) continue;
|
||||
String[] fields = line.split("\t", -1);
|
||||
if (fields.length < 2 || fields.length > 3 || !fields[0].startsWith("0x")) {
|
||||
throw new IllegalArgumentException("REA seed snapshot is malformed");
|
||||
}
|
||||
Address address;
|
||||
try {
|
||||
address = space.getAddress(Long.parseUnsignedLong(fields[0].substring(2), 16));
|
||||
} catch (ghidra.program.model.address.AddressOutOfBoundsException e) {
|
||||
address = null;
|
||||
}
|
||||
seeds.add(new Seed(address, fields[1], fields.length == 3 ? fields[2] : null));
|
||||
}
|
||||
|
||||
// Highest address first: a function body computed before analysis stops at
|
||||
// an existing function entry, so later creations cannot swallow earlier ones.
|
||||
seeds.sort((left, right) -> {
|
||||
if (left.address() == null) return right.address() == null ? 0 : 1;
|
||||
if (right.address() == null) return -1;
|
||||
return right.address().compareTo(left.address());
|
||||
});
|
||||
int unmapped = 0;
|
||||
int logged = 0;
|
||||
int codeDecoded = 0, codeFailed = 0;
|
||||
int functionCreated = 0, functionExisting = 0, functionFailed = 0;
|
||||
int labelApplied = 0, labelFailed = 0;
|
||||
for (String pass : new String[] { "function", "code", "label" }) {
|
||||
for (Seed seed : seeds) {
|
||||
if (!seed.kind().equals(pass)) continue;
|
||||
monitor.checkCancelled();
|
||||
MemoryBlock block = seed.address() == null
|
||||
? null
|
||||
: currentProgram.getMemory().getBlock(seed.address());
|
||||
if (block == null || (!pass.equals("label") && !block.isInitialized())) {
|
||||
unmapped++;
|
||||
continue;
|
||||
}
|
||||
switch (pass) {
|
||||
case "code" -> {
|
||||
if (decoded(seed.address())) codeDecoded++;
|
||||
else {
|
||||
codeFailed++;
|
||||
if (logged++ < 20) println("REA seed failed: code " + seed.address() + " " + why(seed.address()));
|
||||
}
|
||||
}
|
||||
case "function" -> {
|
||||
Function existing = getFunctionAt(seed.address());
|
||||
if (existing != null) {
|
||||
functionExisting++;
|
||||
if (seed.name() != null) {
|
||||
existing.setName(seed.name(), SourceType.USER_DEFINED);
|
||||
}
|
||||
} else if (decoded(seed.address()) &&
|
||||
createFunction(seed.address(), seed.name()) != null) {
|
||||
functionCreated++;
|
||||
} else {
|
||||
functionFailed++;
|
||||
if (logged++ < 20) println("REA seed failed: function " + seed.address() + " " + why(seed.address()));
|
||||
}
|
||||
}
|
||||
default -> {
|
||||
try {
|
||||
createLabel(seed.address(), seed.name(), true, SourceType.USER_DEFINED);
|
||||
labelApplied++;
|
||||
} catch (Exception e) {
|
||||
labelFailed++;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
String report = "{\"format\":\"rea-ghidra-seeds-v1\",\"sha256\":\"" + sha256 + "\"" +
|
||||
",\"entries\":" + seeds.size() +
|
||||
",\"function_created\":" + functionCreated +
|
||||
",\"function_existing\":" + functionExisting +
|
||||
",\"function_failed\":" + functionFailed +
|
||||
",\"code_decoded\":" + codeDecoded +
|
||||
",\"code_failed\":" + codeFailed +
|
||||
",\"label_applied\":" + labelApplied +
|
||||
",\"label_failed\":" + labelFailed +
|
||||
",\"unmapped\":" + unmapped + "}";
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(REPORT_OPTION, report);
|
||||
println("REA seeds applied: " + report);
|
||||
}
|
||||
|
||||
private String why(Address address) {
|
||||
if (getDataAt(address) != null) return "defined data " + getDataAt(address).getDataType().getName();
|
||||
var containing = getInstructionContaining(address);
|
||||
if (containing != null && !containing.getAddress().equals(address))
|
||||
return "inside instruction at " + containing.getAddress();
|
||||
Function owner = getFunctionContaining(address);
|
||||
if (owner != null) return "inside function " + owner.getEntryPoint();
|
||||
return getInstructionAt(address) == null ? "undecodable" : "function creation refused";
|
||||
}
|
||||
|
||||
/** Decode one instruction flow at an address, keeping an existing decode. */
|
||||
private boolean decoded(Address address) {
|
||||
if (getInstructionAt(address) != null) return true;
|
||||
if (getDataAt(address) != null || getInstructionContaining(address) != null) {
|
||||
return false;
|
||||
}
|
||||
disassemble(address);
|
||||
return getInstructionAt(address) != null;
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,14 +0,0 @@
|
||||
import org.osgi.framework.BundleActivator;
|
||||
import org.osgi.framework.BundleContext;
|
||||
|
||||
// Compile the no-op activator with REA's scripts. Otherwise Ghidra generates
|
||||
// one afterward and opens a second Java compiler and file manager.
|
||||
public final class ReaGhidraBundleActivator implements BundleActivator {
|
||||
@Override
|
||||
public void start(BundleContext context) {
|
||||
}
|
||||
|
||||
@Override
|
||||
public void stop(BundleContext context) {
|
||||
}
|
||||
}
|
||||
@@ -1,113 +0,0 @@
|
||||
import com.google.gson.JsonArray;
|
||||
import com.google.gson.JsonNull;
|
||||
import com.google.gson.JsonObject;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.data.DataType;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.util.exception.CancelledException;
|
||||
import ghidra.util.task.TaskMonitor;
|
||||
import java.lang.reflect.InvocationTargetException;
|
||||
import java.net.URLClassLoader;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.LinkOption;
|
||||
import java.nio.file.Path;
|
||||
import java.security.MessageDigest;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
/** Session-owned extension loading; analyzer algorithms stay behind each adapter ABI. */
|
||||
final class ReaGhidraExtensions implements AutoCloseable {
|
||||
private record Loaded(URLClassLoader loader, Object instance, String sha256) {}
|
||||
private final List<Loaded> loaded = new ArrayList<>();
|
||||
private final JsonArray reports = new JsonArray();
|
||||
|
||||
void analyze(JsonArray descriptors, Program program, TaskMonitor monitor) throws Exception {
|
||||
for (var item : descriptors) {
|
||||
monitor.checkCancelled();
|
||||
JsonObject descriptor = item.getAsJsonObject();
|
||||
String id = descriptor.get("id").getAsString();
|
||||
String entry = descriptor.get("entry_class").getAsString();
|
||||
String digest = descriptor.get("sha256").getAsString();
|
||||
JsonObject report = new JsonObject();
|
||||
report.addProperty("id", id);
|
||||
report.addProperty("sha256", digest);
|
||||
URLClassLoader loader = null;
|
||||
try {
|
||||
if (!id.equals("nativeaot") || !entry.equals("rea.extensions.nativeaot.NativeAotExtension") ||
|
||||
descriptor.get("integration_api").getAsInt() != 1)
|
||||
throw new IllegalArgumentException("Unregistered Ghidra extension ABI: " + id);
|
||||
Path path = Path.of(descriptor.get("path").getAsString());
|
||||
if (!path.isAbsolute() || !Files.isRegularFile(path, LinkOption.NOFOLLOW_LINKS) || Files.size(path) > 8 * 1024 * 1024)
|
||||
throw new IllegalArgumentException("Invalid private extension artifact: " + path);
|
||||
String measured = java.util.HexFormat.of().formatHex(MessageDigest.getInstance("SHA-256").digest(Files.readAllBytes(path)));
|
||||
if (!measured.equals(digest)) throw new IllegalArgumentException("Private extension digest mismatch: " + id);
|
||||
loader = new URLClassLoader(new java.net.URL[] { path.toUri().toURL() }, getClass().getClassLoader());
|
||||
Class<?> type = Class.forName(entry, true, loader);
|
||||
Object instance = type.getConstructor().newInstance();
|
||||
Object response = type.getMethod("analyze", Program.class, TaskMonitor.class).invoke(instance, program, monitor);
|
||||
if (!(response instanceof JsonObject result)) throw new IllegalArgumentException("Invalid Ghidra extension result: " + id);
|
||||
report.addProperty("status", result.get("status").getAsString());
|
||||
report.add("reason", result.get("reason").deepCopy());
|
||||
report.add("result", result);
|
||||
loaded.add(new Loaded(loader, instance, digest));
|
||||
loader = null;
|
||||
} catch (InvocationTargetException ex) {
|
||||
if (ex.getCause() instanceof CancelledException cancelled) throw cancelled;
|
||||
failure(report, ex.getCause());
|
||||
} catch (Exception | LinkageError ex) {
|
||||
failure(report, ex);
|
||||
} finally {
|
||||
if (loader != null) loader.close();
|
||||
}
|
||||
reports.add(report);
|
||||
}
|
||||
}
|
||||
|
||||
JsonArray reports() { return reports.deepCopy(); }
|
||||
|
||||
JsonArray inspectImage() throws Exception {
|
||||
JsonArray summaries = new JsonArray();
|
||||
for (Loaded extension : loaded) {
|
||||
Object response = extension.instance.getClass().getMethod("inspectImage").invoke(extension.instance);
|
||||
if (!(response instanceof JsonObject summary)) throw new IllegalArgumentException("Invalid extension image summary");
|
||||
summary.addProperty("analysis_artifact_sha256", extension.sha256);
|
||||
summaries.add(summary);
|
||||
}
|
||||
return summaries;
|
||||
}
|
||||
|
||||
JsonObject inspectDataType(Program program, DataType type, Address address, TaskMonitor monitor) throws Exception {
|
||||
for (Loaded extension : loaded) {
|
||||
try {
|
||||
Object response = extension.instance.getClass().getMethod("inspectDataType", Program.class, DataType.class, Address.class, TaskMonitor.class)
|
||||
.invoke(extension.instance, program, type, address, monitor);
|
||||
if (response instanceof JsonObject result) return result;
|
||||
if (response != null) throw new IllegalArgumentException("Invalid extension type metadata result");
|
||||
} catch (InvocationTargetException ex) {
|
||||
if (ex.getCause() instanceof CancelledException cancelled) throw cancelled;
|
||||
throw new IllegalStateException("Ghidra extension type inspection failed: " + ex.getCause(), ex.getCause());
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private static void failure(JsonObject report, Throwable ex) {
|
||||
String reason = "Extension loading/analysis failed: " + ex;
|
||||
if (ex.getCause() != null) reason += "; caused by " + ex.getCause();
|
||||
report.addProperty("status", "failed");
|
||||
report.addProperty("reason", reason);
|
||||
JsonObject result = new JsonObject();
|
||||
result.addProperty("loader_failure", reason);
|
||||
report.add("result", result);
|
||||
}
|
||||
|
||||
@Override public void close() throws Exception {
|
||||
Exception failure = null;
|
||||
for (Loaded extension : loaded) {
|
||||
try { extension.loader.close(); }
|
||||
catch (Exception ex) { if (failure == null) failure = ex; else failure.addSuppressed(ex); }
|
||||
}
|
||||
loaded.clear();
|
||||
if (failure != null) throw failure;
|
||||
}
|
||||
}
|
||||
@@ -1,176 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.database.SpecExtension;
|
||||
import ghidra.program.model.data.AbstractFloatDataType;
|
||||
import ghidra.program.model.data.DataType;
|
||||
import ghidra.program.model.data.DoubleDataType;
|
||||
import ghidra.program.model.data.FloatDataType;
|
||||
import ghidra.program.model.data.FunctionDefinition;
|
||||
import ghidra.program.model.data.IntegerDataType;
|
||||
import ghidra.program.model.data.ParameterDefinition;
|
||||
import ghidra.program.model.data.TypeDef;
|
||||
import ghidra.program.model.lang.CompilerSpec;
|
||||
import ghidra.program.model.lang.PrototypeModel;
|
||||
import ghidra.program.model.lang.Register;
|
||||
import ghidra.program.model.listing.Function;
|
||||
import ghidra.program.model.listing.Parameter;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.listing.VariableStorage;
|
||||
import ghidra.program.model.pcode.XmlEncode;
|
||||
import java.io.StringReader;
|
||||
import java.io.StringWriter;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Iterator;
|
||||
import java.util.List;
|
||||
import java.util.regex.Pattern;
|
||||
import javax.xml.parsers.DocumentBuilderFactory;
|
||||
import javax.xml.transform.OutputKeys;
|
||||
import javax.xml.transform.TransformerFactory;
|
||||
import javax.xml.transform.dom.DOMSource;
|
||||
import javax.xml.transform.stream.StreamResult;
|
||||
import org.w3c.dom.Document;
|
||||
import org.w3c.dom.Element;
|
||||
import org.w3c.dom.Node;
|
||||
import org.w3c.dom.NodeList;
|
||||
import org.xml.sax.InputSource;
|
||||
|
||||
/**
|
||||
* Soft-float o32 passes and returns float/double values exactly where same-size integers go.
|
||||
* Ghidra's MIPS32 compiler specifications model only hard-float o32, so every typed float
|
||||
* prototype (functions, externals, thunks via their targets, and function-pointer definitions)
|
||||
* would otherwise read $f12/$f14/$f0. Derive a program-specific prototype model from the
|
||||
* program's own default o32 model with its floating-point entries and rules removed, install it
|
||||
* as a specification extension, and assign it to every callable whose formal types use floats.
|
||||
*/
|
||||
public final class ReaGhidraMipsSoftFloat extends GhidraScript {
|
||||
static final String REPORT_KEY = "REA MIPS soft-float storage";
|
||||
static final String POLICY = "o32-soft-float-prototype-v1";
|
||||
static final String PROTOTYPE = "__rea_soft_float";
|
||||
private static final Pattern FPU_REGISTER = Pattern.compile("f\\d+(_\\d+)?");
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String processor = currentProgram.getLanguage().getProcessor().toString();
|
||||
if (!processor.equals("MIPS") || currentProgram.getDefaultPointerSize() != 4)
|
||||
throw new IllegalStateException(
|
||||
"REA soft-float storage requires a 32-bit MIPS program, not " + processor);
|
||||
CompilerSpec spec = currentProgram.getCompilerSpec();
|
||||
PrototypeModel base = spec.getDefaultCallingConvention();
|
||||
new SpecExtension(currentProgram)
|
||||
.addReplaceCompilerSpecExtension(softFloatPrototype(base, spec), monitor);
|
||||
PrototypeModel soft = currentProgram.getCompilerSpec().getCallingConvention(PROTOTYPE);
|
||||
if (soft == null)
|
||||
throw new IllegalStateException("REA soft-float prototype was not installed");
|
||||
requireIntegerStorage(
|
||||
soft.getStorageLocations(
|
||||
currentProgram,
|
||||
new DataType[] {
|
||||
FloatDataType.dataType, IntegerDataType.dataType, DoubleDataType.dataType,
|
||||
FloatDataType.dataType, DoubleDataType.dataType
|
||||
},
|
||||
false),
|
||||
"soft-float prototype self-check");
|
||||
int functions = 0;
|
||||
List<Function> callables = new ArrayList<>();
|
||||
currentProgram.getFunctionManager().getFunctions(true).forEach(callables::add);
|
||||
currentProgram.getFunctionManager().getExternalFunctions().forEach(callables::add);
|
||||
for (Function function : callables) {
|
||||
// A thunk's signature is its target's; the target is in this list.
|
||||
if (function.isThunk() || !usesFloat(function)) continue;
|
||||
if (function.hasCustomVariableStorage()) function.setCustomVariableStorage(false);
|
||||
function.setCallingConvention(PROTOTYPE);
|
||||
List<VariableStorage> storage = new ArrayList<>();
|
||||
storage.add(function.getReturn().getVariableStorage());
|
||||
for (Parameter parameter : function.getParameters()) storage.add(parameter.getVariableStorage());
|
||||
requireIntegerStorage(storage.toArray(VariableStorage[]::new), function.getName(true));
|
||||
functions += 1;
|
||||
}
|
||||
int definitions = 0;
|
||||
Iterator<DataType> types = currentProgram.getDataTypeManager().getAllDataTypes();
|
||||
while (types.hasNext()) {
|
||||
if (!(types.next() instanceof FunctionDefinition definition) || !usesFloat(definition))
|
||||
continue;
|
||||
definition.setCallingConvention(PROTOTYPE);
|
||||
definitions += 1;
|
||||
}
|
||||
// Written last: any failure above leaves no report, and REA refuses the session.
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(
|
||||
REPORT_KEY,
|
||||
"{\"policy\":\"" + POLICY + "\",\"functions\":" + functions +
|
||||
",\"definitions\":" + definitions + "}");
|
||||
println("REA MIPS soft-float prototype: functions=" + functions + " definitions=" + definitions);
|
||||
}
|
||||
|
||||
/** The default o32 model without floating-point registers or float-consuming rules. */
|
||||
private static String softFloatPrototype(PrototypeModel base, CompilerSpec spec) throws Exception {
|
||||
XmlEncode encoder = new XmlEncode();
|
||||
base.encode(encoder, spec.getPcodeInjectLibrary());
|
||||
DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
|
||||
factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
|
||||
Document document =
|
||||
factory.newDocumentBuilder().parse(new InputSource(new StringReader(encoder.toString())));
|
||||
Element root = document.getDocumentElement();
|
||||
if (!root.getTagName().equals("prototype"))
|
||||
throw new IllegalStateException("Unexpected default prototype encoding " + root.getTagName());
|
||||
root.setAttribute("name", PROTOTYPE);
|
||||
List<Node> removed = new ArrayList<>();
|
||||
collect(root.getElementsByTagName("pentry"), "metatype", "float", null, removed);
|
||||
collect(root.getElementsByTagName("consume"), "storage", "float", "rule", removed);
|
||||
if (removed.isEmpty())
|
||||
throw new IllegalStateException("Default prototype has no floating-point entries to remove");
|
||||
for (Node node : removed) node.getParentNode().removeChild(node);
|
||||
StringWriter xml = new StringWriter();
|
||||
var transformer = TransformerFactory.newInstance().newTransformer();
|
||||
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
|
||||
transformer.transform(new DOMSource(document), new StreamResult(xml));
|
||||
return xml.toString();
|
||||
}
|
||||
|
||||
/** Select matching elements, or their nearest `ancestor` element when one is named. */
|
||||
private static void collect(
|
||||
NodeList nodes, String attribute, String value, String ancestor, List<Node> selected) {
|
||||
for (int index = 0; index < nodes.getLength(); index += 1) {
|
||||
Element element = (Element) nodes.item(index);
|
||||
if (!value.equals(element.getAttribute(attribute))) continue;
|
||||
Node target = element;
|
||||
while (ancestor != null && target != null && !ancestor.equals(target.getNodeName()))
|
||||
target = target.getParentNode();
|
||||
if (target != null && !selected.contains(target)) selected.add(target);
|
||||
}
|
||||
}
|
||||
|
||||
private static void requireIntegerStorage(VariableStorage[] storage, String owner) {
|
||||
for (VariableStorage item : storage) {
|
||||
if (item.isBadStorage() || item.isUnassignedStorage())
|
||||
throw new IllegalStateException("REA soft-float storage unresolved for " + owner);
|
||||
List<Register> registers = item.getRegisters();
|
||||
if (registers == null) continue; // stack-only storage
|
||||
for (Register register : registers) {
|
||||
if (FPU_REGISTER.matcher(register.getName()).matches())
|
||||
throw new IllegalStateException(
|
||||
"REA soft-float storage still uses " + register.getName() + " for " + owner);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private static boolean usesFloat(Function function) {
|
||||
if (isFloat(function.getReturn().getFormalDataType())) return true;
|
||||
for (Parameter parameter : function.getParameters()) {
|
||||
if (isFloat(parameter.getFormalDataType())) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static boolean usesFloat(FunctionDefinition definition) {
|
||||
if (isFloat(definition.getReturnType())) return true;
|
||||
for (ParameterDefinition argument : definition.getArguments()) {
|
||||
if (isFloat(argument.getDataType())) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static boolean isFloat(DataType type) {
|
||||
DataType current = type;
|
||||
while (current instanceof TypeDef typeDef) current = typeDef.getBaseDataType();
|
||||
return current instanceof AbstractFloatDataType;
|
||||
}
|
||||
}
|
||||
@@ -1,123 +0,0 @@
|
||||
import ghidra.app.cmd.function.CreateFunctionCmd;
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.listing.*;
|
||||
import ghidra.program.model.symbol.FlowType;
|
||||
import ghidra.program.model.symbol.Reference;
|
||||
import ghidra.program.model.pcode.PcodeOp;
|
||||
import java.util.ArrayDeque;
|
||||
import java.util.Deque;
|
||||
import java.util.HashSet;
|
||||
import java.util.Set;
|
||||
import java.util.TreeSet;
|
||||
|
||||
/**
|
||||
* Undo wrong no-return flags from auto-analysis. A function flagged no-return that contains a
|
||||
* real RET is an ordinary function (stream readers, cxa_guard_acquire, ...). Known returning
|
||||
* libc/TLS imports have no body to inspect, so clear their flags by exact external ABI name.
|
||||
* Callers treated the call as a terminator and the decompiler truncated them. Clear the flag, then re-disassemble only
|
||||
* the call sites of the functions that were cleared and refit their owning functions in place.
|
||||
*/
|
||||
public final class ReaGhidraNoReturnFix extends GhidraScript {
|
||||
private static final Set<String> RETURNING_IMPORTS = Set.of(
|
||||
"malloc", "__tls_get_addr", "memcpy", "memset", "strlen", "__errno_location");
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
// Windows P0 does not admit analysis-database mutation.
|
||||
if (System.getProperty("os.name", "").startsWith("Windows")) return;
|
||||
FunctionManager fm = currentProgram.getFunctionManager();
|
||||
Listing li = currentProgram.getListing();
|
||||
Set<Address> cleared = new HashSet<>();
|
||||
Deque<Address> pending = new ArrayDeque<>();
|
||||
for (Function f : fm.getExternalFunctions()) {
|
||||
monitor.checkCancelled();
|
||||
// Do not apply import-name assumptions to local functions or to arbitrary externals.
|
||||
if (!f.hasNoReturn() || !RETURNING_IMPORTS.contains(externalAbiName(f))) continue;
|
||||
f.setNoReturn(false);
|
||||
enqueueReturning(f, cleared, pending);
|
||||
}
|
||||
for (Function f : fm.getFunctions(true)) {
|
||||
monitor.checkCancelled();
|
||||
if (clearDecodedReturn(f, li)) {
|
||||
enqueueReturning(f, cleared, pending);
|
||||
}
|
||||
}
|
||||
int redone = 0;
|
||||
TreeSet<Address> owners = new TreeSet<>();
|
||||
while (!pending.isEmpty()) {
|
||||
Address target = pending.removeFirst();
|
||||
TreeSet<Address> sites = new TreeSet<>();
|
||||
// References include computed import calls after Ghidra resolves their destination.
|
||||
// Collect before redisassembly mutates the reference table.
|
||||
for (Reference reference : currentProgram.getReferenceManager().getReferencesTo(target)) {
|
||||
monitor.checkCancelled();
|
||||
sites.add(reference.getFromAddress());
|
||||
}
|
||||
for (Address s : sites) {
|
||||
monitor.checkCancelled();
|
||||
Instruction ins = li.getInstructionAt(s);
|
||||
if (ins == null) continue;
|
||||
FlowType ft = ins.getFlowType();
|
||||
if (!(ft.isCall() && ft.isTerminal())) continue;
|
||||
Address[] flows = ins.getFlows();
|
||||
boolean callsTarget = false;
|
||||
for (Address flow : flows) {
|
||||
if (flow.equals(target)) callsTarget = true;
|
||||
}
|
||||
if (!callsTarget) continue;
|
||||
// Do not redisassemble into an independently identified function.
|
||||
Address next = ins.getMaxAddress().add(1);
|
||||
if (fm.getFunctionAt(next) != null || !currentProgram.getMemory().contains(next)) continue;
|
||||
Function owner = fm.getFunctionContaining(s);
|
||||
if (owner != null && !owner.isThunk()) owners.add(owner.getEntryPoint());
|
||||
clearListing(ins.getMinAddress(), ins.getMaxAddress());
|
||||
disassemble(s);
|
||||
redone++;
|
||||
// Refit the body in place so the name, signature and calling convention survive.
|
||||
if (owner != null && !owner.isThunk()) {
|
||||
CreateFunctionCmd.fixupFunctionBody(currentProgram, owner, monitor);
|
||||
// Auto-analysis may have propagated the false flag to this truncated caller.
|
||||
// Its RET becomes visible only now; repair its callers in the same pass.
|
||||
if (clearDecodedReturn(owner, li)) enqueueReturning(owner, cleared, pending);
|
||||
}
|
||||
}
|
||||
}
|
||||
// No analyzeChanges here: on libpl2.so it adds about 70 s, which pushes startup past the 330 s deadline.
|
||||
println(
|
||||
"REA no-return fix: cleared=" + cleared.size() + " sites=" + redone + " refit=" + owners.size());
|
||||
}
|
||||
|
||||
private void enqueueReturning(Function function, Set<Address> cleared, Deque<Address> pending) {
|
||||
if (cleared.add(function.getEntryPoint())) pending.addLast(function.getEntryPoint());
|
||||
// A thunk inherits the flag of the function it forwards to, including through other thunks.
|
||||
Address[] thunks = function.getFunctionThunkAddresses(true);
|
||||
if (thunks == null) return;
|
||||
for (Address thunk : thunks) {
|
||||
if (cleared.add(thunk)) pending.addLast(thunk);
|
||||
}
|
||||
}
|
||||
|
||||
private String externalAbiName(Function function) {
|
||||
String name = function.getName();
|
||||
// Mach-O stores C-linkage symbols with one object-format underscore prefix.
|
||||
if (currentProgram.getExecutableFormat().contains("Mach-O") && name.startsWith("_"))
|
||||
return name.substring(1);
|
||||
return name;
|
||||
}
|
||||
|
||||
private boolean clearDecodedReturn(Function function, Listing listing) throws Exception {
|
||||
if (!function.hasNoReturn() || function.isThunk()) return false;
|
||||
for (Instruction instruction : listing.getInstructions(function.getBody(), true)) {
|
||||
monitor.checkCancelled();
|
||||
FlowType flow = instruction.getFlowType();
|
||||
if (flow.isJump() || flow.isCall()) continue;
|
||||
for (PcodeOp operation : instruction.getPcode()) {
|
||||
if (operation.getOpcode() != PcodeOp.RETURN) continue;
|
||||
function.setNoReturn(false);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import java.math.BigInteger;
|
||||
|
||||
/** Seed the declared DOS COM analysis context before default auto-analysis. */
|
||||
public final class ReaGhidraPrepareCom extends GhidraScript {
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
if (!currentProgram.getLanguageID().getIdAsString().equals("x86:LE:16:Real Mode") ||
|
||||
!currentProgram.getExecutableFormat().equals("Raw Binary")) {
|
||||
throw new IllegalArgumentException("DOS COM preparation requires raw x86 real-mode import");
|
||||
}
|
||||
Address entry = currentProgram.getAddressFactory().getDefaultAddressSpace().getAddress("1000:0100");
|
||||
Address start = currentProgram.getMinAddress();
|
||||
Address end = currentProgram.getMaxAddress();
|
||||
if (!entry.equals(start) || end.subtract(start) + 1 > 0xff00) {
|
||||
throw new IllegalArgumentException("DOS COM import must map 1..65280 bytes at 1000:0100");
|
||||
}
|
||||
for (String name : new String[] { "CS", "DS", "ES", "SS" }) {
|
||||
monitor.checkCancelled();
|
||||
currentProgram.getProgramContext().setValue(
|
||||
currentProgram.getRegister(name), start, end, BigInteger.valueOf(0x1000)
|
||||
);
|
||||
}
|
||||
addEntryPoint(entry);
|
||||
if (!disassemble(entry) || createFunction(entry, "entry") == null) {
|
||||
throw new IllegalStateException("DOS COM entry cannot be decoded as a function");
|
||||
}
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA DOS COM prepared", true);
|
||||
}
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
import ghidra.app.script.GhidraScript;
|
||||
import ghidra.program.model.address.Address;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.program.model.mem.MemoryBlock;
|
||||
import ghidra.program.model.mem.MemoryBlockSourceInfo;
|
||||
|
||||
/** Seed the caller-supplied raw-binary entry before default auto-analysis. */
|
||||
public final class ReaGhidraPrepareRaw extends GhidraScript {
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
if (arguments.length != 2) {
|
||||
throw new IllegalArgumentException(
|
||||
"Raw binary preparation requires the committed language and entry address"
|
||||
);
|
||||
}
|
||||
String expectedLanguage = arguments[0];
|
||||
String entryHex = arguments[1];
|
||||
if (!currentProgram.getLanguageID().getIdAsString().equals(expectedLanguage) ||
|
||||
!currentProgram.getExecutableFormat().equals("Raw Binary")) {
|
||||
throw new IllegalArgumentException(
|
||||
"Raw binary preparation requires the committed language and BinaryLoader import"
|
||||
);
|
||||
}
|
||||
Address entry = currentProgram.getAddressFactory().getDefaultAddressSpace().getAddress(entryHex);
|
||||
MemoryBlock block = entry == null ? null : currentProgram.getMemory().getBlock(entry);
|
||||
boolean fileStart = false;
|
||||
if (block != null) {
|
||||
// Processor-defined zero page/stack blocks can precede the imported file.
|
||||
for (MemoryBlockSourceInfo source : block.getSourceInfos()) {
|
||||
if (source.getFileBytes().isPresent() && source.getFileBytesOffset() == 0 &&
|
||||
entry.equals(source.getMinAddress())) {
|
||||
fileStart = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!fileStart) {
|
||||
throw new IllegalArgumentException(
|
||||
"Raw binary import must map file offset 0 at the caller-supplied entry"
|
||||
);
|
||||
}
|
||||
addEntryPoint(entry);
|
||||
if (!disassemble(entry) || createFunction(entry, "entry") == null) {
|
||||
throw new IllegalStateException("Raw binary entry cannot be decoded as a function");
|
||||
}
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setBoolean("REA raw binary prepared", true);
|
||||
}
|
||||
}
|
||||
+338
-850
File diff suppressed because it is too large.
Load diff
@@ -1,384 +0,0 @@
|
||||
"""REA offline adapter using mitmproxy's unchanged native tnetstring decoder.
|
||||
|
||||
No FlowReader migrations, HAR conversion, proxy listener, or target requests.
|
||||
The containing mitmdump process and its configuration directory are REA-owned.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import ipaddress
|
||||
import json
|
||||
import math
|
||||
import re
|
||||
import resource
|
||||
from pathlib import Path
|
||||
from urllib.parse import unquote_to_bytes
|
||||
|
||||
from mitmproxy import ctx, version
|
||||
from mitmproxy.io import tnetstring
|
||||
|
||||
PROFILE = "12.2.3"
|
||||
OUTPUT_BYTES = 96 * 1024 * 1024
|
||||
CREDENTIALS = {b"authorization", b"proxy-authorization", b"cookie", b"set-cookie"}
|
||||
|
||||
|
||||
class CaptureFailure(Exception):
|
||||
def __init__(self, reason, message, pointer=""):
|
||||
self.reason, self.pointer = reason, pointer
|
||||
super().__init__(message)
|
||||
|
||||
|
||||
def pointer(parent, key):
|
||||
return parent + "/" + str(key).replace("~", "~0").replace("/", "~1")
|
||||
|
||||
|
||||
class BoundedReader:
|
||||
"""Refuse malicious declared lengths before the upstream parser allocates."""
|
||||
def __init__(self, handle, size):
|
||||
self.handle, self.size = handle, size
|
||||
|
||||
def read(self, count):
|
||||
if count < 0 or count > self.size - self.handle.tell():
|
||||
raise CaptureFailure("format", "Native record declares bytes beyond the retained capture.")
|
||||
return self.handle.read(count)
|
||||
|
||||
|
||||
SPECIAL_SCHEMES = {"ftp", "file", "http", "https", "ws", "wss"}
|
||||
SCHEME_PREFIX = r"^[\x00-\x20]*([a-z][a-z0-9+.-]*):"
|
||||
NETWORK_PREFIX = r"^[\x00-\x20]*[/\\]{2}"
|
||||
URL_IGNORABLE_CONTROLS = "\t\n\r"
|
||||
URL_CONTROL_TRANSLATION = {ord(char): None for char in URL_IGNORABLE_CONTROLS}
|
||||
C0_CONTROL_OR_SPACE = "".join(chr(code) for code in range(0x21))
|
||||
FORBIDDEN_DOMAIN_CODE_POINTS = set(C0_CONTROL_OR_SPACE) | set("#%/:<>?@[\\]^|\x7f")
|
||||
|
||||
|
||||
def authority_start(text, base_scheme=None):
|
||||
"""Return where an authority begins and whether it is special, or None.
|
||||
|
||||
Follows WHATWG URL parsing as src/domain/recordedTransportUrl.ts does:
|
||||
special schemes (and scheme-relative references) skip every leading "/"
|
||||
or "\\" before the authority, so `https://\\user:pass@host` still has
|
||||
userinfo. File URLs never carry userinfo: an "@" in a file host makes
|
||||
WHATWG reject the whole URL, so there are no credentials to remove.
|
||||
"""
|
||||
scheme = re.match(SCHEME_PREFIX, text, flags=re.I)
|
||||
name = scheme.group(1).lower() if scheme is not None else None
|
||||
if name == "file" or (name is None and base_scheme == "file"):
|
||||
return None
|
||||
if name is not None and name in SPECIAL_SCHEMES:
|
||||
# A same-scheme Location without two slash-equivalent separators is
|
||||
# relative to the recorded request, so its @ belongs to the path.
|
||||
following = text[scheme.end():scheme.end() + 2]
|
||||
if name == base_scheme and (len(following) < 2 or any(char not in "/\\" for char in following)):
|
||||
return None
|
||||
start = scheme.end()
|
||||
else:
|
||||
if scheme is not None:
|
||||
if text[scheme.end():scheme.end() + 2] != "//":
|
||||
return None
|
||||
return scheme.end() + 2, False
|
||||
# A non-special base treats backslashes as authority data and only
|
||||
# recognizes two forward slashes as a network reference.
|
||||
if base_scheme is not None and base_scheme not in SPECIAL_SCHEMES:
|
||||
prefix = re.match(r"^[\x00-\x20]*//", text)
|
||||
return None if prefix is None else (prefix.end(), False)
|
||||
prefix = re.match(NETWORK_PREFIX, text)
|
||||
if prefix is None:
|
||||
return None
|
||||
start = prefix.end()
|
||||
while start < len(text) and text[start] in "/\\":
|
||||
start += 1
|
||||
return start, True
|
||||
|
||||
|
||||
IPV4_DIGITS = {8: "01234567", 10: "0123456789", 16: "0123456789abcdefABCDEF"}
|
||||
|
||||
|
||||
def ipv4_number(part):
|
||||
"""WHATWG IPv4 number parser: decimal, 0x hex or 0-prefixed octal, else None."""
|
||||
if part == "":
|
||||
return None
|
||||
radix, digits = 10, part
|
||||
if part[:2] in ("0x", "0X"):
|
||||
radix, digits = 16, part[2:]
|
||||
elif len(part) > 1 and part[0] == "0":
|
||||
radix, digits = 8, part[1:]
|
||||
if digits == "":
|
||||
return 0
|
||||
if any(char not in IPV4_DIGITS[radix] for char in digits):
|
||||
return None
|
||||
# Bound decimal text before int(): Python limits decimal conversions to
|
||||
# 4300 digits, and anything past 10 significant digits exceeds IPv4.
|
||||
significant = digits.lstrip("0")
|
||||
if radix == 10 and len(significant) > 10:
|
||||
return 2**32
|
||||
return int(digits, radix)
|
||||
|
||||
|
||||
def domain_parses(domain):
|
||||
"""Apply WHATWG's ends-in-a-number check and IPv4 parser to a domain."""
|
||||
parts = domain.split(".")
|
||||
if parts[-1] == "" and len(parts) > 1:
|
||||
parts.pop()
|
||||
last = parts[-1]
|
||||
if not ((last != "" and last.isascii() and last.isdigit()) or ipv4_number(last) is not None):
|
||||
return True
|
||||
numbers = [ipv4_number(part) for part in parts]
|
||||
if len(parts) > 4 or any(number is None for number in numbers):
|
||||
return False
|
||||
return all(number <= 255 for number in numbers[:-1]) and numbers[-1] < 256 ** (5 - len(numbers))
|
||||
|
||||
|
||||
def special_host_parses(host_port):
|
||||
"""Whether WHATWG could accept a special authority's host and port.
|
||||
|
||||
A rejected host fails the whole URL, so it has no credentials to remove.
|
||||
This mirrors the forbidden-code-point, empty-host, IPv4, IPv6 and port
|
||||
rules for ASCII hosts. UTS #46 mapping is not modeled: a non-ASCII host
|
||||
may map to an accepted one (fullwidth digits, ideographic full stops,
|
||||
ignored code points), so it is treated as accepted and still redacts.
|
||||
"""
|
||||
if host_port.startswith("["):
|
||||
host_end = host_port.find("]") + 1
|
||||
literal = host_port[1 : host_end - 1]
|
||||
if host_end == 0 or "%" in literal:
|
||||
return False
|
||||
try:
|
||||
ipaddress.IPv6Address(literal)
|
||||
except ValueError:
|
||||
return False
|
||||
else:
|
||||
colon = host_port.find(":")
|
||||
host_end = len(host_port) if colon < 0 else colon
|
||||
host = unquote_to_bytes(host_port[:host_end]).decode("utf-8", errors="replace")
|
||||
# Mapping never removes ASCII, so these rejections hold either way.
|
||||
if host == "" or any(char in FORBIDDEN_DOMAIN_CODE_POINTS for char in host):
|
||||
return False
|
||||
if host.isascii() and not domain_parses(host.lower()):
|
||||
return False
|
||||
port = host_port[host_end:]
|
||||
if port == "":
|
||||
return True
|
||||
digits = port[1:]
|
||||
if port[0] != ":" or not (digits == "" or (digits.isascii() and digits.isdigit())):
|
||||
return False
|
||||
# Bound the text first: int() refuses decimal strings over 4300 digits.
|
||||
significant = digits.lstrip("0")
|
||||
return len(significant) <= 5 and int(significant or "0") <= 65535
|
||||
|
||||
|
||||
def without_userinfo(value, base_scheme=None):
|
||||
"""Remove an authority's userinfo using the WHATWG authority boundary.
|
||||
|
||||
Special schemes and scheme-relative references also end the authority
|
||||
at a backslash, so later path text containing "@" is not userinfo.
|
||||
"""
|
||||
binary = isinstance(value, bytes)
|
||||
text = value.decode("latin-1") if binary else value
|
||||
# WHATWG removes these controls before interpreting scheme/separators.
|
||||
# Keep that parse view separate from the exact source spelling and bytes.
|
||||
# It also drops trailing C0 controls and spaces; offsets are unchanged.
|
||||
parsed_text = text.translate(URL_CONTROL_TRANSLATION).rstrip(C0_CONTROL_OR_SPACE)
|
||||
located = authority_start(parsed_text, base_scheme)
|
||||
if located is None:
|
||||
return value
|
||||
start, special = located
|
||||
terminators = "/?#\\" if special else "/?#"
|
||||
end = start
|
||||
while end < len(parsed_text) and parsed_text[end] not in terminators:
|
||||
end += 1
|
||||
at = parsed_text.rfind("@", start, end)
|
||||
if at < start or (special and not special_host_parses(parsed_text[at + 1 : end])):
|
||||
return value
|
||||
parsed_offset = 0
|
||||
raw_start = 0
|
||||
for raw_offset, char in enumerate(text):
|
||||
if char in URL_IGNORABLE_CONTROLS:
|
||||
continue
|
||||
if parsed_offset == start:
|
||||
raw_start = raw_offset
|
||||
if parsed_offset == at:
|
||||
safe = text[:raw_start] + text[raw_offset + 1 :]
|
||||
return safe.encode("latin-1") if binary else safe
|
||||
parsed_offset += 1
|
||||
raise CaptureFailure("decoder", "Normalized URL authority could not be bound to its source.")
|
||||
|
||||
|
||||
def project_record(state, secrets):
|
||||
binaries, numbers, redactions = [], [], []
|
||||
byte_secrets = []
|
||||
for secret in secrets:
|
||||
try:
|
||||
byte_secrets.append(secret.encode("utf-8", errors="strict"))
|
||||
except UnicodeEncodeError:
|
||||
# Non-scalar JSON strings have no UTF-8 byte representation.
|
||||
# Keep them in secrets for the ordinary string projection.
|
||||
pass
|
||||
|
||||
def redact(text, path, original):
|
||||
if any(secret in text or secret in original for secret in secrets):
|
||||
redactions.append({"pointer": path, "reason": "explicit-sensitive-value"})
|
||||
return None
|
||||
return text
|
||||
|
||||
def transport_value(value, path, transport_url, base_scheme):
|
||||
if not (transport_url or re.fullmatch(r"/(?:backup/)*request/(?:path|authority)", path)):
|
||||
return value, False
|
||||
binary = isinstance(value, bytes)
|
||||
separator = b"@" if binary else "@"
|
||||
if transport_url:
|
||||
safe_url = without_userinfo(value, base_scheme)
|
||||
elif path.endswith("/authority"):
|
||||
safe_url = value.rsplit(separator, 1)[-1]
|
||||
else:
|
||||
# Native request.path is origin-form unless it explicitly carries
|
||||
# a scheme (absolute-form). Leading separators are path evidence,
|
||||
# not a header URL reference's network authority.
|
||||
text = value.decode("latin-1") if binary else value
|
||||
safe_url = without_userinfo(value) if re.match(
|
||||
SCHEME_PREFIX, text.translate(URL_CONTROL_TRANSLATION), flags=re.I
|
||||
) else value
|
||||
changed = safe_url != value
|
||||
if changed:
|
||||
redactions.append({"pointer": path, "reason": "transport-credential"})
|
||||
return safe_url, changed
|
||||
|
||||
def validate_omitted(value, path, depth):
|
||||
before = len(binaries), len(numbers), len(redactions)
|
||||
visit(value, path, depth)
|
||||
del binaries[before[0]:]
|
||||
del numbers[before[1]:]
|
||||
del redactions[before[2]:]
|
||||
|
||||
def visit(value, path, depth=0, credential=False, transport_url=False, base_scheme=None):
|
||||
if depth > 64:
|
||||
raise CaptureFailure("input-limit", "Native capture exceeds the 64-level evidence nesting budget.", path)
|
||||
if credential:
|
||||
validate_omitted(value, path, depth)
|
||||
redactions.append({"pointer": path, "reason": "transport-credential"})
|
||||
if isinstance(value, bytes):
|
||||
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted", "content_base64": None, "bytes": None, "sha256": None})
|
||||
return None
|
||||
if isinstance(value, bytes):
|
||||
safe_bytes, credential_url = transport_value(value, path, transport_url, base_scheme)
|
||||
content_base64 = base64.b64encode(value).decode("ascii")
|
||||
digest = hashlib.sha256(value).hexdigest()
|
||||
hidden = any(secret in value or secret in safe_bytes for secret in byte_secrets) or any(secret in content_base64 or secret in digest or secret in str(len(value)) for secret in secrets)
|
||||
safe_text = None
|
||||
try:
|
||||
safe_text = safe_bytes.decode("utf-8", errors="strict")
|
||||
except UnicodeDecodeError:
|
||||
pass
|
||||
if hidden:
|
||||
redactions.append({"pointer": path, "reason": "explicit-sensitive-value"})
|
||||
exclude = hidden or credential_url
|
||||
binaries.append({"pointer": path, "representation": "producer-bytes", "state": "redacted" if exclude else "retained", "content_base64": None if exclude else content_base64, "bytes": None if exclude else len(value), "sha256": None if exclude else digest})
|
||||
if hidden:
|
||||
return None
|
||||
return safe_text
|
||||
if isinstance(value, str):
|
||||
safe_text, _ = transport_value(value, path, transport_url, base_scheme)
|
||||
return redact(safe_text, path, value)
|
||||
if isinstance(value, bool) or value is None:
|
||||
return value
|
||||
if isinstance(value, (int, float)):
|
||||
literal = repr(value)
|
||||
if any(secret in literal for secret in secrets):
|
||||
redactions.append({"pointer": path, "reason": "explicit-sensitive-value"})
|
||||
return None
|
||||
numbers.append({"pointer": path, "producer_type": "integer" if isinstance(value, int) else "float", "literal": literal})
|
||||
return value if (abs(value) <= 9007199254740991 if isinstance(value, int) else math.isfinite(value)) else None
|
||||
if isinstance(value, (list, tuple)):
|
||||
fields = re.fullmatch(r"/(?:backup/)*(?:request|response)/(?:headers|trailers)", path)
|
||||
if fields:
|
||||
result = []
|
||||
for index, field in enumerate(value):
|
||||
if not isinstance(field, (list, tuple)) or len(field) != 2 or not isinstance(field[0], bytes):
|
||||
raise CaptureFailure("format", "Native header/trailer is not an ordered byte pair.", pointer(path, index))
|
||||
result.append([visit(field[0], pointer(pointer(path, index), 0), depth + 2), visit(field[1], pointer(pointer(path, index), 1), depth + 2, field[0].lower() in CREDENTIALS, field[0].lower() in {b"location", b"referer", b"origin"}, base_scheme if field[0].lower() == b"location" else None)])
|
||||
return result
|
||||
return [visit(item, pointer(path, index), depth + 1, base_scheme=base_scheme) for index, item in enumerate(value)]
|
||||
if isinstance(value, dict):
|
||||
if path == "" or re.fullmatch(r"/backup(?:/backup)*", path):
|
||||
request = value.get("request")
|
||||
scheme = request.get("scheme") if isinstance(request, dict) else None
|
||||
if isinstance(scheme, bytes):
|
||||
scheme = scheme.decode("latin-1")
|
||||
base_scheme = scheme.lower() if isinstance(scheme, str) and re.fullmatch(r"[a-z][a-z0-9+.-]*", scheme, flags=re.I) else None
|
||||
result = {}
|
||||
for key, item in value.items():
|
||||
if not isinstance(key, str):
|
||||
raise CaptureFailure("format", "Native record has a non-string dictionary key.", path)
|
||||
if any(secret in key for secret in secrets):
|
||||
# Validate the omitted subtree too, then discard all of its
|
||||
# coordinates rather than inventing a replacement identity.
|
||||
validate_omitted(item, pointer(path, key), depth + 1)
|
||||
redactions.append({"pointer": path, "reason": "explicit-sensitive-value", "scope": "property-name"})
|
||||
continue
|
||||
result[key] = visit(item, pointer(path, key), depth + 1, base_scheme=base_scheme)
|
||||
return result
|
||||
raise CaptureFailure("decoder", "Native parser returned an unexpected value type: " + type(value).__name__ + ".", path)
|
||||
|
||||
return {"reported": visit(state, ""), "binary_fields": binaries, "numeric_literals": numbers, "redactions": redactions}
|
||||
|
||||
|
||||
def decode(request):
|
||||
if version.VERSION != PROFILE:
|
||||
raise CaptureFailure("unsupported", "Expected mitmproxy " + PROFILE + "; observed " + version.VERSION + ".")
|
||||
records = []
|
||||
capture = Path(request["snapshot_path"])
|
||||
size = capture.stat().st_size
|
||||
with capture.open("rb") as handle:
|
||||
reader = BoundedReader(handle, size)
|
||||
while handle.tell() < size:
|
||||
start = handle.tell()
|
||||
try:
|
||||
state = tnetstring.load(reader)
|
||||
except CaptureFailure:
|
||||
raise
|
||||
except RecursionError:
|
||||
raise CaptureFailure("input-limit", "Native parser exceeded its recursion budget at byte offset " + str(start) + ".") from None
|
||||
except (ValueError, IndexError, UnicodeError, EOFError):
|
||||
raise CaptureFailure("format", "Malformed native record at byte offset " + str(start) + ".") from None
|
||||
if not isinstance(state, dict):
|
||||
raise CaptureFailure("format", "Native flow record must be a dictionary at byte offset " + str(start) + ".")
|
||||
records.append({"ordinal": len(records), "location": {"kind": "byte-range", "offset": start, "bytes": handle.tell() - start}, **project_record(state, request["sensitive_values"]), "limitations": ["Fields are original native states without FlowReader migration. Binary fields contain exact producer bytes; reported UTF-8 strings for those fields are derived display views. Unknown flow/version extensions are retained without interpretation."]})
|
||||
return {"decoder": {"id": "mitmproxy-native-tnetstring", "name": "REA offline native mitmproxy adapter", "version": PROFILE}, "container": {"reported": None, "numeric_literals": [], "redactions": [], "records_pointer": None}, "total_records": len(records), "records": records}
|
||||
|
||||
|
||||
def failure_reply(error):
|
||||
"""Keep producer constraints, resource exhaustion and adapter defects distinct."""
|
||||
if isinstance(error, CaptureFailure):
|
||||
return {"ok": False, "reason": error.reason, "message": str(error), "pointer": error.pointer}
|
||||
if isinstance(error, MemoryError):
|
||||
return {"ok": False, "reason": "resource-limit", "message": "Native decoder exhausted its 768 MiB address-space budget.", "pointer": ""}
|
||||
return {"ok": False, "reason": "decoder", "message": "Offline native decoder failed unexpectedly: " + type(error).__name__ + ".", "pointer": ""}
|
||||
|
||||
|
||||
class OfflineCapture:
|
||||
def load(self, loader):
|
||||
loader.add_option("rea_request_path", str, "", "REA private offline capture request")
|
||||
|
||||
def running(self):
|
||||
resource.setrlimit(resource.RLIMIT_AS, (768 * 1024 * 1024, 768 * 1024 * 1024))
|
||||
resource.setrlimit(resource.RLIMIT_CPU, (30, 30))
|
||||
resource.setrlimit(resource.RLIMIT_FSIZE, (OUTPUT_BYTES, OUTPUT_BYTES))
|
||||
resource.setrlimit(resource.RLIMIT_CORE, (0, 0))
|
||||
request = json.loads(Path(ctx.options.rea_request_path).read_text(encoding="utf-8"))
|
||||
try:
|
||||
response = {"ok": True, "value": decode(request)}
|
||||
except Exception as error:
|
||||
response = failure_reply(error)
|
||||
try:
|
||||
try:
|
||||
output = json.dumps(response, ensure_ascii=True, allow_nan=False, separators=(",", ":")).encode("ascii")
|
||||
except Exception as error:
|
||||
output = json.dumps(failure_reply(error), separators=(",", ":")).encode("ascii")
|
||||
if len(output) > OUTPUT_BYTES:
|
||||
output = b'{"ok":false,"reason":"limit","message":"Native reply exceeds the complete-evidence output budget.","pointer":""}'
|
||||
Path(request["reply_path"]).write_bytes(output)
|
||||
finally:
|
||||
ctx.master.shutdown()
|
||||
|
||||
|
||||
addons = [OfflineCapture()]
|
||||
@@ -1,265 +0,0 @@
|
||||
"""Source-owned tests for the mitmproxy adapter's record projection."""
|
||||
|
||||
import importlib.util
|
||||
import os
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
|
||||
def load_capture():
|
||||
"""Load the adapter with fresh mitmproxy stand-ins scoped to its import."""
|
||||
fake_mitmproxy = types.ModuleType("mitmproxy")
|
||||
fake_mitmproxy.ctx = types.SimpleNamespace()
|
||||
fake_mitmproxy.version = types.SimpleNamespace(VERSION="12.2.3")
|
||||
fake_io = types.ModuleType("mitmproxy.io")
|
||||
fake_io.tnetstring = types.ModuleType("mitmproxy.io.tnetstring")
|
||||
stand_ins = {
|
||||
"mitmproxy": fake_mitmproxy,
|
||||
"mitmproxy.io": fake_io,
|
||||
"mitmproxy.io.tnetstring": fake_io.tnetstring,
|
||||
}
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"rea_mitmproxy_capture", os.path.join(os.path.dirname(__file__), "capture.py")
|
||||
)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
with mock.patch.dict(sys.modules, stand_ins):
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
capture = load_capture()
|
||||
|
||||
|
||||
def location(value, scheme=None):
|
||||
state = {"response": {"headers": [[b"Location", value]]}}
|
||||
if scheme is not None:
|
||||
state["request"] = {"scheme": scheme}
|
||||
record = capture.project_record(state, [])
|
||||
sidecar = next(
|
||||
field for field in record["binary_fields"] if field["pointer"] == "/response/headers/0/1"
|
||||
)
|
||||
return record["reported"]["response"]["headers"][0][1], sidecar, record["redactions"]
|
||||
|
||||
|
||||
class TransportUserinfoTests(unittest.TestCase):
|
||||
def test_ignorable_controls_cannot_hide_authority_credentials(self):
|
||||
for control in [b"\t", b"\n", b"\r"]:
|
||||
for value in [b"https://" + control + b"/user:pass@host/path", b"ht" + control + b"tps://user:" + control + b"pass@host/path"]:
|
||||
with self.subTest(value=value):
|
||||
text, sidecar, redactions = location(value, b"https")
|
||||
self.assertNotIn("user", text)
|
||||
self.assertNotIn("pass", text)
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
|
||||
def test_mixed_network_separators_remove_credentials(self):
|
||||
for prefix in [b"\\\\", b"/\\", b"\\/"]:
|
||||
with self.subTest(prefix=prefix):
|
||||
text, sidecar, redactions = location(prefix + b"user:pass@host/path", b"https")
|
||||
self.assertEqual(text, prefix.decode("ascii") + "host/path")
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
|
||||
def test_non_special_base_keeps_backslashes_inside_network_authorities(self):
|
||||
text, sidecar, redactions = location(b"//user\\name:pass@host/x", b"foo")
|
||||
self.assertEqual(text, "//host/x")
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
text, sidecar, redactions = location(b"/\\invoice@customer", b"foo")
|
||||
self.assertEqual(text, "/\\invoice@customer")
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_native_request_paths_preserve_origin_form_and_redact_absolute_form(self):
|
||||
for value in [b"/\\invoice@customer", b"//invoice@customer", b"\\\\invoice@customer"]:
|
||||
with self.subTest(value=value):
|
||||
record = capture.project_record({"request": {"path": value}}, [])
|
||||
self.assertEqual(record["reported"]["request"]["path"], value.decode("ascii"))
|
||||
self.assertEqual(record["binary_fields"][0]["state"], "retained")
|
||||
self.assertEqual(record["redactions"], [])
|
||||
record = capture.project_record({"request": {"path": b"https://user:pass@host/x"}}, [])
|
||||
self.assertEqual(record["reported"]["request"]["path"], "https://host/x")
|
||||
self.assertEqual(record["binary_fields"][0]["state"], "redacted")
|
||||
|
||||
def test_same_scheme_locations_preserve_relative_path_text_and_bytes(self):
|
||||
for value in [b"https:invoice@customer", b"https:/invoice@customer", b"https:\\invoice@customer"]:
|
||||
with self.subTest(value=value):
|
||||
text, sidecar, redactions = location(value, b"https")
|
||||
self.assertEqual(text, value.decode("ascii"))
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_location_authorities_still_remove_credentials_with_a_base_scheme(self):
|
||||
for value, scheme, expected in [
|
||||
(b"https://user:pass@host/path", b"https", "https://host/path"),
|
||||
(b"https:\\\\user:pass@host/path", b"https", "https:\\\\host/path"),
|
||||
(b"https:user:pass@host/path", b"http", "https:host/path"),
|
||||
]:
|
||||
with self.subTest(value=value, scheme=scheme):
|
||||
text, sidecar, redactions = location(value, scheme)
|
||||
self.assertEqual(text, expected)
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
|
||||
def test_backup_location_uses_its_own_request_scheme(self):
|
||||
record = capture.project_record({
|
||||
"request": {"scheme": b"http"},
|
||||
"backup": {
|
||||
"request": {"scheme": b"https"},
|
||||
"response": {"headers": [[b"Location", b"https:invoice@customer"]]},
|
||||
},
|
||||
}, [])
|
||||
self.assertEqual(record["reported"]["backup"]["response"]["headers"][0][1], "https:invoice@customer")
|
||||
self.assertEqual(record["redactions"], [])
|
||||
|
||||
def test_backslash_ends_a_special_scheme_authority(self):
|
||||
text, sidecar, redactions = location(b"https://example.test\\part@tail")
|
||||
self.assertEqual(text, "https://example.test\\part@tail")
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertIsNotNone(sidecar["content_base64"])
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_userinfo_before_a_backslash_is_still_removed(self):
|
||||
text, sidecar, redactions = location(b"https://user:pw@example.test\\path")
|
||||
self.assertEqual(text, "https://example.test\\path")
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(
|
||||
redactions,
|
||||
[{"pointer": "/response/headers/0/1", "reason": "transport-credential"}],
|
||||
)
|
||||
|
||||
def test_ordinary_userinfo_is_removed(self):
|
||||
text, _sidecar, _redactions = location(b"https://user:pw@example.test/a@b")
|
||||
self.assertEqual(text, "https://example.test/a@b")
|
||||
|
||||
def test_empty_userinfo_marker_is_removed(self):
|
||||
self.assertEqual(
|
||||
capture.without_userinfo("https://@host"), "https://host"
|
||||
)
|
||||
|
||||
def test_special_schemes_skip_leading_slashes_before_userinfo(self):
|
||||
for value, expected in [
|
||||
(b"https://\\user:pass@host/path", b"https://\\host/path"),
|
||||
(b"https:\\\\user:pass@host/path", b"https:\\\\host/path"),
|
||||
(b"https:user:pass@host", b"https:host"),
|
||||
(b"//\\user:pass@host", b"//\\host"),
|
||||
]:
|
||||
with self.subTest(value=value):
|
||||
self.assertEqual(capture.without_userinfo(value), expected)
|
||||
text, sidecar, redactions = location(b"https://\\user:pass@host/path")
|
||||
self.assertEqual(text, "https://\\host/path")
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
|
||||
def test_file_base_keeps_paths_after_the_fixed_network_boundary(self):
|
||||
for value in [b"///u@h", b"////u@h"]:
|
||||
with self.subTest(value=value):
|
||||
text, sidecar, redactions = location(value, b"file")
|
||||
self.assertEqual(text, value.decode("ascii"))
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertIsNotNone(sidecar["content_base64"])
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_file_urls_keep_path_text(self):
|
||||
self.assertEqual(capture.without_userinfo("file:///a@b"), "file:///a@b")
|
||||
|
||||
def test_file_urls_reject_userinfo_and_keep_their_source(self):
|
||||
# WHATWG rejects an "@" in a file host, so nothing parses as credentials.
|
||||
self.assertEqual(
|
||||
capture.without_userinfo("file://user:pass@host/x"), "file://user:pass@host/x"
|
||||
)
|
||||
for value in [b"\\\\user:pass@host/x", b"/\\user:pass@host/x", b"//user:pass@host/x"]:
|
||||
with self.subTest(value=value):
|
||||
text, sidecar, redactions = location(value, b"file")
|
||||
self.assertEqual(text, value.decode("ascii"))
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_rejected_special_hosts_keep_their_source(self):
|
||||
# A rejected host or port fails the whole URL, so it has no userinfo.
|
||||
for value in [
|
||||
"https:user@%",
|
||||
"https://user@",
|
||||
"https://user:pass@h^st/x",
|
||||
"https://user@ho st/x",
|
||||
"https://user@host:99999/x",
|
||||
"https://user@host:12a",
|
||||
"https://user@[::1",
|
||||
]:
|
||||
with self.subTest(value=value):
|
||||
self.assertEqual(capture.without_userinfo(value), value)
|
||||
text, sidecar, redactions = location(b"https://user:pass@h^st/x", b"https")
|
||||
self.assertEqual(text, "https://user:pass@h^st/x")
|
||||
self.assertEqual(sidecar["state"], "retained")
|
||||
self.assertEqual(redactions, [])
|
||||
|
||||
def test_accepted_special_hosts_still_remove_userinfo(self):
|
||||
for value, expected in [
|
||||
("https:user@%41", "https:%41"),
|
||||
("https://user@[::1]:8443/x", "https://[::1]:8443/x"),
|
||||
("https://user@host:/x", "https://host:/x"),
|
||||
("wss://user:pass@host:65535", "wss://host:65535"),
|
||||
("https://user@0x7f.1/", "https://0x7f.1/"),
|
||||
("https://user@1.2.3.4./", "https://1.2.3.4./"),
|
||||
("https://user@4294967295/", "https://4294967295/"),
|
||||
("https://user@1.2.3.08a/", "https://1.2.3.08a/"),
|
||||
("https://user@[::ffff:1.2.3.4]/", "https://[::ffff:1.2.3.4]/"),
|
||||
]:
|
||||
with self.subTest(value=value):
|
||||
self.assertEqual(capture.without_userinfo(value), expected)
|
||||
|
||||
def test_unicode_hosts_that_may_map_to_accepted_hosts_remove_userinfo(self):
|
||||
# UTS #46 maps these to 1.2.3.4; mapping is not modeled, so redact.
|
||||
for host in ["%EF%BC%91.2.3.4", "1。2.3.4", "%C2%AD1.2.3.4", "１.2.3.4"]:
|
||||
with self.subTest(host=host):
|
||||
self.assertEqual(
|
||||
capture.without_userinfo(f"https://user:pw@{host}/"), f"https://{host}/"
|
||||
)
|
||||
text, sidecar, redactions = location(b"https://user:pw@%EF%BC%91.2.3.4/", b"https")
|
||||
self.assertEqual(text, "https://%EF%BC%91.2.3.4/")
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
# ASCII rejections still hold for non-ASCII hosts.
|
||||
self.assertEqual(capture.without_userinfo("https://user@１^/"), "https://user@１^/")
|
||||
|
||||
def test_trailing_controls_and_spaces_do_not_hide_userinfo(self):
|
||||
# WHATWG strips trailing C0 controls and spaces before parsing.
|
||||
for suffix in [b" ", b"\x01", b"\x00 \x1f"]:
|
||||
with self.subTest(suffix=suffix):
|
||||
value = b"https://user:pw@example.test" + suffix
|
||||
text, sidecar, redactions = location(value, b"https")
|
||||
self.assertEqual(text, "https://example.test" + suffix.decode("latin-1"))
|
||||
self.assertEqual(sidecar["state"], "redacted")
|
||||
self.assertEqual(len(redactions), 1)
|
||||
|
||||
def test_rejected_ip_literals_keep_their_source(self):
|
||||
for value in [
|
||||
"https://user:pass@256.256.256.256/path",
|
||||
"https://user@1.2.3.4.5/",
|
||||
"https://user@1.2.3.09/",
|
||||
"https://user@4294967296/",
|
||||
"https://user@a.0x/",
|
||||
"https://user@1..2/",
|
||||
"https://user@1." + "9" * 5000 + "/",
|
||||
"https://user@[1::2::3]/",
|
||||
"https://user@[::1%25eth0]/",
|
||||
]:
|
||||
with self.subTest(value=value[:40]):
|
||||
self.assertEqual(capture.without_userinfo(value), value)
|
||||
|
||||
def test_overlong_ports_are_compared_without_numeric_overflow(self):
|
||||
# Python 3.11+ refuses int() on decimal strings over 4300 digits.
|
||||
accepted = "https://user@host:" + "0" * 5000 + "80/x"
|
||||
self.assertEqual(capture.without_userinfo(accepted), "https://host:" + "0" * 5000 + "80/x")
|
||||
rejected = "https://user@host:" + "9" * 5000 + "/x"
|
||||
self.assertEqual(capture.without_userinfo(rejected), rejected)
|
||||
|
||||
def test_non_special_schemes_keep_backslashes_in_the_authority(self):
|
||||
self.assertEqual(capture.without_userinfo("foo://a\\b@host/x"), "foo://host/x")
|
||||
self.assertEqual(capture.without_userinfo(b"//a\\b@host"), b"//a\\b@host")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,27 +0,0 @@
|
||||
import Foundation
|
||||
|
||||
struct ChildBatch<Element> {
|
||||
let values: [Element]
|
||||
let complete: Bool
|
||||
}
|
||||
|
||||
struct ChildCount {
|
||||
let value: Int?
|
||||
let error: Int32?
|
||||
}
|
||||
|
||||
func serializeHelperOutput(_ value: Any) throws -> Data {
|
||||
try JSONSerialization.data(withJSONObject: value, options: [.sortedKeys, .withoutEscapingSlashes])
|
||||
}
|
||||
|
||||
func captureChildCount(status: Int32, success: Int32, value: Int) -> ChildCount {
|
||||
guard status == success else { return ChildCount(value: nil, error: status) }
|
||||
return ChildCount(value: value, error: nil)
|
||||
}
|
||||
|
||||
func captureChildValues<Element>(requestedCount: Int, copy: () -> [Element]?) -> ChildBatch<Element> {
|
||||
guard requestedCount > 0 else { return ChildBatch(values: [], complete: true) }
|
||||
let returned = copy() ?? []
|
||||
let values = Array(returned.prefix(requestedCount))
|
||||
return ChildBatch(values: values, complete: values.count == requestedCount)
|
||||
}
|
||||
@@ -1,171 +0,0 @@
|
||||
import AppKit
|
||||
import ApplicationServices
|
||||
import CryptoKit
|
||||
import ScreenCaptureKit
|
||||
import Darwin
|
||||
|
||||
struct Request: Decodable {
|
||||
var pid: Int32
|
||||
var window_id: UInt32
|
||||
var executable: String
|
||||
var sha256: String
|
||||
var launch_time: Double?
|
||||
var screenshot: Bool
|
||||
var accessibility: Bool
|
||||
var max_nodes: Int
|
||||
var action: Action?
|
||||
}
|
||||
struct Action: Decodable { var kind: String; var path: [Int]?; var direction: String?; var text: String? }
|
||||
struct BoundaryFailure: Error { var code: String; var message: String }
|
||||
func fail(_ code: String, _ message: String) throws -> Never { throw BoundaryFailure(code: code, message: message) }
|
||||
func attribute(_ element: AXUIElement, _ key: String) -> CFTypeRef? {
|
||||
var value: CFTypeRef?
|
||||
return AXUIElementCopyAttributeValue(element, key as CFString, &value) == .success ? value : nil
|
||||
}
|
||||
func childCount(_ element: AXUIElement) -> ChildCount {
|
||||
var count = 0
|
||||
let status = AXUIElementGetAttributeValueCount(element, kAXChildrenAttribute as CFString, &count)
|
||||
return captureChildCount(status: status.rawValue, success: AXError.success.rawValue, value: count)
|
||||
}
|
||||
func children(_ element: AXUIElement, count: Int) -> ChildBatch<AXUIElement> {
|
||||
captureChildValues(requestedCount: count) {
|
||||
var value: CFArray?
|
||||
guard AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, 0, count, &value) == .success else { return nil }
|
||||
return value as? [AXUIElement]
|
||||
}
|
||||
}
|
||||
func text(_ element: AXUIElement, _ key: String) -> Any { (attribute(element, key) as? String) ?? NSNull() as Any }
|
||||
func windowBounds(_ element: AXUIElement) -> CGRect? {
|
||||
guard let p = attribute(element, kAXPositionAttribute), let s = attribute(element, kAXSizeAttribute),
|
||||
CFGetTypeID(p) == AXValueGetTypeID(), CFGetTypeID(s) == AXValueGetTypeID() else { return nil }
|
||||
var point = CGPoint.zero; var size = CGSize.zero
|
||||
guard AXValueGetValue(p as! AXValue, .cgPoint, &point), AXValueGetValue(s as! AXValue, .cgSize, &size) else { return nil }
|
||||
return CGRect(origin: point, size: size)
|
||||
}
|
||||
func observe(_ request: Request) async throws -> [String: Any] {
|
||||
guard let app = NSRunningApplication(processIdentifier: request.pid), let url = app.executableURL else {
|
||||
try fail("target-mismatch", "Selected PID does not expose an application executable URL")
|
||||
}
|
||||
guard let canonicalPointer = realpath(url.path, nil) else { try fail("target-mismatch", "Running executable path cannot be resolved") }
|
||||
defer { free(canonicalPointer) }
|
||||
let canonicalExecutable = String(cString: canonicalPointer)
|
||||
guard canonicalExecutable == request.executable else {
|
||||
try fail("target-mismatch", "Selected PID executable \(canonicalExecutable) differs from active target \(request.executable)")
|
||||
}
|
||||
guard let launch = app.launchDate?.timeIntervalSince1970 else {
|
||||
try fail("process-identity-unavailable", "Selected application does not expose a launch time; stable PID reuse checks are unavailable")
|
||||
}
|
||||
if let expected = request.launch_time, expected != launch { try fail("process-replaced", "Selected process was replaced") }
|
||||
let executableSize = (try FileManager.default.attributesOfItem(atPath: url.path)[.size] as? NSNumber)?.int64Value ?? -1
|
||||
guard executableSize >= 0 && executableSize <= 256 * 1024 * 1024 else { try fail("target-limit", "Running executable exceeds the 256 MiB identity-check budget") }
|
||||
let executableBytes = try Data(contentsOf: url, options: .mappedIfSafe)
|
||||
guard SHA256.hash(data: executableBytes).map({ String(format: "%02x", $0) }).joined() == request.sha256 else {
|
||||
try fail("target-mismatch", "Running executable bytes differ from the active target digest")
|
||||
}
|
||||
guard let windows = CGWindowListCopyWindowInfo([.optionIncludingWindow], request.window_id) as? [[String: Any]],
|
||||
windows.count == 1, let window = windows.first,
|
||||
(window[kCGWindowOwnerPID as String] as? NSNumber)?.int32Value == request.pid,
|
||||
(window[kCGWindowLayer as String] as? NSNumber)?.intValue == 0,
|
||||
let boundsDictionary = window[kCGWindowBounds as String] as? [String: Any],
|
||||
let bounds = CGRect(dictionaryRepresentation: boundsDictionary as CFDictionary) else {
|
||||
try fail("window-mismatch", "Selected window is absent or belongs to another process")
|
||||
}
|
||||
if request.screenshot && !CGPreflightScreenCaptureAccess() {
|
||||
try fail("screen-recording-denied", "Grant Screen Recording to the REA host in System Settings > Privacy & Security, then retry; no broad capture fallback is used")
|
||||
}
|
||||
var selected: AXUIElement?
|
||||
if request.accessibility || request.action != nil {
|
||||
guard AXIsProcessTrusted() else { try fail("accessibility-denied", "Grant Accessibility to the REA host in System Settings > Privacy & Security, then retry") }
|
||||
let application = AXUIElementCreateApplication(request.pid)
|
||||
AXUIElementSetMessagingTimeout(application, 2)
|
||||
guard let axWindows = attribute(application, kAXWindowsAttribute) as? [AXUIElement] else {
|
||||
try fail("accessibility-unavailable", "Selected application does not expose accessibility windows")
|
||||
}
|
||||
let matches = axWindows.filter { windowBounds($0) == bounds }
|
||||
guard matches.count == 1 else { try fail("ambiguous-window", "Accessibility window geometry does not identify exactly one selected window; selected CG bounds \(bounds), AX candidates \(axWindows.map { String(describing: windowBounds($0)) })") }
|
||||
selected = matches[0]
|
||||
}
|
||||
if let action = request.action, let root = selected {
|
||||
var element = root
|
||||
for index in action.path ?? [] {
|
||||
var selectedChild: CFArray?
|
||||
let count = childCount(element)
|
||||
guard let available = count.value else {
|
||||
try fail("element-count-unavailable", "Cannot validate selected accessibility path because child count failed with AXError \(count.error ?? -1)")
|
||||
}
|
||||
guard index >= 0 && index < available,
|
||||
AXUIElementCopyAttributeValues(element, kAXChildrenAttribute as CFString, index, 1, &selectedChild) == .success,
|
||||
let item = (selectedChild as? [AXUIElement])?.first else { try fail("element-missing", "Selected accessibility path no longer exists") }
|
||||
element = item
|
||||
}
|
||||
var owner: pid_t = 0
|
||||
guard AXUIElementGetPid(element, &owner) == .success && owner == request.pid else { try fail("element-owner-mismatch", "Selected element belongs to another process") }
|
||||
let outcome: AXError
|
||||
switch action.kind {
|
||||
case "click": outcome = AXUIElementPerformAction(element, kAXPressAction as CFString)
|
||||
case "scroll": outcome = AXUIElementPerformAction(element, (action.direction == "increment" ? kAXIncrementAction : kAXDecrementAction) as CFString)
|
||||
case "key-entry": outcome = AXUIElementSetAttributeValue(element, kAXValueAttribute as CFString, (action.text ?? "") as CFString)
|
||||
default: try fail("unsupported-action", "Only selected-element press, increment/decrement and text-value entry are admitted")
|
||||
}
|
||||
guard outcome == .success else { try fail("action-failed", "Accessibility action failed with AXError \(outcome.rawValue)") }
|
||||
}
|
||||
var nodes: [[String: Any]] = []; var truncated = false; var gaps: [String] = []
|
||||
if request.accessibility, let root = selected {
|
||||
var pending: [(AXUIElement, [Int])] = [(root, [])]
|
||||
while let (element, path) = pending.popLast() {
|
||||
if nodes.count >= request.max_nodes { truncated = true; break }
|
||||
let childCountResult = childCount(element)
|
||||
var actions: CFArray?
|
||||
AXUIElementCopyActionNames(element, &actions)
|
||||
nodes.append(["path": path, "role": text(element, kAXRoleAttribute), "title": text(element, kAXTitleAttribute), "value": text(element, kAXValueAttribute), "actions": actions as? [String] ?? [], "children_count": childCountResult.value.map { $0 as Any } ?? NSNull()])
|
||||
guard let totalChildren = childCountResult.value else {
|
||||
truncated = true
|
||||
gaps.append("AX child count unavailable at path \(path): AXError \(childCountResult.error ?? -1)")
|
||||
continue
|
||||
}
|
||||
if path.count >= 32 { if totalChildren > 0 { truncated = true }; continue }
|
||||
let available = max(0, request.max_nodes - nodes.count - pending.count)
|
||||
let count = min(available, totalChildren)
|
||||
let batch = children(element, count: count)
|
||||
if !batch.complete { truncated = true }
|
||||
if totalChildren > count { truncated = true }
|
||||
for (index, item) in batch.values.enumerated().reversed() {
|
||||
pending.append((item, path + [index]))
|
||||
}
|
||||
}
|
||||
}
|
||||
var screenshot: Any = NSNull()
|
||||
if request.screenshot {
|
||||
guard #available(macOS 14, *) else { try fail("unsupported-host", "Selected-window screenshots require macOS 14 or later") }
|
||||
let content = try await SCShareableContent.excludingDesktopWindows(true, onScreenWindowsOnly: false)
|
||||
guard let scWindow = content.windows.first(where: { $0.windowID == request.window_id && $0.owningApplication?.processID == request.pid }) else {
|
||||
try fail("capture-window-missing", "Selected window disappeared before capture")
|
||||
}
|
||||
guard bounds.width > 0 && bounds.height > 0 && bounds.width <= 100000 && bounds.height <= 100000 else { try fail("window-bounds", "Selected window has invalid or oversized geometry") }
|
||||
let filter = SCContentFilter(desktopIndependentWindow: scWindow)
|
||||
let configuration = SCStreamConfiguration()
|
||||
let scale = min(1, 2048 / max(bounds.width, bounds.height))
|
||||
configuration.width = max(1, Int(bounds.width * scale)); configuration.height = max(1, Int(bounds.height * scale))
|
||||
configuration.showsCursor = false
|
||||
let image = try await SCScreenshotManager.captureImage(contentFilter: filter, configuration: configuration)
|
||||
guard let png = NSBitmapImageRep(cgImage: image).representation(using: .png, properties: [:]) else {
|
||||
try fail("capture-failed", "Selected-window PNG encoding failed")
|
||||
}
|
||||
screenshot = ["mime_type": "image/png", "base64": png.base64EncodedString(), "sha256": SHA256.hash(data: png).map({ String(format: "%02x", $0) }).joined(), "width": image.width, "height": image.height]
|
||||
}
|
||||
return ["window": ["pid": request.pid, "window_id": request.window_id, "executable": request.executable, "launch_time": launch, "title": window[kCGWindowName as String] as? String ?? ""], "nodes": nodes, "truncated": truncated, "screenshot": screenshot, "gaps": request.accessibility ? gaps : ["Accessibility capture was disabled"]]
|
||||
}
|
||||
Task { @MainActor in
|
||||
do {
|
||||
let request = try JSONDecoder().decode(Request.self, from: Data(CommandLine.arguments[1].utf8))
|
||||
let result = try await observe(request)
|
||||
let output = try serializeHelperOutput(["ok": true, "result": result])
|
||||
FileHandle.standardOutput.write(output)
|
||||
} catch {
|
||||
let failure = error as? BoundaryFailure
|
||||
let output: [String: Any] = ["ok": false, "code": failure?.code ?? "capture-failed", "message": failure?.message ?? error.localizedDescription]
|
||||
if let data = try? serializeHelperOutput(output) { FileHandle.standardOutput.write(data) }
|
||||
}
|
||||
exit(0)
|
||||
}
|
||||
RunLoop.main.run()
|
||||
Loaded 100 of 5724 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user