Compare commits
177
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
38998e465f | ||
|
|
e52a71602d | ||
|
|
8443692e2b | ||
|
|
7b2087e449 | ||
|
|
f5d1012fab | ||
|
|
d2a3269d0c | ||
|
|
fd7da77fc0 | ||
|
|
44521d3d75 | ||
|
|
eac296a7da | ||
|
|
55cce6fc11 | ||
|
|
57e76599c0 | ||
|
|
498a667234 | ||
|
|
1195929fa5 | ||
|
|
c4007f4542 | ||
|
|
a74d859a19 | ||
|
|
2b2e362a00 | ||
|
|
497037e6e3 | ||
|
|
fa36657251 | ||
|
|
d43d3568f2 | ||
|
|
25d3805e7f | ||
|
|
82902465c4 | ||
|
|
cf5194f68e | ||
|
|
52d1a72af0 | ||
|
|
e50a271c72 | ||
|
|
c374c2fe72 | ||
|
|
0c440bfc07 | ||
|
|
a4dbb36d8e | ||
|
|
0e83d92c58 | ||
|
|
ee12d517a2 | ||
|
|
9cf7a7cb0d | ||
|
|
00a055be0f | ||
|
|
22278b20c0 | ||
|
|
3cf6206614 | ||
|
|
4714c7eaa3 | ||
|
|
cec8f1e8f3 | ||
|
|
21e0c580e7 | ||
|
|
f01dccab77 | ||
|
|
5f0bc86d1a | ||
|
|
f69332c94c | ||
|
|
2e03251b61 | ||
|
|
108c0e7e49 | ||
|
|
266dd1d958 | ||
|
|
cac12ae334 | ||
|
|
861c11ec77 | ||
|
|
9051809343 | ||
|
|
9db5f1b276 | ||
|
|
3a9176098d | ||
|
|
483678bf9f | ||
|
|
052956af34 | ||
|
|
aa314ab682 | ||
|
|
c23e161629 | ||
|
|
1417bf943b | ||
|
|
e84468c0bb | ||
|
|
afde435c94 | ||
|
|
e9fc8fb30c | ||
|
|
c284168c4b | ||
|
|
be20ac3740 | ||
|
|
cc73587ec4 | ||
|
|
98edca9f96 | ||
|
|
98a5e56380 | ||
|
|
7c6142efdc | ||
|
|
562dab00c5 | ||
|
|
3b02179560 | ||
|
|
599fdac029 | ||
|
|
696832caa3 | ||
|
|
459a356edb | ||
|
|
419502c8d0 | ||
|
|
1006714271 | ||
|
|
e367750b1b | ||
|
|
cf3288d8cf | ||
|
|
d532b18123 | ||
|
|
074768901f | ||
|
|
2538214e2c | ||
|
|
b5d540f3a2 | ||
|
|
4eaf3a117b | ||
|
|
0134732fcd | ||
|
|
799e8265b2 | ||
|
|
4668cea246 | ||
|
|
58d6043303 | ||
|
|
d148ce0905 | ||
|
|
3fca2e0ca6 | ||
|
|
4e716b0607 | ||
|
|
d72021b0da | ||
|
|
f93a0335a8 | ||
|
|
5fa4defcbc | ||
|
|
833136507d | ||
|
|
82c76e2cba | ||
|
|
bf1eeee94f | ||
|
|
7a0c47bfa4 | ||
|
|
b4938aed58 | ||
|
|
71ae02c6ef | ||
|
|
10f1994fe1 | ||
|
|
909b9f7b6b | ||
|
|
e84e17e30f | ||
|
|
1c2c2f1e78 | ||
|
|
1b3ac94242 | ||
|
|
b44cb5bd30 | ||
|
|
2dd5effe2a | ||
|
|
0e62094b69 | ||
|
|
5199c672ff | ||
|
|
c4aaf813e0 | ||
|
|
aacf740a04 | ||
|
|
0949313d8f | ||
|
|
4af71a45f9 | ||
|
|
60bd646478 | ||
|
|
255d906e15 | ||
|
|
49b4edfe77 | ||
|
|
3ea0d9489e | ||
|
|
dc936f433c | ||
|
|
49e67aa1d9 | ||
|
|
0fc8462be1 | ||
|
|
b999e72476 | ||
|
|
a0edd145d3 | ||
|
|
64abd4aefb | ||
|
|
69aa5465cd | ||
|
|
2ec602b191 | ||
|
|
44b6ec97ee | ||
|
|
69fe86c5c7 | ||
|
|
00dd959b82 | ||
|
|
435efe7afc | ||
|
|
4cbcead46c | ||
|
|
91884c3743 | ||
|
|
048d70755e | ||
|
|
39275420ea | ||
|
|
ca71e91d23 | ||
|
|
ba866a26ec | ||
|
|
e8da7e1edd | ||
|
|
2e5171e391 | ||
|
|
87ab054ea3 | ||
|
|
7f4e7d5577 | ||
|
|
9aca543f46 | ||
|
|
715d93e954 | ||
|
|
e4c3d72396 | ||
|
|
9af0b6113c | ||
|
|
974487f00f | ||
|
|
dfe8378cf5 | ||
|
|
d6ca7b3f20 | ||
|
|
d3627b1038 | ||
|
|
273bb66276 | ||
|
|
6296fdf560 | ||
|
|
c85b44e1b1 | ||
|
|
95c8f57aa2 | ||
|
|
92dadf9334 | ||
|
|
559af3d3ff | ||
|
|
a80575c736 | ||
|
|
fdaffe21cc | ||
|
|
291059adb4 | ||
|
|
65884350ff | ||
|
|
22ddd82014 | ||
|
|
d1adc60795 | ||
|
|
32f0e51209 | ||
|
|
29910d16b9 | ||
|
|
50b6975d86 | ||
|
|
1d5bf5ff08 | ||
|
|
3537e58c1a | ||
|
|
cee95b2ae6 | ||
|
|
eb2c3b589e | ||
|
|
d4e233946a | ||
|
|
292e7bef27 | ||
|
|
36d156ff05 | ||
|
|
7201e8a443 | ||
|
|
9e259f87c8 | ||
|
|
8307ea3717 | ||
|
|
fd728f271e | ||
|
|
b0b485abff | ||
|
|
23f984b1a8 | ||
|
|
093ce695f0 | ||
|
|
63c970812b | ||
|
|
c12d9eda87 | ||
|
|
a25eb5fbb4 | ||
|
|
27dc42d102 | ||
|
|
7c458f04b9 | ||
|
|
8f7d12e30d | ||
|
|
b83b9caf4a | ||
|
|
bb212ffecb | ||
|
|
6b180783ed | ||
|
|
8cbaf6c702 |
No files matched your search
@@ -2,7 +2,7 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "42"
|
||||
version: "43"
|
||||
---
|
||||
|
||||
# REA
|
||||
@@ -169,6 +169,17 @@ explicit path or endpoint and do not need it.
|
||||
absent metadata does not prove a binary is not Go, and unparsed text records
|
||||
remain explicit. See the
|
||||
[Go binary guide](https://github.com/morluto/rea/blob/main/docs/go-binaries.md).
|
||||
- Unity IL2CPP names, member owners and string literals from `global-metadata.dat`:
|
||||
`inspect_il2cpp_metadata` with the explicit local file; MCP defaults to compact
|
||||
counts. Query sections, images, types, methods, fields, properties, events,
|
||||
parameters or string_literals through `inspect_analysis_view`
|
||||
with the returned `parent_evidence_id`, using exact filters, batch indices and
|
||||
selected fields. Parameters accept `method_indices`, avoiding a scan of all
|
||||
parameter pages. Missing collections in historical Evidence remain unknown.
|
||||
Versions 24, 27, 29, 31 and 39; no Unity, IL2CPP
|
||||
binary or disassembler is run. Unrecognized magic (possibly encrypted) and
|
||||
other versions are unsupported. Optional matching `binary_path` enables static native addresses and type references; query `method_addresses` with indices or addresses/states, preserving every shared-body method and both artifact identities. Without a linked binary native counts remain unknown. Byte-pattern fallback is inferred; no runtime behavior or same-build pairing is proved. See the
|
||||
[IL2CPP metadata guide](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
|
||||
- Embedded cargo-auditable Rust dependency metadata: `inspect_rust_binary`
|
||||
with the explicit local ELF, PE, or thin Mach-O file. This built-in
|
||||
target-free inspection requires no Rust installation or disassembler. Preserve
|
||||
@@ -247,6 +258,12 @@ Provider candidate facts describe possible providers, while top-level session
|
||||
capabilities describe the current route and `analysis_provider_binding` names the
|
||||
selected provider. Do not treat those scopes as interchangeable.
|
||||
|
||||
For a Ghidra startup timeout on a large native target, follow
|
||||
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
|
||||
before abandoning deep analysis or repeating the same cold import. The default
|
||||
330 seconds is configurable; changing the server's environment requires a new
|
||||
REA process.
|
||||
|
||||
## Work summary-first
|
||||
|
||||
Start with the default result and use its inline Evidence and graph context.
|
||||
|
||||
@@ -24,6 +24,17 @@ retain its complete source and receive that view immediately. Without that
|
||||
advertised choice, keep the existing complete call. CLI inspection returns
|
||||
portable complete Evidence; save it before projecting a selected inline view.
|
||||
|
||||
For native procedure/name/string inventories, select `detail: "summary"` when
|
||||
the connected tool advertises it and the task needs counts or selected rows.
|
||||
Use its parent Evidence ID with
|
||||
`view: {kind: "native-inventory", offset: 0, limit: 32, fields: ["value"]}`.
|
||||
Search tools retain the existing pattern/mode/case/document selection. Omit
|
||||
`fields` for complete records; `record.address` and `source_index` remain on
|
||||
selected rows. Selecting `value` keeps its string/decoding qualifications.
|
||||
Parent Evidence ID plus source index preserves same-address
|
||||
alternatives. Follow `coverage.next_offset` only as far as the question needs;
|
||||
exhausting retained rows does not establish complete program discovery.
|
||||
|
||||
Use `record_unknown` to track unresolved questions in the user's investigation
|
||||
and name the authority or environment still required. Supply supporting and
|
||||
contradicting evidence IDs. It updates the session registry directly.
|
||||
@@ -69,3 +80,17 @@ When the connected schema advertises `include_query_parameter_names`, select
|
||||
it in the view for distinct observed names with original record ordinals. Query
|
||||
values are omitted; the names establish neither required parameters nor a
|
||||
complete server schema. Keep the original parent Evidence for exact URLs.
|
||||
|
||||
Use `inspect_analysis_view` with `view: {kind: "environment-manifest"}` on the
|
||||
same retained HAR Evidence to project the primary origin's request lines into
|
||||
named tools with path templates, parameters, observed query names, value-free
|
||||
body shapes and record pointers for an agent environment. Opt into
|
||||
`template_rules` only when recorded identifiers need them; every template,
|
||||
parameter and merged sibling is an inference, no header, cookie or body value is
|
||||
copied, and the three stated unknowns stay unresolved.
|
||||
|
||||
Use `view: {kind: "environment-state"}` with the manifest's `template_rules` to
|
||||
extract the data those reads observed instead: entity types and identity
|
||||
fields, entities with recorded values, documents, and per-tool render plans
|
||||
checked against the recordings. The state comes only from reads before the
|
||||
first related write; types, identity and list membership are inferences.
|
||||
@@ -1,5 +1,32 @@
|
||||
# Native, managed, and packaged artifacts
|
||||
|
||||
For release comparisons, export `list_procedures` Evidence from each target before
|
||||
closing it. Close clears session records: import both bundles after collecting the
|
||||
two targets, or supply portable inline Evidence. `match_native_functions` pairs unique exact names and, when supplied,
|
||||
complete observed body bytes. Reuse optional same-context `list_names`,
|
||||
`procedure_info`, `read_bytes` and existing `analyze_function` Evidence; avoid
|
||||
decompiling every function merely to match it. Its summary retains a parent for
|
||||
`inspect_analysis_view` `function-matches` collections `matches` and `functions`.
|
||||
Select columns, batch IDs and page individual facts; aliases remain unresolved.
|
||||
Unpaired observations do not prove additions/removals. Existing matched dossier
|
||||
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
|
||||
|
||||
## Desktop installation directories
|
||||
|
||||
When an installation directory is supplied and its runtime is unknown, use
|
||||
`inspect_desktop_application` with its absolute path. Default MCP summary
|
||||
delivery retains complete Evidence. Read `files`, `omissions`, `candidates`, `components` or
|
||||
`next_probes` with `inspect_analysis_view` kind `desktop-application`, selecting
|
||||
exact paths, batched file/omission IDs or families and only needed fields and pages.
|
||||
Summary `probe_issues` retains counts and bounded previews. Select files with
|
||||
`has_issues: true` and fields `path`, `read_status`, `issues` to retrieve exact
|
||||
failures directly; preview text is display data, while `file_id` selects full facts. Preserve source
|
||||
coverage, read-range identity and unknowns; directory snapshot identity is not
|
||||
a digest of every file's contents. Component path markers do not establish
|
||||
runtime presence. For compact CLI output, use
|
||||
`inspect-desktop-application PATH --detail summary --evidence_output BUNDLE`.
|
||||
See [desktop installations](https://github.com/morluto/rea/blob/main/docs/desktop-applications.md).
|
||||
|
||||
## Go build metadata
|
||||
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or thin Mach-O when the
|
||||
@@ -11,6 +38,24 @@ unparsed module-text line. A missing record is an unknown, not proof of a
|
||||
non-Go binary; observed metadata does not authenticate its claims. Function,
|
||||
source and type recovery remain a separate native-analysis question.
|
||||
|
||||
## IL2CPP metadata
|
||||
|
||||
Use `inspect_il2cpp_metadata` for an explicit local Unity `global-metadata.dat`
|
||||
(versions 24, 27, 29, 31 and 39) when the question concerns assembly images,
|
||||
names, member owners, string literals, namespaces or index ranges. No `open_binary`, Unity installation or target
|
||||
execution is required. The CLI equivalent is `inspect-il2cpp-metadata PATH`.
|
||||
MCP defaults to counts with complete retained Evidence. Query `sections`,
|
||||
`images`, `types`, `methods`, `fields`, `properties`, `events`, `parameters` or
|
||||
`string_literals` through `inspect_analysis_view`; use exact names,
|
||||
batch indices and selected fields. Row indices are scoped by artifact SHA-256
|
||||
and collection; page offsets count filtered matches. Parameters support batch
|
||||
`method_indices` to read one method's parameters directly. Historical reports
|
||||
without member collections retain unknown counts rather than empty inventories. Preserve
|
||||
artifact SHA-256 and index widths. Unrecognized magic may mean encryption and
|
||||
is unsupported, not decrypted. Linking names to `GameAssembly` or
|
||||
`UnityFramework` code addresses remains a separate native-analysis question.
|
||||
See [IL2CPP metadata](https://github.com/morluto/rea/blob/main/docs/il2cpp-metadata.md).
|
||||
|
||||
## Rust dependency metadata
|
||||
|
||||
Use `inspect_rust_binary` for an explicit local ELF, PE, or thin Mach-O when
|
||||
@@ -124,6 +169,36 @@ operation's full ordered list. Candidate capabilities describe possible
|
||||
providers. Top-level session capabilities describe the current route, and
|
||||
`analysis_provider_binding` identifies the selected provider.
|
||||
|
||||
## Ghidra startup deadline recovery
|
||||
|
||||
`provider_timeout` during the first Ghidra query can cover import, auto-analysis,
|
||||
bridge startup, and readiness. The default 330,000 ms is configurable through
|
||||
`REA_GHIDRA_STARTUP_TIMEOUT_MS`; a smaller query still needs the same cold import.
|
||||
|
||||
Preserve useful Evidence before closing the session or restarting REA. Check
|
||||
`close_binary` and resolve reported cleanup failures. Set a larger startup budget
|
||||
in the environment that launches REA, then restart REA and reconnect the MCP
|
||||
client before reopening the same target. Closing and reopening a binary in the
|
||||
existing server keeps its original configuration and starts import again.
|
||||
Environment changes in an agent's shell do not update an already running MCP
|
||||
server. If its launch configuration cannot be changed, report that constraint
|
||||
and the exact retry setting instead of claiming a fixed technical limit.
|
||||
|
||||
The client's tool-call deadline must allow startup plus the requested query;
|
||||
server settings and progress notifications cannot override it. CLI analysis
|
||||
avoids the MCP client deadline, but retains the provider deadline. For example:
|
||||
|
||||
```sh
|
||||
REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000 rea analyze /absolute/path/to/app.exe --provider ghidra --json
|
||||
```
|
||||
|
||||
Thirty minutes is an example budget, not a completion guarantee. Keep host CPU
|
||||
and memory controls in place. Disable specific expensive analyzers only when
|
||||
their missing coverage is acceptable and retain the resulting limitations.
|
||||
Exported C and embedded resources remain useful independent evidence, but a
|
||||
timeout does not establish complete analysis or absence of code. See the
|
||||
[deadline and client guide](https://github.com/morluto/rea/blob/main/docs/mcp-contracts.md#ghidra-first-query-deadlines-and-recovery).
|
||||
|
||||
## Static memory and original file bytes
|
||||
|
||||
For Ghidra, `read_bytes` reads initialized analysis memory, including loader
|
||||
@@ -295,3 +370,22 @@ with two complete inspection Evidence records (or current-session retained
|
||||
references) and explicit scope. Preserve unknown types/duplicate names; static
|
||||
interface equality is not runtime compatibility. CLI accepts complete inline
|
||||
Evidence in `rea compare-wasm-interfaces input.json --json`.
|
||||
|
||||
## Godot PCK
|
||||
|
||||
For a standalone PCK, `open_binary` then `inspect_artifact` with
|
||||
`detail: "summary"` retains the complete static inventory. Use the returned
|
||||
`parent_evidence_id` with `inspect_analysis_view` inventory occurrences filtered
|
||||
by exact `logical_paths` or `occurrence_ids`, and join `artifact_id` to nodes
|
||||
filtered by `artifact_ids`. Select only needed fields; identity links remain.
|
||||
Prefer `inspect-artifact --detail summary --evidence_output PATH` for CLI tasks;
|
||||
the same CLI discovers embedded packs via their EOF trailer without opening a
|
||||
native analysis provider. Normal native opening remains unchanged.
|
||||
|
||||
Distinguish declared MD5 from observed SHA-256 and graph/page completeness from
|
||||
`source_coverage`. Encrypted directories and sparse bundles have unknown entry
|
||||
coverage; encrypted members have unknown content identity. An empty filtered
|
||||
page cannot prove absence in unknown coverage. Retained views are historical,
|
||||
not fresh filesystem checks. No decryption, Godot resource decoding, GDScript
|
||||
decompilation or target execution is provided. See
|
||||
[Godot PCK](https://github.com/morluto/rea/blob/main/docs/godot-pck.md).
|
||||
@@ -69,6 +69,12 @@ with `inspect_analysis_view` and `view: {kind: "capture", detail: "preview", off
|
||||
Select complete record pages only for needed headers, metadata or payloads; views
|
||||
read the historical parent without repeating capture decoding.
|
||||
|
||||
For selected protobuf/gRPC bodies, keep raw bytes as the source and use
|
||||
`inspect_analysis_view` kind `protobuf`, collection `bodies`, `frames` or
|
||||
`fields`. Batch exact record ordinals and field numbers/IDs, and select
|
||||
`schema_value` or `text` only when needed. Supply a local descriptor set and
|
||||
message type for schema interpretation; compressed payloads remain unknown.
|
||||
|
||||
Inspect native `.saz` snapshots with `inspect_web_network_capture` and
|
||||
`format: "saz"`; encrypted archives are unsupported. For live
|
||||
traffic analysis, connect Everywhere's separate MCP server and use its
|
||||
|
||||
@@ -59,6 +59,7 @@ jobs:
|
||||
web_network: ${{ steps.plan.outputs.web_network }}
|
||||
web_runtime: ${{ steps.plan.outputs.web_runtime }}
|
||||
web_source_map: ${{ steps.plan.outputs.web_source_map }}
|
||||
agentenv: ${{ steps.plan.outputs.agentenv }}
|
||||
nativeaot: ${{ steps.plan.outputs.nativeaot }}
|
||||
ghidra_bridge: ${{ steps.plan.outputs.ghidra_bridge }}
|
||||
website: ${{ steps.plan.outputs.website }}
|
||||
@@ -302,7 +303,8 @@ jobs:
|
||||
needs.changes.outputs.recorded_crash == 'true' ||
|
||||
needs.changes.outputs.web_network == 'true' ||
|
||||
needs.changes.outputs.web_runtime == 'true' ||
|
||||
needs.changes.outputs.web_source_map == 'true'
|
||||
needs.changes.outputs.web_source_map == 'true' ||
|
||||
needs.changes.outputs.agentenv == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
@@ -748,6 +750,9 @@ jobs:
|
||||
tests/boundary/providers/native/artifactProvider.extractionCompleteness.test.ts
|
||||
tests/boundary/filesystem/setupUnavailableProvider.test.ts
|
||||
tests/boundary/filesystem/windowsNpxRegistration.test.ts
|
||||
tests/boundary/filesystem/clientConfigurationEnvironment.test.ts
|
||||
tests/boundary/filesystem/clientConfigurationLayeredEnvironment.test.ts
|
||||
tests/acceptance/setup/registrationRuntimeEnvironmentCli.test.ts
|
||||
tests/boundary/android/workspaceGuardianRetry.test.ts
|
||||
tests/acceptance/setup/homeDirectoryFallback.test.ts
|
||||
tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts
|
||||
@@ -909,6 +914,13 @@ jobs:
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
agentenv:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.agentenv == 'true'
|
||||
uses: ./.github/workflows/real-agentenv.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
|
||||
nativeaot:
|
||||
needs: [changes, static]
|
||||
if: needs.changes.outputs.nativeaot == 'true'
|
||||
@@ -953,6 +965,7 @@ jobs:
|
||||
web_network,
|
||||
web_runtime,
|
||||
web_source_map,
|
||||
agentenv,
|
||||
nativeaot,
|
||||
ghidra_bridge,
|
||||
website,
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
name: Real agentenv environment bundle
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
NODE_OPTIONS: --max-old-space-size=768
|
||||
PIP_DISABLE_PIP_VERSION_CHECK: "1"
|
||||
|
||||
jobs:
|
||||
real-agentenv:
|
||||
name: Export and run an environment bundle with the agentenv runner
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 30
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
|
||||
with:
|
||||
python-version: "3.12"
|
||||
# Public ECR caps anonymous pulls by data per source IP and hosted runners share IPs, so the
|
||||
# framework's image builds can fail here for hours. mirror.gcr.io serves the same Docker
|
||||
# official images under the same digests, so each base image the pinned framework builds from
|
||||
# is pulled there and named as its Dockerfile names it, as the framework's own CI does.
|
||||
# Digest-pinned names are set in containerd, whose images Docker reads only on the containerd
|
||||
# image store; switching stores restarts dockerd, so it happens before anything uses Docker.
|
||||
- name: Use the containerd image store
|
||||
run: |
|
||||
[ -s /etc/docker/daemon.json ] || echo '{}' | sudo tee /etc/docker/daemon.json >/dev/null
|
||||
jq '.features["containerd-snapshotter"] = true' /etc/docker/daemon.json > "$RUNNER_TEMP/daemon.json"
|
||||
sudo mv "$RUNNER_TEMP/daemon.json" /etc/docker/daemon.json
|
||||
sudo systemctl restart docker
|
||||
docker info --format '{{json .DriverStatus}}' | grep -q io.containerd.snapshotter
|
||||
- name: Acquire the pinned agentenv runner in isolated runner storage and check host Docker
|
||||
shell: bash
|
||||
run: |
|
||||
runner_root="${RUNNER_TEMP}/rea-agentenv-runner"
|
||||
python -m venv "${runner_root}"
|
||||
"${runner_root}/bin/python" -m pip install --require-virtualenv --no-cache-dir 'agentenv-framework==0.9.1334'
|
||||
"${runner_root}/bin/agent-env" --version
|
||||
docker version --format '{{.Server.Version}}'
|
||||
echo "REA_AGENT_ENV_COMMAND=${runner_root}/bin/agent-env" >> "${GITHUB_ENV}"
|
||||
echo "REA_DOCKER_COMMAND=$(command -v docker)" >> "${GITHUB_ENV}"
|
||||
- name: Pull the pinned framework's public ECR base images from mirror.gcr.io
|
||||
shell: bash
|
||||
run: |
|
||||
package="$("${REA_AGENT_ENV_COMMAND%/agent-env}/python" -c 'import agent_env, pathlib; print(pathlib.Path(agent_env.__file__).parent)')"
|
||||
grep -rhoE '^FROM +public\.ecr\.aws/docker/library/[^ ]+' --include='Dockerfile*' "${package}" | awk '{print $2}' | sort -u |
|
||||
while read -r image; do
|
||||
name=${image#public.ecr.aws/docker/library/}
|
||||
source=mirror.gcr.io/library/$name
|
||||
case $name in *@*) source=mirror.gcr.io/library/${name%%[:@]*}@${name#*@} ;; esac
|
||||
for attempt in 1 2 3; do
|
||||
docker pull --quiet "$source" && break
|
||||
if [ "$attempt" = 3 ]; then echo "::warning::couldn't pull $source; $image's build will pull it"; continue 2; fi
|
||||
sleep $((attempt * 10))
|
||||
done
|
||||
case ${name%@*} in *:*) docker tag "$source" "${image%@*}" ;; esac
|
||||
case $image in *@*) sudo ctr --namespace moby images tag "$source" "$image" >/dev/null ;; esac
|
||||
done
|
||||
- name: Export a bundle from the source-owned capture and run it with the agentenv runner
|
||||
run: npm run verify:agentenv:environment
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "6.6.0"
|
||||
".": "6.7.0"
|
||||
}
|
||||
@@ -1,5 +1,21 @@
|
||||
# Changelog
|
||||
|
||||
## [6.7.0](https://github.com/morluto/rea/compare/rea-agents-6.6.0...rea-agents-6.7.0) (2026-10-18)
|
||||
|
||||
### Highlights
|
||||
|
||||
- Client configuration edits follow the duplicate-member semantics used by MCP clients.
|
||||
|
||||
### Changes
|
||||
|
||||
- Includes the [6.6.0 changes and migration notes](https://github.com/morluto/rea/releases/tag/rea-agents-6.6.0). That version was tagged but was not published to npm.
|
||||
|
||||
### Fixes
|
||||
|
||||
- **Client configuration:** Setup edits the last repeated JSON/JSONC member, matching the value clients actually read. Setup and uninstall refuse a removal that would reactivate a shadowed duplicate, preserving the original configuration. ([#4957](https://github.com/morluto/rea/pull/4957), [morluto/rea#4952](https://github.com/morluto/rea/issues/4952)). Thanks @kaoru0822-kitauji, @xu91102.
|
||||
|
||||
Full contribution record: [4 commits and 1 merged PR](https://github.com/morluto/rea/blob/rea-agents-6.7.0/docs/releases/6.7.0.contributions.json).
|
||||
|
||||
## [6.6.0](https://github.com/morluto/rea/compare/rea-agents-6.5.0...rea-agents-6.6.0) (2026-10-18)
|
||||
|
||||
### Highlights
|
||||
|
||||
@@ -174,31 +174,34 @@ CLI commands use the same workflows.
|
||||
REA requires Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+, plus npm.
|
||||
Additional tools and host support depend on the target:
|
||||
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ---------------------- | ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
|
||||
| Target | What REA returns | Requirements and guide |
|
||||
| ---------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Desktop installations | Runtime candidates, supporting files, component markers and next probes | Built-in bounded directory/header inspection; [desktop applications](docs/desktop-applications.md) |
|
||||
| Native binaries | Pseudocode, assembly, strings, symbols, calls and references | Hopper, Ghidra or IDA; [native analysis](docs/native-investigation.md) |
|
||||
| Offline ELF layout | Sections, segments, original symbols/relocations and static mitigation candidates | Caller-supplied pwntools on Linux x64; [binary diagnostics](docs/binary-diagnostics.md) |
|
||||
| Go binaries | Embedded compiler version, modules, replacements and build settings | Built-in static ELF/PE/thin Mach-O inspection; no Go installation required |
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
| Recorded Linux crashes | Raw notes, every recorded thread's registers/signals and optional mapping candidates | Caller-supplied pwntools on Linux x64; optional GDB/pwndbg; [recorded crashes](docs/recorded-crashes.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC and native add-on relationships | Node.js and npm; Linux: procps-compatible `ps`; [application analysis](docs/javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations and requested screenshots | A Chrome-family browser; [browser analysis](docs/browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
|
||||
| Flutter apps | Dart snapshot hash, per-ABI library digests and engine build provenance | None (static APK parsing); [Flutter guide](docs/flutter-analysis.md) |
|
||||
| Firmware | Regions, extraction results and native-analysis handoffs | Binwalk / Unblob on Linux; [firmware guide](docs/firmware-analysis.md) |
|
||||
| Recorded packets | Selected fields, repeated values, filters and decode-as rules | Linux: TShark 4.x + prlimit; [capture guide](docs/packet-captures.md) |
|
||||
| Packet summaries | Endpoints, DNS, TLS SNI, protocol distribution and HTTP requests | Caller-supplied tshark and capinfos (Linux-verified); [pcap guide](docs/pcap-analysis.md) |
|
||||
| Recorded logic signals | Decoder annotations, sample ranges and diagnostics | Linux: sigrok-cli 0.7.2 + prlimit; [signal recordings](docs/signal-recordings.md) |
|
||||
| Packages and resources | File inventories, digests, plists, Apple bundle anatomy and extracted resources | [Artifact and JavaScript guide](docs/javascript-artifact-reconstruction.md), [Apple applications](docs/apple-application-analysis.md) |
|
||||
| Process behavior | Terminal output, interactions, exit and filesystem observations, and run comparisons | Linux/macOS with a native PTY; [process capture](docs/process-capture.md) |
|
||||
|
||||
Static JavaScript and .NET inspection read the supplied files without running
|
||||
the application. Runtime capture runs or interacts with the selected target
|
||||
|
||||
@@ -458,6 +458,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
FunctionAnalysisObservation observation = analyzeFunctionObservation(request.params);
|
||||
JsonObject result = functionObservation(observation.dossier(), observation.limitations());
|
||||
result.add("relationship_coverage", observation.relationshipCoverage());
|
||||
if (observation.decompilerFailed()) result.addProperty("cache_policy", "live");
|
||||
yield result;
|
||||
}
|
||||
case "set_address_name", "set_addresses_names" -> {
|
||||
@@ -1540,12 +1541,23 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
requireKeys(params, Set.of("procedure"));
|
||||
Function function = resolveProcedure(requireString(params, "procedure"));
|
||||
InstructionScan scan = scanInstructions(function, Integer.MAX_VALUE);
|
||||
DecompileResults decompilation = decompile(function);
|
||||
DecompileResults decompilation;
|
||||
String decompilerFailure = null;
|
||||
try {
|
||||
decompilation = decompile(function);
|
||||
} catch (RequestFailure failure) {
|
||||
// The listing, reference and CFG facts below do not depend on the
|
||||
// native decompiler. Cancellation still ends the request.
|
||||
if (!failure.code.equals("decompile_failed")) throw failure;
|
||||
decompilation = null;
|
||||
decompilerFailure = failure.getMessage().strip();
|
||||
}
|
||||
String pseudocode = decompiledText(decompilation);
|
||||
JsonObject availability = new JsonObject();
|
||||
availability.add("pseudocode", facetAvailability(
|
||||
pseudocode != null,
|
||||
function.isExternal() ? "Ghidra does not decompile external functions."
|
||||
decompilerFailure != null ? decompilerFailure
|
||||
: function.isExternal() ? "Ghidra does not decompile external functions."
|
||||
: function.getBody().isEmpty() ? "Ghidra has no function body to decompile."
|
||||
: "Ghidra completed decompilation without returning pseudocode."
|
||||
));
|
||||
@@ -1594,9 +1606,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
result.add("basic_blocks", blocks);
|
||||
result.add(
|
||||
"native_api",
|
||||
nativeApiBoundary(function, decompilation)
|
||||
decompilerFailure == null
|
||||
? nativeApiBoundary(function, decompilation)
|
||||
: unavailableNativeApiBoundary(decompilerFailure)
|
||||
);
|
||||
result.add(
|
||||
"native_value_flow",
|
||||
decompilerFailure == null
|
||||
? nativeValueFlow(decompilation)
|
||||
: unavailableNativeValueFlow(decompilerFailure)
|
||||
);
|
||||
result.add("native_value_flow", nativeValueFlow(decompilation));
|
||||
// Successful full queries distinguish collected empty sets from missing facts.
|
||||
// Availability does not claim that Ghidra resolved every runtime relationship.
|
||||
for (String facet : List.of("comments", "callers", "callees", "references", "strings_names", "cfg")) {
|
||||
@@ -1643,6 +1662,12 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
!decompilation.getErrorMessage().isBlank()) {
|
||||
limitations.add("Ghidra decompiler diagnostic: " + decompilation.getErrorMessage());
|
||||
}
|
||||
if (decompilerFailure != null) {
|
||||
limitations.add(
|
||||
decompilerFailure + (decompilerFailure.endsWith(".") ? " " : ". ") +
|
||||
"Pseudocode, the native API boundary and p-code value flow are unavailable for this procedure; assembly, comments, callers, callees, references, strings, names and basic blocks come from the Ghidra program listing and were still collected."
|
||||
);
|
||||
}
|
||||
result.add("limitations", limitations);
|
||||
monitor.checkCancelled();
|
||||
int missingCallers = 0;
|
||||
@@ -1651,25 +1676,16 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
if (unrepresentedCaller(reference, callerFunctions)) missingCallers++;
|
||||
}
|
||||
return new FunctionAnalysisObservation(result, calls.limitations(),
|
||||
callRelationshipCoverage(calls, missingCallers));
|
||||
callRelationshipCoverage(calls, missingCallers), decompilerFailure != null);
|
||||
}
|
||||
|
||||
private JsonObject nativeValueFlow(DecompileResults decompilation) throws Exception {
|
||||
HighFunction highFunction =
|
||||
decompilation == null ? null : decompilation.getHighFunction();
|
||||
if (highFunction == null) {
|
||||
JsonObject unavailable = new JsonObject();
|
||||
unavailable.addProperty("available", false);
|
||||
unavailable.addProperty(
|
||||
"reason",
|
||||
return unavailableNativeValueFlow(
|
||||
"Ghidra has no HighFunction p-code model for this procedure"
|
||||
);
|
||||
JsonArray limitations = new JsonArray();
|
||||
limitations.add(
|
||||
"No p-code operations or def-use relationships were recovered for this procedure."
|
||||
);
|
||||
unavailable.add("limitations", limitations);
|
||||
return unavailable;
|
||||
}
|
||||
|
||||
List<PcodeOpAST> selected = new ArrayList<>();
|
||||
@@ -2357,6 +2373,18 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
return result;
|
||||
}
|
||||
|
||||
private static JsonObject unavailableNativeValueFlow(String reason) {
|
||||
JsonObject unavailable = new JsonObject();
|
||||
unavailable.addProperty("available", false);
|
||||
unavailable.addProperty("reason", reason);
|
||||
JsonArray limitations = new JsonArray();
|
||||
limitations.add(
|
||||
"No p-code operations or def-use relationships were recovered for this procedure."
|
||||
);
|
||||
unavailable.add("limitations", limitations);
|
||||
return unavailable;
|
||||
}
|
||||
|
||||
private static JsonObject unavailableNativeApiBoundary(String reason) {
|
||||
JsonObject unavailable = new JsonObject();
|
||||
unavailable.addProperty("available", false);
|
||||
@@ -4875,7 +4903,7 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
private record InventoryItem(Address address, String value, JsonObject facts) {}
|
||||
private record AddressNameRequest(String inputAddress, Address address, String name) {}
|
||||
private record FunctionEntry(Function function, InventoryItem item) {}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage) {}
|
||||
private record FunctionAnalysisObservation(JsonObject dossier, List<String> limitations, JsonObject relationshipCoverage, boolean decompilerFailed) {}
|
||||
private record CallBoundaryObservation(List<String> limitations, Integer unrepresentedCalleeCalls, JsonArray unresolvedCalls) {}
|
||||
private record SwiftStringLiteralDecode(JsonArray literals, JsonArray omissions) {}
|
||||
private record SwiftLiteralLayout(
|
||||
|
||||
+53
-23
@@ -236,6 +236,15 @@ def _name_matches(document, address, name):
|
||||
return observed == name or (not name and observed is None)
|
||||
|
||||
|
||||
def _bookmark_name(document, address):
|
||||
getter = getattr(document, "getBookmarkName", None)
|
||||
if not callable(getter):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose getBookmarkName"
|
||||
)
|
||||
return getter(address)
|
||||
|
||||
|
||||
def _address(document, value=None):
|
||||
"""Keep explicit hexadecimal coordinates distinct from symbol names."""
|
||||
if value is None:
|
||||
@@ -431,10 +440,6 @@ def _procedure_map(document):
|
||||
return result
|
||||
|
||||
|
||||
def _strings(document):
|
||||
return {address: record["value"] for address, record in _string_inventory(document).items()}
|
||||
|
||||
|
||||
def _string_record(segment, address, display):
|
||||
"""Read one native typed object, retaining its shortened display separately."""
|
||||
length = segment.getObjectLength(address)
|
||||
@@ -521,7 +526,7 @@ def _search_inventory(document, kind):
|
||||
if kind == "procedure":
|
||||
values = _procedure_map(document)
|
||||
elif kind == "string":
|
||||
values = _strings(document)
|
||||
values = _string_inventory(document)
|
||||
elif kind == "name":
|
||||
values = {_hex(address): value for address, value in _name_map(document).items()}
|
||||
else:
|
||||
@@ -548,23 +553,17 @@ def _search_results(document, kind, params):
|
||||
else:
|
||||
raise CapabilityUnavailableError("Regex matching requires REA's supervised worker; the native bridge only executes literal searches")
|
||||
|
||||
selected = []
|
||||
for item in _search_inventory(document, kind):
|
||||
if not matches(item[1]):
|
||||
continue
|
||||
selected.append(item)
|
||||
if not selected:
|
||||
return []
|
||||
if kind == "string":
|
||||
records = _string_inventory(document)
|
||||
return [{"address": address, **records[address]} for address, _ in selected]
|
||||
return [
|
||||
{
|
||||
"address": address,
|
||||
"value": value,
|
||||
}
|
||||
for address, value in selected
|
||||
return [
|
||||
{"address": address, **record}
|
||||
for address, record in _search_inventory(document, kind)
|
||||
if matches(record["value"])
|
||||
]
|
||||
return [
|
||||
{"address": address, "value": value}
|
||||
for address, value in _search_inventory(document, kind)
|
||||
if matches(value)
|
||||
]
|
||||
|
||||
|
||||
def _unavailable(reason):
|
||||
@@ -1004,15 +1003,46 @@ def _dispatch(method, params):
|
||||
observed = getter(address)
|
||||
return observed == comment or (comment == "" and observed is None)
|
||||
if method == "list_bookmarks":
|
||||
return [{"address": _hex(item), "name": document.getBookmarkName(item)} for item in document.getBookmarks()]
|
||||
return [{"address": _hex(item), "name": _bookmark_name(document, item)} for item in document.getBookmarks()]
|
||||
if method == "set_bookmark":
|
||||
address = _address(document, params.get("address"))
|
||||
_segment(document, address)
|
||||
name = params.get("name")
|
||||
if name is not None:
|
||||
_api_text(name, "Bookmark name")
|
||||
document.setBookmarkAtAddress(address, name)
|
||||
return document.hasBookmarkAtAddress(address)
|
||||
has_bookmark = getattr(document, "hasBookmarkAtAddress", None)
|
||||
if not callable(has_bookmark):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose hasBookmarkAtAddress"
|
||||
)
|
||||
exists = has_bookmark(address)
|
||||
if name is None and exists:
|
||||
# Omission means preserve the existing bookmark and its name.
|
||||
return True
|
||||
if not callable(getattr(document, "getBookmarkName", None)):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose getBookmarkName"
|
||||
)
|
||||
if name == "" and exists:
|
||||
rename = getattr(document, "renameBookmarkAtAddress", None)
|
||||
if not callable(rename):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose renameBookmarkAtAddress"
|
||||
)
|
||||
rename(address, name)
|
||||
else:
|
||||
setter = getattr(document, "setBookmarkAtAddress", None)
|
||||
if not callable(setter):
|
||||
raise CapabilityUnavailableError(
|
||||
"Hopper's public Python API does not expose setBookmarkAtAddress"
|
||||
)
|
||||
setter(address, name)
|
||||
if not has_bookmark(address):
|
||||
return False
|
||||
observed = _bookmark_name(document, address)
|
||||
if name is None:
|
||||
return observed is None or observed == ""
|
||||
return observed == name or (name == "" and observed is None)
|
||||
if method == "unset_bookmark":
|
||||
address = _address(document, params.get("address"))
|
||||
# Permit removing a legacy orphan bookmark, but do not report success
|
||||
|
||||
@@ -12,16 +12,29 @@ struct ProcessIdentityObservation: Encodable {
|
||||
let pid: Int32
|
||||
let state: String
|
||||
let identity: String?
|
||||
let effective_uid: UInt32?
|
||||
let lifecycle: String?
|
||||
let relation: ProcessRelationObservation?
|
||||
let reason: String?
|
||||
}
|
||||
|
||||
private struct ProcessMetadata {
|
||||
struct ProcessRelationObservation: Encodable {
|
||||
let state: String
|
||||
let parent_pid: Int32
|
||||
let process_group_id: Int32
|
||||
}
|
||||
|
||||
struct ProcessMetadata {
|
||||
let identity: String
|
||||
let effectiveUid: UInt32
|
||||
let lifecycle: String
|
||||
let parentPid: Int32
|
||||
let processGroupId: Int32
|
||||
let pointerSize: Int
|
||||
}
|
||||
|
||||
struct ProcessSnapshot {
|
||||
fileprivate let metadata: [Int32: ProcessMetadata]
|
||||
let metadata: [Int32: ProcessMetadata]
|
||||
let failure: String?
|
||||
}
|
||||
|
||||
@@ -75,6 +88,10 @@ func readProcessSnapshot(
|
||||
process.kp_proc.p_pid,
|
||||
ProcessMetadata(
|
||||
identity: "\(start.tv_sec):\(start.tv_usec)",
|
||||
effectiveUid: process.kp_eproc.e_ucred.cr_uid,
|
||||
lifecycle: process.kp_proc.p_stat == SZOMB ? "zombie" : "non-zombie",
|
||||
parentPid: process.kp_eproc.e_ppid,
|
||||
processGroupId: process.kp_eproc.e_pgid,
|
||||
pointerSize: process.kp_proc.p_flag & P_LP64 != 0 ? 8 : 4
|
||||
)
|
||||
)
|
||||
@@ -87,20 +104,61 @@ func readProcessSnapshot(
|
||||
}
|
||||
|
||||
func readProcessIdentities(pids: [Int32]) -> [ProcessIdentityObservation] {
|
||||
let snapshot = readProcessSnapshot()
|
||||
readProcessIdentities(pids: pids, snapshot: readProcessSnapshot())
|
||||
}
|
||||
|
||||
func readProcessIdentities(
|
||||
pids: [Int32],
|
||||
snapshot: ProcessSnapshot
|
||||
) -> [ProcessIdentityObservation] {
|
||||
return pids.map { pid in
|
||||
guard pid > 0 else {
|
||||
return unavailableIdentity(pid: pid, reason: "invalid_process_id")
|
||||
}
|
||||
guard let metadata = snapshot.metadata[pid] else {
|
||||
return unavailableIdentity(
|
||||
if let failure = snapshot.failure {
|
||||
return unavailableIdentity(pid: pid, reason: failure)
|
||||
}
|
||||
// This records absence from this successful KERN_PROC_ALL snapshot only.
|
||||
// Unlike the Linux procfs path, it has no PID-specific kill(0)
|
||||
// corroboration; callers must not treat it as universal or future exit
|
||||
// proof.
|
||||
return ProcessIdentityObservation(
|
||||
pid: pid,
|
||||
reason: snapshot.failure ?? "process_not_in_snapshot"
|
||||
state: "absent",
|
||||
identity: nil,
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
reason: "not-in-snapshot"
|
||||
)
|
||||
}
|
||||
return ProcessIdentityObservation(pid: pid, state: "readable", identity: metadata.identity, reason: nil)
|
||||
return ProcessIdentityObservation(
|
||||
pid: pid,
|
||||
state: "readable",
|
||||
identity: metadata.identity,
|
||||
effective_uid: metadata.effectiveUid,
|
||||
lifecycle: metadata.lifecycle,
|
||||
relation: ProcessRelationObservation(
|
||||
state: "observed",
|
||||
parent_pid: metadata.parentPid,
|
||||
process_group_id: metadata.processGroupId
|
||||
),
|
||||
reason: nil
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
private func unavailableIdentity(pid: Int32, reason: String) -> ProcessIdentityObservation {
|
||||
ProcessIdentityObservation(pid: pid, state: "unavailable", identity: nil, reason: reason)
|
||||
ProcessIdentityObservation(
|
||||
pid: pid,
|
||||
state: "unavailable",
|
||||
identity: nil,
|
||||
effective_uid: nil,
|
||||
lifecycle: nil,
|
||||
relation: nil,
|
||||
reason: reason
|
||||
)
|
||||
}
|
||||
|
||||
enum RunTokenReadError: Error {
|
||||
|
||||
@@ -95,7 +95,7 @@ def main():
|
||||
try:
|
||||
encoded = json.dumps(reply, ensure_ascii=True, allow_nan=False).encode("utf-8")
|
||||
if len(encoded) > OUTPUT_BYTES:
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds64 MiB reply budget."}).encode("utf-8")
|
||||
encoded = json.dumps({"ok": False, "reason": "output-limit", "message": "Complete debugger context exceeds the 64 MiB reply budget."}).encode("utf-8")
|
||||
descriptor = os.open(request["reply_path"], os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
|
||||
with os.fdopen(descriptor, "wb") as handle:
|
||||
handle.write(encoded)
|
||||
|
||||
@@ -14,7 +14,7 @@ BRIDGE = Path(__file__).with_name("core_context.py")
|
||||
|
||||
|
||||
class CoreContextTests(unittest.TestCase):
|
||||
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian"):
|
||||
def inspect(self, failure=None, connection="core", symbols=None, architecture="x86_64-little-endian", objfile="recorded-executable"):
|
||||
commands, reads, queries = [], [], []
|
||||
state = {"executable": None, "symbols": None, "connection": None}
|
||||
gdb = types.ModuleType("gdb")
|
||||
@@ -50,7 +50,7 @@ class CoreContextTests(unittest.TestCase):
|
||||
def maps():
|
||||
self.assertIsNone(state["executable"], "Helper executable must be discarded before mapping queries")
|
||||
queries.append("maps")
|
||||
return [types.SimpleNamespace(start=4096, end=8192, offset=0, flags=5, objfile="recorded-executable")]
|
||||
return [types.SimpleNamespace(start=4096, end=8192, offset=0, flags=5, objfile=objfile)]
|
||||
|
||||
pwndbg.aglib.vmmap.get = maps
|
||||
modules = {module.__name__: module for module in [
|
||||
@@ -104,6 +104,15 @@ class CoreContextTests(unittest.TestCase):
|
||||
self.assertEqual(reads, [])
|
||||
self.assertEqual(queries, [])
|
||||
|
||||
def test_output_limit_message_separates_the_words_and_names_the_limit(self):
|
||||
result, _, reads, queries = self.inspect(objfile="m" * (64 * 1024 * 1024))
|
||||
self.assertFalse(result["ok"])
|
||||
self.assertEqual(result["reason"], "output-limit")
|
||||
self.assertNotIn("exceeds64", result["message"])
|
||||
self.assertEqual(result["message"], "Complete debugger context exceeds the 64 MiB reply budget.")
|
||||
self.assertEqual(reads, [])
|
||||
self.assertEqual(queries, ["maps"])
|
||||
|
||||
def test_wrong_connection_or_remaining_symbols_never_queries_mappings(self):
|
||||
for options in [{"connection": "native"}, {"symbols": "unexpected-symbols"}]:
|
||||
with self.subTest(options=options):
|
||||
|
||||
@@ -29,6 +29,7 @@ export default defineConfig({
|
||||
items: [
|
||||
{ text: "Native binaries", link: "/native-investigation" },
|
||||
{ text: "Go build metadata", link: "/go-binaries" },
|
||||
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
|
||||
{ text: "Rust dependencies", link: "/rust-binaries" },
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
@@ -43,12 +44,14 @@ export default defineConfig({
|
||||
text: "Web network captures",
|
||||
link: "/web-network-captures",
|
||||
},
|
||||
{ text: "Agent environments", link: "/agent-environments" },
|
||||
{ text: "Electron observation", link: "/electron-observation" },
|
||||
{ text: "Electron fuses", link: "/electron-fuses" },
|
||||
{ text: "Process capture", link: "/process-capture" },
|
||||
{ text: ".NET assemblies", link: "/managed-code-analysis" },
|
||||
{ text: "Android applications", link: "/android-analysis" },
|
||||
{ text: "Apple applications", link: "/apple-application-analysis" },
|
||||
{ text: "Desktop installations", link: "/desktop-applications" },
|
||||
{ text: "Firmware", link: "/firmware-analysis" },
|
||||
{ text: "SQLite databases", link: "/sqlite-databases" },
|
||||
{ text: "Packet captures", link: "/packet-captures" },
|
||||
|
||||
@@ -0,0 +1,426 @@
|
||||
# Agent environments from recorded captures
|
||||
|
||||
Use `rea environment-serve` to serve an environment manifest as an agent
|
||||
environment: an MCP server whose tools answer from the records of one retained
|
||||
web network capture, plus the environment card the agentenv framework reads to
|
||||
discover it. Nothing is fetched from the recorded application; every answer is
|
||||
retained Evidence. `rea environment-export` writes the same pair as a bundle
|
||||
folder `agent-env run` builds and runs; see [Export a bundle](#export-a-bundle).
|
||||
|
||||
```sh
|
||||
rea inspect-web-network-capture /absolute/session.har har --format json > capture.json
|
||||
rea inspect-analysis-view view.json --format json > manifest.json
|
||||
rea environment-serve manifest.json --evidence capture.json --port 8765
|
||||
```
|
||||
|
||||
The first command inspects the capture; the second projects the
|
||||
[environment manifest](web-network-captures.md#environment-manifest) view from
|
||||
it, where `view.json` wraps the capture Evidence as the inline source:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": { "kind": "inline", "evidence": { "evidence_id": "ev_…" } },
|
||||
"view": {
|
||||
"kind": "environment-manifest",
|
||||
"template_rules": ["after-uid-literal", "opaque-token", "sibling-variance"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The third command serves that manifest. Both served files are complete REA
|
||||
Evidence JSON as the commands write it with `--format json`; the `evidence`
|
||||
member of `view.json` is the whole of `capture.json`.
|
||||
|
||||
## From a browser scenario
|
||||
|
||||
The capture can also come from REA itself: a
|
||||
[browser scenario](browser-scenario-contract.md) records a scripted session
|
||||
in a browser REA launches, and its Evidence is a manifest parent like a HAR
|
||||
inspection. Select the `network` event family and response bodies. A replayed
|
||||
reply carries up to 256 KiB of a recorded body, but the scenario keeps at most
|
||||
`max_network_body_bytes` of each body (64 KiB by default), and only within its
|
||||
allocation budgets: base64 is charged about 11 bytes per body byte, and
|
||||
redaction more the shorter a declared secret is (about 35 bytes per body byte
|
||||
for a 12-character `admin_password`). A body over
|
||||
`max_network_body_allocation_bytes` (1 MiB by default) is kept as `unavailable`
|
||||
with that reason; one over `max_event_allocation_bytes` (4 MiB) or
|
||||
`max_total_event_allocation_bytes` (16 MiB) loses its whole content event,
|
||||
headers included, which the capture lists among its omissions. Either way its
|
||||
tool replays no body. The example raises all four so that 256 KiB bodies are
|
||||
kept when the declared secret has at least 12 characters:
|
||||
|
||||
```json
|
||||
{
|
||||
"browser": { "mode": "launch", "executable_path": "/usr/bin/chromium" },
|
||||
"start_url": { "url": "http://127.0.0.1:5230/auth", "query": [] },
|
||||
"secrets": [
|
||||
{ "secret_id": "admin_password", "environment_variable": "APP_PASSWORD" }
|
||||
],
|
||||
"actions": [
|
||||
{
|
||||
"step_id": "user",
|
||||
"action": "fill",
|
||||
"locator": { "kind": "css", "selector": "input[name=username]" },
|
||||
"value": { "source": "literal", "value": "admin" }
|
||||
},
|
||||
{
|
||||
"step_id": "password",
|
||||
"action": "fill",
|
||||
"locator": { "kind": "css", "selector": "input[type=password]" },
|
||||
"value": { "source": "secret", "secret_id": "admin_password" }
|
||||
},
|
||||
{
|
||||
"step_id": "sign-in",
|
||||
"action": "press",
|
||||
"locator": { "kind": "css", "selector": "input[type=password]" },
|
||||
"key": "Enter"
|
||||
},
|
||||
{
|
||||
"step_id": "first-memo",
|
||||
"action": "goto",
|
||||
"destination": { "url": "http://127.0.0.1:5230/m/1", "query": [] },
|
||||
"wait_until": "load"
|
||||
}
|
||||
],
|
||||
"capture": {
|
||||
"events": ["network"],
|
||||
"network": {
|
||||
"request_body": true,
|
||||
"response_body": true,
|
||||
"header_values": false
|
||||
},
|
||||
"limits": {
|
||||
"max_network_body_bytes": 262144,
|
||||
"max_network_body_allocation_bytes": 12582912,
|
||||
"max_event_allocation_bytes": 12582912,
|
||||
"max_total_event_allocation_bytes": 67108864
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```sh
|
||||
APP_PASSWORD=… rea capture-browser-scenario scenario.json --format json > capture.json
|
||||
rea inspect-analysis-view view.json --format json > manifest.json # view.json wraps capture.json as before
|
||||
rea environment-serve manifest.json --evidence capture.json --port 8765
|
||||
```
|
||||
|
||||
Each network transaction becomes one record, in request order, and the
|
||||
record's ordinal is its zero-based position; a transaction whose URL carried a
|
||||
redacted secret is left unrepresented rather than templated, and one the
|
||||
capture cannot attribute to a single request and response is left
|
||||
unrepresented too. Tools are chosen by response media type, so a scenario that
|
||||
selected neither response bodies nor header values is refused. The browser
|
||||
scenario redacts credential headers and the secrets it declares; values the
|
||||
application returns in response bodies, such as session tokens, are kept, and
|
||||
the replayed body is the bytes the browser exposes for each response, with any
|
||||
declared secret replaced by its redaction marker (a body that redaction had to
|
||||
empty replays no body).
|
||||
|
||||
## What a manifest is
|
||||
|
||||
An environment manifest is a projection of one application's recorded request
|
||||
lines into a tool surface: one tool per method and path template on the
|
||||
primary origin, with inferred path parameters, observed query parameter
|
||||
names, value-free body shapes and pointers to the parent records each tool
|
||||
cites. It carries no bodies and no headers. The manifest's
|
||||
`parent_evidence_id` names the capture Evidence it was projected from; the
|
||||
serve command refuses a manifest whose parent is not the supplied `--evidence`
|
||||
file with an `invalid_request` error naming both identifiers, so a manifest can
|
||||
never be served from records it did not cite.
|
||||
|
||||
The MCP catalog and environment card use a 1 KiB UTF-8 budget for each
|
||||
captured display sample. Observation, body-key and cited-record lists show
|
||||
whole values with omission counts; long method, path and query-name displays
|
||||
show a marked prefix. Complete values remain in manifest Evidence. Argument
|
||||
names longer than 64 characters use collision-safe `path_<n>` or `query_<n>` aliases; replay maps
|
||||
those aliases to the complete captured names.
|
||||
|
||||
Discovery is admitted as a complete catalog under a 1 MiB serialization
|
||||
reservation before the listener binds. Reservation charges up to six bytes per
|
||||
UTF-16 code unit for JSON escaping/UTF-8 expansion, plus schema overhead;
|
||||
a catalog may therefore be refused even when its compact JSON is smaller.
|
||||
This was calibrated with 1500 query fields and eight concurrent SDK discovery
|
||||
requests (183,798-byte card, 507.27 MiB sampled process-family RSS). Over-capacity
|
||||
catalogs return `resource_constraint`, retain complete input Evidence, and require
|
||||
a smaller capture or manifest. Fields and tools are never silently omitted.
|
||||
|
||||
## What replay means and does not mean
|
||||
|
||||
Each MCP tool call selects one of the records its manifest tool cites and
|
||||
returns that record's recorded response. Selection compares the supplied path
|
||||
parameters against the record's path segments at the parameter indexes, then
|
||||
the supplied query values against the record's query string, and keeps the
|
||||
record with the most matches; ties keep manifest order. The result reports
|
||||
where the answer came from:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": 200,
|
||||
"content_type": "application/json",
|
||||
"body": { "id": 2, "login": "probe" },
|
||||
"replay": {
|
||||
"record_ordinal": 5,
|
||||
"evidence_id": "ev_…",
|
||||
"exact_match": true,
|
||||
"recorded_path": "/api/users/2",
|
||||
"recorded_query": null,
|
||||
"recorded_line_truncated": false,
|
||||
"body_recorded": true,
|
||||
"body_encoding": "utf-8",
|
||||
"truncated": false
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`exact_match` is true when every supplied path parameter and query value
|
||||
equals the recorded one; it says nothing about parameters the caller omitted.
|
||||
Recorded query parsing runs only when supplied query arguments affect selection.
|
||||
A path value matches a recorded segment as written or after percent-decoding;
|
||||
when both kinds of match are available, the record whose segment equals the
|
||||
value as written is chosen. `recorded_query` keeps the `?` of an explicit empty
|
||||
query, and a recorded path or query longer than 1 KiB is cut in the reply with
|
||||
`recorded_line_truncated: true`; selection always uses the full recorded line.
|
||||
`content_type` also has a 1 KiB display bound with `content_type_truncated`; body
|
||||
interpretation uses the full recorded media type.
|
||||
When no record matches, the tool still answers from the tool's first cited
|
||||
record and reports `exact_match: false`. `body` is parsed JSON when the
|
||||
recorded content type is JSON-like (`application/json`, `application/*+json`,
|
||||
`text/plain`, `text/markdown`) and the text parses with safe numeric conversion; otherwise
|
||||
it is the recorded text. Large integer IDs and other numbers that would lose
|
||||
information stay in the original JSON text. Bodies over 256 KiB are cut at a UTF-8 boundary and returned as
|
||||
text with `truncated: true`. `body_recorded` is false when the capture kept the
|
||||
response's status and content type but no body; `body` is then `null`, which
|
||||
is not a recorded JSON `null`. `body_encoding` is `base64` when the capture
|
||||
retained response bytes that are not valid UTF-8; `body` is then exactly those
|
||||
bytes in base64, never a lossy decoding. A retained base64 body is decoded only as far as the 256 KiB a reply can carry, and a leading byte-order mark is kept. For a `base64` body the limit applies to the recorded bytes, not their longer base64 spelling, and a cut keeps whole base64 groups. Request bodies passed to `POST`, `PUT` and `PATCH`
|
||||
tools are accepted and ignored: nothing is written, and no call changes what a
|
||||
later call returns. Arguments the tool's input schema does not name are
|
||||
refused by the MCP server as invalid arguments, so a misspelled parameter never
|
||||
degrades into an omitted one.
|
||||
|
||||
Replay is not emulation. A served environment does not establish the
|
||||
application's authorization model, response semantics or state transitions,
|
||||
which the manifest lists as unknowns; it does not validate supplied values; and
|
||||
it does not answer requests the capture never recorded. Replayed content is
|
||||
target content: every tool result, including the argument errors the MCP server
|
||||
returns before a replay runs, carries `_meta.provenance.untrusted_content: true`
|
||||
with `source: "analysis_target"`. The server advertises REA's untrusted-content
|
||||
instructions plus a warning that its tool names, descriptions, parameter names
|
||||
and observed values come from the capture too, and agents must treat all of it
|
||||
as data, never as instructions.
|
||||
That same warning prefixes every tool description in the standalone card, so
|
||||
card discovery carries it before an MCP client initializes. It labels the
|
||||
captured text without changing the tool names or schema fields clients use.
|
||||
|
||||
## The state a capture observed
|
||||
|
||||
Replay answers each call from one recording, so the answers reflect whichever
|
||||
moment of the capture recorded them. The
|
||||
[environment state](web-network-captures.md#environment-state) view extracts
|
||||
instead the data the capture's reads observed before the first related write:
|
||||
entity types with an identity field, entities with their recorded values,
|
||||
documents for written paths, and a render plan per read tool, kept only where
|
||||
it reproduces that tool's recordings from the state. It is the starting state
|
||||
an environment can hold, load and report; serving from it is a later increment
|
||||
and does not change replay.
|
||||
|
||||
## Serve a manifest
|
||||
|
||||
```sh
|
||||
rea environment-serve manifest.json --evidence capture.json [--host 127.0.0.1] [--port 0]
|
||||
```
|
||||
|
||||
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the
|
||||
environment, then to `127.0.0.1` and an ephemeral port, exactly as
|
||||
[`rea mcp --http`](cli.md#serve-mcp-over-streamable-http) resolves them. The
|
||||
bound MCP URL is written to stderr as one JSON log line
|
||||
(`"msg":"MCP transport listening"` with `url`); `REA_LOG_LEVEL` overrides that
|
||||
log level. `ENVIRONMENT_NAME` renames the served environment; otherwise the
|
||||
manifest's `name` is used, and a set but blank value is refused as
|
||||
`configuration_invalid` before binding rather than silently falling back. The
|
||||
listener serves one MCP client at a time with the same session, `Host` header
|
||||
and shutdown contract as [transports](mcp-contracts.md#transports) describes.
|
||||
It runs until SIGINT or SIGTERM, then closes the listener, prints one
|
||||
`{"status":"stopped", …}` document and exits `130` or `143`.
|
||||
Signals during input loading or DNS/bind cancel the owning reader/listener before
|
||||
a listening announcement;
|
||||
that path returns a structured `cancelled` result instead of a stopped-listener
|
||||
document, with the same signal exit status.
|
||||
|
||||
Three routes are served:
|
||||
|
||||
| Route | Answer |
|
||||
| --------------------------------- | ---------------------------------------------------------------------- |
|
||||
| `POST /mcp` | The MCP server: one tool per manifest tool, replaying retained records |
|
||||
| `GET /.well-known/agent-env.json` | The environment card |
|
||||
| `POST /agentenv` | JSON-RPC `-32601 method not found: <method>` for every method |
|
||||
|
||||
Numeric JSON-RPC IDs outside JavaScript's safe range or whose precision would
|
||||
be lost are refused as invalid requests with a null ID. They are never echoed
|
||||
as a different number.
|
||||
|
||||
The data plane at `/agentenv` is advertised by the card but carries no
|
||||
operations in this release; the error names the requested method so a caller
|
||||
can see that the method, not the transport, is unsupported. Its request body is
|
||||
read within the same 4 MiB bound the MCP endpoint applies: a larger body is
|
||||
answered `413` with a JSON-RPC `-32000` error before any of it is parsed, and a
|
||||
body that is not JSON is answered `400` with `-32700`. A notification (no `id`)
|
||||
gets `202` and no body, a batch gets one reply per request in it, and an empty
|
||||
batch, a batch of more than 100 messages, or a message that is not a JSON-RPC
|
||||
request (including an `id` that is not a string, number or null, or `params`
|
||||
that are not an object or array) gets `-32600 invalid request`.
|
||||
|
||||
## The environment card
|
||||
|
||||
The card is the discovery document the agentenv framework reads before
|
||||
connecting to `/mcp`:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "grafana-example",
|
||||
"protocolVersion": "1.0",
|
||||
"url": "/agentenv",
|
||||
"preferredTransport": "JSONRPC",
|
||||
"additionalInterfaces": [{ "url": "/mcp", "transport": "mcp" }],
|
||||
"capabilities": {
|
||||
"extensions": null,
|
||||
"tools": [
|
||||
{
|
||||
"name": "get_users_by_id",
|
||||
"description": "GET /api/users/{numeric}; replays retained records [5, 6]; evidence ev_…",
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"description": "Path segment 2 (numeric); observed: 2, 3"
|
||||
}
|
||||
},
|
||||
"required": ["id"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
}
|
||||
],
|
||||
"operations": []
|
||||
},
|
||||
"children_environments": null
|
||||
}
|
||||
```
|
||||
|
||||
`capabilities.tools` lists the same names, descriptions and input schemas that
|
||||
`tools/list` advertises over MCP. Path parameters are required strings whose
|
||||
description names the segment index, the templating rule and the observed
|
||||
values (sampled within the list budget above; the manifest keeps them all);
|
||||
query parameter names are optional strings; write methods accept an
|
||||
optional `body` object described by the recorded top-level key types. Each
|
||||
input keeps its own argument: a query name equal to a path parameter's name is
|
||||
advertised as `query_<name>`, a query name longer than 64 characters as
|
||||
`query_<n>` (its position) with the captured name sampled in its description,
|
||||
and the write body as `request_body` when a query parameter is named `body`. Every
|
||||
schema closes with `additionalProperties: false`, a key the echo example's
|
||||
schema does not carry.
|
||||
`extensions`, `operations` and `children_environments` are fixed to `null`,
|
||||
`[]` and `null`: a replay environment declares no extensions, no data-plane
|
||||
operations and no children.
|
||||
|
||||
## Export a bundle
|
||||
|
||||
`rea environment-export` writes the same pair as a bundle folder the agentenv
|
||||
framework runs directly, shaped like its own echo example:
|
||||
|
||||
```sh
|
||||
rea environment-export manifest.json --evidence capture.json ./grafana-example --json
|
||||
agent-env run ./grafana-example
|
||||
```
|
||||
|
||||
```text
|
||||
grafana-example/
|
||||
README.md provenance, replay limits, run command, rights note
|
||||
envs/grafana-example/Dockerfile pinned image: node:24.18.0-bookworm-slim + rea-agents@<this version>
|
||||
envs/grafana-example/env.toml environment_name = "grafana-example"
|
||||
envs/grafana-example/manifest.json byte-exact copy of manifest.json
|
||||
envs/grafana-example/evidence.json byte-exact copy of capture.json
|
||||
artifacts/check_grafana-example/verify.py the replay verifier
|
||||
tasks/smoke.json deploy_env (local sandbox), then env_outcome_verifier
|
||||
```
|
||||
|
||||
The command reads each file once, after checking that reading and parsing the
|
||||
whole file, including sealing traversal, fits the process's live heap headroom (a file that does not is
|
||||
refused as a `resource_constraint` before it is read), then authenticates and cross-checks the
|
||||
pair as `environment-serve` does, so a manifest projected from another capture
|
||||
is refused with the same `invalid_request` before anything is written, and the
|
||||
bytes it copies are the bytes it authenticated. The bundle directory must be
|
||||
absent or empty; an occupied directory is refused and nothing in it changes.
|
||||
The complete tool catalog must also fit the server's advertised catalog budget
|
||||
before any files are written. Privately parsed JSON is sealed once so authentication
|
||||
and replay share immutable response trees rather than copying the capture repeatedly.
|
||||
`--name` replaces the manifest's portable kebab-case name as the env folder and env id
|
||||
(Windows device names such as `con` and `lpt1` are refused);
|
||||
`--package-tarball <path>` copies a packed `rea-agents` tarball into the env
|
||||
folder and makes the Dockerfile install it instead of the registry package and
|
||||
remove it in the same layer, which is how a checkout rather than a release is
|
||||
run. The command prints the
|
||||
written files with their sizes and the pinned package source.
|
||||
|
||||
Export uses the shared output-tree owner to check directory identities around
|
||||
exclusive creates, verify written bytes, and reject unowned entries before
|
||||
reporting success. It does not delete an inspectable partial folder on failure.
|
||||
Node's portable pathname APIs leave a syscall-boundary race, so the output's
|
||||
parent directories must be trusted. On POSIX output files use mode `0600`;
|
||||
Windows inherits ACLs and REA does not establish owner-only access there. Choose
|
||||
an already private Windows destination when captured content needs it.
|
||||
|
||||
The Dockerfile sets `MCP_HOST=0.0.0.0` and `MCP_PORT=18765`, the port the
|
||||
framework maps for an MCP server env; the framework's own deploy supplies
|
||||
`MCP_HOST`, `MCP_PORT` and `ENVIRONMENT_NAME`, so the served card carries the
|
||||
env's name. It installs Debian's `python3` because the framework's local
|
||||
sandbox probes a container's readiness with `python3` inside it, which the slim
|
||||
Node image lacks. It copies the two inputs owned by the image's non-root `node`
|
||||
user, so a private host umask cannot leave them unreadable, runs as that user and starts
|
||||
`rea environment-serve /env/manifest.json --evidence /env/evidence.json`. The
|
||||
`env.toml` names the environment because the framework otherwise reads the
|
||||
name from Python sources the image does not have.
|
||||
|
||||
`verify.py` is the framework's `async def verify(mcp_url)` shape over its
|
||||
Python MCP client: it lists the tools and checks the count equals the
|
||||
manifest's, calls the first `GET` tool without path parameters with no
|
||||
arguments and checks the call did not fail and its replayed `status` equals the
|
||||
recorded status of that tool's first cited record (an unknown recorded status
|
||||
stays unknown; it is not borrowed from another record), and returns one
|
||||
`replay` result whose description
|
||||
reports both checks. A manifest with no such tool yields a verifier that checks
|
||||
the count only and says so. `tasks/smoke.json` deploys the env on the local
|
||||
sandbox and runs that verifier against the env's MCP endpoint; the first run
|
||||
also builds the framework's own infrastructure, and the run tears everything
|
||||
down when it ends.
|
||||
|
||||
The README written into the bundle records the capture artifact's SHA-256, the
|
||||
capture and manifest Evidence ids, the manifest view digest, the tool count and
|
||||
the package source, and states that the recorded content belongs to the
|
||||
captured application and must not be published without the rights to it.
|
||||
|
||||
## Compatibility verification
|
||||
|
||||
`npm run verify:agentenv:environment` is the hosted `real-agentenv` lane
|
||||
described in [testing](testing.md#exported-environment-bundles). It projects
|
||||
the Grafana-shaped source-owned capture through the built CLI, packs the
|
||||
checkout, exports a bundle with `--package-tarball`, runs it with the pinned
|
||||
`agentenv-framework` release on the runner's Docker and requires the smoke task
|
||||
to pass with no container left behind other than the framework's own
|
||||
image-registry helper, which the receipt reports. It proves that one exported bundle runs
|
||||
on one framework release; it does not claim any hosted platform or other
|
||||
framework versions.
|
||||
|
||||
## Resource and lifecycle boundaries
|
||||
|
||||
Both Evidence files are read once at startup through the bounded JSON input
|
||||
reader and authenticated as Evidence before the listener binds; a file that is
|
||||
not Evidence, a manifest of another view kind, a capture of another format, or
|
||||
a manifest projected from a different capture is refused before any port is
|
||||
opened. Because authentication covers the capture's records, a manifest whose
|
||||
parent check passes cites only records the capture retains. The server keeps
|
||||
only the cited records' request lines and response text in memory. No outbound
|
||||
network request is made, no process is launched and nothing is written to
|
||||
disk. A listener that fails after binding is logged and ends the command with
|
||||
a `resource_constraint` error and exit `1` instead of serving nothing.
|
||||
@@ -363,6 +363,14 @@ export REA_ANDROID_TEST_APK="$PWD/_reference/apk-integration/ApiDemos-debug.apk"
|
||||
npm run verify:android
|
||||
```
|
||||
|
||||
The main lane and Windows Java-selection checks validate the selected JDK and
|
||||
engine archive through `validateVerifierJadxConfiguration`. This shared verifier
|
||||
boundary supplies the production validator's resource owner and waits for
|
||||
retained cleanup before proceeding. Failed closes retain their exact owners for
|
||||
retry; prerequisite failures must not be mistaken for completed bridge tests.
|
||||
Recovery checks match the typed Java or archive failure and its applicable
|
||||
remediation.
|
||||
|
||||
The lane also uses the selected full JDK's `jlink` to create a disposable JRE
|
||||
without the compiler module. It checks real Java startup failures, missing
|
||||
selectors, recovery after failures, and CLI cancellation with owned cleanup.
|
||||
|
||||
@@ -8,7 +8,7 @@ installs Apktool or a Java runtime.
|
||||
|
||||
## Supply tools
|
||||
|
||||
Use Apktool 2.7.0 or newer (verified with 2.7.0) and a Java runtime the
|
||||
Use Apktool 2.7.0 or newer (verified with 2.7.0 and 3.0.3 on Linux) and a Java runtime the
|
||||
launcher can find. Select the launcher through the environment:
|
||||
|
||||
```sh
|
||||
@@ -63,6 +63,7 @@ rea inspect-apktool-client
|
||||
rea decode-android-resources /targets/Example.apk
|
||||
rea decode-android-resources /targets/Example.apk --locale de
|
||||
rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR
|
||||
rea decode-android-resources /targets/Example.apk --locales de --locales pt-BR --string-tables-only
|
||||
rea decode-android-resources /targets/Example.apk --no-strings
|
||||
```
|
||||
|
||||
@@ -71,8 +72,19 @@ Equivalent MCP methods are `inspect_apktool_client` and
|
||||
primary `locale` (project `res/values-<locale>/strings.xml` instead of the default
|
||||
table), `locales` (repeatable array of additional localized string tables to project
|
||||
from the same single decode pass), and `include_strings` (default true). Regional locales such as `pt-BR`
|
||||
select Android's `values-pt-rBR`; Android qualifiers such as `pt-rBR` and
|
||||
`b+zh+Hans+CN` may also be supplied directly.
|
||||
select Android's `values-pt-rBR`. A three-digit region such as `es-419`
|
||||
selects `values-b+es+419`; `es-r419` is rejected because Android's legacy
|
||||
`-r` form requires a two-letter region. Android qualifiers such as `pt-rBR`
|
||||
and `b+zh+Hans+CN` may also be supplied directly.
|
||||
|
||||
Set `string_tables_only: true` (CLI `--string-tables-only`) to return string
|
||||
entries only inside `string_tables`, including when selecting a single table.
|
||||
The top-level `strings` array is then empty by request; it does not establish
|
||||
that the primary table is empty. Table status, counts and coverage still describe
|
||||
the examined source. This avoids repeating primary entries in multi-locale
|
||||
responses and charges the aggregate output budget for one copy per table.
|
||||
Omitting the option preserves existing responses. `include_strings: false`
|
||||
still suppresses all string projection, including in this mode.
|
||||
|
||||
## Interpret the result
|
||||
|
||||
@@ -108,7 +120,8 @@ select Android's `values-pt-rBR`; Android qualifiers such as `pt-rBR` and
|
||||
each projected table with its locale, status (`present`, `missing`,
|
||||
`oversized`, `malformed`, or `skipped`), entry counts, and coverage,
|
||||
bounded at an aggregate budget of 10,000 strings and 32 MiB XML. When `locales`
|
||||
is omitted, legacy single-table payloads are preserved without adding `string_tables`.
|
||||
is omitted and `string_tables_only` is not true, legacy single-table payloads are preserved
|
||||
without adding `string_tables`.
|
||||
- **Smali is skipped by design** (`apktool d --no-src`): Java-source
|
||||
recovery belongs to the JADX family. Resource smali edits and rebuild are
|
||||
a follow-up lane with mutation authority.
|
||||
@@ -143,3 +156,11 @@ locales — launcher version, on-disk digest agreement, apktool.yml metadata
|
||||
(1.2.3, SDK 24–34), manifest package agreement, default and `de` string
|
||||
tables projected. Parser goldens in the test suite come from the same
|
||||
decode. See `docs/testing.md` for the lane summary.
|
||||
|
||||
Multi-locale and table-only projection were also verified through real CLI and
|
||||
stdio MCP workflows on Linux with Apktool 3.0.3/JDK 21 against NewPipe 0.29.1,
|
||||
ApiDemos 6.0.18 and Appium Settings 8.2.1. Independent XML projection checks,
|
||||
exact facts/Evidence parity and bounded task costs are recorded in the
|
||||
[shared-decode and table-only checkpoint](../experiments/analysis-efficiency/apktool-string-tables.md).
|
||||
This qualifies selected resource facts, not application execution or independent
|
||||
binary resource-decoder completeness.
|
||||
@@ -1,8 +1,8 @@
|
||||
graph TD
|
||||
subgraph "Entry Points"
|
||||
CLI["src/cli.ts + src/cli/<br/>One-shot CLI adapter<br/>setup, diagnostics, analysis, decompilation"]
|
||||
MAIN["src/main.ts + src/main/<br/>MCP stdio server adapter<br/>config, session, process lifetime"]
|
||||
DISPATCH["scripts/rea.mjs<br/>Executable dispatcher<br/>routes mcp / one-shot"]
|
||||
MAIN["src/main.ts + src/main/<br/>MCP server adapter: stdio default, streamable HTTP opt-in<br/>config, session, process lifetime"]
|
||||
DISPATCH["scripts/rea.mjs<br/>Executable dispatcher<br/>routes mcp / mcp --http / one-shot"]
|
||||
end
|
||||
|
||||
subgraph "Production Composition"
|
||||
@@ -10,7 +10,7 @@ graph TD
|
||||
end
|
||||
|
||||
subgraph "Adapters"
|
||||
SERVER["src/server/<br/>MCP request translation<br/>createServer, tool registration, toolResult"]
|
||||
SERVER["src/server/<br/>MCP request translation<br/>createServer, tool registration, toolResult<br/>http/: streamable HTTP listener and node:http bridge"]
|
||||
HOPPER_BRIDGE["bridge/hopper_bridge.py<br/>Runs inside Hopper<br/>adapts operations to Hopper Python API"]
|
||||
GHIDRA_BRIDGE["bridge/ghidra/ReaGhidraBridge.java<br/>Packaged HeadlessScript<br/>inventory, decompiler, references, CFG"]
|
||||
INSTALL["src/cli/ + src/application/Setup*<br/>Host installation adapters<br/>macOS: verified vendor DMG to ~/Applications<br/>Linux: verified vendor package + apt/dnf/pacman"]
|
||||
@@ -34,7 +34,7 @@ graph TD
|
||||
GHIDRA["src/ghidra/<br/>Ghidra discovery, profile & function analysis<br/>snapshot, Unix/Windows transport, bounded lifecycle"]
|
||||
IDA["src/ida/<br/>Upstream MCP contract adaptation<br/>live GUI binding or owned headless database"]
|
||||
IDA_MCP["External ida-pro-mcp<br/>GUI proxy or database supervisor<br/>upstream installation and engine lifecycle"]
|
||||
AUX["src/artifacts/ + src/native/<br/>artifact inventory and extraction<br/>format readers and native capabilities"]
|
||||
AUX["src/artifacts/ + src/native/<br/>artifact inventory, bounded directory triage and extraction<br/>format readers and native capabilities"]
|
||||
BROWSER["src/browser/<br/>CDP / Playwright capture<br/>captured scripts and native module resolution<br/>execution/: coverage, listener and source attribution"]
|
||||
JS_CODEC["src/javascript/sourceMaps/<br/>pinned upstream map codec<br/>owned heap-limited Node process, complete point evidence"]
|
||||
INSPECTOR["src/inspector/<br/>passive V8 Inspector discovery and capture<br/>script and target locations"]
|
||||
|
||||
@@ -160,7 +160,12 @@ launch builder. It is a configuration fact, not measured memory use or proof of
|
||||
the effective JVM limit; JVM environment options may affect that limit.
|
||||
When a Windows script wrapper selects the JVM, this value remains unknown.
|
||||
Historical stdout and stderr remain available, but classification uses only the
|
||||
failing operation's `failure_output`. Output draining waits at most one second,
|
||||
failing operation's `failure_output`. Retained stdout and stderr share an 8 MiB
|
||||
budget, but every drained byte is scanned for `java.lang.OutOfMemoryError`:
|
||||
`failure_output.observed_oome` records the latest confirmed token's bounded line per stream from
|
||||
the failing operation, with its byte offset, even after the budget is spent. These observations own OOME detection; retained text that ends at a class-name prefix cannot confirm a complete token. `line_complete` distinguishes a known line end from an incomplete trailing context; `invalid_utf8` marks replacement-decoded context.
|
||||
Incomplete operation output without an observed OOME keeps the original failure code and explains that a memory cause cannot be ruled out. Incompleteness alone does not confirm a memory constraint or justify heap-specific advice. Token context stays within the failing operation's checkpoint; a class-name prefix in a different identifier is not OOME evidence. A trailing bare token requires an observed stream EOF to establish its boundary.
|
||||
Output draining waits at most one second,
|
||||
subject to earlier cancellation or the startup deadline. `output_incomplete`
|
||||
marks a snapshot taken before stream closure; cleanup ownership remains with
|
||||
the session until release succeeds.
|
||||
|
||||
@@ -350,6 +350,40 @@ validation and resource bounds. Omitted script metadata makes its affected
|
||||
section explicitly truncated.
|
||||
Disallowed-origin entries are filtered before retention.
|
||||
|
||||
Selected script-source collection shares a 32 MiB aggregate UTF-8 retention
|
||||
budget and a 20-second deadline across the capture. An oversized source is
|
||||
omitted while later smaller sources may still fit. When the deadline expires,
|
||||
remaining source reads are skipped. Script metadata, per-item omission reasons,
|
||||
and `script_sources` completeness remain available. A single CDP response can
|
||||
still be materialized up to the separate 64 MiB transport limit before source
|
||||
admission; the retention budget does not bound that transient allocation.
|
||||
|
||||
Bundle parsing and semantic inspection run sequentially in an owned child,
|
||||
without executing captured code. `max_heap_mb` defaults to 1024 MiB V8
|
||||
old-space; `analysis_timeout_ms` defaults to 30000 per script, including startup
|
||||
and transfer. A failed or timed-out script becomes an explicit unknown while
|
||||
completed findings and later scripts remain available. `total_analysis_timeout_ms`
|
||||
defaults to 300000 across script analysis; expiry returns a typed timeout with
|
||||
partial Evidence and identifies unstarted scripts. These controls are also
|
||||
available as CLI flags with hyphenated names. Capture and optional source-map
|
||||
fetching have separate budgets.
|
||||
|
||||
One web-bundle capture/analysis is admitted per REA process at a time. Completed
|
||||
script results share an 8 MiB serialized transfer budget, with host heap
|
||||
reservation checked before decoding and validating another result.
|
||||
Exhausted result capacity stops further script launches and returns a typed
|
||||
resource failure with completed facts and explicit unstarted-script unknowns.
|
||||
Worker control frames are limited to 64 KiB. Cleanup must confirm child termination
|
||||
before releasing its private files; failed cleanup retains its exact owner and
|
||||
blocks replacement until retry succeeds. Cancellation retains completed bundle
|
||||
facts, including cancellation during final cleanup.
|
||||
|
||||
The worker heap setting is not a process-family RSS limit. Node enforces V8
|
||||
old-space flags; Bun ignores them. Windows Bun instead uses an owned process
|
||||
commit limit with a 512 MiB runtime reserve; other Bun hosts have no enforced
|
||||
memory limit. These bounds do not establish an aggregate host-memory limit
|
||||
across other REA operations or external processes.
|
||||
|
||||
Resource-tree inventories distinguish a valid empty `resources` array from a
|
||||
missing or malformed collection. Invalid resource entries are omitted with
|
||||
incomplete coverage, while valid resources from that frame and other frames
|
||||
|
||||
@@ -107,6 +107,11 @@ Through MCP, pass the same object as the `capture_browser_scenario` arguments.
|
||||
The final step's DOM and accessibility artifacts retain the page state at that
|
||||
capture time inline. REA still launches and cleans up the browser profile.
|
||||
|
||||
Action `timeout_ms` and `wait_for_timeout.duration_ms` accept integers from 1
|
||||
through 2,147,483,647 milliseconds, matching the underlying timer representation.
|
||||
Larger values receive an input error before browser launch. Omit `timeout_ms`
|
||||
to keep an action pending until completion or request cancellation.
|
||||
|
||||
The click's `timeout_ms` covers the click action. A non-navigating `fetch` or
|
||||
delayed render can finish later, so use `wait_for` to observe a page-specific
|
||||
ready condition before final capture. A page that updates synchronously can
|
||||
|
||||
+148
-3
@@ -30,6 +30,13 @@ jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evi
|
||||
rea inspect-analysis-view app-view.json
|
||||
```
|
||||
|
||||
The request can also name the saved file instead of embedding it:
|
||||
`{"source": {"kind": "evidence-file", "path": "app-evidence.json"}, "view": {"kind": "summary"}}`.
|
||||
The CLI resolves a relative path against the working directory; MCP clients
|
||||
pass an absolute path. REA reads and authenticates the file, then projects it
|
||||
like retained Evidence, so a large file is bounded by the file reader rather
|
||||
than charged as inline input.
|
||||
|
||||
For example, a 334 MB Obsidian application Evidence file yields a summary view
|
||||
of about 10 KB. Replace the view with
|
||||
`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
|
||||
@@ -58,8 +65,10 @@ assembly-wide inventory, so the output can still be large.
|
||||
|
||||
Choose the workflow from the artifact's contents rather than its `.exe` or
|
||||
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
|
||||
unpack .NET single-file hosts or decode IL2CPP metadata. NativeAOT method bodies
|
||||
require native analysis. See the [managed-code guide](managed-code-analysis.md)
|
||||
unpack .NET single-file hosts or decode IL2CPP. For the image, type and method
|
||||
names in a Unity `global-metadata.dat`, use
|
||||
[IL2CPP metadata](il2cpp-metadata.md). NativeAOT method bodies require native
|
||||
analysis. See the [managed-code guide](managed-code-analysis.md)
|
||||
for supported deployment forms, selection semantics, and coverage limits.
|
||||
|
||||
## SQLite snapshots
|
||||
@@ -83,6 +92,34 @@ The shared CLI/MCP workflow reports compiler version, modules, replacements,
|
||||
build settings, original bytes and SHA-256. See [Go binaries](go-binaries.md)
|
||||
for supported containers and absent or malformed metadata.
|
||||
|
||||
Inspect a Godot PCK without running the game:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact game.pck --detail summary --evidence_output evidence.json --json
|
||||
rea extract-artifact game.pck --json
|
||||
```
|
||||
|
||||
Formats 2, 3 and 4 retain header, directory and integrity facts. Compact delivery
|
||||
keeps complete Evidence in the requested bundle. See [Godot PCK](godot-pck.md)
|
||||
for embedded packs, batch queries and unknown encrypted coverage.
|
||||
|
||||
Inspect Unity IL2CPP metadata names without Unity or a disassembler:
|
||||
|
||||
```bash
|
||||
rea inspect-il2cpp-metadata /absolute/path/global-metadata.dat --json
|
||||
```
|
||||
|
||||
The command decodes a version 24, 27, 29, 31 or 39 `global-metadata.dat` and reports assembly
|
||||
images, type, method and member names, string literals, index ranges, header
|
||||
sections and SHA-256. With `--binary PATH` it also reads the matching
|
||||
`GameAssembly.dylib`, `GameAssembly.dll` or `libil2cpp.so` (never runs it) and
|
||||
links each method to its code address; `--ghidra-seeds PATH` then writes Ghidra
|
||||
function seeds. It does not decrypt metadata. MCP defaults to compact counts
|
||||
with retained Evidence; query indexed rows with filters, batch indices and
|
||||
selected fields. The CLI defaults to complete Evidence;
|
||||
`--detail summary --evidence_output FILE` exports the full result to a new file
|
||||
and returns a compact overview. See [IL2CPP metadata](il2cpp-metadata.md).
|
||||
|
||||
Inspect an embedded cargo-auditable Rust dependency list without installing
|
||||
Rust or configuring a disassembler:
|
||||
|
||||
@@ -286,6 +323,12 @@ reported separately as configured-secret exclusions.
|
||||
Re-import saved graphs whose pattern-based exclusions lack `pattern` before
|
||||
using them in source-to-bundle comparisons.
|
||||
|
||||
Entries that change during an import retain an unavailable-entry diagnostic
|
||||
alongside readable siblings. If enumeration of an already admitted root fails,
|
||||
the graph keeps its observed children and records the root failure as a
|
||||
top-level limitation. That partial inventory does not establish absence among
|
||||
unexamined children.
|
||||
|
||||
JavaScript and TypeScript import parsing requires valid UTF-8. Malformed source
|
||||
bytes retain their original hashes and sizes with a decoding diagnostic; REA
|
||||
does not infer module targets from replacement characters.
|
||||
@@ -310,6 +353,80 @@ Choose the guide for the target and intended interaction:
|
||||
Runtime requests name the target and actions. Launched targets run with your
|
||||
user permissions; consult the chosen guide for host requirements and effects.
|
||||
|
||||
## Serve MCP over streamable HTTP
|
||||
|
||||
`rea mcp` speaks stdio to the client that launched it. To serve the same MCP
|
||||
catalog over streamable HTTP for one local client, such as an agent environment
|
||||
runtime, start the dispatcher entry with `--http`:
|
||||
|
||||
```sh
|
||||
rea mcp --http --port 8765
|
||||
```
|
||||
|
||||
`--host` and `--port` default to `MCP_HOST` and `MCP_PORT` from the environment,
|
||||
then to `127.0.0.1` and an ephemeral port. The bound URL is written to stderr as
|
||||
one JSON log line (`"msg":"MCP transport listening"`) whatever `REA_LOG_LEVEL`
|
||||
is set to; nothing is written to stdout. `rea mcp --http --help` lists the accepted options. The listener serves
|
||||
one client at a time: a session ends with the client's `DELETE`, and a client
|
||||
that connects while another session is open takes it over, so a client that
|
||||
exited without ending its session needs no restart. The listener runs until
|
||||
SIGINT or SIGTERM and does not stop when stdin closes. See
|
||||
[transports](mcp-contracts.md#transports) for the session, `Host` header and
|
||||
shutdown contract. Client registrations written by setup and the MCP registry
|
||||
entry keep stdio.
|
||||
|
||||
## Serve an environment manifest
|
||||
|
||||
`rea environment-serve <manifest.json> --evidence <capture.json> [--host H]
|
||||
[--port N]` serves an environment manifest projected by `inspect-analysis-view`
|
||||
as an agent environment: an MCP server at `/mcp` with one tool per manifest
|
||||
tool, each replaying the retained capture records it cites, the environment
|
||||
card at `/.well-known/agent-env.json`, and a `/agentenv` data plane that
|
||||
answers every method with JSON-RPC `-32601`. The manifest's
|
||||
`parent_evidence_id` must equal the evidence file's `evidence_id`; otherwise
|
||||
the command refuses with `invalid_request` before binding.
|
||||
|
||||
```sh
|
||||
rea environment-serve manifest.json --evidence capture.json --port 8765
|
||||
```
|
||||
|
||||
`--host` and `--port` resolve like `rea mcp --http`: flags, then `MCP_HOST` and
|
||||
`MCP_PORT`, then `127.0.0.1` and an ephemeral port. `ENVIRONMENT_NAME` renames
|
||||
the served environment; set but blank, it is refused as `configuration_invalid`
|
||||
like a blank `MCP_HOST`. The bound URL is written to stderr as one JSON log line
|
||||
(`"msg":"MCP transport listening"`). See
|
||||
[agent environments](agent-environments.md) for what replay returns and does
|
||||
not establish.
|
||||
|
||||
## Export an environment bundle
|
||||
|
||||
`rea environment-export <manifest.json> --evidence <capture.json> <dir>
|
||||
[--package-tarball <path>] [--name <override>]` writes the same manifest and
|
||||
capture pair as a bundle folder the agentenv framework runs with
|
||||
`agent-env run <dir>`. It reads each file once, authenticates and cross-checks
|
||||
the pair with the same checks as `environment-serve`, copies the bytes it
|
||||
authenticated, refuses to write into a directory that exists and is not empty,
|
||||
and never overwrites a file.
|
||||
|
||||
```sh
|
||||
rea environment-export manifest.json --evidence capture.json ./grafana-example --json
|
||||
```
|
||||
|
||||
The folder holds a `README.md` with the bundle's provenance, an env folder
|
||||
`envs/<name>/` with a `Dockerfile` that installs this exact `rea-agents`
|
||||
version and serves the copied `manifest.json` and `evidence.json` (written
|
||||
with mode `0600` on POSIX; Windows retains inherited ACLs) with
|
||||
`environment-serve`, a Python verifier under `artifacts/check_<name>/` and the
|
||||
smoke task `tasks/smoke.json`. `--package-tarball` copies a packed `rea-agents`
|
||||
tarball into the env folder and installs it instead of the registry package,
|
||||
which is how the hosted lane proves a checkout rather than a release. `--name`
|
||||
replaces the manifest's kebab-case name as the env folder and env id; a device
|
||||
name Windows reserves (`con`, `nul`, `com1` and the like) is refused. The
|
||||
command prints one JSON document listing the written files with their sizes
|
||||
and the pinned package source. See
|
||||
[agent environments](agent-environments.md#export-a-bundle) for the folder
|
||||
layout and what the smoke task checks.
|
||||
|
||||
## Output and exit status
|
||||
|
||||
`rea mcp doctor --json` checks production MCP startup and reports child logs in
|
||||
@@ -328,6 +445,11 @@ redirected, which keeps large results smaller for agents and scripts; pipe
|
||||
through `jq .` for indented files. Output selection and formatting do not change
|
||||
operation status.
|
||||
|
||||
`--filter-output` selects fields from successful command data. Classified failures
|
||||
retain their complete typed diagnostic projection even when the selector matches
|
||||
a field such as `code` or `message`, or names a missing field. `--full-output`
|
||||
controls whether the outer `{ok, error, meta}` envelope is included.
|
||||
|
||||
For `cutter-command`, `--json-output` requests command-specific JSON from
|
||||
Cutter when available. This is separate from `--format json`, which selects the
|
||||
CLI result representation; the global `--json` flag selects that same CLI
|
||||
@@ -344,6 +466,28 @@ output formatting.
|
||||
| `1` | The operation could not complete. Structured output identifies invalid input, permissions, cancellation, timeouts or another failure when available. |
|
||||
| `128 + N` | Signal-derived status. An unhandled signal may have ended the process, with the shell or runtime preserving the conventional status; a command may instead have handled the signal, waited for its operation and cleanup to settle, and assigned the same status (`130` for SIGINT, `143` for SIGTERM). In the handled case, it may also have printed a structured `cancelled` result. |
|
||||
|
||||
`rea mcp --http` is long-running and prints no structured result. It exits
|
||||
`130` on SIGINT or `143` on SIGTERM after closing the listener, the session and
|
||||
its providers, and `1` when that cleanup fails.
|
||||
|
||||
`rea environment-serve` is also long-running: it serves until SIGINT or
|
||||
SIGTERM, closes the listener, then prints one structured
|
||||
`{"status":"stopped", …}` document naming the URL, environment and tool count
|
||||
and exits `130` or `143` through the shared cancellation handler below.
|
||||
Refusals before binding (unreadable files, non-Evidence JSON, capture Evidence
|
||||
in a format other than HAR, a manifest whose parent is not the supplied
|
||||
evidence, an unbindable port) print a structured
|
||||
error and exit `1`, as does a listener that fails after binding.
|
||||
`rea environment-export` is one-shot: it prints the written files and exits
|
||||
`0`, or prints a structured `invalid_request` error and exits `1` for the same
|
||||
pair refusals, a non-empty or file-occupied directory, a missing package
|
||||
tarball or a name that is not a kebab-case label or is a Windows device name,
|
||||
writing nothing in each case.
|
||||
A write that fails after the first file (a permission or disk-space failure
|
||||
on a later path) propagates as the host's failure and leaves the partial
|
||||
folder in place for inspection; remove it before retrying, since the next
|
||||
attempt refuses the non-empty directory.
|
||||
|
||||
A `cancelled` result does not by itself determine the exit status; it depends on
|
||||
the command's signal handling. Commands routed through the shared
|
||||
`withCommandCancellation` handler in the CLI set status `130` for SIGINT or
|
||||
@@ -356,7 +500,8 @@ browser, web and Electron commands (including `analyze-web-bundle`,
|
||||
`capture-electron-scenario`), and the platform and provider command families
|
||||
for Android/ADB, Apktool/JEB/Flutter, firmware, PCAP and signal analysis,
|
||||
WASM/EVM, Go/Rust and binary diagnostics, Frida instrumentation, the Cutter
|
||||
command, objdump, Rizin and GDB. In these paths, a returned structured
|
||||
command, objdump, Rizin and GDB, and the agent environment server
|
||||
(`environment-serve`). In these paths, a returned structured
|
||||
`cancelled` result can be printed before the wrapper assigns the
|
||||
signal-derived status. This describes the current source routing; it is not a
|
||||
claim that every command or provider runtime has been independently exercised.
|
||||
|
||||
@@ -457,7 +457,9 @@ disposable HOME and CODEX_HOME under `~/.cache/rea-agent-evaluations/`. Set
|
||||
`REA_AGENT_EVAL_ROOT` to choose another parent directory outside the OS temporary
|
||||
directory: current Codex refuses helper aliases under `/tmp`, preventing shell
|
||||
commands and skill reads even if MCP works. Each owned run directory is removed
|
||||
after verification unless fixture retention is requested. The lane
|
||||
after verification unless fixture retention is requested. Cleanup attempts the
|
||||
remaining run tree even if auth removal fails, and reports unresolved cleanup
|
||||
together with the original verification failure. The lane
|
||||
installs the packaged skill and MCP registration through public `rea setup`,
|
||||
rather than copying a skill into a project that also inherits the caller's
|
||||
skills and configuration. Only authentication is copied, when present, and it
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
# Desktop installation directories
|
||||
|
||||
Use `inspect_desktop_application` when an installation directory is the target
|
||||
and its runtime is unknown. It reads directory entries, selected headers and
|
||||
bounded metadata, then reports runtime candidates, supporting files, component
|
||||
path markers and concrete follow-up operations. No native provider or active
|
||||
binary session is needed. It does not run the application, extract archives,
|
||||
follow child symlinks, contact services or change target files.
|
||||
|
||||
## MCP
|
||||
|
||||
```json
|
||||
{ "path": "/artifacts/MyApplication" }
|
||||
```
|
||||
|
||||
Default `detail: "summary"` retains complete source Evidence in the session and
|
||||
returns candidates with representative supporting paths, counts, coverage,
|
||||
limitations and grouped next probes. `normalized_result.parent_evidence_id`
|
||||
identifies the complete record. A server without Evidence retention refuses
|
||||
summary delivery before reading the target; select `detail: "complete"` to
|
||||
receive every retained fact inline.
|
||||
|
||||
Read exact files in one request:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_<parent-sha256>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "desktop-application",
|
||||
"collection": "files",
|
||||
"paths": ["Game_Data/Managed/Assembly-CSharp.dll", "GameAssembly.dll"],
|
||||
"fields": ["path", "read_status", "ranges", "facts", "issues"],
|
||||
"offset": 0,
|
||||
"limit": 16
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Collections are `files`, `omissions`, `candidates`, `components` and `next_probes`.
|
||||
Files and omissions accept exact `paths` and batched `file_ids` / `omission_ids`.
|
||||
Files also accept `has_issues: true` to select probe diagnostics directly, or
|
||||
`false` to select rows without reported issues. This filter uses the retained
|
||||
issue facts; it does not re-read targets or infer that uninspected files are healthy.
|
||||
When both are supplied, all filters apply; an empty selection returns no rows.
|
||||
Omission records retain the observed path, kind and stable `omission_id`; their
|
||||
payloads are not read, and other special-entry metadata remains unavailable.
|
||||
`candidates` accepts exact `families`.
|
||||
Omitted selectors match all retained rows; an empty selector matches none.
|
||||
Every collection supports `fields`, `offset` and `limit`; its stable row ID is
|
||||
always included. Selection changes delivery, not source inspection. Pages keep
|
||||
source coverage and parent Evidence links. Page exhaustion does not establish
|
||||
absence in a partial source. Complete Evidence remains accessible through
|
||||
`get_evidence` and Evidence bundle export.
|
||||
|
||||
The summary's `probe_issues` counts files with reported issues and their retained
|
||||
issue entries. It previews at most two such files, including stable `file_id`,
|
||||
read status and the first issue. Path/reason previews use at most 256 UTF-16 units
|
||||
each without splitting surrogate pairs; `text_truncated` marks shortened text,
|
||||
and `preview_truncated` marks additional files. These are display prefixes;
|
||||
full caller paths and all reasons remain in Evidence. Counts describe retained
|
||||
observations, while source coverage still identifies partial inspection.
|
||||
|
||||
Use `inspect_analysis_view` with `collection: "files", has_issues: true` and
|
||||
`fields: ["path", "read_status", "issues"]` for exact reasons, with the usual
|
||||
paging and parent links. It intersects with exact paths and batched file IDs.
|
||||
|
||||
An ASAR inventory probe first selects its exact path with `open_binary`; then
|
||||
call `inspect_artifact` on that active target. Directory inspection itself does
|
||||
not select or replace the session target. The JavaScript application workflow
|
||||
accepts its own selected path directly.
|
||||
|
||||
## CLI
|
||||
|
||||
The one-shot CLI returns complete Evidence by default. For compact output,
|
||||
explicitly retain it in a bundle:
|
||||
|
||||
```bash
|
||||
rea inspect-desktop-application /artifacts/MyApplication \
|
||||
--detail summary --evidence_output /reports/application.evidence.json --json
|
||||
```
|
||||
|
||||
This returns `summary` Evidence and an `evidence_bundle` publication receipt.
|
||||
Existing output files are never overwritten. Summary delivery requires
|
||||
`--evidence_output`, checked before analysis. `inspect-evidence-bundle` selects
|
||||
saved records; an exact saved record can also feed `inspect-analysis-view`.
|
||||
`--detail complete` returns source Evidence inline and can optionally write the
|
||||
same bundle. Export writes only the caller-selected output file.
|
||||
|
||||
## Facts and uncertainty
|
||||
|
||||
| Family | Evidence used |
|
||||
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| .NET | A PE optional header declares a nonempty CLI directory. |
|
||||
| Unity Mono | A managed assembly below `*_Data/Managed` or `Contents/Resources/Data/Managed` has an exact UnityEngine assembly reference. |
|
||||
| Unity IL2CPP | An IL2CPP metadata header coexists with a conventionally named native `GameAssembly.dll`, `GameAssembly.so`, `GameAssembly.dylib` or `libil2cpp.so` image. Build correspondence remains unknown. |
|
||||
| Godot | A `GDPC` header at file start or a validated executable trailer offset. |
|
||||
| GameMaker | A bounded `FORM` container starts with a `GEN8` chunk. |
|
||||
| Unreal | `Content/Paks` path conventions or an IoStore TOC signature; arbitrary `.pak` filenames are insufficient. |
|
||||
| Electron | A `resources/app.asar` or `resources/default_app.asar` path convention; archive validity is unverified. |
|
||||
|
||||
An observed metadata header with supported raw version 29, 31 or 39 offers an
|
||||
`inspect_il2cpp_metadata` follow-up at its admitted file path, including when no
|
||||
native library was found. This probe does not establish full metadata validity
|
||||
or correspondence to a native binary. See [IL2CPP metadata](il2cpp-metadata.md).
|
||||
|
||||
Candidates are derived or inferred from these facts and do not prove loading or
|
||||
execution. BepInEx, MelonLoader, Easy Anti-Cheat and BattlEye labels identify
|
||||
path conventions only; runtime presence stays unknown. No bypass is provided.
|
||||
|
||||
`file_id` includes the admitted root namespace, recorded path and available filesystem identity. The caller-selected root path and admitted canonical path/root identity are both retained; `local_path` identifies each observed file for follow-up operations. The selected root may be a symlink; child symlinks are not followed. Exact
|
||||
metadata integers remain decimal strings. Read ranges have offsets, byte counts
|
||||
and SHA-256 digests; typed facts expose CLI RVAs, metadata/pack versions and
|
||||
assembly tokens. Directory and symlink identities unavailable from traversal
|
||||
remain null.
|
||||
|
||||
The directory's `snapshot_sha256` commits to retained paths, omission kinds,
|
||||
available metadata and range digests, composed through the admitted root,
|
||||
entry and omission identities. Caller ceilings, coverage counters, diagnostic
|
||||
wording and interpretation facts remain in Evidence, outside source identity.
|
||||
The same observations under different generous limits share a snapshot digest;
|
||||
Evidence IDs still preserve request parameters and reported coverage. Different
|
||||
retained entries or read ranges change the source snapshot.
|
||||
Omission records share the entry and representation budgets and can be fetched
|
||||
independently through their view.
|
||||
**It is not a digest of all file contents.** These are sequential stable entry
|
||||
observations, not an atomic directory snapshot. Unread bytes remain unknown.
|
||||
|
||||
## Resource bounds and reuse
|
||||
|
||||
Defaults are 20,000 examined entries and 8 MiB of total probe reads. Callers may
|
||||
select up to 100,000 entries and 64 MiB. A separate 32 MiB conservative
|
||||
representation budget bounds retained paths and facts before growth. Limits
|
||||
and failed probes produce partial coverage. Traversal and reads are serial.
|
||||
Each reader admits one inspection at a time; overlapping requests receive a
|
||||
retryable capacity refusal without cancelling the admitted caller.
|
||||
|
||||
Unity reference decoding stops after a witness in each `Data/Managed` directory;
|
||||
other assemblies' references remain unexamined, while their headers are still
|
||||
checked. AssemblyRef facts decode names and tokens only; they do not decode or
|
||||
hash unrelated key/culture/hash payloads or establish runtime binding. Pack contents, encrypted data, metadata/address correspondence and
|
||||
section-embedded PCKs without a trailer are separate capabilities.
|
||||
|
||||
Filesystem results are not cached across calls. Retained views reuse validated
|
||||
immutable metadata without rescanning files or retaining another inventory.
|
||||
@@ -33,6 +33,12 @@ encoded path separators, ASAR directories or links, and nonexistent paths are
|
||||
rejected. The endpoint exposes every eligible target and its local path
|
||||
metadata.
|
||||
|
||||
Cancellation stops further file authorization after an already-started filesystem
|
||||
operation settles. ASAR descriptor-close failures remain typed incomplete cleanup
|
||||
with an exact retryable owner; they are not treated as a missing or unauthorized
|
||||
page. A later authorization and provider shutdown retry retained descriptors
|
||||
before reporting their release.
|
||||
|
||||
## Workflow
|
||||
|
||||
`list_electron_targets` returns every eligible target served by the supplied endpoint in one inline array.
|
||||
@@ -68,7 +74,12 @@ Script metadata retains its execution-context frame ID when CDP supplies one. Th
|
||||
inventories worker, service-worker, and shared-worker targets with
|
||||
validated opener-target and parent-frame IDs. Worker discovery uses passive
|
||||
target metadata; REA does not attach to or execute code in those targets.
|
||||
Collection counts and aggregate script-source bytes are not capped. Like every
|
||||
Script-source collection retains at most 32 MiB of UTF-8 text across scripts
|
||||
and shares a 20-second collection deadline. Omitted sources retain their script
|
||||
metadata and an explicit reason; later smaller sources may still fit after an
|
||||
oversized source is omitted. The separate 64 MiB CDP response limit bounds one
|
||||
wire message before retention admission, rather than total transient memory.
|
||||
The source budget adds no script-count cap. Like every
|
||||
target, frame, script, and resource, a worker URL must resolve to a local
|
||||
file before it is retained. Relationship IDs improve
|
||||
attribution but do not prove which static module started a worker or that its
|
||||
@@ -193,8 +204,25 @@ Caller-defined `.listener` properties can change the identity exposed by
|
||||
wrapper recognition. Re-registering a raw recording wrapper on another channel
|
||||
retains the original registration's channel attribution.
|
||||
Mutating `.listener` on a raw recording wrapper can also prevent removal through
|
||||
application decorators; the reproduced identity conflict is tracked in
|
||||
[#4513](https://github.com/morluto/rea/issues/4513).
|
||||
application decorators that bypass the hook's removal methods. When a removal
|
||||
request reaches the hook's methods and its registration order is known, private
|
||||
callback ownership preserves removal by the original function and the newest
|
||||
matching registration when public metadata is changed or deleted. Native removal
|
||||
still determines getter execution, exceptions and removal meta-events.
|
||||
|
||||
If registration or removal methods differ from the Node methods captured at hook
|
||||
initialization, the hook leaves listener callbacks unwrapped and records a
|
||||
`listener-registration` observation gap. The application keeps its original
|
||||
callback and receiver; callback execution remains unobserved. This comparison
|
||||
assumes Node's EventEmitter prototype has not itself been globally decorated
|
||||
before hook initialization. The provider loads the hook before the application
|
||||
and excludes `NODE_OPTIONS` from the child environment.
|
||||
|
||||
An opaque later listener or unsupported emitter state can prevent proof of
|
||||
removal order. The hook then delegates removal
|
||||
to the native method and records a `listener-removal` gap rather than claiming
|
||||
which registration was removed. These gaps are distinct from dropped records
|
||||
and do not establish IPC delivery or successful removal.
|
||||
|
||||
### Target fuse prerequisites
|
||||
|
||||
|
||||
+587
-17
@@ -28,6 +28,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -64,6 +91,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -100,6 +154,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -136,6 +217,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -172,6 +280,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -208,6 +343,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -244,6 +406,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -280,6 +469,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -316,6 +532,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -352,6 +595,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -388,25 +658,82 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"logical_path": { "type": "string" },
|
||||
"declared_sha256": {
|
||||
"anyOf": [{ "type": "string" }, { "type": "null" }]
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"logical_path": { "type": "string" },
|
||||
"declared_sha256": {
|
||||
"anyOf": [{ "type": "string" }, { "type": "null" }]
|
||||
},
|
||||
"calculated_sha256": {
|
||||
"anyOf": [{ "type": "string" }, { "type": "null" }]
|
||||
},
|
||||
"unpacked": { "type": "boolean" }
|
||||
},
|
||||
"required": [
|
||||
"logical_path",
|
||||
"declared_sha256",
|
||||
"calculated_sha256",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"calculated_sha256": {
|
||||
"anyOf": [{ "type": "string" }, { "type": "null" }]
|
||||
},
|
||||
"unpacked": { "type": "boolean" }
|
||||
},
|
||||
"required": [
|
||||
"logical_path",
|
||||
"declared_sha256",
|
||||
"calculated_sha256",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": false
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"logical_path": { "type": "string" },
|
||||
"algorithm": { "type": "string", "const": "md5" },
|
||||
"declared_md5": {
|
||||
"type": "string",
|
||||
"pattern": "^[a-f0-9]{32}$"
|
||||
},
|
||||
"calculated_md5": {
|
||||
"anyOf": [
|
||||
{ "type": "string", "pattern": "^[a-f0-9]{32}$" },
|
||||
{ "type": "null" }
|
||||
]
|
||||
},
|
||||
"unpacked": { "type": "boolean" }
|
||||
},
|
||||
"required": [
|
||||
"logical_path",
|
||||
"algorithm",
|
||||
"declared_md5",
|
||||
"calculated_md5",
|
||||
"unpacked"
|
||||
],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
@@ -446,6 +773,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -482,6 +836,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -518,6 +899,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -554,6 +962,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -590,6 +1025,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -626,6 +1088,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -662,6 +1151,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -698,6 +1214,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
@@ -734,6 +1277,33 @@
|
||||
"required": ["action"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"recovery": {
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_input_contract" },
|
||||
"operation": { "type": "string", "minLength": 1 }
|
||||
},
|
||||
"required": ["kind", "operation"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"kind": { "type": "string", "const": "inspect_timeout_state" },
|
||||
"operation": { "type": "string", "minLength": 1 },
|
||||
"timeout_ms": {
|
||||
"type": "integer",
|
||||
"exclusiveMinimum": 0,
|
||||
"maximum": 9007199254740991
|
||||
}
|
||||
},
|
||||
"required": ["kind", "operation", "timeout_ms"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
]
|
||||
},
|
||||
"details": {
|
||||
"type": "object",
|
||||
"propertyNames": { "type": "string" },
|
||||
|
||||
@@ -158,7 +158,17 @@ tree semantics, vulnerability scanning, emulation or hardware runtime capture.
|
||||
Supply a stable copy within the allowance. No in-place source modification.
|
||||
- One queued heavy operation per provider instance; Binwalk one thread and
|
||||
Unblob one worker. No automatic parallel extraction engines.
|
||||
- **120-second** deadline; inherited **1 GiB per-process address-space** limit,
|
||||
- **120-second** analysis deadline through inventory. Publication and final
|
||||
cleanup start a separate **120-second** budget; select
|
||||
`publication_timeout_ms` in MCP or `--publication-timeout-ms` in CLI for a slow
|
||||
destination. Valid budgets are integers from 1 through 2,147,483,647 ms (the
|
||||
timer representation limit). Publication timeouts identify their phase and
|
||||
verified file progress; verified writes before atomic commit are not a claim
|
||||
that output remains published after rollback. Caller cancellation applies to
|
||||
both phases and final cleanup. Phase handoff, publication commit and final
|
||||
completion also check monotonic elapsed time, so delayed timer callbacks
|
||||
cannot revive an expired budget.
|
||||
- Inherited **1 GiB per-process address-space** limit,
|
||||
120 CPU seconds and zero core-dump allowance through `prlimit`.
|
||||
- JSON reports: at most **8 MiB** and valid UTF-8; malformed encoding and
|
||||
oversized reports fail without repairing or truncating producer data.
|
||||
|
||||
@@ -23,7 +23,9 @@ rea inspect-dart-aot /targets/Example.apk --abi armeabi-v7a
|
||||
The equivalent MCP methods are `identify_flutter_build` (with `path`) and
|
||||
`inspect_dart_aot` (with `path` and an optional `abi`; the default prefers
|
||||
arm64-v8a, then x86_64, then x86, then armeabi-v7a, then the remaining
|
||||
ABIs alphabetically).
|
||||
ABIs alphabetically). An explicit `abi` is the directory label exactly as
|
||||
`identify_flutter_build` reports it, so a payload under a nonstandard label
|
||||
stays selectable and keeps that label in the result.
|
||||
|
||||
## Optional retained AOT delivery
|
||||
|
||||
|
||||
@@ -8,6 +8,6 @@ MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`,
|
||||
|
||||
Failed startup cleanup retains acquired Frida handles, including a paused spawn that has not attached, for later cleanup retries. Its remote connection stays leased until session detach and target resume complete. CLI commands await provider cleanup before returning; MCP retains these resources for cleanup when its server closes. Server shutdown stops admitting new Frida operations and waits for admitted startup, discovery, and queued work before releasing their resources. A failed cleanup remains reported and can be retried by closing the server again, rather than treating an unregistered session as fully released.
|
||||
|
||||
Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Message capture is bounded to 16 MiB and reports truncation. Script source is not stored in Evidence; its selected path and SHA-256 digest are. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. A cancelled script load on an existing session releases only its new script before queued operations can run; the session and previously loaded scripts remain available. Once the provider commits a successful load, later request cancellation does not unload that script; use `unload_frida_script` to release it. If script cleanup fails, the script remains owned and a paused target cannot resume until the pending instrumentation is released. Queued script loads, resumes, and unloads check cancellation before dispatching native effects. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed.
|
||||
Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Message capture is bounded to 16 MiB and reports truncation. Script source is not stored in Evidence; its selected path and SHA-256 digest are. For a file, `source_sha256` identifies the original selected bytes. Invalid UTF-8 continues to use replacement decoding; Evidence then records that limitation and `raw_result.source_representation` with the encoding, decoding mode, and `loaded_source_sha256` of the UTF-8 string passed to Frida. Valid UTF-8 files and inline scripts do not need this additional representation fact. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. A cancelled script load on an existing session releases only its new script before queued operations can run; the session and previously loaded scripts remain available. If cancellation is observed when the native call returns, partial Evidence retains the source identity and captured messages even if that call rejects, including messages emitted during awaited cleanup. The caller's cancellation reason remains primary and a native rejection is retained as a sanitized diagnostic. A failure already observed before cleanup stays primary if cancellation arrives during cleanup. The cancellation cutoff is REA's observation of native completion, rather than a claim about unobservable native event order. `script_id` is REA's load-attempt identity. When creation rejects without returning a handle, `script_cleanup: "no_handle"` means REA acquired no handle to unload; it does not prove that the engine performed no work. Once the provider commits a successful load, later request cancellation does not unload that script; use `unload_frida_script` to release it. If script cleanup fails, the script remains owned and a paused target cannot resume until the pending instrumentation is released. Queued script loads, resumes, and unloads check cancellation before dispatching native effects. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed.
|
||||
|
||||
The [Frida Core API](https://frida.re/docs/frida-core-api/) documents device management, sessions, and script loading; the [JavaScript API](https://frida.re/docs/javascript-api/) documents target-side instrumentation. [Luma](https://luma.frida.re/) is Frida's standalone GUI; REA does not automate or control it. `npm run verify:frida` requires a Linux host with the optional Frida binding and uses a short-lived local Node.js fixture created by the verifier. Add `-- --address HOST:PORT --pid PID` to run the separate remote attach lane against a caller-selected target. Mock tests do not establish real device/server compatibility.
|
||||
@@ -0,0 +1,108 @@
|
||||
# Godot PCK archives
|
||||
|
||||
`inspect-artifact` and `extract-artifact` read ordinary Godot PCK formats 2, 3
|
||||
and 4 without running the game or requiring Godot. Official Godot 4.4.1, 4.6.1
|
||||
and 4.7.1 produced these respective formats in verification; other engine
|
||||
releases have not been verified. Format 1 is unsupported.
|
||||
|
||||
Inspection retains the original header and directory facts, including engine
|
||||
version, flags, declared file count, source paths, offsets and declared MD5.
|
||||
Observed plaintext bytes receive content-addressed SHA-256 identities. MD5 is a
|
||||
container integrity claim, separate from the observed SHA-256. A mismatch fails
|
||||
by default; `--integrity-policy record-and-continue` retains the observed bytes
|
||||
and an explicit untrusted contradiction. An all-zero MD5 is an unknown claim,
|
||||
not a verified digest.
|
||||
|
||||
## CLI
|
||||
|
||||
Start with a compact summary and keep the complete Evidence bundle:
|
||||
|
||||
```sh
|
||||
rea inspect-artifact /artifacts/game.pck --detail summary \
|
||||
--evidence_output /evidence/game.json --json
|
||||
rea extract-artifact /artifacts/game.pck --json
|
||||
```
|
||||
|
||||
Summary delivery requires `--evidence_output` and never overwrites that file.
|
||||
The response contains `summary` Evidence and an `evidence_bundle` receipt.
|
||||
Without `--detail summary`, inspection keeps its complete inline delivery.
|
||||
Extraction creates a new temporary directory and returns its path. It rejects
|
||||
unsafe paths and verifies container identity before creating output.
|
||||
|
||||
The same CLI commands locate a PCK at the beginning of a file or through its
|
||||
12-byte EOF trailer in an executable carrier. They do not scan arbitrary
|
||||
payloads or discover packs behind additional trailing signatures. The root
|
||||
content SHA-256 covers the whole carrier; its content classification remains
|
||||
native, while the inventory records the PCK role and exact pack bounds.
|
||||
Ordinary native `open_binary` detection remains unchanged.
|
||||
|
||||
## MCP and selected facts
|
||||
|
||||
For a standalone pack, call `open_binary` with its absolute path, then
|
||||
`inspect_artifact` with `{"detail":"summary"}`. The complete inspection and its
|
||||
nested inventory stay in the session. Use the summary's
|
||||
`normalized_result.parent_evidence_id` as the source of subsequent views:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_analysis_view",
|
||||
"arguments": {
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "<parent_evidence_id>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "inventory",
|
||||
"collection": "occurrences",
|
||||
"logical_paths": ["scripts/main.gd", "project.binary"],
|
||||
"fields": [
|
||||
"logical_path",
|
||||
"declared_integrity",
|
||||
"hash_status",
|
||||
"format_metadata"
|
||||
],
|
||||
"offset": 0,
|
||||
"limit": 32
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Occurrences retain `occurrence_id`, `artifact_id` and `parent_occurrence_id`
|
||||
even when `fields` is empty. Batch occurrence IDs with `occurrence_ids`, or
|
||||
join returned content IDs to `collection: "nodes"`, `artifact_ids: [...]` and
|
||||
`fields: ["sha256", "size"]`. Filters intersect; an omitted filter selects all
|
||||
available rows and an empty array selects none. Pagination follows the filtered
|
||||
canonical order. Multiple occurrences may share one content ID.
|
||||
|
||||
Views reuse authenticated immutable inventory facts and lazy lookup indexes;
|
||||
they do not reopen the source. These are historical observations, including
|
||||
after the source changes or disappears. Close the target with `close_binary`
|
||||
when finished. To read an exported bundle through the CLI, select its complete
|
||||
inspection record as `source: {kind: "inline", evidence: ...}` in a JSON request
|
||||
file for `rea inspect-analysis-view request.json --json`.
|
||||
|
||||
## Coverage and limits
|
||||
|
||||
PCK summaries expose root format metadata, available graph counts,
|
||||
`available_hash_status_counts`, and `source_coverage`. Inventory pages preserve
|
||||
producer coverage alongside page coverage. `source_complete` means all declared
|
||||
retained graph rows are available; it does not establish that an encrypted or
|
||||
otherwise unknown directory was examined. Missing matches in unknown coverage
|
||||
are not evidence of absence.
|
||||
|
||||
Encrypted directories retain their header and declared file count with unknown
|
||||
entry coverage. Individually encrypted members retain directory metadata with
|
||||
unknown content identity; REA does not decrypt them. Sparse bundles likewise
|
||||
retain header facts with unknown member coverage. Extraction refuses an
|
||||
unreadable directory or encrypted member. This capability does not recover
|
||||
keys, decode Godot resources, decompile GDScript bytecode, analyze signals,
|
||||
apply patch/removal entries or execute targets.
|
||||
|
||||
Readable logical member bytes, including repeated physical ranges, share the ZIP
|
||||
archive expansion policy: at most the larger of 256 MiB or 500 times the PCK's
|
||||
encoded byte span. Embedded carrier bytes do not enlarge that budget. The full
|
||||
directory is checked before any member is yielded, hashed or extracted. Encrypted
|
||||
members are not read and do not consume this plaintext-work allowance. Small
|
||||
bounded aliases remain valid. This is a processing/output resource bound, separate
|
||||
from the directory metadata retention budget and format validity.
|
||||
@@ -0,0 +1,419 @@
|
||||
# IL2CPP metadata
|
||||
|
||||
`inspect_il2cpp_metadata` / `inspect-il2cpp-metadata` decodes one selected local
|
||||
Unity IL2CPP `global-metadata.dat` file and returns its assembly images, type
|
||||
definitions, methods and named members with their owners, plus managed string
|
||||
literals, by stable index, namespace, range and count,
|
||||
with the header sections and serialized index widths. It reads the file without
|
||||
running it and requires no Unity installation, IL2CPP binary, active target,
|
||||
Hopper, Ghidra or IDA.
|
||||
|
||||
Metadata versions 24, 27, 29, 31 and 39 are supported. For example, Unity 2019.4.41f2
|
||||
wrote version 24, 2020.3.49f1 version 27, 2022.3.30f1
|
||||
wrote version 29, 2022.3.62f3 wrote 31 and 6000.4.2f1 wrote 39; other Unity
|
||||
releases that write these numbers have not been checked. Other versions are
|
||||
rejected as unsupported. Version 24 decoding covers the 264-byte header layout
|
||||
used by the cited Unity 2019.4 build; earlier 24.x layouts are not inferred from
|
||||
the version number and are refused when their header or records differ. The path
|
||||
must name a regular file; directories and symbolic links are rejected.
|
||||
|
||||
With `binary_path`, the tool also reads the matching native binary and links
|
||||
each method to its code address. See [Native method addresses](#native-method-addresses).
|
||||
|
||||
## MCP
|
||||
|
||||
MCP defaults to a compact summary and retains the complete decoded Evidence in
|
||||
the session. MCP requires an absolute filesystem path:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_il2cpp_metadata",
|
||||
"arguments": {
|
||||
"path": "/artifacts/Data/Managed/Metadata/global-metadata.dat"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The summary retains the complete decoded Evidence in the session and returns a
|
||||
linked `inspect_analysis_view` overview: artifact identity, metadata version,
|
||||
index widths and `counts` (`sections`, `images`, `types`, `methods`,
|
||||
`nested_types`, `fields`, `properties`, `events`, `parameters` and
|
||||
`string_literals`) without rows. Use `normalized_result.parent_evidence_id` to page
|
||||
the original rows without reopening or decoding the file:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "<parent_evidence_id>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "il2cpp-metadata",
|
||||
"collection": "types",
|
||||
"offset": 0,
|
||||
"limit": 64
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Collections are `sections`, `images`, `types`, `methods`, `fields`,
|
||||
`properties`, `events`, `parameters`, `string_literals`, `method_addresses`,
|
||||
`return_types`, `field_types`, `parameter_types` and `code_gen_modules`. Every returned row includes
|
||||
its source `index`, scoped by artifact SHA-256 and collection. All collections
|
||||
accept batch `indices` and `fields`; collections whose rows have a `name` also
|
||||
accept exact `names`. Types also accept `image_indices`, `namespaces` and
|
||||
`declaring_types`; methods, fields, properties and events accept
|
||||
`declaring_types`. Parameters accept exact `method_indices`, so a batch of
|
||||
method indices retrieves their parameters without reading the entire parameter
|
||||
table. `parameter_types` accepts the same `method_indices`, resolved through the
|
||||
retained `parameters` owners; Evidence without `parameters` refuses that filter. Filters intersect, duplicate indices are returned once, and
|
||||
rows retain metadata order. An empty filter selects no rows; `fields: []`
|
||||
returns indices only. Omitted fields return the complete selected rows.
|
||||
|
||||
For example, after resolving a type index, retrieve selected method facts:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "<parent_evidence_id>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "il2cpp-metadata",
|
||||
"collection": "methods",
|
||||
"declaring_types": [1555],
|
||||
"names": ["SendMessage"],
|
||||
"fields": ["name", "declaring_type"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After finding method indices, parameters can be requested directly:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "<parent_evidence_id>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "il2cpp-metadata",
|
||||
"collection": "parameters",
|
||||
"method_indices": [9785],
|
||||
"fields": ["name", "method"],
|
||||
"limit": 64
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Historical Evidence that predates these member collections remains queryable.
|
||||
A missing collection has an unknown (`null`) summary count, and requesting it
|
||||
reports that it was not collected; it is never presented as an empty inventory. The same holds for Evidence that predates binary linking: its summary
|
||||
omits `binary`, and `method_addresses`, `method_address_states`,
|
||||
`return_types`, `field_types` and `parameter_types` counts are `null`.
|
||||
Existing empty collections retain a zero count. Full Evidence is preserved.
|
||||
|
||||
`offset` counts matching rows, not source indices. `limit` defaults to 64;
|
||||
`coverage.next_offset` continues that filtered page. Coverage describes retained
|
||||
metadata facts only; runtime behavior and correspondence to native code remain
|
||||
unknown. Each view preserves the parent Evidence link, artifact identity and
|
||||
limitations. Queries work after the source file is removed. Select
|
||||
`detail: "complete"` to return every row inline within the separate MCP response
|
||||
budget.
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
rea inspect-il2cpp-metadata /absolute/path/global-metadata.dat --json
|
||||
```
|
||||
|
||||
The CLI defaults to complete Evidence. For compact delivery, explicitly preserve
|
||||
the full Evidence in a new file:
|
||||
|
||||
```bash
|
||||
rea inspect-il2cpp-metadata /absolute/path/global-metadata.dat \
|
||||
--detail summary --evidence_output /tmp/metadata-evidence.json --json
|
||||
jq -c '{source: {kind: "inline", evidence: .records[0]}, view: {kind: "il2cpp-metadata", collection: "methods", indices: [9785], fields: ["name", "declaring_type"]}}' \
|
||||
/tmp/metadata-evidence.json > /tmp/metadata-query.json
|
||||
rea inspect-analysis-view /tmp/metadata-query.json --json
|
||||
```
|
||||
|
||||
Use the exported Evidence with the CLI's inline Evidence source format described
|
||||
in [CLI and Evidence](cli.md#run-a-command). Compact delivery requires
|
||||
`--evidence_output`; existing files are never overwritten. The full export
|
||||
preserves every decoded row for subsequent queries without the original file.
|
||||
|
||||
`--binary PATH` links method addresses and `--architecture arm64|amd64` selects
|
||||
the slice of a universal Mach-O.
|
||||
|
||||
## Observations
|
||||
|
||||
The result contains:
|
||||
|
||||
- `artifact`: path, SHA-256 and byte length;
|
||||
- `magic`: unsigned little-endian header signature (`0xFAB11BAF`, delivered as a JSON number);
|
||||
- `metadata_version`: `24`, `27`, `29`, `31` or `39`;
|
||||
- `index_widths`: the serialized widths (1, 2 or 4 bytes) of
|
||||
`type_definition`, `generic_container`, `parameter` and `type` indices, and
|
||||
`type_width_source`. Version 39 derives the type index width from the
|
||||
`interface-offsets-record` or the `type-definition-record`; versions 24, 27, 29
|
||||
and 31 always use 4-byte indices (`fixed-width-version`);
|
||||
- `sections`: all version-specific header sections in header order, each with `offset`, `size`
|
||||
and `count`. Versions 24 to 31 record no counts, so `count` is `null` and
|
||||
record counts come from the section sizes;
|
||||
- `images`: `name`, `type_start` and `type_count`;
|
||||
- `types`: `name`, `namespace`, `method_start`, `method_count`,
|
||||
`nested_type_start`, `nested_type_count` and `declaring_type`;
|
||||
- `methods`: `name` and `declaring_type`;
|
||||
- `fields`, `properties` and `events`: `name` and `declaring_type`, the type
|
||||
whose member range holds the row (`-1` when none does);
|
||||
- `parameters`: `name` and `method`, the method whose parameter range holds it;
|
||||
- `string_literals`: managed string literals as `value`, or `value_base64` with
|
||||
the exact bytes when they are not valid UTF-8;
|
||||
- `limitations`.
|
||||
|
||||
A row's position in its collection is its metadata index, so `type_start`,
|
||||
`method_start` and `declaring_type` refer to positions in `types` or `methods`.
|
||||
`nested_type_start` instead indexes the metadata's `nestedTypes` table, which is
|
||||
not returned; each nested type's `declaring_type` carries that relationship.
|
||||
A type's `declaring_type` is derived from the `nestedTypes` section and is `-1`
|
||||
for top-level types. The start of an empty range can retain the original value
|
||||
`-1`.
|
||||
|
||||
The complete inventory of a large game is several megabytes and can exceed
|
||||
the MCP response budget. Summary delivery and filtered, projected pages keep
|
||||
unneeded rows out of agent context without discarding the source Evidence.
|
||||
|
||||
## Native method addresses
|
||||
|
||||
`binary_path` names the native binary built with the metadata: macOS
|
||||
`GameAssembly.dylib` (Mach-O), Windows `GameAssembly.dll` (64-bit PE) or
|
||||
Android `libil2cpp.so` (64-bit ELF, AArch64 or x86-64). A
|
||||
universal Mach-O needs `binary_architecture` (`arm64` or `amd64`); a thin binary
|
||||
rejects it. The file is read, never loaded or executed.
|
||||
|
||||
A position-independent `libil2cpp.so` stores zero in most pointer slots and
|
||||
keeps the value in `R_AARCH64_RELATIVE` or `R_X86_64_RELATIVE` relocations
|
||||
(`DT_RELA`). The reader writes those values into a private copy before
|
||||
searching it, so addresses are the link-time addresses and RVAs are relative to
|
||||
the first loaded segment. Android packed
|
||||
relocations (`DT_ANDROID_RELA`) are refused. No target code is executed.
|
||||
|
||||
The binary's `Il2CppCodeRegistration` points to one `Il2CppCodeGenModule` per
|
||||
assembly image. Each module record starts with its module name, its method
|
||||
pointer count and its method pointer array. The reader finds, for every metadata
|
||||
image, the single record whose name pointer reaches that image's name string and
|
||||
whose count equals the image's method count. It then requires exactly one
|
||||
`codeGenModulesCount`/`codeGenModules` pair that lists exactly those records. No
|
||||
symbols are needed, so stripped builds work. A method's token row id selects its
|
||||
slot in the module's method pointer array.
|
||||
|
||||
Module records establish names through their declared NUL-terminated string pointers. NUL/0xFF padding, spaces, Unicode and suffix-shared storage are supported. A pointer to a longer string does not match just its suffix; a pointer directly into a shared suffix may name it. Exact reversed UTF-8 matching avoids rescanning the file for every name and retains duplicate-name owners until module records resolve them. A contributor investigated a Linux
|
||||
`GameAssembly.so` from Unity 2023.2.13f1 whose static registration was not located. When no static pair is resolved on x86-64, the reader searches executable ranges for bytes consistent with RIP-relative LEA/MOV references to module-array candidates. Exactly one referenced array candidate must remain. `code_gen_modules_field` is then `null`, and Evidence is **inferred**: instruction boundaries, the absence of a static registration elsewhere, and runtime use were not verified. The module records and slots themselves remain file observations. Static-table associations are **derived**. Otherwise the result is `code-registration-not-found`.
|
||||
|
||||
The binary/metadata pair is caller-selected. Name/count/token agreement is structural evidence, not a cryptographic same-build proof. File-offset locations refer to the primary metadata artifact; linked virtual addresses refer to `binary.artifact`. Both SHA-256 identities remain available in compact native pages as `artifact` and `binary_source`.
|
||||
|
||||
Contributor-reported verification (not all rerun by the follow-up): macOS arm64 and x86_64 slices from Unity 2021.3.45f2, 2022.3.30f1, 2022.3.62f3 and
|
||||
6000.0.84f1 against dSYM symbols; Android arm64 and x86-64 from 2022.3.62f3
|
||||
against `libil2cpp.sym.so`; Windows x64 from 2023.2.13f1 (metadata version 29,
|
||||
the MIT-licensed game Between Two Worlds 1.0.5, `GameAssembly.dll` SHA-256
|
||||
`a3ae7f6ecd1f671297b7d98d12713ef024062d93cd7a4205298102656c93b67a`) against
|
||||
Il2CppDumper, where all 43,198 linked methods share an address with a
|
||||
same-named Il2CppDumper method and identical code folding groups 10,448 methods
|
||||
into 1,735 addresses. A macOS x86_64 build from Unity 2020.3.49f1 (metadata version 27, which Il2CppDumper reads as 27.1) links 10,555 methods, all at addresses where Il2CppDumper names the same method; its other addresses are generic instances. A build from Unity 2021.1.28f1 (also read as 27.1) gives the same result: 10,345 methods linked, every address named the same by Il2CppDumper, and its other 3,486 addresses generic instances.
|
||||
|
||||
These checks compare addresses with symbols or with another static tool. They
|
||||
do not establish that a linked address is the code the game runs (a runtime may
|
||||
patch, hook or replace it), that builds from other Unity versions, compilers or
|
||||
obfuscators link the same way, or that a method without an address has no code:
|
||||
generic instances, adjustor thunks and invokers are not linked, and an
|
||||
`unknown` row was not looked up at all.
|
||||
|
||||
The result adds:
|
||||
|
||||
- `binary`: artifact identity, `format`, `architecture`, `image_base`,
|
||||
`code_gen_modules_field` (the address of `codeGenModulesCount`, or `null` as
|
||||
described above), `code_gen_modules_array` (the module pointer array) and
|
||||
`code_gen_modules` (`image`, record `address`, `method_pointer_count`);
|
||||
- `method_addresses`: one row per method index with `state`, `address` and
|
||||
`rva` (address minus `image_base`), both hexadecimal strings. `state` is
|
||||
`code` when the module slot holds an unambiguously mapped executable address, `no-body` when an observed slot is zero,
|
||||
as for abstract, extern and most generic methods, and `unknown` when no type
|
||||
range in any image holds the method or a symbol relocation leaves its slot unresolved.
|
||||
`address` and `rva` are `null` unless `state` is `code`. The summary counts
|
||||
each state in `counts.method_address_states`. Ghidra seeds are written only
|
||||
for `code` rows.
|
||||
|
||||
- `return_types`, `field_types` and `parameter_types`: one row per method,
|
||||
field and parameter index with `type`, the index into the binary's
|
||||
`Il2CppType` table, and `name`, rendered from that table: C# keywords for
|
||||
primitives (`int`, `string`, `object`), `Namespace.Outer+Inner` for type
|
||||
definitions, `T[]`, `T[,]` and `T*`, generic instances as
|
||||
`System.Collections.Generic.List<string>` and generic parameters by their
|
||||
metadata name (`TKey`). A trailing `&` marks a by-reference type. `name` is
|
||||
`null` only for type index `-1`.
|
||||
|
||||
`binary` also records `types_field` and `types_count`: the address of
|
||||
`Il2CppMetadataRegistration.typesCount` and its value. That record is found as
|
||||
the single place whose `fieldOffsetsCount` and `typeDefinitionsSizesCount` both
|
||||
equal the metadata type-definition count and whose type table is file mapped.
|
||||
|
||||
Without `binary_path`, `binary` is `null` and the four binary collections are
|
||||
empty in the complete producer report to indicate unrequested observations. Their summary counts, including state counts, are `null`, and native page requests report unavailable observations. This does not establish an empty native inventory. Missing native fields in historical Evidence remain unknown too.
|
||||
|
||||
For metadata version 27, type names are `null` and only type indices are
|
||||
reported: versions 27.0 to 27.2 share that header number but place the
|
||||
Il2CppType byref flag in different bits, so a rendered name could gain or lose
|
||||
`&` without warning.
|
||||
|
||||
The builds checked are listed after the module search rules above.
|
||||
|
||||
| Reason | Category | Meaning |
|
||||
| --------------------------------- | ----------- | ------------------------------------------------------------------------------------------------- |
|
||||
| `unsupported-container` | unsupported | Not a 64-bit little-endian Mach-O, PE or ELF image (ELF: AArch64 or x86-64 with dynamic section). |
|
||||
| `slice-selection-required` | unsupported | A universal Mach-O was given without `binary_architecture`. |
|
||||
| `slice-not-found` | unsupported | The requested slice is absent, or a thin binary was given an architecture. |
|
||||
| `unsupported-pointer-encoding` | unsupported | Mach-O chained fixups or Android packed (APS2) ELF relocations; pointers not decoded. |
|
||||
| `malformed-container` | input error | The container headers are damaged. |
|
||||
| `code-gen-modules-not-found` | input error | An image has no module record; often a binary from a different build. |
|
||||
| `ambiguous-code-gen-module` | input error | More than one module record matches an image. |
|
||||
| `code-registration-not-found` | input error | No unique module table or executable-byte array candidate was resolved. |
|
||||
| `invalid-method-token` | input error | A method token is not a method row inside its module's pointer count. |
|
||||
| `invalid-method-pointer` | input error | A slot is unavailable or its nonzero address lacks unambiguous executable file coverage. |
|
||||
| `metadata-registration-not-found` | input error | No single MetadataRegistration matches the type-definition count. |
|
||||
| `invalid-type` | input error | A type index or `Il2CppType` record is out of range, unmapped or malformed. |
|
||||
| `unsupported-type-kind` | unsupported | A valid type kind has no implemented rendering. |
|
||||
|
||||
Format and selection failures name `binary_path`; capacity refusals remain typed resource errors. Failed linking preserves the metadata source identity, validated counts, selected binary path/architecture and its SHA-256 when observed, plus prior producer observations. It never publishes an incomplete native table as complete. The binary input is bounded to1GiB.
|
||||
|
||||
Scans yield cooperatively in bounded batches. Name-address retention and reverse-trie construction are each limited to65,536 entries/nodes and fallback module-array candidates to64 (`native-index-capacity`); exceeding capacity does not prove absence. Rendered type text has a64MiB UTF-8 cache-workspace guard, separate from the64MiB encoded-report admission, and64 nested rendering frames (`native-type-text-capacity` / `type-nesting-capacity`). Genuine cycles are malformed; unsupported type kinds are explicit coverage failures. Native row/text admission runs before allocation or retention. Unused generic parameter names are read on demand from the same stable metadata snapshot. There is no cross-inspection file cache; validated immutable Evidence is reused for session views.
|
||||
|
||||
ELF RELATIVE writes into zero-fill are kept as a private virtual-pointer overlay. Unresolved supported-width symbol relocations, including RELA PLT entries, remain unknown; stored zeros never establish no-body. COPY, TLS descriptor, REL, packed APS2 and other unsupported relocation encodings fail closed. Input bytes are unchanged.
|
||||
|
||||
Native address queries batch exact64-bit hexadecimal `addresses` and `states` (`code`, `no-body`, `unknown`), intersecting indices and pagination. Numerically equal address spellings match; original row indices and every shared-body alternative survive:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "<parent_evidence_id>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "il2cpp-metadata",
|
||||
"collection": "method_addresses",
|
||||
"addresses": ["0x180c56ed0"],
|
||||
"states": ["code"],
|
||||
"fields": ["address", "rva"],
|
||||
"offset": 0,
|
||||
"limit": 64
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Ghidra function names
|
||||
|
||||
The CLI can turn a linked report into a
|
||||
[`REA_GHIDRA_SEED_FILE`](installation.md) so Ghidra names every IL2CPP method
|
||||
before auto-analysis:
|
||||
|
||||
```bash
|
||||
rea inspect-il2cpp-metadata global-metadata.dat --binary libil2cpp.so \
|
||||
--ghidra-seeds il2cpp-seeds.tsv --json > /dev/null
|
||||
export REA_GHIDRA_SEED_FILE="$PWD/il2cpp-seeds.tsv"
|
||||
```
|
||||
|
||||
Each linked method becomes one `function` seed named `Namespace.Type$$Method`
|
||||
(nested types joined with `.`, whitespace replaced by `_`), the Il2CppDumper
|
||||
convention. Seeds are at `seed-base + rva`. The default base is where Ghidra's
|
||||
loaders place the binary: `0x100000` for an ELF linked at 0, and the link base
|
||||
for Mach-O and PE; `--seed-base 0x...` overrides it for a different import. When
|
||||
identical code folding gives several methods one address, the first method names
|
||||
the function and a `# folded <address> <kept> also <names>` comment line after
|
||||
the seeds lists the others, so the kept name is visibly one of several. Ghidra still names a seeded function that is a thunk after its
|
||||
target. The file is atomically published with exclusive creation; an existing destination is never replaced. Failed staging retains its cleanup owner for retry. Source digest and link/seed-base comments make the mapping inspectable. Unknown slots are skipped separately from observed no-body slots. `--seed-base` requires `--ghidra-seeds`; overflowing rebased addresses and names/outputs beyond the existing Ghidra protocol limits (2,000 name characters,32MiB file,1,000,000 entries) are refused before publication. The CLI and reusable service use one formatter and owned output workflow; ordinary MCP inspection remains read-only.
|
||||
|
||||
## Failures
|
||||
|
||||
A file whose magic is not `0xFAB11BAF` fails as an unsupported target with
|
||||
`unrecognized-magic`. The file may be encrypted or may not be IL2CPP metadata.
|
||||
A version other than 24, 27, 29, 31 or 39 fails as an unsupported target with
|
||||
`unsupported-version`.
|
||||
|
||||
Damaged metadata returns an input error (`invalid_format`) whose
|
||||
message names one reason:
|
||||
|
||||
| Reason | Meaning |
|
||||
| ------------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
||||
| `truncated-header` | The file is shorter than the header. |
|
||||
| `unexpected-header-size` | `stringLiterals.offset` is not the header size: 264 bytes for version 24, 256 for 27, 29 and 31, 380 for 39. |
|
||||
| `section-out-of-bounds` | A section extends beyond the file, or a non-empty section starts inside the header. |
|
||||
| `section-overlap` | Two sections overlap. Zero-size sections are exempt and gaps are allowed. |
|
||||
| `record-size-mismatch` | A section size does not divide into its records, or the record size differs from the one the index widths imply. |
|
||||
| `type-index-width` | The record used to derive the type index width implies no 1, 2 or 4 byte width. |
|
||||
| `index-out-of-range` | A serialized index or range points outside its target collection. |
|
||||
| `nested-type-conflict` | A type is listed as nested in two types, in itself, or in a cycle of declaring types. |
|
||||
| `declaring-type-mismatch` | A method inside a type's method range names a different type, or a method lies outside its type's range. |
|
||||
| `image-range-overlap` | Two images claim the same type definitions. |
|
||||
| `invalid-string` | A name index does not start a string, or the name is unterminated or not valid UTF-8. |
|
||||
|
||||
A version field that disagrees with the file's layout, for example a version
|
||||
39 file relabeled as 29, usually surfaces as `unexpected-header-size` or
|
||||
`record-size-mismatch`.
|
||||
|
||||
Two checks are stricter than the IL2CPP runtime: a name index must start a
|
||||
string, and an empty range's start must lie between -1 and the collection size.
|
||||
Every Unity-written file checked so far satisfies both; metadata rewritten by
|
||||
other tools may be rejected.
|
||||
|
||||
An incomplete inventory is never reported as complete. Failures after a verified
|
||||
snapshot retain partial source Evidence with its SHA-256, byte length and any
|
||||
validated section row counts; these counts do not establish that names and
|
||||
relationships were decoded. A file that changes during reading and a cancelled
|
||||
request keep distinct errors. Decode checkpoints yield for cancellation, and
|
||||
close waits for active work. Each reader admits one inspection at a time.
|
||||
Resource guards bound the selected file to 256 MiB and the encoded complete
|
||||
report to 64 MiB, checking row counts before allocation and JSON/UTF-8/base64 expansion during
|
||||
decoding, before serialization. Names and identical literal spans are decoded
|
||||
once within one stable file snapshot; no filesystem results are cached across
|
||||
inspections. Ownership assignment also yields to cancellation. MCP also applies its
|
||||
[response transport budget](mcp-contracts.md); a complete report within the
|
||||
report limit can still require the summary and paging path.
|
||||
|
||||
## Outside this tool
|
||||
|
||||
- Decrypting or deobfuscating protected metadata.
|
||||
- Linking Mach-O binaries that use chained fixups, ELF images with Android
|
||||
packed (APS2) relocations, and 32-bit images; generic method instances,
|
||||
adjustor thunks and invokers.
|
||||
- Full method signatures beyond the rendered field, parameter and return type
|
||||
names, generic method instances, and type names for metadata version 27.
|
||||
- Running Ghidra or another disassembler. `--ghidra-seeds` only writes a seed
|
||||
file for [`REA_GHIDRA_SEED_FILE`](installation.md); use the
|
||||
[native workflow](native-investigation.md) for native code.
|
||||
- Metadata versions other than 24, 27, 29, 31 and 39.
|
||||
|
||||
Names come from the metadata file only. They do not establish which methods
|
||||
were compiled into the native binary or how they behave at runtime.
|
||||
|
||||
## Real producer verification
|
||||
|
||||
`npm run verify:il2cpp:metadata -- --metadata PATH` exercises a caller-supplied
|
||||
`global-metadata.dat` through the public CLI and MCP. It is opt-in, like
|
||||
`verify:pcap`, because no real game metadata ships with the repository. Normal
|
||||
inspection does not need it.
|
||||
|
||||
The follow-up was checked against the public Cpp2IL
|
||||
[`Simple_2022_3_35` sample](https://github.com/SamboyCoding/Cpp2IL/tree/b5ad444b82267cb1e4b88b8b373c008105bdea52/TestFiles/Simple_2022_3_35):
|
||||
version 31, 1,892,268 bytes, SHA-256
|
||||
`2441635a7c234a1a34deccc6a9906d6f5abcbdab078a38b2bf488a52a4962c48`.
|
||||
An independent byte-layout oracle matched all 36 images, 2,468 types and 12,409
|
||||
methods. Corresponding managed DLLs independently matched names and ownership
|
||||
for 35 images, 2,463 types and 12,406 methods; the generated image has no matching
|
||||
DLL. This does not establish real-producer coverage for versions 29 or 39.
|
||||
|
||||
Compact native summaries omit the `binary.code_gen_modules` table and keep its
|
||||
count in `counts.code_gen_modules`. Query `collection: "code_gen_modules"`
|
||||
with batch `image_indices` or source `indices`, selected `fields` and pagination.
|
||||
Module row `index` is its retained collection position; `image` joins the
|
||||
metadata image collection. Scalar registration facts and both source identities
|
||||
remain inline. Complete Evidence retains the original full table.
|
||||
+13
-11
@@ -47,17 +47,19 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
|
||||
|
||||
## Choose a guide
|
||||
|
||||
| Task | Guides |
|
||||
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
|
||||
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
|
||||
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
|
||||
| Inspect assemblies, packages, or firmware | [.NET](managed-code-analysis.md), [Android](android-analysis.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
|
||||
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
|
||||
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
|
||||
| Capture and compare a command | [Process capture](process-capture.md) |
|
||||
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
|
||||
| Task | Guides |
|
||||
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
|
||||
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
|
||||
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
|
||||
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [.NET](managed-code-analysis.md), [Android](android-analysis.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
|
||||
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
|
||||
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
|
||||
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
|
||||
| Capture and compare a command | [Process capture](process-capture.md) |
|
||||
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
|
||||
|
||||
For development, use [testing](testing.md), [client verification](client-verification.md),
|
||||
[tool design](tool-design.md), and [releasing](releasing.md).
|
||||
|
||||
+54
-7
@@ -276,14 +276,14 @@ Register with the variable unset:
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```bash
|
||||
claude mcp add --scope local rea -- npx -y rea-agents@6.6.0 mcp
|
||||
claude mcp add --scope local rea -- npx -y rea-agents@6.7.0 mcp
|
||||
```
|
||||
|
||||
On native Windows, use the same project directory and wrap the package-runner
|
||||
shim so the stdio client can launch it:
|
||||
|
||||
```powershell
|
||||
claude mcp add --scope local rea -- cmd.exe /d /c npx -y rea-agents@6.6.0 mcp
|
||||
claude mcp add --scope local rea -- cmd.exe /d /c npx -y rea-agents@6.7.0 mcp
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
@@ -376,7 +376,17 @@ Uninstall preserves Hopper, Node.js, Evidence files, captures, unrelated skills
|
||||
and other MCP servers. Purging removes only REA's cache and state under
|
||||
`~/.rea`. A client configuration that is malformed, unreadable, or at an unsafe
|
||||
path stops the operation before anything is removed, and a client that fails
|
||||
while being updated stops the remaining removals. A purge path that is a
|
||||
while being updated stops the remaining removals. Uninstall reports each
|
||||
candidate `configuration_path` with its outcome. A client with multiple files
|
||||
keeps their individual `actions` alongside its aggregate status and detail,
|
||||
including earlier removals when a later file fails. The paths name
|
||||
the discovered configuration candidates, not resolved physical file identities.
|
||||
For JSON clients, uninstall removes every repeated `rea` member in the effective
|
||||
server table only when each registration is REA-owned. An unowned earlier
|
||||
member, or an unowned OpenCode V1 entry that removing V2 would expose, stops
|
||||
preflight before any
|
||||
removal. Repeats inside an inactive earlier copy of the server table stay
|
||||
unchanged. A purge path that is a
|
||||
symbolic link is retained and reported rather than followed. Uninstall checks
|
||||
REA's managed shared and Claude Code personal skill locations. See the
|
||||
[CLI guide](cli.md#output-and-exit-status) for exit statuses.
|
||||
@@ -599,6 +609,20 @@ host mount and the rest of `/tmp` remain unchanged; this fallback never invokes
|
||||
`sudo`. `rea doctor --provider hopper --json` reports the selected
|
||||
strategy and both host and effective mount facts.
|
||||
|
||||
Hopper also embeds Python; the Python executable and standard library alone
|
||||
are insufficient on Ubuntu. Approved setup checks the system interpreter
|
||||
at `/usr/bin/python3` and, when necessary, installs its matching
|
||||
`libpython3.X` package, then verifies that its shared library can load.
|
||||
APT resolves the concrete package (for example, `libpython3.12t64` on Ubuntu
|
||||
24.04). Doctor includes these fresh observations in the `embedded_python`
|
||||
details of its `hopper-demo-runtime` check, alongside the private-display
|
||||
diagnostic. Failed inspections remain unknown rather than proving a library
|
||||
absent. Run `rea setup --yes --install-hopper` to repair a supported install.
|
||||
This prerequisite check does not launch the bridge or validate a custom Python
|
||||
interpreter configured inside Hopper; that interpreter needs its own matching
|
||||
shared runtime. Custom installation prefixes or shared-library ABIs require
|
||||
manual dependency repair.
|
||||
|
||||
When the package-manager step fails, setup reports the failing command and its
|
||||
captured output, so a package conflict is distinguishable from an authorization
|
||||
failure. On CachyOS, an unversioned compatibility package such as
|
||||
@@ -963,6 +987,24 @@ can time out earlier even when Ghidra would complete within its startup deadline
|
||||
See [Ghidra first-query deadlines and recovery](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
|
||||
for client options and the close/reopen recovery flow.
|
||||
|
||||
The 330-second default is configurable. Configuration is read when REA starts:
|
||||
changing an environment variable in another shell, or closing and reopening a
|
||||
binary in the same MCP server, does not change that server's startup allowance.
|
||||
For a CLI retry with an example 30-minute allowance:
|
||||
|
||||
```sh
|
||||
REA_GHIDRA_STARTUP_TIMEOUT_MS=1800000 rea analyze /absolute/path/to/app.exe --provider ghidra --json
|
||||
```
|
||||
|
||||
For MCP, export useful Evidence before restarting, close the binary and resolve
|
||||
any cleanup failure, update the environment in the client's REA server launch
|
||||
configuration, then restart REA and reconnect before reopening the target. Allow
|
||||
startup plus the query in the client's own tool-call deadline. Every retry uses
|
||||
a fresh ephemeral import; smaller result requests do not reduce the prerequisite
|
||||
auto-analysis. A larger time allowance does not raise the JVM heap limit or
|
||||
guarantee completion. Keep [heap and host CPU controls](#ghidra-heap-and-cpu-controls)
|
||||
appropriate for the machine.
|
||||
|
||||
One session contains exactly one imported Program; use `provider_id: "ghidra"`, `--provider ghidra`, or
|
||||
`REA_ANALYSIS_PROVIDER=ghidra` when both Hopper and Ghidra support the target.
|
||||
One persistent decompiler is owned by the Program, and a serial queue keeps
|
||||
@@ -1065,9 +1107,14 @@ and hyphens; at most 32 names are admitted, each at most 120 characters.
|
||||
Names must identify analyzers registered for the selected program. Unknown names
|
||||
or options that remain enabled abort the headless import before auto-analysis
|
||||
and the bridge, so an unsuccessful override cannot produce a session claiming
|
||||
that the analyzer was disabled. For Windows PE resource analysis, the exact name
|
||||
is `WindowsResourceReference`. Disabling an analyzer skips the facts it would
|
||||
have produced.
|
||||
that the analyzer was disabled. The setting applies to every import in the REA
|
||||
server. A format-specific analyzer may not be registered for another target, so
|
||||
the same override can prevent that target from opening. For mixed-target work,
|
||||
omit the override or use a separate REA server configured for the affected
|
||||
format. Changing the environment requires restarting the server; reopening a
|
||||
target alone keeps the current setting. For Windows PE resource analysis, the
|
||||
exact name is `WindowsResourceReference`. Disabling an analyzer skips the facts
|
||||
it would have produced.
|
||||
|
||||
### Ghidra heap and CPU controls
|
||||
|
||||
@@ -1187,7 +1234,7 @@ For a client that requires manual configuration, use:
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@6.6.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@6.7.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -195,6 +195,17 @@ earlier read or captured alias. Unproved ordering across branches or function
|
||||
activations remains conservative. Returned-reference effects use the same
|
||||
read-position facts as call resolution.
|
||||
|
||||
Returned-reference effects share one property-projection state per source read.
|
||||
When several paths reach that read, differing keys at the same depth become
|
||||
wildcards; differing depths become an unknown-depth selection. This retains a
|
||||
conservative superset of the represented local targets rather than enumerating
|
||||
every alias history or dropping paths at a query cap. It can add possible effects,
|
||||
including for finite projection cycles. Static limitations disclose this loss of
|
||||
precision; semantic graphs retain `dynamic-property` unknowns at the affected
|
||||
callee coordinates, and captured web bundles identify the affected script keys.
|
||||
These candidates do not prove runtime calls or mutations. Existing source-worker
|
||||
and result-admission limits still apply.
|
||||
|
||||
Mutation tracking covers explicit member assignments, updates, deletion, and
|
||||
loop assignment targets, including the supported local aliases and shared
|
||||
nested references. Calls conservatively invalidate exposed mutable references;
|
||||
|
||||
@@ -50,6 +50,12 @@ The equivalent MCP input is:
|
||||
`input_path` must be absolute. `format` accepts `auto`, `asar`, or `directory`.
|
||||
`integrity_policy` accepts `fail` or `record-and-continue`; both CLI routes and
|
||||
the MCP tool default to `fail`.
|
||||
Evidence parameters retain the caller-selected `input_path`, including a
|
||||
symlink alias. Both CLI routes anchor relative inputs to their working
|
||||
directory without resolving symlinks. The subject and normalized result use
|
||||
the resolved path for artifact identity; the selected path remains a separate
|
||||
request fact. Older Evidence can omit this parameter; retained records are
|
||||
validated without resolving their paths against the live filesystem.
|
||||
The result retains the canonical local path, root artifact digest, artifact
|
||||
manifest and graph commitments, JavaScript Application Graph, static
|
||||
Electron summary, reconstruction statistics, and explicit limitations. It does
|
||||
|
||||
@@ -152,6 +152,15 @@ and uncollected scripts, but scripts collected before attachment may be
|
||||
missing. Absence from the observation window never proves that a script or
|
||||
behavior does not occur.
|
||||
|
||||
Cancellation stops new target-file authorization after the current filesystem
|
||||
operation settles. Cancellation diagnostics retain the examined target prefix;
|
||||
unexamined targets remain unknown. Observation cancellation keeps its original
|
||||
reason and any completed Evidence. Finalization waits for REA's bounded socket
|
||||
close (up to two seconds for graceful and forced confirmation). A failed close
|
||||
retains its exact transport owner and blocks replacement connections until
|
||||
cleanup succeeds; CLI finalization and MCP shutdown also retry retained cleanup.
|
||||
Closing REA's connection does not stop the caller-owned runtime.
|
||||
|
||||
## Static correlation
|
||||
|
||||
Pass the resulting observation Evidence and an
|
||||
|
||||
+14
-4
@@ -8,7 +8,16 @@ Start a JEB client that serves MCP and point REA at it:
|
||||
|
||||
- Default endpoint: `http://127.0.0.1:8425/mcp`. Override with `REA_JEB_MCP_URL` (absolute `http`/`https` URL with literal host `127.0.0.1` or `[::1]`; redirects, remote hosts, credentials, query strings, and fragments are rejected rather than silently dropped).
|
||||
- In the JEB GUI client, start the MCP server from the _File_ menu (or via the VIBRE assistant).
|
||||
- Headless, use JEB's documented programmatic route (`JebMcpServerInstance` from the JEB API) with the JEB client on your classpath; JEB 5.48's `-c` headless mode does not accept `--mcp` directly.
|
||||
- Headless, the official Linux CE build `5.48.0.202610071539` supports `-c --mcp`. From the JEB installation directory, select JDK 21 or newer through `JAVA_HOME`, choose your target and bind explicitly to loopback:
|
||||
|
||||
```sh
|
||||
"$JAVA_HOME/bin/java" -jar bin/app/jeb.jar -c --mcp --interface=127.0.0.1 \
|
||||
--port=8425 --infile=/absolute/path/to/target
|
||||
```
|
||||
|
||||
Complete the vendor's first-use EULA and license-key activation when prompted. The Linux audit used an isolated installation, a 1536 MiB Java heap and one active processor inside a separately bounded process family; JVM heap limits alone do not bound the whole family.
|
||||
|
||||
- JEB's documented programmatic route (`JebMcpServerInstance` from the JEB API) remains an alternative. The earlier licensed 5.48 headless verification used that route because its CLI did not accept `--mcp`. Check your installed build's `-c --help` and live tool catalog; the version number alone does not establish the launcher options or supported operations.
|
||||
|
||||
For setup registration, set `REA_JEB_MCP_URL` in the REA process environment
|
||||
before running `rea setup`. The approved setup plan preserves the exact selected
|
||||
@@ -31,7 +40,7 @@ or establish one end-to-end deadline across a whole CLI or MCP operation. A
|
||||
request that reaches its budget returns a typed timeout, while caller
|
||||
cancellation remains a cancellation.
|
||||
|
||||
Verified against JEB 5.48.0 (Linux x64, licensed build) serving `jeb-mcp-server` 1.3.0 over streamable HTTP.
|
||||
Earlier verification used JEB 5.48.0 (Linux x64, licensed build) serving `jeb-mcp-server` 1.3.0 over streamable HTTP. The official Linux x64 CE build `5.48.0.202610071539` was also verified through real CLI and production stdio MCP native-code workflows; see the [Linux CE audit](../experiments/analysis-efficiency/jeb-linux-native.md) for measurements and limits. CE verification does not establish licensed/Demo Android decompilation or GUI behavior.
|
||||
|
||||
## Operations
|
||||
|
||||
@@ -42,15 +51,16 @@ Verified against JEB 5.48.0 (Linux x64, licensed build) serving `jeb-mcp-server`
|
||||
| `list_jeb_units` / `list-jeb-units` | Page units by stable path and type (`filter`, `parent-unit-path`, `index`, `count`) |
|
||||
| `decompile_jeb_item` / `decompile-jeb-item` | Decompile one type or method to engine pseudo-code |
|
||||
|
||||
All four are read-only with respect to the target file; only `open_jeb_project` changes engine-side project state, and Evidence records the endpoint, the caller-selected parameters, and the engine-reported digests. Mutation workflows the engine also offers (renames, comments, `save_project`, `close_project`) are intentionally not exposed yet.
|
||||
All four preserve target-file bytes. `open_jeb_project` selects or creates an engine-side project; analysis queries can populate computed analysis within it. Evidence records the endpoint, caller-selected parameters, and available engine-reported digests. Mutation workflows the engine also offers (renames, comments, `save_project`, `close_project`) are intentionally not exposed yet.
|
||||
|
||||
**Engine coverage note (verified):** JEB 5.48.0 headless MCP instances do not advertise `open_project` — the target must be selected when the engine client starts (`--infile` on the programmatic launcher, or the GUI's file opening). REA preserves that refusal with the engine's reason instead of failing silently; `open_jeb_project` completes only against engine builds that advertise the tool, such as GUI clients.
|
||||
**Engine coverage note:** The earlier licensed 5.48.0 headless verification did not advertise `open_project`; it selected the target when the engine client started. For that build, use `--infile` on the programmatic launcher or open the file in the GUI. The audited CE headless catalog also omitted `open_project`; the verified CLI route above selects its input at startup. Use `binary_session` to inspect the current engine's advertised operations. REA preserves an unavailable tool's refusal; `open_jeb_project` requires an engine that advertises `open_project`. The CE audit does not establish GUI project-opening support.
|
||||
|
||||
## Semantics
|
||||
|
||||
- Paths in `open_jeb_project` are resolved by the JEB process, not by REA; REA does not require the artifact to exist locally.
|
||||
- The engine caps `list_units` pages at 100; a full page is reported as `coverage: "partial"` because more units may exist. An empty remainder is `complete` only when fewer units than requested were returned.
|
||||
- When `decompile_jeb_item` omits `unit_path`, the engine implicitly selects the project's first code unit; REA reports `unit_path: null` and a limitation instead of guessing the effective unit.
|
||||
- `item_address` is a JEB code-item reference, such as a value returned by the engine's `list_code_items`. Preserve that representation and select its code unit explicitly. For the audited CE SQLite unit, the exported method reference was `sqlite3_libversion_number`; using its raw ELF symbol address did not produce a successful decompilation. File offsets, ELF virtual addresses and engine item references have different semantics. REA currently exposes unit listing, while code-item discovery remains an upstream JEB operation.
|
||||
- Engine refusals (`success: false` envelopes) are preserved as unsupported-target errors carrying the engine's own message, for example a nonexistent item address.
|
||||
- Successfully parsed client information is reused for the lifetime of the MCP connection, including when the engine omits its version. Explicit client inspection and availability probes refresh it.
|
||||
- An unreachable endpoint is a provider-availability failure with recovery guidance; REA does not retry or spawn the engine.
|
||||
|
||||
@@ -13,6 +13,14 @@ rea doctor --provider kuna
|
||||
|
||||
Select it explicitly with `--provider kuna`, MCP's `open_binary.provider_id: "kuna"`, or `REA_ANALYSIS_PROVIDER=kuna`. With Kuna and another provider configured for the same target, `auto` reports `ambiguous` instead of choosing one.
|
||||
|
||||
Ordinary readiness cancellation returns `cancelled` (CLI SIGINT/SIGTERM exits
|
||||
130/143); an earlier recorded timeout or physical failure keeps its diagnostic.
|
||||
Completed version, hash and script observations remain in `kuna_installation`
|
||||
with `status: "partial"` and `completed_hash` when available. Typed errors carry
|
||||
this in `details.diagnostics`; doctor cleanup checks carry it in their `details`.
|
||||
Omitted fields remain unobserved; availability remains unknown until readiness
|
||||
completes. Cleanup retries keep both the observations and the exact file owner.
|
||||
|
||||
| Setting | Default | Meaning |
|
||||
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `REA_KUNA_COMMAND` | none (required) | Absolute path of the `kuna` executable. |
|
||||
|
||||
@@ -119,9 +119,11 @@ returning an empty one.
|
||||
|
||||
The canonical parser admits PE/CLI bytes and their metadata/CIL without loading
|
||||
an assembly. It reports observed implementation markers and unavailable facts;
|
||||
it does not unpack .NET single-file hosts, decode IL2CPP metadata, or infer a
|
||||
NativeAOT identity from an ordinary native PE. Inputs without admitted CLI
|
||||
metadata do not become managed assemblies through naming or routing guesses.
|
||||
it does not unpack .NET single-file hosts or infer a NativeAOT identity from an
|
||||
ordinary native PE. It does not decode IL2CPP; use
|
||||
[IL2CPP metadata](il2cpp-metadata.md) for the image, type and method names in a
|
||||
Unity `global-metadata.dat`. Inputs without admitted CLI metadata do not
|
||||
become managed assemblies through naming or routing guesses.
|
||||
Inspect separately obtained components explicitly and preserve their identities.
|
||||
|
||||
A valid implementation marker establishes a candidate native boundary, not
|
||||
|
||||
+240
-5
@@ -1,8 +1,13 @@
|
||||
# MCP runtime contracts
|
||||
|
||||
For pairing native functions before comparing selected dossiers, see
|
||||
[native function matching](native-function-matching.md). It operates on retained
|
||||
inventories, preserves aliases/unknowns and exposes selected fact pages.
|
||||
|
||||
Use this reference to call REA tools and interpret their replies on one MCP
|
||||
connection. Start with [tool results](#tool-results), [retained Evidence](#retained-application-evidence-inputs),
|
||||
and [session discovery](#identity-and-discovery). See [cancellation](#progress-and-cancellation)
|
||||
and [session discovery](#identity-and-discovery). See [transports](#transports)
|
||||
for the opt-in HTTP listener, [cancellation](#progress-and-cancellation)
|
||||
for partial-result recovery and [first-query deadlines](#ghidra-first-query-deadlines-and-recovery)
|
||||
when a cold provider exceeds the client's wait.
|
||||
|
||||
@@ -21,6 +26,14 @@ directly to `compare_functions`, and `inspect_artifact` Evidence to
|
||||
`compare_artifacts`. Use `get_evidence_bundle` when the task needs broader
|
||||
retained session history or an explicit bundle for transfer.
|
||||
|
||||
Ghidra inventories and function replies use `raw_result: null` when their facts
|
||||
are already preserved in `normalized_result`, limitations, locations and
|
||||
relationship coverage. Distinct overview details, measured load-image
|
||||
observations, file-offset selection roles and database types supplemented with
|
||||
derived metadata keep their upstream representation. Read function results from
|
||||
`normalized_result`; older retained Evidence keeps its original representation
|
||||
and identity.
|
||||
|
||||
Artifact inventories declare `coverage.scope` (`root` or `container-entries`)
|
||||
and `coverage.status` (`complete` or `unknown`) independently of diagnostic
|
||||
wording. Comparison requires matching scopes, complete graph members and complete
|
||||
@@ -58,7 +71,13 @@ Call comparison additionally retains the Evidence relationship coverage
|
||||
checks. Ghidra and Hopper declare their observed dossier facets and complete
|
||||
provider-observed caller/callee sets, including empty sets. Missing decompiler
|
||||
output remains unavailable; complete call coverage does not claim every runtime
|
||||
target. Hopper's unavailable reference-kind metadata and reported unresolved
|
||||
target. When Ghidra's decompiler fails on one function, its dossier is still
|
||||
returned: `pseudocode`, `native_api` and `native_value_flow` are unavailable
|
||||
with the decompiler's failure as the reason, while the listing-backed facets
|
||||
are collected as usual. These failed decompiler observations and workflows
|
||||
derived from them retain Evidence but skip snapshot reuse, so a later request
|
||||
can observe backend recovery; healthy dossiers keep their existing cache policy.
|
||||
`procedure_pseudo_code` still fails for that function. Hopper's unavailable reference-kind metadata and reported unresolved
|
||||
calls still prevent equality conclusions for those dimensions. Analysis profiles
|
||||
version these declarations so older snapshots are not reused as current results.
|
||||
IDA declares the facets its selected adapter cannot provide; an empty
|
||||
@@ -71,13 +90,30 @@ metadata in the supported JSON payloads (`parameters`, `raw_result`,
|
||||
|
||||
Tool execution failures return `isError: true` and the complete canonical
|
||||
`{ "error": ... }` diagnostic as JSON in text `content`. Read that diagnostic
|
||||
for the error code, target details, partial observations, and remediation.
|
||||
for the error code, target details, partial observations, remediation, and the
|
||||
optional bounded `recovery` next step. `recovery.kind=inspect_input_contract`
|
||||
names the rejected operation: re-read that operation's advertised input schema
|
||||
(call `tools/list` if the catalog may be stale), correct the listed fields, and
|
||||
do not repeat the unchanged call. `recovery.kind=inspect_timeout_state` names
|
||||
the operation and the REA server/provider deadline; inspect session/provider
|
||||
state before retrying, and raise the MCP client's tool deadline only when the
|
||||
client ended the call before REA returned this response. These hints never
|
||||
include the complete catalog or caller arguments.
|
||||
Failures omit `structuredContent`, since each advertised `outputSchema`
|
||||
describes successful results. Successful replies retain their schema-validated
|
||||
structured data and matching text projection. Oversized errors are retained
|
||||
as Evidence before delivery of a bounded diagnostic with an export reference;
|
||||
if retention is unavailable or fails, the diagnostic reports that reason.
|
||||
|
||||
SDK-generated input validation failures are normalized into the same canonical
|
||||
`invalid_request` text when the bounded diagnostic fits the response budget.
|
||||
The complete field/type diagnostic is preserved when normalization fits the
|
||||
response budget, including request-specific rejected fields and reasons. Oversized
|
||||
SDK diagnostics keep the existing retained-Evidence recovery path. An unknown
|
||||
tool remains an MCP JSON-RPC `-32602` error, but its message directs the caller
|
||||
to `tools/list` and an advertised operation instead of repeating the unknown
|
||||
name.
|
||||
|
||||
`get_call_graph` returns a `graph` keyed by traversal depth and a top-level
|
||||
`limitations` array with deduplicated execution caveats. Each node keeps its
|
||||
relationship coverage and read-specific `limitations`, alongside observed
|
||||
@@ -92,8 +128,9 @@ unresolved relationship reads. Cancellation remains a tool error.
|
||||
|
||||
REA prepares complete MCP results within the pinned stdio client's 10 MiB
|
||||
receive-buffer budget, including both text and structured representations and
|
||||
room for the JSON-RPC envelope. Tool errors carry only their text
|
||||
representation and `isError` flag, so only those count against the budget. Response budget settings are captured at startup;
|
||||
room for the JSON-RPC envelope. Tool errors omit `structuredContent` from the
|
||||
wire; their text, `isError`, provenance marker and negotiated protocol framing
|
||||
count toward the wire budget. Response budget settings are captured at startup;
|
||||
restart or recreate the server to apply changes. If a result cannot fit, REA returns
|
||||
`resource_constraint` with `details.resource: "transport"` before constructing
|
||||
a document-sized string. Analysis Evidence remains complete in the current
|
||||
@@ -103,6 +140,52 @@ that reference for a summary, one section/module, or a stable page, or call
|
||||
`export_evidence_bundle` with a destination path. Complete
|
||||
bundle exports stream canonical JSON into an atomically published file.
|
||||
|
||||
On protocol revision `2026-07-28`, completed replies retain the SDK's
|
||||
`resultType: "complete"`, including the replacement for an oversized error.
|
||||
This marks completion of the MCP exchange; Evidence coverage, partial facts
|
||||
and unknowns retain their separate meaning. Legacy replies keep their existing
|
||||
framing.
|
||||
|
||||
`list_procedures`, `search_procedures`, `list_strings`, `search_strings` and
|
||||
`list_names` accept `detail: "summary"`. Complete inline delivery remains the
|
||||
default. Summary retains the original complete Evidence and returns its row
|
||||
count, provider identity, analysis-profile digest, original query, parent Evidence
|
||||
ID, limitations and explicit unknown discovery coverage. Delivery detail is excluded from provider arguments
|
||||
and original Evidence identity.
|
||||
|
||||
Read selected rows with `inspect_analysis_view`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_<source digest>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "native-inventory",
|
||||
"offset": 0,
|
||||
"limit": 32,
|
||||
"fields": ["value"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Each row has `source_index` and `record`; parent Evidence ID plus source index
|
||||
identifies same-address alternatives. Order is the original retained order.
|
||||
Omit `fields` for complete records; a selected field list always retains
|
||||
`record.address`. Selecting `value` also retains available `string` and
|
||||
`decoding` qualifications, so native display text and unterminated fragments
|
||||
remain distinguishable. Search tools apply their existing filters before retention.
|
||||
Follow `coverage.next_offset`; exhaustion describes the retained list, not
|
||||
complete discovery of all code or strings. An unbound source keeps
|
||||
`artifact: null` and the derived Evidence keeps `subject: null`.
|
||||
|
||||
The CLI uses the same projector. For an original complete Evidence JSON file,
|
||||
write `{"source":{"kind":"evidence-file","path":"/absolute/source.json"},"view":{"kind":"native-inventory","limit":32}}`
|
||||
to `/absolute/view-request.json`, then run
|
||||
`rea inspect-analysis-view /absolute/view-request.json --json`. This also reads
|
||||
pre-summary source records without changing their identity; no live provider is started.
|
||||
|
||||
For oversized native function Evidence, `details.reported_limits.continuation`
|
||||
contains a ready-to-call `inspect_analysis_view` request for the retained
|
||||
pseudocode's first page. Use its `name` and `arguments` directly, then follow
|
||||
@@ -111,6 +194,8 @@ limit is the view contract's default, not a guarantee that arbitrary source or
|
||||
metadata fits; reduce the limit or export when needed. Continuations are
|
||||
offered for completed `analyze_function` results only after session recording is
|
||||
acknowledged; the complete result and its retention reference remain intact.
|
||||
Oversized native inventories likewise include a ready-to-call first
|
||||
`native-inventory` page. Reduce its limit or select fields when a row is large.
|
||||
|
||||
A broad follow-up or `get_evidence_bundle` can also exceed the response budget; exporting
|
||||
preserves the complete session without sending it through a single MCP frame.
|
||||
@@ -132,6 +217,12 @@ is null. `total` counts all matching heads; each page preserves complete records
|
||||
in stable unknown ID order. Filters apply before paging. Pages reflect current
|
||||
session state, so keep the unknown ledger unchanged while traversing it.
|
||||
|
||||
Automatically retained workflow questions include their immutable result
|
||||
Evidence ID. Distinct results retain independent questions and source links;
|
||||
repeating the same result does not create another unknown. Workflow Evidence
|
||||
keeps the original residual question text. Explicit `record_unknown` requests
|
||||
keep caller-selected identity and reject duplicate creation.
|
||||
|
||||
For a portable exported bundle, the CLI shares this projection:
|
||||
`rea inspect-evidence-bundle '{"path":"/tmp/evidence.json","detail":"summary","filters":{"operation":"analyze_function"}}' --format json`.
|
||||
The CLI validates and materializes that local bundle under its existing memory
|
||||
@@ -210,6 +301,21 @@ inferred Evidence whenever any group is templated, and derived Evidence only whe
|
||||
no group was templated.
|
||||
See [derived API surfaces](web-network-captures.md#derived-api-surface).
|
||||
|
||||
`inspect_analysis_view` also accepts `environment-manifest` for retained HAR capture
|
||||
Evidence and `capture_browser_scenario` Evidence, projecting the primary origin's request lines into named tools with path
|
||||
templates, parameters, observed query names, value-free body shapes and record
|
||||
pointers, plus excluded groups and three fixed unknowns. Templates, parameters and
|
||||
names are inferences; the Evidence is inferred whenever any tool or excluded group
|
||||
is templated, and derived otherwise.
|
||||
See [environment manifests](web-network-captures.md#environment-manifest).
|
||||
|
||||
`inspect_analysis_view` also accepts `environment-state` for the same parents,
|
||||
extracting the initial state the recorded reads observed: entity types with an
|
||||
identity field, entities, documents for written non-entity paths, and a render
|
||||
plan per read tool that reproduces its recordings from the state or names why it
|
||||
cannot. Types and identity are inferences, so the Evidence is always inferred.
|
||||
See [environment state](web-network-captures.md#environment-state).
|
||||
|
||||
`inspect_analysis_view` projects a caller-selected view of already completed
|
||||
`inspect_binary_layout`, `analyze_javascript_application`, or `analyze_function`
|
||||
Evidence. Native views select procedure, pseudocode, assembly, basic blocks,
|
||||
@@ -355,6 +461,17 @@ checking that a client received the full catalog. An operation absent from all
|
||||
groups is missing from the complete inventory; it is not an implicit
|
||||
unavailable entry.
|
||||
|
||||
`get_call_graph` and `trace_call_path` report `forward` and `backward` modes.
|
||||
Each requires `procedure_address` plus `procedure_callees` or `procedure_callers`,
|
||||
respectively. The tool is available when either traversal mode is supported;
|
||||
`default_mode_available` describes the default forward direction. Read each mode's
|
||||
missing operations and remediation before selecting a direction. These facts
|
||||
describe traversal prerequisites, not relationship completeness or a guarantee
|
||||
for every selector. A trace whose start and goal resolve to the same address can
|
||||
finish without reading relationships, even when neither traversal mode is
|
||||
available. Discovery does not block that direct invocation or replace its typed
|
||||
partial/error outcomes.
|
||||
|
||||
The `effects` and `annotations` arrays list the true members of their fixed
|
||||
vocabularies. A known member omitted from either array is false. This applies
|
||||
only to these closed flag sets; it does not make an omitted tool, provider, mode,
|
||||
@@ -418,6 +535,96 @@ inspection fails or is cancelled before a provider client exists. If no client
|
||||
remains, no provider run or health observation is available; follow the cleanup
|
||||
error's reported resources.
|
||||
|
||||
## Transports
|
||||
|
||||
stdio is the default transport and the only one that client registrations,
|
||||
`rea mcp doctor` and the MCP registry entry describe. `rea mcp --http [--host H]
|
||||
[--port N]` is an opt-in alternative that serves the same catalog over the
|
||||
streamable HTTP transport at `/mcp`; it replaces stdio for that process rather
|
||||
than running beside it. Host and port come from the flags, then `MCP_HOST` and
|
||||
`MCP_PORT`, then `127.0.0.1` and an ephemeral port. The bound URL is announced
|
||||
as one JSON log line on stderr (`"msg":"MCP transport listening"` with `url`)
|
||||
at every `REA_LOG_LEVEL`, because an ephemeral port is otherwise unreachable;
|
||||
stdout stays unused.
|
||||
|
||||
This endpoint uses plain HTTP without TLS or client authentication. Use it only
|
||||
within a trusted local or isolated environment; an explicit non-loopback bind
|
||||
makes the same tools and active analysis state reachable at that address. It
|
||||
does not provide separate users or independent target sessions. Standard client
|
||||
registrations continue to use a dedicated stdio process.
|
||||
|
||||
The HTTP server is stateful and serves one client at a time on one server
|
||||
instance. `initialize` returns an `Mcp-Session-Id`; `notifications/initialized`
|
||||
is acknowledged with `202`; responses arrive as JSON or SSE frames; `DELETE
|
||||
/mcp` with the session id ends the session with `200`. Ending a session is not
|
||||
a server shutdown: the process keeps listening, the instance keeps its providers
|
||||
and the shared active target, and the next client opens a fresh session against
|
||||
them. A client ending or replacing a session cancels its running tool calls and
|
||||
prompt completions, stops request-body reads and fences further message dispatch,
|
||||
then waits up to ten seconds for request dispatch, handlers and cleanup to finish before reusing the
|
||||
server. A timeout returns `503`; the old calls remain owned, and later requests
|
||||
retry the drain rather than starting another session against those resources.
|
||||
Requests that finish parsing or SDK dispatch after their session closed are
|
||||
refused, and their response streams are cancelled. Tool callbacks whose
|
||||
validation finishes after cancellation never start.
|
||||
Body forwarding stops when SDK request processing ends, including early header
|
||||
and size refusals, rather than remaining active until the session closes.
|
||||
HTTP input/route dispatch is limited to eight concurrent requests by default,
|
||||
shared by MCP and plain routes. Excess requests receive `503` with a closing
|
||||
connection before their bodies are converted, read or cloned. An embedder can
|
||||
select `maxConcurrentDispatches`; the SDK's per-body size limit still applies.
|
||||
MCP responses with fully read or bodyless requests do not hold admission slots,
|
||||
so an idle MCP SSE stream does not prevent later calls. Plain routes retain their
|
||||
slots through response delivery and response-source cancellation cleanup. An
|
||||
unread upload retains its slot until input finishes or either side closes,
|
||||
even with a long response. Sequential keep-alive is supported; a peer pipelining
|
||||
behind an unfinished response is disconnected before queued requests dispatch.
|
||||
Concurrent MCP requests use separate connections.
|
||||
This bounds concurrent input work, not total provider or process memory.
|
||||
Provider-backed prompt completions use the same request owner and drain, so a
|
||||
cancelled completion cannot keep using providers after handover or shutdown.
|
||||
HTTP factories enable `RequestTrackingMcpServer` tracking so completion is tracked
|
||||
even when cancellation suppresses the SDK's reply; stdio retains the SDK's
|
||||
original callback behavior. Full server shutdown stops the listener and waits
|
||||
for MCP work, plain-route handlers and response-source cancellation to settle
|
||||
before running provider shutdown hooks or closing providers. Response producers
|
||||
must finish provider work before closing their stream and expose asynchronous
|
||||
disconnect cleanup through the stream's `cancel()` result. Cancellation does not
|
||||
delay delivery of a HEAD reply; a caller retaining a sibling of a teed response
|
||||
must consume or cancel that sibling before shutdown can finish. Failed response
|
||||
cleanup remains owned, blocks further admission and prevents provider teardown.
|
||||
The server continues cleanup
|
||||
automatically when they finish, including after a client handover timed out.
|
||||
All routes return `503` until the server factory succeeds; startup failure closes
|
||||
the listener and joins its pending replies before rejecting the factory.
|
||||
The newest client replaces an open session: an `initialize` that arrives
|
||||
while a session is open ends that session instead of being refused, because a
|
||||
client that exits without `DELETE` must not wedge the listener; the replaced
|
||||
client's open streams are closed and its later requests are refused. Only an
|
||||
`initialize` that carries the streamable headers (`Content-Type:
|
||||
application/json` and an `Accept` naming both `application/json` and
|
||||
`text/event-stream`) and passes the Host and Origin checks can replace a
|
||||
session; any other sessionless POST goes to the live transport, which refuses
|
||||
it without disturbing the session. Requests that carry no valid session id are
|
||||
refused with `400` or `404` as the SDK transport defines. A bind is loopback by
|
||||
address class, not spelling, and is classified from the address the listener
|
||||
actually bound: any `127.0.0.0/8` address, `::1`, or an IPv4-mapped loopback.
|
||||
Configured names, including `localhost` and its aliases, do not establish that
|
||||
the bound address is local; names are added only after classifying the actual IP.
|
||||
On such a bind, requests whose `Host` header is outside the SDK loopback allowlist plus
|
||||
the configured name and the bound address are refused with `403`, and a browser
|
||||
`Origin` outside that allowlist is refused the same way (a missing `Origin`
|
||||
passes); a non-loopback bind relaxes both checks and logs the relaxation. A reply sent before a request body was read (a refused `Host` or `Origin`, an unknown route, the startup `503`) closes the connection after the reply, so an unread upload cannot keep a socket open. Request targets are taken verbatim as origin-form paths (`//host/path` is a path, not another origin); an absolute-form target, a target containing a raw fragment delimiter (`#`), or one whose path URL parsing would rewrite (dot segments such as `/public/../mcp`), is refused with `400`, and a method Fetch does not support with `405`. Percent-encoded `%23` remains valid query data. Only a lone `initialize` can replace an open session; a batch that carries one is left to the transport's refusal. Scoped
|
||||
IPv6 literals (`fe80::1%eth0`) are refused before binding, because no URL can
|
||||
name them. Listener errors reach the runtime's connection-loss handler only
|
||||
after a successful bind; a bind failure prints the listen-failure message alone. Request bodies are bounded by the SDK's
|
||||
4 MiB limit, and the response budget keeps its stdio-derived floor. The
|
||||
listener ends on SIGINT or SIGTERM only, not on stdin EOF, and exits `130` or
|
||||
`143` after cleanup settles. A signal that arrives while startup is still
|
||||
running is recorded and acted on as soon as the listener and its providers are
|
||||
owned, so startup never ends without cleanup; a second signal during startup
|
||||
exits immediately.
|
||||
|
||||
## Progress and cancellation
|
||||
|
||||
REA accepts ordinary `tools/call` progress tokens. The numeric `progress` counts
|
||||
@@ -652,6 +859,13 @@ the current client, so it is not a fresh-start recovery. On the same connection:
|
||||
`provider_id: "ghidra"`, then retry the first query with an appropriate client
|
||||
deadline. The connection and selected provider do not need to change.
|
||||
|
||||
If REA's provider startup allowance also needs raising, preserve Evidence and
|
||||
close as above, then update `REA_GHIDRA_STARTUP_TIMEOUT_MS` in REA's launch
|
||||
environment and restart the server. Reconnect the MCP client before reopening
|
||||
the target. Configuration is read at process startup; close/reopen on the same
|
||||
connection cannot apply a changed environment. See the
|
||||
[CLI retry example](installation.md#ghidra).
|
||||
|
||||
This close/reopen flow was exercised on one real Linux stdio connection after a
|
||||
controlled startup timeout, followed by a successful overview and function
|
||||
analysis. A separate Windows x64 stdio run with Ghidra 12.1.4, JDK 21 and Node
|
||||
@@ -751,6 +965,12 @@ after SDK conversion and check advertised validation and actual calls.
|
||||
Individual model APIs can impose additional nesting limits; complete producer
|
||||
captures can exceed ten structural levels.
|
||||
|
||||
`search_bytes` advertises its pattern and mask as descriptive strings. Its
|
||||
shared canonical validator scans byte pairs, whitespace, wildcards and aligned
|
||||
masks without a repeating regular expression or temporary byte arrays. The
|
||||
server rejects malformed or unconstrained patterns before provider dispatch;
|
||||
the advertised strings alone do not express those rules.
|
||||
|
||||
Boolean literal constraints in input schemas are relaxed only in this
|
||||
advertisement. For example, a boolean `const` is presented as `type: boolean`;
|
||||
an affected `not` is omitted, conditional predicates and their branches are
|
||||
@@ -857,6 +1077,21 @@ includes all dimension limitations. Equal CFGs establish only equal observed
|
||||
block sizes and successor relationships. Text from different provider
|
||||
identities remains unknown.
|
||||
|
||||
The `references` dimension preserves each edge's direction, endpoint identity
|
||||
and available kind facts. Within one artifact it compares exact raw endpoint
|
||||
coordinates, including the address space; hexadecimal case and zero padding
|
||||
do not change identity. Resolving an endpoint to a containing procedure does
|
||||
not replace the raw coordinate. Unparseable coordinates stay unknown.
|
||||
|
||||
Across artifacts, reference endpoints use stable procedure names. Missing or
|
||||
address-generated names remain unknown. A name observed at multiple procedure
|
||||
entries retains its label constraint but cannot establish equal endpoints.
|
||||
Pairing preserves duplicate counts and considers alternative compatible
|
||||
pairings; array order never identifies an edge. Compatible records with
|
||||
unknown endpoint identity or kind facts remain `unknown`. Independent known
|
||||
facts that contradict every possible pairing still establish `changed`.
|
||||
Source Evidence retains the original addresses, names and unavailable facts.
|
||||
|
||||
### Comparison input shapes
|
||||
|
||||
`compare_web_captures` accepts exactly one of two input shapes:
|
||||
|
||||
+7
-5
@@ -108,11 +108,13 @@ list does not establish that the target has no procedures.
|
||||
| Artifact occurrence | Occurrence IDs present in schema-valid retained artifact inventory pages. |
|
||||
| Residual unknown | Current non-resolved unknown heads only. |
|
||||
|
||||
Suggestions are Unicode-normalized, case-insensitive prefix matches. The server
|
||||
deduplicates them and sorts by code point for deterministic results. Procedure
|
||||
completion derives candidates from the complete inventory, so a name is offered
|
||||
only when it identifies one address in that inventory. A completion reply
|
||||
carries at most 100 values, the MCP limit. `total` counts every match and
|
||||
Prefix matching is Unicode-normalized and case-insensitive; suggestions retain
|
||||
their original spelling. The server deduplicates them and sorts by code point
|
||||
for deterministic results. Procedure completion derives candidates from the
|
||||
complete inventory, so a name is offered only when it identifies one address in
|
||||
that inventory. Every name at a shared address participates in this check,
|
||||
including later rows that establish ambiguity. A completion reply carries at
|
||||
most 100 values, the MCP limit. `total` counts every match and
|
||||
`hasMore` is `true` when values were omitted. Completion has no paging
|
||||
controls. A longer prefix reaches an omitted value only when it excludes other
|
||||
matches; matching uses normalized, case-insensitive forms, so a value can stay
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
# Match native functions across builds
|
||||
|
||||
`match_native_functions` pairs two existing `list_procedures` Evidence inventories. Select `list_procedures` `detail: "summary"` to retain
|
||||
complete source rows without transferring the entire inventory. Its
|
||||
`parent_evidence_id` is the reference matching consumes.
|
||||
Targets may be opened sequentially; matching never opens a target, rereads its
|
||||
file, executes it or starts a provider. Preserve each inventory before closing its
|
||||
target, or import its Evidence bundle later. `close_binary` clears session
|
||||
records. Export each bundle before closing; after collecting both sides and
|
||||
closing the last target, import both bundles, then use their references. Keeping
|
||||
only an Evidence ID across close is insufficient. Complete inline Evidence and
|
||||
CLI `source_bundle` inputs are portable alternatives. Pass Evidence or exact
|
||||
session references as `left` and `right`:
|
||||
|
||||
```json
|
||||
{
|
||||
"left": { "kind": "retained-evidence", "evidence_id": "ev_<left digest>" },
|
||||
"right": { "kind": "retained-evidence", "evidence_id": "ev_<right digest>" }
|
||||
}
|
||||
```
|
||||
|
||||
Unique exact reported names are paired first. Optional `list_names` Evidence
|
||||
identifies dynamic/default names to exclude. If origin is unavailable, the
|
||||
existing automatic-name exclusion rule applies and each function retains that
|
||||
limitation. Exact names establish name correspondence, not unchanged code or
|
||||
original-source identity.
|
||||
|
||||
For unmatched candidates, optionally supply `left_observations` and
|
||||
`right_observations`. `procedure_info` or existing `analyze_function` Evidence
|
||||
gives exact inclusive body ranges; `read_bytes` supplies memory bytes. Observations
|
||||
must belong to the side's exact target/provider/profile and document context.
|
||||
Adjacent, consistent overlapping and partially returned reads compose by their
|
||||
observed extent; conflicting intervals remain gaps. A byte match requires every
|
||||
body byte, equal total size, equal ordered range lengths, a unique SHA-256
|
||||
fingerprint and the same known architecture. Extents are never guessed from the
|
||||
next entry, and relocations are not normalized. A larger mapped read can serve
|
||||
several observed bodies; only needed ranges need to be captured.
|
||||
|
||||
This first slice reuses existing range observations. Stripped builds can require
|
||||
many lightweight range queries when those facts are not retained yet. Matching
|
||||
does not hide that acquisition cost or silently start those queries. Full
|
||||
decompilation is unnecessary.
|
||||
|
||||
Repeated candidates stay `unresolved` groups without a Cartesian pair expansion.
|
||||
`only_left`/`only_right` mean no exact counterpart in the supplied observations;
|
||||
renaming, inlining, stripping, changed bytes or incomplete discovery can explain
|
||||
them. They do not prove source addition/removal. `source_coverage` explicitly keeps
|
||||
inventory completeness unknown and separately counts known byte fingerprints and
|
||||
resolved correspondence. Provider limitations remain accessible.
|
||||
|
||||
Default `detail: "summary"` retains complete matching Evidence in MCP and returns
|
||||
counts, both identities and coverage. Query its `parent_evidence_id`:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "ev_<matching digest>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "function-matches",
|
||||
"collection": "functions",
|
||||
"states": ["matched"],
|
||||
"sides": ["left"],
|
||||
"fields": ["name", "body_sha256"],
|
||||
"offset": 0,
|
||||
"limit": 20
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`collection: "matches"` returns relation identities, states, basis and member
|
||||
counts. Large member lists are omitted by default; select `left`/`right` columns
|
||||
explicitly or page individual `functions`. Batch `match_ids`/`function_ids`, exact function `names`, state
|
||||
and side filters intersect. Mandatory identities and provenance survive column
|
||||
selection. Page coverage covers selected retained facts. Function identity binds
|
||||
inventory Evidence and the owning inventory’s `source_index`, preserving same-address alternatives.
|
||||
One validation, lazy member index and bounded recent filter are retained per
|
||||
immutable parent; no live target state is cached.
|
||||
|
||||
When both functions already have unambiguous `analyze_function` dossiers, a matched
|
||||
relation includes `compare_functions_input` with directly usable retained
|
||||
references. Otherwise analyze just the selected canonical addresses in their
|
||||
respective targets, then compare those dossiers. Matching never claims behavioral
|
||||
equivalence. Optional informative CFG/call/string fingerprints from supplied
|
||||
dossiers connect unresolved candidate groups only. They require observed CFG,
|
||||
callees and strings, complete observed call coverage, no unresolved calls, and
|
||||
the same analysis-profile identity. Empty leaf-function buckets are excluded.
|
||||
`structural_candidate_sha256` identifies that derived candidate signature; its
|
||||
dossier link preserves the full supporting facts. Group membership does not
|
||||
assert that every member pair is a candidate or equivalent.
|
||||
|
||||
CLI uses the same service and JSON input:
|
||||
|
||||
```bash
|
||||
rea match-native-functions request.json \
|
||||
--source_bundle left-bundle.json --source_bundle right-bundle.json \
|
||||
--evidence_output matching-bundle.json --json
|
||||
```
|
||||
|
||||
`source_bundle` resolves exact references from authenticated local bundles.
|
||||
Default summary requires `evidence_output`, retaining full matching Evidence and
|
||||
its selected source closure, including transitive Evidence links across supplied
|
||||
bundles. Missing dependencies fail with an integrity error; unrelated records
|
||||
are omitted. Publication never overwrites. Set `detail: "complete"`
|
||||
in the JSON for all groups inline. Import the bundle into MCP for views, or
|
||||
extract its matching Evidence record to a JSON file for the existing CLI
|
||||
`inspect-analysis-view` `evidence-file` source. Input bundles are unchanged.
|
||||
|
||||
Representation admission is bounded at 96 MiB before cloning selected observations
|
||||
and expanding matching indexes and repeated per-function source links. Work yields to cancellation. No worker/provider
|
||||
concurrency is introduced; this budget is not a bound on total process RSS.
|
||||
|
||||
The self-owned source fixture is [nativeFunctionMatching.c](https://github.com/morluto/rea/blob/main/tests/fixtures/nativeFunctionMatching.c).
|
||||
On Linux x86-64, compile with `cc -O2 -g0 -fno-pie -no-pie -DVERSION=1` and
|
||||
`-DVERSION=2`; copy the second binary and run `strip --strip-all` on that copy.
|
||||
It includes unchanged/changed code, a byte-identical rename, a removed/added
|
||||
observation, an inlined function, identical-code aliases and a renamed changed
|
||||
function with an observed call/string signature. Analyze without executing the
|
||||
fixture. Full-provider verification still needs caller-supplied Ghidra and Java.
|
||||
@@ -216,6 +216,36 @@ sites remain distinct. A static route does not establish runtime reachability.
|
||||
Swift/closure dispatch requires resolved static references; unavailable ABI
|
||||
forms remain unknown.
|
||||
|
||||
Incomplete reference kinds do not discard known typed calls or partial
|
||||
provider call references. A `provider_call` with `observed` resolution retains
|
||||
the provider-reported call site and endpoint without claiming generic direct,
|
||||
indirect or Objective-C dispatch semantics. Its `call_reference` evidence
|
||||
preserves the original endpoints, classification, raw provider type and
|
||||
provenance, even when the target node is a containing procedure. Hopper's
|
||||
`none`, `unknown`, `direct` and `objective_c` labels stay provider-scoped;
|
||||
unknown numeric types remain available in the raw type field. Target-only
|
||||
callee fallback uses `provider_call` with `inferred` resolution.
|
||||
|
||||
Callee fallback adds inferred targets while each retained route keeps its
|
||||
original provider and call site. If enumeration fails, known routes remain alongside an unresolved
|
||||
failure; cancellation still terminates the request. A node-limit omission keeps
|
||||
the route's known relation while marking its target unresolved. Distinct call
|
||||
sites to the same omitted target retain separate evidence.
|
||||
Reports sharing a call site and containing target procedure retain all their
|
||||
original endpoint evidence on the shared graph edge. Different reported
|
||||
relations retain separate edges, independent of enumeration order.
|
||||
|
||||
Session discovery describes two automatic source routes for this trace:
|
||||
`authored_ui_dispatch` needs Interface Builder decoding and dispatch metadata;
|
||||
`native_seed_call_trace` needs native address resolution, dispatch metadata and
|
||||
a call reader. Metadata can come from direct inspection or `list_names`;
|
||||
the call reader can supply typed `procedure_references` or inferred
|
||||
`procedure_callees`. An authored UI-to-handler edge remains useful without
|
||||
call expansion, and native-seed tracing can proceed without UI decoding.
|
||||
These routes add no request option or default mode, so discovery omits
|
||||
`default_mode_available`. Availability describes source prerequisites;
|
||||
the result reports the query's actual matches, coverage and failures.
|
||||
|
||||
`trace_native_values` / `rea trace-native-values <binary> <procedure> --provider
|
||||
ghidra` composes high-p-code def-use graphs across statically resolved calls.
|
||||
Logical CALL arguments bind to recovered parameter ordinals; recovered RETURN
|
||||
@@ -337,6 +367,12 @@ complete observation.
|
||||
Object `kind` describes the decoded archive value: data, dates and numbers
|
||||
remain `scalar` when their JSON value uses a `$plist_type` wrapper. An actual
|
||||
dictionary with the same fields remains `dictionary`.
|
||||
XML must be well-formed. Data literals use
|
||||
[CoreFoundation's raw ASCII quartet rules](https://github.com/swiftlang/swift-corelibs-foundation/blob/384ff680be7c7445fee6ce0c55e9ae11a46e07c5/Sources/CoreFoundation/CFPropertyList.c#L1588):
|
||||
non-alphabet ASCII characters are ignored and an incomplete final quartet
|
||||
emits no bytes. Character references retain their source spelling for data
|
||||
decoding. Non-ASCII content, child markup and self-closing `<data/>` are format
|
||||
failures; explicit `<data></data>` represents empty data.
|
||||
UID 0 cannot distinguish conditional nil from ordinary nil. Limits are
|
||||
64 MiB of input, 200,000 objects/references and 128 nesting levels.
|
||||
XML dictionary entries decoded as `__proto__` are omitted with an explicit
|
||||
@@ -424,6 +460,12 @@ complete observation.
|
||||
address is retained as `reported_address` in the symbol's evidence, whose
|
||||
location also remains unknown. Nonempty provider address syntax is preserved.
|
||||
|
||||
Session discovery reports this workflow available when the bound provider
|
||||
offers direct metadata inspection or a `list_names` symbol inventory. The
|
||||
workflow tries direct metadata first and uses the symbol fallback only when
|
||||
that operation is unavailable; runtime failures remain failures rather than
|
||||
being replaced by symbol-only results.
|
||||
|
||||
## Native desktop observation
|
||||
|
||||
`observe_native_ui` captures one explicitly selected existing PID/window ID.
|
||||
@@ -598,9 +640,11 @@ digests identify the original serialized bytes.
|
||||
|
||||
The archive reader observes XML number element types and binary number markers.
|
||||
XML values and their number facts are interpreted together from the validated
|
||||
DOM. Hexadecimal `<integer>` text, including an optional sign, retains its value;
|
||||
DOM and its original scalar source spans. Hexadecimal `<integer>` text, including
|
||||
an optional sign, retains its value;
|
||||
`<real>` text and CDATA segments are combined without inferring text from nested
|
||||
elements. The XML path does not serialize or reparse character references.
|
||||
elements. Real character references expand outside CDATA; integer and date
|
||||
literals use raw source text.
|
||||
Integers beyond the exact range of a JSON number are reported with the original
|
||||
decimal text as `{ "$plist_type": "integer", "decimal": "<exact digits>" }` when
|
||||
that source value can be associated unambiguously. Integral real values remain
|
||||
@@ -609,9 +653,9 @@ markers remain decoded with an explicit precision limitation. The limitation
|
||||
also counts observed unsafe integer literals, including observations outside the
|
||||
selected object page. Incomplete supplemental metadata produces an explicit
|
||||
precision note without rejecting an archive accepted by the byte decoder.
|
||||
Mixed XML number content (comments, CDATA, processing instructions or child
|
||||
elements) and integer text the supplemental reader cannot classify disable exact
|
||||
association and retain decoded numbers with that precision note. Nonnumeric
|
||||
Malformed XML scalar content becomes an explicit unknown rather than a prefix
|
||||
number, guessed timestamp, or tag-only boolean. These unknowns do not disable
|
||||
exact association for unrelated observed numbers. Nonnumeric
|
||||
`UID` and `CF$UID` dictionary values remain ordinary data and are projected
|
||||
recursively; numeric reference markers retain their original representation.
|
||||
|
||||
@@ -632,11 +676,15 @@ decode budget reserves estimated room for number metadata and attribute
|
||||
projection; these estimates are not a hard process-memory bound. Native
|
||||
NIBArchive decoding is unchanged by this plist-specific representation.
|
||||
|
||||
Dates are reported as `{ "$plist_type": "date", "iso8601": "<timestamp>" }`. A
|
||||
date the decoder cannot represent, such as an XML date with a year beyond 9999
|
||||
or a negative year, is reported as `{ "$plist_type": "date", "iso8601": null }`
|
||||
with a counted limitation; the rest of the plist or archive is still decoded.
|
||||
`inspect_plist` and Interface Builder decoding use the same projection.
|
||||
Dates are reported as `{ "$plist_type": "date", "iso8601": "<timestamp>" }`.
|
||||
XML dates require a complete UTC spelling with fixed two-digit month, day and
|
||||
time fields; short, negative and extended years are interpreted as UTC year
|
||||
fields within JavaScript's date range. Unsupported calendar fields (including
|
||||
leap seconds) and out-of-range dates remain explicit unknowns. REA does not
|
||||
emulate CoreFoundation's normalization of overflowing calendar fields. Invalid
|
||||
dates returned by other decoders retain `{ "$plist_type": "date", "iso8601": null }`
|
||||
with a counted limitation. `inspect_plist` and Interface Builder decoding use
|
||||
the same projection.
|
||||
|
||||
A negative-zero real is reported as `{ "$plist_type": "real", "value": "-0" }`
|
||||
because a JSON number cannot carry the sign of zero. `inspect_plist` and
|
||||
@@ -653,14 +701,20 @@ for those details. XML CFPropertyList literals (`nan`, `+infinity`, `-infinity`,
|
||||
`infinity`, `inf`, `+inf`, `-inf`, case-insensitively) decode to the same known
|
||||
values. Case and aliases normalize; original archive bytes retain source text.
|
||||
|
||||
XML numeric text whose identity the decoder cannot establish, including
|
||||
unrecognized non-finite text and finite literals that overflow the numeric
|
||||
representation, remains `{ "$plist_type": "real", "value": null }` with a
|
||||
counted limitation. The XML decoder marks these values explicitly before shared
|
||||
projection, so they cannot be confused with known binary IEEE values.
|
||||
Incomplete numeric markup retains its existing compatibility behavior described
|
||||
above. Known non-finite and unresolved numeric values have separate counts.
|
||||
Native `inspect_plist` uses these same labels for plutil's XML conversion.
|
||||
XML scalars whose identity cannot be established use
|
||||
`{ "$plist_type": "unknown", "element": "integer" | "real" | "date" | "true" | "false", "literal": "<original content>", "reason": "invalid-literal" | "unsupported-markup" | "unrepresentable" }`.
|
||||
A counted limitation accompanies these wrappers. Literal content retains its
|
||||
original entity spelling, markup and line endings. Whole XML documents must
|
||||
still pass strict well-formedness checks. Empty integer, real and date elements
|
||||
are unknown scalars; boolean elements must be empty. Complete decimal/exponent
|
||||
reals and signed decimal/hexadecimal integers accept their respective native
|
||||
leading whitespace but reject trailing text. Arbitrary-precision integers
|
||||
remain an intentional REA extension beyond CoreFoundation's integer range.
|
||||
Finite real spellings that overflow the numeric representation remain unknown,
|
||||
rather than becoming observed infinities. Known non-finite and unresolved scalar
|
||||
values have separate counts. This replaces the former prefix-number and
|
||||
incomplete-metadata fallback for malformed XML numeric content. Native
|
||||
`inspect_plist` uses the same projection for plutil's XML conversion.
|
||||
|
||||
## Locate a byte signature
|
||||
|
||||
|
||||
@@ -38,6 +38,10 @@ supervision on macOS. REA removes that directory when it exits. REA does not
|
||||
install Xcode, Command Line Tools, or other software. Capability checks prepare
|
||||
this helper before reporting macOS capture as available.
|
||||
|
||||
Stopping process sampling or cancelling an ownership query waits for its issued
|
||||
inspection command to exit. Closing a macOS ownership reader rejects new queries
|
||||
and waits for admitted helper commands before removing their executable directory.
|
||||
|
||||
## Capture a command
|
||||
|
||||
For a small first capture, save this as `scenario.json`:
|
||||
@@ -241,6 +245,15 @@ follow also remain unknown. Partial enumeration of one root does not erase
|
||||
known effects in another, and unavailable content hashes do not by themselves
|
||||
make path absence unknown.
|
||||
|
||||
When a follow-up read fails after an entry's metadata was observed, the
|
||||
snapshot keeps unaffected observations and records an `entry_unavailable`
|
||||
enumeration omission with the root-aliased path and system error code.
|
||||
Successfully observed identity replacements use `identity_changed` with a null
|
||||
system code. Unstable symlink targets are omitted; already observed directory
|
||||
metadata remains available. The affected root stays incomplete, so omitted
|
||||
entries do not prove deletion. A path missing at its initial lookup remains an
|
||||
observed absence. Cancellation and unexpected I/O failures still stop capture.
|
||||
|
||||
Terminal frame `data` is the comparison text after the selected normalization.
|
||||
New captures also preserve `raw_data` when normalization changes a PTY chunk;
|
||||
otherwise `data` is already the original text. Older captures may lack the
|
||||
@@ -419,6 +432,17 @@ Every capture requires `truncation_details`, with separate accounting for:
|
||||
imply incomplete path enumeration.
|
||||
- `process`: sampling limit and whether sampling ended partially. Coverage
|
||||
remains `sampled`, including when that limit was not exhausted.
|
||||
`sampling_failures` counts failed read observations plus at most one
|
||||
unfinished pass interrupted by sampler shutdown. `first_sampling_failure`
|
||||
retains the first attributed read diagnostic, with shutdown context appended
|
||||
when applicable; interruption alone has an explicit shutdown diagnostic.
|
||||
Denied or failed procfs reads mark
|
||||
sampling partial while preserving readable process rows and thread children;
|
||||
a process with unreadable identity is not attributed. Expected process-exit
|
||||
races are separate from failed observations. Shutdown counts only when it
|
||||
interrupts a read or leaves traversal unfinished; a completed pass does not
|
||||
become partial merely because the sampler stops. Completed rows
|
||||
remain available and issued reads settle before final coverage is returned.
|
||||
|
||||
The aggregate `truncated` flag summarizes these observations. Comparisons retain
|
||||
results for unaffected dimensions and mark affected dimensions unknown; an
|
||||
@@ -435,12 +459,36 @@ Each filesystem checkpoint starts with a fresh `file_bytes` budget across all
|
||||
selected roots; it is not a per-file allowance. File contents are read in
|
||||
64 KiB chunks for hashing and are not embedded in capture Evidence.
|
||||
|
||||
Ownership inspections bind run-token reads to native start identities and retain
|
||||
related generations through reparenting, group changes, omitted rows, and cleanup
|
||||
retries. The captured run keeps one evidence owner through settlement and cleanup.
|
||||
An absent or changed token leaves a previously related generation unresolved;
|
||||
current token proof is still required before signaling. Swept cleanup verifies the
|
||||
remaining family after its group signals. Coverage is bounded by the native
|
||||
snapshots and generations inspected; sampled group IDs select candidates.
|
||||
|
||||
When the host withholds an unrelated process’s ownership token, REA leaves that
|
||||
process untouched and records its PID and reason in `cleanup.unverified_processes`
|
||||
and process residual unknowns. Successful cleanup verifies the owned group;
|
||||
it does not attribute those unrelated processes. Related or otherwise unexplained
|
||||
unreadable processes still prevent successful cleanup.
|
||||
|
||||
Failed cleanup retains its exact resources for the owning session to retry.
|
||||
The CLI retries before projecting its error; MCP retries pending cleanup before
|
||||
admitting another capture and during session cleanup. Once every attributed
|
||||
owner confirms release, the live error status clears `cleanup_incomplete`.
|
||||
The original failure, observations and `cleanup_report` retain the facts from
|
||||
the earlier attempt, including any cleanup failure recorded then.
|
||||
|
||||
Snapshot file handles, directory handles, and the selected executable reader
|
||||
follow the same ownership rule. A failed close preserves completed observations
|
||||
and any primary read failure; it cannot turn a completed digest into a successful
|
||||
capture with unresolved cleanup. Interrupted snapshots retain examined entries
|
||||
with incomplete coverage in `partial_observation`, alongside the failure.
|
||||
File-handle retry uses the exact retained owner. After a directory close rejects,
|
||||
Node does not expose enough handle validity to authorize another native close;
|
||||
REA retains that unresolved owner and blocks further captures in the session.
|
||||
|
||||
On macOS, changing a Node process's `process.title` can make its run token
|
||||
unreadable. Node documents that [setting the title overwrites argv memory](https://nodejs.org/download/release/v24.18.0/docs/api/process.html#processtitle).
|
||||
With the pinned toolchain, a live child retained a readable start identity
|
||||
|
||||
@@ -146,6 +146,7 @@ npx rea-agents@latest setup
|
||||
| شيفرة EVM البايتية | محددات التوجيه، وإزاحات البايتات، والمعاملات المستنتجة، وقابلية تغيير الحالة | مدخل محلي يحمل بايتات خامًا أو تمثيلًا سداسيًا عشريًا على Linux x64؛ [دليل EVM دون تشغيل](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| أعطال Linux المسجّلة | سجلات note الخام، ومسجّلات المعالج/الإشارات لكل خيط مسجّل، وترشيحات اختيارية لخرائط الذاكرة | pwntools يوفّره المستدعي على Linux x64؛ GDB/pwndbg اختياريان؛ [الأعطال المسجّلة](../recorded-crashes.md) |
|
||||
| تثبيتات سطح المكتب | عائلات وقت التشغيل، والملفات الداعمة، وعلامات المكونات، والخطوات التالية | فحص محدود للملفات والترويسات;[تطبيقات سطح المكتب](../desktop-applications.md) |
|
||||
| JavaScript / Electron | الوحدات، وعمليات الاستيراد، وخرائط المصدر، والمسارات، وIPC، وعلاقات إضافات الشيفرة الأصلية | Node.js وnpm؛ [تحليل التطبيقات](../javascript-application-workflows.md) |
|
||||
| المواقع | بنية الصفحة، والسكربتات، وملاحظات الشبكة، ولقطات الشاشة المطلوبة | متصفح من عائلة Chrome؛ [تحليل المتصفح](../browser-observation.md) |
|
||||
| تسجيلات الشبكة المحفوظة | الطلبات، والاستجابات، ومحتويات الحمولة المتاحة، ومواضعها في المصدر | HAR؛ وmitmdump على Linux لتسجيلات mitmproxy بصيغتها الأصلية؛ [دليل تسجيلات الشبكة](../web-network-captures.md) |
|
||||
|
||||
@@ -146,6 +146,7 @@ REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätz
|
||||
| EVM-Bytecode | Dispatch-Selektoren, Byte-Offsets, abgeleitete Argumente und Veränderbarkeit des Zustands | Lokale Eingabe mit Rohbytes oder Hexdarstellung unter Linux x64; [Offline-EVM-Leitfaden](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Aufgezeichnete Linux-Abstürze | Unverarbeitete Note-Einträge, Register/Signale aller aufgezeichneten Threads und optionale Zuordnungskandidaten | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; optional GDB/pwndbg; [aufgezeichnete Abstürze](../recorded-crashes.md) |
|
||||
| Desktop-Installationen | Laufzeitkandidaten, Quelldateien, Komponenten und nächste Schritte | Begrenzte Verzeichnis- und Headerprüfung;[Desktop-Anwendungen](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Module, Imports, Source Maps, Routen, IPC und Beziehungen zu nativen Add-ons | Node.js und npm; [Anwendungsanalyse](../javascript-application-workflows.md) |
|
||||
| Websites | Seitenstruktur, Skripte, Netzwerkbeobachtungen und angeforderte Screenshots | Ein Browser aus der Chrome-Familie; [Browseranalyse](../browser-observation.md) |
|
||||
| Gespeicherte Netzwerkaufzeichnungen | Anfragen, Antworten, zugängliche Nutzdaten und Quellpositionen | HAR; mitmdump unter Linux für Aufzeichnungen im nativen mitmproxy-Format; [Leitfaden zu Netzwerkaufzeichnungen](../web-network-captures.md) |
|
||||
|
||||
@@ -153,6 +153,7 @@ npx rea-agents@latest setup
|
||||
| Bytecode EVM | Selectors επιλογής συνάρτησης, μετατοπίσεις σε bytes, εκτιμώμενα ορίσματα και δυνατότητα μεταβολής της κατάστασης | Τοπική είσοδος σε ακατέργαστα bytes ή δεκαεξαδική μορφή σε Linux x64· [οδηγός EVM χωρίς εκτέλεση](../evm-bytecode.md) |
|
||||
| Βάσεις δεδομένων SQLite | Πίνακες, στήλες, ευρετήρια, όψεις, triggers και επιλεγμένες γραμμές κανονικών πινάκων | Τοπικό στιγμιότυπο βάσης δεδομένων/WAL· Node.js 24.x >=24.15 ή 26+ με εγγενή όρια SQLite |
|
||||
| Καταγεγραμμένες καταρρεύσεις στο Linux | Ακατέργαστες εγγραφές notes, καταχωρητές/σήματα κάθε καταγεγραμμένου νήματος και προαιρετικές πιθανές αντιστοιχίσεις μνήμης | pwntools που παρέχει ο χρήστης σε Linux x64· προαιρετικά GDB/pwndbg· [καταγεγραμμένες καταρρεύσεις](../recorded-crashes.md) |
|
||||
| Εγκαταστάσεις επιφάνειας εργασίας | Υποψήφια περιβάλλοντα εκτέλεσης, αρχεία τεκμηρίωσης, στοιχεία και επόμενα βήματα | Περιορισμένη επιθεώρηση καταλόγων και κεφαλίδων;[Εφαρμογές επιφάνειας εργασίας](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Modules, εισαγωγές, source maps, διαδρομές, IPC και σχέσεις με πρόσθετα κώδικα μηχανής | Node.js και npm· [ανάλυση εφαρμογών](../javascript-application-workflows.md) |
|
||||
| Ιστότοποι | Δομή σελίδων, scripts, παρατηρήσεις δικτύου και ζητούμενα στιγμιότυπα οθόνης | Πρόγραμμα περιήγησης της οικογένειας Chrome· [ανάλυση μέσω προγράμματος περιήγησης](../browser-observation.md) |
|
||||
| Αποθηκευμένες καταγραφές δικτύου | Αιτήματα, αποκρίσεις, διαθέσιμα δεδομένα φορτίου και θέσεις στην πηγή | HAR ή SAZ του Fiddler Everywhere· mitmdump σε Linux για την εγγενή μορφή mitmproxy· [οδηγός καταγραφών](../web-network-captures.md) |
|
||||
|
||||
@@ -146,6 +146,7 @@ REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las h
|
||||
| Bytecode EVM | Selectores de despacho, desplazamientos en bytes, argumentos inferidos y mutabilidad | Entrada local con bytes sin procesar o representación hexadecimal, en Linux x64; [guía de EVM sin ejecutar](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Fallos de Linux registrados | Registros note sin procesar, registros/señales de cada hilo registrado y posibles correspondencias de memoria opcionales | pwntools proporcionado por quien llama, en Linux x64; GDB/pwndbg opcionales; [fallos registrados](../recorded-crashes.md) |
|
||||
| Instalaciones de escritorio | Posibles entornos de ejecución, archivos de evidencia, componentes y siguientes pasos | Inspección acotada de directorios y cabeceras;[Aplicaciones de escritorio](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Módulos, importaciones, mapas de código fuente, rutas, IPC y relaciones con complementos nativos | Node.js y npm; [análisis de aplicaciones](../javascript-application-workflows.md) |
|
||||
| Sitios web | Estructura de la página, scripts, observaciones de red y capturas de pantalla solicitadas | Un navegador de la familia Chrome; [análisis del navegador](../browser-observation.md) |
|
||||
| Capturas de red guardadas | Solicitudes, respuestas, contenido accesible de las cargas útiles y ubicaciones de origen | HAR; mitmdump en Linux para capturas en formato nativo de mitmproxy; [guía de capturas](../web-network-captures.md) |
|
||||
|
||||
@@ -138,6 +138,7 @@ REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، بههمراه np
|
||||
| بایتکد EVM | selectorهای dispatch، offsetهای بایتی، آرگومانهای استنباطشده و قابلیت تغییر وضعیت | دادهٔ خام/hex محلی روی Linux x64؛ [راهنمای EVM آفلاین](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| کرشهای ثبتشدهٔ لینوکس | یادداشتهای خام، رجیسترها/سیگنالهای تمام threadهای ثبتشده و گزینههای احتمالی نگاشت | pwntools تأمینشده توسط فراخواننده روی Linux x64؛ GDB/pwndbg اختیاری؛ [کرشهای ثبتشده](../recorded-crashes.md) |
|
||||
| نصبهای دسکتاپ | نامزدهای زمان اجرا، فایلهای شاهد، نشانههای مؤلفهها و مراحل بعدی | بررسی محدود پوشهها و سرآیندها;[برنامههای دسکتاپ](../desktop-applications.md) |
|
||||
| JavaScript / Electron | ماژولها، importها، source mapها، مسیرها، IPC و ارتباط افزونههای بومی | Node.js و npm؛ [تحلیل برنامه](../javascript-application-workflows.md) |
|
||||
| وبسایتها | ساختار صفحه، اسکریپتها، مشاهدات شبکه و اسکرینشاتهای درخواستی | مرورگر مبتنی بر Chrome؛ [تحلیل مرورگر](../browser-observation.md) |
|
||||
| دادههای ضبطشدهٔ شبکه | درخواستها، پاسخها، payloadهای آشکارشده و مکان آنها در منبع | HAR؛ ابزار mitmdump در لینوکس برای ضبطهای بومی mitmproxy؛ [راهنمای ضبط شبکه](../web-network-captures.md) |
|
||||
|
||||
@@ -150,6 +150,7 @@ REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les
|
||||
| Bytecode EVM | Sélecteurs de dispatch, offsets en octets, arguments inférés et mutabilité | Fichier local brut ou hexadécimal sous Linux x64 ; [guide EVM hors ligne](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Crashs Linux enregistrés | Notes brutes, registres et signaux de chaque thread enregistré, et correspondances de mappings facultatives | pwntools fourni par l'utilisateur sous Linux x64 ; GDB/pwndbg en option ; [crashs enregistrés](../recorded-crashes.md) |
|
||||
| Installations de bureau | Environnements candidats, fichiers justificatifs, composants et étapes suivantes | Inspection bornée des répertoires et en-têtes;[Applications de bureau](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC et relations avec les modules natifs | Node.js et npm ; [analyse d'applications](../javascript-application-workflows.md) |
|
||||
| Sites web | Structure des pages, scripts, observations réseau et captures d'écran demandées | Un navigateur de la famille Chrome ; [analyse du navigateur](../browser-observation.md) |
|
||||
| Captures réseau enregistrées | Requêtes, réponses, payloads exposés et emplacements dans le code source | HAR ; mitmdump sous Linux pour les captures mitmproxy natives ; [guide des captures](../web-network-captures.md) |
|
||||
|
||||
@@ -153,6 +153,7 @@ Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis:
|
||||
| Bytecode EVM | Selector dispatch, offset byte, argumen yang disimpulkan, dan sifat mutabilitas | Pembawa data raw/hex lokal; [panduan EVM offline](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Crash Linux yang direkam | Catatan mentah, register/sinyal setiap thread yang direkam, dan kandidat pemetaan opsional | pwntools yang disediakan pemanggil; GDB/pwndbg opsional; [crash yang direkam](../recorded-crashes.md) |
|
||||
| Instalasi desktop | Kandidat runtime, berkas pendukung, penanda komponen, dan langkah berikutnya | Pemeriksaan direktori dan header terbatas;[Aplikasi desktop](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](../javascript-application-workflows.md) |
|
||||
| Situs web | Struktur halaman, skrip, pengamatan jaringan, dan tangkapan layar yang diminta | Peramban berbasis Chrome; [analisis peramban](../browser-observation.md) |
|
||||
| Rekaman jaringan tersimpan | Permintaan, respons, payload yang terekspos, dan lokasi sumber | HAR; mitmdump di Linux untuk rekaman native mitmproxy; [panduan rekaman](../web-network-captures.md) |
|
||||
|
||||
@@ -150,6 +150,7 @@ REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strument
|
||||
| Bytecode EVM | Selettori di dispatch, offset dei byte, argomenti inferiti e mutabilità | Carrier raw/hex locale; [guida EVM offline](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Crash Linux registrati | Note grezze, registri/segnai di ogni thread registrato e candidati mapping opzionali | pwntools fornito dal chiamante; GDB/pwndbg opzionale; [crash registrati](../recorded-crashes.md) |
|
||||
| Installazioni desktop | Runtime candidati, file di supporto, componenti e passi successivi | Ispezione limitata di directory e intestazioni;[Applicazioni desktop](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Moduli, import, source map, route, IPC e relazioni con add-on nativi | Node.js e npm; [analisi delle applicazioni](../javascript-application-workflows.md) |
|
||||
| Siti web | Struttura della pagina, script, osservazioni di rete e screenshot richiesti | Un browser della famiglia Chrome; [analisi browser](../browser-observation.md) |
|
||||
| Acquisizioni di rete salvate | Richieste, risposte, payload esposti e posizioni sorgente | HAR; mitmdump su Linux per acquisizioni mitmproxy native; [guida alle acquisizioni](../web-network-captures.md) |
|
||||
|
||||
@@ -146,6 +146,7 @@ REA には Node.js 22.x(>=22.19)、24.x(>=24.11)、または 26+ と npm
|
||||
| EVM バイトコード | ディスパッチセレクター、バイトオフセット、推定された引数と状態変更の可否 | Linux x64 上のローカルの生バイト/16 進数の入力ファイル;[オフライン EVM ガイド](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| 記録済みの Linux クラッシュ | 生の note レコード、記録された全スレッドのレジスター/シグナル、任意のマッピング候補 | Linux x64 で利用者が用意する pwntools;GDB/pwndbg は任意;[記録済みクラッシュ](../recorded-crashes.md) |
|
||||
| デスクトップのインストール先 | ランタイム候補、根拠ファイル、コンポーネントの目印、次の調査手順 | 上限付きのディレクトリ・ヘッダー検査;[デスクトップアプリ](../desktop-applications.md) |
|
||||
| JavaScript / Electron | モジュール、インポート、ソースマップ、ルート、IPC、ネイティブアドオンの関係 | Node.js と npm;[アプリケーション分析](../javascript-application-workflows.md) |
|
||||
| ウェブサイト | ページ構造、スクリプト、ネットワークの観測結果、依頼されたスクリーンショット | Chrome 系ブラウザー;[ブラウザー分析](../browser-observation.md) |
|
||||
| 保存済みのネットワークキャプチャ | リクエスト、レスポンス、取得可能なペイロード、ソース内の位置 | HAR;mitmproxy ネイティブ形式のキャプチャには Linux 上の mitmdump;[キャプチャガイド](../web-network-captures.md) |
|
||||
|
||||
+30
-25
@@ -117,6 +117,10 @@ REA는 빠르게 바뀌며 새 릴리스에는 버그 수정이 자주 포함됩
|
||||
rea update
|
||||
```
|
||||
|
||||
`rea --update`는 이 검증된 업데이트 절차의 별칭이며 같은 전역 출력 플래그를 사용할 수 있습니다. 다른 명령 제어 옵션과 함께 사용할 때는 `rea update`를 사용하세요.
|
||||
|
||||
업데이트 명령은 stdout과 stderr를 합쳐 최대 4 MiB까지만 보관합니다. 한도를 넘긴 메타데이터 응답은 명령이 종료된 뒤 거부됩니다. 상세 출력을 사용하는 설치에서는 진단 정보를 한도 내에서만 보관하고 설치를 계속 진행합니다. 사람이 읽는 진행 출력은 터미널이 받아들이는 속도에 맞춰 전달되며, 이미 제출한 쓰기가 끝날 때까지 기다립니다.
|
||||
|
||||
Bun 전역 업데이트에는 Bun 1.2.17 이상이 필요합니다. 이전 버전의 Bun을 사용한다면 `rea update`를 실행하기 전에 Bun을 직접 업그레이드하세요.
|
||||
|
||||
에이전트 등록과 스킬을 갱신하려면 업데이트 과정에서 출력된 설정 명령을 실행하세요.
|
||||
@@ -143,31 +147,32 @@ npx rea-agents@latest setup
|
||||
|
||||
REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니다. 추가 도구와 호스트 지원은 대상에 따라 다릅니다.
|
||||
|
||||
| 대상 | REA가 반환하는 내용 | 요구 사항 및 가이드 |
|
||||
| --------------------- | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 네이티브 바이너리 | 의사 코드, 어셈블리, 문자열, 심볼, 호출, 참조 | Hopper, Ghidra 또는 IDA; [네이티브 분석](../native-investigation.md) |
|
||||
| 오프라인 ELF 레이아웃 | 섹션, 세그먼트, 원본 심볼/재배치 정보, 정적 분석으로 얻은 보안 완화 기법 후보 | Linux x64에서 호출자가 제공하는 pwntools; [바이너리 진단](../binary-diagnostics.md) |
|
||||
| Go 바이너리 | 내장된 컴파일러 버전, 모듈, `replace` 지시어, 빌드 설정 | REA 내장 ELF/PE/thin Mach-O 정적 검사; Go 설치 불필요 |
|
||||
| Rust 바이너리 | cargo-auditable이 내장한 크레이트 이름, 버전, 소스, 의존성 관계 | REA 내장 ELF/PE/thin Mach-O 정적 검사; [Rust 바이너리](../rust-binaries.md) |
|
||||
| WASM 아티팩트 | 유효성 검사, 섹션, import/export의 정확한 WAT 형식, 디코딩된 WAT | 호출자 제공 WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM 바이트코드 | 디스패치 선택자, 바이트 오프셋, 추론한 인자와 상태 변경 가능 여부 | Linux x64의 로컬 원시 바이트/16진수 입력 파일; [오프라인 EVM 가이드](../evm-bytecode.md) |
|
||||
| SQLite 데이터베이스 | 테이블, 열, 인덱스, 뷰, 트리거, 선택한 일반 테이블의 행 | 로컬 데이터베이스/WAL 스냅샷; Node.js 24.15 이상의 24.x 또는 26 이상(네이티브 SQLite 한도 적용) |
|
||||
| 기록된 Linux 크래시 | 원시 note 레코드, 기록된 모든 스레드의 레지스터/시그널, 선택적 매핑 후보 | Linux x64에서 호출자가 제공하는 pwntools; GDB/pwndbg는 선택 사항; [기록된 크래시](../recorded-crashes.md) |
|
||||
| JavaScript / Electron | 모듈, 임포트, 소스 맵, 라우트, IPC, 네이티브 애드온 관계 | Node.js와 npm; Linux: procps 호환 `ps`; [애플리케이션 분석](../javascript-application-workflows.md) |
|
||||
| 웹사이트 | 페이지 구조, 스크립트, 네트워크 관찰 결과, 요청한 스크린샷 | Chrome 계열 브라우저; [브라우저 분석](../browser-observation.md) |
|
||||
| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR 또는 Fiddler Everywhere SAZ; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](../web-network-captures.md) |
|
||||
| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 별도 도구 불필요(정적 검사만); [관리 코드 가이드](../managed-code-analysis.md) |
|
||||
| Android APK | 매니페스트 선언, 클래스, 디컴파일된 메서드, 참조 | Linux/macOS/Windows x64의 헤드리스 JADX와 전체 JDK; [Android 가이드](../android-analysis.md) |
|
||||
| Android 기기 | 기기, 패키지, 프로세스, 로그, 덤프, 화면 캡처, APK/파일 전송 | 호출자 제공 adb(에뮬레이터 또는 실제 기기); [ADB 가이드](../adb-device-analysis.md) |
|
||||
| JEB 프로젝트 | 엔진 기반 프로젝트 유닛, 아티팩트 해시, 타입/메서드 의사코드 | 호출자가 시작하고 MCP를 제공하는 JEB 클라이언트(REA_JEB_MCP_URL로 선택); [JEB 가이드](../jeb-analysis.md) |
|
||||
| Android 리소스 | 디코딩된 매니페스트, 문자열 테이블, 로캘, 버전 정보 | 호출자 제공 Apktool 런처; [리소스 가이드](../apktool-resource-analysis.md) |
|
||||
| Flutter 앱 | Dart 스냅샷 해시, ABI별 라이브러리 해시, 엔진 빌드 출처 | 없음 (정적 APK 파싱); [Flutter 가이드](../flutter-analysis.md) |
|
||||
| 펌웨어 | 영역, 추출 결과, 네이티브 분석으로 전달할 정보 | Linux의 Binwalk / Unblob; [펌웨어 가이드](../firmware-analysis.md) |
|
||||
| 기록된 패킷 | 선택 필드, 반복 값, 필터 및 디코딩 규칙 | Linux: TShark 4.x + prlimit; [캡처 가이드](../packet-captures.md) |
|
||||
| 패킷 요약 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos(Linux에서 검증됨); [pcap 가이드](../pcap-analysis.md) |
|
||||
| 기록된 논리 신호 | 디코더 주석, 샘플 범위, 진단 정보 | Linux: sigrok-cli 0.7.2 + prlimit; [신호 기록 가이드](../signal-recordings.md) |
|
||||
| 패키지 및 리소스 | 파일 목록, 다이제스트, plist, Apple 번들 구조, 추출한 리소스 | [아티팩트 및 JavaScript 가이드](../javascript-artifact-reconstruction.md), [Apple 애플리케이션](../apple-application-analysis.md) |
|
||||
| 프로세스 동작 | 터미널 출력, 상호작용, 종료 및 파일 시스템 관찰 결과, 실행 비교 | 네이티브 PTY를 지원하는 Linux/macOS; [프로세스 캡처](../process-capture.md) |
|
||||
| 대상 | REA가 반환하는 내용 | 요구 사항 및 가이드 |
|
||||
| ---------------------- | ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| 네이티브 바이너리 | 의사 코드, 어셈블리, 문자열, 심볼, 호출, 참조 | Hopper, Ghidra 또는 IDA; [네이티브 분석](../native-investigation.md) |
|
||||
| 오프라인 ELF 레이아웃 | 섹션, 세그먼트, 원본 심볼/재배치 정보, 정적 분석으로 얻은 보안 완화 기법 후보 | Linux x64에서 호출자가 제공하는 pwntools; [바이너리 진단](../binary-diagnostics.md) |
|
||||
| Go 바이너리 | 내장된 컴파일러 버전, 모듈, `replace` 지시어, 빌드 설정 | REA 내장 ELF/PE/thin Mach-O 정적 검사; Go 설치 불필요 |
|
||||
| Rust 바이너리 | cargo-auditable이 내장한 크레이트 이름, 버전, 소스, 의존성 관계 | REA 내장 ELF/PE/thin Mach-O 정적 검사; [Rust 바이너리](../rust-binaries.md) |
|
||||
| WASM 아티팩트 | 유효성 검사, 섹션, import/export의 정확한 WAT 형식, 디코딩된 WAT | 호출자 제공 WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM 바이트코드 | 디스패치 선택자, 바이트 오프셋, 추론한 인자와 상태 변경 가능 여부 | Linux x64의 로컬 원시 바이트/16진수 입력 파일; [오프라인 EVM 가이드](../evm-bytecode.md) |
|
||||
| SQLite 데이터베이스 | 테이블, 열, 인덱스, 뷰, 트리거, 선택한 일반 테이블의 행 | 로컬 데이터베이스/WAL 스냅샷; Node.js 24.15 이상의 24.x 또는 26 이상(네이티브 SQLite 한도 적용) |
|
||||
| 기록된 Linux 크래시 | 원시 note 레코드, 기록된 모든 스레드의 레지스터/시그널, 선택적 매핑 후보 | Linux x64에서 호출자가 제공하는 pwntools; GDB/pwndbg는 선택 사항; [기록된 크래시](../recorded-crashes.md) |
|
||||
| 데스크톱 설치 디렉터리 | 런타임 후보, 근거 파일, 구성 요소 표시와 다음 조사 단계 | 범위가 제한된 디렉터리 및 헤더 검사;[데스크톱 애플리케이션](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 모듈, 임포트, 소스 맵, 라우트, IPC, 네이티브 애드온 관계 | Node.js와 npm; Linux: procps 호환 `ps`; [애플리케이션 분석](../javascript-application-workflows.md) |
|
||||
| 웹사이트 | 페이지 구조, 스크립트, 네트워크 관찰 결과, 요청한 스크린샷 | Chrome 계열 브라우저; [브라우저 분석](../browser-observation.md) |
|
||||
| 저장된 네트워크 캡처 | 요청, 응답, 접근 가능한 페이로드, 소스 위치 | HAR 또는 Fiddler Everywhere SAZ; mitmproxy 네이티브 캡처에는 Linux의 mitmdump; [캡처 가이드](../web-network-captures.md) |
|
||||
| .NET 어셈블리 | 메타데이터, CIL 명령, 선언된 네이티브 의존성, 빌드 비교 | 별도 도구 불필요(정적 검사만); [관리 코드 가이드](../managed-code-analysis.md) |
|
||||
| Android APK | 매니페스트 선언, 클래스, 디컴파일된 메서드, 참조 | Linux/macOS/Windows x64의 헤드리스 JADX와 전체 JDK; [Android 가이드](../android-analysis.md) |
|
||||
| Android 기기 | 기기, 패키지, 프로세스, 로그, 덤프, 화면 캡처, APK/파일 전송 | 호출자 제공 adb(에뮬레이터 또는 실제 기기); [ADB 가이드](../adb-device-analysis.md) |
|
||||
| JEB 프로젝트 | 엔진 기반 프로젝트 유닛, 아티팩트 해시, 타입/메서드 의사코드 | 호출자가 시작하고 MCP를 제공하는 JEB 클라이언트(REA_JEB_MCP_URL로 선택); [JEB 가이드](../jeb-analysis.md) |
|
||||
| Android 리소스 | 디코딩된 매니페스트, 문자열 테이블, 로캘, 버전 정보 | 호출자 제공 Apktool 런처; [리소스 가이드](../apktool-resource-analysis.md) |
|
||||
| Flutter 앱 | Dart 스냅샷 해시, ABI별 라이브러리 해시, 엔진 빌드 출처 | 없음 (정적 APK 파싱); [Flutter 가이드](../flutter-analysis.md) |
|
||||
| 펌웨어 | 영역, 추출 결과, 네이티브 분석으로 전달할 정보 | Linux의 Binwalk / Unblob; [펌웨어 가이드](../firmware-analysis.md) |
|
||||
| 기록된 패킷 | 선택 필드, 반복 값, 필터 및 디코딩 규칙 | Linux: TShark 4.x + prlimit; [캡처 가이드](../packet-captures.md) |
|
||||
| 패킷 요약 | 엔드포인트, DNS, TLS SNI, 프로토콜 분포, HTTP 요청 | 호출자 제공 tshark 및 capinfos(Linux에서 검증됨); [pcap 가이드](../pcap-analysis.md) |
|
||||
| 기록된 논리 신호 | 디코더 주석, 샘플 범위, 진단 정보 | Linux: sigrok-cli 0.7.2 + prlimit; [신호 기록 가이드](../signal-recordings.md) |
|
||||
| 패키지 및 리소스 | 파일 목록, 다이제스트, plist, Apple 번들 구조, 추출한 리소스 | [아티팩트 및 JavaScript 가이드](../javascript-artifact-reconstruction.md), [Apple 애플리케이션](../apple-application-analysis.md) |
|
||||
| 프로세스 동작 | 터미널 출력, 상호작용, 종료 및 파일 시스템 관찰 결과, 실행 비교 | 네이티브 PTY를 지원하는 Linux/macOS; [프로세스 캡처](../process-capture.md) |
|
||||
|
||||
정적 JavaScript 및 .NET 검사는 제공된 파일을 읽으며 애플리케이션을 실행하지 않습니다. 런타임 캡처는 사용자의 권한으로 선택한 대상을 실행하거나 대상과 상호작용합니다. 각 런타임 가이드에서 그 영향을 설명합니다.
|
||||
|
||||
|
||||
@@ -155,6 +155,7 @@ REA-യ്ക്ക് Node.js 22.x (>=22.19), 24.x (>=24.11), അല്ലെ
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, അനുമാനിച്ച arguments, mutability | Linux x64-ൽ local raw/hex carrier; [offline EVM guide](../evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers, തിരഞ്ഞെടുത്ത സാധാരണ table rows | Local database/WAL snapshot; native SQLite പരിമിതികളോടെ Node.js 24.x >=24.15 അല്ലെങ്കിൽ 26+ |
|
||||
| Recorded Linux crashes | Raw notes, രേഖപ്പെടുത്തിയ ഓരോ thread-ന്റെയും registers/signals, ഐച്ഛിക mapping candidates | Linux x64-ൽ caller നൽകുന്ന pwntools; ഐച്ഛിക GDB/pwndbg; [recorded crashes](../recorded-crashes.md) |
|
||||
| ഡെസ്ക്ടോപ്പ് ഇൻസ്റ്റലേഷനുകൾ | റൺടൈം സാധ്യതകൾ, തെളിവ് ഫയലുകൾ, ഘടക സൂചനകൾ, തുടർ നടപടികൾ | പരിധിയുള്ള ഡയറക്ടറി, ഹെഡർ പരിശോധന;[ഡെസ്ക്ടോപ്പ് ആപ്ലിക്കേഷനുകൾ](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC, native add-on ബന്ധങ്ങൾ | Node.js, npm; Linux: procps-compatible `ps`; [application analysis](../javascript-application-workflows.md) |
|
||||
| Websites | Page structure, scripts, network observations, ആവശ്യപ്പെട്ട screenshots | Chrome-family browser; [browser analysis](../browser-observation.md) |
|
||||
| Saved network captures | Requests, responses, പുറത്തുകാണുന്ന payload-ുകൾ, source locations | HAR അല്ലെങ്കിൽ Fiddler Everywhere SAZ; native mitmproxy-നായി Linux-ൽ mitmdump; [capture guide](../web-network-captures.md) |
|
||||
|
||||
@@ -146,6 +146,7 @@ REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe na
|
||||
| Kod bajtowy EVM | Selektory wywołań, przesunięcia bajtowe, wywnioskowane argumenty i mutowalność | Lokalny plik z surowymi bajtami lub zapisem szesnastkowym w systemie Linux x64; [przewodnik EVM offline](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Zarejestrowane awarie systemu Linux | Surowe rekordy note, rejestry/sygnały każdego zarejestrowanego wątku i opcjonalni kandydaci na mapowania | pwntools dostarczone przez użytkownika w systemie Linux x64; opcjonalnie GDB/pwndbg; [zarejestrowane awarie](../recorded-crashes.md) |
|
||||
| Instalacje desktopowe | Kandydaci środowisk uruchomieniowych, pliki źródłowe dowodów, komponenty i dalsze kroki | Ograniczona inspekcja katalogów i nagłówków;[Aplikacje desktopowe](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Moduły, importy, mapy źródeł, trasy, IPC i powiązania z natywnymi dodatkami | Node.js i npm; [analiza aplikacji](../javascript-application-workflows.md) |
|
||||
| Strony internetowe | Struktura strony, skrypty, obserwacje sieciowe i zamówione zrzuty ekranu | Przeglądarka z rodziny Chrome; [analiza w przeglądarce](../browser-observation.md) |
|
||||
| Zapisane przechwycenia ruchu sieciowego | Żądania, odpowiedzi, dostępne dane przesyłane i lokalizacje źródłowe | HAR; mitmdump w systemie Linux dla natywnych przechwyceń mitmproxy; [przewodnik przechwytywania](../web-network-captures.md) |
|
||||
|
||||
@@ -146,6 +146,7 @@ O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As fer
|
||||
| Bytecode EVM | Seletores de despacho, offsets de bytes, argumentos inferidos e mutabilidade | Entrada local com bytes brutos ou representação hexadecimal no Linux x64; [guia de EVM sem execução](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Falhas do Linux registradas | Registros note brutos, registradores/sinais de cada thread registrada e candidatos opcionais a mapeamentos | pwntools fornecido pelo chamador no Linux x64; GDB/pwndbg opcionais; [falhas registradas](../recorded-crashes.md) |
|
||||
| Instalações desktop | Ambientes candidatos, arquivos de evidência, componentes e próximos passos | Inspeção limitada de diretórios e cabeçalhos;[Aplicações desktop](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Módulos, importações, source maps, rotas, IPC e relações com complementos nativos | Node.js e npm; [análise de aplicativos](../javascript-application-workflows.md) |
|
||||
| Sites | Estrutura da página, scripts, observações de rede e capturas de tela solicitadas | Um navegador da família Chrome; [análise do navegador](../browser-observation.md) |
|
||||
| Capturas de rede salvas | Requisições, respostas, payloads acessíveis e localizações de origem | HAR; mitmdump no Linux para capturas no formato nativo do mitmproxy; [guia de capturas](../web-network-captures.md) |
|
||||
|
||||
+23
-22
@@ -139,28 +139,29 @@ npx rea-agents@latest setup
|
||||
|
||||
REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а также npm. Дополнительные инструменты и поддерживаемые системы зависят от цели:
|
||||
|
||||
| Цель | Что возвращает REA | Требования и руководство |
|
||||
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Нативные бинарные файлы | Псевдокод, ассемблер, строки, символы, вызовы и ссылки | Hopper, Ghidra или IDA; [нативный анализ](../native-investigation.md) |
|
||||
| Структура ELF без запуска | Секции, сегменты, исходные символы/релокации и возможные механизмы защиты по данным статического анализа | pwntools, предоставленный вызывающей стороной, в Linux x64; [диагностика бинарных файлов](../binary-diagnostics.md) |
|
||||
| Байт-код EVM | Селекторы диспетчеризации, смещения байтов, предполагаемые аргументы и изменяемость состояния | Локальные необработанные байты или шестнадцатеричное представление в Linux x64; [руководство по EVM без запуска](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Записанные сбои Linux | Необработанные записи note, регистры/сигналы каждого записанного потока и необязательные кандидаты сопоставления с памятью | pwntools, предоставленный вызывающей стороной, в Linux x64; необязательные GDB/pwndbg; [записанные сбои](../recorded-crashes.md) |
|
||||
| JavaScript / Electron | Модули, импорты, карты исходного кода, маршруты, IPC и связи с нативными дополнениями | Node.js и npm; [анализ приложений](../javascript-application-workflows.md) |
|
||||
| Сайты | Структура страницы, скрипты, наблюдения за сетью и запрошенные снимки экрана | Браузер семейства Chrome; [анализ браузера](../browser-observation.md) |
|
||||
| Сохранённые сетевые захваты | Запросы, ответы, доступные полезные данные и позиции в источнике | HAR; mitmdump в Linux для захватов в собственном формате mitmproxy; [руководство по сетевым захватам](../web-network-captures.md) |
|
||||
| Сборки .NET | Метаданные, инструкции CIL, объявленные нативные зависимости и сравнения сборок | Статическое исследование; [руководство по управляемому коду](../managed-code-analysis.md) |
|
||||
| Android APK | Объявления манифеста, классы, декомпилированные методы и ссылки | JADX без графического интерфейса и полный JDK в Linux/macOS/Windows x64; [руководство по Android](../android-analysis.md) |
|
||||
| Устройства Android | Устройства, пакеты, процессы, журналы, дампы, снимки экрана и передача APK/файлов | adb, предоставленный вызывающей стороной (эмулятор или устройство); [руководство по ADB](../adb-device-analysis.md) |
|
||||
| Проекты JEB | Единицы проекта от движка, дайджесты артефактов и псевдокод типов/методов | Запущенный вызывающей стороной клиент JEB с интерфейсом MCP, выбирается через REA_JEB_MCP_URL; [руководство по JEB](../jeb-analysis.md) |
|
||||
| Ресурсы Android | Декодированный манифест, таблицы строк, языки и сведения о версии | Запуск Apktool, предоставленный вызывающей стороной; [руководство по ресурсам](../apktool-resource-analysis.md) |
|
||||
| Приложения Flutter | Хеш снапшота Dart, дайджесты библиотек по ABI и происхождение сборки движка | Не требуется (статический разбор APK); [руководство по Flutter](../flutter-analysis.md) |
|
||||
| Прошивки | Области, результаты извлечения и передача в нативный анализ | Binwalk / Unblob в Linux; [руководство по прошивкам](../firmware-analysis.md) |
|
||||
| Записанные пакеты | Выбранные поля, повторяющиеся значения, фильтры и правила декодирования | Linux: TShark 4.x + prlimit; [руководство по захватам](../packet-captures.md) |
|
||||
| Захват пакетов | Конечные точки, DNS, TLS SNI, распределение протоколов и HTTP-запросы | Предоставленные вызывающей стороной tshark и capinfos; [руководство по pcap](../pcap-analysis.md) |
|
||||
| Записанные логические сигналы | Аннотации декодера, диапазоны отсчётов и диагностика | Linux: sigrok-cli 0.7.2 + prlimit; [руководство по сигналам](../signal-recordings.md) |
|
||||
| Пакеты и ресурсы | Списки файлов, хеши, plist, структура пакетов Apple и извлечённые ресурсы | [Руководство по артефактам и JavaScript](../javascript-artifact-reconstruction.md), [приложения Apple](../apple-application-analysis.md) |
|
||||
| Поведение процессов | Вывод терминала, взаимодействия, наблюдения за завершением и файловой системой, сравнения запусков | Linux/macOS с нативным PTY; [захват процессов](../process-capture.md) |
|
||||
| Цель | Что возвращает REA | Требования и руководство |
|
||||
| ----------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Нативные бинарные файлы | Псевдокод, ассемблер, строки, символы, вызовы и ссылки | Hopper, Ghidra или IDA; [нативный анализ](../native-investigation.md) |
|
||||
| Структура ELF без запуска | Секции, сегменты, исходные символы/релокации и возможные механизмы защиты по данным статического анализа | pwntools, предоставленный вызывающей стороной, в Linux x64; [диагностика бинарных файлов](../binary-diagnostics.md) |
|
||||
| Байт-код EVM | Селекторы диспетчеризации, смещения байтов, предполагаемые аргументы и изменяемость состояния | Локальные необработанные байты или шестнадцатеричное представление в Linux x64; [руководство по EVM без запуска](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Записанные сбои Linux | Необработанные записи note, регистры/сигналы каждого записанного потока и необязательные кандидаты сопоставления с памятью | pwntools, предоставленный вызывающей стороной, в Linux x64; необязательные GDB/pwndbg; [записанные сбои](../recorded-crashes.md) |
|
||||
| Установленные настольные приложения | Возможные среды выполнения, файлы с доказательствами, компоненты и дальнейшие шаги | Ограниченная проверка каталогов и заголовков;[Настольные приложения](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Модули, импорты, карты исходного кода, маршруты, IPC и связи с нативными дополнениями | Node.js и npm; [анализ приложений](../javascript-application-workflows.md) |
|
||||
| Сайты | Структура страницы, скрипты, наблюдения за сетью и запрошенные снимки экрана | Браузер семейства Chrome; [анализ браузера](../browser-observation.md) |
|
||||
| Сохранённые сетевые захваты | Запросы, ответы, доступные полезные данные и позиции в источнике | HAR; mitmdump в Linux для захватов в собственном формате mitmproxy; [руководство по сетевым захватам](../web-network-captures.md) |
|
||||
| Сборки .NET | Метаданные, инструкции CIL, объявленные нативные зависимости и сравнения сборок | Статическое исследование; [руководство по управляемому коду](../managed-code-analysis.md) |
|
||||
| Android APK | Объявления манифеста, классы, декомпилированные методы и ссылки | JADX без графического интерфейса и полный JDK в Linux/macOS/Windows x64; [руководство по Android](../android-analysis.md) |
|
||||
| Устройства Android | Устройства, пакеты, процессы, журналы, дампы, снимки экрана и передача APK/файлов | adb, предоставленный вызывающей стороной (эмулятор или устройство); [руководство по ADB](../adb-device-analysis.md) |
|
||||
| Проекты JEB | Единицы проекта от движка, дайджесты артефактов и псевдокод типов/методов | Запущенный вызывающей стороной клиент JEB с интерфейсом MCP, выбирается через REA_JEB_MCP_URL; [руководство по JEB](../jeb-analysis.md) |
|
||||
| Ресурсы Android | Декодированный манифест, таблицы строк, языки и сведения о версии | Запуск Apktool, предоставленный вызывающей стороной; [руководство по ресурсам](../apktool-resource-analysis.md) |
|
||||
| Приложения Flutter | Хеш снапшота Dart, дайджесты библиотек по ABI и происхождение сборки движка | Не требуется (статический разбор APK); [руководство по Flutter](../flutter-analysis.md) |
|
||||
| Прошивки | Области, результаты извлечения и передача в нативный анализ | Binwalk / Unblob в Linux; [руководство по прошивкам](../firmware-analysis.md) |
|
||||
| Записанные пакеты | Выбранные поля, повторяющиеся значения, фильтры и правила декодирования | Linux: TShark 4.x + prlimit; [руководство по захватам](../packet-captures.md) |
|
||||
| Захват пакетов | Конечные точки, DNS, TLS SNI, распределение протоколов и HTTP-запросы | Предоставленные вызывающей стороной tshark и capinfos; [руководство по pcap](../pcap-analysis.md) |
|
||||
| Записанные логические сигналы | Аннотации декодера, диапазоны отсчётов и диагностика | Linux: sigrok-cli 0.7.2 + prlimit; [руководство по сигналам](../signal-recordings.md) |
|
||||
| Пакеты и ресурсы | Списки файлов, хеши, plist, структура пакетов Apple и извлечённые ресурсы | [Руководство по артефактам и JavaScript](../javascript-artifact-reconstruction.md), [приложения Apple](../apple-application-analysis.md) |
|
||||
| Поведение процессов | Вывод терминала, взаимодействия, наблюдения за завершением и файловой системой, сравнения запусков | Linux/macOS с нативным PTY; [захват процессов](../process-capture.md) |
|
||||
|
||||
Статическое исследование JavaScript и .NET читает предоставленные файлы, не запуская приложение. Захват во время выполнения запускает выбранную цель или взаимодействует с ней с правами вашего пользователя; соответствующие руководства описывают последствия.
|
||||
|
||||
|
||||
@@ -146,6 +146,7 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26
|
||||
| ไบต์โค้ด EVM | ตัวเลือกฟังก์ชันสำหรับการเรียก ออฟเซ็ตไบต์ อาร์กิวเมนต์และความสามารถในการเปลี่ยนสถานะที่อนุมานได้ | ไฟล์บนเครื่องที่มีไบต์ดิบหรือข้อมูลเลขฐานสิบหกบน Linux x64; [คู่มือ EVM แบบออฟไลน์](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ข้อมูลการแครชของ Linux ที่บันทึกไว้ | ระเบียน note ดิบ รีจิสเตอร์/สัญญาณของทุกเธรดที่บันทึกไว้ และตัวเลือกการแมปเพิ่มเติม | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; เลือกใช้ GDB/pwndbg เพิ่มเติมได้; [การแครชที่บันทึกไว้](../recorded-crashes.md) |
|
||||
| ไดเรกทอรีติดตั้งแอปเดสก์ท็อป | รันไทม์ที่เป็นไปได้ ไฟล์หลักฐาน ตัวบ่งชี้ส่วนประกอบ และขั้นตอนถัดไป | ตรวจสอบไดเรกทอรีและส่วนหัวโดยจำกัดขอบเขต;[แอปเดสก์ท็อป](../desktop-applications.md) |
|
||||
| JavaScript / Electron | โมดูล การนำเข้า ซอร์สแมป เส้นทาง IPC และความสัมพันธ์กับส่วนเสริมเนทีฟ | Node.js และ npm; [การวิเคราะห์แอปพลิเคชัน](../javascript-application-workflows.md) |
|
||||
| เว็บไซต์ | โครงสร้างหน้า สคริปต์ ข้อมูลเครือข่ายที่สังเกตได้ และภาพหน้าจอที่ร้องขอ | เบราว์เซอร์ในตระกูล Chrome; [การวิเคราะห์ผ่านเบราว์เซอร์](../browser-observation.md) |
|
||||
| ข้อมูลเครือข่ายที่บันทึกไว้ | คำขอ คำตอบ เพย์โหลดที่ปรากฏ และตำแหน่งต้นทาง | HAR; mitmdump บน Linux สำหรับข้อมูลที่บันทึกในรูปแบบเนทีฟของ mitmproxy; [คู่มือการบันทึกข้อมูล](../web-network-captures.md) |
|
||||
|
||||
@@ -151,6 +151,7 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç
|
||||
| EVM bayt kodu | Yönlendirme seçicileri, bayt ofsetleri, çıkarımla belirlenen argümanlar ve durum değiştirilebilirliği | Linux x64 üzerinde yerel ham bayt/onaltılık girdi; [çevrimdışı EVM rehberi](../evm-bytecode.md) |
|
||||
| SQLite veritabanları | Tablolar, sütunlar, index'ler, view'lar, trigger'lar ve seçilen sıradan tablo satırları | Yerel database/WAL snapshot'ı; native SQLite sınırlarıyla Node.js 24.x >=24.15 veya 26+ |
|
||||
| Kaydedilmiş Linux çökmeleri | Ham note kayıtları, kaydedilen her iş parçacığının yazmaçları/sinyalleri ve isteğe bağlı eşleme adayları | Linux x64 üzerinde çağıranın sağladığı pwntools; isteğe bağlı GDB/pwndbg; [kaydedilmiş çökmeler](../recorded-crashes.md) |
|
||||
| Masaüstü kurulumları | Çalışma zamanı adayları, kanıt dosyaları, bileşen işaretleri ve sonraki adımlar | Sınırlı dizin ve başlık incelemesi;[Masaüstü uygulamaları](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Modüller, içe aktarmalar, kaynak haritaları, rotalar, IPC ve yerel eklenti ilişkileri | Node.js ve npm; [uygulama analizi](../javascript-application-workflows.md) |
|
||||
| Web siteleri | Sayfa yapısı, betikler, ağ gözlemleri ve istenen ekran görüntüleri | Chrome ailesinden bir tarayıcı; [tarayıcı analizi](../browser-observation.md) |
|
||||
| Kaydedilmiş ağ yakalamaları | İstekler, yanıtlar, erişilebilir yük içerikleri ve kaynak konumları | HAR veya Fiddler Everywhere SAZ; Linux'ta native mitmproxy için mitmdump; [ağ yakalama rehberi](../web-network-captures.md) |
|
||||
|
||||
@@ -146,6 +146,7 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та
|
||||
| Байт-код EVM | Селектори диспетчеризації, зміщення байтів, виведені припущення про аргументи та змінюваність стану | Локальні необроблені байти або шістнадцяткове представлення у Linux x64; [посібник EVM без запуску](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Записані аварійні завершення в Linux | Необроблені записи note, регістри/сигнали кожного записаного потоку й необов'язкові кандидати зіставлення пам'яті | pwntools, наданий стороною, що викликає інструмент, у Linux x64; необов'язкові GDB/pwndbg; [записані аварійні завершення](../recorded-crashes.md) |
|
||||
| Встановлені настільні програми | Можливі середовища виконання, файли доказів, компоненти та наступні кроки | Обмежена перевірка каталогів і заголовків;[Настільні програми](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Модулі, імпорти, карти вихідного коду, маршрути, IPC і зв'язки з нативними доповненнями | Node.js і npm; [аналіз застосунків](../javascript-application-workflows.md) |
|
||||
| Сайти | Структура сторінки, скрипти, мережеві спостереження й запитані знімки екрана | Браузер сімейства Chrome; [аналіз браузера](../browser-observation.md) |
|
||||
| Збережені мережеві захоплення | Запити, відповіді, доступні корисні дані та місця в джерелі | HAR; mitmdump у Linux для захоплень у власному форматі mitmproxy; [посібник мережевих захоплень](../web-network-captures.md) |
|
||||
|
||||
+23
-22
@@ -139,28 +139,29 @@ Tác nhân gọi REA qua MCP để kiểm tra mục tiêu và lần theo mã li
|
||||
|
||||
REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với npm. Công cụ bổ sung và nền tảng chạy REA được hỗ trợ tùy thuộc vào mục tiêu:
|
||||
|
||||
| Mục tiêu | REA trả về gì | Yêu cầu và hướng dẫn |
|
||||
| ----------------------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Tệp nhị phân mã máy | Mã giả, hợp ngữ, chuỗi, ký hiệu, lời gọi và tham chiếu | Hopper, Ghidra hoặc IDA; [phân tích mã máy](../native-investigation.md) |
|
||||
| Bố cục ELF ngoại tuyến | Các section, segment, ký hiệu/thông tin tái định vị gốc và các biện pháp giảm thiểu có thể có từ phân tích tĩnh | pwntools do bên gọi cung cấp trên Linux x64; [chẩn đoán tệp nhị phân](../binary-diagnostics.md) |
|
||||
| Bytecode EVM | Bộ chọn điều phối, độ lệch byte, tham số suy luận và khả năng thay đổi trạng thái | Đầu vào cục bộ chứa byte thô hoặc dạng thập lục phân trên Linux x64; [hướng dẫn EVM ngoại tuyến](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Sự cố Linux đã ghi lại | Bản ghi note thô, thanh ghi/tín hiệu của mọi luồng được ghi lại và các ứng viên ánh xạ tùy chọn | pwntools do bên gọi cung cấp trên Linux x64; GDB/pwndbg tùy chọn; [sự cố đã ghi lại](../recorded-crashes.md) |
|
||||
| JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](../javascript-application-workflows.md) |
|
||||
| Trang web | Cấu trúc trang, script, quan sát mạng và ảnh chụp màn hình được yêu cầu | Trình duyệt thuộc họ Chrome; [phân tích trình duyệt](../browser-observation.md) |
|
||||
| Bản ghi lưu lượng mạng đã lưu | Yêu cầu, phản hồi, payload có thể truy cập và vị trí nguồn | HAR; mitmdump trên Linux cho bản ghi định dạng gốc của mitmproxy; [hướng dẫn bản ghi mạng](../web-network-captures.md) |
|
||||
| Bắt gói tin | Điểm cuối, DNS, TLS SNI, phân bố giao thức và yêu cầu HTTP | tshark và capinfos do nơi gọi cung cấp; [hướng dẫn pcap](../pcap-analysis.md) |
|
||||
| Assembly .NET | Siêu dữ liệu, lệnh CIL, các phụ thuộc mã máy được khai báo và so sánh bản dựng | Kiểm tra tĩnh; [hướng dẫn mã được quản lý](../managed-code-analysis.md) |
|
||||
| APK Android | Khai báo manifest, lớp, phương thức được dịch ngược và tham chiếu | JADX không giao diện đồ họa và JDK đầy đủ trên Linux/macOS/Windows x64; [hướng dẫn Android](../android-analysis.md) |
|
||||
| Thiết bị Android | Thiết bị, gói, tiến trình, nhật ký, dump, ảnh chụp màn hình và truyền APK/tệp | adb do nơi gọi cung cấp (trình giả lập hoặc thiết bị); [hướng dẫn ADB](../adb-device-analysis.md) |
|
||||
| Dự án JEB | Đơn vị dự án từ engine, hash của artifact và mã giả của kiểu/phương thức | Một ứng dụng JEB do nơi gọi khởi động, phục vụ MCP, chọn bằng REA_JEB_MCP_URL; [hướng dẫn JEB](../jeb-analysis.md) |
|
||||
| Tài nguyên Android | Manifest đã giải mã, bảng chuỗi, ngôn ngữ và thông tin phiên bản | Trình khởi chạy Apktool do nơi gọi cung cấp; [hướng dẫn tài nguyên](../apktool-resource-analysis.md) |
|
||||
| Ứng dụng Flutter | Băm snapshot Dart, hash thư viện theo ABI và nguồn gốc build engine | Không cần (phân tích APK tĩnh); [hướng dẫn Flutter](../flutter-analysis.md) |
|
||||
| Phần sụn | Các vùng, kết quả trích xuất và dữ liệu chuyển sang phân tích mã máy | Binwalk / Unblob trên Linux; [hướng dẫn phần sụn](../firmware-analysis.md) |
|
||||
| Gói tin đã ghi | Trường được chọn, giá trị lặp, bộ lọc và quy tắc giải mã | Linux: TShark 4.x + prlimit; [hướng dẫn bản ghi gói tin](../packet-captures.md) |
|
||||
| Tín hiệu logic đã ghi | Chú thích bộ giải mã, phạm vi mẫu và chẩn đoán | Linux: sigrok-cli 0.7.2 + prlimit; [hướng dẫn tín hiệu](../signal-recordings.md) |
|
||||
| Gói và tài nguyên | Danh sách tệp, giá trị băm, plist, cấu trúc bundle Apple và tài nguyên được trích xuất | [Hướng dẫn hiện vật phần mềm và JavaScript](../javascript-artifact-reconstruction.md), [ứng dụng Apple](../apple-application-analysis.md) |
|
||||
| Hành vi tiến trình | Đầu ra terminal, tương tác, quan sát khi thoát và hệ thống tệp, cùng so sánh các lần chạy | Linux/macOS có PTY gốc; [ghi lại tiến trình](../process-capture.md) |
|
||||
| Mục tiêu | REA trả về gì | Yêu cầu và hướng dẫn |
|
||||
| --------------------------------- | --------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Tệp nhị phân mã máy | Mã giả, hợp ngữ, chuỗi, ký hiệu, lời gọi và tham chiếu | Hopper, Ghidra hoặc IDA; [phân tích mã máy](../native-investigation.md) |
|
||||
| Bố cục ELF ngoại tuyến | Các section, segment, ký hiệu/thông tin tái định vị gốc và các biện pháp giảm thiểu có thể có từ phân tích tĩnh | pwntools do bên gọi cung cấp trên Linux x64; [chẩn đoán tệp nhị phân](../binary-diagnostics.md) |
|
||||
| Bytecode EVM | Bộ chọn điều phối, độ lệch byte, tham số suy luận và khả năng thay đổi trạng thái | Đầu vào cục bộ chứa byte thô hoặc dạng thập lục phân trên Linux x64; [hướng dẫn EVM ngoại tuyến](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Sự cố Linux đã ghi lại | Bản ghi note thô, thanh ghi/tín hiệu của mọi luồng được ghi lại và các ứng viên ánh xạ tùy chọn | pwntools do bên gọi cung cấp trên Linux x64; GDB/pwndbg tùy chọn; [sự cố đã ghi lại](../recorded-crashes.md) |
|
||||
| Thư mục cài đặt ứng dụng máy tính | Các runtime có khả năng được dùng, tệp bằng chứng, dấu hiệu thành phần và bước tiếp theo | Kiểm tra thư mục và phần đầu tệp có giới hạn;[Ứng dụng máy tính](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](../javascript-application-workflows.md) |
|
||||
| Trang web | Cấu trúc trang, script, quan sát mạng và ảnh chụp màn hình được yêu cầu | Trình duyệt thuộc họ Chrome; [phân tích trình duyệt](../browser-observation.md) |
|
||||
| Bản ghi lưu lượng mạng đã lưu | Yêu cầu, phản hồi, payload có thể truy cập và vị trí nguồn | HAR; mitmdump trên Linux cho bản ghi định dạng gốc của mitmproxy; [hướng dẫn bản ghi mạng](../web-network-captures.md) |
|
||||
| Bắt gói tin | Điểm cuối, DNS, TLS SNI, phân bố giao thức và yêu cầu HTTP | tshark và capinfos do nơi gọi cung cấp; [hướng dẫn pcap](../pcap-analysis.md) |
|
||||
| Assembly .NET | Siêu dữ liệu, lệnh CIL, các phụ thuộc mã máy được khai báo và so sánh bản dựng | Kiểm tra tĩnh; [hướng dẫn mã được quản lý](../managed-code-analysis.md) |
|
||||
| APK Android | Khai báo manifest, lớp, phương thức được dịch ngược và tham chiếu | JADX không giao diện đồ họa và JDK đầy đủ trên Linux/macOS/Windows x64; [hướng dẫn Android](../android-analysis.md) |
|
||||
| Thiết bị Android | Thiết bị, gói, tiến trình, nhật ký, dump, ảnh chụp màn hình và truyền APK/tệp | adb do nơi gọi cung cấp (trình giả lập hoặc thiết bị); [hướng dẫn ADB](../adb-device-analysis.md) |
|
||||
| Dự án JEB | Đơn vị dự án từ engine, hash của artifact và mã giả của kiểu/phương thức | Một ứng dụng JEB do nơi gọi khởi động, phục vụ MCP, chọn bằng REA_JEB_MCP_URL; [hướng dẫn JEB](../jeb-analysis.md) |
|
||||
| Tài nguyên Android | Manifest đã giải mã, bảng chuỗi, ngôn ngữ và thông tin phiên bản | Trình khởi chạy Apktool do nơi gọi cung cấp; [hướng dẫn tài nguyên](../apktool-resource-analysis.md) |
|
||||
| Ứng dụng Flutter | Băm snapshot Dart, hash thư viện theo ABI và nguồn gốc build engine | Không cần (phân tích APK tĩnh); [hướng dẫn Flutter](../flutter-analysis.md) |
|
||||
| Phần sụn | Các vùng, kết quả trích xuất và dữ liệu chuyển sang phân tích mã máy | Binwalk / Unblob trên Linux; [hướng dẫn phần sụn](../firmware-analysis.md) |
|
||||
| Gói tin đã ghi | Trường được chọn, giá trị lặp, bộ lọc và quy tắc giải mã | Linux: TShark 4.x + prlimit; [hướng dẫn bản ghi gói tin](../packet-captures.md) |
|
||||
| Tín hiệu logic đã ghi | Chú thích bộ giải mã, phạm vi mẫu và chẩn đoán | Linux: sigrok-cli 0.7.2 + prlimit; [hướng dẫn tín hiệu](../signal-recordings.md) |
|
||||
| Gói và tài nguyên | Danh sách tệp, giá trị băm, plist, cấu trúc bundle Apple và tài nguyên được trích xuất | [Hướng dẫn hiện vật phần mềm và JavaScript](../javascript-artifact-reconstruction.md), [ứng dụng Apple](../apple-application-analysis.md) |
|
||||
| Hành vi tiến trình | Đầu ra terminal, tương tác, quan sát khi thoát và hệ thống tệp, cùng so sánh các lần chạy | Linux/macOS có PTY gốc; [ghi lại tiến trình](../process-capture.md) |
|
||||
|
||||
Kiểm tra tĩnh JavaScript và .NET đọc các tệp được cung cấp mà không chạy ứng dụng. Ghi lại khi chạy sẽ khởi chạy hoặc tương tác với mục tiêu đã chọn bằng quyền người dùng của bạn; từng hướng dẫn mô tả tác động của nó.
|
||||
|
||||
|
||||
@@ -150,6 +150,7 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
|
||||
| EVM 位元組碼 | 分派選擇器、位元組偏移、推斷的參數和狀態可變性 | Linux x64 上的本地原始位元組/十六進位輸入載體;[離線 EVM 指南](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| 已記錄的 Linux 崩潰 | 原始 note 記錄、每個已記錄執行緒的暫存器/訊號及可選的對映候選項 | Linux x64 上由呼叫方提供的 pwntools;可選的 GDB/pwndbg;[已記錄崩潰](../recorded-crashes.md) |
|
||||
| 桌面安裝目錄 | 執行階段候選、支持檔案、元件標記與後續操作 | 有界的目錄與檔頭檢查;[桌面應用](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm;[應用分析](../javascript-application-workflows.md) |
|
||||
| 網站 | 頁面結構、腳本、網路觀察結果和按請求獲取的螢幕擷取畫面 | Chrome 系瀏覽器;[瀏覽器分析](../browser-observation.md) |
|
||||
| 已保存的網路擷取 | 請求、回應、可訪問的酬載和來源位置 | HAR;原生 mitmproxy 擷取需要 Linux 上的 mitmdump;[擷取指南](../web-network-captures.md) |
|
||||
|
||||
@@ -150,6 +150,7 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
|
||||
| EVM 字节码 | 分派选择器、字节偏移、推断的参数和状态可变性 | Linux x64 上的本地原始字节/十六进制输入载体;[离线 EVM 指南](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| 已记录的 Linux 崩溃 | 原始 note 记录、每个已记录线程的寄存器/信号及可选的映射候选项 | Linux x64 上由调用方提供的 pwntools;可选的 GDB/pwndbg;[已记录崩溃](../recorded-crashes.md) |
|
||||
| 桌面安装目录 | 运行时候选、支持文件、组件标记和后续操作 | 有界的目录与文件头检查;[桌面应用](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm;[应用分析](../javascript-application-workflows.md) |
|
||||
| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器;[浏览器分析](../browser-observation.md) |
|
||||
| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR;原生 mitmproxy 捕获需要 Linux 上的 mitmdump;[捕获指南](../web-network-captures.md) |
|
||||
|
||||
@@ -105,6 +105,27 @@ GDB/Python/pwndbg/strace. Fixture generation explicitly launches its owned test
|
||||
program; subsequent inspection only reads the resulting core. Real acceptance
|
||||
checks recorded threads, high registers, missing/malformed notes, opaque owners,
|
||||
CLI/MCP parity, optional debugger context and historical PID collision with an
|
||||
owned live sentinel. The syscall trace also rejects access to the still-present
|
||||
owned live sentinel. An observed Node thread group must launch the exact ownership
|
||||
`ps` inventory before it can read that live process's `stat` or `status` for native
|
||||
identity/UID facts. Only traced `CLONE_THREAD` members share this allowance;
|
||||
exec-replaced threads, ordinary children, debugger and decoder processes cannot
|
||||
use it. The purpose of stat/status reads within the same Node thread group
|
||||
cannot be distinguished by strace. A second recording uses a valid
|
||||
historical PID above the host's assignable PID range and rejects resolved lookups,
|
||||
including Node `stat`/`status` reads, to detect historical-PID-driven access separately
|
||||
from live inventory. Descriptor paths come from strace's `-yy` decoding; exact
|
||||
PID-directory opens and relative accesses through those handles are rejected.
|
||||
Failed FD-relative pathname attempts are checked across syscall variants.
|
||||
Legacy relative pathname calls without dirfd/cwd evidence and symlink aliases
|
||||
remain unverified; the verification report declares these gaps rather than
|
||||
establishing their absence.
|
||||
One syscall/selector contract supplies trace selection, target interpretation
|
||||
and the report's explicit coverage catalog. Unlisted PID-directed syscalls remain
|
||||
unverified. The catalog includes PID-directed resource
|
||||
limits, scheduling, priority, process metadata, signals and pidfd controls.
|
||||
Self, group and UID selectors retain their syscall-defined meanings; unresolved
|
||||
targets cannot qualify. Coverage is reported once for all CLI/MCP traces and
|
||||
does not establish containment of every possible kernel interaction.
|
||||
The syscall trace also rejects access to the still-present
|
||||
source executable. The CI workflow defines Ubuntu 24.04 and Debian 13 (GDB 16)
|
||||
container lanes. Original inputs and host configuration remain unchanged.
|
||||
@@ -0,0 +1,112 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"repository": "morluto/rea",
|
||||
"version": "6.7.0",
|
||||
"base": "2c4338db1fdb7773e05376301c558cff2a0b4b80",
|
||||
"target": "b83b9caf4a5f0241a5fd370792f1aed1bffbb2cf",
|
||||
"commits": [
|
||||
{
|
||||
"sha": "8cbaf6c7021b97dd4de6b5771c35fbed06b240e9",
|
||||
"subject": "fix(tests): isolate command finalization module mocks",
|
||||
"body": "fix(tests): isolate command finalization module mocks\n\nMove the five registered WASM finalization cases into the existing per-file isolated project. Preserve all result, exit-code, observation and cleanup assertions. A preloaded shared CLI graph reproduces three failures before the move; the isolated covered packet passes.\n\nFixes #4959\n",
|
||||
"parents": ["2c4338db1fdb7773e05376301c558cff2a0b4b80"],
|
||||
"authors": [
|
||||
{
|
||||
"name": "morluto",
|
||||
"email": "76467478+morluto@users.noreply.github.com",
|
||||
"user": {
|
||||
"login": "morluto",
|
||||
"__typename": "User"
|
||||
}
|
||||
}
|
||||
],
|
||||
"pullRequests": []
|
||||
},
|
||||
{
|
||||
"sha": "6b180783ed04ebc87548c8385c80e05725804c47",
|
||||
"subject": "fix(tests): budget CLI coverage from parsed options",
|
||||
"body": "fix(tests): budget CLI coverage from parsed options\n\nUse the pinned Vitest parser for coverage and shard flags. Keep ordinary CLI workers at 384 MiB and covered workers at the tested 512 MiB with unchanged concurrency. Retain complete coverage and add actual-config regressions for false, provider-only and spaced-shard options.\n\nValidation: 27 focused cases; 16 covered cases with CLI preloaded; 5 docs checks; typecheck/lint; exact discovery of 1578 files. Measured covered control: 22.18s elapsed, 46.52s CPU, about 2.33 GiB sampled summed process-family RSS on macOS arm64. Full Linux CI pending. No runtime/device verification claimed.\n\nFixes #4960\n",
|
||||
"parents": ["8cbaf6c7021b97dd4de6b5771c35fbed06b240e9"],
|
||||
"authors": [
|
||||
{
|
||||
"name": "morluto",
|
||||
"email": "76467478+morluto@users.noreply.github.com",
|
||||
"user": {
|
||||
"login": "morluto",
|
||||
"__typename": "User"
|
||||
}
|
||||
}
|
||||
],
|
||||
"pullRequests": []
|
||||
},
|
||||
{
|
||||
"sha": "bb212ffecb47e129cc569b9867b2b6e7c9b65685",
|
||||
"subject": "fix(tests): inherit server identity deadline policy",
|
||||
"body": "fix(tests): inherit server identity deadline policy\n\nRemove per-case ten-second overrides from MCP catalog and identity checks. Keep the suite-owned bounded ordinary and coverage deadlines, all assertions, and session cleanup.\n\nFixes #4967\n",
|
||||
"parents": ["6b180783ed04ebc87548c8385c80e05725804c47"],
|
||||
"authors": [
|
||||
{
|
||||
"name": "morluto",
|
||||
"email": "76467478+morluto@users.noreply.github.com",
|
||||
"user": {
|
||||
"login": "morluto",
|
||||
"__typename": "User"
|
||||
}
|
||||
}
|
||||
],
|
||||
"pullRequests": []
|
||||
},
|
||||
{
|
||||
"sha": "b83b9caf4a5f0241a5fd370792f1aed1bffbb2cf",
|
||||
"subject": "fix(setup): edit the duplicate JSON member that clients read (#4957)",
|
||||
"body": "fix(setup): edit the duplicate JSON member that clients read (#4957)\n\nJSON.parse, JSONC parse, and native Claude Code keep the last duplicate\nobject member, and parseClientConfiguration therefore validates the last\none. The shared setup/uninstall serializer instead edited text through\njsonc-parser's modify and findNodeAtLocation, which select the first\nmatching member, so setup and uninstall on configurations with repeated\nserver tables or rea members wrote into a shadowed member and failed\nreadback.\n\nLocate every JSON text edit by the effective (last) member in one place,\nreplacing the first-member modify path. Removing an effective member\nthat shadows an earlier duplicate would activate that duplicate, whatever\nits ownership, so the serializer now returns a tagged shadowed_member\nfailure and setup and uninstall refuse before writing a backup or the\nconfiguration.\n\nFixes #4952\n\nCo-authored-by: Claude Opus 5.5 <noreply@anthropic.com>",
|
||||
"parents": ["bb212ffecb47e129cc569b9867b2b6e7c9b65685"],
|
||||
"authors": [
|
||||
{
|
||||
"name": "Kaoru0822",
|
||||
"email": "kanadekitauji@proton.me",
|
||||
"user": {
|
||||
"login": "kaoru0822-kitauji",
|
||||
"__typename": "User"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "Claude Opus 5.5",
|
||||
"email": "noreply@anthropic.com",
|
||||
"user": {
|
||||
"login": "claude",
|
||||
"__typename": "User"
|
||||
}
|
||||
}
|
||||
],
|
||||
"pullRequests": [4957]
|
||||
}
|
||||
],
|
||||
"pullRequests": [
|
||||
{
|
||||
"number": 4957,
|
||||
"title": "fix(setup): edit the duplicate JSON member that clients read",
|
||||
"url": "https://github.com/morluto/rea/pull/4957",
|
||||
"author": {
|
||||
"login": "kaoru0822-kitauji",
|
||||
"__typename": "User"
|
||||
},
|
||||
"mergeCommit": {
|
||||
"oid": "b83b9caf4a5f0241a5fd370792f1aed1bffbb2cf"
|
||||
},
|
||||
"issues": [
|
||||
{
|
||||
"number": 4952,
|
||||
"url": "https://github.com/morluto/rea/issues/4952",
|
||||
"author": {
|
||||
"login": "xu91102",
|
||||
"__typename": "User"
|
||||
},
|
||||
"repository": {
|
||||
"nameWithOwner": "morluto/rea"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -18,8 +18,11 @@ can run.
|
||||
|
||||
For DWARF, REA does not follow local debug links by default. Set `follow_debug_links` to `true` to pass `-WK`; external debug-file identities are not bound to Evidence when following links is enabled. The CLI flag is `--follow-debug-links`; `--no-follow-debug-links` or `--follow-debug-links=false` keeps the default. A separate `true` or `false` word after the flag is not its value. REA clears `DEBUGINFOD_URLS` in the objdump child environment to disable debuginfod network access on GNU binutils versions that do not support `-WE`. The complete raw output is authoritative; REA does not normalize it into a version-independent disassembly schema.
|
||||
|
||||
For a known native symbol, use `operation: "disassemble", symbol: "my_function"` in MCP or `--symbol my_function` in CLI. REA passes `--disassemble=my_function` to objdump before collecting output, rather than producing the whole disassembly and filtering it afterward. Omit `symbol` to keep full code-section disassembly. Other operations reject this selector. Use a nonempty, NUL-free name with well-formed Unicode; unsupported values are rejected before reading the artifact or starting objdump. Symbol names are passed unchanged; matching, versioned symbols, aliases and the end of the disassembly follow the installed GNU objdump's semantics. Evidence records the selector and actual arguments. An empty match does not establish artifact-wide absence, and the selected observation does not contain the omitted disassembly; request the complete operation when needed.
|
||||
|
||||
```sh
|
||||
rea inspect-with-objdump ./program disassemble
|
||||
rea inspect-with-objdump ./program disassemble --symbol my_function
|
||||
rea inspect-with-objdump ./program dwarf
|
||||
rea inspect-with-objdump ./program dwarf --follow-debug-links
|
||||
```
|
||||
@@ -42,15 +45,17 @@ close_gdb_session {"session_id":"…"}
|
||||
|
||||
## Rizin
|
||||
|
||||
`start_rizin_debug_session` and `rea debug-with-rizin` require an explicit `inferior_tty` on Linux. Rizin launches the target with stdout and stderr routed to that caller-selected TTY and `dbg.exitkills=false`; the target can therefore continue running and writing to the TTY after REA closes the debugger. This is a visible target output effect, and the caller is responsible for selecting a suitable terminal that remains available. Before closing, REA serializes after the active command, parses the structured `olj` descriptor list, detaches only a unique `ptrace://` descriptor, confirms its absence, reads back `dbg.exitkills=false`, and verifies `/proc/<pid>/fd/1` and `/proc/<pid>/fd/2` still identify the selected TTY by device and inode. Any malformed or ambiguous readback keeps the Rizin session owned for retry. Persistent Rizin debugger sessions are unavailable outside Linux because this verification depends on `/proc`.
|
||||
`start_rizin_debug_session` and `rea debug-with-rizin` require an explicit `inferior_tty` on Linux. Rizin launches the target with stdout and stderr routed to that caller-selected TTY and `dbg.exitkills=false`; the target can therefore continue running and writing to the TTY after REA closes the debugger. This is a visible target output effect, and the caller is responsible for selecting a suitable terminal that remains available. Before closing, REA serializes after the active command, parses the structured `olj` descriptor list, detaches only a unique `ptrace://` descriptor, confirms its absence, reads back `dbg.exitkills=false`, and verifies `/proc/<pid>/fd/1` and `/proc/<pid>/fd/2` still identify the selected TTY by device and inode. If a previously identified inferior exits after Rizin confirms detach, REA verifies that the original process has ended instead of requiring output descriptors from an exited process. Unresolved identity or ambiguous readback keeps the Rizin session owned for retry. Persistent Rizin debugger sessions are unavailable outside Linux because this verification depends on `/proc`.
|
||||
|
||||
Persistent command Evidence retains the caller-selected and resolved startup paths in `parameters.launch_provenance`. These paths describe launch provenance, not the current target after unrestricted commands. `rizin_debug_session_status.output_invalid_utf8` reports whether lossy frame decoding has been observed during the session. Command Evidence sets `output_invalid_utf8` only when loss was observed in frames collected for that command, including retained partial output; earlier startup or command frames do not mark later valid output as lossy.
|
||||
|
||||
Session status retains up to 64 recent protocol frames. Each history entry keeps a complete-character UTF-8 prefix of at most 16 KiB, plus a truncation marker when needed; `recent_output_truncated` reports omitted history. This history budget also accounts for replacement characters from invalid UTF-8. It does not shorten command Evidence, which has its own 16 MiB output budget.
|
||||
|
||||
```sh
|
||||
rea debug-with-rizin --inferior-tty /dev/pts/7 ./program dr
|
||||
```
|
||||
|
||||
`execute_rizin_command` and `rea execute-rizin-command` run one selected command against a local artifact. The one-shot profile uses `-N -q -c`, suppressing user settings and scripts while leaving plugins available. Commands and plugins run with the REA process user's security context and may access files, the network, or perform other actions. REA preserves raw output and does not claim that command-specific JSON has one universal schema.
|
||||
`execute_rizin_command` and `rea execute-rizin-command` run one selected command against a local artifact. The one-shot profile uses `-N -q -e scr.color=0 -c`, suppressing user settings and scripts while leaving plugins available. Native terminal presentation defaults to plain text; the caller can explicitly enable colour with `e scr.color=1` in the selected command. REA preserves raw output, including literal ANSI bytes, and does not claim that command-specific JSON has one universal schema. Commands and plugins run with the REA process user's security context and may access files, the network, or perform other actions.
|
||||
|
||||
```sh
|
||||
rea execute-rizin-command ./program iI
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
# Implementation plan
|
||||
|
||||
1. Extend `src/application/environment/environmentSource.ts` so
|
||||
`loadEnvironmentSource` takes the reporting operation and retains the
|
||||
capture artifact's SHA-256, letting the export reuse the serve command's
|
||||
authentication and cross-check unchanged.
|
||||
2. Add `environmentBundle.ts` to plan the bundle from the source alone: the
|
||||
name rule, the Dockerfile for the registry and tarball package sources, the
|
||||
verifier, the smoke task, the README and the copy targets.
|
||||
3. Add `environmentExport.ts` to read each input once through the
|
||||
regular-file reader, refuse an occupied directory, a missing tarball or an
|
||||
unreadable input before writing, then write the planned files with
|
||||
exclusive creation and copy the authenticated bytes and the tarball
|
||||
byte-exact.
|
||||
4. Register `environment-export` in `src/cli/environmentCommands.ts` beside
|
||||
`environment-serve`, sharing the argument descriptions, and in
|
||||
`CLI_COMMANDS`.
|
||||
5. Add the `agentenv` scope, `real-agentenv.yml`, the package script and
|
||||
`scripts/verify-agentenv-environment.mjs`, wire `ci.yml` and extend the
|
||||
`ciRequiredLanes` and `ciChangeScope` tables.
|
||||
6. Add services, conformance and cli-boundary tests; document the command in
|
||||
`docs/cli.md`, the bundle and lane in `docs/agent-environments.md` and
|
||||
`docs/testing.md`; record the spec.
|
||||
|
||||
No new dependency is introduced; the CLI inventory gains one command.
|
||||
@@ -0,0 +1,116 @@
|
||||
# Export an environment manifest as an agent environment bundle
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
`rea environment-export <manifest.json> --evidence <capture.json> <dir>
|
||||
[--package-tarball <path>] [--name <override>]` writes the manifest and
|
||||
capture pair that `environment-serve` serves as a bundle folder the agentenv
|
||||
framework runs with `agent-env run <dir>`: a README with provenance, an env
|
||||
folder whose Dockerfile installs this exact `rea-agents` version and serves the
|
||||
copied pair, a Python replay verifier and a smoke task. The hosted
|
||||
`real-agentenv` lane exports such a bundle from the source-owned Grafana-shaped
|
||||
capture through the built CLI, packs the checkout into the image and runs it
|
||||
with the pinned framework release on the runner's Docker. This increment
|
||||
completes the agent-environment work of the issue; serving came in the
|
||||
previous increment.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Read each of the two files once through the regular-file reader, parse it
|
||||
from those bytes and authenticate and cross-check the pair with
|
||||
`loadEnvironmentSource` as `environment-serve` does, so the bytes copied into
|
||||
the bundle are the bytes that were authenticated; every refusal reports
|
||||
operation `environment-export`, and a manifest projected from a different
|
||||
capture is refused before anything is written.
|
||||
- Refuse a bundle directory that exists and is not empty, or that names a file,
|
||||
with an `invalid_request` whose issue path is `directory`; an absent or empty
|
||||
directory is accepted. Write every file with the exclusive-create flag so
|
||||
nothing is ever overwritten.
|
||||
- Write `README.md`, `envs/<name>/Dockerfile`, `envs/<name>/env.toml`,
|
||||
`envs/<name>/manifest.json` and `envs/<name>/evidence.json` as byte-exact
|
||||
copies of the inputs (mode `0600` on POSIX; Windows retains inherited ACLs
|
||||
and does not provide an owner-only guarantee), `artifacts/check_<name>/verify.py` and
|
||||
`tasks/smoke.json`, where `<name>` defaults to the manifest's kebab-case name
|
||||
and `--name` must be a kebab-case label of at most 64 characters that is not a
|
||||
device name Windows reserves (`con`, `prn`, `aux`, `nul`, `com1`–`com9`,
|
||||
`lpt1`–`lpt9`), since the env folder could not be created there.
|
||||
- The Dockerfile reads `FROM node:24.18.0-bookworm-slim`, sets
|
||||
`MCP_HOST=0.0.0.0` and `MCP_PORT=18765`, installs Debian's `python3` for the
|
||||
framework's readiness probe, installs
|
||||
`rea-agents@<PRODUCT_IDENTITY.packageVersion>` from the registry or, with
|
||||
`--package-tarball`, copies the tarball as `rea-agents.tgz`, installs it
|
||||
from `/tmp/` and removes it in the same layer, copies the pair to `/env/`,
|
||||
switches to the image's `node` user
|
||||
and runs `rea environment-serve /env/manifest.json --evidence
|
||||
/env/evidence.json`.
|
||||
- `verify.py` mirrors the framework's echo example: `async def verify(mcp_url)`
|
||||
over `mcp.client.streamable_http`, lists the tools and checks the count
|
||||
equals the manifest's, calls the first `GET` tool without path parameters
|
||||
with no arguments and checks the replayed `status` equals the recorded status
|
||||
of that tool's first cited record, prints the description and returns
|
||||
`[{"id": "replay", "result": <bool>, "description": <both checks>}]`; with no
|
||||
such tool it checks the count only and says so.
|
||||
- `tasks/smoke.json` is `deploy_env` of `<name>` on the local sandbox followed
|
||||
by `env_outcome_verifier` with `file_artifact_id` `check_<name>` and
|
||||
`verifier_id` `replay`. The README records the capture artifact SHA-256, both
|
||||
Evidence ids, the manifest view digest, the tool count and the package
|
||||
source, states what replay does and does not establish, names the run
|
||||
command as `agent-env run .` from the bundle root (the folder it sits in,
|
||||
whatever the caller named it), and carries the rights note.
|
||||
- Print one JSON document with `status: "exported"`, the directory, name, tool
|
||||
count, both Evidence ids, the pinned `package_source` and the written files
|
||||
with sizes. Register the flat `environment-export` command in the CLI
|
||||
inventory; no MCP tool is added.
|
||||
- Add the `agentenv` CI scope, the reusable `real-agentenv.yml` workflow
|
||||
(`workflow_dispatch` and `workflow_call` with `build_cache_artifact`,
|
||||
`ubuntu-24.04`, SHA-pinned actions, `setup-python` 3.12, a virtual
|
||||
environment with `agentenv-framework==0.9.1334`, the runner's Docker, 30
|
||||
minutes), the `verify:agentenv:environment` script and
|
||||
`scripts/verify-agentenv-environment.mjs`, which requires absolute
|
||||
`REA_AGENT_ENV_COMMAND` and `REA_DOCKER_COMMAND` like every other lane
|
||||
verifier, runs `agent-env run <dir> --sandbox local` bounded to 15 minutes
|
||||
and resolves nothing from `PATH`; wire `ci.yml` at the four spots and extend
|
||||
the two CI guard tests.
|
||||
|
||||
## Acceptance and limits
|
||||
|
||||
Services tests cover the planned file set, the Dockerfile for both package
|
||||
sources, the verifier text, the smoke task, the README, the name override and
|
||||
its refusals, writing into an absent and an empty directory with byte-exact
|
||||
copies and correct sizes, the tarball copy, and the refusals of a non-empty
|
||||
directory, a file path, a missing tarball and an inline manifest. The
|
||||
conformance test pins the registry install to the `package.json` version. The
|
||||
cli-boundary test writes the folder from the fixture pair through the built
|
||||
CLI, refuses a non-empty directory leaving it unchanged, and refuses a
|
||||
mismatched pair without creating the directory. The CI guard tests enumerate
|
||||
the new consumer, route and classification cases. The local end-to-end run
|
||||
recorded in `verification.md` is the proof that one exported bundle runs under
|
||||
`agent-env run` with the smoke task passed and no container left behind.
|
||||
|
||||
The export copies bytes and never inspects the tarball; it does not build or
|
||||
run the image itself. Its writes are not transactional: a host failure after
|
||||
the first file leaves the partial folder for inspection, as `docs/cli.md`
|
||||
states. The verifier checks one replayed status and the tool count, not every
|
||||
tool. The lane passes `--sandbox local` as designed although
|
||||
`tasks/smoke.json` already pins `sandbox_type: local` on its deploy step, so
|
||||
the flag restates what the task fixes. The lane verifier imports the
|
||||
Grafana-shaped HAR builder from `tests/fixtures/historicalHar.ts` through
|
||||
Node's type stripping rather than carrying a second copy under `scripts/`; the
|
||||
`agentenv` scope lists that fixture so a change to it runs the lane. The lane
|
||||
proves one framework release on one hosted runner and claims nothing about
|
||||
any hosted platform. Three additions to the
|
||||
designed Dockerfile and file set were needed by the framework, each found by
|
||||
the local run: `MCP_PORT=18765` as the listener default for the port the
|
||||
framework maps, Debian's `python3` because the framework's local sandbox
|
||||
probes readiness with `python3 -c "import socket; …"` inside the container and
|
||||
the slim Node image has none, and `env.toml` naming the environment, because
|
||||
the framework otherwise reads the name from Python sources the Node image does
|
||||
not have.
|
||||
|
||||
## Deferred issue criteria
|
||||
|
||||
This increment closes the agent-environment scope of #4933 together with the
|
||||
previous three. Running the lane on the maintainer's CI and any follow-up on
|
||||
hosted platforms are outside this PR.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the export contract, requirements and the lane.
|
||||
- [x] Thread the operation name and capture digest through the source loader.
|
||||
- [x] Plan the bundle: name rule, Dockerfile for both package sources, verifier, smoke task, README.
|
||||
- [x] Write the bundle with directory, tarball and input refusals and exclusive creation.
|
||||
- [x] Register the flat CLI command with described arguments and options.
|
||||
- [x] Add the agentenv scope, workflow, package script and verifier; wire ci.yml; extend the guard tests.
|
||||
- [x] Add services, conformance and cli-boundary coverage.
|
||||
- [x] Document the command, the bundle and the lane; record the spec.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates and the local end-to-end run.
|
||||
- [x] Repair the review findings (README run command, absolute tool paths, single read, tarball cleanup) and re-run the gates and the end-to-end run.
|
||||
@@ -0,0 +1,362 @@
|
||||
# Verification
|
||||
|
||||
The initial author record below describes the original macOS tree. The Linux
|
||||
followup record at the end covers the integrated serve changes and subsequent
|
||||
repairs in this PR. The two records have different toolchains and cache states.
|
||||
|
||||
Verified base: 24574576e4fa74ff56eda965ae287555b7552c03, the tip of the environment-serve increment on fetched main b83b9caf (this increment builds on it).
|
||||
|
||||
The worktree carries the manifest-view, streamable-HTTP and serve increments
|
||||
as commits on upstream main b83b9caf; this increment's changes are uncommitted
|
||||
edits on top. Every check below ran as written on this tree after the review
|
||||
repairs; nothing is reported that did not run. Timings are macOS
|
||||
`/usr/bin/time -l` command accounting (elapsed, user, system seconds and
|
||||
maximum resident set size of the command and the children it waited for) on
|
||||
macOS 26.5.1 with Node 22.23.1, npm 12.0.2, Docker 29.5.3 and a Python 3.12.12
|
||||
virtual environment holding `agentenv-framework 0.9.1334`.
|
||||
|
||||
## Review findings addressed
|
||||
|
||||
An adversarial review of the first pass found two should-fix findings and
|
||||
seven nits; each was repaired at its owning boundary before the gates below
|
||||
ran.
|
||||
|
||||
- The bundle README told the reader to run `agent-env run <name>`, which only
|
||||
works from the bundle's parent when the folder happens to carry the env's
|
||||
name. The README now says `agent-env run .` from the bundle root and names
|
||||
the folder's path for any other working directory; the services test
|
||||
asserts the new block and the absence of the old command.
|
||||
- The lane verifier resolved `agent-env` and `docker` from `PATH` when the
|
||||
`REA_*` variables were unset. It now requires absolute
|
||||
`REA_AGENT_ENV_COMMAND` and `REA_DOCKER_COMMAND`, throwing the standard
|
||||
actionable message naming the npm script and that nothing is installed, and
|
||||
checks both paths exist; the workflow exports `REA_DOCKER_COMMAND` from
|
||||
`command -v docker` beside the virtual environment's `agent-env`;
|
||||
`docs/testing.md` names both variables and drops the `PATH` default. No
|
||||
Linux preflight was added, so the lane stays runnable on a macOS host.
|
||||
- The brittle `assert.doesNotMatch(stdout, /failed/)` is gone; exit code `0`
|
||||
and the `tasks/smoke.json v<n>: passed` summary already decide the case.
|
||||
- The inner `agent-env run` timeout is 15 minutes inside the 30-minute job,
|
||||
so a hung run ends with the verifier's failed receipt rather than the
|
||||
runner's kill; `docs/testing.md` says so.
|
||||
- The pair was read twice: parsed once for authentication and re-read from
|
||||
disk for the byte-exact copies, so a file replaced between the reads was
|
||||
copied unchecked. `readEnvironmentExportInputs` now reads each file once
|
||||
through the regular-file reader, parses the value from those bytes with the
|
||||
shared UTF-8 JSON parser, and the command passes the same buffers to the
|
||||
export, which copies them; the refusals of inline JSON, a directory and a
|
||||
file that is not JSON are typed by role, and the spec, plan and both guides
|
||||
state the single read.
|
||||
- The tarball Dockerfile left the 6 MB `rea-agents.tgz` in `/tmp` of the
|
||||
image; the install layer now removes it (`&& rm /tmp/rea-agents.tgz`),
|
||||
confirmed by listing `/tmp` in the built image below.
|
||||
- The verifier passes `--sandbox local` as the design specifies; the spec
|
||||
records that the task's `sandbox_type: local` already pins the provider and
|
||||
the flag restates it.
|
||||
- The verifier's import of the HAR builder from `tests/fixtures/` through
|
||||
Node's type stripping is kept and recorded in the spec's limits with the
|
||||
reason (no second copy under `scripts/`; the scope lists the fixture).
|
||||
- Writes are not transactional; `docs/cli.md` now states that a host failure
|
||||
after the first file leaves the partial folder for inspection and that the
|
||||
next attempt refuses the non-empty directory, and the spec's limits say so.
|
||||
|
||||
## Setup
|
||||
|
||||
`npm ci` completed (exit 0; 373 top-level entries). `/usr/bin/time -l npm run
|
||||
build` passed with 3 tasks successful in 7.60 s elapsed (14.81 s user, 2.69 s
|
||||
sys, 3013410816 bytes); `/usr/bin/time -l npm run test:prepare` passed with 6
|
||||
tasks successful, 3 cached, in 1.63 s (3.47 s user, 0.81 s sys, 400752640
|
||||
bytes).
|
||||
|
||||
## Static gates
|
||||
|
||||
Every gate ran on the final tree after the last source edit, sequentially:
|
||||
`npm run check:fast` (typecheck and lint with module boundaries; 2 tasks
|
||||
successful; 5.28 s elapsed, 23.22 s user, 2.93 s sys, 4564680704 bytes);
|
||||
`npm run format` rewrote nothing outside this increment's files ("Finished in
|
||||
635ms on 4077 files"), and `npm run format:check` passed ("All matched files
|
||||
use the correct format"; 0.87 s, 4.30 s user, 0.64 s sys, 224051200 bytes);
|
||||
`npx knip` reported no findings (3.49 s, 5.04 s user, 0.72 s sys, 805191680
|
||||
bytes); `npm run verify:test-discovery` reported "Discovered all 1597 test
|
||||
files exactly once" (0.55 s, 0.38 s user, 0.14 s sys, 141492224 bytes);
|
||||
`npm run check:ci:tasks` (typecheck, lint:built, format:check, knip,
|
||||
metadata:check, jscpd, scan:todos) passed with 8 tasks successful, lint "3764
|
||||
files, 0 errors, 1690 warnings" and jscpd "3171 files, 2587 clones, 3.34%
|
||||
duplicated lines" under the 3.5% threshold (4.89 s elapsed, 15.19 s user,
|
||||
3.86 s sys, 874627072 bytes); `npm run docs:check` passed including the README
|
||||
consistency test (6 tasks, 5 tests; 4.00 s elapsed, 3.86 s user, 0.94 s sys,
|
||||
290111488 bytes). `actionlint 1.7.7` (`-shellcheck= -pyflakes=`, from the
|
||||
review's scratch virtual environment) reported no findings on
|
||||
`.github/workflows/real-agentenv.yml` and `.github/workflows/ci.yml`. A
|
||||
case-insensitive grep for the forbidden words and marker comments over the 23
|
||||
new or modified files matched only the `scan:todos` script name in
|
||||
`package.json` and this file.
|
||||
|
||||
## Test projects
|
||||
|
||||
Source group: `/usr/bin/time -l npx vitest run --project services --project
|
||||
conformance` passed 122 files, 1524 tests in 11.08 s elapsed, 23.10 s user,
|
||||
2.24 s sys, 1631240192 bytes. The increment's files are
|
||||
`environmentBundle.test.ts` (7 tests: the planned file set and copy targets,
|
||||
the registry Dockerfile pinned to `PACKAGE_METADATA.version` with the
|
||||
readiness-probe layer, the tarball Dockerfile installing and removing
|
||||
`/tmp/rea-agents.tgz`, the verifier text naming `get_annotations` as the
|
||||
first parameterless `GET` tool with recorded status 200 of record 25, the
|
||||
smoke task, the README's provenance, `agent-env run .` block and rights note,
|
||||
the name override and six refused names), `environmentExport.test.ts` (6
|
||||
tests: the reader returning each file's bytes and parsed value, its refusals
|
||||
of inline JSON, a directory and malformed JSON by role, writing into an absent
|
||||
directory with the read bytes copied unchanged and reported sizes, the
|
||||
tarball copy, the refusals of a non-empty directory, a file path and a missing
|
||||
tarball with nothing written, and an existing empty directory accepted),
|
||||
`environmentSource.test.ts` (4 tests; the loader takes the reporting
|
||||
operation) and the conformance test `environmentBundlePackagePin.test.ts` (1
|
||||
test: the registry install line ends with `rea-agents@<package.json version>`
|
||||
and never uses `latest` or a range).
|
||||
|
||||
Built group after `npm run test:prepare`: `/usr/bin/time -l npx vitest run
|
||||
--project cli-boundary --project boundary` over `environmentExportCli` (3),
|
||||
`ciChangeScope` (228), `ciRequiredLanes` (17), `workflowActionPinning` (5),
|
||||
`ciCachePolicy` (6), `cliToolCoverage` (1), `cliStartupImports` (5) and
|
||||
`ciTestProjects` (11) passed 8 files, 276 tests in 432.80 s elapsed, 303.47 s
|
||||
user, 92.63 s sys, 422412288 bytes; `tests/boundary/filesystem/productCatalog`
|
||||
passed separately (10 tests, 5.32 s). The cli-boundary file writes the folder
|
||||
from the fixture pair through the built CLI and checks the seven written
|
||||
paths, the byte-exact copies and the Dockerfile's first line and `CMD`;
|
||||
refuses a non-empty directory leaving its one file in place; and refuses a
|
||||
mismatched pair without creating the directory.
|
||||
|
||||
## Local end-to-end run
|
||||
|
||||
With the built CLI in this worktree, from a fresh scratch directory:
|
||||
`inspect-web-network-capture grafana-like.har har --format json --json` wrote
|
||||
capture Evidence `ev_4ee4759b…` (28 records); `inspect-analysis-view` with the
|
||||
inline source and all three template rules wrote manifest Evidence
|
||||
`ev_6553a8e5…` naming that parent, `grafana-example`, 19 tools, view digest
|
||||
`8d1eb3ee…`. `npm pack --ignore-scripts` of the checkout produced
|
||||
`rea-agents-6.6.0.tgz` (6039238 bytes). `environment-export manifest.json
|
||||
--evidence capture.json <tmp>/grafana-example --package-tarball <tgz> --json`
|
||||
exited 0 and printed `status: exported`, `tool_count: 19`, `package_source:
|
||||
{kind: tarball}` and eight files: `README.md` (2076 bytes),
|
||||
`envs/grafana-example/env.toml` (37), `envs/grafana-example/Dockerfile` (441),
|
||||
`artifacts/check_grafana-example/verify.py` (1534), `tasks/smoke.json` (291),
|
||||
`envs/grafana-example/manifest.json` (15701),
|
||||
`envs/grafana-example/evidence.json` (58296) and
|
||||
`envs/grafana-example/rea-agents.tgz` (6039238); `cmp` confirmed the two
|
||||
Evidence copies and the tarball byte-identical to the inputs. The Dockerfile's
|
||||
install line reads `RUN npm install -g --ignore-scripts --no-fund --no-audit
|
||||
/tmp/rea-agents.tgz && rm /tmp/rea-agents.tgz`, and the README's run section
|
||||
is the `agent-env run .` block.
|
||||
|
||||
`/usr/bin/time -l agent-env run <tmp>/grafana-example` exited 0 in 43.66 s
|
||||
elapsed (5.19 s user, 3.26 s sys, 129318912 bytes), output
|
||||
|
||||
```text
|
||||
[tasks/smoke.json] step 1/2 env done in 25.1s
|
||||
[tasks/smoke.json] step 2/2 replay (env_outcome_verifier)
|
||||
replay: listed 19 tools, expected 19; get_annotations replayed status 200, recorded 200
|
||||
[tasks/smoke.json] passed in 25.1s
|
||||
|
||||
Tasks:
|
||||
tasks/smoke.json v1: passed (replay: 1), 25.1s, instance @local/…/grafana-example/smoke-nwvvojs7
|
||||
|
||||
Tore down 1 sandbox.
|
||||
```
|
||||
|
||||
`docker ps -aq` before and after the run listed the same three pre-existing
|
||||
containers (a Grafana container, an exited container and the framework's
|
||||
registry from earlier sessions, unrelated to this run); no container was
|
||||
added, and the image count rose from 19 to 20 for the built env image
|
||||
`5a5b3bf62aae`. `docker run --rm --entrypoint sh 5a5b3bf62aae -c 'ls -la
|
||||
/tmp; rea --version; id -un'` listed only `node-compile-cache` in `/tmp` (no
|
||||
`rea-agents.tgz`), printed `6.6.0` and ran as `node`.
|
||||
|
||||
`REA_AGENT_ENV_COMMAND=<venv>/bin/agent-env REA_DOCKER_COMMAND=$(command -v
|
||||
docker) /usr/bin/time -l node scripts/verify-agentenv-environment.mjs` exited
|
||||
0 in 37.24 s elapsed (10.42 s user, 4.45 s sys, 268533760 bytes) and printed
|
||||
the receipt `status: passed`, `public_cases: 5`, `upstream: agentenv-framework
|
||||
0.9.1334`, `runner: darwin arm64; host Docker`, `docker_before: {containers:
|
||||
3, images: 20}`, `agent_env_run: {exit_code: 0, seconds: 32}` with the same
|
||||
passed summary in its stdout tail, `docker_after: {containers: 3, images: 21,
|
||||
new_containers: []}` and `process_lineage.status: verified` with no
|
||||
descendants. The first hosted run of the lane, on a runner with no prior
|
||||
framework state, created the framework's image-store helper `agentenv-registry`
|
||||
(`registry:2`, container `b2e047f4c6cd`) during `agent-env run`; the run itself
|
||||
passed (`exit_code: 0`, 354 s, `tasks/smoke.json v1: passed`, "Tore down 1
|
||||
sandbox.") but the first form of the leak check counted the helper and failed.
|
||||
The check now lists containers with their names, excludes
|
||||
`agentenv-registry[-<port>]` and reports it under `image_store_helpers`; the
|
||||
local receipt gained `image_store_helpers: []` because the helper already
|
||||
existed before the run here. Public ECR's anonymous pull quota refused the
|
||||
framework's `postgres:16-alpine` pull with `429 Too Many Requests, Data limit
|
||||
exceeded` on two hosted runs out of six (the second and, after the stack was
|
||||
rebuilt, the sixth); the verifier now retries the run once, after 90 seconds,
|
||||
when the failure carries that registry's signature, and records
|
||||
`attempts: 2` with `retried_for` in the receipt. The local receipt reports
|
||||
`attempts: 1`. Removing that helper (`docker rm -f
|
||||
agentenv-registry`) and running the verifier again reproduced the fresh-host
|
||||
path locally: the run recreated it and the receipt passed with
|
||||
`image_store_helpers: ["agentenv-registry"]`, `new_containers: []`
|
||||
(`agent_env_run: {exit_code: 0, seconds: 33}`). Without `REA_DOCKER_COMMAND` the verifier exited 1 before
|
||||
touching anything with "verify:agentenv:environment requires absolute
|
||||
REA_DOCKER_COMMAND; see docs/testing.md. No dependencies are installed by this
|
||||
verifier."; with a relative `REA_AGENT_ENV_COMMAND=agent-env` it exited 1 with
|
||||
the same sentence naming that variable.
|
||||
|
||||
Refusals through the built CLI: exporting into the already written bundle
|
||||
directory exited 1 with `invalid_request`, issue path `["directory"]` and the
|
||||
message "is not empty (4 entries); choose an absent or empty directory";
|
||||
exporting the manifest against the capture of one selected record
|
||||
(`--record 0`, `ev_3249d233…`) exited 1 with `invalid_request`, issue path
|
||||
`["evidence", "evidence_id"]` and "The manifest was projected from a
|
||||
different capture than the supplied evidence file.", creating no directory;
|
||||
a truncated `broken.json` as the evidence exited 1 with `invalid_request`,
|
||||
issue path `["evidence"]`, reason `invalid_format` and "broken.json is not
|
||||
valid JSON: Unexpected end of JSON input", creating no directory. An export
|
||||
without `--package-tarball` wrote a Dockerfile whose install line is
|
||||
`RUN npm install -g --ignore-scripts --no-fund --no-audit rea-agents@6.6.0`,
|
||||
the `package.json` version, with no tarball beside it; the tarball variant
|
||||
copies `rea-agents.tgz` and installs and removes `/tmp/rea-agents.tgz`.
|
||||
|
||||
## Independent review
|
||||
|
||||
The adversarial review of the first pass is the independent review of this
|
||||
increment; its findings and their repairs are listed above. One further
|
||||
observation surfaced while repairing them: the single-read helper routes the
|
||||
export's input refusals through the export module rather than the CLI's
|
||||
shared JSON-input parser, so the inline-JSON refusal is now a typed
|
||||
`invalid_request` on the `manifest` or `evidence` issue path instead of the
|
||||
parser's `input_path`/`input_reason` envelope; the cli-boundary tests did not
|
||||
depend on that envelope.
|
||||
|
||||
## Scope and measurement limits
|
||||
|
||||
The framework readiness probe, the port variable default and the `env.toml`
|
||||
name were found by running the bundle in the first pass, not by reading a
|
||||
published contract; they are recorded in the spec as the three additions to
|
||||
the designed file set. The local run and the verifier used the host's Docker
|
||||
with the framework's gateway and service-database images already built from
|
||||
earlier sessions, so the "first run also builds the infrastructure" path was
|
||||
not timed here. The verifier's container check compares `docker ps -aq`
|
||||
before and after; images built by the run remain by design and are only
|
||||
counted. `npm pack --json` has the npm 12 object shape on this host and the
|
||||
npm 11 array shape on the hosted runner; the verifier accepts both but only
|
||||
the former was exercised locally. The hosted `real-agentenv` workflow was not
|
||||
dispatched from this tree. The built projects ran on this host's Node 22.23.1
|
||||
rather than the pinned 24.18.0. The `/usr/bin/time -l` figures account for
|
||||
each command and the children it waited for; Docker's own work runs in the
|
||||
daemon and is outside them. Windows was not run.
|
||||
|
||||
## Linux followup verification (2026-10-19)
|
||||
|
||||
The original PR was updated from main after #4955 and #4975 merged. The
|
||||
followups retain the author's commits and portable-name checks, seal private
|
||||
JSON parser output once for shared immutable authentication/replay, reserve
|
||||
sealing work before reading, preflight the served catalog before exporting,
|
||||
translate changed-file failures by input role, defer export imports during CLI
|
||||
registration, and verify the supplied framework executable before attributing
|
||||
its results. The default `rea mcp` transport remains stdio.
|
||||
|
||||
Executed on Linux x64 with Node 24.18.0, npm 11.16.0, Docker 20.10.24,
|
||||
Python 3.12.13 and an isolated `agentenv-framework==0.9.1334` environment:
|
||||
|
||||
- `turbo run typecheck lint:built format:check docs:check:built knip metadata:check artifacts:skills --concurrency=1`: ten tasks passed, one cached, 289.315 s elapsed, 284.552 s CPU, 4450.18 MiB peak process-family RSS.
|
||||
- The focused source/compiled CLI/SDK/lifecycle/configuration run passed 399 tests in 24 files. A later run passed the updated 231 CI-selection cases, three executable-identity cases and ten product-catalog cases. The latter needed its generated catalog first; its initial missing-file failure was preparation, not a passing check.
|
||||
- Four previously recorded, unauthenticated real GitHub API responses were exported with the checkout tarball. The generated Dockerfile built unchanged. A fresh non-root container replayed every response twice through the official MCP SDK, matching body, status, Evidence ordinal/id and untrusted-content provenance. Discovery/card parity and tarball removal passed. The standalone test supplies `ENVIRONMENT_NAME` as the framework does. Runtime verification with the built image took 12.393 s / 6.002 s CPU / 313.11 MiB process-family RSS; image-build cost is excluded. A Docker stats sample was 250.5 MiB, not a measured container peak.
|
||||
- The production `verify-agentenv-environment.mjs` observed `agent-env 0.9.1334`, exported its 28-record source-owned Grafana-shaped fixture, deployed it with the real framework and passed `tasks/smoke.json`. The receipt reported five public cases, one sandbox torn down, no new task containers and no supervised descendants. This successful run used already prepared infrastructure: 141.697 s / 101.971 s CPU / 501.61 MiB process-family RSS. Earlier attempts failed because this host's proxy environment required optional SOCKS support and its default home sandbox directory was unwritable. Only the test process's proxy variables were cleared; supported `XDG_STATE_HOME` and `AGENT_ENV_LOCAL_SANDBOX_DIR` pointed into temporary storage. The shared Docker daemon was not reconfigured or restarted. The framework's persistent registry helper and images remain as its documented behavior.
|
||||
|
||||
A matched public `environment-serve`/SDK task compared the pre-followup serve
|
||||
build with the final implementation. Both used the same GitHub capture with
|
||||
1,000 synthetic HAR-extension items (each containing additional containers),
|
||||
one first call and twenty warm identical replays. Fixture generation was
|
||||
excluded. This is controlled stress data, not newly observed application data.
|
||||
Larger 25,000/75,000-item projection attempts returned the existing typed
|
||||
representation-budget refusal; that budget was not relaxed.
|
||||
|
||||
| Measurement | Before | After |
|
||||
| ----------------------- | ----------: | ----------: |
|
||||
| Cold listener startup | 2189.064 ms | 2862.860 ms |
|
||||
| First call | 76.950 ms | 45.265 ms |
|
||||
| Twenty warm calls | 324.119 ms | 267.132 ms |
|
||||
| Completed task elapsed | 3.229 s | 3.885 s |
|
||||
| Completed task CPU | 3.196 s | 3.827 s |
|
||||
| Peak process-family RSS | 466.43 MiB | 470.21 MiB |
|
||||
|
||||
These single completed samples do not establish an overall speed or memory
|
||||
improvement. Separate identity regressions establish that large reported JSON
|
||||
trees are shared rather than copied and that caller-owned mutable trees are
|
||||
not frozen. Token usage was not measured. CPU is child user+system time;
|
||||
process-family memory sums sampled descendant RSS every 50 ms and may count
|
||||
shared pages twice. Docker daemon/container CPU and peaks are outside that
|
||||
process-family measurement and remain unmeasured. Builds used one task at a
|
||||
time, one CPU and a soft 3 GiB Go limit; Docker builds were serialized and
|
||||
limited to one CPU/4 GiB, and the standalone replay container to one CPU/2 GiB.
|
||||
Windows execution and the published registry package were not tested in this
|
||||
followup; the checkout tarball proves the code before its release.
|
||||
|
||||
### Output ownership and final review followups
|
||||
|
||||
The output boundary now uses the existing `SafeOutputTree` owner instead of
|
||||
separate directory admission and pathname writes. It checks the admitted root,
|
||||
parents, file identities and complete entry inventory around publication, keeps
|
||||
failed-close authority, and preserves an existing empty caller root on rollback.
|
||||
The optional package is held as one regular-file descriptor before output
|
||||
writes, streamed with hash/readback verification, and checked for changes before
|
||||
release. POSIX file modes are set and verified as `0600` independently of umask;
|
||||
newly owned directories are `0700`. Existing root permissions remain caller owned.
|
||||
Windows privacy depends on the selected destination ACL; no owner-only Windows
|
||||
ACL guarantee or Windows runtime validation is claimed. Portable pathname
|
||||
operations still require a trusted destination and parents against concurrent
|
||||
replacement.
|
||||
|
||||
A compiled CLI counterexample under umask `0400` failed before the mode fix and
|
||||
succeeded afterward with every generated file readable at `0600` and exact
|
||||
Evidence bytes. Public CLI coverage also verifies that `symlink/..` retains its
|
||||
filesystem meaning for the destination and package path. Real-filesystem tests
|
||||
exercise destination replacement, foreign entries, parent symlinks, read-only
|
||||
permissions and admitted-tarball replacement. Controlled close failures exercise
|
||||
post-publication recovery and persistent retained ownership.
|
||||
|
||||
The inventory owner accepts canonically equivalent Unicode directory listings
|
||||
without collapsing distinct filenames: an NFC multiset comparison is followed
|
||||
by the original path identity checks. Tests simulate decomposed `readdir`
|
||||
spelling over a real owned file and create an extra distinct Linux decomposed
|
||||
foreign file. These establish the boundary behavior, not actual HFS+ execution.
|
||||
Manifest, Evidence and tarball descriptors use the shared resource scope's close
|
||||
retry. Persistent failures retain the canonical cleanup lease and produce a
|
||||
tagged `environment-export` error. The compiled JSON CLI regression injects a
|
||||
persistent manifest close failure before any destination write.
|
||||
|
||||
The framework verifier reuses `execNpm` and honors selected `npm_execpath`.
|
||||
Four process-boundary attribution cases pass, including a conflicting PATH npm.
|
||||
Those controlled executables verify attribution only.
|
||||
|
||||
After merging main `b4938aed5`, the generated Dockerfile built unchanged with the
|
||||
checkout package and replayed all four captured public GitHub API responses
|
||||
twice through the official MCP SDK in a fresh non-root container. Input bytes,
|
||||
status/body/Evidence/provenance, discovery/card parity and package removal passed.
|
||||
Warm base-image build plus pack/export/runtime verification: **40.699 s elapsed,
|
||||
8.274 s process CPU, 421.04 MiB peak process-family RSS**. The real pinned
|
||||
`agent-env 0.9.1334` verifier then passed all five public cases and the smoke task,
|
||||
tore down its sandbox, and reported no new task containers or supervised
|
||||
descendants: **156.895 s / 102.871 s CPU / 500.34 MiB**. Framework infrastructure
|
||||
was already prepared. The later Unicode, close-failure and npm-selection repairs
|
||||
have separate boundary regressions; they do not establish new real-platform
|
||||
coverage.
|
||||
|
||||
A matched export comparison before the main merge used the same manifest
|
||||
(6,416 bytes), capture (98,519 bytes) and checkout tarball (6,221,957 bytes), with
|
||||
all copied inputs byte-exact. Before/after the output-owner repair: **1.827/2.604
|
||||
s elapsed, 1.807/2.545 s CPU, 316.43/334.11 MiB** peak process-family RSS. These are
|
||||
single process samples with uncontrolled host filesystem cache; the additional
|
||||
ownership checks increased measured cost. They do not establish a speed or
|
||||
memory improvement.
|
||||
|
||||
All followup heavy work uses one CPU and one test/build task at a time. Docker
|
||||
builds are limited to one CPU/4 GiB, standalone runtime containers one CPU/2 GiB.
|
||||
Process-family RSS is sampled every 50 ms and may double-count shared pages.
|
||||
Docker daemon/container CPU and peak memory, agent tokens and fresh-framework
|
||||
infrastructure cost are unmeasured. The previously documented early catalog
|
||||
heap refusal is not represented as a passing run.
|
||||
|
||||
Final focused verification on the reviewed source passed **452 tests in 33 files** (including 232 CI-selection cases and the compiled CLI regressions): **161.276 s elapsed / 159.900 s CPU / 1242.43 MiB** peak process-family RSS. The final typecheck/build/lint gate passed: **218.427 s / 218.130 s CPU / 4449.45 MiB**.
|
||||
@@ -0,0 +1,36 @@
|
||||
# Implementation plan
|
||||
|
||||
1. Add `src/application/environment/environmentSource.ts` to authenticate the
|
||||
two Evidence files, require the manifest view kind and the capture
|
||||
operation, refuse a parent mismatch, and reduce every cited record to its
|
||||
request line, status, content type and retained response text through the
|
||||
slice-1 request-line reader, so the server holds only what replay returns.
|
||||
2. Add `environmentTools.ts` to map manifest tools to MCP tool definitions with
|
||||
one hand-built JSON Schema per tool that both `registerTool` (through the
|
||||
SDK's `fromJsonSchema`) and the card consume, and to build the card in the
|
||||
echo example's key order.
|
||||
3. Add `environmentReplay.ts` to score cited records by path-segment matches,
|
||||
then query matches, keep manifest order on ties, and shape the replay result
|
||||
with JSON-like parsing, UTF-8-safe truncation and the replay provenance
|
||||
block; export the slice-1 JSON-like predicate instead of restating it.
|
||||
4. Add `src/server/environment/serveEnvironment.ts` to build the `McpServer`
|
||||
whose handlers attach REA's target-content `_meta`, the card and `/agentenv`
|
||||
routes (the latter reading its body through the SDK's bounded reader), and
|
||||
to serve them through the slice-2 `serveStreamableHttp` with its `routes`
|
||||
and `onerror` seams; add `src/config/environmentName.ts` for
|
||||
`ENVIRONMENT_NAME`.
|
||||
5. Add `src/cli/environmentCommands.ts` registering `environment-serve` through
|
||||
`withCommandCancellation` and `logCliCommand`, resolving host and port with
|
||||
the slice-2 endpoint parser, reading files through `parseCliJsonInput`,
|
||||
importing the listener lazily, awaiting the cancellation signal or a
|
||||
listener failure and returning one `stopped` document or a typed error;
|
||||
register the name in `CLI_COMMANDS` and the lazy module in the startup
|
||||
import probe.
|
||||
6. Share the fixture pair through `tests/fixtures/environmentServe.ts` and the
|
||||
listener spawn through a generalised `startListeningProcess`; add services,
|
||||
adapters, mcp-boundary, cli-boundary and process-boundary tests.
|
||||
7. Document the command in `docs/cli.md`, add `docs/agent-environments.md` with
|
||||
its sidebar and index entries, and record the spec.
|
||||
|
||||
No new dependency is introduced. REA's tool contracts, catalog digests and the
|
||||
stdio transport are unchanged; the CLI inventory gains one command.
|
||||
@@ -0,0 +1,133 @@
|
||||
# Serve an environment manifest as a replay environment
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
`rea environment-serve <manifest.json> --evidence <capture.json> [--host H]
|
||||
[--port N]` serves an environment manifest, projected by `inspect-analysis-view`
|
||||
from retained HAR capture Evidence, as an agent environment: an MCP server at
|
||||
`/mcp` with one tool per manifest tool that replays the retained records the
|
||||
tool cites, the environment card at `/.well-known/agent-env.json` in the shape
|
||||
the agentenv framework's echo example publishes, and a `/agentenv` data plane
|
||||
that answers every JSON-RPC method as not found. Nothing is fetched from the
|
||||
recorded application; every answer is retained Evidence marked as target
|
||||
content. This increment adds the serve command only; exporting a bundle is a
|
||||
later increment.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Read both files as bounded JSON input, authenticate each as REA Evidence and
|
||||
require `inspect_analysis_view` Evidence of kind `environment-manifest` and
|
||||
complete `inspect_web_network_capture` Evidence; refuse any other pair with an
|
||||
`invalid_request` whose issue path names the offending file.
|
||||
- Forward caller cancellation to file reads and check startup boundaries before
|
||||
authentication/listener work; an already observed abort must not bind or
|
||||
announce a listener.
|
||||
- Refuse a manifest whose `parent_evidence_id` differs from the evidence file's
|
||||
`evidence_id` before binding any port; Evidence authentication covers the
|
||||
capture's records, so no further record-coverage refusal is defined.
|
||||
- Build an `McpServer` from the manifest alone: each tool's input schema lists
|
||||
every path parameter as a required string described by its segment index,
|
||||
rule and observed values, every observed query name as an optional string,
|
||||
and an optional `body` object for `POST`, `PUT` and `PATCH` (a query name
|
||||
that collides with a path parameter becomes `query_<name>`, a query name
|
||||
longer than 64 characters becomes `query_<n>`, and the body
|
||||
becomes `request_body` when a query parameter is named `body`), closed with
|
||||
`additionalProperties: false` so unknown arguments are refused; the
|
||||
description reads `METHOD template; replays retained records [ordinals];
|
||||
evidence ev_…`. The served catalog stays separate from REA's own tool
|
||||
contracts and digests.
|
||||
- Prefix tool descriptions with the shared captured-catalog warning so
|
||||
standalone card readers receive the trust notice before MCP initialization.
|
||||
- Bound each path-observation, body-key and cited-ordinal description sample to 1 KiB
|
||||
of UTF-8 whole values before joining them, with a separate omitted-count and
|
||||
manifest-Evidence notice. Keep complete arrays and replay semantics unchanged.
|
||||
- Bound captured method, path and query-name prose to UTF-8-safe display prefixes
|
||||
under the same budget, with an explicit full-value Evidence notice. Preserve
|
||||
complete lookup values; long path and query names use collision-safe aliases.
|
||||
- Parse a recorded query only when supplied query arguments can affect selection.
|
||||
- Replay by selecting, among the tool's cited records, the one whose path
|
||||
segments at the parameter indexes match the most supplied values, then the
|
||||
most of those segments equal to the value as written rather than only after
|
||||
percent-decoding, then the most supplied query values, keeping manifest order
|
||||
on ties; return `status`,
|
||||
`content_type`, `body` (parsed JSON for JSON-like content types with safe numeric conversion,
|
||||
else text; bodies over 256 KiB cut at a UTF-8 boundary with
|
||||
`truncated: true`) and `replay` with `record_ordinal`, `evidence_id`,
|
||||
`exact_match`, `recorded_path` and `recorded_query` (each cut to 1 KiB, with
|
||||
`recorded_line_truncated`; an explicit empty query is `?`), `body_recorded` (false
|
||||
when the capture kept no response body, so a `null` body is not mistaken for
|
||||
a recorded one), `body_encoding` (`base64` when the retained bytes are not
|
||||
UTF-8, so they are returned exactly rather than decoded lossily) and
|
||||
`truncated` (a large retained base64 body is decoded only up to that limit),
|
||||
as both
|
||||
structured content and text, with REA's existing target-content provenance
|
||||
marker in `_meta`.
|
||||
- Serve the card with exactly the echo example's keys and nesting: `name`
|
||||
(`ENVIRONMENT_NAME` from the environment snapshot, else the manifest name; a
|
||||
set but blank value is refused as `configuration_invalid`),
|
||||
`protocolVersion` `1.0`, `url` `/agentenv`, `preferredTransport` `JSONRPC`,
|
||||
`additionalInterfaces` `[{url: /mcp, transport: mcp}]`, `capabilities` with
|
||||
`extensions: null`, the tools and `operations: []`, and
|
||||
`children_environments: null`. Read the `/agentenv` body within the SDK's
|
||||
4 MiB bound and answer `413` with JSON-RPC `-32000` beyond it.
|
||||
- Resolve host and port as `rea mcp --http` does (flags, `MCP_HOST`/`MCP_PORT`,
|
||||
then `127.0.0.1` and port `0`), announce the bound URL as one pino JSON line
|
||||
on stderr, serve through the shared streamable HTTP listener, and run until
|
||||
SIGINT or SIGTERM only, then close, print one `stopped` document and exit
|
||||
`130` or `143` through the shared command cancellation handler; a signal
|
||||
during loading returns `cancelled` before binding; a listener
|
||||
failure after binding is logged and ends the command with a
|
||||
`resource_constraint` error and exit `1`.
|
||||
- Register the flat `environment-serve` command in the CLI inventory with
|
||||
described arguments and no MCP tool contract; keep the CLI wiring thin, the
|
||||
mapping and replay in the application layer and the server assembly in the
|
||||
server layer; load the listener lazily so the command catalog registers
|
||||
without it.
|
||||
|
||||
## Acceptance and limits
|
||||
|
||||
Services tests cover source loading and each refusal, tool mapping and schema
|
||||
derivation, the card shape, exact and best-effort selection, query matching,
|
||||
tie order, text and non-parsing bodies, truncation at a UTF-8 boundary and
|
||||
base64 content. Adapters tests byte-compare the served card with the echo card
|
||||
shape carrying the fixture tools, drive `/agentenv` including a body over the
|
||||
bound, and call tools over the SDK streamable HTTP client with provenance and
|
||||
the unknown-argument refusal asserted. The mcp-boundary test
|
||||
spawns the built CLI on a manifest and capture produced from the slice-1
|
||||
Grafana-shaped fixture and asserts tool count, an exact `get_users_by_id`
|
||||
replay with `body.login === "probe"`, an inexact replay, the card under
|
||||
`ENVIRONMENT_NAME` and the data-plane refusal; the cli-boundary test covers
|
||||
the mismatched-parent refusal, the missing `--evidence` usage error and an
|
||||
unreadable path; the process-boundary test asserts a `143` exit with a
|
||||
released port. Replay validates no supplied values, performs no network
|
||||
activity and establishes no authorization, response semantics or state
|
||||
transitions. One MCP client is served at a time, as the shared listener
|
||||
defines. Non-loopback binds relax the `Host` check and are not exercised; a
|
||||
listener `error` after binding is wired but not provoked by a test.
|
||||
|
||||
## Resource and cancellation boundaries
|
||||
|
||||
- Whole discovery admission reserves at most 1 MiB using six bytes per UTF-16
|
||||
unit plus schema overhead before retaining each schema property. Aggregate
|
||||
tools, arguments and selected environment identity participate; oversized
|
||||
catalogs fail with a non-retryable capacity error before bind. The calibration
|
||||
used 1500 query fields with eight concurrent discovery requests; compact wire
|
||||
size can be smaller than the conservative reservation.
|
||||
- Query matching scans selected names one field at a time; path scoring scans
|
||||
once and retains only supplied parameter coordinates. Native form-decoding
|
||||
semantics, malformed escape handling and complete captured facts are preserved.
|
||||
- Content-type display is bounded to 1 KiB and reports `content_type_truncated`;
|
||||
complete media-type semantics remain available to body interpretation.
|
||||
- Cancellation also owns DNS/listen startup; an observed abort closes the
|
||||
listener before its URL is announced.
|
||||
|
||||
## Deferred issue criteria
|
||||
|
||||
Exporting a manifest and its evidence as an agent-environment bundle with a
|
||||
verifier, and the hosted verification lane, are later independent increments
|
||||
of #4933. This PR must reference rather than close the whole issue.
|
||||
|
||||
Numeric data-plane IDs that lose value through JSON number conversion are invalid
|
||||
requests with null response IDs, rather than silently changed correlation values.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the serve contract, requirements and deferred increments.
|
||||
- [x] Load and cross-check the manifest and capture Evidence with typed refusals.
|
||||
- [x] Map manifest tools to MCP input schemas and the environment card.
|
||||
- [x] Select and shape replayed records with exact-match, truncation and provenance.
|
||||
- [x] Assemble the MCP server, card and data-plane routes on the shared listener.
|
||||
- [x] Register the flat CLI command with host, port and environment-name resolution.
|
||||
- [x] Add services, adapters, mcp-boundary, cli-boundary and process-boundary coverage.
|
||||
- [x] Document the command and the agent environment page; record the spec.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates and the
|
||||
built-CLI exchange with the Python MCP client.
|
||||
@@ -0,0 +1,245 @@
|
||||
# Verification
|
||||
|
||||
Verified base: cf4cc6faab0fb2c444ca054db8b98cb981bae6c9 (fetched main).
|
||||
|
||||
The worktree carries the manifest-view and streamable-HTTP increments as
|
||||
commits on that base; this increment's changes are uncommitted edits on top.
|
||||
Every check below ran as written on this tree in the review-response pass;
|
||||
nothing is reported that did not run. Timings are macOS `/usr/bin/time -l`
|
||||
command accounting (elapsed, user, system seconds and maximum resident set
|
||||
size of the command and the children it waited for).
|
||||
|
||||
## Review findings addressed
|
||||
|
||||
An adversarial review of the first pass found one should-fix and six nits;
|
||||
each was repaired at its owning boundary before the gates below ran.
|
||||
|
||||
- `POST /agentenv` read its body with `request.json()` on the streamed
|
||||
`node:http` body, so a 60 MB payload was buffered before the `-32700`
|
||||
answer. The route now reads through the SDK's `readRequestBody` with
|
||||
`DEFAULT_MAX_REQUEST_BODY_SIZE`, answering `413` with JSON-RPC `-32000`
|
||||
(`id: null`) before parsing, and parses within the bound through the
|
||||
domain's `safeParseJson`. The JSON-RPC error helper of the shared listener
|
||||
is exported as `jsonRpcErrorResponse` and reused rather than restated.
|
||||
- A set but blank `ENVIRONMENT_NAME` refuses with `configuration_invalid`,
|
||||
as a blank `MCP_HOST` does in the sibling parser; `docs/cli.md` and
|
||||
`docs/agent-environments.md` now say so, and the spec records the
|
||||
divergence from the design's `||` reading as deliberate.
|
||||
- The record-coverage refusal in `loadEnvironmentSource` was unreachable for
|
||||
authenticated pairs (the Evidence id is computed over `normalized_result`,
|
||||
so a matching parent id implies identical records, each with a request
|
||||
line by construction of the projector). The branch is gone: cited records
|
||||
are indexed unconditionally, a record without a request line is left out
|
||||
and the tool refuses at call time through the existing no-record path. The
|
||||
claim was removed from `docs/agent-environments.md` and `spec.md`, and the
|
||||
services test now asserts an empty manifest indexes nothing and a tampered
|
||||
capture is refused by authentication.
|
||||
- Every advertised input schema closes with `additionalProperties: false`,
|
||||
so an unknown argument is refused as invalid arguments instead of silently
|
||||
degrading to an omitted one. The agentenv framework's card consumers
|
||||
(`verify_env_card.py`, the gateway's `tool.inputSchema` pass-through and
|
||||
the spec-conformance normaliser) read `properties` and `required` only, so
|
||||
the extra key does not disturb them; the guide names it as a key the echo
|
||||
schema does not carry.
|
||||
- The tool description now reads `METHOD template; replays retained records
|
||||
[ordinals]; evidence ev_…`, as the design text specifies.
|
||||
- `serveEnvironment` passes an `onerror` to the shared listener that logs
|
||||
`{ error }` at `error` level and forwards to the caller; the command races
|
||||
the cancellation signal against that failure and, on a listener failure
|
||||
after binding, closes the listener and returns the same typed
|
||||
`resource_constraint` error the bind failure returns (exit `1`) instead of
|
||||
serving nothing. The two constructions share one `transportError` helper.
|
||||
- `tests/boundary/cli/cliStartupImports.test.ts` lists
|
||||
`dist/server/environment/serveEnvironment.js` as `environment-server` in
|
||||
`DEFERRED_IMPORTS`, so the `--help` probe would fail if the serve command
|
||||
ever imported the listener eagerly.
|
||||
|
||||
## Setup
|
||||
|
||||
`npm ci` exited 0 (540 packages added, 541 audited; 4.79 s elapsed, 3.46 s
|
||||
user, 7.34 s sys, 880803840 bytes). After the edits above, `npm run build`
|
||||
exited 0 (three turbo tasks; 4.61 s elapsed, 14.76 s user, 3.23 s sys,
|
||||
3053060096 bytes) and `npm run test:prepare` exited 0 (six tasks; 1.28 s
|
||||
elapsed, 2.94 s user, 0.67 s sys, 424935424 bytes), so every built-group run
|
||||
and built-CLI check below used a `dist/` carrying the revised serve command.
|
||||
|
||||
## Static gates
|
||||
|
||||
Each exit 0 on the final tree: `npm run check:fast` (typecheck and lint,
|
||||
2 tasks; 4.76 s elapsed, 23.36 s user, 2.56 s sys, 4458840064 bytes). `npm run
|
||||
format` (4064 files; 0.93 s elapsed, 4.12 s user, 0.76 s sys, 226344960
|
||||
bytes) then `npm run format:check` (all 4064 files formatted; 0.64 s elapsed,
|
||||
4.19 s user, 0.66 s sys, 219906048 bytes). `npx knip` (no findings; 3.49 s
|
||||
elapsed, 4.92 s user, 0.63 s sys, 871645184 bytes). `npm run check:ci:tasks`
|
||||
(8 tasks successful, 2 served from the turbo cache; `lint:built` 3756 files,
|
||||
0 errors; jscpd 3164 files, 2583 clones, 3.34% duplicated lines under the
|
||||
3.5% threshold; the stray-marker scan; 7.16 s elapsed, 26.46 s user, 5.88 s
|
||||
sys, 2939142144 bytes). `npm run verify:architecture-guards` passed across
|
||||
1523 source files and `npm run verify:test-discovery` discovered all 1592
|
||||
test files exactly once. `npm run docs:check` (six tasks plus the README
|
||||
consistency test, 5 cases; 3.37 s elapsed, 3.49 s user, 0.84 s sys,
|
||||
291618816 bytes). After the last edit of this record, `npm run format`,
|
||||
`npm run format:check` and `npm run docs:check` were rerun (each exit 0) and
|
||||
the same static set was rerun uncached as `npx turbo run typecheck lint:built
|
||||
format:check knip metadata:check jscpd scan:todos --force` (8 tasks
|
||||
successful, 0 cached, 3.34% duplication; 8.40 s elapsed, 48.82 s user, 7.77 s
|
||||
sys, 4638474240 bytes). A case-insensitive grep for the forbidden words and
|
||||
marker comments over all 28 new or modified files found nothing.
|
||||
|
||||
## Test projects
|
||||
|
||||
Source group on the final tree: `/usr/bin/time -l npx vitest run --project
|
||||
services --project adapters` passed 343 files, 5196 tests with 8 skipped, in
|
||||
99.55 s elapsed, 93.60 s user, 28.88 s sys, 1781055488 bytes. The increment's
|
||||
files are `environmentSource.test.ts` (4 tests: pair authentication and
|
||||
exchange indexing, parent mismatch as `invalid_request` with path
|
||||
`["evidence","evidence_id"]`, other view kind, non-capture evidence and
|
||||
non-Evidence JSON, an empty manifest indexing nothing with a tampered capture
|
||||
refused by authentication), `environmentTools.test.ts` (5 tests: ordered
|
||||
mapping and the `; evidence` description, required path parameters with rule,
|
||||
observed values and `additionalProperties: false`, optional query strings
|
||||
without a body for reads, a body object for writes, the card's key order),
|
||||
`environmentReplay.test.ts` (7 tests, unchanged in this pass) and
|
||||
`serveEnvironment.test.ts` (5 adapters tests: the card text byte-equal to the
|
||||
echo card shape with the fixture tools, `/agentenv` answering `-32601` with
|
||||
`-32700` for a malformed body, a route-level body of
|
||||
`DEFAULT_MAX_REQUEST_BODY_SIZE` plus envelope answered `413` `-32000` with
|
||||
`id: null`, a full listener driven by the SDK streamable HTTP client with
|
||||
`_meta` equal to `TARGET_CONTENT_RESULT_META`, exact and inexact replays, an
|
||||
argument-less call and an unknown argument both refused with `isError`, and
|
||||
the `renamed` override with an oversized body answered `413` through
|
||||
`node:http` and a released port).
|
||||
|
||||
Focused built group after `test:prepare`: `npx vitest run --project
|
||||
cli-boundary --project mcp-boundary --project process-boundary --project
|
||||
boundary` over `environmentServeCli`, `environmentServeMcp`,
|
||||
`environmentServeShutdown`, `mcpHttpShutdown`, `streamableHttpMcp`,
|
||||
`cliToolCoverage`, `cliStartupImports` (now carrying the `environment-server`
|
||||
deferred import), `dispatcher`, `productCatalog`, `documentationFacts` and
|
||||
`ciTestProjects` passed 11 files, 33 tests (16.38 s elapsed, 25.77 s user,
|
||||
5.94 s sys, 941948928 bytes).
|
||||
|
||||
Full built-group projects ran one after another on the final `dist/` from a
|
||||
background shell while the built-CLI exchange below ran in the foreground.
|
||||
`/usr/bin/time -l npx vitest run --project mcp-boundary` passed 156 files
|
||||
with 2 skipped, 914 tests with 7 skipped, in 166.23 s elapsed, 319.51 s user,
|
||||
46.18 s sys, 4452171776 bytes. `/usr/bin/time -l npx vitest run --project
|
||||
cli-boundary` exited 1: 109 files passed and 3 failed, 663 tests passed,
|
||||
2 skipped and 7 failed, in 432.45 s elapsed, 725.28 s user, 213.32 s sys,
|
||||
641597440 bytes. No failing file belongs to this increment.
|
||||
`javascriptApplicationJsonOutput.test.ts` (4 cases) and
|
||||
`javascriptApplicationProgress.test.ts` (2 cases) failed with the worker's
|
||||
result-capacity constraint ("The parent's result capacity was exhausted
|
||||
after retaining static facts for main.js"); rerun together on the idle host
|
||||
after the chain they passed 2 files, 17 tests (17.05 s elapsed, 26.40 s user,
|
||||
8.47 s sys, 282394624 bytes), the load classification the previous pass
|
||||
recorded. `closedStdout.test.ts` "stops quietly when the reader closes during
|
||||
streamed JSON" exits `1` instead of `141` with 1009 bytes received, alone as
|
||||
well as in the project, the environmental failure the previous pass
|
||||
reproduced on a clean clone (macOS, Node 22.23.1, npm 12.0.2).
|
||||
`commandFinalizationCancellation.test.ts`, which the previous pass saw fail
|
||||
on worker order, passed in this run.
|
||||
|
||||
`/usr/bin/time -l npx vitest run --project process-boundary` exited 1: 56
|
||||
files passed, 2 skipped and 3 failed, 463 tests passed, 19 skipped and 5
|
||||
failed, in 431.71 s elapsed, 148.34 s user, 114.81 s sys, 710000640 bytes.
|
||||
`environmentServeShutdown.test.ts` and `mcpHttpShutdown.test.ts` passed inside
|
||||
it. `jadxIntegration.test.ts` failed three cases at the 30 s test and 10 s
|
||||
hook timeouts ("retires an idle engine", "completes large class inventories",
|
||||
"close cancels active and queued work"), the lane the previous pass recorded
|
||||
as moving between runs on this host. `releaseArtifactCanary.test.ts` fails
|
||||
parsing `npm pack --json` ("expected tuple, received object"), the npm 12.0.2
|
||||
shape the previous pass reproduced on a clean clone.
|
||||
`processCaptureLifecycle.test.ts` "keeps the process deadline bounded when the
|
||||
wall clock moves backward" failed with `expected false to be true` on
|
||||
`result.ok`; rerun alone on the idle host it failed the same way with 23
|
||||
passed (38.12 s elapsed, 9.84 s user, 18.07 s sys, 323076096 bytes). The file
|
||||
and `src/process/` are untouched by this increment and the case mocks
|
||||
`Date.now` around a real process capture with a 1.5 s fallback abort; it is
|
||||
recorded as a host-dependent failure of that lane, not classified further
|
||||
here.
|
||||
|
||||
## Built-CLI exchange on the prototype capture
|
||||
|
||||
`node scripts/rea.mjs inspect-web-network-capture grafana-like.har har
|
||||
--format json` wrote capture Evidence `ev_4ee4759b…` with 28 records;
|
||||
`inspect-analysis-view` with the inline source and all three template rules
|
||||
wrote manifest Evidence `ev_6553a8e5…` naming that parent, `grafana-example`,
|
||||
19 tools. `environment-serve manifest.json --evidence capture.json --port 0
|
||||
--json < /dev/null` announced one stderr line
|
||||
`{"transport":"streamable-http","url":"http://127.0.0.1:62305/mcp","msg":"MCP transport listening"}`
|
||||
and nothing on stdout until shutdown.
|
||||
|
||||
`GET /.well-known/agent-env.json` returned a card whose top-level keys,
|
||||
`capabilities` keys and tool keys equal the echo card's in order; its
|
||||
`inputSchema` keys are `type`, `properties`, `required` and
|
||||
`additionalProperties`, the echo card's three plus the closing key. Fixed
|
||||
fields: `protocolVersion` `1.0`, `url` `/agentenv`, `preferredTransport`
|
||||
`JSONRPC`, `additionalInterfaces` `[{url:/mcp,transport:mcp}]`, `extensions`
|
||||
null, `operations` `[]`, `children_environments` null, 19 tools, with
|
||||
`get_users_by_id` described as `GET /api/users/{numeric}; replays retained
|
||||
records [5, 6]; evidence ev_4ee4759b…`.
|
||||
|
||||
The Python `mcp` 1.30.0 client (httpx 0.28.1) initialised against
|
||||
`grafana-example 6.6.0` (protocol 2025-11-25), listed 19 tools, called
|
||||
`get_users_by_id {"id":"2"}` and read `body.login == "probe"`,
|
||||
`replay.exact_match == True`, `_meta.provenance.untrusted_content == True`
|
||||
with `source == "analysis_target"`; `{"id":"999"}` gave `exact_match ==
|
||||
False`; `get_folders_by_uid {"uid":"general"}` gave `recorded_path ==
|
||||
"/api/folders/general"`; `get_search {"query":"probe","type":"dash-db"}`
|
||||
returned a list body whose first title is `probe dashboard` with
|
||||
`recorded_query == "?query=probe&type=dash-db"`; `get_users_by_id
|
||||
{"id":"2","unknown":"x"}` returned `isError == True` with no structured
|
||||
content.
|
||||
|
||||
`POST /agentenv {"jsonrpc":"2.0","id":1,"method":"data/reset"}` returned
|
||||
`{"jsonrpc":"2.0","id":1,"error":{"code":-32601,"message":"method not found: data/reset"}}`.
|
||||
A 60 MB non-JSON POST with a `Content-Length` header was answered `413`
|
||||
`{"jsonrpc":"2.0","error":{"code":-32000,"message":"Request body must not exceed 4194304 bytes"},"id":null}`
|
||||
after curl had sent 768332 bytes, in 1.4 ms, with the server's resident set
|
||||
269024 KiB before and 269040 KiB after; the same file sent chunked without a
|
||||
length was answered `413` after 5237311 bytes in 9.9 ms, resident set 267952
|
||||
KiB before and 274208 KiB after, so the read stops at the bound on both
|
||||
paths. A malformed body within the bound (`{`) was answered `400`
|
||||
`{"jsonrpc":"2.0","error":{"code":-32700,"message":"parse error"},"id":null}`.
|
||||
|
||||
SIGTERM ended the process with status 143 after one stdout document
|
||||
`{"status":"stopped","url":"http://127.0.0.1:62305/mcp","environment":"grafana-example","tools":19,"evidence_id":"ev_4ee4759b…","manifest_evidence_id":"ev_6553a8e5…"}`.
|
||||
A restart with `ENVIRONMENT_NAME=renamed` served a card named `renamed`,
|
||||
reported `"environment":"renamed"` in its stopped document and also exited 143. `ENVIRONMENT_NAME=` (set, blank) exited 1 before binding with
|
||||
`{"code":"configuration_invalid","details":{"settings":[{"setting":"ENVIRONMENT_NAME",…}]}}`.
|
||||
The manifest served against the capture of one selected record (`--record
|
||||
0`, `ev_3249d233…`) was refused with exit 1 and
|
||||
`{"code":"invalid_request","details":{"operation":"environment-serve","issues":[{"path":["evidence","evidence_id"],"reason":"invalid_value","expected":"ev_4ee4759b…",…}]}}`
|
||||
whose message names both identifiers; omitting `--evidence` exited 1 with
|
||||
Incur's `VALIDATION_ERROR` naming `evidence`. No `rea.mjs` process or
|
||||
listener remained after these runs.
|
||||
|
||||
## Independent review
|
||||
|
||||
The adversarial review of the first pass is the independent review of this
|
||||
increment; its findings and their repairs are listed above. One further
|
||||
observation surfaced while repairing them: the byte-identical card assertion
|
||||
in `serveEnvironment.test.ts` compares the `inputSchema` object exactly, so
|
||||
closing the schema required extending that expectation as well as the
|
||||
services tests, which is the intended coupling between the card and the
|
||||
advertised MCP schema.
|
||||
|
||||
## Scope and measurement limits
|
||||
|
||||
The echo-card comparison is exact on keys, nesting and tool entries; the
|
||||
property names inside `inputSchema.properties` necessarily differ from the
|
||||
echo tool's single `text` property, and `inputSchema` carries one key the echo
|
||||
schema does not. `exact_match` is defined over supplied values only; omitted
|
||||
parameters cannot mismatch, and a call without arguments is exact on the
|
||||
tool's first record. Request bodies on write tools are accepted and ignored.
|
||||
The post-bind listener `error` path is wired and type-checked but not
|
||||
provoked by a test, since the shared listener does not expose its
|
||||
`node:http` server. Non-loopback binds and the relaxed `Host` check were not
|
||||
exercised; the Python exchange, curl checks and every test used `127.0.0.1`.
|
||||
Replay selection and body parsing were measured on the 28-record fixture
|
||||
only; no large-capture timing was taken. Resident-set figures are `ps` samples
|
||||
of the serving process before and after one request each. Token usage and
|
||||
full process-family peak are unmeasured; the `/usr/bin/time -l` figures
|
||||
account for each command and the children it waited for. Windows was not
|
||||
run.
|
||||
@@ -0,0 +1,32 @@
|
||||
# Implementation plan
|
||||
|
||||
1. Widen the runtime `serve` dependency to a transport-neutral factory that may
|
||||
bind asynchronously. `startMcpTransport` awaits it, hands it the server
|
||||
logger, and keeps stdio as the default; the shutdown handle type loses its
|
||||
stdio name.
|
||||
2. Add `src/server/http/nodeHttpBridge.ts` to turn an IncomingMessage into a web
|
||||
Request (headers, streamed body, abort on client close) and to pipe a web
|
||||
Response back with flushed headers, and `src/server/http/serveStreamableHttp.ts`
|
||||
to own one `McpServer` instance and one stateful transport per session,
|
||||
ended by `DELETE` or replaced by the next client's `initialize` without
|
||||
running the instance's shutdown hooks, plus plain routes, loopback Host
|
||||
validation, the listening log line and idempotent close. The typed listen
|
||||
error lives in its own module so the stdio entry never loads the listener.
|
||||
3. Parse `--http`, `--host` and `--port` in `src/main/mcpHttpArguments.ts` from
|
||||
one schema that also renders help; resolve `MCP_HOST`/`MCP_PORT` in
|
||||
`src/config/mcpHttpEndpoint.ts` as a `Result` that names rejected settings
|
||||
without their values.
|
||||
4. Route the dispatcher to `runMcpHttpEntrypoint` when the `mcp` entry carries
|
||||
`--http`; compose HTTP dependencies in `src/main.ts` with the shared
|
||||
`run()` path so configuration, session, providers and shutdown stay common.
|
||||
5. Register signals only for HTTP mode through `registerSignalShutdown`, a
|
||||
projection of `registerProcessShutdown` without stdin that sets the signal
|
||||
exit status before the idempotent shutdown runs; stdio keeps its stdin
|
||||
end/close triggers.
|
||||
6. Document the transport in docs/mcp-contracts.md and docs/cli.md, relabel the
|
||||
architecture diagram, and cover each boundary with adapters, dispatcher,
|
||||
mcp-boundary and process-boundary tests plus a Python SDK client exchange.
|
||||
|
||||
No new dependency is introduced: the transport, Host validation and body bound
|
||||
come from the already pinned `@modelcontextprotocol/server` root export. Tool
|
||||
contracts, catalog digests, response budgets and stdio behaviour are unchanged.
|
||||
@@ -0,0 +1,79 @@
|
||||
# Streamable HTTP transport for the MCP server
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
`rea mcp --http [--host H] [--port N]` serves the existing MCP tool catalog over
|
||||
the pinned SDK's streamable HTTP transport instead of stdio, so an agent
|
||||
environment runtime or any HTTP MCP client can reach REA on a local URL. The
|
||||
listener is opt-in: bare `rea mcp`, `rea mcp doctor`, `rea mcp add` and every
|
||||
client registration keep stdio.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Route `mcp --http ...` and `--mcp --http ...` to the MCP runtime in the
|
||||
dispatcher without changing bare `mcp`, `mcp doctor` or `mcp add`.
|
||||
- Resolve host and port from the flags, then `MCP_HOST`/`MCP_PORT` from the
|
||||
environment snapshot, then `127.0.0.1` and port `0`; announce the bound URL as
|
||||
one pino JSON line on stderr at every log level and leave stdout unused.
|
||||
- Use `WebStandardStreamableHTTPServerTransport` in stateful mode around one
|
||||
server instance built once, after the listener has bound. `DELETE` ends the session on its transport without
|
||||
running the instance's shutdown hooks, so the same instance keeps its
|
||||
providers and serves the next client on a fresh transport; the hooks run when
|
||||
the listener closes.
|
||||
- Let a new client's `initialize` replace a session that is still open, closing
|
||||
the earlier transport the same way, so a client that left without `DELETE`
|
||||
cannot block the listener; the replaced client's later requests are refused
|
||||
(`404` while the replacing session is open).
|
||||
- Bridge node:http to web Request/Response in `src/server/http/`, streaming
|
||||
response chunks as they are produced and turning a client disconnect into a
|
||||
request abort and body cancellation.
|
||||
- Classify a loopback bind by the actual bound IP (`127.0.0.0/8`, `::1`, or
|
||||
IPv4-mapped loopback), without using the configured DNS name as evidence; validate
|
||||
the `Host` and `Origin` headers with the SDK loopback allowlist plus the
|
||||
configured name and the bound address on such binds, and relax both for
|
||||
other binds and log that relaxation. Refuse scoped IPv6 literals before
|
||||
binding, close the listener if its URL cannot be formed, and forward
|
||||
listener errors to `onerror` only after a successful bind.
|
||||
- Let the caller own instance cleanup (`closeInstance: false`) when its factory
|
||||
owner already closes every instance, as the MCP runtime does.
|
||||
- Let only an `initialize` that carries the streamable headers and passes those
|
||||
checks replace an open session; refuse requests once the slot is closed, join
|
||||
an arming session during shutdown, and keep close stages retryable.
|
||||
- End the HTTP listener on SIGINT/SIGTERM only, never on stdin EOF, and exit with
|
||||
the signal status (`130`/`143`) after cleanup settles; latch a signal that
|
||||
arrives during startup so it is acted on as soon as the lifecycle owner exists,
|
||||
and exit at once on a second one.
|
||||
- Export `serveStreamableHttp({ host, port, createMcpServer, routes, logger })`
|
||||
returning `{ url, close }` so an agent environment server can add plain routes
|
||||
beside `/mcp` without a second bridge.
|
||||
- Keep listener failure causes out of stderr behind a fixed message while the
|
||||
typed listen error carries the system cause (such as `EADDRINUSE`) into the
|
||||
error-level log line.
|
||||
|
||||
## Acceptance and limits
|
||||
|
||||
Adapters tests cover the bridge (method, URL, headers, body, GET without body,
|
||||
incremental streaming, abort and cancel) and the listener (listening line, one
|
||||
factory call across three sessions, replacement of an open session by the next
|
||||
`initialize`, two `initialize` requests racing to replace one session while the
|
||||
instance's shutdown hook stays installed, a working tool call after `DELETE`,
|
||||
the instance's shutdown hook firing only when the listener closes, plain
|
||||
routes, 404, Host refusal, port release, typed bind failure naming
|
||||
`EADDRINUSE`). The dispatcher test covers the new routes and keeps `mcp add
|
||||
--http` with the CLI. An mcp-boundary test spawns the built CLI and drives
|
||||
initialize, tools/list, `binary_session` and `DELETE` with the SDK streamable
|
||||
HTTP client, then proves `binary_session` still answers in the next session and
|
||||
in a session that replaced one left open; a process-boundary test asserts
|
||||
`143`/`130` exits and a released port. Concurrent clients are not supported:
|
||||
the newest `initialize` owns the server. The non-loopback relaxation is covered
|
||||
only through its pure allowlist helper; no wide bind runs in tests. Response
|
||||
budgets keep their stdio-derived floor; registrations, `rea mcp doctor` and
|
||||
`server.json` stay stdio.
|
||||
|
||||
## Deferred issue criteria
|
||||
|
||||
Serving an environment manifest with replayed records, its environment card and
|
||||
the exported bundle are later increments of #4933. This PR must reference rather
|
||||
than close the issue.
|
||||
@@ -0,0 +1,11 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Define single-feature requirements and implementation plan before code.
|
||||
- [x] Widen the runtime serve seam and shutdown handle to transport-neutral types.
|
||||
- [x] Implement the node:http bridge and the streamable HTTP listener module.
|
||||
- [x] Parse flags and `MCP_HOST`/`MCP_PORT`, route the dispatcher, add signals-only shutdown.
|
||||
- [x] Document the transport, CLI entry, exit status and architecture label.
|
||||
- [x] Verify bridge, listener, dispatcher, SDK client, Python client and shutdown behavior.
|
||||
- [x] Keep providers alive across sessions, replace a session left open, name the bind cause in the log.
|
||||
- [x] Resolve the re-review: capture the shutdown hook once, one shutdown registration primitive, flags-only HTTP routing.
|
||||
- [x] Run relevant repository checks and prepare a separately scoped PR.
|
||||
@@ -0,0 +1,297 @@
|
||||
# Verification
|
||||
|
||||
Verified base: 18aa9af2b2c26fd114113dd39528a91224801840 (fetched main).
|
||||
|
||||
The worktree was moved onto that base with the slice's changes kept
|
||||
uncommitted, then reinstalled with `npm ci` (exit 0; 540 packages added, 541
|
||||
audited; 6.56 s elapsed, 3.85 s user, 8.07 s sys, 881459200 bytes maximum
|
||||
resident set size). After the edits of this pass the tree was rebuilt with
|
||||
`npm run build` (exit 0, three turbo tasks; 5.23 s elapsed, 13.94 s user,
|
||||
2.93 s sys, 2964652032 bytes) and `npm run test:prepare` (exit 0, six tasks
|
||||
with three cache hits; 1.37 s elapsed, 3.07 s user, 0.76 s sys, 422625280
|
||||
bytes); `dist/mcp/` carries the rebuilt `serveStreamableHttp-*.js` chunk beside
|
||||
`main.js`.
|
||||
|
||||
Static gates on the final tree: `npm run check:fast` passed (typecheck and
|
||||
lint, two tasks; 4.69 s elapsed, 24.25 s user, 3.00 s sys, 4480139264 bytes).
|
||||
`npx oxlint` over the files touched in this pass reported no findings. After
|
||||
the last edit, including this record, `npm run format` ran over 4032 files
|
||||
(0.68 s elapsed, 4.11 s user, 0.74 s sys, 227393536 bytes) and `npm run
|
||||
format:check` passed (0.69 s elapsed, 4.09 s user, 0.62 s sys, 216989696
|
||||
bytes). `npx knip` exited 0 (3.80 s elapsed, 5.10 s user, 0.72 s sys,
|
||||
868679680 bytes). `npm run check:ci:tasks` passed all eight tasks, seven of
|
||||
them turbo cache hits from the run on the same sources (0.88 s elapsed, 4.52 s
|
||||
user, 1.04 s sys, 215171072 bytes): jscpd reported 2576 clones and 3.35%
|
||||
duplicated lines across 3143 files under the 3.5% threshold, lint:built 0
|
||||
errors over 3734 files, and format:check ran fresh. `npm run docs:check`
|
||||
passed, including the README consistency test (5 cases; 3.27 s elapsed,
|
||||
3.43 s user, 0.80 s sys, 288866304 bytes). `git diff --stat` lists 13 tracked
|
||||
files, none under `src/domain/`.
|
||||
|
||||
Source-group tests: `/usr/bin/time -l npx vitest run --project adapters
|
||||
src/server/http src/server/shutdownRegistration.test.ts` passed 3 files, 17
|
||||
tests in 2.74 s elapsed, 3.12 s user, 0.93 s sys, 289767424 bytes.
|
||||
`serveStreamableHttp.test.ts` gained the concurrent case: a slot with one open
|
||||
session receives two `initialize` requests through `Promise.all`, exactly one
|
||||
answers `200` with a session id and the other `400`, the instance's `onclose`
|
||||
hook has fired zero times, and `slot.close()` fires it once. Run before the fix
|
||||
against the same file, that case failed with the hook count `0` instead of `1`
|
||||
(1 failed, 5 passed; 1.07 s elapsed, 0.84 s user, 0.21 s sys, 158187520
|
||||
bytes), which is the pinned no-op hook the review described.
|
||||
`shutdownRegistration.test.ts` asserts that `registerProcessShutdown` hands
|
||||
the handler `SIGINT`/`SIGTERM` by name and `undefined` for stdin end, that
|
||||
omitting stdin registers no input listener, and that `registerSignalShutdown`
|
||||
reports `143`/`130`.
|
||||
|
||||
Built-group tests ran after `npm run test:prepare`. The targeted dispatcher
|
||||
run, `/usr/bin/time -l npx vitest run --project cli-boundary
|
||||
tests/boundary/cli/dispatcher.test.ts`, passed 3 tests in 1.08 s elapsed,
|
||||
0.89 s user, 0.26 s sys, 153026560 bytes, now including `mcp add --http`
|
||||
routed to the CLI beside `mcp --http --port 0`, `--mcp --http` and `mcp --host
|
||||
127.0.0.1 --http` routed to the HTTP entry and `mcp doctor --http`, `mcp add`
|
||||
and `--mcp extra` keeping their routes.
|
||||
|
||||
The four full projects ran in sequence while a sibling worktree ran its own
|
||||
`cli-boundary`, `mcp-boundary` and `acceptance` chain on the same host.
|
||||
`/usr/bin/time -l npx vitest run --project adapters` passed 247 files, 3945
|
||||
tests with 8 skipped, in 93.77 s elapsed, 80.21 s user, 30.69 s sys, 344260608
|
||||
bytes. `/usr/bin/time -l npx vitest run --project mcp-boundary` passed 155
|
||||
files with 2 skipped, 910 tests with 7 skipped, in 167.51 s elapsed, 321.24 s
|
||||
user, 51.30 s sys, 4923490304 bytes (the Vitest parent, its workers and the
|
||||
spawned CLI children). `/usr/bin/time -l npx vitest run --project
|
||||
process-boundary` exited 1: 56 files passed, 2 skipped and 2 failed, 460 tests
|
||||
passed, 19 skipped and 7 failed, in 544.81 s elapsed, 160.89 s user, 133.49 s
|
||||
sys, 1028931584 bytes. `/usr/bin/time -l npx vitest run --project
|
||||
cli-boundary` exited 1: 108 files passed and 3 failed, 658 tests passed, 2
|
||||
skipped and 7 failed, in 452.34 s elapsed, 734.29 s user, 234.02 s sys,
|
||||
648495104 bytes. No failing file belongs to this slice.
|
||||
|
||||
The failures split into two kinds. Load: six `jadxIntegration.test.ts` cases
|
||||
hit the 30 s test or 10 s hook timeout, and six `javascriptApplication*.test.ts`
|
||||
cases failed with the worker's result-capacity constraint; re-run alone in this
|
||||
worktree after the other chains ended, `jadxIntegration.test.ts` passed all 48
|
||||
tests and both JavaScript files passed all 20 tests (`cli-boundary` three files:
|
||||
20 passed, 1 failed, 23.38 s elapsed, 32.99 s user, 10.97 s sys, 343244800
|
||||
bytes; `process-boundary` two files: 48 passed, 1 failed, 62.48 s elapsed,
|
||||
21.89 s user, 17.55 s sys, 686620672 bytes). Pre-existing on main in this
|
||||
environment: `closedStdout.test.ts` "stops quietly when the reader closes
|
||||
during streamed JSON" exits `1` instead of `141` with 1009 bytes received and
|
||||
empty stderr, and `releaseArtifactCanary.test.ts` fails parsing `npm pack
|
||||
--json`, which npm 12.0.2 emits as an object keyed by package name rather than
|
||||
the tuple the test expects. Both reproduce identically on the clean sibling
|
||||
clone at `52b73ee857029a3ab8f5c6eeef233b295951a46c` (no tracked or untracked
|
||||
changes; `closedStdout` 1 failed, 3 passed, 8.86 s elapsed, 6.98 s user, 2.29 s
|
||||
sys, 348012544 bytes; the canary 1 failed, 7.91 s elapsed, 7.76 s user, 1.84 s
|
||||
sys, 668729344 bytes). That clone predates the verified base by 15 commits,
|
||||
none touching either test or `scripts/rea.mjs`. Because the dispatcher is on
|
||||
the `closedStdout` path, its scenario was also replayed against this
|
||||
worktree's built `dist/` with the `18aa9af2` `scripts/rea.mjs` and with the
|
||||
edited one, three runs each: all six exited `1` with 1009 bytes received and
|
||||
empty stderr. Nothing in this pass was changed for these failures.
|
||||
|
||||
Cross-implementation check with the Python `mcp` 1.30.0 client (httpx 0.28.1,
|
||||
Python 3.12.12) against the built `node scripts/rea.mjs mcp --http --port 0 <
|
||||
/dev/null`: stderr announced
|
||||
`{"transport":"streamable-http","url":"http://127.0.0.1:57093/mcp","msg":"MCP transport listening"}`.
|
||||
Session A: `initialize` POST returned `200 text/event-stream` with
|
||||
`mcp-session-id`; `notifications/initialized` returned `202` with no body; the
|
||||
standalone GET and the `tools/list` and `tools/call` POSTs returned `200
|
||||
text/event-stream`; `tools/list` reported 207 tools including
|
||||
`binary_session`; `binary_session` with `{}` returned `isError=False`; `DELETE
|
||||
/mcp` returned `200`. Session B, opened after that `DELETE`, initialized with a
|
||||
new session id, listed the same 207 tools, and `binary_session` again returned
|
||||
`isError=False` before its own `DELETE` returned `200`. The server logged two
|
||||
`MCP HTTP session opened` and two `MCP HTTP session closed by the client`
|
||||
lines, SIGTERM then exited it with status `143`, and stdout stayed empty. The
|
||||
methods, statuses and session-id handling of session A match the recorded
|
||||
reference exchange step for step; the reference's echo server labelled its
|
||||
`202` and `DELETE` replies `application/json`, where this listener sends no
|
||||
content type for those bodiless replies.
|
||||
|
||||
## Independent review
|
||||
|
||||
An adversarial review of the first implementation reproduced that, after a
|
||||
`DELETE`, the next session could initialize and list tools but every tool call
|
||||
failed with a `cancelled` error because the SDK's `DELETE` handling closed the
|
||||
transport and the server's `onclose` hooks ran provider teardown. It also found
|
||||
that a session ended without `DELETE` blocked the listener until restart, that
|
||||
the bind-failure log line omitted the system cause, a duplicated foreign-`Host`
|
||||
probe, a missing trailing newline on `rea mcp --http --help`, and a static
|
||||
import of the HTTP listener on the stdio path. Each finding is addressed and
|
||||
covered by the adapters, mcp-boundary and Python checks in this document.
|
||||
|
||||
The re-review of the rebased tree raised four points, all resolved here. Two
|
||||
concurrent `initialize` POSTs could leave the no-op `onclose` hook installed,
|
||||
because each session end captured the hook current at that moment and the
|
||||
later `finally` restored the other call's no-op; `McpSessionSlot` now captures
|
||||
the instance's hook once, after the factory returns the fully wrapped server,
|
||||
and every session end restores exactly that, with the adapters case above
|
||||
failing before and passing after the change. The optional stdin parameter of
|
||||
`registerProcessShutdown` was dead in production; `registerSignalShutdown` is
|
||||
now a projection of `registerProcessShutdown(handler, { signals })`, so one
|
||||
primitive owns registration while the handler learns the signal and keeps
|
||||
`130`/`143`. The dispatcher routed any `mcp` invocation containing `--http` to
|
||||
the HTTP entry; it now does so only when the first token after `mcp` or
|
||||
`--mcp` is a flag, so `rea mcp add --http` keeps the CLI's own diagnostic, and
|
||||
the dispatcher test covers that case. The formatter hunk in
|
||||
`src/domain/propertyListXml.test.ts` noted in the earlier record is gone after
|
||||
the rebase, as `git diff --stat` confirms. The replacement rule stays as
|
||||
documented, and `docs/mcp-contracts.md` now states it in one sentence with its
|
||||
rationale so maintainers can decide on it.
|
||||
|
||||
## Scope and measurement limits
|
||||
|
||||
The concurrent-`initialize` case is exercised at the slot with in-memory
|
||||
`Request` objects, where both calls advance in lockstep; over TCP two
|
||||
connections are served in separate event-loop turns, so the race is reachable
|
||||
only in-process and the listener-level behaviour remains "the newest client
|
||||
owns the server, the other sees `400` or `404`". The non-loopback relaxation
|
||||
path is covered only through the pure allowlist helper and the loopback smoke
|
||||
runs; no wide bind was exercised. Sessions are served one at a time;
|
||||
concurrent clients are not supported. The two failures that reproduce on the
|
||||
clean clone are environmental (macOS with Node 22.23.1 and npm 12.0.2) and are
|
||||
left for their owners; the load-induced timeouts were classified by isolated
|
||||
re-runs, not by repeating the full projects on an idle host. The error-level
|
||||
log line that names the bind cause was observed in earlier runs, not asserted
|
||||
by `mainErrors.test.ts`. Response budgets keep the stdio-derived floor and were
|
||||
not re-measured over HTTP. Windows was not run.
|
||||
|
||||
## Automated review findings on the PR (2026-10-18)
|
||||
|
||||
Codex and Greptile left eight inline findings on morluto/rea#4955 at
|
||||
46dd8f0e; each was checked against the code and addressed in one follow-up
|
||||
commit, with a regression test where the behaviour is observable:
|
||||
|
||||
- A late `initialize` could arm a fresh transport after shutdown (Codex, P1):
|
||||
the slot now records that it is closed, `close()` joins an arming session
|
||||
before ending it, and every request after that is refused with `503`. Test:
|
||||
an initialize racing `close()` gets `503`, the shutdown hook fires once, and
|
||||
the next request gets `503`.
|
||||
- A failed `close()` was memoized forever (both reviewers, P1): the close runs
|
||||
three stages once each and a failed stage is retried by the next call. Test:
|
||||
the instance's first `close()` rejects, the second call resolves and the port
|
||||
is released.
|
||||
- Loopback aliases skipped Host validation (both reviewers, P2 and P1): a bind
|
||||
is now loopback by address class (`127.0.0.0/8`, `::1`, IPv4-mapped loopback,
|
||||
`localhost` names) after the same normalization the SDK applies to the Host
|
||||
header, and the allowlist adds the bind's own hostname so requests to
|
||||
`127.0.0.2` or `localhost.` are still served. Test: twelve spellings.
|
||||
- A sessionless POST could close the live session before any header check
|
||||
(Greptile, P1): on loopback binds the route validates `Origin` with the same
|
||||
allowlist as `Host`, and only a POST carrying `Content-Type:
|
||||
application/json` and an `Accept` naming both `application/json` and
|
||||
`text/event-stream` may replace a session; others reach the live transport,
|
||||
which refuses them. Test: a `text/plain` initialize and one with a foreign
|
||||
`Origin` are refused (4xx and 403) while the connected client keeps echoing.
|
||||
- The bound URL disappeared at quiet log levels (Greptile, P2): the listening
|
||||
line is written through a child logger pinned to `info`. Test: a `warn`
|
||||
logger still receives the line with the URL.
|
||||
- `rea mcp -h --http` reached the ordinary CLI (Greptile, P2): the dispatcher
|
||||
predicate accepts any flag after the entry. Test: the dispatcher case.
|
||||
|
||||
Checks after the change: `oxfmt --check .` clean; `tsc --noEmit` clean;
|
||||
`oxlint` on the touched sources clean after extracting the staged close into
|
||||
its own function; `npx knip` no findings; `npm run jscpd` 3.35 % duplicated
|
||||
lines; `scan:todos` none; `verify:test-discovery` 1,583 files once; `vitest
|
||||
run` over the listener and bridge adapters tests 14 passed in 2 files; after
|
||||
`npm run build:cached`, `vitest run` over the dispatcher, option, streamable
|
||||
HTTP, shutdown and main-error boundary tests 22 passed in 5 files.
|
||||
|
||||
A second automated round on the fix (Codex, two P2 findings) was addressed in
|
||||
the next commit: the replacement guard compares the media type proper, so
|
||||
`application/json-patch+json` no longer counts as JSON (test: such an
|
||||
initialize is refused while the connected client keeps echoing), and the
|
||||
server instance is created only after the listener has bound, so a bind
|
||||
failure has nothing to clean up and a factory failure releases the listener
|
||||
and rethrows (test: a throwing factory rejects without announcing a URL).
|
||||
|
||||
A third round (Greptile and Codex, five findings) was addressed next: the
|
||||
listener's `error` forwarding to `onerror` is registered after the bind, so
|
||||
`EADDRINUSE` prints only the listen-failure message (test: an occupied port
|
||||
rejects without calling `onerror`); scoped IPv6 literals are refused before
|
||||
binding and a URL that cannot be formed from the bound address closes the
|
||||
listener (test: `fe80::1%lo0` rejects before the factory runs); loopback is
|
||||
classified from the bound address with the configured name added to the
|
||||
allowlist (test: `ip6-localhost` bound to `::1`, `dev.local` bound to
|
||||
`127.0.0.1`); and the MCP runtime passes `closeInstance: false` so the factory
|
||||
owner in `startMcpTransport` remains the single owner of instance cleanup
|
||||
(test: `close` leaves the instance open until its owner closes it).
|
||||
|
||||
A fourth round (Codex, two findings) was addressed next: the replacement
|
||||
guard parses `Accept` into media ranges and requires exact `application/json`
|
||||
and `text/event-stream` entries (test: `application/json-patch+json,
|
||||
text/event-streaming` is refused while the connected client keeps echoing),
|
||||
and a session whose transport shutdown rejected stays owned so the next close
|
||||
retries it (test: a failing transport close rejects `close()` once, the second
|
||||
call completes, the shutdown hook fires once and later requests get `503`).
|
||||
|
||||
A fifth round (Codex, three findings) was addressed next: the instance's
|
||||
shutdown hook now runs exactly once even when the listener closes without a
|
||||
live session (test: two closes of an idle slot fire it once; an idle listener
|
||||
close fires it once); the initialize probe no longer awaits cancelling its
|
||||
cloned body, which is one branch of a tee and would wait for the original
|
||||
request to drain (test: a 4.5 MB sessionless initialize is refused promptly
|
||||
while the connected client keeps echoing); and the node:http bridge passes
|
||||
repeated `Set-Cookie` values as an array instead of collapsing them (test: two
|
||||
cookies from a composed route both arrive).
|
||||
|
||||
A sixth round (Codex, one P1) was addressed next: signal handlers were only
|
||||
installed after the asynchronous startup returned, so a SIGINT or SIGTERM
|
||||
during target opening, provider loading or the HTTP bind ended the process
|
||||
without cleanup. The HTTP entry now latches signals before startup, the
|
||||
lifecycle owner picks the recorded signal up as soon as it registers, and a
|
||||
second signal during startup exits at once (test: the latch records `143`,
|
||||
exits on the second signal, hands the status over on release and leaves no
|
||||
listeners behind).
|
||||
|
||||
Both reviewers then noted that the latched signal's status was overwritten by
|
||||
the entry's normal `0` return: the HTTP entry now owns its exit status through
|
||||
`createExitStatus`, where a `0` after a recorded signal keeps the signal's
|
||||
status and a real failure still wins (unit test), and a process test sends the
|
||||
signal the moment the listening line appears and asserts the signal status.
|
||||
|
||||
An eighth finding (Codex): a reply sent before the MCP handler read the body
|
||||
left that body unread on a paused bridged stream, so a slow uploader could
|
||||
hold the socket. The bridge now closes the connection after any reply whose
|
||||
request body was never read (test: an unknown-route POST and a foreign-Host
|
||||
POST are answered with `Connection: close`, while a keep-alive GET without a
|
||||
body keeps its connection).
|
||||
|
||||
A ninth round (Greptile and Codex): request conversion ran outside the guard,
|
||||
so a method Fetch refuses (`TRACE`) could crash the process; the conversion is
|
||||
now guarded and answered with `405` and a closed connection (test: `TRACE`
|
||||
gets `405` and the next client still echoes). The unread-body decision now
|
||||
comes from the incoming message's own body declaration, so a `GET` or `HEAD`
|
||||
that carries a body also closes, and the regression uses a keep-alive agent
|
||||
and checks that the socket actually closes after the early reply while a
|
||||
bodiless `GET` keeps its socket.
|
||||
|
||||
A tenth round (Codex, three findings): the takeover guard now requires a lone
|
||||
`initialize`, mirroring the transport (test: a batch of `initialize` plus
|
||||
`notifications/initialized` is refused while the connected client keeps
|
||||
echoing); the shutdown hook records completion only after it returns, so a
|
||||
hook that threw stays eligible for the next close (test: a hook that throws
|
||||
once runs twice across two closes and then no more); and the bridge joins the
|
||||
request target onto the listener's origin verbatim, refusing absolute-form
|
||||
targets (test: `//attacker.example/card` is a 404 for that path, a plain route
|
||||
sees the bound host only, and `http://attacker.example/card` gets `400`).
|
||||
|
||||
An eleventh finding (Codex): URL construction canonicalized dot segments, so
|
||||
`/public/../mcp` reached `/mcp`. The bridge now refuses any target whose path
|
||||
differs after URL parsing, so routing always sees the path the client sent
|
||||
(test: `/public/../mcp`, `/public/%2e%2e/mcp` and `/./mcp` get `400`; `/card?x=1`
|
||||
still routes).
|
||||
|
||||
A twelfth finding (Codex): `bodyUsed` means disturbed, not consumed, so a
|
||||
composed route that read one chunk of an upload left the rest paused on a
|
||||
keep-alive socket. The close decision now uses the incoming message's end of
|
||||
stream (test: a route that reads one chunk of a 256 KiB upload is answered
|
||||
with `Connection: close` and a closed socket; a route that reads it all keeps
|
||||
the connection).
|
||||
|
||||
A thirteenth round (Codex): an `Accept` range with `q=0` no longer counts toward
|
||||
the takeover guard (test: `application/json;q=0, text/event-stream` is refused
|
||||
while the connected client keeps echoing). The same round noted that a failed
|
||||
startup leaves cleanup of the already opened session and providers to a single
|
||||
attempt; that path is the pre-existing stdio startup-failure cleanup, which the
|
||||
HTTP mode now reaches on `EADDRINUSE`, and is left for the maintainers to decide.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Implementation plan
|
||||
|
||||
1. Lift the api-surface accumulator's shared pieces into
|
||||
`webNetworkRouteTemplates.ts` (format rules, media-type essence, request-line
|
||||
and query-name readers, route scanning, byte allowances) and an abstract
|
||||
`WebNetworkProjectionAccumulator` (retained/temporary budget, admission,
|
||||
limitations) with `admitHarCaptureParent` for the shared parent guards, so
|
||||
both grouping views own one interpretation.
|
||||
2. Add `environmentManifestRules.ts` as pure functions: the opt-in template
|
||||
rules, re-bucketing and sibling merging, parameter naming, path templates and
|
||||
deterministic tool names.
|
||||
3. Add `EnvironmentManifestAccumulator.ts`: per-record pointers, group state,
|
||||
primary-origin selection, body-shape parsing through `parseJsonObject` with
|
||||
top-level key types only, without walking nested values, exclusion reasons,
|
||||
fixed unknowns and the finished manifest fields.
|
||||
4. Add `environmentManifestView.ts` with the strict request schema (defaults
|
||||
applied in the echoed view), the result fields and the projector; register
|
||||
the kind first in `projectAnalysisView`, extend the result union and
|
||||
`UnsignedAnalysisView`, and wire the accumulator and confidence rule in
|
||||
`AnalysisViewService`.
|
||||
5. Publish the `inspect_analysis_view` description and a fixture-generated
|
||||
example; port the Grafana-shaped HAR into a source-owned TypeScript builder.
|
||||
6. Cover domain, accumulator budget, service, MCP boundary, CLI boundary and
|
||||
acceptance behavior; extend the real-capture verifier; document the view in
|
||||
the capture guide, the contracts reference and the authored skill.
|
||||
|
||||
No new tool, provider, side effect or network activity is introduced. The
|
||||
manifest is a projection of already retained Evidence; replaying or serving it
|
||||
is deferred.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Environment manifest view from web network captures
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
`inspect_analysis_view` accepts `view: {"kind": "environment-manifest"}` for
|
||||
retained HAR Evidence and projects the primary origin's recorded request lines
|
||||
into a replayable tool surface: deterministic tool names, path templates,
|
||||
parameters, observed query names, value-free body shapes, status and
|
||||
content-type sets and record pointers, with excluded groups, three fixed
|
||||
unknowns and the shared projection representation budget. This increment adds
|
||||
the view kind only; serving and exporting an agent environment from the
|
||||
manifest are later increments.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Accept only `inspect_web_network_capture` HAR parents; refuse native
|
||||
mitmproxy, SAZ and qlog parents, and parents whose Evidence does not record
|
||||
`sensitive_value_count: 0`, with the same typed unsupported errors as the
|
||||
api-surface view.
|
||||
- Without opt-in rules, every tool template must equal the api-surface template
|
||||
of the same group; the four format rules always apply with `basis: inferred`.
|
||||
- Apply `after-uid-literal`, `opaque-token` and `sibling-variance` only when
|
||||
listed, each at most once, in that order. With `sibling-variance` selected,
|
||||
fold literal siblings within its comparison scope into existing opt-in
|
||||
parameters unless they are stop words, whichever was recorded first. With only
|
||||
`opaque-token`, a non-opaque literal remains a separate tool. Never fold a
|
||||
literal into a format-rule parameter or template a stop word;
|
||||
`sibling-variance` compares routes of at most 32 segments and the manifest
|
||||
states when a route exceeded that.
|
||||
- Derive tool names from the method and route literals deterministically, at
|
||||
most 60 characters, unique within one manifest.
|
||||
- Derive `body_shape` for `POST`, `PUT` and `PATCH` from the tool's earliest
|
||||
record when it carries a JSON-typed request body, as top-level key types
|
||||
only without walking nested values; copy no header, cookie, query or body
|
||||
value anywhere in the result.
|
||||
- List records as pointers (`ordinal`, `method`, `path`, `status`,
|
||||
`content_type`) and excluded groups with reason `other-origin`,
|
||||
`ui-page-or-asset` or `non-json`.
|
||||
- Mark the Evidence inferred whenever any tool or excluded group is templated,
|
||||
derived otherwise; keep the parent's authority.
|
||||
- Admit the capture clone, the inline envelope, every retained pointer, group,
|
||||
route part, observed value, query name and body-shape key before allocation,
|
||||
and charge body parsing as temporary bytes, within the existing 96 MiB budget.
|
||||
|
||||
## Acceptance and limits
|
||||
|
||||
A source-owned Grafana-shaped HAR fixture (28 recorded exchanges on two
|
||||
origins) drives domain, service, MCP, CLI and acceptance coverage. Without
|
||||
rules the templates equal the api-surface grouping; with all three rules the
|
||||
fixture yields `/api/dashboards/uid/{uid}`, `/api/folders/{uid}` folding
|
||||
`general`, `/api/plugins/{plugin_id}/settings` and
|
||||
`/api/datasources/uid/{uid}/health`. A published contract example is
|
||||
reproduced byte-for-byte from fixture bytes through the real decoder and view
|
||||
service. The view performs no network activity and replays nothing; it does
|
||||
not establish handler identity, schemas, authorization or state transitions.
|
||||
|
||||
## Deferred issue criteria
|
||||
|
||||
Serving a manifest over streamable HTTP, exporting an agent environment from a
|
||||
manifest and the hosted verification lane are later independent increments of
|
||||
#4933. This PR must reference rather than close the whole issue.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the view contract, requirements and deferred increments.
|
||||
- [x] Factor shared route, media-type and budget helpers out of the api-surface
|
||||
accumulator without changing its behavior.
|
||||
- [x] Implement the opt-in template rules, naming and manifest accumulator.
|
||||
- [x] Register the `environment-manifest` kind, result schema, service wiring and
|
||||
confidence rule.
|
||||
- [x] Port the Grafana-shaped HAR fixture and publish a reproducible example.
|
||||
- [x] Add domain, service, MCP, CLI and acceptance coverage.
|
||||
- [x] Document the view in the capture guide, contracts reference and skill.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
@@ -0,0 +1,186 @@
|
||||
# Verification
|
||||
|
||||
Verified base: 18aa9af2b2c26fd114113dd39528a91224801840 (fetched main).
|
||||
|
||||
Executed on the implementation worktree after `npm ci`, `npm run build` and
|
||||
`npm run test:prepare`; every check below ran as written in this pass. The
|
||||
built-group projects and the built-CLI checks ran against a `dist/` rebuilt
|
||||
after the last source change.
|
||||
|
||||
- `npm run check:fast`: typecheck and lint passed (2 tasks successful).
|
||||
- `npm run format` then `npm run format:check`: all 4028 matched files use the
|
||||
correct format.
|
||||
- `npx knip`: ok (exit 0, no unused files, exports or dependencies reported).
|
||||
- `npm run check:ci:tasks`: typecheck, lint:built (3730 files, 0 errors), format:check, knip,
|
||||
metadata:check, jscpd (3.35% duplicated lines over 3140 files and 2581
|
||||
clones, under the 3.5% gate) and the tech-debt marker scan passed (8 tasks,
|
||||
3 replayed from this worktree's cache).
|
||||
- `npm run docs:check`: the six built documentation checks and the README
|
||||
consistency conformance test (5 tests) passed; no tracked generated file
|
||||
changed.
|
||||
- `npx vitest run --project domain --project services` (full source group):
|
||||
312 files, 5412 tests passed, 3 pre-existing skips.
|
||||
- `npx vitest run --project cli-boundary` (full project): 111 files; 659 tests passed, 2
|
||||
pre-existing skips, 8 failed in three files that drive no analysis-view
|
||||
command (`closedStdout`, `commandFinalizationCancellation`,
|
||||
`javascriptApplicationJsonOutput`). Re-run in isolation on this worktree the
|
||||
three WASM interrupt cases and the four JavaScript JSON-output cases passed
|
||||
and only `closedStdout` "stops quietly when the reader closes during streamed
|
||||
JSON" (`inspect-sqlite-database`) exited 1 instead of 141. The same three
|
||||
files run on a clean export of 18aa9af2 (`git archive`, fresh `npm ci`,
|
||||
`npm run build`, `npm run test:prepare`, nothing from this slice) reproduced
|
||||
exactly that one failure and passed the other 16, so none of the eight is
|
||||
attributable to this slice and none was changed here.
|
||||
- `npx vitest run --project mcp-boundary` (full project): 153 files passed, 2
|
||||
pre-existing file skips; 901 tests passed, 7 pre-existing skips, no failures.
|
||||
- `npx vitest run --project acceptance` (full project): 123 files passed, 1
|
||||
pre-existing file skip; 608 tests passed, 13 pre-existing skips, no failures.
|
||||
- Built CLI end-to-end on the 28-record Grafana-shaped capture:
|
||||
`inspect-web-network-capture ... har --format json` then
|
||||
`inspect-analysis-view` with `environment-manifest`, with and without the
|
||||
three opt-in rules, and with `api-surface`, twice each. Without rules the 22
|
||||
tool templates plus the 3 primary-origin exclusions equal the 25 api-surface
|
||||
groups on that origin and every `path_template` equals its
|
||||
`api_surface_template`; with rules `/api/dashboards/uid/{uid}`,
|
||||
`/api/folders/{uid}` (observed values `fg1jo91e57e2ob` and `general`, records
|
||||
12 and 13), `/api/plugins/{plugin_id}/settings` and
|
||||
`/api/datasources/uid/{uid}/health` appear across 19 tools with 13 templated
|
||||
groups; `normalized_result` and `evidence_id` are identical across the two
|
||||
runs of each view; records carry only pointer keys; no recorded body or query
|
||||
value appears; three unknowns and `view_digest` are present; confidence is
|
||||
`inferred` with authority `historical-reference`.
|
||||
- Built CLI on the same capture extended with `GET /api/teams/search` and
|
||||
`GET /api/teams/mine` (30 records), all three rules selected:
|
||||
`/api/teams/{numeric}` keeps observed value `1` only and `get_teams_search`
|
||||
and `get_teams_mine` stay their own tools; 21 tools, 13 templated groups.
|
||||
|
||||
## Independent review
|
||||
|
||||
Findings repaired at their owning boundary:
|
||||
|
||||
1. A `__proto__` key in a recorded JSON body became a prototype rather than an
|
||||
own property, and the schema clone then dropped it silently. The manifest
|
||||
accumulator builds body shapes with `Object.fromEntries`, omits that key and
|
||||
states the omission as a limitation; a domain regression covers it.
|
||||
2. The contract example carried a readonly tuple that the mutable JSON example
|
||||
type rejects; the example keeps a literal-typed mutable tuple.
|
||||
3. Folding a literal sibling into an already templated position took the parts
|
||||
of the earliest group, so a literal recorded before the token kept its
|
||||
literal, dropped the parameter and claimed the token's record under a
|
||||
template it does not match. `mergeGroups` now takes the templated group as
|
||||
the shape for parts while the earliest group still names
|
||||
`api_surface_template`, `body_shape` and record order; a domain regression
|
||||
with the literal recorded first covers it.
|
||||
4. The request-body parse accepted `text/plain` and `text/markdown` bodies; it
|
||||
accepts only `application/json` and `application/*+json`, and the capture
|
||||
guide states the earliest-record rule; a domain test covers both media
|
||||
types.
|
||||
5. A repeated entry in `template_rules` was accepted, echoed and hashed into
|
||||
`view_digest` while the accumulator deduplicated it; the request schema
|
||||
refuses repeats with `template_rules must not repeat a rule`.
|
||||
6. The sibling-variance fold-in absorbed a non-conforming literal into a
|
||||
parameter produced by one of the four format rules, so `/api/teams/mine`
|
||||
became an observed value of `/api/teams/{numeric}` and the published rule no
|
||||
longer described every observed value. Fold-in is restricted to parameters
|
||||
produced by the opt-in rules; literal siblings of a format-rule parameter
|
||||
stay their own tools or merge among themselves under `sibling-variance`
|
||||
only when identifier-like. A domain test with `/api/teams/1`,
|
||||
`/api/teams/2`, `/api/teams/search` and `/api/teams/mine`, the capture
|
||||
guide and the spec state the rule; the built-CLI check above exercises it.
|
||||
|
||||
## Hosted lane
|
||||
|
||||
- `real-web-network-captures` (hosted ubuntu-24.04, workflow run
|
||||
https://github.com/earakely-scale/rea/actions/runs/39097105064 on the fork
|
||||
at 2d95597f) passed with the manifest block of
|
||||
`scripts/verify-web-network-captures.mjs` executed against the lane's
|
||||
producer and mitmproxy captures. The first run at 61d35369 failed only in
|
||||
that block: the refusal check stringified the whole MCP result, so the
|
||||
declared `[` literal matched the result envelope; the follow-up compares the
|
||||
parsed refusal as the api-surface branch does.
|
||||
|
||||
## Scope and measurement limits
|
||||
|
||||
macOS `/usr/bin/time -l`, command accounting only: `npm run check:fast`
|
||||
reported 5.23 seconds elapsed, 24.20 user and 2.92 system seconds, and
|
||||
4449140736 bytes maximum RSS; `npm run format` 0.74 seconds elapsed, 4.14
|
||||
user and 0.73 system seconds, and 223363072 bytes maximum RSS;
|
||||
`npm run format:check` 0.66 seconds elapsed, 4.03 user and 0.61 system
|
||||
seconds, and 218759168 bytes maximum RSS; the full source group reported
|
||||
25.10 seconds elapsed, 67.26 user and 2.72 system seconds, and 2659172352
|
||||
bytes maximum RSS; `npx knip` 3.68 seconds elapsed, 5.24 user and 0.70 system
|
||||
seconds, and 865599488 bytes maximum RSS; `npm run check:ci:tasks` 4.65
|
||||
seconds elapsed, 15.03 user and 4.36 system seconds, and 843333632 bytes
|
||||
maximum RSS; `npm run docs:check` 3.61 seconds elapsed, 3.81 user and 0.99
|
||||
system seconds, and 289472512 bytes maximum RSS; the full `cli-boundary`
|
||||
project 425.66 seconds elapsed, 703.52 user and 206.77 system seconds, and
|
||||
630456320 bytes maximum RSS; the full `mcp-boundary` project 164.19 seconds
|
||||
elapsed, 302.85 user and 47.08 system seconds, and 4457529344 bytes maximum
|
||||
RSS; the full `acceptance` project 960.25 seconds elapsed, 1520.91 user and
|
||||
520.68 system seconds, and 800063488 bytes maximum RSS. `npx knip` ran while
|
||||
the `cli-boundary` project was executing, and the clean 18aa9af2 export's
|
||||
`npm ci`, build and test preparation overlapped the `mcp-boundary` and
|
||||
`acceptance` runs, so those elapsed figures include host contention. Token usage and full process-family peak are
|
||||
unmeasured. Local real-tool verification ran on macOS arm64; the hosted Linux lane result is recorded above.
|
||||
|
||||
## Automated review findings on the PR (2026-10-18)
|
||||
|
||||
Codex and Greptile left seven inline findings on morluto/rea#4954 at
|
||||
b6d18d23; each was checked against the code and addressed in one follow-up
|
||||
commit, with a regression test where the behaviour is observable:
|
||||
|
||||
- Body shapes walked nested values (both reviewers, P1): `#bodyShape` passed
|
||||
the text to `inferJsonShape`, whose per-node path copies grow with the square
|
||||
of depth, so a small deeply nested body could exceed the budget it had
|
||||
reserved. The reader now parses once and reads top-level key types from the
|
||||
parsed object; a test feeds a 2,500-level nested body and gets the four
|
||||
top-level types back.
|
||||
- Literal route parts were not admitted (Codex, P2): a path of many short
|
||||
literal segments materialized one part object per segment without a charge.
|
||||
Every route part is now charged before the group is created; a test shows a
|
||||
5,000-segment path refused under a 1 MiB budget while a 100-segment path is
|
||||
retained.
|
||||
- `sibling-variance` rebuilt a key over the whole route at every position
|
||||
(Greptile, P1): routes longer than 32 segments are no longer compared, the
|
||||
manifest states it in a limitation, and a test shows two 39-segment siblings
|
||||
left unmerged beside a merged short pair.
|
||||
- `after-uid-literal` templated stop words (Greptile, P1): the rule now skips
|
||||
them; a test keeps `/api/items/uid/search` literal beside
|
||||
`/api/items/uid/{uid}`.
|
||||
- Tool-name suffixes restarted at 2 on every collision (Greptile, P2): the
|
||||
registry keeps the next suffix per base; a test projects 300 dotted literals
|
||||
into `get_item` and `get_item_2` through `get_item_300`.
|
||||
- An exclusion row could lose its templated flag when two groups printed the
|
||||
same row (Codex, P2): the flag is now the logical or of both. The scenario
|
||||
needs a literal segment spelled `{numeric}`, which the WHATWG URL parser
|
||||
percent-encodes in recorded paths, so no capture reaches it; the one-line
|
||||
defence is kept without a test.
|
||||
|
||||
Checks after the change: `oxfmt --check .` clean; `tsc --noEmit` clean;
|
||||
`npm run lint` and `verify:module-boundaries` clean (the complexity warnings
|
||||
on `#append` predate the change); `npx knip` no findings; `npm run jscpd`
|
||||
3.35 % duplicated lines; `scan:todos` none; `verify:test-discovery` 1,580
|
||||
files once; `vitest run` over the view, accumulator, JSON-shape and contract
|
||||
example tests 32 passed in 4 files; `vitest run` over the service, CLI and
|
||||
MCP boundary tests for analysis views 73 passed in 3 files.
|
||||
|
||||
A sixth automated finding (Codex) was addressed after the collaborator's
|
||||
merge of main: `sibling-variance` merged a whole bucket when any one literal
|
||||
carried a digit, so `/api/items/mine` could be folded into `{item_id}`. The
|
||||
rule now partitions the bucket and merges only the identifier-like siblings
|
||||
(each with a digit or colon, or three or more hyphenated names under a plural
|
||||
literal), leaving the other literals as their own tools; with the rule alone
|
||||
the fixture therefore keeps `/api/folders/general` and
|
||||
`/api/folders/fg1jo91e57e2ob` separate, while all three rules still fold
|
||||
`general` into `/api/folders/{uid}` through the opaque-token parameter. Tests:
|
||||
the new partition case and the updated single-rule table.
|
||||
|
||||
Follow-up mixed-cohort regressions reproduced two omissions on `dd87f54e`
|
||||
with main `8307ea37`: two coded siblings prevented three qualifying hyphenated
|
||||
siblings from joining the parameter, and one coded sibling was omitted when
|
||||
three hyphenated siblings merged. Eligibility now belongs to one Set that
|
||||
combines both qualifying cohorts before the minimum group size is checked.
|
||||
The consumer reuses that Set and materializes its merge order once. Three
|
||||
cases cover both omissions and preserve the three-hyphen minimum when only
|
||||
two hyphenated siblings accompany coded identifiers; `mine` and `search`
|
||||
remain literal and every source record remains represented.
|
||||
@@ -0,0 +1,18 @@
|
||||
# Plan
|
||||
|
||||
1. Add the `evidence-file` source to `analysisViewSourceSchema`
|
||||
(`src/domain/analysisView/analysisView.ts`).
|
||||
2. Add `inspectAnalysisViewWithFiles` / `inspectAnalysisViewValidatedWithFiles`
|
||||
to `AnalysisViewService`: read with `readJsonFile`, authenticate with
|
||||
`parseEvidence`, then project through the retained path with a lookup that
|
||||
returns the file's Evidence. The synchronous path refuses an unread file
|
||||
source.
|
||||
3. Route the CLI command (with `source.path` resolved against the working
|
||||
directory) and the MCP tool through the file-aware entry point.
|
||||
4. Add `sibling-pair-variance` to `TEMPLATE_RULES`, with its merge pass before
|
||||
`sibling-variance`, its parameter names and its segment-limit limitation.
|
||||
5. Document both in the tool contract, `docs/cli.md` and
|
||||
`docs/web-network-captures.md`.
|
||||
6. Tests: service (file equals inline; a capture refused inline is accepted
|
||||
from a file; refusals), CLI and MCP boundaries, and the pair rule's merges
|
||||
and non-merges.
|
||||
@@ -0,0 +1,56 @@
|
||||
# Rich captures for the environment manifest
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
A capture large enough to hold a rich application state can be projected into
|
||||
an environment manifest whose tool list stays within what model providers
|
||||
accept. Two changes:
|
||||
|
||||
- `inspect_analysis_view` accepts a third source,
|
||||
`{"kind": "evidence-file", "path": "<absolute path>"}`. REA reads one
|
||||
completed Evidence JSON file, authenticates it, and projects it like retained
|
||||
Evidence, so its strings are bounded by the file reader instead of being
|
||||
charged as inline input.
|
||||
- The environment-manifest view gains an opt-in template rule,
|
||||
`sibling-pair-variance`, that templates two adjacent literal segments after a
|
||||
plural literal, such as `/repos/{owner}/{repo}/…`.
|
||||
|
||||
## Requirements
|
||||
|
||||
- An `evidence-file` path must be absolute and free of NUL bytes; the CLI
|
||||
resolves a relative path against the working directory before validation,
|
||||
and MCP clients pass an absolute path.
|
||||
- The file is read through the shared JSON file reader (regular file, heap and
|
||||
string-length guards) and parsed as complete Evidence. A missing or unreadable
|
||||
file is an `EvidenceFileError`; JSON that is not complete Evidence is invalid
|
||||
input reported under `source.path`.
|
||||
- Over MCP the read follows the request's cancellation and the server's
|
||||
resource scope, so a failed descriptor close stays owned for retry; a request
|
||||
cancelled before authentication, before projection or during projection
|
||||
returns typed cancellation and records nothing. Before authentication, which
|
||||
copies and hashes the parsed Evidence, REA checks that live heap headroom holds
|
||||
those copies and otherwise returns `resource_constraint`.
|
||||
- A file source projects exactly what the same Evidence projects inline or
|
||||
retained, with the same parent Evidence ID, view parameters and result.
|
||||
- The synchronous projection API refuses an unread `evidence-file` source with
|
||||
a typed capability error; the CLI and MCP adapters read files first.
|
||||
- `sibling-pair-variance` applies only when listed, before `sibling-variance`, so
|
||||
a single-position merge cannot template one half of a pair first. It
|
||||
compares groups equal in method, origin and segment count except at two
|
||||
adjacent literal positions at index 2 or later that follow a plural literal
|
||||
and precede another literal (a trailing pair, such as
|
||||
`/users/alice/followers`, names a sub-resource rather than a namespace and
|
||||
name), and merges a bucket only when it holds at least three distinct pairs and both
|
||||
positions vary. Stop words and `v1`-style versions never qualify, and routes
|
||||
of more than 32 segments are not compared, with a limitation saying so.
|
||||
- Its parameters are named from the plural literal, singular plus `_namespace`
|
||||
and then `_name`, and labelled `sibling-pair-variance`.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No change to the 96 MiB projection budget or to inline admission.
|
||||
- No change to `sibling-variance`; pairs are a separate rule so every inferred
|
||||
parameter keeps naming the rule that produced it.
|
||||
- No inference of semantic names (`owner`, `repo`) beyond the plural literal.
|
||||
@@ -0,0 +1,10 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the outcome, requirements and non-goals.
|
||||
- [x] Add the `evidence-file` source and the file-aware service entry point.
|
||||
- [x] Route the CLI command and the MCP tool through it.
|
||||
- [x] Add the `sibling-pair-variance` rule, its names and its limitation.
|
||||
- [x] Document both; extend the tool contract text.
|
||||
- [x] Add service, CLI, MCP and rule tests.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
- [x] Rebuild a rich capture with both changes and record the results.
|
||||
@@ -0,0 +1,104 @@
|
||||
# Verification
|
||||
|
||||
Verified commit: a64367a9 on main 2dd5effe.
|
||||
|
||||
## Gates
|
||||
|
||||
Run on macOS 26.5.1 with Node 22.23.1 on that tree: `npm run typecheck`, `lint`
|
||||
(oxlint plus `verify:module-boundaries`, 2172 files), `format:check`, `knip`,
|
||||
`metadata:check`, `docs:check`, `jscpd` (3.43% duplicated lines), `scan:todos`
|
||||
and `mcp-catalog:check` all exit 0. oxlint on the 12 changed TypeScript files
|
||||
reports no finding that main does not already report for the same files.
|
||||
|
||||
`npm run test:prepare`, then `npm run test:changed` (the tests related to the
|
||||
changed files): 84 files, 1531 passed, 3 skipped, 0 failed. They include:
|
||||
|
||||
- the service: an Evidence file projects exactly what the same Evidence projects
|
||||
inline; a capture whose response text exceeds the inline budget is refused
|
||||
inline with `resource_constraint` and projected from its file; a relative
|
||||
path, a missing file and JSON that is not Evidence are refused; the
|
||||
synchronous API refuses an unread file source;
|
||||
- the compiled CLI: a request naming an Evidence file projects the same
|
||||
manifest as the same Evidence inline;
|
||||
- MCP: inline, retained and file sources give identical manifests, outputs
|
||||
validate against the advertised schema, and a relative path is refused;
|
||||
- the file source module: the size estimate, the authentication headroom
|
||||
refusal, a cancelled request refused before reading, and a read within the
|
||||
caller's resource scope; the service returns typed cancellation for a
|
||||
cancelled file request; a request aborted while it runs, even after its read, returns typed
|
||||
cancellation because the service yields before each check;
|
||||
- the pair rule: three owner/repo pairs merge for collection and numeric detail
|
||||
routes, while two pairs, or three pairs sharing one namespace, stay literal;
|
||||
with `sibling-variance` also selected the pair still merges, because pairs
|
||||
are merged first. Each pair parameter keeps its role when
|
||||
`sibling-variance` later templates the segment before the pair.
|
||||
|
||||
## A rich capture
|
||||
|
||||
A Gitea 28.1.0 instance seeded through its API with an invented world (12
|
||||
people, 4 organisations, 16 public repositories, 320 issues with about 600
|
||||
comments, labels, milestones, commits, pull requests and releases) was read by
|
||||
one `capture-browser-scenario` run of 1,217 API reads: 1,218 transactions in
|
||||
24 s, 10.7 MB of Evidence, nothing dropped. Scenario parents are not on main,
|
||||
so this run used this commit merged onto the scenario-capture change
|
||||
(morluto/rea#5182) and packed for the host.
|
||||
|
||||
| | Inline source, previous rules | Evidence-file source with `sibling-pair-variance` |
|
||||
| --------------------- | ------------------------------------- | ------------------------------------------------- |
|
||||
| Manifest | `resource_constraint` (96 MiB budget) | built in 4.1 s, 459 MB maximum RSS |
|
||||
| Tools | 446 (with the budget raised by hand) | 63 |
|
||||
| Recorded exchanges | 1,218 | 1,218 |
|
||||
| Tools with one record | 77% | 49% |
|
||||
|
||||
The pair rule templated 25 route families as `{repo_namespace}/{repo_name}`.
|
||||
A model provider that accepts at most 128 tools refused every request for the
|
||||
446-tool manifest with HTTP 400.
|
||||
|
||||
The 63-tool environment was then exported, and in the AgentEnv runner its
|
||||
smoke task passed and twelve generated retrieval tasks scored 12/12 with
|
||||
`openai/gpt-oss-120b` and 12/12 with `zai-org/GLM-5.3-Flash`, with no provider
|
||||
errors. The 446-tool manifest had scored 0/12 and 8/12 on its own generated
|
||||
tasks, the 0 being the provider's tool limit rather than the model.
|
||||
|
||||
Review then found that the rule as run here also merged one id's
|
||||
sub-resources (`/users/alice/followers`, `/users/bob/following` and
|
||||
`/users/carol/starred` became one tool), and a pair now qualifies only when a
|
||||
literal follows it. The Gitea capture above predates that guard and has not
|
||||
been projected with it: whether any of the 63 tools merged sub-resources this
|
||||
way, and the tool count with bare `/repos/{owner}/{repo}` routes kept one per
|
||||
pair, are not established here.
|
||||
|
||||
## Linux follow-up (2026-10-20)
|
||||
|
||||
Based on `dd2bf3565`, including main `5fa4defcb`, with Node 24.18.0 and npm
|
||||
11.16.0. The file reader now owns authentication; invalid Evidence returns the
|
||||
selected file path and file-relative `content_issues`. Service and advertised
|
||||
MCP error-envelope regressions cover this boundary.
|
||||
|
||||
- Domain analysis views, application analysis views and error projection:
|
||||
16 files, 298 tests passed.
|
||||
- Compiled analysis-view CLI/MCP and environment serve/export boundaries:
|
||||
5 files, 48 tests passed.
|
||||
- Serial Turbo run: typecheck, lint, format, knip, package metadata and required
|
||||
runtime/contract artifacts: 11 tasks passed. Documentation/catalog validation:
|
||||
8 tasks passed. Architecture guards passed across 1,595 source files.
|
||||
- Real Gitea 28.2.0 official Linux binary (published compressed SHA-256 checked),
|
||||
isolated SQLite database, two users and three public repositories with large
|
||||
issue bodies. Actual Chromium navigation captured nine API transactions,
|
||||
45 events, no drops, 12,584,312 bytes of Evidence. Inline manifest projection
|
||||
was refused with `resource_constraint`; file CLI and default stdio MCP
|
||||
projections agreed on five tools with all template rules selected. The
|
||||
three trailing user sub-resources stayed separate. Malformed file diagnostics
|
||||
named the file's JSON root. HTTP SDK replay verified six complete small
|
||||
responses and twelve exact 256 KiB prefixes of large responses, each marked
|
||||
truncated. Owned Gitea, browser and replay-server processes were stopped.
|
||||
|
||||
One CPU and one test worker were used. Sampled sum of descendant RSS every
|
||||
50 ms (shared pages can count twice): focused tests 15.834 s / 15.327 CPU s /
|
||||
764.69 MiB peak; boundaries 194.916 s / 190.121 CPU s / 1,795.15 MiB;
|
||||
static/build 266.311 s / 259.658 CPU s / 4,677.44 MiB; real Gitea workflow
|
||||
31.631 s / 24.999 CPU s / 2,146.29 MiB. These are individual runs, not a
|
||||
cold/warm comparison; model token usage was not measured. The first external
|
||||
probe stopped at the expected CLI refusal because its driver rejected exit 1;
|
||||
the driver was corrected and the complete workflow passed. No new macOS,
|
||||
Windows, model-task scoring or AgentEnv runner verification is claimed here.
|
||||
@@ -0,0 +1,20 @@
|
||||
# Implementation plan
|
||||
|
||||
1. Add `src/domain/analysisView/scenarioCaptureRecords.ts`: index scenario
|
||||
events by transaction, project each request event as a HAR-shaped
|
||||
`WebNetworkCaptureRecord`, and admit a scenario parent for the manifest
|
||||
view (clone budget, schema, network-content selection).
|
||||
2. Route `projectEnvironmentManifestView` through one capture-parent gate that
|
||||
selects the HAR or scenario admission by parent operation, and widen the
|
||||
environment-manifest result's `parent_operation`.
|
||||
3. Branch `parseCaptureEvidence` in `environmentSource.ts` on the Evidence
|
||||
operation, carry the capture kind and digest on the source, and word the
|
||||
bundle README and CLI help for both kinds.
|
||||
4. Add a schema-valid scenario capture fixture and domain and services tests:
|
||||
the record projection, the manifest from a scenario parent, the refusal,
|
||||
replay of UTF-8 and binary bodies, and the bundle README.
|
||||
5. Document scenario parents in `docs/agent-environments.md` and
|
||||
`docs/web-network-captures.md`, extend the tool contract text, and record
|
||||
the spec.
|
||||
|
||||
No new dependency is introduced and no command is added.
|
||||
@@ -0,0 +1,65 @@
|
||||
# Build environments from browser scenario captures
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
The `environment-manifest` view, `environment-serve` and `environment-export`
|
||||
accept `capture_browser_scenario` Evidence as the manifest's parent, in
|
||||
addition to `inspect_web_network_capture` HAR Evidence. A session REA records
|
||||
itself, in a browser it launches, with credential headers and declared secrets
|
||||
redacted, becomes an agent environment without an external capture script or
|
||||
HAR scrubbing step. The replay, the tool surface and the bundle layout do not
|
||||
change.
|
||||
|
||||
## Requirements
|
||||
|
||||
- A scenario parent is admitted only when its scenario selected
|
||||
`capture.network.response_body` or `header_values`, because tools are chosen
|
||||
by response media type; otherwise the view refuses with
|
||||
`unsupported_target`.
|
||||
- Each network transaction becomes one record in request order; its ordinal
|
||||
is the zero-based position of its `request` event, and its location is the
|
||||
JSON pointer of that event in the capture. The record is shaped like a HAR
|
||||
entry: request method and URL, the request body as `postData` when it was
|
||||
captured and decodes as UTF-8, the response status, and the response media
|
||||
type with its retained bytes as base64 content and a retained binary field.
|
||||
- A transaction whose URL the scenario redacted keeps a null URL, so no path
|
||||
template is formed from a redaction marker. A transaction that does not
|
||||
resolve to exactly one request event, at most one response event and at most
|
||||
one content event per phase keeps no request line. Both are counted as
|
||||
unrepresented records, as HAR records without a usable request line are.
|
||||
- The response media type is the captured body's `media_type`, or the
|
||||
`content-type` header value when only headers were captured.
|
||||
- The existing projection representation budget reserves, before schema body
|
||||
validation or request-body decoding and for retained and file parents alike,
|
||||
the scratch of validating the largest body (two bytes per base64 character)
|
||||
and the decoded text of every request body; response bodies stay borrowed,
|
||||
as HAR response text does, so a large capture read from its file is not
|
||||
charged per character for them. Over-budget captures return
|
||||
`resource_constraint` before those allocations. Replay projection omits the
|
||||
request text used only for manifest body-shape inference; retained response
|
||||
bytes and their integrity checks are unchanged.
|
||||
- The manifest result names `parent_operation: "capture_browser_scenario"`,
|
||||
carries an empty `artifact` (`path: ""`, `sha256: null`) and states, as
|
||||
limitations, the ordinal rule, any dropped capture events, how many
|
||||
transactions with a usable request line retained no response body (tools
|
||||
citing them replay only the recorded status and content type, either of
|
||||
which may be absent) and how many retained response bodies carry redaction
|
||||
markers. A body that redaction emptied, because a declared secret could not
|
||||
be cut out, is treated as not retained rather than replayed as an empty
|
||||
reply.
|
||||
- `environment-serve` and `environment-export` authenticate either parent
|
||||
kind through the same loader, project scenario transactions through the same
|
||||
adapter, and keep the cross-check that the manifest's parent is the supplied
|
||||
Evidence. The bundle README names the capture kind and, for a scenario, the
|
||||
SHA-256 of the scenario definition in place of a capture artifact digest.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No new browser action: scenarios still cover what the UI requests plus GET
|
||||
reads through `goto`. A fetch or script action would be a new effect and
|
||||
needs its own scope decision.
|
||||
- No change to what the scenario redacts. Values the application returns in
|
||||
response bodies are retained, as they are in HAR captures.
|
||||
- The api-surface view keeps HAR parents only.
|
||||
@@ -0,0 +1,10 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the scenario-parent contract, requirements and non-goals.
|
||||
- [x] Project scenario transactions as HAR-shaped records and admit scenario parents.
|
||||
- [x] Route the manifest view through one capture-parent gate and widen its result schema.
|
||||
- [x] Load scenario Evidence in serve and export; name the capture kind in the README and help.
|
||||
- [x] Add the scenario capture fixture and domain and services tests.
|
||||
- [x] Document scenario parents and extend the tool contract text.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
- [x] Build environments from real browser scenario captures and record the results.
|
||||
@@ -0,0 +1,144 @@
|
||||
# Verification
|
||||
|
||||
Verified commit: 3c2a8db2, this increment on top of the environment-export increment (481c9883).
|
||||
|
||||
## Gates
|
||||
|
||||
Run on macOS 26.5.1 with Node 22.23.1 on that tree:
|
||||
|
||||
- `npx tsc --noEmit`: no errors.
|
||||
- `oxfmt --check`: all files formatted. `oxlint` on the changed files: no new findings.
|
||||
`npm run verify:module-boundaries`: 2133 files checked.
|
||||
- `npm run jscpd`: 3.35% duplicated lines, unchanged. `npm run scan:todos`: none.
|
||||
`npm run mcp-catalog:check`: passed. `npx knip`: one unresolved import that predates
|
||||
this change (`#javascript-analysis-worker`).
|
||||
- `npm run test:prepare`, then vitest over `src/domain/analysisView`, `src/application`,
|
||||
`src/contracts`, `src/server/environment`, `tests/conformance` and the analysis-view,
|
||||
environment-serve, environment-export and browser-scenario boundary tests: 1946 tests in
|
||||
157 files, 1941 passed, 5 skipped, 0 failed.
|
||||
|
||||
## Real browser captures
|
||||
|
||||
`npm pack` of this commit (`rea-agents-6.7.0.tgz`) was installed on Ubuntu 24.04 amd64 hosts
|
||||
(8 vCPU, 32 GB) with Chromium 155.0.8059.79 at `/snap/bin/chromium`, Node 24, Docker and
|
||||
`agentenv-framework` 0.9.1334. Every capture below is `rea capture-browser-scenario` output;
|
||||
no HAR was recorded, scoped or edited.
|
||||
|
||||
### Fixture
|
||||
|
||||
A loopback fixture served a page that lists photos from `/api/photos`, three PNG thumbnails, and
|
||||
a form that POSTs JSON to `/api/albums`. The scenario waited for the list, opened
|
||||
`/api/photos/1..3` with `goto`, filled and submitted the form.
|
||||
|
||||
- Capture: 14 transactions, 70 events, 0 dropped, authority `controlled-replay`.
|
||||
- `environment-manifest`: `parent_operation` `capture_browser_scenario`, origin
|
||||
`http://127.0.0.1:3199`, confidence `inferred`. Tools `get_photos` (records 1, 9),
|
||||
`get_photos_by_id` (`{numeric}` id, records 5, 6, 7) and `post_albums` (record 13, body shape
|
||||
`title: string`). The page and the three PNGs were excluded as `ui-page-or-asset`.
|
||||
- `environment-serve`: `get_photos_by_id {id: "3"}` returned the recorded JSON with
|
||||
`record_ordinal` 7 and `exact_match` true.
|
||||
- The POST's response body was missing: the scenario ended at the action that sent it, the
|
||||
transaction was still unfinished, and the observer does not read bodies of unfinished
|
||||
responses. The manifest stated "1 of 14 scenario transactions retained no response body" and
|
||||
`post_albums` replayed status 201 with no body. Ending the scenario with a 1.5 s
|
||||
`wait_for_timeout` finished all 14 transactions and retained the body.
|
||||
- `environment-export` of that capture wrote a README naming the browser scenario capture and the
|
||||
scenario definition's SHA-256; `agent-env run <bundle> --task smoke` passed in 30.4 s.
|
||||
|
||||
### Five applications
|
||||
|
||||
Each environment ran manifest, export, the bundle's smoke task and 6 to 12 generated retrieval
|
||||
tasks, with one run per task for two models through the Hugging Face router.
|
||||
|
||||
| Application | Capture | Actions | Transactions | Tools (rules / none) | Smoke | Retrieval tasks | gpt-oss-120b | GLM-5.3-Flash |
|
||||
| ---------------------------- | ------------------------------------------------------------------ | ------- | ------------ | -------------------- | ------ | --------------- | ------------ | ------------- |
|
||||
| PhotoPrism 261007 | self-hosted, no login (public mode) | 55 | 277 | 13 / 28 | passed | 10 | 10/10 | 10/10 |
|
||||
| Lychee 6.10.4 | self-hosted, session cookie seeded from a declared secret | 152 | 134 | 15 / 27 | passed | 12 | 11/12 | 12/12 |
|
||||
| Pinry 2.1.13 | self-hosted, session and CSRF cookies seeded from declared secrets | 148 | 568 | 12 / 12 | passed | 6 | 6/6 | 6/6 |
|
||||
| Art Institute of Chicago API | public, `goto` per URL | 52 | 26 | 11 / 12 | passed | 12 | 11/12 | 12/12 |
|
||||
| Library of Congress JSON API | public, `goto` per URL | 55 | 29 | 11 / 11 | passed | 12 | 11/12 | 12/12 |
|
||||
|
||||
Every scenario step completed and no capture dropped events. Each self-hosted capture includes
|
||||
one create, one edit and one delete made through the application's own forms; all three became
|
||||
tools with their recorded statuses, including a 204 DELETE without a content type. Declared
|
||||
secrets did not appear in any capture; where the application echoed a declared value in a
|
||||
response body (PhotoPrism's session reply, Pinry's owner token) the capture held the redaction
|
||||
marker instead.
|
||||
|
||||
Two other targets were tried and replaced. The Met Collection API projected 3 tools: its data
|
||||
paths are `/departments`, `/objects/{id}` and a search, and numeric object ids merge into one
|
||||
tool. The NASA Image and Video Library projected 16 tools, but its Collection+JSON bodies nest
|
||||
record fields deeper than the task generator reads, so no tasks could be derived.
|
||||
|
||||
## Observed limits
|
||||
|
||||
These were observed during the runs and are outside this increment:
|
||||
|
||||
- A scenario's last action can lose its response body unless the scenario ends with a wait
|
||||
(above).
|
||||
- Large asset bodies count against the analysis-view budget. One Pinry capture at
|
||||
`max_network_body_bytes` 262144 held 12.5 MB of Evidence, mostly CSS and JavaScript reloaded
|
||||
on 33 page loads, and the manifest view returned `resource_constraint`. A 20,000-byte body
|
||||
limit kept it within budget. Scenarios have no per-origin or per-media-type body selection.
|
||||
- Image responses outside an API path are excluded as UI assets, so their bytes are replayable
|
||||
only when an API path serves them (PhotoPrism's `/api/v1/t/...` tiles became tools).
|
||||
- With the default `max_total_event_allocation_bytes`, one PhotoPrism capture dropped its last
|
||||
68 transactions, including all three writes, while the command exited 0; 64 MiB dropped none.
|
||||
- Actions have no default deadline: two runs waited on a locator for 277 s and 7.5 min until
|
||||
cancelled. Every scenario above sets `timeout_ms` on its actions.
|
||||
- A redirect is its own transaction; Django's trailing-slash 301 produced a second tool
|
||||
(`get_pins_by_id` and `get_pins_by_id_2`) for one resource.
|
||||
|
||||
## Linux followup verification (2026-10-19)
|
||||
|
||||
The original PR was merged with current main `7a0c47bfa`, after #4955, #4975
|
||||
and #4999 merged. The export filesystem ownership, mode, tarball, cleanup and
|
||||
caller-path fixes from main are retained. Default MCP remains stdio.
|
||||
|
||||
The retained-parent projection previously decoded every request body before
|
||||
per-record budget checks. It now reserves copied/transformed text through the
|
||||
shared admission owner before schema validation or decoding, including retained
|
||||
parents. Replay projection omits request text used only for body-shape inference;
|
||||
response bytes still undergo the producer schema's integrity validation. The
|
||||
intermediate projected-array copy and unused capture wrapper were removed.
|
||||
Windows test inventories normalize filesystem separators to the portable bundle
|
||||
path representation. CI selects the agentenv lane for the new projection and its
|
||||
shared admission dependencies.
|
||||
|
||||
Executed on Linux x64, Node 24.18.0 / npm 11.16.0, Chrome for Testing
|
||||
156.0.8078.4:
|
||||
|
||||
- `turbo run typecheck lint:built format:check docs:check:built knip artifacts:skills artifacts:mcp-catalog --concurrency=1 --summarize`: eleven tasks passed, one cached, 210.023 s elapsed / 209.860 s CPU / 4562.07 MiB peak process-family RSS. This first complete gate preceded the final lint cleanup and CI-selection additions; later checks are reported on the PR.
|
||||
- Initial source verification passed 111 tests in 13 files, including HAR/manifest regressions, scenario loading/replay/export and new capacity/replay-mode tests. CI-selection verification passed 235 cases.
|
||||
- Real public CLI capture of the source-owned interactive browser network site: ten transactions, fifty events, two tools, no dropped events. Its POST response replayed twice through the official HTTP SDK with exact body/status/Evidence ordinal and untrusted-content provenance. Declared secrets and transport credentials were excluded; the ordinary selected `password` field value remained in retained body bytes.
|
||||
- Real public CLI capture of three live GitHub API responses in Chromium, without a HAR: three transactions, fifteen events, three tools, no dropped events. Every response replayed twice through the HTTP SDK with exact body/status/Evidence/provenance. Explicit navigation waits let response-body capture settle. Complete two-scenario capture/projection/serve task: **22.527 s / 16.678 s CPU / 1397.93 MiB** peak process-family RSS, including owned browser processes. Both listeners exited 143 and newly owned browser profiles were gone.
|
||||
- The same two actual captures produced identical manifest results through CLI and official SDK **stdio** MCP, with HTTP environment variables present: **2.795 s / 2.778 s CPU / 476.67 MiB**. This verifies adapter parity and default transport independently of explicit environment HTTP.
|
||||
|
||||
The first local probe mistakenly searched base64 body content for plaintext; its
|
||||
assertion was corrected to inspect decoded retained bytes. A later SDK probe
|
||||
initially imported stdio transport from the SDK root instead of its documented
|
||||
subpath; the corrected probe passed. An early live navigation also reported one
|
||||
`body-read-failed` response, which the projection preserved as unavailable. No
|
||||
browser-provider behavior was changed; the successful run used explicit waits.
|
||||
These earlier attempts are not counted as passing runs.
|
||||
|
||||
A matched retained-scenario projection comparison bundles the original helper
|
||||
from `3aae3579e` and the followup helper, supplies the same 72,771-byte capture
|
||||
with three JSON writes, and completes one cold plus 100 warm full manifest
|
||||
projections. Both outputs have the same SHA-256
|
||||
`990ac431670374ff68c77a20d77985f626e4681bb0e2f491e5af18f45ce79991`.
|
||||
Before/after process totals: **0.970/0.876 s elapsed, 0.934/0.865 s CPU,
|
||||
184.89/184.39 MiB** RSS. First task: **17.378/13.073 ms**; 100 warm tasks:
|
||||
**261.419/236.131 ms**. Single samples do not establish a speed or memory
|
||||
improvement. Bundle preparation is excluded; process totals include fixture
|
||||
startup. CLI, provider and transport costs are excluded from this comparison.
|
||||
|
||||
Heavy work is serialized on one CPU with one test worker. RSS sums sampled
|
||||
process descendants every 50 ms and can double-count shared pages. Agent token
|
||||
usage is unmeasured. These followups do not rerun the author's five application
|
||||
reconstruction/model tasks, nor establish Windows or macOS behavior. Historical
|
||||
response replay does not reconstruct application state.
|
||||
|
||||
A retained-parent counterexample used a 128 KiB projection budget and one 16 KiB JSON request: the original helper admitted it and decoded once; the repaired helper returned `AnalysisResourceConstraintError` with zero UTF-8 decodes. The test intentionally uses a small budget without inducing an OOM. Final targeted verification passed 242 tests (including 235 CI-selection cases): 41.042 s / 40.666 s CPU / 644.37 MiB peak process-family RSS.
|
||||
|
||||
The broader d2416ca54 regression run passed 657 tests in 47 files plus ten product-catalog cases (295.763 s / 293.370 s CPU / 1921.69 MiB family RSS; catalog checked separately). The author then pushed 655e62334 with redaction/body-budget documentation and withheld-empty-body behavior. That commit was retained; the followup counts bodiless/redacted observations against represented transactions, preserving the complete request ordinals. The related source run passed 145 tests in 14 files (9.793 s / 9.618 s CPU / 688.66 MiB). Docker execution was held at this author-update checkpoint, so no earlier bundle is attributed to the new source. Final committed checks and actual scenario-container/framework results are reported on the PR.
|
||||
@@ -0,0 +1,28 @@
|
||||
# Implementation plan
|
||||
|
||||
1. Move response-body decoding from the environment source into
|
||||
`recordedResponseBody.ts` with the caller's byte limit, and exact JSON
|
||||
parsing into `exactJsonNumber.ts`, so replay and state extraction share one
|
||||
owner each.
|
||||
2. Add `environmentStateRules.ts` as pure helpers: write classification, API
|
||||
bases and roots, entity location, identity names, JSON pointers of depth one,
|
||||
value bytes and the entity and list renderers.
|
||||
3. Add `EnvironmentStateExtraction.ts` for planning item and list tools,
|
||||
observing entities and assembling the result, `environmentStateMerge.ts` for
|
||||
merging observations, and `environmentStateGate.ts` for reproducing each
|
||||
plan's recordings; every body is parsed through a budget callback and
|
||||
released before the next.
|
||||
4. Add `EnvironmentStateAccumulator.ts`, extending the manifest accumulator so
|
||||
grouping is shared, retaining record lines and references to the admitted
|
||||
records, and charging bodies and kept values.
|
||||
5. Add `environmentStateView.ts` with the request schema (sharing the
|
||||
manifest's `template_rules` schema), the result fields and the projector;
|
||||
share capture admission and record grouping with the manifest view; register
|
||||
the kind and pass accumulators to `projectAnalysisView` as one object;
|
||||
always seal the result inferred.
|
||||
6. Cover domain, service, MCP, CLI, catalog and acceptance behavior; document
|
||||
the view in the capture guide, the agent environments guide, the contracts
|
||||
reference and the authored skill.
|
||||
|
||||
No new tool, provider, side effect or network activity is introduced. The state
|
||||
is a projection of already retained Evidence; serving it is deferred.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Environment state view from captures
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
`inspect_analysis_view` accepts `view: {"kind": "environment-state"}` for the
|
||||
parents the environment manifest accepts and extracts the initial state the
|
||||
capture's reads observed: entity types with an identity field, entities with
|
||||
their recorded values, documents for written paths whose reads are not
|
||||
entities, and a render plan per read tool that reproduces the tool's latest
|
||||
initial recordings from the state, or a recorded view with the reason it
|
||||
cannot. This increment adds the view kind only; serving from the state, loading
|
||||
another state through the AgentEnv data plane and modelling write effects are
|
||||
later increments.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Accept `inspect_web_network_capture` HAR parents and
|
||||
`capture_browser_scenario` parents that selected response bodies, with the
|
||||
manifest's admission: the same format, subject and sensitive-value refusals.
|
||||
- Group records exactly as the manifest does for the same `template_rules`, so
|
||||
tool names match the manifest the state accompanies.
|
||||
- Order primary-origin records by recorded start time, or by record order when
|
||||
any start time is missing, and say which. Bound each type's state by the first
|
||||
write under its root (API base, version segments, one segment); a query
|
||||
`POST` is not a write, a read overlapping the write is left out, and a write
|
||||
whose URL was redacted bounds every type, with a limitation counting it.
|
||||
- Prove an item type's identity field from the requested path value its
|
||||
responses repeat; otherwise infer it from a fixed list of identity names that
|
||||
every listed item carries distinctly, which needs at least one listed item.
|
||||
Never treat an envelope that is itself an entity as a list.
|
||||
- Merge observations into one entity per collection and identity: item views'
|
||||
latest values win over list views', differing views keep their own values,
|
||||
and a value that changed between repeats of one request line with no recorded
|
||||
write is volatile and keeps the latest; a change across a write under another
|
||||
root, including between two tools of the type, leaves every tool of the type
|
||||
recorded. Compare list membership, including empty lists, and field presence before selecting each request line's latest recording; a change across a write also leaves the type recorded. Different query strings remain distinct; omitted fields do not erase their earlier observations on other request lines.
|
||||
- A list has `all` membership only when no item-total envelope member (`total`,
|
||||
`count` and their compounds, not page counts) differs from the number of
|
||||
entities it lists and no `has_more`, `next` or next-page token is set.
|
||||
- Keep an item or list plan only when it reproduces each request line's latest
|
||||
initial recording from the state, comparing JSON values without regard to
|
||||
member order; otherwise record the tool with a reason. Parse bodies only when
|
||||
JSON-like, UTF-8, at most 256 KiB, exactly representable and free of
|
||||
`__proto__` members.
|
||||
- Mark the Evidence inferred; state unknowns for authorization, response
|
||||
semantics and write effects.
|
||||
- Charge the manifest's admission and grouping, every record line, every body
|
||||
as temporary bytes before decoding and parsing, and every kept value, within
|
||||
the existing 96 MiB budget.
|
||||
- Read response bodies through one domain owner shared with replay, and parse
|
||||
exact JSON through the owner of the exact-number policy.
|
||||
|
||||
## Acceptance and limits
|
||||
|
||||
The Grafana-shaped HAR fixture with distinct start times drives domain,
|
||||
service, MCP, CLI and acceptance coverage: folders merge their detail and list
|
||||
views under a proved `uid`, the dashboard entity is read from its `/dashboard`
|
||||
wrapper, and the annotation list after the annotation write has no initial
|
||||
recording. Synthetic captures cover write bounds, volatile fields, complete
|
||||
lists with counters, each recorded reason and documents; the scenario fixture
|
||||
covers record order and the response-body requirement. The view performs no
|
||||
network activity and serves nothing. Values copied outside entity objects,
|
||||
derived values and cross-type effects are not tied to the state.
|
||||
|
||||
## Deferred issue criteria
|
||||
|
||||
Serving tool calls from the state, `data/reset`, `data/add` and `data/get` on
|
||||
the environment data plane, exporting the state with a bundle, and write
|
||||
effects are later increments of #4933. So are documents for item-shaped
|
||||
written paths (one per identity rather than per template) and for paths that
|
||||
are only deleted; until then their reads stay recorded. This PR must reference rather than
|
||||
close the issue.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the view contract, requirements and deferred increments.
|
||||
- [x] Move body decoding and exact JSON parsing to their domain owners without
|
||||
changing replay.
|
||||
- [x] Implement the extraction rules, merge, gate and accumulator.
|
||||
- [x] Register the `environment-state` kind, result schema, service wiring and
|
||||
confidence rule.
|
||||
- [x] Add the timed Grafana-shaped fixture and synthetic captures.
|
||||
- [x] Add domain, service, MCP, CLI, catalog and acceptance coverage.
|
||||
- [x] Document the view in the guides, contracts reference and skill.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
@@ -0,0 +1,156 @@
|
||||
# Verification
|
||||
|
||||
Verified base: bf1eeee9 (fetched main).
|
||||
|
||||
Executed on the implementation worktree after `npm ci` and `npm run
|
||||
build:cached`; every check below ran as written in this pass, after the last
|
||||
source change.
|
||||
|
||||
- `npm run check:fast`: typecheck and lint passed.
|
||||
- `npm run format:check`: passed.
|
||||
- `npx knip`: passed (exit 0).
|
||||
- `npm run jscpd`: 3.40% duplicated lines over 3331 files, under the 3.5%
|
||||
gate; no clone involves a file this change adds.
|
||||
- `npm run scan:todos`: passed.
|
||||
- `npm run docs:check`: the built documentation checks and the README
|
||||
consistency conformance test (5 tests) passed.
|
||||
- `npx vitest run --project domain --project services`: 338 files, 5830 tests
|
||||
passed, 5 pre-existing skips.
|
||||
- `npm run test:focused --` the analysis-view MCP and CLI boundaries, the
|
||||
environment serve and export CLI boundaries, the environment serve MCP
|
||||
boundary, the compact input-schema MCP budget, the analysis-detail and
|
||||
advertised-input-example contracts and the qlog acceptance test: 9 files, 68
|
||||
tests passed, 1 pre-existing skip.
|
||||
|
||||
Real captures (outside the test suite): the built view ran through the
|
||||
application service on 44 recorded captures (39 HAR, 5 browser scenario) with
|
||||
all three template rules, in 7 to 46 ms each. 42 produced a state: 568 types,
|
||||
2,213 entities, 2 documents, and 710 of 1,258 read tools state-backed (211
|
||||
item, 497 list, 2 document views). The other read tools are recorded views:
|
||||
306 not entities (singletons, configuration and computed views), 129 without
|
||||
an identity field, 32 with unusable bodies (most are Nomad bodies with
|
||||
integers beyond exact native conversion), 31 lists and 7 items that do not
|
||||
reproduce, 26 without an initial recording, 12 query `POST`s and 5 documents
|
||||
read under several queries. The two browser-scenario captures that did not
|
||||
produce a state are refused inline with `resource_constraint`, as the
|
||||
environment manifest view refuses them on unchanged main.
|
||||
|
||||
Not covered here: serving tool calls from the state, the AgentEnv data plane
|
||||
and write effects, which are later increments.
|
||||
|
||||
## Linux follow-up (2026-10-20)
|
||||
|
||||
Node 24.18.0 / npm 11.16.0, with the author's `47f8a27a9` main integration
|
||||
preserved and main `d72021b0d` integrated. Eight regression cases failed across
|
||||
two pre-fix runs: membership additions/removals across a write, field presence,
|
||||
sparse field observations, and compact compound totals. The fixes compare
|
||||
membership before selecting latest recordings, retain each field's observation
|
||||
across sparse reads, and recognize `totalcount`/`itemcount`. API-base grouping
|
||||
appends to its owned arrays; identity intersection uses a set and skips the
|
||||
first response's self-intersection.
|
||||
|
||||
Executed checks:
|
||||
|
||||
- Relevant domain, application and contract suites: 39 files, 493 passed,
|
||||
3 existing skips. Includes empty/stable lists, query variants, field additions
|
||||
and removals, sparse reads, identity proof and deferred document paths.
|
||||
- Broader compiled CLI/MCP/environment, compact schema and qlog acceptance:
|
||||
7 files, 65 passed, 1 existing skip. After the final state fixes, both selected
|
||||
compiled state CLI/MCP cases passed again (36 other cases deselected).
|
||||
- Final serial static/build/artifact gate: 11 tasks passed. Typecheck, lint,
|
||||
format, knip, package metadata and runtime/catalog generation all ran.
|
||||
- Final documentation/catalog/format validation: 9 tasks passed; architecture
|
||||
guards passed across 1,602 source files; README consistency: 5 tests passed.
|
||||
|
||||
Real Gitea 28.2.0 used an isolated SQLite database. An owned Node fetch HAR
|
||||
recorder captured actual API exchanges, omitting transport credentials:
|
||||
|
||||
- Three exchanges: empty repository search, creation through `/user/repos`,
|
||||
then the same search. On exactly the same authenticated Evidence, pre-fix
|
||||
`29e745923` published one created repository as initial state; the final
|
||||
source publishes none and keeps the search recorded with `fields-differ`.
|
||||
- Eleven exchanges: initial issue list/details and topics, PATCH, PUT and
|
||||
DELETE, then subsequent reads. The state keeps the two original issues
|
||||
before ordinal 4 and the original topics document at ordinal 3. File CLI and
|
||||
default stdio MCP agree; six HTTP SDK replays reproduce original responses.
|
||||
- Actual Chromium `capture_browser_scenario`: two transactions, eight events,
|
||||
zero dropped. The state uses record order, extracts one issue, and keeps list
|
||||
and item plans; file CLI and default stdio MCP agree. The earlier 12.6 MB
|
||||
real browser capture also projects, keeping oversized bodies recorded with
|
||||
`body-unusable` rather than parsing them into state. Owned servers and browser
|
||||
processes were stopped.
|
||||
|
||||
### Resource and performance measurements
|
||||
|
||||
One CPU and one worker. Process-family RSS sums were sampled every 50 ms and
|
||||
may count shared pages twice. Final source suites: 39.669 s wall / 39.486 CPU s /
|
||||
898.42 MiB peak; broader boundaries: 270.171 s / 267.136 CPU s / 2,130.78 MiB;
|
||||
final static/build: 199.683 s / 199.523 CPU s / 4,689.07 MiB; final live HAR
|
||||
workflow: 22.836 s / 19.880 CPU s / 1,072.43 MiB; actual browser-state workflow:
|
||||
12.734 s / 10.941 CPU s / 1,549.80 MiB. The qlog acceptance lane intentionally
|
||||
exhausts an isolated decoder heap and verifies that MCP remains usable; no
|
||||
workspace OOM occurred.
|
||||
|
||||
The first follow-up benchmark used pre-fix `29e745923` and `6ad63bcc2`, the same
|
||||
completed Evidence files, Node and CPU affinity. One fresh process per revision,
|
||||
with a first call and two warm repeats; filesystem caches were not cleared.
|
||||
Complete normalized output digests matched for both synthetic projection tasks.
|
||||
|
||||
| Work | Before: first / warm (ms) | After: first / warm (ms) |
|
||||
| ----------------------------------------------- | ------------------------- | ------------------------ |
|
||||
| Group 8,000 templates (helper only) | 320.08 / 81.01–81.46 | 11.89 / 3.88–4.33 |
|
||||
| File-to-state projection, 2,000 collections | 1,321.57 / 732.00–939.57 | 1,268.76 / 781.34–811.05 |
|
||||
| File-to-state projection, one 15,001-field item | 657.05 / 494.82–501.19 | 377.78 / 347.62–380.87 |
|
||||
|
||||
Combined benchmark processes, including imports, three helper calls, six
|
||||
projections and the live-capture comparison: before 6.536 s wall / 6.496 CPU s /
|
||||
370.82 MiB peak; after 5.077 s / 5.056 CPU s / 386.53 MiB. Grouping and wide
|
||||
identity inference improved; the 2,000-collection projection has overlapping
|
||||
warm timings, and peak memory did not decrease. These few runs do not establish
|
||||
a general speedup. Model token usage was not measured; no model-task scoring
|
||||
was run. Initial external probe attempts had incorrect CLI positional arguments
|
||||
and assumed coverage counted records rather than grouped tools; those driver
|
||||
errors were corrected before the successful complete runs.
|
||||
|
||||
No new macOS/Windows runtime, state serving, AgentEnv data-plane or write-effects
|
||||
claim is made. A type first observed only after an unrelated-root write may
|
||||
still lack the earlier observation needed to detect its effect. Item-shaped
|
||||
and delete-only document paths remain recorded, as specified.
|
||||
|
||||
## Post-merge write-cutoff follow-up
|
||||
|
||||
Base `3fca2e0ca` includes #5319 and the validated immutable-capture reuse from
|
||||
#5349. The remaining review finding was repeated write-root parsing for each
|
||||
read type and again during output assembly. One admitted prefix/wildcard index
|
||||
now owns parsed write roots, with per-projection positive/negative cutoff
|
||||
caching. Backed type membership is collected once, replacing another plan scan.
|
||||
The old linear matcher is removed. Prefix, parameter, redacted-root and timeline
|
||||
semantics are preserved; no cross-request cache or new authority is introduced.
|
||||
|
||||
- Source checks: 41 files, 508 passed, 3 existing skips, including independent
|
||||
matching-oracle and 12,000-segment iterative-lookup checks.
|
||||
- Serial static/build/artifact gate: 11 tasks passed, 197.356 s wall /
|
||||
197.195 CPU s / 4,744.92 MiB peak process-family RSS.
|
||||
- Both selected compiled state CLI/MCP cases passed after rebuilding.
|
||||
- Live Gitea HAR workflows and fresh actual Chromium capture passed again:
|
||||
initial values, recorded fallbacks, CLI/default stdio parity and six exact
|
||||
HTTP replays. HAR workflow: 22.834 s wall / 19.623 CPU s / 1,067.36 MiB;
|
||||
browser workflow: 22.858 s / 20.912 CPU s / 1,546.59 MiB. Owned processes stopped.
|
||||
|
||||
The same completed Evidence with 1,000 collection reads and 1,000 unrelated
|
||||
writes was projected before and after on one CPU. Two larger full-HAR inputs
|
||||
(3,000/4,000 total records) were refused by the existing 96 MiB admission budget;
|
||||
the limit was retained. One fresh process per revision, first call plus one
|
||||
warm repeat, filesystem caches not cleared:
|
||||
|
||||
| Complete file-to-state projection | Main before | Follow-up |
|
||||
| -------------------------------------------------------- | ------------------- | ------------------- |
|
||||
| First call wall / CPU (ms) | 1,727.89 / 1,727.36 | 1,244.82 / 1,244.05 |
|
||||
| Warm call wall / CPU (ms) | 1,300.84 / 1,300.42 | 624.01 / 623.89 |
|
||||
| Process wall / CPU (s), including imports and both calls | 4.690 / 4.363 | 3.370 / 3.062 |
|
||||
| Sampled process-family peak RSS (MiB) | 365.84 | 360.03 |
|
||||
|
||||
Parent Evidence IDs and complete normalized output digests were identical.
|
||||
RSS sampling is every 50 ms, includes descendants and can double-count shared
|
||||
pages; these few runs do not establish a general memory reduction. Model token
|
||||
usage remains unmeasured. No workspace OOM occurred.
|
||||
+98
-11
@@ -92,7 +92,7 @@ without serial scheduling. Pure domain/service, composition, conformance,
|
||||
evaluation, CLI boundary, and MCP boundary projects share module graphs. Their
|
||||
tests avoid process-global state and keep resource cleanup test-scoped. Each
|
||||
MCP session still owns its resources, and CLI cases retain fresh command
|
||||
instances or subprocesses. CLI module mocks belong in `tests/process-global/`
|
||||
instances or subprocesses. CLI and MCP module mocks belong in `tests/process-global/`
|
||||
so they keep per-file isolation. Product catalog verification stays with the
|
||||
isolated filesystem boundary tests because it loads both source and compiled
|
||||
module graphs. Adapter, other boundary, acceptance, and process-global projects
|
||||
@@ -110,9 +110,14 @@ rendered declarations. Keep canonical rejection checks separate from the
|
||||
intentionally reduced compact advertisement, and probe actual client renderers
|
||||
when changing schema representations.
|
||||
|
||||
The reused CLI boundary workers cap V8 old space at 384 MiB, based on their
|
||||
profiled allocation and cleanup behavior. This encourages collection between
|
||||
files; spawned REA processes retain their own heap settings.
|
||||
The reused CLI boundary workers cap V8 old space at 384 MiB for ordinary runs
|
||||
and 512 MiB with coverage enabled. Covered ADB fixtures exhaust the ordinary
|
||||
384 MiB budget; those fixtures also complete with the shared CLI graph preloaded
|
||||
at 512 MiB. This is a measured capacity control, not a suite-wide speedup claim.
|
||||
Both profiles keep the
|
||||
same bounded worker count, and spawned REA processes retain their own heap
|
||||
settings. Explicitly disabling coverage retains the ordinary profile; coverage
|
||||
provider options alone do not enable instrumentation.
|
||||
|
||||
Vitest worker limits do not bound subprocesses launched within a test. Bound
|
||||
those batches separately so repeated CLI validation cannot oversubscribe the
|
||||
@@ -246,6 +251,15 @@ lineage followed by a confirmed passive family census. Windows controls require
|
||||
a fresh lineage observation and do not signal cached descendant PIDs after root
|
||||
termination. Simulated Windows controls do not establish native Windows behavior.
|
||||
|
||||
Native process-ownership controls distinguish start identity, zombie lifecycle,
|
||||
current parent/group relation, and token-read evidence. Retain generation-bound
|
||||
candidates through smaller lineage observations, table omissions, reparenting,
|
||||
group changes, and failed cleanup retries. A readable token with unavailable
|
||||
identity stays unresolved. Native UID facts, rather than an earlier process-table
|
||||
row, determine whether a token denial concerns another user. Darwin tests compile
|
||||
the helper and exercise snapshot growth, absence, lifecycle, and UID observations;
|
||||
controlled Linux procfs tests do not establish real Linux exit or PID-reuse behavior.
|
||||
|
||||
For atomic writes, exercise cancellation on both sides of the documented commit
|
||||
point and preserve any publication receipt. A late interruption does not prove
|
||||
that no output was published.
|
||||
@@ -350,6 +364,11 @@ The process-global Vitest configuration contract rejects new direct temporary-ro
|
||||
creation outside the workspace seam and its narrowly documented boundary/package
|
||||
exceptions.
|
||||
|
||||
The shared temporary-directory and workspace helpers preserve caller labels
|
||||
under an owned `rea-` prefix when needed. `npm run verify:test-temp-hygiene`
|
||||
uses the same ownership rule to find retained fixtures under a private temporary
|
||||
root. Third-party caches remain outside that ownership rule.
|
||||
|
||||
Real Hopper, Ghidra, IDA, browser, package, and managed-code claims belong to their
|
||||
explicit `npm run verify:*` lanes. The reconstruction-readiness lane also
|
||||
checks deterministic rerun, tamper, and stale-input handling; those checks do
|
||||
@@ -741,11 +760,12 @@ See [Android analysis](android-analysis.md) for boundaries and resource budgets.
|
||||
|
||||
`verify:jeb` requires a caller-started JEB client serving MCP at
|
||||
`REA_JEB_MCP_URL` (default `http://127.0.0.1:8425/mcp`) with a project
|
||||
already open, and verifies real CLI client inspection, unit listing coverage,
|
||||
already open, and exercises shared application/provider client inspection, unit listing coverage,
|
||||
and method decompilation. The script records the engine's exact response to
|
||||
`open_jeb_project`; JEB 5.48.0 headless instances do not advertise that tool.
|
||||
`open_jeb_project`; the earlier licensed 5.48.0 headless verification did not
|
||||
advertise that tool.
|
||||
REA does not install or launch JEB; see [JEB analysis](jeb-analysis.md) for
|
||||
the bring-your-own boundary. Verified against JEB 5.48.0 serving
|
||||
the bring-your-own boundary. The earlier licensed build was verified against JEB 5.48.0 serving
|
||||
`jeb-mcp-server` 1.3.0 at four scales: a compiled Java class fixture, a
|
||||
locally built signed probe APK with a launcher activity (manifest, v1/v2/v3
|
||||
certificates, dex bytecode, filtered and paginated listing), the published
|
||||
@@ -756,7 +776,12 @@ decompilation through both CLI and MCP, and the official Flutter Gallery
|
||||
analyzed as native code (75k methods on arm64) and decompiled through both
|
||||
CLI and MCP with explicit unit selection, plus the thin Java wrapper and
|
||||
decoded manifest. The dedicated Dart snapshot processor and `run_script`
|
||||
remain GUI-only surfaces in JEB 5.48.0 headless.
|
||||
remained GUI-only surfaces in that headless build.
|
||||
The separate [Linux CE native audit](../experiments/analysis-efficiency/jeb-linux-native.md)
|
||||
uses the official CE `5.48.0.202610071539` direct CLI launcher and real
|
||||
CLI/stdio MCP ELF workflows. That audit does not qualify CE Java/Android
|
||||
decompilation or GUI project opening, and it does not replace the licensed
|
||||
verification above.
|
||||
Authenticated IPA and macOS application inventory projection is documented in
|
||||
[Apple application analysis](apple-application-analysis.md).
|
||||
|
||||
@@ -1152,6 +1177,16 @@ identifiable; unknown or shared script changes select everything. Android
|
||||
build-script edits retain the Linux suite and workflow validation; real Android
|
||||
verification stays manual. Renames account for both paths.
|
||||
|
||||
The `agentenv` scope owns the environment application and server modules, the
|
||||
`environment-serve` and `environment-export` command registration, the shared
|
||||
streamable HTTP listener, `scripts/verify-agentenv-environment.mjs` and its
|
||||
workflow `real-agentenv.yml`. The lane exports a bundle from the source-owned
|
||||
Grafana-shaped capture through the built CLI, packs the checkout into the
|
||||
bundle's image and runs `agent-env run` on the hosted runner's Docker with the
|
||||
pinned `agentenv-framework` release; it passes when the smoke task passes and
|
||||
no container remains. It claims nothing about any hosted platform or another
|
||||
framework version.
|
||||
|
||||
This narrower selection trades PR-time coverage for runner capacity. A change to
|
||||
unowned shared runtime code that fails only on arm64, macOS, Windows, the Apple
|
||||
artifact lane, or a provider lane that does not own the file is not caught on
|
||||
@@ -1300,9 +1335,9 @@ Portable owner-based test selection remains observational. Its reports compare
|
||||
proposed omissions with the full run; incomplete or skipped observations do not
|
||||
establish that fewer tests would preserve regression detection.
|
||||
|
||||
The eight automatic browser, binary-layout, EVM, JavaScript-recovery,
|
||||
recorded-crash, and web provider lanes download `rea-build-cache` from their own
|
||||
run after the build and static jobs succeed. Their existing build commands still
|
||||
The nine automatic browser, binary-layout, EVM, JavaScript-recovery,
|
||||
recorded-crash, web and environment-bundle provider lanes download
|
||||
`rea-build-cache` from their own run after the build and static jobs succeed. Their existing build commands still
|
||||
validate task hashes and rebuild mismatched inputs; provider observations always
|
||||
run. Provider-workflow-only edits also admit the build without enabling source
|
||||
test shards. Direct provider dispatches retain the cross-run restore-only cache.
|
||||
@@ -1529,6 +1564,58 @@ This is fixture evidence, not a sandbox claim. Inputs remain unchanged and the
|
||||
sentinel must stay alive; owned cleanup and empty verifier descendants are required. Pass an
|
||||
installed package entrypoint as the script's first argument for package coverage.
|
||||
|
||||
### Exported environment bundles
|
||||
|
||||
`npm run verify:agentenv:environment` is the `real-agentenv` lane. It requires
|
||||
the host's Docker CLI named by absolute `REA_DOCKER_COMMAND`, Python 3.12 with
|
||||
`agentenv-framework==0.9.1334` installed in a virtual environment whose
|
||||
`agent-env` binary is named by absolute `REA_AGENT_ENV_COMMAND`, and outbound
|
||||
network for `pip` on the runner and for `npm` inside the image build. Both
|
||||
variables are required; neither upstream executable is resolved from `PATH`,
|
||||
and the verifier installs nothing. Packing uses the shared npm helper, honoring
|
||||
`npm_execpath` from the caller's selected npm. Direct Node invocations without
|
||||
that value use the helper's platform fallback, so configure the pinned toolchain
|
||||
on `PATH` for those runs. It checks the supplied runner's `--version` against the pin before
|
||||
using Docker and retains the observed version in both passing and failing receipts.
|
||||
Its disposable CI runner creates the virtual environment in runner
|
||||
storage, exports both paths and uses the runner's own Docker; no developer host
|
||||
configuration changes. `agent-env run` attempts share one 22-minute budget
|
||||
inside the 30-minute job (the first attempt at most 15 minutes), so a hung or
|
||||
retried run ends with the verifier's failed receipt rather than the runner's
|
||||
kill. The framework builds its gateway and service-db
|
||||
images from `public.ecr.aws` on every fresh runner, and hosted runners share
|
||||
that registry's anonymous pull quota, so the lane first pulls the base images
|
||||
the pinned framework's Dockerfiles name from `mirror.gcr.io`, which serves the
|
||||
same Docker official images under the same digests, and names them as those
|
||||
Dockerfiles do (on Docker's containerd image store, so digest-pinned names
|
||||
resolve), as the framework's own CI does. The quota handling below covers
|
||||
anything still pulled from that registry. The receipt's `agent_env_run` records the
|
||||
last run whether it passed or failed, with `public_ecr_quota` set to
|
||||
`data_limit` or `rate` when that registry's `429 Too Many Requests` ended it.
|
||||
The data-transfer cap (`toomanyrequests: Data limit exceeded`) is counted per
|
||||
source IP, so it fails the lane at once: a pause on the same runner cannot lift
|
||||
it. When a rate refusal ends the run and at least five minutes of the budget
|
||||
would remain after the pause, the verifier waits 90 seconds and runs once more,
|
||||
recording `attempts: 2` and `retried_for` in the receipt. Any other failure
|
||||
ends the lane at once.
|
||||
|
||||
The verifier writes the Grafana-shaped HAR the view tests own, inspects it and
|
||||
projects the `environment-manifest` view with all three template rules through
|
||||
the built CLI, packs the checkout with `npm pack`, exports the bundle with
|
||||
`--package-tarball` so the image installs this checkout rather than the
|
||||
registry package, and runs `agent-env run` on the bundle. It checks that the
|
||||
export reports the planned file set with the inputs copied byte-exact, that
|
||||
the runner exits `0` with `tasks/smoke.json` reported as passed and the
|
||||
verifier's description naming the tool count, and that no container exists
|
||||
after the run that did not exist before; the container inventory runs even when
|
||||
the run fails or times out, so a failed receipt still lists what it left. The one exception is the framework's
|
||||
own image-store helper, `agentenv-registry` (or `agentenv-registry-<port>`),
|
||||
a `registry:2` container the local provider starts on first use and keeps
|
||||
running across runs; a first run on a fresh host creates it, so the receipt
|
||||
lists it under `image_store_helpers` instead of counting it as a leak. Images
|
||||
built by the run remain and are counted in the receipt. Pass an installed
|
||||
package entrypoint as the script's first argument for package coverage.
|
||||
|
||||
## Agent evaluation and conformance records
|
||||
|
||||
`npm run verify:agent` evaluates investigation scenarios through a real local
|
||||
|
||||
@@ -17,7 +17,7 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
|
||||
13,src/domain/errors.ts:EvidenceIntegrityError,Evidence validation fails,Raw constructor message,May expose validation internals and lacks recovery,"Evidence is invalid or has changed. Recreate or re-import it, then try again.",implemented,focused assertions passed; full `npm run check` passed
|
||||
14,src/domain/errors.ts:UnknownRegistryError,Evidence-state mutation fails,"Residual unknown registry mutation failed: {reason}",Exposes internal subsystem and state token,"Missing IDs identify the session and retry; concurrent revision conflicts refresh current state.",implemented,focused assertions passed; full `npm run check` passed
|
||||
15,src/domain/errors.ts:ProcessCaptureError,Process capture fails,Raw constructor message,May expose process and cleanup internals,"Process capture could not complete. Check command access and limits, then try again.",implemented,focused assertions passed; full `npm run check` passed
|
||||
16,src/domain/errors.ts:AnalysisTimeoutError,Analysis deadline expires,"Analysis operation timed out after {timeoutMs}ms: {operation}",Exposes internal deadline and operation; weak recovery,"Analysis took too long. Try a smaller request, then run `rea doctor` if it continues.",implemented,focused assertions passed; full `npm run check` passed
|
||||
16,src/domain/errors.ts:AnalysisTimeoutError,Analysis deadline expires,"REA's {operation} operation reached its {timeoutMs}ms server/provider deadline; the MCP client may have ended the call earlier",Explains the two deadline owners and avoids blind retry,"Inspect session/provider state first. If the MCP client ended the call, raise its tool deadline above {timeoutMs}ms; otherwise retry with a smaller request and do not repeat an unchanged call.",implemented,verified by projection tests and real MCP/Codex CLI timeout-recovery probes
|
||||
17,src/domain/errors.ts:HopperTimeoutError,Provider deadline expires,"Hopper did not respond within {timeoutMs}ms",Exposes provider and internal deadline,"Analysis took too long. Try a smaller request, then run `rea doctor` if it continues.",implemented,focused assertions passed; full `npm run check` passed
|
||||
18,src/domain/errors.ts:ProviderSelectionError,Deep-provider selection fails,"Analysis provider selection failed ({reason}) for {requestedProviderId}; candidates: {candidateIds}",Retains non-secret local selection context that needs a safe structured projection,Reason-specific guidance with stable candidate IDs and actionable rejection diagnostics,implemented,focused registry/CLI/MCP assertions passed; full `npm run check` passed
|
||||
20,src/domain/errors.ts:AnalysisInputError,Parsed analysis input invalid,"Invalid analysis input for {operation}",Exposes operation identifier and lacks correction,"Analysis input is invalid. Check the arguments and try again.",implemented,focused assertions passed; full `npm run check` passed
|
||||
@@ -35,7 +35,7 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
|
||||
24.93,src/server/registerFunctionComparisonTool.ts:probe rationale,Function comparison is incomplete,Capture complete dossiers with equal providers and analysis limits.,Exposes provider architecture and gives ambiguous equality guidance,"Capture complete dossiers for both functions under the same target context and analysis limits.",implemented,verified by focused function-comparison MCP test
|
||||
24.95,incur:CLI argument validation,Caller omits or mistypes a command argument,VALIDATION_ERROR with raw Zod Details and fieldErrors including code/path/expected/received,Exposes validator representation and buries the useful correction,"REA could not read the command arguments. Check the named argument and its expected type, then try again.",implemented,verified by sanitizer unit tests and real dispatcher missing-argument test
|
||||
25,src/application/ReferenceSourceImport.ts:cancelled,Reference-source import is cancelled,Reference source import cancelled,No recovery timing guidance,"Reference source import was cancelled. Start it again when ready.",implemented,focused assertions passed
|
||||
26,src/application/ReferenceSourceImportPolicy.ts:invalid-limits,Configured import limits are not positive integers,Import limits must be positive integers,Fixed importer ceilings prevented full source imports,Removed fixed byte, entry, depth, and path limits.,removed,verified by greater-than-10000 entry and greater-than-16-MiB regressions
|
||||
26,src/application/ReferenceSourceImportPolicy.ts:invalid-limits,Configured import limits are not positive integers,Import limits must be positive integers,Fixed importer ceilings prevented full source imports,"Removed fixed byte, entry, depth, and path limits.",removed,verified by greater-than-10000 entry and greater-than-16-MiB regressions
|
||||
27,src/application/ReferenceSourceImportPolicy.ts:invalid-root,Source path cannot resolve or is not a directory,Reference source root could not be resolved / Reference source root is not a directory,Multiple low-level copies and no complete path check,"Reference source directory could not be opened. Check that the path exists, is readable, and points to a directory.",implemented,focused assertions passed
|
||||
28,src/application/ReferenceSourceImportPolicy.ts:policy,No approved root exists or requested root is outside policy,No approved reference source roots / Reference source root is outside approved roots,Approval friction duplicates direct caller path selection,Removed the configured root allowlist and permission gate; caller-selected directories are validated and traversed directly.,removed,verified by source search and focused import tests
|
||||
29,src/application/ReferenceSourceImport.ts:io,Reader fails during traversal,Raw reader error message,May expose filesystem details and inconsistent recovery,"Reference source files could not be read. Check directory permissions and try again.",implemented,focused assertions passed
|
||||
@@ -72,7 +72,7 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
|
||||
59,src/application/Setup.ts:unsupported macOS,macOS version is missing or older than 12,Upgrade to macOS 12 or newer.,Low risk; exact support floor,Keep current copy,unchanged,verified by setup host tests
|
||||
60,src/application/Setup.ts:unsupported Linux,Linux distribution is outside supported set,"REA supports Hopper on Ubuntu 24.04+, Fedora 41+, Nobara 44+, 64-bit Arch Linux, and CachyOS.",Low risk; exact supported alternatives,Keep current copy,unchanged,verified by setup unit test
|
||||
61,src/application/process/ProcessCli.ts:readJson read,Process scenario or evidence file cannot be read,Raw filesystem exception through CLI framework,Can expose local paths/codes and no recovery,"Process input file could not be read. Check that the path exists and is readable.",implemented,verified by focused CLI test
|
||||
62,src/application/process/ProcessCli.ts:readJson size,Process scenario or evidence file exceeds 64 MiB,Process JSON input exceeds byte limit,Does not state limit or recovery,"Process input file is too large. Reduce it below 64 MiB, then try again.",implemented,verified by focused sparse-file CLI test
|
||||
62,src/application/JsonInputFile.ts:memory admission,Projected process JSON value exceeds available heap headroom,Insufficient heap headroom to materialize the complete JSON value,Heap admission depends on live runtime capacity rather than a fixed 64 MiB file limit,"Provide a smaller valid JSON value. For Evidence workflows, re-analyze a smaller selection of the original target and use its complete Evidence; splitting JSON text or trimming Evidence fields does not produce valid workflow input.",implemented,Source verified against shared JSON reader and recovery constant; former fixed-size refusal is superseded
|
||||
63,src/application/process/ProcessCli.ts:readJson parse,Process scenario or evidence file is malformed JSON,Raw JSON parser exception through CLI framework,Can expose parser implementation and lacks recovery,"Process input file is not valid JSON. Repair the file, then try again.",implemented,verified by focused CLI test
|
||||
64,src/application/process/ProcessCli.ts:capture scenario parse,Scenario fails Process Capture schema,Raw Zod validation exception through CLI framework,Exposes schema internals and field paths without stable recovery,"Process scenario is invalid. Check its required fields and limits, then try again.",implemented,verified by focused CLI test
|
||||
66,src/application/process/ProcessCli.ts:wrong evidence,Comparison receives valid Evidence from another workflow,Expected REA process capture Evidence,Uses internal schema name and does not explain recovery,"Capture evidence is not from the current process-capture workflow. Create new capture evidence, then try again.",implemented,verified by focused CLI test
|
||||
@@ -114,5 +114,5 @@ Then rerun this installer.",implemented,verified by shell branch inspection
|
||||
88,install.sh:binary absent,npm exits successfully without executable,the rea command was not installed at local path,Leaks local path and no recovery,"npm completed without installing the rea command. Check the npm global bin directory and PATH, then retry.",implemented,verified by exact absent-installed-command test
|
||||
89,install.sh:version read,Installed rea executable cannot report its version,could not read the installed REA version.,No recovery instruction,"The installed REA version could not be read. Reinstall the requested version, then retry.",implemented,verified by exact unreadable-installed-version test
|
||||
90,install.sh:version mismatch,Installed rea reports a different version,installed version ACTUAL does not match EXPECTED.,Low risk; gives exact integrity mismatch,Keep current copy,unchanged,verified by exact installer test
|
||||
90.5,@modelcontextprotocol/server:tool input validation,Caller sends arguments that violate a declared tool schema,"Input validation error: Invalid arguments for tool TOOL: FIELD: Invalid input: expected TYPE, received TYPE",Low risk; public tool and field names plus exact type correction,Keep SDK-generated copy,unchanged,verified by real in-memory MCP client/server call with wrong argument type
|
||||
90.5,@modelcontextprotocol/server:tool input validation,Caller sends arguments that violate a declared tool schema,"Canonical invalid_request with the SDK diagnostic plus recovery.kind=inspect_input_contract and a bounded instruction to read the advertised schema or call tools/list",Keeps field/type correction while telling the agent how to recover; diagnostics over 4 KiB stay on the existing retained-error path,Normalize ordinary validation failures and retain oversized diagnostics,verified by real in-memory and stdio MCP client/server calls with wrong argument types
|
||||
90.6,incur:unknown CLI command,Caller enters an unknown REA command,"'COMMAND' is not a command for 'rea'. Suggested command: rea --help",Low risk; identifies input and exact discovery action,Keep framework-generated copy,unchanged,verified by real packaged dispatcher invocation
|
||||
|
+305
-23
@@ -55,6 +55,12 @@ it as inline Evidence to `inspect-analysis-view`; the same summary and capture
|
||||
page contracts apply. Large complete records can still exceed MCP's response
|
||||
budget; export the retained Evidence when a single record cannot fit.
|
||||
|
||||
If decoding completes before cancellation or private-root cleanup fails, the
|
||||
failure retains the completed capture as Evidence with the requested record
|
||||
selection and explicit sensitivity exclusions. Earlier decode failures remain
|
||||
primary. Failed cleanup stays linked to its exact owner, including through
|
||||
redaction, and its live status clears only after a confirmed retry.
|
||||
|
||||
## Formats and upstream profiles
|
||||
|
||||
HAR uses unchanged `har-schema@2.0.0` draft-06 schemas, `ajv@8.20.0`,
|
||||
@@ -116,29 +122,96 @@ XML node tree; optional `_w.txt` and `_g.txt` files retain opaque WebSocket and
|
||||
gRPC message bytes. Missing members remain null, distinct from empty bodies.
|
||||
See Telerik's [SAZ format reference](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/knowledge-base/fiddler-archives).
|
||||
|
||||
The existing ZIP reader verifies read members; expanded archive contents are
|
||||
limited to 32 MiB in addition to the input/output budgets below. Exceeding this
|
||||
expanded-byte budget returns `invalid_request` / `invalid_input` through both
|
||||
CLI and MCP, with an `out_of_range` issue at
|
||||
`capture_path` and `maximum_expanded_capture_bytes`. Selecting records does not
|
||||
bypass the complete-archive admission check or return partial Evidence. Encrypted
|
||||
archives and unrecognized raw session members are unsupported. SAZ requires an
|
||||
enforced decoder memory bound: Node's fixed heap, or the Windows Job Object
|
||||
commit limit. Bun on hosts without that Job Object limit returns capability
|
||||
unavailable before starting the SAZ decoder. Non-session
|
||||
members are inventoried without decoding their content. Original files remain
|
||||
unchanged. Linux verification includes independently checked public recorded
|
||||
Mattermost captures from [FOREST](https://github.com/dfrc-korea/FOREST/tree/405c9154658a128ea203a6c10040cec55111ed03/api-results/dataset)
|
||||
and the Fiddler author's [encoding corpus](https://github.com/ericlaw1979/Clearinet/blob/b6faf78cde1e72042980364f180ceb34b77d1434/Test/data/Encodings.saz),
|
||||
plus synthetic malformed/limit fixtures. Their exporting product versions are
|
||||
not authenticated. An interactive export from a current Fiddler Everywhere
|
||||
installation and its live MCP server remain unverified. Support here is the
|
||||
bounded documented raw HTTP/XML SAZ layout, not every product/version profile.
|
||||
## Protobuf and gRPC bodies
|
||||
|
||||
For visual inspection, use Everywhere's **Import Snapshot**, then **Inspectors**
|
||||
and **Overview**. HAR remains an alternative: select sessions, choose
|
||||
**Export > HTTPArchive**, and confirm `log.version` is `"1.2"`. Preserve the
|
||||
original SAZ when exporting. See the [import/export instructions](https://www.telerik.com/fiddler/fiddler-everywhere/documentation/collaboration/exporting-and-importing).
|
||||
Selecting `record_ordinals` (CLI `--record`) also decodes retained bodies with
|
||||
recognized protobuf or gRPC content types. Whole-capture inspection does not
|
||||
perform this extra work. The original binary sidecar, SHA-256 and producer
|
||||
ordinal remain the source; decoding adds flat body, frame and field facts.
|
||||
|
||||
Native gRPC admits only data-frame flags 0/1; gRPC-web additionally admits its
|
||||
terminal trailer frame. `grpc-web-text` supports independently padded base64
|
||||
chunks. Its offsets refer to **base64-decoded body bytes**; other offsets refer
|
||||
to retained body bytes. Every field has a body-scoped `field_id` derived from
|
||||
its byte offset, a `frame_index`, and a `parent_field_id` for schema-driven
|
||||
nested messages or groups. Repeated field numbers remain separate occurrences.
|
||||
|
||||
Without a schema, varints and fixed-width values retain exact unsigned decimal
|
||||
representations. Length-delimited UTF-8, nested-message and packed readings
|
||||
are candidates; REA does not choose one or invent field names or signedness.
|
||||
Nested candidate ranges remain accessible without recursively expanding them.
|
||||
Malformed messages retain the parsed prefix and an exact error offset.
|
||||
Compressed frames and encoded HTTP bodies remain unknown; they are not
|
||||
silently treated as empty messages.
|
||||
|
||||
For schema interpretation, supply a local compiled `FileDescriptorSet` and its
|
||||
fully qualified message type. All imports must be included. REA never fetches
|
||||
schemas or executes the selected application. Descriptor identity and parser
|
||||
version remain in the body facts. Scalars, repeated numeric packing and nested
|
||||
messages are interpreted alongside the unchanged wire facts; disagreements
|
||||
remain explicit. Map semantics, extensions and required-field validation are
|
||||
unsupported. A supplied schema does not establish the sender's actual schema.
|
||||
An invalid nested interpretation records `schema_status: "invalid-message"`
|
||||
and `schema_error` locally; later outer fields remain available. Valid string
|
||||
contents, including a leading U+FEFF, are preserved.
|
||||
|
||||
```bash
|
||||
# Compile your own schema with your existing protoc, including local imports:
|
||||
protoc --include_imports --descriptor_set_out=/evidence/api.pb api.proto
|
||||
rea inspect-web-network-capture /evidence/session.har har --record 3 \
|
||||
--protobuf-schema /evidence/api.pb --protobuf-message example.Response \
|
||||
--detail summary --evidence_output /evidence/session-evidence.json --json
|
||||
```
|
||||
|
||||
MCP accepts the same choices as `protobuf: {schema: {descriptor_set_path:
|
||||
"/evidence/api.pb", message_type: "example.Response"}}`, with explicit
|
||||
`record_ordinals`. Optional `body_pointers` batch-select exact binary pointers;
|
||||
`framing` can explicitly select `raw`, `grpc`, `grpc-web` or `grpc-web-text`
|
||||
when MIME metadata is unavailable. CLI exposes `--protobuf-body-pointer` and
|
||||
`--protobuf-framing`. Filters never invent bytes excluded by source redaction. Decoding an extra
|
||||
base64 layer also checks explicit exclusions; newly recognized trailer
|
||||
credentials exclude the entire affected binary sidecar and its reported
|
||||
encoding, preserving a redaction coordinate instead of a credential digest.
|
||||
Credential inspection covers every selected body before descriptor reads or
|
||||
field expansion. Later capacity or schema failures can therefore retain the
|
||||
sanitized source even with explicit sensitivity options. If that source pass
|
||||
is interrupted, unchecked bodies are not published as partial Evidence.
|
||||
Framing overrides apply only to format-owned body coordinates and preserve
|
||||
exclusions for the transport declared in MIME metadata.
|
||||
Unknown or explicitly excluded HTTP coding metadata is not treated as absent:
|
||||
SAZ/native bodies remain opaque until identity coding is established. Raw
|
||||
header values and retained body bytes remain available in source Evidence.
|
||||
Skipping wire analysis still runs the independent credential check. If HTTP
|
||||
coding metadata is unavailable and that check cannot complete, the operation
|
||||
returns unavailable without publishing unchecked capture bodies as Evidence.
|
||||
|
||||
Use `detail: "summary"`, then `inspect_analysis_view` with its retained parent:
|
||||
|
||||
```json
|
||||
{"kind":"protobuf","collection":"bodies","offset":0,"limit":16}
|
||||
{"kind":"protobuf","collection":"frames","record_ordinals":[3],"offset":0,"limit":16}
|
||||
{"kind":"protobuf","collection":"fields","record_ordinals":[3],"field_numbers":[1,4],"fields":["number","schema_name","schema_value"],"offset":0,"limit":32}
|
||||
```
|
||||
|
||||
Filters intersect; an empty filter selects nothing. Field pages also accept
|
||||
`field_ids`, `frame_indices` and `schema_names`. Default field pages omit `text`,
|
||||
`schema_value` and enum alias inventories; select these columns when needed. Identity, source offset
|
||||
basis and body status survive column selection. Complete Evidence retains
|
||||
all decoded values, and ordinary capture previews expose counts. Page exhaustion
|
||||
covers retained facts only; a partial source does not establish absence.
|
||||
|
||||
Authenticated immutable parents reuse validation and the most recent field
|
||||
filter per capture. These caches live only with retained Evidence, hold references
|
||||
to existing facts and never reopen either file. Cached selector text is bounded
|
||||
by retained field cardinality; larger selections use an uncached scan. Mutable
|
||||
input is revalidated.
|
||||
The existing 96 MiB report budget includes the original decoder reply plus
|
||||
conservative charges for new facts, strings and descriptor expansion. The explicit
|
||||
`protobuf_coverage` reports selected and inspected record counts, preserving
|
||||
partial decoding through composed views. Capacity failure retains the source
|
||||
observation as partial Evidence; cancellation yields
|
||||
between bounded work batches. Real elapsed time and process-family memory
|
||||
still depend on the selected capture and schema.
|
||||
|
||||
## Fiddler Everywhere MCP and public skills
|
||||
|
||||
@@ -294,7 +367,15 @@ exclusions and deduplicated limitations. It scans path spans without splitting
|
||||
literal paths into arrays. Before schema parsing, it reserves the capture clone
|
||||
and validation scratch by walking the parent without cloning it. Inline requests
|
||||
borrow the Evidence value through CLI/MCP envelope validation; the application
|
||||
admits the entire portable envelope before authenticated parsing or hashing.
|
||||
admits the entire portable envelope before authenticated parsing or hashing. An
|
||||
`evidence-file` source is read and authenticated first and then admitted like
|
||||
retained Evidence, so its strings are not charged as inline envelope text: a
|
||||
capture too large to project inline can be projected from its file. Before
|
||||
authentication REA checks that live heap headroom holds its copies of the parsed
|
||||
Evidence, and over MCP the read follows the request's cancellation.
|
||||
An invalid Evidence file returns `EvidenceFileError` with the selected `path`
|
||||
and `content_issues` relative to the file's JSON root; those locations do not
|
||||
refer to the request's `source.path` string.
|
||||
The envelope clone, hash scratch and capture clone stay charged during grouping.
|
||||
Temporary URL/path/MIME/query work and retained state must
|
||||
fit together within a 96 MiB projection representation budget. A whole group or
|
||||
@@ -315,6 +396,207 @@ parent envelope, capture and examined records are deduplicated onto the view.
|
||||
Portable captures must retain unique ordinals within their declared total;
|
||||
caller-selected subsets can preserve any requested ordinal order.
|
||||
|
||||
## Environment manifest
|
||||
|
||||
Pass retained HAR Evidence to `inspect_analysis_view` with
|
||||
`view: {"kind":"environment-manifest"}` to project one application's recorded
|
||||
request lines into a replayable tool surface: the environment manifest that an
|
||||
agent environment later serves from retained Evidence. The view reuses the
|
||||
api-surface grouping, then keeps the groups on the primary origin, the origin
|
||||
with the most JSON-like records (`application/json`, `application/*+json`,
|
||||
`text/plain`, `text/markdown`), whose path starts with `/api/` or `/apis/` or
|
||||
whose content types are JSON-like. Every other group is listed under
|
||||
`excluded` with reason `other-origin`, `ui-page-or-asset` or `non-json`, and
|
||||
retained records without a usable request line are counted in a limitation.
|
||||
|
||||
The parent may instead be `capture_browser_scenario` Evidence whose scenario
|
||||
selected response bodies or header values: each network transaction is then
|
||||
one record, in request order, and the result names
|
||||
`parent_operation: "capture_browser_scenario"` with an empty `artifact`, since
|
||||
a scenario records no file. See
|
||||
[agent environments](agent-environments.md#from-a-browser-scenario).
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_analysis_view",
|
||||
"arguments": {
|
||||
"source": { "kind": "retained-evidence", "evidence_id": "ev_..." },
|
||||
"view": {
|
||||
"kind": "environment-manifest",
|
||||
"template_rules": [
|
||||
"after-uid-literal",
|
||||
"opaque-token",
|
||||
"sibling-variance"
|
||||
],
|
||||
"include_query_parameter_names": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`origin` and `name` identify the environment; the name is the kebab-case
|
||||
primary host with dots and ports replaced by hyphens, `grafana-example` for
|
||||
`http://grafana.example`. Each tool carries a deterministic `name`, `method`,
|
||||
`path_template`, `api_surface_template` (the four-rule template of its
|
||||
earliest record), `params`, `query_parameter_names`, `body_shape`,
|
||||
`statuses`, `content_types` and `record_ordinals`. `records` lists one pointer
|
||||
`{ordinal, method, path, status, content_type}` per record a tool cites and
|
||||
never a header, cookie or body. Tools sort by template then method.
|
||||
|
||||
Without `template_rules`, every `path_template` equals the api-surface
|
||||
template and `params` carry the four format rules with names `id`, `uuid`,
|
||||
`hash` and `date`; a repeated name gets a numeric suffix. Opt-in rules apply
|
||||
only when listed, each at most once (a repeated rule is an `invalid_request`),
|
||||
in this order:
|
||||
|
||||
| Rule | Applies to | Parameter |
|
||||
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- |
|
||||
| `after-uid-literal` | The literal segment right after a literal `uid`, unless it is a stop word | `uid` |
|
||||
| `opaque-token` | A lowercase alphanumeric segment of twelve or more characters mixing letters and digits that is not a `v1`-style version | `uid` |
|
||||
| `sibling-pair-variance` | Groups equal in method, origin and segment count that differ in two adjacent literals right after a plural literal at index 2 or later and followed by a literal, with at least three distinct pairs and both positions varying (`/repos/{owner}/{repo}/issues`); stop words and `v1`-style versions never qualify | singular plural literal plus `_namespace`, then `_name` |
|
||||
| `sibling-variance` | Groups equal in method, origin and segment count that differ in one literal at index 2 or later; the siblings whose literal is identifier-like (a digit or `:`, or three or more hyphenated names under a plural literal) merge, never a stop word, and the other literals stay their own tools | singular previous literal plus `_id`, else `id` |
|
||||
|
||||
With `sibling-variance` selected, literal siblings within its comparison scope
|
||||
fold into existing opt-in parameters as observed values unless they are stop words.
|
||||
For example, select both `opaque-token` and `sibling-variance` so
|
||||
`/api/folders/{uid}` lists `general` beside the recorded token. With only
|
||||
`opaque-token`, `/api/folders/general` stays a separate literal tool.
|
||||
A position templated by one of the four format rules
|
||||
absorbs no literal, so `/api/teams/{numeric}` keeps its exact-format meaning
|
||||
and `/api/teams/search` stays its own tool. Opt-in placeholders use the
|
||||
parameter name; format-rule placeholders keep their api-surface spelling. Tool names are
|
||||
`<method>_<literals>` with `api`, `apis`, `namespaces`, `default`, `v`-prefixed
|
||||
version tokens and a `uid` literal before a parameter dropped, dotted literals
|
||||
cut to their first label, `_by_<params>` appended when parameters exist,
|
||||
sanitized to `[a-z0-9_]`, at most 60 characters and made unique with numeric
|
||||
suffixes: `get_dashboards_by_uid`, `get_plugins_settings_by_plugin_id`.
|
||||
|
||||
`sibling-variance` and `sibling-pair-variance` compare routes of at most 32 segments; longer routes keep
|
||||
their grouped templates and the manifest says so in a limitation. `body_shape`
|
||||
is derived for `POST`, `PUT` and `PATCH` from the tool's earliest record when it
|
||||
carries an `application/json` or `application/*+json` request body: top-level
|
||||
keys and their JSON type names, never values and never nested structure; a body
|
||||
on a later record adds nothing. `unknowns` always states three facts: the authorization model, response
|
||||
semantics and state transitions are not established by a capture.
|
||||
`templated_groups` counts tools and excluded groups with a templated segment;
|
||||
whenever it is nonzero the Evidence is inferred, otherwise derived. The view
|
||||
performs no network activity and replays nothing. Native mitmproxy, SAZ and
|
||||
qlog captures, and parents inspected with `sensitive_values`, return the same
|
||||
typed unsupported errors as the api-surface view, including the canonical
|
||||
token `sensitive_values_unretained`. A `capture_browser_scenario` parent is not
|
||||
refused for declaring secrets: the capture redacted them as it recorded, a
|
||||
transaction whose URL was redacted is left unrepresented, and a response body
|
||||
keeps the capture's redaction markers, which a manifest limitation counts.
|
||||
|
||||
The manifest accumulator shares the api-surface admission model: the capture
|
||||
clone, the inline envelope (for an inline source), record pointers, groups, every route part, observed
|
||||
values, query names and body-shape keys are admitted before allocation, and each
|
||||
recorded body is charged as temporary bytes before it is parsed. Retained state and
|
||||
temporary work must fit together within the 96 MiB projection representation
|
||||
budget; exceeding it returns `resource_constraint` without a partial manifest.
|
||||
Name a large capture with `{"kind": "evidence-file", "path": "/abs/capture.json"}`
|
||||
rather than inline: response bodies the manifest never reads are then charged
|
||||
once per value in the capture clone, not per character. A
|
||||
`capture_browser_scenario` parent is also charged, before validation, for
|
||||
checking its largest body and for the decoded text of its request bodies.
|
||||
|
||||
## Environment state
|
||||
|
||||
Pass the same parent to `inspect_analysis_view` with
|
||||
`view: {"kind":"environment-state"}` to extract the initial state the
|
||||
capture's reads observed, and a render plan per read tool that reproduces those
|
||||
reads from it. Where the environment manifest names an application's tools, the
|
||||
state holds its data: entities by type and identity, plus documents for written
|
||||
paths whose reads are not entities. Use the `template_rules` of the manifest the
|
||||
state accompanies so tool names match; the view groups records the same way.
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_analysis_view",
|
||||
"arguments": {
|
||||
"source": { "kind": "retained-evidence", "evidence_id": "ev_..." },
|
||||
"view": {
|
||||
"kind": "environment-state",
|
||||
"template_rules": [
|
||||
"after-uid-literal",
|
||||
"opaque-token",
|
||||
"sibling-variance"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Records on the primary origin are ordered by recorded start time, or by record
|
||||
order when any start time is missing (`timeline`), as for a
|
||||
`capture_browser_scenario` parent. A `POST`, `PUT`, `PATCH` or `DELETE` with a
|
||||
2xx or 3xx status, or no status, may change state, unless it is a `POST` whose
|
||||
path has a segment such as `search`, `query`, `list`, `find`, `fuzzy`,
|
||||
`count` or `stats`, which queries state instead. A type's state comes from its
|
||||
successful reads that finished before the first such write under the same root:
|
||||
the path's API base (the literal segments every tool sharing its first segment
|
||||
has), any `v1`-style version segments, then one more segment. A read that
|
||||
overlaps that write in time is left out. A write whose URL a scenario redacted
|
||||
has an unknown target, so it ends the state of every type, and a limitation
|
||||
counts such writes. Values from a `capture_browser_scenario` parent keep the
|
||||
capture's redaction markers in place of declared secrets; the state is not
|
||||
refused for them, as the manifest is not.
|
||||
|
||||
Each `GET` tool whose template ends in a parameter is an item tool, and its
|
||||
type's `collection_template` is the template without that parameter; any other
|
||||
`GET` tool is a list tool of its own template. An item tool proves its type's
|
||||
`id_field` when every initial response carries the requested path value in the
|
||||
same field, at the root or in one wrapper member (`id_basis: "path-parameter"`).
|
||||
Otherwise the field is the first of `id`, `uid`, `uuid`, `_id`, `ID`, `UID`,
|
||||
`Id`, `guid`, `key`, `slug` and `name` that every listed item carries as a
|
||||
distinct string or number (`id_basis: "field-name"`). A list is the response
|
||||
array, or the one array of objects in an envelope that is not itself an entity.
|
||||
|
||||
`entities` merge every initial observation of one identity within one
|
||||
collection path: an item view's latest values win over a list view's, a view
|
||||
whose latest values differ keeps them in `view_fields`, and a field whose value
|
||||
changed between one tool's observations with no recorded write between is named
|
||||
in the type's `volatile_fields`, keeping the latest value. A value that changed
|
||||
across a write under another root may be that write's effect, so every tool of
|
||||
its type is a recorded view with reason `fields-differ`; so is a value one tool
|
||||
read before a write and another tool read after it. Repeated reads of one full request line also keep the type recorded when list membership changes across a write, including transitions to or from an empty list, or when entity fields are added or removed. Different query strings remain distinct request lines. Sparse responses do not erase the last observation of a field supplied by another request line. Entities are ordered
|
||||
by type, collection and first sighting; that order is a list's state order.
|
||||
Every entity cites its `record_ordinals`.
|
||||
|
||||
`views` lists one plan per read tool in tool order:
|
||||
|
||||
| Shape | Plan |
|
||||
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity object at `pointer` (the root or one member), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares |
|
||||
| `list` | The array at `pointer`, its `fields`, `membership` (`all`: every entity of the collection in state order, when each recording listed exactly that, had no query, no item-total member (`total`, `count`, `totalCount`, `item_count`, `totalcount`, `itemcount`, but not `total_pages` or `per_page`; a number or a string of digits) differing from the number listed, and no `has_more`, `next` or next-page token member set; `recorded`: the entities each recording listed) and `counters` that equal the number listed |
|
||||
| `document` | The path's latest initial value, held in `documents` |
|
||||
| `recorded` | A `reason` the tool replays its recordings instead |
|
||||
|
||||
An item or list plan is kept only when it reproduces the latest initial
|
||||
recording of each request line from the state, comparing JSON values without
|
||||
regard to member order; a volatile field compares its recorded value. Each
|
||||
plan lists the `checked_record_ordinals`. Otherwise the tool is a recorded view
|
||||
with reason `envelope-varies`, `fields-differ` or `list-differs`; the other
|
||||
reasons are `query-body` (a query `POST`), `no-init-recording`,
|
||||
`body-unusable`, `not-entity`, `no-identity` and `query-varies` (a document
|
||||
read under several queries). Documents are held only for list-shaped paths
|
||||
that a `POST`, `PUT` or `PATCH` tool writes: a non-entity read of an item path
|
||||
such as `/api/files/{id}`, or of a path that is only deleted, stays recorded
|
||||
with `no-identity` or `not-entity`. A body is usable when its content type is
|
||||
JSON-like, it is UTF-8 text of at most 256 KiB, every number survives native
|
||||
conversion exactly, and no member is named `__proto__`.
|
||||
|
||||
Values in the state are recorded data, so the view refuses the same parents as
|
||||
the manifest, including those inspected with `sensitive_values`, and a
|
||||
`capture_browser_scenario` parent must have selected response bodies. Types and
|
||||
identity are inferences, so the Evidence is always inferred. `unknowns` states
|
||||
that the state establishes no authorization model, no way the application
|
||||
computes responses or derived values, and no effects of writes. The state
|
||||
accumulator charges the manifest's admission and grouping, every record line,
|
||||
each body as temporary bytes before it is decoded and parsed, and every value
|
||||
it keeps, within the same 96 MiB budget. The view performs no network activity
|
||||
and serves nothing.
|
||||
|
||||
## Resource and lifecycle boundaries
|
||||
|
||||
Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting
|
||||
|
||||
@@ -1,5 +1,43 @@
|
||||
# Analysis efficiency measurements
|
||||
|
||||
The [native inventory view checkpoint](native-inventory-views.md) compares
|
||||
complete delivery with summaries and selected rows on real `true` and jq
|
||||
inventories, including oversized-response recovery, portable old Evidence,
|
||||
CLI/MCP parity, tokenizer estimates and independent-engine variation.
|
||||
|
||||
The [retained-capture validation checkpoint](retained-capture-validation.md)
|
||||
reconciles the earlier audit queue and measures immutable historical-capture
|
||||
validation reuse on public compact and large qlogs, including first-view costs,
|
||||
small-input limits and unchanged Evidence delivery.
|
||||
|
||||
The [Ghidra Evidence projection checkpoint](ghidra-evidence-projection.md)
|
||||
measures removal of redundant raw copies on real `true` and jq workflows,
|
||||
with complete provenance, unchanged response budgets, native CLI/MCP checks
|
||||
and explicit cold-analysis variance.
|
||||
|
||||
For Hopper literal string queries, see the [decoded-record checkpoint](hopper-string-records.md).
|
||||
It measures record reuse through real Linux CLI/MCP workflows, preserving
|
||||
native recognition gaps, guard-stopped attempts and negative whole-task costs.
|
||||
|
||||
The [Linux embedding-runtime follow-up](hopper-linux-python-runtime.md) checks
|
||||
actual clean setup, dependency repair and normal-user analysis, retaining
|
||||
failed attempts and distinguishing prerequisite correctness from speedups.
|
||||
|
||||
For JEB on Linux, see the [CE native audit](jeb-linux-native.md). It qualifies
|
||||
the direct headless launch route and measures selected function reads through
|
||||
real CLI and production stdio MCP, preserving timeout attempts and the limits
|
||||
of wrapper-cache and token claims.
|
||||
|
||||
For Apktool locale queries, see the
|
||||
[shared-decode and table-only checkpoint](apktool-string-tables.md). It separates
|
||||
the existing decode reuse from duplicate string delivery, with three real APKs,
|
||||
CLI/MCP parity, whole-task estimates and negative timing results.
|
||||
|
||||
For immutable imported native/JavaScript parents, see the
|
||||
[validation-reuse checkpoint](immutable-parent-validation.md). It measures the
|
||||
owning format change with the existing retained-query driver, including first
|
||||
view costs, small cases, unchanged token estimates, and noisy whole-task results.
|
||||
|
||||
For authentic old Evidence and mixed investigations, see the
|
||||
[retained-query checkpoint](retained-investigation-workflows.md).
|
||||
`retained-investigation.mjs` compares existing complete delivery with selected
|
||||
|
||||
Loaded 100 of 1229 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user