Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a6380ee11f | ||
|
|
3140f8c002 | ||
|
|
011082201d | ||
|
|
d4b12d8b93 | ||
|
|
86655eac66 | ||
|
|
536f5ce38b | ||
|
|
b6e6d7cd1a | ||
|
|
6714fb1d02 | ||
|
|
92a9591a41 | ||
|
|
2425493e10 | ||
|
|
c592ea6bac | ||
|
|
740f107ff9 | ||
|
|
f33b7bd617 | ||
|
|
4a90059fe2 | ||
|
|
aa521379dd | ||
|
|
5d3762456f | ||
|
|
1f9d1b5de1 | ||
|
|
02e2c9cd7f | ||
|
|
3f35eacad2 | ||
|
|
1ba178befb | ||
|
|
02a0e72f70 | ||
|
|
b4bc6cf3ee | ||
|
|
1684c73d4f | ||
|
|
b5c47f5255 | ||
|
|
7e8ec0e456 | ||
|
|
7436ac8e20 | ||
|
|
de00fdc17d | ||
|
|
75575f2006 | ||
|
|
db8556e724 | ||
|
|
e3a00bb5f9 | ||
|
|
af42f856ec | ||
|
|
809f3b7b5b | ||
|
|
9a21cd78ce | ||
|
|
be4498d394 | ||
|
|
5d3ae66e63 | ||
|
|
26573c3aa4 | ||
|
|
b2048d46d1 | ||
|
|
8ecbb74b00 | ||
|
|
9c81191b1d | ||
|
|
220414682e | ||
|
|
d11fca46fc | ||
|
|
5c8d5b2a8a | ||
|
|
87657c6297 | ||
|
|
af73951405 | ||
|
|
ab2119e9b8 | ||
|
|
6da3839b57 | ||
|
|
54ea293e7c | ||
|
|
25a0f3c3b8 | ||
|
|
6b55802df6 | ||
|
|
4b4babf1e7 | ||
|
|
26bc77ce56 | ||
|
|
7aee876d81 | ||
|
|
49c804b7ec | ||
|
|
295e7efddf | ||
|
|
9a426033ff | ||
|
|
6dc0f60faa | ||
|
|
44a4c1ba87 | ||
|
|
0d5c7b6d79 | ||
|
|
77c72b96aa | ||
|
|
510838bd70 | ||
|
|
49bb6c84b0 | ||
|
|
5fbea61e5a | ||
|
|
791f6328b0 | ||
|
|
e28c11f87d | ||
|
|
b9ca37b51f | ||
|
|
09416de0cd | ||
|
|
888b2afc0d | ||
|
|
106779208d | ||
|
|
7501911f17 | ||
|
|
a6b9312577 | ||
|
|
b43daa166f | ||
|
|
fd5cd1b058 | ||
|
|
304c978368 | ||
|
|
54f51f0d7d | ||
|
|
566a9a1551 | ||
|
|
957bcfa149 | ||
|
|
71dc6fe5bd | ||
|
|
5bda0a7dc3 | ||
|
|
6d977db1b5 | ||
|
|
7ec9b6e56e | ||
|
|
2127095e3b | ||
|
|
584937c761 | ||
|
|
306ed79932 | ||
|
|
5375ff0f9a | ||
|
|
9512a80804 | ||
|
|
3d472bde23 | ||
|
|
8761d4251f | ||
|
|
2846cc0570 | ||
|
|
30fef57f75 | ||
|
|
5d7d6ad9ab | ||
|
|
2589c0074d | ||
|
|
e780148305 | ||
|
|
edccb430c4 | ||
|
|
293832bafc | ||
|
|
e93e3b1e7d | ||
|
|
2c3dd946d5 | ||
|
|
4043e71f6c | ||
|
|
932fbb7d84 | ||
|
|
a4a548718d | ||
|
|
3a42a9e356 | ||
|
|
46cc5ce056 | ||
|
|
3b2d2863da | ||
|
|
aff5e23ecf | ||
|
|
2d91e2d578 | ||
|
|
690a63abd6 | ||
|
|
117ded88f0 | ||
|
|
bfba99827e | ||
|
|
44827a5020 | ||
|
|
055d1a9e1b | ||
|
|
ca4ca65c5e | ||
|
|
31dc035041 | ||
|
|
21f9af9d93 | ||
|
|
c7efd3a388 | ||
|
|
66aacc6fba | ||
|
|
79dea89f1f | ||
|
|
7d0efef62d | ||
|
|
933af60eba | ||
|
|
04fd41b288 | ||
|
|
002e1b68c6 | ||
|
|
73b756430f | ||
|
|
2452efa921 | ||
|
|
b6007e0a58 | ||
|
|
c9b4cae9dc | ||
|
|
86357bad54 | ||
|
|
1639189f26 | ||
|
|
c083fedaf4 | ||
|
|
c02945bc44 | ||
|
|
2edf3b23c1 | ||
|
|
c8ed328538 | ||
|
|
ef5ab1f1f2 | ||
|
|
14877a069b | ||
|
|
7aa1b51282 | ||
|
|
538017d1d2 | ||
|
|
b8c1b45316 | ||
|
|
407a718050 | ||
|
|
51be954c4b |
No files matched your search
@@ -2,7 +2,7 @@
|
||||
name: reverse-engineer-anything
|
||||
description: Reverse engineer native, managed, Electron/JavaScript, packaged, firmware, and browser targets with REA. Use shipped-artifact or requested runtime evidence to explain features, compare versions, decompile code, or guide a reconstruction. Skip REA for ordinary source-repository architecture analysis.
|
||||
metadata:
|
||||
version: "44"
|
||||
version: "45"
|
||||
---
|
||||
|
||||
# REA
|
||||
@@ -32,6 +32,14 @@ original task after repair. Client approval, shell PATH, registration and engine
|
||||
availability have different remedies; avoid repeating setup for all of them.
|
||||
If the host times out without a REA result, do not repeat the unchanged call or
|
||||
assume work stopped. Read the recovery guide's deadline distinction before retrying.
|
||||
If the MCP connection drops, retry that call once only when it is safe to
|
||||
repeat or known not to have started. If it may already have changed the
|
||||
target, or it fails again, tell the user and ask them to restart the MCP
|
||||
client. If REA could not start its connection, retry once, then tell the user
|
||||
and ask them to restart the MCP client. Do not retry a startup failure that
|
||||
returns the same unchanged result. Run `rea doctor` when REA could not start
|
||||
or tools stay missing after that restart. Read the recovery guide before a
|
||||
second repair attempt.
|
||||
|
||||
1. Diagnose without changing REA or client configuration. `npx` may download
|
||||
the package and write to npm's cache. For Codex, run
|
||||
@@ -126,7 +134,10 @@ explicit path or endpoint and do not need it.
|
||||
connect Everywhere's separate MCP server for live traffic.
|
||||
- Retained HAR/SAZ/native mitmproxy/aioquic qlog capture: `inspect_web_network_capture`. Select
|
||||
original record ordinals when useful; inspect producer fields and byte/number
|
||||
sidecars without fetching recorded URLs or claiming live attribution. See
|
||||
sidecars without fetching recorded URLs or claiming live attribution. HAR/SAZ/qlog
|
||||
old-generation heap is `REA_CAPTURE_DECODER_OLD_GENERATION_MIB` (default 192 MiB,
|
||||
floor 8, maximum 16384), set where REA starts. Host memory and `NODE_OPTIONS` do not
|
||||
raise it. See
|
||||
[historical captures](https://github.com/morluto/rea/blob/main/docs/web-network-captures.md) for the exact upstream
|
||||
profile and credential exclusions. Qlog support starts with aioquic 1.5.0
|
||||
JSON 0.3; stream coverage and endpoint loss declarations do not recover bodies
|
||||
@@ -166,6 +177,14 @@ explicit path or endpoint and do not need it.
|
||||
Select JavaScript glue and candidate paths explicitly and preserve URL/path
|
||||
ambiguity. No implicit fetch or target execution; WAT is not original source.
|
||||
See the [WASM guide](https://github.com/morluto/rea/blob/main/docs/wasm-artifacts.md).
|
||||
- Explicit local Android APK (or ZIP/AAB) signing: `inspect_apk_signature`.
|
||||
Prefer its summary or selected signer/digest/lineage page; retain complete
|
||||
Evidence and use `inspect_analysis_view` kind `apk-signature` for follow-ups.
|
||||
Reports which schemes are present (v1 JAR, v2, v3, v3.1), each signer's
|
||||
certificate SHA-256 fingerprint (v1 included, from its PKCS#7 block), v3 key-rotation lineage, source stamp, and
|
||||
whether each signed digest still matches the bytes (self-consistency). This
|
||||
is identity and integrity, not a trust or authenticity verdict.
|
||||
See the [APK signature guide](https://github.com/morluto/rea/blob/main/docs/apk-signature.md).
|
||||
- Explicit local EVM bytecode carrier: `inspect_evm_interface` with caller-selected
|
||||
`raw` or `hex` encoding. The owned worker deadline is `timeout_ms` (default
|
||||
30000 ms, range 1..2147483647, next call). It does not raise the fixed byte,
|
||||
@@ -219,6 +238,33 @@ explicit path or endpoint and do not need it.
|
||||
delivery requires `--evidence_output`.
|
||||
See the
|
||||
[Rust binary guide](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
- PE Authenticode certificate table: `inspect_pe_certificates` with the explicit
|
||||
local PE32/PE32+ file. This built-in target-free inspection requires no
|
||||
Windows SDK. Preserve WIN_CERTIFICATE ranges, types, payload SHA-256, and
|
||||
PKCS#7 signer/X.509 identity fields, including `subject_key_identifier_hex`
|
||||
when the signer uses that form. A missing security directory is observed
|
||||
absence of that table, not proof the image is unsigned in every catalog sense.
|
||||
Signatures are not verified. See the
|
||||
[PE certificate guide](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
|
||||
file. This built-in target-free inspection requires no disassembler. Preserve
|
||||
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
|
||||
build-id or ABI-tag fields. A missing table is observed absence of those
|
||||
declarations, not proof the image never carried notes. See the
|
||||
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
- CPython 3.7–3.13 `.pyc` header, nested marshal code objects and bytecode
|
||||
identity: `inspect_python_pyc` with the explicit local file. This built-in
|
||||
target-free inspection requires no Python installation. Preserve magic,
|
||||
PEP 552 hash or mtime/size, nested names, and bytecode SHA-256. Bytecode is
|
||||
not decompiled, and a decoded header does not prove the matching `.py` is
|
||||
present. See the
|
||||
[CPython pyc guide](https://github.com/morluto/rea/blob/main/docs/python-pyc.md).
|
||||
- Erlang BEAM IFF container: `inspect_erlang_beam` with the explicit local
|
||||
`.beam` file. This built-in target-free inspection requires no Erlang
|
||||
installation. Preserve chunk names, atom order, export/import/lambda tables,
|
||||
and the Code header. Missing optional tables stay null. Bytecode is not
|
||||
executed. See the
|
||||
[Erlang BEAM guide](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
|
||||
with Node.js 24.x >=24.15 or 26+ providing native SQLite limits. Read schema
|
||||
@@ -241,6 +287,8 @@ explicit path or endpoint and do not need it.
|
||||
- PE resource type/name/language trees and static icon group references:
|
||||
`inspect_pe_resources` with an explicit path. This portable operation requires
|
||||
no active target or engine; opaque payloads retain file ranges and SHA-256.
|
||||
- PE export directories and ELF dynamic-symbol exports: `inspect_native_exports`
|
||||
with an explicit path; portable, no active target or engine.
|
||||
- Native executable, library, or analysis database: `open_binary`, then
|
||||
use focused analysis tools directly; call `binary_overview` when metadata or
|
||||
inventory context is useful and available from the selected provider.
|
||||
|
||||
@@ -11,14 +11,15 @@ main and an older release expose the same arguments.
|
||||
|
||||
## Choose the remedy from the observed failure
|
||||
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
| Observation | Meaning and next step |
|
||||
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| No REA tools in the active session | Inspect registration with `rea doctor --client codex --json`. A skill installation alone does not register MCP. |
|
||||
| Missing, malformed or stale registration | Prepare a scoped setup plan; apply only authorized changes with backups. |
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| MCP connection dropped or REA could not start | If the connection drops, retry that call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or the same failure returns, tell the user REA's connection failed and ask them to restart the MCP client. If REA could not start its connection, retry once, then tell the user and ask them to restart the MCP client. Do not retry a startup failure that returns the same unchanged result. Run `rea doctor` when REA could not start or tools stay missing after restart. Doctor cannot restore a live connection by itself. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
|
||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
||||
inspect its launch diagnostic. For example, Codex can refuse helper aliases when
|
||||
@@ -56,8 +57,13 @@ authorized CLI fallback. Do not silently broaden permissions or switch targets.
|
||||
|
||||
A returned REA `provider_timeout` instead means the **server/provider** reached
|
||||
its deadline. Follow its `remediation.action`, reuse accessible partial Evidence,
|
||||
and resolve reported incomplete cleanup before retrying. Raising only the
|
||||
client deadline does not change this server budget. For static JavaScript,
|
||||
and resolve reported incomplete cleanup before retrying. Change a request
|
||||
field or a `REA_*` setting only when the returned remediation explicitly
|
||||
recommends that change. Restart REA when that text says the value is read at
|
||||
startup. A mention that a control does not raise the deadline is not a
|
||||
recommendation to change it. Do not invent a setting the remediation does not
|
||||
name. Raising only the client deadline does not change
|
||||
this server budget. For static JavaScript,
|
||||
`analysis_timeout_ms` (CLI `rea analyze-javascript-application <path>
|
||||
--analysis-timeout-ms <milliseconds>`) controls each source's
|
||||
startup, analysis and transfer deadline in **milliseconds**. Increase that
|
||||
|
||||
@@ -93,10 +93,14 @@ Use `view: {kind: "environment-state"}` with the manifest's `template_rules` to
|
||||
extract the data those reads observed instead: entity types and identity
|
||||
fields, entities with recorded values, documents, and per-tool render plans
|
||||
checked against the recordings. The state comes only from reads before the
|
||||
first related write; types, identity and list membership are inferences.
|
||||
first related write; types, identity and list membership are inferences. The
|
||||
view also gives each state-changing tool a write plan, served only when a later
|
||||
recorded read confirms its effect and none contradicts it.
|
||||
`rea environment-serve manifest.json --evidence capture.json` derives that
|
||||
state at startup, answers read tools from it and serves `data/reset`,
|
||||
`data/add` and `data/get` on `/agentenv`; write tools are not served. It
|
||||
replays the capture instead with `--replay-only` or when the state is
|
||||
unavailable (not derivable, or a catalog over the discovery budget), and its
|
||||
stopped document's `state` says which applied.
|
||||
state at startup, answers read tools from it, serves the write tools whose plan
|
||||
is served, each applying its effect to that state, and serves `data/reset`,
|
||||
`data/add` and `data/get` on `/agentenv`; other write tools are not served, and
|
||||
each plan states why in its `reason`. It replays the capture instead with
|
||||
`--replay-only` or when the state is unavailable (not derivable, or a catalog
|
||||
over the discovery budget), and its stopped document's `state` says which
|
||||
applied.
|
||||
@@ -59,6 +59,20 @@ in the same session. A returned `error.details.partial_observation` reference
|
||||
can be supplied directly to `inspect_analysis_view`. If the SDK cancels locally, retrieve its recorded ID from
|
||||
the Evidence bundle before inspecting the partial analysis.
|
||||
|
||||
## Inventory route candidates without returning the application graph
|
||||
|
||||
When the advertised schema supports `view`, use `detail: "summary"` with
|
||||
`view: {kind: "javascript-routes", collection: "routes", fields: ["path", "confidence"], limit: 20}`.
|
||||
The same view works with `analyze_web_bundle` on already selected captured sources.
|
||||
The complete source Evidence is retained; use its `parent_evidence_id` with
|
||||
`inspect_analysis_view` for `summary`, `routes`, `occurrences`, `sources`, or
|
||||
`unknowns`. Filter and batch IDs before requesting more fields, and follow
|
||||
`coverage.next_offset`. CLI selections require `--evidence_output` to keep the
|
||||
complete record. Static patterns, inferred deployment bases, heuristic system
|
||||
classification, and unavailable sources never establish route reachability or
|
||||
the complete application route denominator. See the
|
||||
[route inventory guide](https://github.com/morluto/rea/blob/main/docs/javascript-route-inventory.md).
|
||||
|
||||
## Read a module without repeating analysis
|
||||
|
||||
Use the actual returned parent ID, on the same connection, as `source.evidence_id`.
|
||||
|
||||
@@ -96,6 +96,51 @@ dependencies. An omitted package kind stays null. A rustc commit hash is
|
||||
inferred from a read-only `/rustc/<40-hex>/` path and does not by itself
|
||||
identify the compiler. See [Rust binaries](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
|
||||
## PE Authenticode certificate tables
|
||||
|
||||
Use `inspect_pe_certificates` for an explicit local PE32/PE32+ image when the
|
||||
question concerns the Authenticode certificate table, WIN_CERTIFICATE types, or
|
||||
embedded PKCS#7 signer and X.509 identity fields. No `open_binary`, Windows SDK,
|
||||
or target execution is required. The CLI equivalent is
|
||||
`inspect-pe-certificates PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
payload hashes, original validity-time spellings, and
|
||||
`subject_key_identifier_hex` when the signer uses that form. A missing
|
||||
`IMAGE_DIRECTORY_ENTRY_SECURITY` directory is observed absence of that table,
|
||||
not proof the image is unsigned in every catalog or overlay sense. Signatures,
|
||||
chains, timestamps, and Authenticode digest coverage remain unobserved. See
|
||||
[PE Authenticode certificate tables](https://github.com/morluto/rea/blob/main/docs/pe-certificates.md).
|
||||
|
||||
## ELF note tables
|
||||
|
||||
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
|
||||
question concerns PT_NOTE/SHT_NOTE owners, GNU build-id, or GNU ABI-tag
|
||||
fields. No `open_binary` or target execution is required. The CLI equivalent
|
||||
is `inspect-elf-notes PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
descriptor hashes, and original owner spellings. A missing table is observed
|
||||
absence of those declarations, not proof the image never carried notes in
|
||||
another container. See
|
||||
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
|
||||
## CPython pyc files
|
||||
|
||||
Use `inspect_python_pyc` for an explicit local CPython 3.7–3.13 `.pyc` when the
|
||||
question concerns the header magic, PEP 552 source hash or timestamp, nested
|
||||
code-object names, or bytecode identity. No `open_binary`, Python installation,
|
||||
or target execution is required. The CLI equivalent is `inspect-python-pyc PATH`.
|
||||
Preserve artifact SHA-256, the 16-byte header fields, nested code names, and
|
||||
bytecode SHA-256. Bytecode is not decompiled. A decoded header does not prove
|
||||
the matching `.py` is present. See [CPython pyc inspection](https://github.com/morluto/rea/blob/main/docs/python-pyc.md).
|
||||
|
||||
## Erlang BEAM files
|
||||
|
||||
Use `inspect_erlang_beam` for an explicit local `FOR1`/`BEAM` file when the
|
||||
question concerns IFF chunks, atom names, exports, imports, lambdas, or the
|
||||
Code header. No `open_binary`, Erlang installation, or bytecode execution is
|
||||
required. The CLI equivalent is `inspect-erlang-beam PATH`. Preserve artifact
|
||||
SHA-256, chunk file ranges, 1-based atom indices, and original atom spellings.
|
||||
Missing optional tables stay null. See
|
||||
[Erlang BEAM files](https://github.com/morluto/rea/blob/main/docs/erlang-beam.md).
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
Use `inspect_sqlite_database` with an explicit local database snapshot for its
|
||||
@@ -150,6 +195,30 @@ and tool profile. Text offsets count UTF-16 code units; follow `next_offset`.
|
||||
The complete source remains retained after an oversized MCP reply. No target
|
||||
execution or network acquisition is implied, and WAT is decoded representation.
|
||||
|
||||
## Offline APK signatures
|
||||
|
||||
Use `inspect_apk_signature` for a selected local APK/ZIP/AAB. Default MCP summaries
|
||||
retain complete Evidence. Select `view.collection: signers/digests/lineage`,
|
||||
with scheme or ID batches, fields and paging; use `inspect_analysis_view` kind
|
||||
`apk-signature` for retained reads after deletion. IDs retain artifact/scheme
|
||||
scope and signer links; `fields: []` keeps identity links. Empty pages preserve
|
||||
partial decoding reasons and unknown authenticity. CLI compact delivery needs
|
||||
`--evidence_output` to preserve the complete parent.
|
||||
It reads the APK
|
||||
Signing Block (v2/v3/v3.1) and v1 `META-INF` signing without executing anything,
|
||||
and reports each signer's certificate facts (for v1, the certificate the PKCS#7
|
||||
block's SignerInfo names), the declared v3 rotation lineage,
|
||||
the declared source-stamp certificate and, for SHA-256/SHA-512 digests, whether
|
||||
the declared content digest matches the current bytes. That match is not a
|
||||
signature verification: anyone who changes the bytes can recompute the digest.
|
||||
`posture` summarises v1-only signing (which cannot tell a v1-only APK from one
|
||||
whose v2/v3 block was removed), observed signer agreement across schemes,
|
||||
whether the v1 signer equals the block-scheme signer (`false` is notable),
|
||||
whether every differing signer appears in a declared (unverified) rotation
|
||||
lineage, and the strongest scheme.
|
||||
It never claims authenticity: no signature or lineage is cryptographically
|
||||
verified.
|
||||
|
||||
## Native targets
|
||||
|
||||
After `open_binary`, use focused search, procedure, or function tools directly.
|
||||
@@ -368,6 +437,8 @@ For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
|
||||
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
|
||||
|
||||
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
|
||||
|
||||
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
|
||||
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
|
||||
`arm`, `arm64`, `6502`, or `m68k`), and `entry_point` (`0x` hex). File offset 0 is mapped at
|
||||
@@ -396,6 +467,16 @@ Other payloads are opaque, and this operation does not decode images or simulate
|
||||
Windows resource loading. Consult the budgets and supported tree profile in
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
|
||||
## Offline PE and ELF export tables
|
||||
|
||||
Use `inspect_native_exports` with an absolute `path`, or
|
||||
`rea inspect-native-exports ./libexample.so --json`, to read a PE export
|
||||
directory or ELF dynamic symbol table on any host without opening a native
|
||||
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
|
||||
partial or unsupported coverage means the inventory is incomplete, not that
|
||||
exports are absent. Use `inspect_macho` for Mach-O. See
|
||||
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
|
||||
|
||||
For exact core-module import/export differences, use `compare_wasm_interfaces`
|
||||
with two complete inspection Evidence records (or current-session retained
|
||||
references) and explicit scope. Preserve unknown types/duplicate names; static
|
||||
|
||||
@@ -72,6 +72,7 @@ jobs:
|
||||
browser: ${{ steps.plan.outputs.browser }}
|
||||
binary_layout: ${{ steps.plan.outputs.binary_layout }}
|
||||
evm: ${{ steps.plan.outputs.evm }}
|
||||
apk_signature: ${{ steps.plan.outputs.apk_signature }}
|
||||
javascript_recovery: ${{ steps.plan.outputs.javascript_recovery }}
|
||||
recorded_crash: ${{ steps.plan.outputs.recorded_crash }}
|
||||
web_network: ${{ steps.plan.outputs.web_network }}
|
||||
@@ -384,6 +385,7 @@ jobs:
|
||||
needs.changes.outputs.browser == 'true' ||
|
||||
needs.changes.outputs.binary_layout == 'true' ||
|
||||
needs.changes.outputs.evm == 'true' ||
|
||||
needs.changes.outputs.apk_signature == 'true' ||
|
||||
needs.changes.outputs.javascript_recovery == 'true' ||
|
||||
needs.changes.outputs.recorded_crash == 'true' ||
|
||||
needs.changes.outputs.web_network == 'true' ||
|
||||
@@ -1003,6 +1005,16 @@ jobs:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
apk_signature:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.apk_signature == 'true'
|
||||
secrets:
|
||||
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
|
||||
uses: ./.github/workflows/real-apk-signature.yml
|
||||
with:
|
||||
build_cache_artifact: rea-build-cache
|
||||
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
|
||||
|
||||
javascript_recovery:
|
||||
needs: [changes, static, build]
|
||||
if: needs.changes.outputs.javascript_recovery == 'true'
|
||||
@@ -1103,6 +1115,7 @@ jobs:
|
||||
browser,
|
||||
binary_layout,
|
||||
evm,
|
||||
apk_signature,
|
||||
javascript_recovery,
|
||||
recorded_crash,
|
||||
web_network,
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
name: Real APK signature inspection
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
build_cache_artifact:
|
||||
description: Portable build cache artifact from the calling CI run.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
remote_cache_write:
|
||||
description: Trusted release qualification may publish the remote cache.
|
||||
required: false
|
||||
type: boolean
|
||||
default: false
|
||||
secrets:
|
||||
TURBO_TOKEN:
|
||||
description: Remote cache token. Only trusted callers pass it.
|
||||
required: false
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
HUSKY: "0"
|
||||
# A called workflow sees the caller event, so workflow_call is not an event
|
||||
# name here. Writes when remote_cache_write is set, or on the default branch
|
||||
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
|
||||
# and use the local cache only.
|
||||
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
|
||||
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
|
||||
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
|
||||
NODE_OPTIONS: "--max-old-space-size=768"
|
||||
RAYON_NUM_THREADS: "1"
|
||||
GOMAXPROCS: "2"
|
||||
UV_THREADPOOL_SIZE: "2"
|
||||
JAVA_TOOL_OPTIONS: "-Xmx512m -XX:ActiveProcessorCount=2"
|
||||
|
||||
jobs:
|
||||
real-apk-signature:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 12
|
||||
steps:
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
|
||||
with:
|
||||
node-version-file: .nvmrc
|
||||
- name: Cache npm package downloads
|
||||
id: npm-cache
|
||||
uses: ./.github/actions/npm-cache
|
||||
- run: npm ci
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Prepare pinned SDK oracle without installing host software
|
||||
shell: bash
|
||||
run: |
|
||||
python3 - <<'PYTHON'
|
||||
import hashlib, os, urllib.request, zipfile
|
||||
from pathlib import Path
|
||||
root = Path(os.environ["RUNNER_TEMP"]) / "rea-apk-signature-sdk"
|
||||
packages = [
|
||||
("build-tools_r36.1_linux.zip", "a7b5889e4a79fcf3b0976bef40d401f4240fb1eed891d9d91169da1111e11d78", 64195204, "build-tools/36.1.0", {"apksigner", "aapt2", "lib/apksigner.jar", "source.properties"}),
|
||||
("platform-36_r02.zip", "37607369a28c5b640b3a7998868d45898ebcb777565a0e85f9acf36f29631d2e", 65878410, "platforms/android-36", {"android.jar", "source.properties"}),
|
||||
]
|
||||
for name, expected, expected_size, directory, selected in packages:
|
||||
archive = root / name
|
||||
archive.parent.mkdir(parents=True, exist_ok=True)
|
||||
digest = hashlib.sha256()
|
||||
total = 0
|
||||
with urllib.request.urlopen("https://dl.google.com/android/repository/" + name, timeout=60) as response, archive.open("wb") as output:
|
||||
while chunk := response.read(1024 * 1024):
|
||||
total += len(chunk)
|
||||
if total > expected_size:
|
||||
raise RuntimeError("SDK archive exceeds the pinned package size budget")
|
||||
digest.update(chunk)
|
||||
output.write(chunk)
|
||||
if total != expected_size or digest.hexdigest() != expected:
|
||||
raise RuntimeError("SDK oracle archive digest mismatch")
|
||||
extracted = set()
|
||||
with zipfile.ZipFile(archive) as source:
|
||||
for entry in source.infolist():
|
||||
relative = entry.filename.partition("/")[2]
|
||||
if relative not in selected:
|
||||
continue
|
||||
target = root / directory / relative
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
target.write_bytes(source.read(entry))
|
||||
target.chmod((entry.external_attr >> 16) & 0o777 or 0o644)
|
||||
extracted.add(relative)
|
||||
if extracted != selected:
|
||||
raise RuntimeError("Pinned SDK package is missing required oracle files")
|
||||
with Path(os.environ["GITHUB_ENV"]).open("a") as environment:
|
||||
environment.write("ANDROID_HOME=" + str(root) + "\n")
|
||||
PYTHON
|
||||
- run: npm run verify:apk-signature -- --require-tools
|
||||
@@ -59,7 +59,7 @@ do not establish fewer tokens or faster analysis.
|
||||
|
||||
## Development
|
||||
|
||||
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`.
|
||||
Use [.nvmrc](.nvmrc) and `package.json#packageManager` for the pinned toolchain. Cursor Cloud may place older Node.js first on `PATH`; environment setup installs the pinned toolchain under `/usr/local/bin`. Prefix `PATH="/usr/local/bin:$PATH"` before `node` and `npm`.
|
||||
|
||||
- `npm ci`: install locked dependencies.
|
||||
- `npm run build:cached`: build the runtime and packaged skill.
|
||||
|
||||
@@ -239,6 +239,14 @@ MinGW-w64, and `tar`; the build script does not run on Windows. See that guide
|
||||
for the build and artifact-verification commands and the separate real Windows
|
||||
verification prerequisites. Keep the generated addon and manifest uncommitted.
|
||||
|
||||
The general source suite (`npm run test:fast`, `npm run test:changed`, and
|
||||
`npm run test:local`) runs on Linux in CI, and Windows CI runs a curated lane.
|
||||
On a Windows checkout, some source tests fail because fixtures use POSIX paths,
|
||||
`#!/bin/sh` tool stubs, or POSIX rename semantics. Compare failures against the
|
||||
base revision, or run the suite on Linux or WSL, before attributing them to a
|
||||
change. See [testing](docs/testing.md#developer-commands) for command scopes
|
||||
and lane selection.
|
||||
|
||||
## Documentation website
|
||||
|
||||
The VitePress site uses the Markdown files in `docs/`. Run `npm run docs:dev`
|
||||
|
||||
@@ -191,6 +191,9 @@ Additional tools and host support depend on the target:
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](docs/pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, imports, lambdas and the Code header | Built-in static BEAM inspection; [Erlang BEAM](docs/erlang-beam.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
@@ -200,6 +203,7 @@ Additional tools and host support depend on the target:
|
||||
| Saved network captures | Requests, responses, exposed payloads and source locations | HAR or Fiddler Everywhere SAZ; mitmdump on Linux for native mitmproxy; [capture guide](docs/web-network-captures.md) |
|
||||
| .NET assemblies | Metadata, CIL instructions, declared native dependencies and build comparisons | None (static only); [managed-code guide](docs/managed-code-analysis.md) |
|
||||
| Android APKs | Manifest declarations, classes, decompiled methods and references | Headless JADX and a full JDK on Linux/macOS/Windows x64; [Android guide](docs/android-analysis.md) |
|
||||
| APK signatures | v1/v2/v3/v3.1 signers, certificate fingerprints, key rotation and digest self-consistency | None (static only); `inspect_apk_signature` |
|
||||
| Android devices | Devices, packages, processes, logs, dumps, screen captures, and APK/file transfers | Caller-supplied adb (emulator or hardware); [ADB guide](docs/adb-device-analysis.md) |
|
||||
| JEB projects | Engine-backed project units, artifact digests and type/method pseudo-code | A caller-started JEB client serving MCP, selected with REA_JEB_MCP_URL; [JEB guide](docs/jeb-analysis.md) |
|
||||
| Android resources | Decoded manifest, string tables, locales and version facts | Caller-supplied Apktool launcher; [resource guide](docs/apktool-resource-analysis.md) |
|
||||
|
||||
@@ -625,12 +625,14 @@ class _BridgeServer(threading.Thread):
|
||||
data = bytearray()
|
||||
deadline = time.monotonic() + AUTH_FRAME_TIMEOUT_SECONDS
|
||||
while b"\n" not in data:
|
||||
if len(data) >= MAX_FRAME_BYTES:
|
||||
if len(data) > MAX_FRAME_BYTES:
|
||||
raise ValueError("frame-limit-exceeded")
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0:
|
||||
raise socket.timeout("authentication frame timed out")
|
||||
connection.settimeout(remaining)
|
||||
# MAX_FRAME_BYTES limits the JSON payload; the terminating LF is
|
||||
# framing and may be the one byte beyond that payload budget.
|
||||
part = connection.recv(min(65536, MAX_FRAME_BYTES + 1 - len(data)))
|
||||
if not part:
|
||||
raise ValueError("incomplete authentication frame")
|
||||
|
||||
@@ -1,38 +1,91 @@
|
||||
import java.util.HashSet;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
|
||||
import com.google.gson.JsonArray;
|
||||
import com.google.gson.JsonObject;
|
||||
|
||||
import ghidra.app.plugin.core.analysis.AutoAnalysisManager;
|
||||
import ghidra.app.services.Analyzer;
|
||||
import ghidra.app.util.headless.HeadlessScript;
|
||||
import ghidra.program.model.listing.Program;
|
||||
import ghidra.util.classfinder.ClassSearcher;
|
||||
|
||||
/** Disable the named auto-analyzers before default auto-analysis. */
|
||||
/**
|
||||
* Disable the named auto-analyzers before default auto-analysis.
|
||||
*
|
||||
* REA_GHIDRA_DISABLED_ANALYZERS is process-wide, but Ghidra registers only the
|
||||
* installed analyzers whose canAnalyze(program) accepts this program. A name an
|
||||
* installed analyzer provides without being registered here is not applicable
|
||||
* and is recorded instead of rejected; a name no installed analyzer provides
|
||||
* aborts the import. The outcome is stored as a program property that the
|
||||
* bridge reports during its handshake.
|
||||
*/
|
||||
public final class ReaGhidraAnalysisOptions extends HeadlessScript {
|
||||
static final String REPORT_OPTION = "REA disabled analyzers";
|
||||
private static final String SETTING = "REA_GHIDRA_DISABLED_ANALYZERS";
|
||||
|
||||
@Override
|
||||
public void run() throws Exception {
|
||||
String[] arguments = getScriptArgs();
|
||||
if (arguments.length != 1 || arguments[0].isBlank()) {
|
||||
throw new IllegalArgumentException(
|
||||
"ReaGhidraAnalysisOptions requires one comma-separated analyzer-name argument");
|
||||
"ReaGhidraAnalysisOptions requires one comma-separated " + SETTING + " argument");
|
||||
}
|
||||
String[] names = arguments[0].split(",");
|
||||
boolean[] registered = new boolean[names.length];
|
||||
AutoAnalysisManager analysis = AutoAnalysisManager.getAnalysisManager(currentProgram);
|
||||
Map<String, String> options = getCurrentAnalysisOptionsAndValues(currentProgram);
|
||||
for (String requestedName : names) {
|
||||
Set<String> installed = null;
|
||||
for (int index = 0; index < names.length; index++) {
|
||||
monitor.checkCancelled();
|
||||
String name = requestedName.trim();
|
||||
if (analysis.getAnalyzer(name) == null || !options.containsKey(name)) {
|
||||
throw new IllegalArgumentException("Unknown Ghidra analyzer for this program: " + name);
|
||||
String name = names[index].trim();
|
||||
names[index] = name;
|
||||
if (analysis.getAnalyzer(name) != null) {
|
||||
if (!options.containsKey(name)) {
|
||||
throw new IllegalStateException(SETTING + " names " + name +
|
||||
", which Ghidra registered for this program without an enablement option");
|
||||
}
|
||||
registered[index] = true;
|
||||
continue;
|
||||
}
|
||||
if (installed == null) installed = installedAnalyzerNames();
|
||||
if (!installed.contains(name)) {
|
||||
throw new IllegalArgumentException(SETTING + " names " + name +
|
||||
", which is not an analyzer in this Ghidra installation");
|
||||
}
|
||||
}
|
||||
for (String requestedName : names) {
|
||||
for (int index = 0; index < names.length; index++) {
|
||||
monitor.checkCancelled();
|
||||
setAnalysisOption(currentProgram, requestedName.trim(), "false");
|
||||
if (registered[index]) setAnalysisOption(currentProgram, names[index], "false");
|
||||
}
|
||||
options = getCurrentAnalysisOptionsAndValues(currentProgram);
|
||||
for (String requestedName : names) {
|
||||
JsonArray analyzers = new JsonArray();
|
||||
for (int index = 0; index < names.length; index++) {
|
||||
monitor.checkCancelled();
|
||||
String name = requestedName.trim();
|
||||
if (!"false".equals(options.get(name))) {
|
||||
throw new IllegalStateException("Ghidra analyzer remained enabled: " + name);
|
||||
String name = names[index];
|
||||
if (registered[index] && !"false".equals(options.get(name))) {
|
||||
throw new IllegalStateException(SETTING + " names " + name +
|
||||
", which remained enabled for this program");
|
||||
}
|
||||
JsonObject analyzer = new JsonObject();
|
||||
analyzer.addProperty("name", name);
|
||||
analyzer.addProperty("status", registered[index] ? "disabled" : "not_applicable");
|
||||
analyzers.add(analyzer);
|
||||
}
|
||||
JsonObject report = new JsonObject();
|
||||
report.addProperty("format", "rea-ghidra-disabled-analyzers-v1");
|
||||
report.add("analyzers", analyzers);
|
||||
currentProgram.getOptions(Program.PROGRAM_INFO).setString(REPORT_OPTION, report.toString());
|
||||
println("REA disabled analyzers: " + report);
|
||||
}
|
||||
|
||||
/** The same analyzer universe AutoAnalysisManager filters by canAnalyze(program). */
|
||||
private static Set<String> installedAnalyzerNames() {
|
||||
Set<String> names = new HashSet<>();
|
||||
for (Analyzer analyzer : ClassSearcher.getInstances(Analyzer.class)) {
|
||||
names.add(analyzer.getName());
|
||||
}
|
||||
return names;
|
||||
}
|
||||
}
|
||||
@@ -523,6 +523,11 @@ public final class ReaGhidraBridge extends HeadlessScript {
|
||||
String seeds = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
.getString("REA analysis seeds", null);
|
||||
if (seeds != null) result.add("analysis_seeds", JsonParser.parseString(seeds));
|
||||
// Written by ReaGhidraAnalysisOptions before auto-analysis; REA checks it
|
||||
// against the committed profile.
|
||||
String disabledAnalyzers = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
.getString("REA disabled analyzers", null);
|
||||
if (disabledAnalyzers != null) result.add("disabled_analyzers", JsonParser.parseString(disabledAnalyzers));
|
||||
// Written by ReaGhidraMipsSoftFloat after analysis; REA requires it
|
||||
// exactly when the committed profile declares soft-float storage.
|
||||
String softFloat = currentProgram.getOptions(ghidra.program.model.listing.Program.PROGRAM_INFO)
|
||||
|
||||
+18
-1
@@ -14,6 +14,9 @@ from typing import Any, Optional, Protocol, Sequence
|
||||
BAD_ADDRESSES = (-1, 0xFFFFFFFFFFFFFFFF, None)
|
||||
# The native label setter truncates NSString names at this UTF-16 extent.
|
||||
HOPPER_SYMBOL_NAME_UTF16_UNITS = 1024
|
||||
# Maximum request JSON payload size in bytes, excluding its trailing LF.
|
||||
# Matches the Cutter bridge's MAX_FRAME_BYTES payload ceiling.
|
||||
MAX_REQUEST_BYTES = 16 * 1024 * 1024
|
||||
_rea_session_document = None
|
||||
_search_inventory_cache = {}
|
||||
_pseudocode_cache = {}
|
||||
@@ -1060,9 +1063,23 @@ def _serve_connection(connection):
|
||||
should_stop = False
|
||||
try:
|
||||
while True:
|
||||
line = file.readline()
|
||||
# Bound the read itself. The payload budget excludes LF, so permit
|
||||
# one additional byte for the delimiter while still refusing an
|
||||
# oversized unterminated peer without waiting for EOF.
|
||||
line = file.readline(MAX_REQUEST_BYTES + 1)
|
||||
if not line:
|
||||
break
|
||||
if not line.endswith(b"\n"):
|
||||
message = (
|
||||
"Bridge request exceeds the %d-byte payload limit"
|
||||
% MAX_REQUEST_BYTES
|
||||
if len(line) > MAX_REQUEST_BYTES
|
||||
else "Incomplete bridge request frame"
|
||||
)
|
||||
_write_message(file, {"id": 0, "error": _safe_diagnostic(
|
||||
InvalidRequestError(message)
|
||||
)})
|
||||
break
|
||||
request_id = None
|
||||
authenticated = False
|
||||
should_stop = False
|
||||
|
||||
@@ -32,6 +32,10 @@ export default defineConfig({
|
||||
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
|
||||
{ text: "Model artifacts", link: "/model-artifacts" },
|
||||
{ text: "Rust dependencies", link: "/rust-binaries" },
|
||||
{ text: "PE certificates", link: "/pe-certificates" },
|
||||
{ text: "ELF notes", link: "/elf-notes" },
|
||||
{ text: "CPython pyc files", link: "/python-pyc" },
|
||||
{ text: "Erlang BEAM", link: "/erlang-beam" },
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
link: "/javascript-artifact-reconstruction",
|
||||
|
||||
+161
-38
@@ -5,7 +5,8 @@ environment: an MCP server whose tools answer from what one retained web
|
||||
network capture observed, plus the environment card the agentenv framework
|
||||
reads to discover it. By default the server derives the
|
||||
[state the capture observed](#the-state-a-capture-observed) at startup and
|
||||
answers read tools from it; `--replay-only` replays the capture's records
|
||||
answers read tools from it, and the write tools whose effect the capture
|
||||
confirmed by changing it; `--replay-only` replays the capture's records
|
||||
instead. Nothing is fetched from the recorded application; every answer comes
|
||||
from retained Evidence. `rea environment-export` writes the same pair as a
|
||||
bundle folder `agent-env run` builds and runs; see
|
||||
@@ -271,11 +272,11 @@ document, with the same signal exit status.
|
||||
|
||||
Three routes are served:
|
||||
|
||||
| Route | Answer |
|
||||
| --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `POST /mcp` | The MCP server: from the state, one tool per read tool, answering from the state where it backs them; in replay, one tool per manifest tool, replaying records |
|
||||
| `GET /.well-known/agent-env.json` | The environment card |
|
||||
| `POST /agentenv` | From the state, `data/reset`, `data/add` and `data/get`; in replay, JSON-RPC `-32601 method not found: <method>` for every method |
|
||||
| Route | Answer |
|
||||
| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `POST /mcp` | The MCP server: from the state, one tool per read tool, answering from the state where it backs them, and one per write tool whose effect the capture confirmed, applying it to the state; in replay, one tool per manifest tool, replaying records |
|
||||
| `GET /.well-known/agent-env.json` | The environment card |
|
||||
| `POST /agentenv` | From the state, `data/reset`, `data/add` and `data/get`; in replay, JSON-RPC `-32601 method not found: <method>` for every method |
|
||||
|
||||
Numeric JSON-RPC IDs outside JavaScript's safe range or whose precision would
|
||||
be lost are refused as invalid requests with a null ID. They are never echoed
|
||||
@@ -376,8 +377,11 @@ In replay, `extensions`, `operations` and `children_environments` are fixed to
|
||||
`null`, `[]` and `null`: a replay environment declares no extensions, no
|
||||
data-plane operations and no children. Served
|
||||
[from the state](#serve-from-the-state), the card declares the three data-plane
|
||||
operations and one intake extension instead, leaves write tools out and
|
||||
describes each tool as answering from the state where the state backs it.
|
||||
operations and one intake extension instead, and the catalog holds the read
|
||||
tools and only the write tools whose effect the capture confirmed: a read tool
|
||||
is described as answering from the state where the state backs it, and a
|
||||
served write tool by the effect it applies to the state and the quoted body members it
|
||||
accepts.
|
||||
|
||||
## Serve from the state
|
||||
|
||||
@@ -400,7 +404,8 @@ A state that is unavailable does not stop the server: it logs a warning naming
|
||||
the error `code` and message before it binds, then serves replay. The state is
|
||||
unavailable when it cannot be derived, such as from a
|
||||
`capture_browser_scenario` capture that selected header values but no response
|
||||
bodies or over the view's budget, and when its catalog under the served name
|
||||
bodies or over the view's budget, when `data/get` would write it past the 32 MiB
|
||||
`data/add` intake limit, and when its catalog under the served name
|
||||
exceeds the [discovery reservation](#what-a-manifest-is). That check runs
|
||||
first, so no state is derived for it, and replay is then admitted exactly as
|
||||
with `--replay-only`. `--replay-only`
|
||||
@@ -417,26 +422,66 @@ member reports which applied:
|
||||
|
||||
Each read tool answers according to its view in the state:
|
||||
|
||||
| View | Answer |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold, or one lacking a field every recording carried, is a tool error; a body is never invented. |
|
||||
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"`, or the recorded members still held. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
|
||||
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
|
||||
| `recorded` | The recording, as replay answers it. |
|
||||
| View | Answer |
|
||||
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error, unless a read showed the application leaves that field out; a body is never invented. |
|
||||
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"` or a line `all_record_ordinals` names, or the recorded members still held. A call with no query argument answers from such a line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. On a line the view's `query` answers, and for a call whose every query argument has a role there (an argument-less call aside), the answer is the query evaluated over the state: filtered, searched, sorted and paged, with counters at the number listed and `query.totals` at the number matched, and `state.membership: "query"`; such a call answers on a query or whole line of its collection with `replay.exact_match` false. A value an `inert` parameter never recorded, or a parameter without a role, replays its recording. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
|
||||
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
|
||||
| `recorded` | The recording, as replay answers it. |
|
||||
|
||||
An item or list view that always spells a clock field otherwise than the
|
||||
entity's stored value names that spelling in `clock_formats`, and a value it
|
||||
never recorded, such as a created entity's clock, is rendered in it: a list that
|
||||
writes `+00:00` where item reads write `Z` lists a new entity the same way.
|
||||
|
||||
A list or document call that names no checked line, including a line recorded
|
||||
after a write or with other query values, answers from the recording. Every
|
||||
answer has the replay shape plus `state`: `{"source": "state", "view": …}`
|
||||
naming the entity, collection or document path, or `{"source": "recording",
|
||||
"reason": …}` with the view's reason, `no-view` or `unrecorded-request`. A
|
||||
"reason": …}` with the view's reason, `no-view`, `unrecorded-request`, `page-exceeded` (a call naming a page of a whole list that the collection has outgrown) or `projection` (a call to an item tool whose recordings asked for some fields only, naming such a parameter for an entity the state holds; the answer is a recording, so it does not show served writes). A
|
||||
state answer carries the base record's status, content type and `replay`
|
||||
provenance with `truncated: false`; one whose JSON would exceed 256 KiB is
|
||||
refused as a `resource_constraint` tool error rather than cut. Extraction does
|
||||
not state-back a document whose rendered JSON exceeds 256 KiB; that read stays
|
||||
recorded (`rendered-oversize`) and replays the captured body. Serving is
|
||||
read-only: `POST`, `PUT`, `PATCH` and `DELETE` tools, other than query `POST`s,
|
||||
are not served, so nothing a call does changes the state. Query `POST`s stay
|
||||
served and answer from their recordings with the reason `query-body`.
|
||||
recorded (`rendered-oversize`) and replays the captured body. Query `POST`s
|
||||
stay served and answer from their recordings with the reason `query-body`.
|
||||
|
||||
A line establishes whole-collection membership only from the beginning:
|
||||
`offset`, `start`, `_start` and `page` must be zero when present. Positive or
|
||||
malformed positions do not by themselves establish whole membership. A separate
|
||||
query model may describe supported paging; an empty later page cannot start
|
||||
returning earlier entities merely because it was empty in the capture.
|
||||
|
||||
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
|
||||
[write plan](web-network-captures.md#environment-state) for them is served:
|
||||
the capture's later reads confirmed the effect the plan models. Other write
|
||||
tools are not served, so no call claims a change it does not make; each one's
|
||||
plan states why in its `reason`. A served write changes the live state, which
|
||||
every MCP session and `data/get` then see:
|
||||
|
||||
| Effect | Call |
|
||||
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, increment, slug, unique and renewed changes the plan models. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
|
||||
Its answer has the status and content type of the plan's recorded response,
|
||||
with the written entity or its identity rendered in, no body, or the recorded
|
||||
body. A call whose body is not an object, sends a member the capture did not
|
||||
show stored, sends a fixed member with another value or a member of another
|
||||
JSON type, or lacks a required member is a tool error naming the accepted
|
||||
members; so is a rendered answer missing a field the recorded response
|
||||
carried, and an update or delete of an entity the state does not hold answers
|
||||
the recorded 400, 404 or 410 of the tool or of an item view of the type, if
|
||||
any. A refused call leaves the state unchanged. Values a write generates are
|
||||
deterministic: the clock starts at `write_clock` and advances one second per
|
||||
served write, sequences take one past both the largest held value and the
|
||||
largest they generated, and UUIDs and tokens derive from the state Evidence
|
||||
id, tool, field and write count, skipping held values. `data/reset` and
|
||||
`data/add` restart the clock, counter and sequences, so the same calls after a
|
||||
reset give the same state. Answers carry `state: {"source": "state", "view":
|
||||
"create" | "update" | "delete", …}` or `"document-write"`.
|
||||
|
||||
The data plane serves three JSON-RPC methods:
|
||||
|
||||
@@ -470,18 +515,29 @@ names its category in `error.data.reason`: `not-found`, `permission`,
|
||||
downloads, and `cancelled` for a download the server's stop or its caller's
|
||||
departure ended.
|
||||
|
||||
Served creates also bound additional collection history since the last reset
|
||||
or `data/add` to 1 MiB of deterministic accounting: UTF-8 bytes for each new
|
||||
collection spelling and its possible sequence keys, plus a 64-byte allowance
|
||||
per collection and potential sequence-key record. The score bounds identifier
|
||||
payload and a fixed record allowance; it is not a measurement or guarantee of
|
||||
process memory. Collections
|
||||
already present in the Evidence or loaded state are part of the state itself;
|
||||
the store indexes their unique spellings so they remain known after deletion,
|
||||
but they do not consume this additional-history budget. A create that would
|
||||
exceed the budget is refused before the state changes. `data/reset` or `data/add`
|
||||
clears generated history; if the same calls would hit the limit again, use
|
||||
fewer distinct collection paths. The 1 MiB score ceiling is 1/32 of the
|
||||
separate 32 MiB state-intake ceiling, reserving a small, explicit allowance
|
||||
for identifiers retained beyond the loaded state. A loaded-state path that is
|
||||
later deleted and then introduced again may consume the additional-history
|
||||
budget when the store must retain it beyond the source entity.
|
||||
|
||||
Stopping the server cancels every pending call, and a caller that leaves before
|
||||
its reply cancels its own: a download in progress ends, and calls not yet run
|
||||
are answered `-32000` with `error.data.reason` `cancelled` without running. A
|
||||
cancelled call leaves the state unchanged, and shutdown never waits for a
|
||||
download.
|
||||
|
||||
A `file://` state file whose close fails stays owned by the server, and
|
||||
the `data/add` gets `-32000`. The next `file://` load retries that close first
|
||||
and, while it still fails, answers `-32000` without reading another file;
|
||||
inline and `https://` documents still load. Closing the server retries it once
|
||||
more and fails with the unfinished close rather than dropping the descriptor.
|
||||
|
||||
One `/agentenv` response carries at most 64 MiB of replies, twice the 32 MiB
|
||||
a loaded document may hold: room for a `data/get` of a loaded state, while a
|
||||
batch of reads cannot multiply a large state into one body. Replies are
|
||||
@@ -565,14 +621,17 @@ The card declares the operations and what `data/add` accepts:
|
||||
}
|
||||
```
|
||||
|
||||
The state is a projection of recorded reads. List and document calls answer
|
||||
from the state only for the lines their views checked, and every recorded view
|
||||
answers from recordings; item calls answer for any held entity but render only
|
||||
the envelope one recording carried. Values copied outside entity objects,
|
||||
derived values and counters other than those that count a list stay as
|
||||
recorded; no write has an effect. An exported bundle carries no state file:
|
||||
its server derives the state at startup, as the export does to predict what it
|
||||
serves.
|
||||
The state starts as a projection of recorded reads and changes only through
|
||||
served write tools and the data plane. List and document calls answer from the
|
||||
state only for the lines their views checked, and every recorded view answers
|
||||
from recordings; item calls answer for any held entity but render only the
|
||||
envelope one recording carried. Values copied outside entity objects, derived
|
||||
values and counters other than those that count a list stay as recorded, and a
|
||||
served write changes only the one entity or document it writes: side effects
|
||||
on counters, relations and copies of entities in other collections, reordering
|
||||
after an update and referential rules on delete are not modelled. An exported
|
||||
bundle carries no state file: its server derives the state at startup, as the
|
||||
export does to predict what it serves.
|
||||
|
||||
## Export a bundle
|
||||
|
||||
@@ -640,8 +699,8 @@ name from Python sources the image does not have.
|
||||
|
||||
`verify.py` is the framework's `async def verify(mcp_url)` shape over its
|
||||
Python MCP client: it lists the tools and checks the count equals the number
|
||||
the bundle serves (the manifest's read tools from the state, every manifest
|
||||
tool in replay), calls one tool with no arguments and checks the call did not
|
||||
the bundle serves (from the state, the manifest's read tools and the write
|
||||
tools whose effect the capture confirmed; every manifest tool in replay), calls one tool with no arguments and checks the call did not
|
||||
fail and its `status` is the expected one, and returns one `serve` result
|
||||
whose description reports both checks. The tool is the first served `GET` tool
|
||||
without path parameters whose answer to no arguments is not an error, and the
|
||||
@@ -659,13 +718,77 @@ The README written into the bundle records the capture artifact's SHA-256, the
|
||||
capture and manifest Evidence ids, the manifest view digest, the served tool
|
||||
count, the derived state's Evidence id or that no state is served, and the
|
||||
package source. It says what the environment serves: read tools from the state
|
||||
the capture observed, the `/agentenv` data plane on it and no write tools yet,
|
||||
replay when the state is unavailable, and `--replay-only` for pure replay; it
|
||||
the capture observed, the `/agentenv` data plane on it and the write tools
|
||||
whose effect the capture's later reads confirmed, replay when the state is
|
||||
unavailable, and `--replay-only` for pure replay; it
|
||||
says which applies to this bundle, with the error code and message when the
|
||||
state is unavailable, and that neither is emulation and served
|
||||
content is untrusted data. It states that the recorded content belongs to the
|
||||
captured application and must not be published without the rights to it.
|
||||
|
||||
## Measure fidelity against the application
|
||||
|
||||
An environment agrees with its own capture by construction. What matters to an agent is how it answers the calls the
|
||||
capture did not make, what it shows after a write, and how it fails. `rea environment-compare` measures that against the
|
||||
application itself:
|
||||
|
||||
```sh
|
||||
rea environment-compare manifest.json --evidence capture.json \
|
||||
--probe probe-crud.json --probe probe-reads.json --json
|
||||
```
|
||||
|
||||
Each `--probe` is capture Evidence (`inspect-web-network-capture` of a HAR, or `capture-browser-scenario`) of one call
|
||||
sequence made against the live application, seeded as it was for the capture and reset before the sequence. The command
|
||||
makes no network call: whoever records the probes drives the application with its own credentials. For each probe it
|
||||
resets the environment's state, takes the requests in the order they were made (record order when any lacks a start time), matches each to the manifest tool of the
|
||||
same method whose path template fits with the most literal segments (a format-rule parameter must fit its format once
|
||||
percent-decoded). A path argument is the raw segment, which the environment decodes once.
|
||||
It answers the call as the served environment would, and compares that answer with the application's.
|
||||
|
||||
A re-seeded application holds the same entities under other ids, so the comparison maps them. Objects in the two answers
|
||||
to one call pair by a natural key, a string field unique on each side that both share, such as a name or a slug; their
|
||||
identifier fields (`id`, `uuid`, `guid`, and names ending in `_id`, `Id`, `ID`, `_uuid` or `Uuid`), and those of two single
|
||||
answers such as a create's, map the live value to the environment's, including a create that returns only the new id.
|
||||
Ids are learned only from an answer the environment
|
||||
gave that call itself, from its state or a recording of that very call, request line and body, never from another call's
|
||||
recording replayed in its place and never from a recording whose body the probe omitted. The answer's own `id`, `uuid` or `guid` also maps within the collection the call names, since
|
||||
two collections can reuse a live id, and an equal pair stays in that collection. A live id in a later request's path, query or body is translated before the
|
||||
environment answers, a path parameter by the collection the live path names (a nested collection keeps its parent values, each decoded once, and a literal route segment stays), except a query value that pages a list, such as `limit`
|
||||
or `page`. Each call is checked against the served tool's input schema first, as the MCP server checks it. After mapping, two clock values compare by format only (ISO-8601 strings, and epoch numbers under
|
||||
time-named fields), and two different values each side generated on its own compare equal: both UUIDs, or one length of
|
||||
eight or more with the same character classes and a digit. The report counts each kind of normalised value.
|
||||
|
||||
A step agrees when the statuses and the normalised bodies are equal. The report gives agreement overall, per probe and
|
||||
per tool, `agreement_ignoring_order` for steps that differ only in item order, and each step's differences, at most 20,
|
||||
with a JSON pointer and both values cut short:
|
||||
|
||||
| Category | Meaning |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| `status` | The statuses differ |
|
||||
| `missing-field` | The application's object has a member the environment's lacks |
|
||||
| `extra-field` | The environment's object has a member the application's lacks |
|
||||
| `value` | Values differ, or one is an object or array and the other is not |
|
||||
| `length` | Arrays differ in length, or not every item pairs |
|
||||
| `order` | Paired items come in a different order |
|
||||
| `clock-format` | Both are clocks, in different formats |
|
||||
| `refused` | The environment refused the call |
|
||||
| `not-served` | The request's tool is one the environment does not offer, such as a withheld write |
|
||||
| `no-tool` | No manifest tool matches the request |
|
||||
| `unmapped-id` | The application answered with success and the environment did not, for a path id the comparison never mapped |
|
||||
|
||||
What a probe did not observe is not counted as the application's behaviour. A record without a usable request line,
|
||||
such as a redacted URL or a transaction the capture cannot attribute, is not replayed, nor is a request to another
|
||||
origin than the application's (`other-origin`: the probe's origin with the most JSON-like records, as the manifest
|
||||
chooses its own), and a request with no recorded
|
||||
answer is made, since it may have changed the application, but not compared: each probe lists them under `unobserved` with the reason, and the summary counts them. A live
|
||||
body the probe did not record compares by status only and is counted as `unobserved_bodies`, unless the answer never
|
||||
carries one (204, 205, 304, `HEAD`). A browser probe captured without `capture.network.response_body` is refused, as
|
||||
is one that makes POST, PUT or PATCH calls without `capture.network.request_body`.
|
||||
|
||||
Agreement covers only the calls the probes make, after the mapping the answers allow. Values compared by form, clocks
|
||||
and generated values, can still differ in content, and an equal answer after a write shows the state matched for that
|
||||
read, not that the write had no other effect.
|
||||
|
||||
## Compatibility verification
|
||||
|
||||
`npm run verify:agentenv:environment` is the hosted `real-agentenv` lane
|
||||
|
||||
@@ -0,0 +1,263 @@
|
||||
# APK signature schemes
|
||||
|
||||
REA reads the signing of a selected local Android APK (or ZIP/AAB) statically,
|
||||
so an analyst can answer **who signed this, under which schemes, and whether the
|
||||
declared content digests match these bytes** before trusting or triaging it. The reader
|
||||
is in-process and needs no external engine: it parses the ZIP tail and the APK
|
||||
Signing Block, reads certificates with the platform X.509 parser, and
|
||||
recomputes the content digest. No Android code executes, and no device or
|
||||
`adb` is involved.
|
||||
|
||||
CLI and MCP share one workflow and return artifact-bound Evidence inline.
|
||||
|
||||
```sh
|
||||
rea inspect-apk-signature /path/app.apk
|
||||
```
|
||||
|
||||
## Selected facts and retained Evidence
|
||||
|
||||
MCP defaults to a summary with scheme presence, signer certificate fingerprints,
|
||||
row counts, posture and explicit coverage. `detail: "complete"` returns the full
|
||||
report. A selected page can answer a narrower question in the acquisition call:
|
||||
|
||||
```json
|
||||
{
|
||||
"path": "/path/app.apk",
|
||||
"view": {
|
||||
"collection": "signers",
|
||||
"schemes": ["v3"],
|
||||
"fields": ["certificate_sha256"],
|
||||
"limit": 32
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The returned `parent_evidence_id` identifies the retained complete report.
|
||||
Subsequent reads use `inspect_analysis_view`, including after the APK is deleted:
|
||||
|
||||
```json
|
||||
{
|
||||
"source": {
|
||||
"kind": "retained-evidence",
|
||||
"evidence_id": "<returned parent_evidence_id>"
|
||||
},
|
||||
"view": {
|
||||
"kind": "apk-signature",
|
||||
"collection": "digests",
|
||||
"signer_ids": ["<returned signer id>"],
|
||||
"fields": ["signature_algorithm_id", "algorithm", "matches_apk"],
|
||||
"offset": 0,
|
||||
"limit": 32
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Collections are `signers`, `digests` and declared `lineage` nodes. `ids`,
|
||||
`signer_ids` and `schemes` filters intersect; omitted filters select all available
|
||||
records and `[]` selects none. Omitted `fields` returns every collection column;
|
||||
`fields: []` retains identity and relationship links. IDs include the complete
|
||||
APK digest, scheme and source ordinal; child records retain `signer_id` and
|
||||
`signer_index`, and declared certificates retain their raw-DER SHA-256 identity.
|
||||
Lineage pages expose `previous_id` when selected. Certificate fingerprints can
|
||||
be composed across versions without treating a declaration as verified identity.
|
||||
|
||||
Page coverage describes examined declared records. Exhausting a page does not
|
||||
establish certificate authenticity, platform acceptance, v1 entry coverage or a
|
||||
verified rotation. Partial decoding and its reason remain visible even when a
|
||||
lineage page contains no rows. A weak cache reuses validated immutable report
|
||||
facts; each read checks its parent subject, and new file inspections read again.
|
||||
|
||||
CLI defaults to complete delivery. A compact one-shot response requires a complete
|
||||
Evidence export:
|
||||
|
||||
```sh
|
||||
rea inspect-apk-signature /path/app.apk --detail summary \
|
||||
--view '{"collection":"signers","fields":["certificate_sha256"]}' \
|
||||
--evidence_output /path/app-signing-evidence.json --json
|
||||
```
|
||||
|
||||
The export contains the same complete parent used by MCP. Historical CLI pages
|
||||
use `inspect-analysis-view` with an `evidence-file` source record. The CLI and MCP
|
||||
share the application workflow, projection, retention and publication owners.
|
||||
|
||||
## What it reports
|
||||
|
||||
- **Schemes present:** v1 (JAR signing, from the central directory), and the
|
||||
APK Signing Block's v2, v3 and v3.1. v1 follows apksig's selection: entries
|
||||
under `META-INF/` at any depth ending in `.SF`, or in `.RSA`/`.DSA`/`.EC`
|
||||
(case-sensitive). v1 is `present` only when a signature block has its
|
||||
matching `.SF` file (`META-INF/x/CERT.RSA` pairs with `META-INF/x/CERT.SF`);
|
||||
`signature_files` and `block_files` list every candidate either way.
|
||||
- **Per v1 signer** (each block paired with its `.SF`): REA inflates the block
|
||||
and reads its PKCS#7 `SignedData`. `certificate` is the certificate the
|
||||
block's one `SignerInfo` names by `issuerAndSerialNumber` or
|
||||
`subjectKeyIdentifier` (not by position), with the same facts and raw-DER
|
||||
SHA-256 as a v2/v3 signer, so a v1-only APK can be fingerprinted and
|
||||
clustered with v2/v3-signed ones. apksig resolves only
|
||||
`issuerAndSerialNumber`, so its v1 verification fails for a block whose
|
||||
`SignerInfo` uses `subjectKeyIdentifier`: REA still reports the declared
|
||||
certificate (`identified_by: subject-key-identifier`), adds a limitation,
|
||||
and treats that signer as unknown in `v1_signers_match_block_signers`. `certificates` lists every certificate the
|
||||
block carries (SHA-256 and subject) and `signer_infos` gives each
|
||||
`SignerInfo`'s certificate and digest/signature algorithm OIDs.
|
||||
`certificate` is `null` when the block has more than one `SignerInfo` (with
|
||||
nothing verified there is no single signer) or names a certificate it does
|
||||
not carry. A block that cannot be read (malformed DER, an unsupported
|
||||
compression method, Zip64 sizes, a local header that disagrees with the
|
||||
central directory on the name, data-descriptor flag, CRC-32 or a size, as
|
||||
apksig refuses it, or over a bound) keeps its names and carries
|
||||
`decode_error`; it is never reported as an absent signer. apksig ignores
|
||||
the ZIP encryption flag (bit 0), so a block that sets it is read as stored:
|
||||
the PKCS#7 decoding decides (bytes that really are encrypted fail to
|
||||
decode), and a limitation names the block.
|
||||
apksig rejects the v1 signature of an archive whose central directory
|
||||
holds any name more than once ("Duplicate entry"). REA checks every entry:
|
||||
a limitation names the first few duplicates (JSON-quoted; a name over 256
|
||||
bytes is cut there and its stored length given) and counts the rest, and a
|
||||
signer whose block, `.SF` or `META-INF/MANIFEST.MF` name is duplicated
|
||||
(one signer per block name) carries `decode_error`
|
||||
`duplicate entry: <name>`, so a duplicated manifest marks every signer.
|
||||
A duplicate outside those names leaves the signers as declared.
|
||||
Names are compared as apksig compares them, as Java strings decoded from
|
||||
UTF-8: malformed bytes become U+FFFD, so `dup\xff` and `dup\xfe` are one
|
||||
name (apksigner: `Duplicate entry: dup�`). REA reproduces Java's
|
||||
replacement rather than WHATWG `TextDecoder`'s; the two differ for encoded
|
||||
surrogates (`ED A0..BF`), which Java replaces as one unit. This was checked
|
||||
against OpenJDK 25's `new String(bytes, UTF_8)` on about 300,000 malformed
|
||||
and well-formed sequences, including a leading byte-order mark, which Java
|
||||
keeps; other Java runtimes are not checked. The same
|
||||
decoding names v1 entries and compares a local header's name with the
|
||||
central directory's; as in apksig, a local name longer in bytes than the
|
||||
directory's is refused before decoding. apksig takes the record count from the classic
|
||||
end-of-central-directory record's 16-bit field and ignores a Zip64 count,
|
||||
so it reads only the first 65,535 records of a larger directory: the
|
||||
limitation then says so, counts as duplicates for apksig only names whose
|
||||
second copy lies within those records, and names separately those
|
||||
duplicated only past them. REA still reads v1 signing entries, and flags
|
||||
duplicated signing names, from every record, so past record 65,535 it
|
||||
reports signers apksig would not see.
|
||||
Bytes after the block's `ContentInfo` are ignored, as apksig's BER parser
|
||||
ignores them, and a limitation names the block and their count.
|
||||
- **Per signer:** the certificate's **SHA-256 fingerprint over the raw embedded
|
||||
DER** (the same value `apksigner --print-certs` prints), plus subject, issuer,
|
||||
serial, ISO 8601 validity and public-key algorithm when the X.509 parser can
|
||||
read them (each is `null` otherwise); raw signature algorithm IDs, declared
|
||||
digest hex and digest algorithms, signature lengths and public-key length; and for
|
||||
v3/v3.1 the signer's declared min/max SDK.
|
||||
- **Key rotation (v3/v3.1):** the declared proof-of-rotation lineage, oldest
|
||||
first. Each node carries its certificate and the capability flags **retained
|
||||
by that node's own certificate** after rotation (installed data, shared UID,
|
||||
permission, rollback, auth). A lineage attribute that cannot be decoded is
|
||||
reported with `decode_error`, never silently emptied.
|
||||
- **Source stamp:** whether a source-stamp block is present and the stamp
|
||||
certificate it declares (the V2 stamp block is preferred over V1, as apksig
|
||||
does). `certificate` is `null` when no stamp is present, or when the block
|
||||
could not be decoded and `decode_error` says why. Stamp signatures are not
|
||||
verified.
|
||||
- **Declared digest consistency:** for SHA-256/SHA-512 digests, REA recomputes
|
||||
the v2/v3 content digest and reports `matches_apk` per digest and
|
||||
`posture.self_consistent`. This only shows that the **declared** digest
|
||||
equals the digest of the current bytes. It is **not a signature
|
||||
verification**: anyone who changes the bytes can recompute the digest and
|
||||
write it into the signed data, and REA still reports `true` while
|
||||
`apksigner verify` rejects the APK. A changed byte with an unchanged
|
||||
declared digest reads `false`.
|
||||
- **Posture** (derived from the facts above):
|
||||
- `v1_only`: v1 is present and no v2/v3/v3.1 block is. The v1
|
||||
`X-Android-APK-Signed` marker is not read, so this cannot distinguish an
|
||||
APK signed only with v1 from one whose v2/v3 block was removed.
|
||||
- `schemes_agree_on_signer` (observed): every present block scheme presents
|
||||
the same _set_ of signer certificates. An extra signer in one scheme is
|
||||
disagreement; a signer without a certificate makes it `null` (unknown).
|
||||
Build-tools 33+ rotate through v3.1 by default, so a rotated app usually
|
||||
reads `false` here and `true` below.
|
||||
- `signer_change_declared_by_rotation` (declared): the signers differ, but
|
||||
each appears in a lineage the APK declares. Read the two together: a
|
||||
rotated APK shows `false` / `true`; signers that differ with no declared
|
||||
lineage containing them show `false` / `false`. It is `null` when a
|
||||
signer's identity is unknown, or when the signers differ and a declared
|
||||
lineage could not be decoded.
|
||||
- `v1_signers_match_block_signers` (observed): the v1 signer certificates
|
||||
equal what the lowest block scheme that has signers (v2, else v3, else
|
||||
v3.1) expects v1 to carry. That is its signer set, except that a v3/v3.1
|
||||
signer declaring a rotation lineage expects the lineage's first
|
||||
(original) certificate, as apksig requires. `false` is notable: the JAR
|
||||
signature names a different key than the block signature. A key rotation
|
||||
keeps v1 (and v2) on the original key, so it reads `true` while
|
||||
`schemes_agree_on_signer` reads `false`, with or without v2. `null` when
|
||||
v1 or a block scheme is absent, any signer's identity is unknown (a
|
||||
`decode_error`, a `null` certificate or a `subjectKeyIdentifier`
|
||||
`SignerInfo`), or the lineage's original certificate cannot be
|
||||
identified; a limitation then says why.
|
||||
- `strongest_scheme`, `rotation_present`, `source_stamp_present`,
|
||||
`self_consistent`.
|
||||
|
||||
## What it does not claim
|
||||
|
||||
This is **declared identity and digest consistency, not trust**. No signature
|
||||
is cryptographically verified (including the v1 PKCS#7 signature and the
|
||||
`.SF`/`MANIFEST.MF` digests, so a v1 signer is reported even where `keytool`
|
||||
or `apksigner` reject the archive), certificate chains and revocation are not
|
||||
evaluated, and nothing here says an app is authentic. `self_consistent: true`
|
||||
means the declared digest equals the digest of the current bytes; it is not a
|
||||
signature verification, and it does not prove who made the bytes. Rotation-lineage node signatures are not verified either, so a lineage is
|
||||
what the APK _declares_: an attacker who re-signs an APK can declare any
|
||||
lineage, which is why `signer_change_declared_by_rotation` is kept apart from
|
||||
the observed agreement. SHA-256 identifies exact bytes, not authenticity.
|
||||
|
||||
Not yet read: v4 `.apk.idsig` (a sibling file), verity (`0x0421`/`0x0423`/`0x0425`) digests (reported with
|
||||
`matches_apk: null`), and the v1 `X-Android-APK-Signed` marker.
|
||||
|
||||
## Bounds and errors
|
||||
|
||||
The archive is read once, bounded at 512 MiB, without following symlinks and
|
||||
with change detection. Every size and offset in the ZIP tail and signing block
|
||||
is checked against its containing section before use, including the complete
|
||||
central-directory extent before ZIP end records. A process-wide queue permits
|
||||
one active whole-file inspection and eight pending scans across provider
|
||||
instances; pending scans retain no APK bytes and can be cancelled with their
|
||||
original reason. Further requests receive a capacity error. The active slot
|
||||
is held through parsing and cleanup. Signers per scheme, v1 signers and
|
||||
lineage nodes are capped at 32 each as resource admission limits, not format
|
||||
maxima (apksig separately accepts at most 10 v1 signers). Each v1 block is inflated to at most 1 MiB, and its DER
|
||||
is read within 32 levels of nesting and 65,536 elements, with at most 32
|
||||
certificates and 32 `SignerInfo`s; BER indefinite lengths are accepted. The report's size is
|
||||
bounded from above by those counts before anything is materialised, and the exact
|
||||
serialized size of the report together with its limitations (which repeat
|
||||
archive names) is then checked against an 8 MiB output budget (the exact check
|
||||
is authoritative); an over-budget report is refused, not truncated.
|
||||
|
||||
The whole-file reader retains read chunks and a concatenated buffer; total
|
||||
process memory exceeds the selected file's size. It also computes the complete
|
||||
file SHA-256, so inspection is not header-only. Content digests are recomputed
|
||||
at most once per SHA-256/SHA-512 algorithm during one inspection. Input,
|
||||
inflation, decoded-record and serialized-output bounds keep work finite;
|
||||
resource refusals are distinct from malformed format errors.
|
||||
|
||||
A file that is not a ZIP is refused as an unsupported target. A damaged ZIP
|
||||
(local headers but no end-of-central-directory), a central directory that
|
||||
cannot be walked or whose declared record count does not account for all of
|
||||
its bytes, or a malformed signing block or signer is an input error naming the
|
||||
scheme and reason. Signer framing includes signature records, the public-key
|
||||
field and matching signed/outer v3 SDK ranges. Suffix bytes beyond known
|
||||
fields are counted as uninterpreted, as Android readers may ignore extensions;
|
||||
their presence does not establish malformed signing.
|
||||
When the central directory is not immediately followed by
|
||||
the end-of-central-directory record (a declared-size mismatch or a Zip64
|
||||
record), the content digest is not recomputed and a limitation says so. A
|
||||
rotation attribute that cannot be decoded still counts as `rotation_present`
|
||||
(the attribute exists) and carries its `decode_error`.
|
||||
|
||||
Verified by the `verify:apk-signature` real lane. It builds a real minimal APK
|
||||
with `aapt2`, signs it with the SDK's `apksigner` (default v2+v3, v1-only with
|
||||
RSA and EC keys, v1+v2+v3 on a legacy target, a rotated key lineage with and
|
||||
without v1, rotation carried by v3.1, a source stamp, and a copy
|
||||
with one changed byte), and cross-checks REA against
|
||||
`apksigner verify --print-certs`, `apksigner lineage --print-certs` and
|
||||
`keytool -printcert -jarfile` through the CLI and MCP.
|
||||
|
||||
The real-toolchain CI lane uses pinned, checksum-verified Build Tools 36.1.0
|
||||
and platform 36 oracle files in its job workspace. It requires the existing
|
||||
host JDK and runs `npm run verify:apk-signature -- --require-tools`; absent
|
||||
oracles fail rather than silently establishing provider coverage. REA itself
|
||||
requires none of these external tools.
|
||||
@@ -29,6 +29,7 @@ graph TD
|
||||
PROVIDER_OWNERS["src/providers/<br/>shared deferred client and inspection owners<br/>exact cleanup retry and attributed failures"]
|
||||
PWNTOOLS["src/native/pwntools/ + src/native/pwndbg/<br/>BYO offline ELF/core adapters<br/>owned snapshots/Python and optional core-only GDB<br/>unchanged upstream parsers, ABI structures and mapping API"]
|
||||
WASM["src/wasm/<br/>caller-supplied WABT; immutable artifact inspection<br/>owned process supervision and retryable workspace cleanup"]
|
||||
APKSIG["src/apkSignature/<br/>in-process APK Signing Block + v1 reader<br/>bounded single read, no device or execution"]
|
||||
EVM["src/evm/<br/>unchanged EVMole/WASM in bounded owned worker<br/>explicit carrier parsing and inferred interface adapter"]
|
||||
HOPPER["src/hopper/<br/>Hopper launch & Unix-socket protocol<br/>BridgeLauncher, HopperClient, protocol"]
|
||||
GHIDRA["src/ghidra/<br/>Ghidra discovery, profile & function analysis<br/>snapshot, Unix/Windows transport, bounded lifecycle"]
|
||||
@@ -101,6 +102,8 @@ graph TD
|
||||
SERVER -->|narrow record ports| RECORDS
|
||||
RECORDS --> DOMAIN
|
||||
COMPOSITION --> WASM
|
||||
COMPOSITION --> APKSIG
|
||||
APKSIG --> DOMAIN
|
||||
WASM --> PROCESS
|
||||
WASM --> DOMAIN
|
||||
COMPOSITION --> PWNTOOLS
|
||||
|
||||
@@ -215,3 +215,136 @@ Large Evidence envelopes may require a caller-configured MCP transport receive
|
||||
buffer because envelope copies add overhead. Requests honor cancellation during
|
||||
snapshot reading, directory traversal and payload hashing. The operation reads
|
||||
regular files only and rejects symbolic links.
|
||||
|
||||
## Byte entropy triage
|
||||
|
||||
`inspect_byte_entropy` / `inspect-byte-entropy` profiles any local regular file
|
||||
without an analysis engine, execution, extraction or network access.
|
||||
|
||||
```sh
|
||||
rea inspect-byte-entropy ./firmware.bin --block-size 4096 --json
|
||||
rea inspect-byte-entropy ./firmware.bin --offset 4096 --length 65536 --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_byte_entropy",
|
||||
"arguments": { "path": "/artifacts/firmware.bin", "detail": "summary" }
|
||||
}
|
||||
```
|
||||
|
||||
Complete Evidence includes `artifact` (original path, byte size and SHA-256),
|
||||
`block_size`, `window`, `file_entropy`, `thresholds`, `blocks`, `runs` and
|
||||
`limitations`. For a 4096-byte zero-filled file, the entropy fields are:
|
||||
|
||||
```json
|
||||
{
|
||||
"block_size": 4096,
|
||||
"window": { "offset": 0, "length": 4096 },
|
||||
"file_entropy": 0,
|
||||
"thresholds": { "high": 7.2, "low": 1 },
|
||||
"blocks": [{ "offset": 0, "length": 4096, "entropy": 0 }],
|
||||
"runs": [{ "offset": 0, "length": 4096, "classification": "low" }]
|
||||
}
|
||||
```
|
||||
|
||||
Offsets and lengths are decimal byte counts. Blocks begin at the requested
|
||||
`offset` (default zero); omitted `length` extends to EOF. Explicit windows past
|
||||
EOF fail rather than silently clipping. The final block may be shorter.
|
||||
`block_size` must be a power of two from 256 through 65536, default 4096. A
|
||||
window may produce at most 32,768 block rows; larger requests fail before the
|
||||
file scan with a capacity error. Choose a shorter window or larger block size.
|
||||
SHA-256 and `file_entropy` always cover the entire file, including when a window
|
||||
is selected. Empty files and empty windows are accepted; empty entropy is zero
|
||||
by convention. The tool reads the size the file reports. A file that reports size
|
||||
0, such as a Linux `/proc` entry, returns no blocks with a limitation that the
|
||||
result does not show the file is empty. File identity, size and timestamps are checked across the read;
|
||||
detected changes produce an error rather than Evidence.
|
||||
|
||||
Shannon entropy is reported in bits per byte from 0 to 8, rounded to four decimal
|
||||
places. Adjacent blocks with reported entropy at least 7.2 form `high` runs;
|
||||
blocks at most 1.0 form `low` runs. Other blocks interrupt runs. Entropy does not
|
||||
prove encryption, and this measure cannot distinguish encrypted from compressed
|
||||
data. Small blocks can underestimate entropy; the result is a triage clue.
|
||||
|
||||
MCP `detail: "summary"` retains complete Evidence in the session and returns
|
||||
file entropy, block/run counts, thresholds, the selected window, and the ten
|
||||
longest runs (ties ordered by file offset). Use its `parent_evidence_id` to
|
||||
retrieve complete Evidence, or request its summary with `inspect_analysis_view`.
|
||||
The CLI returns complete Evidence through the same service. Summary detail
|
||||
requires a session capable of retaining Evidence.
|
||||
|
||||
Reads use one block-sized buffer and a 256-entry histogram. Complete output
|
||||
can become very large because it includes one row per selected block, and
|
||||
summary delivery still retains that full list. Use a shorter length or a
|
||||
larger block size to reduce it. Full Evidence and its serialization require
|
||||
memory proportional to the number of selected blocks. The 32,768-row ceiling bounds the actual retained block objects and
|
||||
their serialized representation: a cap-sized run below produced 1,514,877
|
||||
serialized bytes and increased V8 heap use by 21.2 MiB on the measured host.
|
||||
The cap therefore bounds this output independently of file size; it is not a
|
||||
portable process-memory guarantee. Narrow the window or increase block size to
|
||||
reduce retained rows further.
|
||||
|
||||
Local service measurement (macOS arm64, Node 24.18.0, 100,000,000-byte file,
|
||||
4096-byte blocks, file already in the OS cache): first process run took 1.113 s
|
||||
wall / 1.148 s CPU with 145.3 MiB peak RSS; a second run in the same process took
|
||||
1.015 s / 1.035 s CPU, with cumulative peak RSS 155.8 MiB. Each read 100,000,000
|
||||
bytes and produced 24,415 blocks. Measurements include Evidence construction,
|
||||
summary projection and JSON serialization, with no child processes. Complete
|
||||
Evidence serialized to 1,122,303 bytes and its summary to 2,705 bytes. A
|
||||
cap-sized 134,217,728-byte sparse file with 4096-byte blocks produced 32,768
|
||||
rows and 1,514,877 serialized bytes; V8 heap use increased by 21.2 MiB and RSS
|
||||
by 18.1 MiB in that process. This separate measurement used the same macOS
|
||||
arm64 host and Node version. Cold disk I/O, token counts and an agent
|
||||
investigation were not measured; there was no previous equivalent tool to
|
||||
benchmark. These figures do not establish a portable peak-memory bound,
|
||||
constant memory for retained Evidence or faster agent task completion.
|
||||
|
||||
## Portable native exports
|
||||
|
||||
`inspect_native_exports` / `inspect-native-exports` reads the export table of an
|
||||
explicit PE or ELF image on any supported host, without an active target,
|
||||
native engine, `objdump` or Windows SDK:
|
||||
|
||||
```sh
|
||||
rea inspect-native-exports ./runtimes/linux-x64/native/libexample.so --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_native_exports",
|
||||
"arguments": { "path": "/artifacts/native/example.dll" }
|
||||
}
|
||||
```
|
||||
|
||||
The complete Evidence identifies the stable artifact snapshot by path, byte
|
||||
count and SHA-256.
|
||||
|
||||
- PE rows reproduce the export directory: each used export-address-table slot
|
||||
with its `ordinal`, one row per name (aliases share an ordinal), `address`
|
||||
as the preferred-base virtual address, or `forwarder` text when the slot
|
||||
forwards to another DLL. Ordinal-only exports have a null `name`. The
|
||||
directory's DLL name, ordinal base and timestamp are reported, and an image
|
||||
without an export directory is a complete empty inventory.
|
||||
- ELF rows list defined `.dynsym` symbols with global, weak or GNU-unique
|
||||
binding and default or protected visibility. Each row keeps `value`
|
||||
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`
|
||||
and `binding`. `version` is null for an unversioned symbol, the
|
||||
`.gnu.version_d` index, name and `hidden` flag for a versioned one, or
|
||||
`"unknown"` when the version table cannot be read. A hidden version is not
|
||||
the default version that a by-name `dlsym` lookup selects.
|
||||
|
||||
Unreadable names, out-of-range name ordinals and unreadable forwarders are
|
||||
listed in `issues`, their entries are not reported, and coverage is `partial`.
|
||||
Unresolved symbol versions keep their rows: the version name is null for an
|
||||
index no definition names, and `version` is `"unknown"` when the version table
|
||||
cannot be read; coverage is likewise `partial`. A dynamic symbol table without
|
||||
its null entry is malformed. An ELF without section headers is `unsupported`:
|
||||
enumerating dynamic symbols from the dynamic segment is not
|
||||
implemented. Mach-O exports are reported by `inspect_macho`.
|
||||
|
||||
`max_file_bytes` defaults to 256 MiB (maximum 512 MiB; CLI `--max-file-bytes`).
|
||||
Export directories in PE headers are read like those in sections. An export or
|
||||
dynamic symbol table above 262144 entries, more than 262144 rows, or more than
|
||||
16 MiB of retained names fails without partial success. Export rows do not establish that a symbol is reachable, safe to call
|
||||
or loaded at runtime.
|
||||
@@ -22,6 +22,13 @@ Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate
|
||||
|
||||
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
|
||||
|
||||
For a compact route query, select `detail: "summary"` and a
|
||||
`view: {kind: "javascript-routes", collection: "routes", limit: 20}`. The
|
||||
shared [route inventory](javascript-route-inventory.md) preserves source IDs,
|
||||
precise locations, partial coverage, and the complete parent Evidence for
|
||||
subsequent filtered pages. Static candidates never establish complete application
|
||||
coverage or reachability.
|
||||
|
||||
For `analyze_web_bundle`, each `webmcp_declarations` entry's
|
||||
`schema_property_names` is a source-derived projection of statically known own
|
||||
property keys, not schema validation. `schema_property_names_coverage` reports
|
||||
@@ -333,7 +340,7 @@ REA retains local observation data and removes only URL userinfo credentials, ex
|
||||
- DOM snapshots retain node types, node names, value lengths, and attribute names, but not text or attribute values.
|
||||
- Accessibility structure and roles are retained by default, while names and descriptions require `include_accessibility_text: true`.
|
||||
- Network observations retain method, status, MIME type, size, type, initiator stack location, and complete URLs with userinfo credentials removed. Redirects reported by CDP are attached as ordered `redirects` on the one request ID, with the prior request URL, CDP response URL, method/type, response status/MIME/encoded length, request timestamp, and the next `requestWillBeSent` event timestamp. CDP delivers `redirectResponse` with that next request event, so the latter is an event boundary rather than an exact response timestamp. Redirect response headers and bodies are discarded; hops outside the selected origins are discarded with the request chain. Headers are discarded after an allowlisted projection of length/encoding, structured CSP/Link/policy fields, and untrusted agent hints. Cookies and authorization headers are never retained.
|
||||
- Request/response bodies are not requested or parsed by default. When selected, only allowed-origin JSON media types are read, converted immediately to complete value-free typed paths, types and observation counts, and discarded. Each path segment is either `{ "kind": "property", "name": "raw property name" }` or `{ "kind": "array-element" }`; array-element types include primitives and nested arrays. Literal `*` properties remain distinct from array elements. `Network.getResponseBody` is never sent unless JSON body shape capture is selected. Comparison uses body shapes only when both captures selected them, and compares request/response shapes only where both observations provide those facts. Unavailable or truncated selected coverage makes equivalence unknown; observed differences still report changed.
|
||||
- Request/response bodies are not requested or parsed by default. When selected, only allowed-origin JSON media types are read, converted immediately to complete value-free typed paths, types and observation counts, and discarded. Each path segment is either `{ "kind": "property", "name": "raw property name" }` or `{ "kind": "array-element" }`; array-element types include primitives and nested arrays. Literal `*` properties remain distinct from array elements. `Network.getResponseBody` is never sent unless JSON body shape capture is selected. A JSON-typed request body that CDP reports through `hasPostData` but omits from `postData`, or that cannot be parsed as JSON, makes `json_body_shapes` unavailable. That request's `body_shapes.status` is `partial`, not `included`, when only its response shape was captured. Comparison uses body shapes only when both captures selected them, and compares request/response shapes only where both observations provide those facts. Unavailable or truncated selected coverage makes equivalence unknown; observed differences still report changed.
|
||||
For normalized passive inputs with complete request inventories on both sides,
|
||||
request additions and removals remain observable even when selected body shapes,
|
||||
response metadata, or WebSocket coverage are unavailable. HTTP request and
|
||||
|
||||
@@ -387,6 +387,24 @@ observed) and excludes the sequence reference from normalization.
|
||||
Playwright scenario capture does not supply initiator stacks; passive CDP
|
||||
observation retains its own initiator evidence.
|
||||
|
||||
## Storage value fingerprints
|
||||
|
||||
New storage snapshots include `fingerprint_version: 2`. Non-secret, well-formed
|
||||
JavaScript strings retain their SHA-256 over UTF-8. Strings with unpaired UTF-16
|
||||
surrogates use SHA-256 over a raw `0xff` byte followed by their exact UTF-16LE
|
||||
code units; this prefix cannot occur in UTF-8. Declared secrets remain
|
||||
`redacted-secret` with a null digest, without a second or legacy digest.
|
||||
|
||||
A missing version identifies legacy UTF-8 fingerprints, which replaced unpaired
|
||||
surrogates and cannot establish value identity. Storage containing any legacy
|
||||
hashed value is conservatively compared as `unknown`, including old-to-old
|
||||
comparisons; recapture both sides to compare reliably. This also conservatively
|
||||
withholds some otherwise provable legacy differences. Empty or entirely
|
||||
secret-redacted storage keeps its existing comparison semantics. The version is
|
||||
checked before normalization and excluded from content comparison: changing
|
||||
encoding metadata alone does not establish a website change. Existing captures
|
||||
and their completeness records are not rewritten.
|
||||
|
||||
## Scenario comparison
|
||||
|
||||
`compare_web_captures` and `compare-web-captures` also accept two complete
|
||||
@@ -419,6 +437,15 @@ sorted canonically before application, making the same policy reproducible
|
||||
regardless of input order. Rules operate only on already-redacted durable
|
||||
capture fields and are preserved in the result.
|
||||
|
||||
Cookies, local storage, and session storage are compared as unordered inventories
|
||||
after caller literal rules apply. Enumeration order alone does not establish a
|
||||
storage change. Canonical ordering uses the complete entry, preserving cookie
|
||||
scope/flags, value fingerprints, and duplicate entries even when redaction or
|
||||
literal rules make names collide. Captured storage arrays retain their original
|
||||
order. The `compare_storage_entries_as_unordered_collection` built-in rule
|
||||
participates in the normalization commitment; new storage comparison digests
|
||||
therefore differ from earlier order-sensitive comparisons.
|
||||
|
||||
Each aligned step reports changed, unchanged, or unknown. Changed and unknown
|
||||
action, screenshot, DOM, accessibility, URL, history, storage, and event
|
||||
artifacts carry their normalized before/after digests and capture states.
|
||||
|
||||
+61
-1
@@ -155,6 +155,49 @@ dependency edges, and the section's file range. A missing `.dep-v0` section is
|
||||
null metadata, not proof that the binary has no dependencies. See
|
||||
[Rust binaries](rust-binaries.md).
|
||||
|
||||
Inspect a PE Authenticode certificate table without executing the image or
|
||||
verifying signatures:
|
||||
|
||||
```bash
|
||||
rea inspect-pe-certificates /absolute/path/to/application.exe --json
|
||||
```
|
||||
|
||||
The workflow reports `WIN_CERTIFICATE` entries and, for PKCS#7 signedData,
|
||||
embedded X.509 serial, issuer, subject, and validity fields. A missing security
|
||||
directory is observed absence of that table. See
|
||||
[PE Authenticode certificate tables](pe-certificates.md).
|
||||
|
||||
Inspect ELF PT_NOTE and SHT_NOTE tables without executing the image:
|
||||
|
||||
```bash
|
||||
rea inspect-elf-notes /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The workflow reports note owners, types, descriptor hashes, and GNU build-id
|
||||
or ABI-tag fields when those descriptors match. A missing note table is
|
||||
observed absence of those declarations. See [ELF note tables](elf-notes.md).
|
||||
|
||||
Inspect a CPython 3.7–3.13 `.pyc` header and nested marshal code objects
|
||||
without executing bytecode or installing Python:
|
||||
|
||||
```bash
|
||||
rea inspect-python-pyc /absolute/path/to/module.pyc --json
|
||||
```
|
||||
|
||||
The workflow reports the pinned CPython version for the magic, PEP 552 hash or
|
||||
mtime/size fields, nested function names, and bytecode SHA-256. Bytecode is
|
||||
not decompiled. See [CPython pyc inspection](python-pyc.md).
|
||||
|
||||
Inspect an Erlang BEAM file without executing bytecode:
|
||||
|
||||
```bash
|
||||
rea inspect-erlang-beam /absolute/path/to/app.beam --json
|
||||
```
|
||||
|
||||
The workflow reports IFF chunk names, atom tables, exports, imports, lambdas,
|
||||
and the Code chunk header. Missing optional tables stay null. See
|
||||
[Erlang BEAM files](erlang-beam.md).
|
||||
|
||||
Configure [Hopper or Ghidra](installation.md#hopper), the
|
||||
[IDA adapter](ida-provider.md), or [Kuna](kuna-provider.md) before analyzing a
|
||||
native target. Substitute
|
||||
@@ -414,7 +457,8 @@ evidence file's `evidence_id`; otherwise the command refuses with
|
||||
`invalid_request` before binding. Before binding it derives the
|
||||
`environment-state` view of the capture with the manifest's `template_rules`,
|
||||
as `inspect-analysis-view` would, and serves from it: read tools whose views
|
||||
are state-backed answer from a live state, write tools are not served, and
|
||||
are state-backed answer from a live state, write tools are served only where
|
||||
the capture's later reads confirmed their effect, and
|
||||
`/agentenv` serves `data/reset`, `data/add` and `data/get`. When the state
|
||||
cannot be derived, or its catalog exceeds the discovery reservation, it logs a
|
||||
warning with the error code and message and serves replay; `--replay-only` serves replay without deriving. In replay every
|
||||
@@ -470,6 +514,22 @@ of the serve command's stopped document. See
|
||||
[agent environments](agent-environments.md#export-a-bundle) for the folder
|
||||
layout and what the smoke task checks.
|
||||
|
||||
## Compare an environment with its application
|
||||
|
||||
`rea environment-compare <manifest.json> --evidence <capture.json> --probe <probe.json> [--probe <probe.json> ...]`
|
||||
compares the environment the pair serves with probe captures of the application it came from. Each probe is capture
|
||||
Evidence of one call sequence recorded against the application after a reset; the environment's state is reset before
|
||||
each probe, every request to the application's origin is answered as the served environment answers it, and the
|
||||
answer is compared with the application's after ids are mapped and clocks and generated values are normalised. The command reads files only and
|
||||
makes no network call. It prints the report and exits `0`, or prints a structured `invalid_request` error and exits `1`
|
||||
for the same pair refusals as `environment-serve`, a probe that is not capture Evidence, or a browser probe captured
|
||||
without response bodies, or without the request bodies its writes need. See
|
||||
[agent environments](agent-environments.md#measure-fidelity-against-the-application) for the report.
|
||||
|
||||
```sh
|
||||
rea environment-compare manifest.json --evidence capture.json --probe probe.json --json
|
||||
```
|
||||
|
||||
## Output and exit status
|
||||
|
||||
`rea mcp doctor --json` checks production MCP startup and reports child logs in
|
||||
|
||||
@@ -73,12 +73,17 @@ to rerun it.
|
||||
A process that the MSIX app started, such as a terminal in Claude Desktop's
|
||||
Code tab, sees the package's private files at their `%APPDATA%` paths even
|
||||
though it reports no package identity. Setup compares one of those files at
|
||||
both paths. When it finds that redirected view, the app is running, so setup
|
||||
and uninstall refuse to change Claude Desktop's configuration; rerun them from
|
||||
a terminal that Claude Desktop did not start. If the package has no private
|
||||
file to compare, setup and uninstall cannot rule that out and also refuse; open
|
||||
Claude Desktop once if it has never run, quit it fully, then rerun. Doctor
|
||||
reads the same file from either kind of process.
|
||||
both paths: first one from the package's private `Claude` folder, and when
|
||||
that folder has none (some installations keep writing the real
|
||||
`%APPDATA%\Claude`), one from the package's other private roaming folders,
|
||||
which the package redirects in the same way. When it finds that redirected
|
||||
view, the app is running, so setup and uninstall refuse to change Claude
|
||||
Desktop's configuration; rerun them from a terminal that Claude Desktop did not
|
||||
start. If the package keeps no private roaming file at all, setup and
|
||||
uninstall cannot rule that out and also refuse. With Claude Desktop fully
|
||||
quit, add REA's entry by hand to the file setup names, which is the one the
|
||||
app reads, or remove it from both the private copy and the `%APPDATA%` file.
|
||||
Doctor reads the same file from either kind of process.
|
||||
|
||||
The same process also writes everything under `%APPDATA%` and
|
||||
`%LOCALAPPDATA%` into the package's private storage, where other apps never
|
||||
@@ -249,7 +254,19 @@ registration supports `startup_timeout_sec` for connection initialization and
|
||||
startup timeout and a tool timeout derived from the known provider
|
||||
allowance plus 120 seconds of operation headroom. This yields 450 seconds for
|
||||
the default Ghidra allowance; an explicit Ghidra startup timeout of 2700
|
||||
seconds yields 2820 seconds. Setup preserves positive caller-selected timeout
|
||||
seconds yields 2820 seconds. That 30-second Codex figure is client guidance.
|
||||
REA's own phase budgets are `REA_MCP_INITIALIZE_BUDGET_MS` (default 15000),
|
||||
`REA_MCP_TOOLS_LIST_BUDGET_MS` (default 5000),
|
||||
`REA_MCP_FIRST_CATALOG_BUDGET_MS` (default 20000), and
|
||||
`REA_MCP_DOCTOR_DEADLINE_MS` (default 30000). Each is a positive decimal
|
||||
integer of milliseconds from 1 to 2147483647. Unset keeps the default.
|
||||
Invalid values fail configuration and name the setting. Setup copies a
|
||||
caller-selected value into the registration, and a change is read from the
|
||||
environment that starts REA, so it needs a restart. Doctor and packaged
|
||||
startup probes name the effective milliseconds, whether they came from the
|
||||
default or the setting, the default, the floor, and the maximum. REA does
|
||||
not enforce Codex's 30 seconds as the doctor deadline. The 120000 ms
|
||||
tool-operation headroom and the 64 KiB doctor stderr cap stay fixed. Setup preserves positive caller-selected timeout
|
||||
values in the REA entry, and doctor reports a non-blocking advisory when a tool
|
||||
timeout is shorter than the known provider allowance. An explicitly selected
|
||||
JEB request budget also contributes to the allowance. A configured IDA
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
# ELF note tables
|
||||
|
||||
`inspect_elf_notes` / `inspect-elf-notes` answers whether one selected local
|
||||
ELF32/ELF64 image declares `PT_NOTE` program headers or `SHT_NOTE` sections,
|
||||
which note owners and types those tables store, and which GNU build-id or
|
||||
ABI-tag fields those notes carry. It reads the file without executing it and
|
||||
does not require a disassembler, debug package, or package catalog.
|
||||
|
||||
```bash
|
||||
rea inspect-elf-notes /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The CLI resolves relative paths against the caller's working directory. MCP
|
||||
requires an absolute filesystem path and preserves that exact selection:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_elf_notes",
|
||||
"arguments": { "path": "/artifacts/program" }
|
||||
}
|
||||
```
|
||||
|
||||
## Observations
|
||||
|
||||
The inline Evidence result contains the artifact path, SHA-256, and byte length.
|
||||
`tables` lists each declared note program header or section, including the
|
||||
section name when a string table is present. Each `entries[]` item reports owner,
|
||||
type, descriptor SHA-256, and a hex preview for descriptors of 64 bytes or less.
|
||||
|
||||
For GNU notes, `gnu.build_id_hex`, `gnu.abi_tag`, or `gnu.gold_version` are
|
||||
filled when the descriptor matches that type. Other owners keep `gnu` null.
|
||||
|
||||
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
|
||||
appear twice when both tables declare it.
|
||||
|
||||
A missing table list is observed absence of those declarations. It is not proof
|
||||
the image never carried notes in another container or overlay.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
32 note entries are examined by default across every table. Retained note
|
||||
names, section names, and GNU descriptor text share that file's byte budget,
|
||||
and repeated table ranges count again. Each descriptor range is hashed once,
|
||||
and each GNU version's trailing NUL run is scanned once; neither walk may
|
||||
exceed that byte ceiling. Malformed ELF headers, truncated header
|
||||
tables, declared note tables that extend past the file, and truncated
|
||||
note entries fail without partial success. ELF extended
|
||||
numbering (`PN_XNUM`, including a section count stored when `e_shnum` is 0) is
|
||||
unsupported.
|
||||
|
||||
This workflow does not execute the image or verify GNU build-id uniqueness
|
||||
against package catalogs.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Erlang BEAM files
|
||||
|
||||
`inspect_erlang_beam` / `inspect-erlang-beam` answers whether one selected
|
||||
local file is a `FOR1`/`BEAM` IFF container, which chunks that form stores, and
|
||||
which atom, export, import, lambda, and Code-header fields those chunks carry.
|
||||
It reads the file without executing bytecode and does not require an Erlang
|
||||
installation or BEAM emulator.
|
||||
|
||||
```bash
|
||||
rea inspect-erlang-beam /absolute/path/to/app.beam --json
|
||||
```
|
||||
|
||||
The CLI resolves relative paths against the caller's working directory. MCP
|
||||
requires an absolute filesystem path and preserves that exact selection:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_erlang_beam",
|
||||
"arguments": { "path": "/artifacts/app.beam" }
|
||||
}
|
||||
```
|
||||
|
||||
## Observations
|
||||
|
||||
The inline Evidence result contains the artifact path, SHA-256, and byte length.
|
||||
`chunks` lists every IFF chunk name, file range, and payload SHA-256.
|
||||
|
||||
When `AtU8` or `Atom` is present, `atoms` is the 1-based name table and
|
||||
`module` is atom 1. Both the pre-OTP 28 table and the OTP 28+ compact table
|
||||
(negated count, compact-encoded name lengths) are read. `exports`, `imports`, and `lambdas` resolve atom indices to
|
||||
those names when the table is present; unresolved indices keep `function` or
|
||||
`module` null.
|
||||
|
||||
`code` reports the Code chunk instruction-set, opcode maximum, label count,
|
||||
function count, and the instruction stream's file range. Missing optional
|
||||
tables are null, not empty-everywhere.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
64 chunks and 4096 atoms are examined by default, and at most 65536 export,
|
||||
import, and lambda rows together. Malformed `FOR1`/`BEAM` headers, truncated
|
||||
chunks, and `AtU8` names that are not valid UTF-8 fail without partial success.
|
||||
|
||||
This workflow does not execute bytecode, inflate `LitT` literals, or decode
|
||||
`Attr`/`CInf` external-term payloads.
|
||||
+13
-13
@@ -47,19 +47,19 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
|
||||
|
||||
## Choose a guide
|
||||
|
||||
| Task | Guides |
|
||||
| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
|
||||
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
|
||||
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
|
||||
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [.NET](managed-code-analysis.md), [Python packages](python-packaged-applications.md), [Android](android-analysis.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
|
||||
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
|
||||
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
|
||||
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
|
||||
| Capture and compare a command | [Process capture](process-capture.md) |
|
||||
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
|
||||
| Task | Guides |
|
||||
| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [CPython pyc files](python-pyc.md), [PE certificates](pe-certificates.md), [ELF notes](elf-notes.md), [Erlang BEAM](erlang-beam.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
|
||||
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
|
||||
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
|
||||
| Inspect assemblies, packages, or firmware | [Godot PCK](godot-pck.md), [.NET](managed-code-analysis.md), [Python packages](python-packaged-applications.md), [Android](android-analysis.md), [APK signatures](apk-signature.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
|
||||
| Observe or compare a browser scenario | [CDP observation](browser-observation.md), [scenarios](browser-scenario-contract.md), [runtime evidence](web-runtime.md), [historical captures](web-network-captures.md) |
|
||||
| Serve recorded behavior as an agent environment | [Agent environments](agent-environments.md) |
|
||||
| Trace captured website source | [Source locations](web-source-location.md), [module traces](website-module-trace.md), [script export](website-script-export.md) |
|
||||
| Capture and compare a command | [Process capture](process-capture.md) |
|
||||
| Use CLI/MCP and retain evidence | [CLI](cli.md), [MCP contracts](mcp-contracts.md), [agent prompts](mcp-prompts.md), [obligation ledgers](reconstruction-obligation-ledgers.md) |
|
||||
|
||||
For development, use [testing](testing.md), [client verification](client-verification.md),
|
||||
[tool design](tool-design.md), and [releasing](releasing.md).
|
||||
|
||||
+36
-17
@@ -104,9 +104,11 @@ your investigation. Failed steps and diagnostics remain in terminal history.
|
||||
|
||||
Select exact clients in scripts with repeatable `--client` flags. Each explicit
|
||||
client skips interactive selection. Use `--all-detected` only when you intend
|
||||
to configure every detected supported client. `--skill=false` omits the
|
||||
workflow, and `--dry-run` returns a read-only plan with status `planned` and
|
||||
exit code 0:
|
||||
to configure every detected supported client. Combining `--client` with
|
||||
`--all-detected` configures both the explicitly selected and detected clients,
|
||||
even when an explicit client is not detected. Each client is configured once.
|
||||
`--skill=false` omits the workflow, and `--dry-run` returns a read-only plan
|
||||
with status `planned` and exit code 0:
|
||||
|
||||
```bash
|
||||
rea setup --client codex --client cursor --skill=false --dry-run
|
||||
@@ -540,8 +542,10 @@ Hopper is separate commercial software with its own license. Its free demo has
|
||||
vendor-defined limits, and a paid license is optional. REA reuses any detected
|
||||
installation and preserves Hopper during uninstall.
|
||||
|
||||
The automated Hopper installer supports macOS 12+, Ubuntu 24.04+, Fedora 41+,
|
||||
Nobara 44+, 64-bit Arch Linux, CachyOS, or Omarchy. REA itself runs on any
|
||||
The automated Hopper installer supports macOS 12+ and x86_64 hosts running
|
||||
Ubuntu 24.04+, Fedora 41+, Nobara 44+, Arch Linux, CachyOS, or Omarchy. The
|
||||
official Linux packages are x86_64-only, so setup does not offer Hopper on other
|
||||
Linux architectures; configure [Ghidra](#ghidra) there instead. REA itself runs on any
|
||||
Linux distribution with a recognized apt, dnf, or pacman package family, where
|
||||
an existing Hopper launcher is selected through `HOPPER_LAUNCHER_PATH`. Ghidra
|
||||
and IDA have their own provider-specific host requirements; Windows Ghidra uses
|
||||
@@ -923,7 +927,17 @@ It also checks that the reported JVM `os.arch` and `os.name` map to the same nat
|
||||
directory validated for the host; unrecognized reported architectures or a
|
||||
non-host operating system are refused. LaunchSupport overrides of either
|
||||
property, including through helper-environment JVM option variables, are
|
||||
rejected at launch.
|
||||
rejected at launch. `JDK_JAVA_OPTIONS` argument files (`@file`) are expanded
|
||||
and inspected with that same check using the Java launcher's argfile quoting,
|
||||
comments and continuation rules. The reader admits regular UTF-8 files, retains
|
||||
failed descriptor cleanup, and bounds all admitted file contents and the prepared
|
||||
environment to 64 KiB per launch. Unreadable, oversized or nested `@` references
|
||||
are refused before the analysis JVM starts. Validated bytes are snapshotted under
|
||||
the owned runtime root, so later helpers and the analysis JVM do not reopen mutable
|
||||
caller files. Repeated references share that launch's snapshot; no live state is
|
||||
cached across launches. `JAVA_TOOL_OPTIONS`
|
||||
and `_JAVA_OPTIONS` do not expand `@file` tokens, so they remain
|
||||
literal-token checks.
|
||||
When Java is found through `PATH`, setup records its observed JDK home so GUI
|
||||
MCP clients do not depend on an incidental shell path. Setup shows every exact
|
||||
environment entry in its plan, writes only after approval, and never downloads,
|
||||
@@ -1115,17 +1129,22 @@ override in `analyzer_preset` (for example
|
||||
open when its committed profile and the setting differ. A name starts with a
|
||||
letter or digit and contains only letters, digits, spaces, periods, underscores,
|
||||
and hyphens; at most 32 names are admitted, each at most 120 characters.
|
||||
Names must identify analyzers registered for the selected program. Unknown names
|
||||
or options that remain enabled abort the headless import before auto-analysis
|
||||
and the bridge, so an unsuccessful override cannot produce a session claiming
|
||||
that the analyzer was disabled. The setting applies to every import in the REA
|
||||
server. A format-specific analyzer may not be registered for another target, so
|
||||
the same override can prevent that target from opening. For mixed-target work,
|
||||
omit the override or use a separate REA server configured for the affected
|
||||
format. Changing the environment requires restarting the server; reopening a
|
||||
target alone keeps the current setting. For Windows PE resource analysis, the
|
||||
exact name is `WindowsResourceReference`. Disabling an analyzer skips the facts
|
||||
it would have produced.
|
||||
|
||||
The setting applies to every import in the REA server, but Ghidra registers
|
||||
format-specific analyzers, such as `Objective-C Message Analyzer` or
|
||||
`Demangler Swift`, only for programs they can analyze. A name that the Ghidra
|
||||
installation provides but did not register for the selected program is not
|
||||
applicable: the import continues, the profile still commits the configured
|
||||
names, and result limitations report the applied preset with that name marked,
|
||||
for example `ghidra-default+disabled:Objective-C Message Analyzer(n/a)`. A name
|
||||
that no installed analyzer provides, or an option that remains enabled, aborts
|
||||
the headless import before auto-analysis and the bridge with an error naming
|
||||
`REA_GHIDRA_DISABLED_ANALYZERS` and the analyzer, so a misspelled override
|
||||
cannot produce a session claiming that the analyzer was disabled. Changing the
|
||||
environment requires restarting the server; reopening a target alone keeps the
|
||||
current setting. For Windows PE resource analysis, the exact name is
|
||||
`WindowsResourceReference`. Disabling an analyzer skips the facts it would have
|
||||
produced.
|
||||
|
||||
### Ghidra heap and CPU controls
|
||||
|
||||
|
||||
@@ -80,8 +80,8 @@ Graph coverage describes the analyzed artifact scope and resource limits, not
|
||||
runtime execution. A partial target alone does not imply omitted source.
|
||||
|
||||
The directed relation vocabulary is `contains`, `loads`, `imports`, `maps_to`,
|
||||
`exposes`, `sends`, `invokes`, `handles`, `calls`, `persists_to`, `observed_as`,
|
||||
and `changed_from`. Provider names are deliberately absent from both
|
||||
`exposes`, `sends`, `invokes`, `handles`, `calls`, `accesses`, `persists_to`,
|
||||
`observed_as`, and `changed_from`. Provider names are deliberately absent from both
|
||||
vocabularies.
|
||||
|
||||
For a `maps_to` edge targeting a source-map node, the edge carries
|
||||
|
||||
@@ -183,6 +183,11 @@ registrations/removals and dispatch, Node timers and cancellation handles, async
|
||||
`node:child_process` creation and ownership, configuration sources and direct
|
||||
defaults, request construction and response consumers, parse/coercion/
|
||||
validation boundaries, and built-in resource acquisition/release.
|
||||
Global timer, `fetch`, `WebSocket`, `JSON.parse`, and coercion calls receive
|
||||
these built-in effects only when the file has no visible possible write to that
|
||||
global, including a computed write through a global object. The check does not
|
||||
evaluate order or reachability, so a later or guarded polyfill write also
|
||||
removes the resolved effect.
|
||||
|
||||
Local callback candidates do not establish a complete target set when a
|
||||
conditional write may leave a parameter, catch binding, or uninitialized
|
||||
|
||||
@@ -392,6 +392,11 @@ observation, because callers outside the traced paths may pass other values.
|
||||
A wrapper call that is both its own entry and reached through another entry
|
||||
is recorded once when both paths find the same facts; calls that supply
|
||||
different handlers or channels stay distinct.
|
||||
A contextBridge exposure reached through wrapper calls records one API per key
|
||||
those invocations supply, each with its own bounded member paths. Invocations
|
||||
that supply the same key to one exposure call describe one API covering each
|
||||
alternative object. A traced invocation whose key, API object or world stays
|
||||
unresolved or guarded keeps the call's own unresolved observation.
|
||||
Unknown spreads, reassigned bindings, accessors,
|
||||
dynamic channels and unsupported syntax do not become exact implementations.
|
||||
This pass neither executes the bundle nor proves that runtime guards pass.
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
# Static JavaScript route inventory
|
||||
|
||||
`analyze_javascript_application` and `analyze_web_bundle` retain a shared
|
||||
`route_inventory` alongside their existing graph and endpoint observations.
|
||||
The inventory answers which route patterns the supplied code suggests, where
|
||||
each pattern came from, and which parts remain unknown. It never requests a
|
||||
route, executes scripts, expands a sitemap, or establishes reachability.
|
||||
|
||||
Select a compact view in either MCP analysis call:
|
||||
|
||||
```json
|
||||
{
|
||||
"detail": "summary",
|
||||
"view": {
|
||||
"kind": "javascript-routes",
|
||||
"collection": "routes",
|
||||
"where": { "path_prefix": "/reports" },
|
||||
"fields": ["path", "classification", "confidence"],
|
||||
"limit": 20
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Keep the original analysis inputs in that call. For the CLI, pass the same
|
||||
selection as `--view '<JSON>' --detail summary --evidence_output routes.json`.
|
||||
The complete Evidence bundle is written without overwriting existing files.
|
||||
Unselected delivery retains the existing complete result. A web-bundle summary
|
||||
requires an explicit route view; application summaries also support the existing
|
||||
general summary when `view` is omitted.
|
||||
|
||||
Use the returned `normalized_result.parent_evidence_id` with
|
||||
`inspect_analysis_view` on the same connection. The CLI's exported bundle can be
|
||||
reopened with `inspect-evidence-bundle`; its `records` contain the complete
|
||||
Evidence. To use `inspect_analysis_view`'s `evidence-file` source, extract the
|
||||
selected parent record into a separate local Evidence JSON file (that source
|
||||
accepts one Evidence record, not a bundle). Views do not reopen source files or
|
||||
reconnect to a browser. Historical Evidence without `route_inventory` is
|
||||
unmeasured, rather than an empty route set.
|
||||
|
||||
Collections are `summary`, `routes`, `occurrences`, `sources`, and `unknowns`.
|
||||
All row collections support exact batch `where.ids`, field selection, `offset`,
|
||||
and `limit`. An explicit empty filter array matches nothing. Routes also support
|
||||
exact `paths`, literal `path_prefix`, `classifications`, `mechanisms`, and
|
||||
`min_confidence`; occurrences support `route_ids`, `source_ids`, and `mechanisms`;
|
||||
sources support `keys` and `statuses`; unknowns support `source_ids` and `kinds`.
|
||||
Follow `coverage.next_offset` until `coverage.exhausted`; a page may stop early
|
||||
at the 1 MiB serialized view budget. Oversized indivisible facts return a
|
||||
resource error carrying the retained parent Evidence ID.
|
||||
|
||||
Pattern IDs bind the exact path. Source IDs bind the source key, path, original
|
||||
URL, content digest and container digest. Occurrences identify a detector mechanism at a source location; one location
|
||||
may support multiple mechanisms. `source_count` counts distinct sources. Lookup IDs use 128 bits of SHA-256;
|
||||
full content and inventory digests remain SHA-256. Field selection always keeps
|
||||
row identity and necessary provenance. The page's deduplicated `sources` table
|
||||
contains the source descriptors referenced by returned rows. Retrieve all
|
||||
occurrences by route ID when independent supporting evidence matters. Source
|
||||
ranges use one-based lines and zero-based UTF-16 columns with exclusive ends;
|
||||
filename-only discoveries have no invented AST range.
|
||||
|
||||
The shared parser handles literal router calls, nested route objects, typed
|
||||
`_$` trees with explicitly inferred property-name fallbacks, Next App Router
|
||||
page paths, and navigation targets. It follows stable lexical bindings and
|
||||
keeps unresolved parents, mutations, spreads, and dynamic fields explicit.
|
||||
A selected route-builder root owns its configuration even when mutation prevents
|
||||
trusting its value. Ordinary wrapper keys do not become route ancestors. Missing
|
||||
outer ancestors of unbound typed subtrees remain unknown rather than invented.
|
||||
Generic Next runtime path constants require a top-level application segment
|
||||
supported by structural or navigation candidates across the supplied sources.
|
||||
This limits framework filesystem noise, while leaving unsupported candidates
|
||||
unknown. The parser shares the existing AST traversal and lexical index.
|
||||
|
||||
[Next dynamic segments](https://nextjs.org/docs/app/api-reference/file-conventions/dynamic-routes)
|
||||
map to `:id`, `:slug+`, or `:slug*`; groups and parallel slots do not contribute
|
||||
URL segments. Intercepted routes remain unresolved. A prefix before `/_next/`
|
||||
is an **inferred** routing base: [assetPrefix](https://nextjs.org/docs/app/api-reference/config/next-config-js/assetPrefix)
|
||||
can differ from the routing base. Every affected occurrence keeps
|
||||
`unprefixed_path`, `inferred_base_path`, and low confidence. `system` classification
|
||||
is explicitly a path-name heuristic; `page` uses page-file or structured/typed configuration support. Name-only
|
||||
`route`/`addRoute` calls remain `unknown` because they can also register server APIs.
|
||||
|
||||
Known capture omissions remain inline in `summary.reported_capture_omissions`
|
||||
and queryable as `unknowns` of kind `capture-omission`. Each keeps a stable gap ID,
|
||||
its script-record/script-source unit, and the reported count (including `null`
|
||||
for unknown). They do not invent missing script identities or source ranges.
|
||||
|
||||
Counts describe deduplicated patterns in the provided code sources. They are
|
||||
not a denominator for complete application or capture coverage. In particular,
|
||||
server-only, lazy, inaccessible, or unselected sources may be missing.
|
||||
`coverage.application_routes` always remains `unknown`; parsing and candidate
|
||||
enumeration have their own completeness. Static work and fact admission share
|
||||
the existing 8 MiB per-source transfer ceiling, and the root inventory has an
|
||||
8 MiB admission budget. Exhaustion preserves a prefix with explicit capacity;
|
||||
the serialization owner also admits route facts within the actual space left by
|
||||
legacy facts. `result-bytes` capacity preserves an earlier work-capacity reason;
|
||||
if even that envelope cannot fit, the optional route facet remains unmeasured.
|
||||
These are accounting bounds, not a claim that total process memory is 8 MiB.
|
||||
|
||||
Projection only reduces delivery, not analysis work. Retained immutable Evidence
|
||||
can reuse validated tables through a weak cache with the snapshot's lifetime;
|
||||
subject identity is checked on every query. No live state or source content is
|
||||
cached as current. Use existing source selection and resource controls when
|
||||
they match the investigation, keeping their partial coverage visible.
|
||||
@@ -57,7 +57,11 @@ are complete Evidence objects, not only evidence IDs:
|
||||
`static_layers` requires exactly one `application` role and permits additional
|
||||
`cache` and `assets` roles. A `file-root` mapping translates a canonical runtime
|
||||
filesystem root to an artifact-relative prefix. A `url-prefix` mapping does the
|
||||
same for one exact HTTP(S) origin and path prefix. Mappings cannot add CDP
|
||||
same for one exact HTTP(S) origin and path prefix. Each URL path segment below
|
||||
the prefix is percent-decoded separately: an encoded slash (`%2F`) stays inside
|
||||
its segment and never names a nested artifact, so such a location, like one
|
||||
with malformed or non-UTF-8 escapes, has no artifact path under that mapping.
|
||||
An encoded backslash (`%5C`) remains a literal name byte. Mappings cannot add CDP
|
||||
origins, broaden Electron roots, read files, or authorize source-map access;
|
||||
they are caller-declared inference inputs applied only to already-authorized
|
||||
locations.
|
||||
@@ -133,6 +137,8 @@ classification:
|
||||
- `unknown` covers incomplete captures, layers outside runtime scope, and
|
||||
unresolved correspondence. Nodes outside a mapped artifact prefix stay
|
||||
unknown even when another prefix in the same static layer was observed.
|
||||
Nodes beneath a mapped artifact prefix also stay unknown when a runtime
|
||||
script or worker under that mapping has no artifact path.
|
||||
|
||||
None of these states proves feature execution, initialization order,
|
||||
reachability, or causality. In particular, `not-observed-in-capture` never means
|
||||
|
||||
@@ -68,7 +68,7 @@ The background export belongs to the session. `close_binary`, or the end of a CL
|
||||
|
||||
The assembly label index is scanned once per export reader. Cancelling a request stops its wait without interrupting other readers; closing the session aborts and drains the shared scan before releasing its files.
|
||||
|
||||
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a five-second matching deadline and a worker old-generation heap. Literal matching remains the default. Cancellation or a matching deadline leaves the session available. This release has no runtime override for that deadline. The worker requests 128 MiB. A positive Node.js `--max-old-space-size` on the process that starts REA, including `NODE_OPTIONS`, overrides that request and sets the effective heap. `--max-old-space-size-percentage` takes precedence over that flag in either order. REA does not turn the percentage into a MiB, so the effective heap stays unestablished instead of the 128 MiB request or the absolute flag. The reported `NODE_OPTIONS` value is the process initial environment when that block can be read. A later edit to the environment is not the value Node applied, and an unreadable initial value leaves the effective heap unestablished. The launch Node options are the initial command line on Linux when that buffer still has a program entry, not a later edit to `process.execArgv`. A preload can clear that array after V8 has applied the flags. Setting `process.title` or `--title` overwrites the command line, so a buffer without a program entry leaves the effective heap unestablished. `--env-file` or `--env-file-if-exists` can supply `NODE_OPTIONS` when the initial environment has none. That file is not part of the initial environment block, so the effective heap stays unestablished instead of the 128 MiB request. An initial `NODE_OPTIONS` is still used, because Node does not let the file replace it. A positive `--max-heap-size`, including the underscore form and launch `NODE_OPTIONS` on Node versions that allow it, sets the total heap rather than the old generation and can be lower than the 128 MiB request. The effective old generation stays unestablished while that value is in effect. A zero or empty value clears it. `--experimental-config-file` or `--experimental-default-config-file` can set `nodeOptions`, including a heap percentage, from a file that is not read back. The effective heap stays unestablished when either flag is present, including when an absolute `--max-old-space-size` is also on the command line or in the recovered `NODE_OPTIONS`. Node accepts the config path as `=` or the next argument.
|
||||
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a matching deadline and a worker old-generation heap. Literal matching remains the default. Cancellation or a matching deadline leaves the session available. The deadline is `REA_REGEX_MATCHING_DEADLINE_MS`, a positive decimal integer from 1 to 2147483647 ms. Unset uses 5000 ms. A larger value is refused because Node schedules a delay past that signed 32-bit bound as 1 ms. Invalid values are configuration errors and do not fall back to the default. The setting is read from the environment that starts REA, so a change needs a restart. A deadline failure names the effective milliseconds, whether they came from the default or the setting, the default, and the maximum. The worker old-generation heap does not raise it. The worker heap request is `REA_REGEX_WORKER_HEAP_REQUEST_MIB`, a decimal integer from 7 to 16384 MiB. Unset uses 128 MiB. A request below 7 MiB aborts the worker during startup, and a larger value is refused. Invalid values are configuration errors and do not fall back to 128. The setting is read from the environment that starts REA, so a change needs a restart. A heap failure names the effective request, whether it came from the default or the setting, the default, the floor, and the maximum. A positive Node.js `--max-old-space-size` on the process that starts REA, including `NODE_OPTIONS`, overrides that request and sets the effective heap. `--max-old-space-size-percentage` takes precedence over that flag in either order. REA does not turn the percentage into a MiB, so the effective heap stays unestablished instead of the worker heap request or the absolute flag. The reported `NODE_OPTIONS` value is the process initial environment when that block can be read. A later edit to the environment is not the value Node applied, and an unreadable initial value leaves the effective heap unestablished. The launch Node options are the initial command line on Linux when that buffer still has a program entry, not a later edit to `process.execArgv`. A preload can clear that array after V8 has applied the flags. Setting `process.title` or `--title` overwrites the command line, so a buffer without a program entry leaves the effective heap unestablished. `--env-file` or `--env-file-if-exists` can supply `NODE_OPTIONS` when the initial environment has none. That file is not part of the initial environment block, so the effective heap stays unestablished instead of the worker heap request. An initial `NODE_OPTIONS` is still used, because Node does not let the file replace it. A positive `--max-heap-size`, including the underscore form and launch `NODE_OPTIONS` on Node versions that allow it, sets the total heap rather than the old generation and can be lower than the worker heap request. The effective old generation stays unestablished while that value is in effect. A zero or empty value clears it. `--experimental-config-file` or `--experimental-default-config-file` can set `nodeOptions`, including a heap percentage, from a file that is not read back. The effective heap stays unestablished when either flag is present, including when an absolute `--max-old-space-size` is also on the command line or in the recovered `NODE_OPTIONS`. Node accepts the config path as `=` or the next argument.
|
||||
|
||||
Single-function subprocess output and each cached source facet share a 16 MiB buffered-text capacity. Cached spans are checked before allocation; larger text remains in the local export and the error identifies the file and range. `analyze_function` initially returns the complete dossier, which can still be large; later retained views do not remove that first transfer. Prefer the individual text tools for a focused question.
|
||||
|
||||
|
||||
@@ -428,6 +428,32 @@ Declared import inventory supports a bounded positive claim. Its absence does
|
||||
not exclude dynamic resolution, generated code, protected code, native helpers,
|
||||
or server-side behavior.
|
||||
|
||||
## Application graph relations
|
||||
|
||||
`project_managed_application_graph` turns member Evidence into `contains`
|
||||
edges for declarations, `imports` edges for P/Invoke declarations, and two
|
||||
relation kinds read from method bodies:
|
||||
|
||||
- `calls` links two `managed-method` nodes when a `call`, `callvirt`, `newobj`
|
||||
or `jmp` site in the caller targets a MethodDef in the inspected module.
|
||||
- `accesses` links a `managed-method` node to a `managed-field` node for each
|
||||
field it loads, stores or takes the address of.
|
||||
|
||||
Each edge stands for every site between one source and one target. Its
|
||||
properties carry `site_count` and per-opcode `opcodes` counts; `accesses`
|
||||
edges also list `access` as `read`, `write` and/or `address`. Edge evidence
|
||||
links the member Evidence.
|
||||
|
||||
Some sites are counted rather than projected: calls to MemberRefs (members of
|
||||
other assemblies or of generic type instantiations), generic method
|
||||
instantiations (`MethodSpec`), unknown targets, `ldftn`/`ldvirtftn` function
|
||||
pointer loads, recursive self-calls (the graph has no self-loops), MemberRef
|
||||
fields, and sites whose method or field node is absent, for example under a
|
||||
type selection. Limitations name each counted class, and graph coverage is
|
||||
`partial` with `omitted_count` equal to the omitted sites when nothing else is
|
||||
incomplete. A missing `calls` edge is therefore not evidence that a method is
|
||||
never called.
|
||||
|
||||
## Evidence record shape
|
||||
|
||||
Member comparison, application graph projection, and managed/native verification
|
||||
|
||||
+12
-6
@@ -95,11 +95,15 @@ optional bounded `recovery` next step. `recovery.kind=inspect_input_contract`
|
||||
names the rejected operation: re-read that operation's advertised input schema
|
||||
(call `tools/list` if the catalog may be stale), correct the listed fields, and
|
||||
do not repeat the unchanged call. `recovery.kind=inspect_timeout_state` names
|
||||
the operation and the REA server/provider deadline; inspect partial Evidence
|
||||
and reported cleanup state, then follow producer-specific remediation or reduce
|
||||
supported work. Generic errors do not assume adjustable timeout controls. Raising
|
||||
only the MCP client's deadline cannot extend this budget. These hints never
|
||||
include the complete catalog or caller arguments.
|
||||
the operation and the REA server/provider deadline. Inspect partial Evidence
|
||||
and reported cleanup state, then follow the returned remediation. Change a
|
||||
request field or REA_* setting only when the returned remediation explicitly
|
||||
recommends that change. Restart REA when that text says the value is read at
|
||||
startup. A mention that a control does not raise the deadline is not a
|
||||
recommendation to change it. When it names no setting, reduce supported work;
|
||||
do not invent one. Raising only the MCP
|
||||
client's deadline cannot extend this budget. These hints never include the
|
||||
complete catalog or caller arguments.
|
||||
Failures omit `structuredContent`, since each advertised `outputSchema`
|
||||
describes successful results. Successful replies retain their schema-validated
|
||||
structured data and matching text projection. Oversized errors are retained
|
||||
@@ -310,7 +314,9 @@ that complete Evidence in the session and returns the derived
|
||||
`normalized_result.parent_evidence_id` and `evidence_links` name the retained
|
||||
analysis, so later views and application workflows read it without repeating
|
||||
analysis or moving the complete record through the response. A server without
|
||||
session retention refuses summary detail with `capability_unavailable`.
|
||||
session retention refuses summary detail with `capability_unavailable`;
|
||||
`inspect_artifact`, `inspect_wasm_artifact`, `inspect_macho` and
|
||||
`decode_interface_builder` refuse before any analysis starts.
|
||||
For `inspect_artifact`, omission also keeps the original request parameters and
|
||||
complete output shape. Delivery selection never reaches the inventory engine;
|
||||
the complete inspection and its atomic inventory Evidence remain retained.
|
||||
|
||||
@@ -58,7 +58,10 @@ lengths; affected totals and storage validation remain unknown.
|
||||
Safetensors reports the header extent, exact string `__metadata__` values, dtype,
|
||||
shape and declared data ranges. JSON duplicate keys, invalid UTF-8, out-of-file
|
||||
ranges, overlaps, gaps, trailing unindexed data and known dtype/shape byte-length
|
||||
contradictions are rejected with a typed `file_format_rejected` constraint (public code `target_unavailable`, category `unsupported_target`). Empty
|
||||
contradictions are rejected with a typed `file_format_rejected` constraint (public code `target_unavailable`, category `unsupported_target`).
|
||||
A missing, non-regular or otherwise unusable selected path is also `target_unavailable`;
|
||||
host I/O or resource failures on a valid selection, such as `EMFILE` or `EIO`, are
|
||||
`execution_failure` with the path and reason in `details.diagnostics`. Empty
|
||||
tensors and scalar shapes are supported; packed sub-byte types require whole
|
||||
byte storage. Unknown dtypes retain declared ranges and an explicit limitation
|
||||
that shape-to-storage validation is unavailable. Each tensor records
|
||||
|
||||
@@ -202,14 +202,28 @@ Function dossiers retain this same string evidence. Native call edges retain
|
||||
Hopper's partial `CallReference` classification and exact endpoints; detailed
|
||||
reference flags remain unavailable rather than being invented.
|
||||
Regex searches use ECMAScript Unicode syntax in a cancellable worker with a
|
||||
five-second matching deadline and a worker old-generation heap. Deadline or
|
||||
cancellation stops matching while leaving the Hopper API available. This
|
||||
release has no runtime override for that deadline. The worker requests 128
|
||||
MiB. A positive Node.js `--max-old-space-size` on the process that starts
|
||||
matching deadline and a worker old-generation heap. Deadline or cancellation
|
||||
stops matching while leaving the Hopper API available. The deadline is
|
||||
`REA_REGEX_MATCHING_DEADLINE_MS`, a positive decimal integer from 1 to
|
||||
2147483647 ms. Unset uses 5000 ms. A larger value is refused because Node
|
||||
schedules a delay past that signed 32-bit bound as 1 ms. Invalid values are
|
||||
configuration errors and do not fall back to the default. The setting is read
|
||||
from the environment that starts REA, so a change needs a restart. A deadline
|
||||
failure names the effective milliseconds, whether they came from the default
|
||||
or the setting, the default, and the maximum. The worker old-generation heap
|
||||
does not raise it. Kuna regex search uses the same deadline. The worker
|
||||
heap request is `REA_REGEX_WORKER_HEAP_REQUEST_MIB`, a decimal integer from
|
||||
7 to 16384 MiB. Unset uses 128 MiB. A request below 7 MiB aborts the worker
|
||||
during startup, and a larger value is refused. Invalid values are
|
||||
configuration errors and do not fall back to 128. The setting is read from
|
||||
the environment that starts REA, so a change needs a restart. A heap failure
|
||||
names the effective request, whether it came from the default or the setting,
|
||||
the default, the floor, and the maximum. A positive Node.js
|
||||
`--max-old-space-size` on the process that starts
|
||||
REA, including `NODE_OPTIONS`, overrides that request and sets the effective
|
||||
heap. `--max-old-space-size-percentage` takes precedence over that flag in
|
||||
either order. REA does not turn the percentage into a MiB, so the effective
|
||||
heap stays unestablished instead of the 128 MiB request or the absolute flag.
|
||||
heap stays unestablished instead of the worker heap request or the absolute flag.
|
||||
The reported `NODE_OPTIONS` value is the process initial environment when that
|
||||
block can be read. A later edit to the environment is not the value Node
|
||||
applied, and an unreadable initial value leaves the effective heap
|
||||
@@ -221,11 +235,11 @@ buffer without a program entry leaves the effective heap unestablished.
|
||||
`--env-file` or `--env-file-if-exists` can supply `NODE_OPTIONS`
|
||||
when the initial environment has none. That file is not part of the initial
|
||||
environment block, so the effective heap stays unestablished instead of the
|
||||
128 MiB request. An initial `NODE_OPTIONS` is still used, because Node does
|
||||
worker heap request. An initial `NODE_OPTIONS` is still used, because Node does
|
||||
not let the file replace it. A positive `--max-heap-size`, including the
|
||||
underscore form and launch `NODE_OPTIONS` on Node versions that allow it,
|
||||
sets the total heap rather than the old generation and can be lower than the
|
||||
128 MiB request. The effective old generation stays unestablished while that
|
||||
worker heap request. The effective old generation stays unestablished while that
|
||||
value is in effect. A zero or empty value clears it.
|
||||
`--experimental-config-file` or `--experimental-default-config-file` can set
|
||||
`nodeOptions`, including a heap percentage, from a file that is not read
|
||||
@@ -321,6 +335,15 @@ Mach-O fixtures check each case against source-owned return values.
|
||||
|
||||
## Static inspection
|
||||
|
||||
`inspect_artifact` and `extract_artifact` retain universal Mach-O slices under
|
||||
their exact `lipo` names, including 32-bit ARM implementations such as `armv7`,
|
||||
`armv7s`, and `armv7k`. These names are reconciled with the FAT CPU subtype;
|
||||
the structural reader's `arm` family label alone does not identify a slice.
|
||||
Available CPU type, subtype/capability bits, byte range, and alignment remain
|
||||
checked against the source bytes. Inventory records source ranges and hashes, and
|
||||
extraction copies those ranges without running the target. This does not
|
||||
expand any analysis provider's supported architectures.
|
||||
|
||||
`inspect_plist.format` describes the selected source, not plutil's conversion
|
||||
format. Sources explicitly identified as XML, binary or JSON retain those
|
||||
labels. For generic text, a bounded source-prefix probe recognizes leading XML
|
||||
@@ -462,9 +485,9 @@ complete observation.
|
||||
`rea inspect-native-dispatch-metadata <app-or-binary>` prefers a validated macOS Mach-O byte reader. It decodes
|
||||
64-bit little-endian Objective-C class/metaclass records, superclass pointers,
|
||||
absolute/relative method entries, ivar offsets/sizes/alignment and protocol
|
||||
declarations. It also decodes simple Swift conformances, static synchronous
|
||||
witness slots, signed-relative pointers and non-generic, non-resilient class
|
||||
vtable descriptors. Records include exact virtual addresses/file offsets and
|
||||
declarations. It also decodes Swift nominal type declarations, simple Swift
|
||||
conformances, static synchronous witness slots, signed-relative pointers and
|
||||
non-generic, non-resilient class vtable descriptors. Records include exact virtual addresses/file offsets and
|
||||
artifact evidence. Vtable indexes are metadata word offsets; witness indexes
|
||||
start after the conformance header. Names unavailable in metadata remain null.
|
||||
The byte reader holds the whole selected file in memory, so it admits files up
|
||||
@@ -475,6 +498,20 @@ complete observation.
|
||||
(`default` or `caller`) and the bytes retained. A refusal names the observed
|
||||
file size, the effective budget and its source, and either the budget to
|
||||
select or, above 1 GiB, a deep analysis provider.
|
||||
`swift_decls` lists each `__swift5_types` nominal type descriptor, which
|
||||
survives stripping: `kind` (`class`, `struct`, `enum`, or `actor` from the
|
||||
class actor flag), the descriptor's own `name`, `module`, `qualified_name`
|
||||
(`Outer.Inner`), `generic`, the metadata `accessor_address`, the descriptor
|
||||
`location`, and evidence citing its `__swift5_types` record.
|
||||
Private and local types set `anonymous_context` because their anonymous
|
||||
context is omitted from `qualified_name`. When a parent context cannot be
|
||||
resolved, such as an extension of a type named only by a mangled name, a
|
||||
missing module, or a cyclic chain, `qualified_name` is null and the entry is
|
||||
partial with a reason. Access level, superclass, conformances, and fields are
|
||||
not reported by this facet. The `swift_declarations` coverage counters count
|
||||
examined type records and fully decoded declarations. Class vtable recovery
|
||||
reads the same descriptors, so an unreadable type record also keeps
|
||||
`swift_class_vtable_descriptors` partial.
|
||||
The `swift_class_vtable_descriptors` coverage counters count examined vtable
|
||||
slots and successfully decoded slots, rather than type-section entries.
|
||||
The `objc_class_method_ivar_metadata` counters count admitted class and
|
||||
|
||||
@@ -10,10 +10,17 @@ Supply an existing TShark 4.x installation **on Linux**, with util-linux
|
||||
`prlimit` available at `/usr/bin/prlimit`. Override its absolute path with
|
||||
`REA_TSHARK_PRLIMIT_COMMAND` if required. Unsupported hosts, including macOS
|
||||
and Windows, are refused before inspection because this integration requires
|
||||
an enforced address-space limit:
|
||||
an enforced address-space limit.
|
||||
|
||||
TShark selection follows the same rule as [`inspect_pcap_capture`](pcap-analysis.md):
|
||||
an absolute `REA_TSHARK_COMMAND`, otherwise `tshark` from the first absolute
|
||||
`PATH` entry that holds an executable file. A relative `REA_TSHARK_COMMAND` is
|
||||
refused rather than looked up. Either way the selected executable is resolved,
|
||||
hashed and rechecked after each run, and Evidence records its
|
||||
`selection_source` (`environment` or `path`):
|
||||
|
||||
```sh
|
||||
export REA_TSHARK_COMMAND=/absolute/path/to/tshark
|
||||
export REA_TSHARK_COMMAND=/absolute/path/to/tshark # optional when tshark is on PATH
|
||||
rea inspect-packet-capture /artifacts/device.pcapng \
|
||||
--fields ip.addr --fields udp.port --fields data.data \
|
||||
--display_filter 'udp' --format json
|
||||
|
||||
+10
-1
@@ -17,7 +17,16 @@ rea inspect-pcap-capture /captures/session.pcapng --json
|
||||
```
|
||||
|
||||
Without explicit tool commands, this family resolves them through PATH and
|
||||
records that selection source. Client inspection reads no capture file. Capture
|
||||
records that selection source. On Linux and macOS it runs the tool found in the
|
||||
first absolute PATH entry holding an executable file, the same selection the
|
||||
[packet capture tools](packet-captures.md) make; relative PATH entries are
|
||||
skipped. Windows keeps PATH/PATHEXT resolution in the process launcher. Client
|
||||
inspection reads no capture file; on hosts where PATH/PATHEXT does not expose
|
||||
an absolute candidate, its executable digest and resolved path are reported as
|
||||
unresolved. Otherwise the canonical executable path and SHA-256 are reported.
|
||||
The pcap provider and packet tools share the selected-tool identity check, and
|
||||
changes to a resolved executable during a run fail with artifact-changed
|
||||
Evidence. Capture
|
||||
inspection requires Linux with util-linux prlimit or Windows with owned Job
|
||||
Objects. macOS capture inspection returns a capability gap because the new
|
||||
native memory boundary is unverified; its version probes remain available.
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
# PE Authenticode certificate tables
|
||||
|
||||
`inspect_pe_certificates` / `inspect-pe-certificates` answers whether one
|
||||
selected local PE32/PE32+ image declares an `IMAGE_DIRECTORY_ENTRY_SECURITY`
|
||||
table, which `WIN_CERTIFICATE` entries that table stores, and which PKCS#7
|
||||
signer and X.509 fields those entries carry. It reads the file without
|
||||
executing it and does not require a Windows SDK, catalog file, or disassembler.
|
||||
|
||||
```bash
|
||||
rea inspect-pe-certificates /absolute/path/to/application.exe --json
|
||||
```
|
||||
|
||||
The CLI resolves relative paths against the caller's working directory. MCP
|
||||
and the advertised input schema require an absolute filesystem path and
|
||||
preserve that exact selection:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_pe_certificates",
|
||||
"arguments": { "path": "/artifacts/application.exe" }
|
||||
}
|
||||
```
|
||||
|
||||
## Observations
|
||||
|
||||
The inline Evidence result contains the artifact path, SHA-256, and byte length.
|
||||
`table` is the file-offset security directory, or `null` when the directory is
|
||||
absent or zero. Each `entries[]` item reports `WIN_CERTIFICATE` length,
|
||||
revision, type, and the payload SHA-256.
|
||||
|
||||
For `WIN_CERTIFICATE` type 2 (PKCS#7 signedData) `pkcs7` contains digest
|
||||
algorithm OIDs, embedded X.509 serial/issuer/subject/validity fields, and
|
||||
signer identifier facts. A signer identified by subject key identifier reports
|
||||
that value as `subject_key_identifier_hex`. Other certificate types keep
|
||||
`pkcs7` null.
|
||||
|
||||
A missing table is observed absence of that directory. It is not proof the
|
||||
image is unsigned in every catalog or overlay sense.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
eight `WIN_CERTIFICATE` entries are examined by default. Malformed PE headers,
|
||||
truncated certificate entries, and PKCS#7 that is not structurally signedData
|
||||
fail without partial success. A security directory that ends in fewer than
|
||||
eight header bytes, including zeros, is a truncated `WIN_CERTIFICATE` and
|
||||
fails the same way. SignedData must contain at least one `SignerInfo`, and
|
||||
each `SignerInfo` must carry a non-empty signature value. Each embedded X.509
|
||||
certificate must include `subjectPublicKeyInfo` and `signatureValue`. Decoded
|
||||
certificate name text, serials, validity spellings, and subject key identifiers
|
||||
share a 64 KiB budget per PKCS#7 payload. Unsupported certificate name
|
||||
encodings, including TeletexString (T.61) and GeneralString, fail explicitly
|
||||
rather than returning names decoded with an assumed character set.
|
||||
|
||||
This workflow does not verify signatures, certificate chains, timestamps, or
|
||||
Authenticode digest coverage of image bytes.
|
||||
@@ -252,7 +252,14 @@ Successfully observed identity replacements use `identity_changed` with a null
|
||||
system code. Unstable symlink targets are omitted; already observed directory
|
||||
metadata remains available. The affected root stays incomplete, so omitted
|
||||
entries do not prove deletion. A path missing at its initial lookup remains an
|
||||
observed absence. Cancellation and unexpected I/O failures still stop capture.
|
||||
observed absence.
|
||||
|
||||
Before launch, a selected root below a non-directory, or one whose lookup is
|
||||
denied or otherwise unavailable, fails capture. After admission, a root or descendant lookup that
|
||||
reports a missing path or a non-directory ancestor is an observed absence. A
|
||||
denied or otherwise unavailable lookup records an `entry_unavailable` omission
|
||||
for that path instead, and its root stays incomplete. Cancellation and
|
||||
unexpected I/O failures still stop capture.
|
||||
|
||||
Terminal frame `data` is the comparison text after the selected normalization.
|
||||
New captures also preserve `raw_data` when normalization changes a PTY chunk;
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
# CPython pyc inspection
|
||||
|
||||
`inspect_python_pyc` / `inspect-python-pyc` answers which CPython version wrote
|
||||
one selected local `.pyc` file, how its 16-byte header records source identity,
|
||||
and which nested code objects marshal stored. It reads the file without running
|
||||
bytecode and requires no Python installation, active binary target, Hopper,
|
||||
Ghidra, or IDA.
|
||||
|
||||
```bash
|
||||
rea inspect-python-pyc /absolute/path/to/module.pyc --json
|
||||
```
|
||||
|
||||
The CLI resolves relative paths against the caller's working directory. MCP
|
||||
requires an absolute filesystem path and preserves that exact selection:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_python_pyc",
|
||||
"arguments": { "path": "/artifacts/module.pyc" }
|
||||
}
|
||||
```
|
||||
|
||||
## Observations
|
||||
|
||||
The inline Evidence result contains the artifact path, SHA-256, and byte length.
|
||||
`header` contains:
|
||||
|
||||
- `magic`, the 16-bit little-endian field;
|
||||
- `python_version` and `code_layout`, mapped from the pinned CPython 3.7–3.13
|
||||
table (`py37`, `py38`, or `py311`); 3.11 alpha magics before 3494 used
|
||||
other code layouts and are unsupported;
|
||||
- `flags`, `hash_based`, and `check_source` from PEP 552;
|
||||
- `source_hash_hex` for hash-based pycs, or `source_mtime` and `source_size`
|
||||
for timestamp pycs.
|
||||
|
||||
`code_objects` lists each distinct marshal code object once, the top-level
|
||||
module code first. `nested_code` holds the indexes of the distinct code objects
|
||||
reachable from that object's constants, including code nested in tuples and
|
||||
other containers and the owner itself when a constant container reaches it.
|
||||
A function shared through a marshal reference is named by index once per owner,
|
||||
even when that owner reaches it again through the same container. Relationships
|
||||
are collected after the marshal graph is complete, so a container that a nested
|
||||
code object also references is not summarized while its items are still missing.
|
||||
Each object reports name, optional qualname,
|
||||
filename,
|
||||
argument counts, flags, names, locals (or `localsplusnames` on 3.11+), const
|
||||
summaries, and bytecode length plus SHA-256.
|
||||
|
||||
Bytecode is not decompiled. Const strings may be truncated in the summary while
|
||||
the marshal walk still consumed the original bytes; a string summary's `bytes`
|
||||
is that marshal length. Integer consts report `bits`; their decimal `value` is
|
||||
`null` above 256 bits. Non-finite float and complex parts use CPython's `inf`,
|
||||
`-inf`, and `nan` spellings. Dictionary and set lengths follow CPython:
|
||||
unhashable keys and members fail the walk, equal set members count once, equal
|
||||
dictionary keys keep the first key object and replace only its value, code
|
||||
objects are valid keys and members, and a null dictionary value ends the dict
|
||||
without that entry.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
The selected file is limited to 16 MiB by default (64 MiB maximum). Marshal
|
||||
object count, nesting depth, and per-string size are bounded; exceeding a
|
||||
budget fails without partial success. Argument counts, positional-only counts,
|
||||
keyword-only counts, local counts, and stack size are signed CPython fields; a
|
||||
negative encoding is malformed. On the 3.11+ layout, `co_code` must cover a
|
||||
whole number of code units. Every layout rejects argument counts that do not
|
||||
fit the local table CPython checks (`co_varnames`, including `*args` /
|
||||
`**kwargs`, or fast locals on 3.11+). Those files are malformed, so the
|
||||
workflow does not report complete coverage. 3.7–3.10 still accept an odd
|
||||
`co_code`, matching those constructors. Nested code collection walks reference
|
||||
chains iteratively and keeps one shared descendant list for a container that
|
||||
merely wraps another, instead of copying that list onto every wrapper. Set and
|
||||
dictionary keys walk tuple and frozenset reference chains the same way, so a
|
||||
long `TYPE_REF` chain stays within the call stack. A shared child stored twice
|
||||
still counts its expanded key against the string budget. Code-object equality
|
||||
keys record each name table's length and each entry's boundaries, so `()` and
|
||||
`("",)` stay distinct, and `co_varnames`, `co_freevars`, and `co_cellvars` stay
|
||||
separate tables. Names repeated through a marshal reference, including each
|
||||
code-constant name the report copies, count against the file budget once per
|
||||
copy. Empty strings add no bytes, so every copied
|
||||
name-table entry (`co_names`, varnames, freevars, cellvars, and
|
||||
`localsplusnames`) counts against the object budget before its array is built.
|
||||
Copied constant tables and each reported nested code relationship count
|
||||
against that same object budget. Integer digits and expanded
|
||||
set or dictionary keys, including a shared tuple repeated at each level, count
|
||||
against the string budget. Magics outside the pinned 3.7–3.13 table,
|
||||
truncated headers, trailing marshal bytes, and a non-code top-level object are
|
||||
typed failures.
|
||||
|
||||
A decoded header does not prove the matching `.py` is present. `python_version`
|
||||
is the magic table, not a running interpreter. This workflow does not recover
|
||||
source.
|
||||
@@ -143,8 +143,11 @@ npx rea-agents@latest setup
|
||||
| ------------------------------- | ---------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
|
||||
| الملفات الثنائية للشيفرة الأصلية | شيفرة شبه برمجية، وتعليمات تجميع، وسلاسل نصية، ورموز، واستدعاءات، ومراجع | Hopper أو Ghidra أو IDA؛ [تحليل الشيفرة الأصلية](../native-investigation.md) |
|
||||
| بنية ELF دون تشغيل | الأقسام والمقاطع والرموز ومعلومات إعادة التموضع الأصلية، وآليات الحماية المحتملة المستدلّ عليها بالفحص الثابت | pwntools يوفّره المستدعي على Linux x64؛ [تشخيص الملفات الثنائية](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| شيفرة EVM البايتية | محددات التوجيه، وإزاحات البايتات، والمعاملات المستنتجة، وقابلية تغيير الحالة | مدخل محلي يحمل بايتات خامًا أو تمثيلًا سداسيًا عشريًا على Linux x64؛ [دليل EVM دون تشغيل](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| أعطال Linux المسجّلة | سجلات note الخام، ومسجّلات المعالج/الإشارات لكل خيط مسجّل، وترشيحات اختيارية لخرائط الذاكرة | pwntools يوفّره المستدعي على Linux x64؛ GDB/pwndbg اختياريان؛ [الأعطال المسجّلة](../recorded-crashes.md) |
|
||||
| تثبيتات سطح المكتب | عائلات وقت التشغيل، والملفات الداعمة، وعلامات المكونات، والخطوات التالية | فحص محدود للملفات والترويسات;[تطبيقات سطح المكتب](../desktop-applications.md) |
|
||||
| JavaScript / Electron | الوحدات، وعمليات الاستيراد، وخرائط المصدر، والمسارات، وIPC، وعلاقات إضافات الشيفرة الأصلية | Node.js وnpm؛ [تحليل التطبيقات](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -143,8 +143,11 @@ REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätz
|
||||
| ----------------------------------- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Native Binärdateien | Pseudocode, Assemblercode, Zeichenketten, Symbole, Aufrufe und Verweise | Hopper, Ghidra oder IDA; [native Analyse](../native-investigation.md) |
|
||||
| Offline-ELF-Struktur | Sektionen, Segmente, ursprüngliche Symbole/Relokationen und mögliche Schutzmaßnahmen aus statischer Analyse | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; [Binärdiagnostik](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| EVM-Bytecode | Dispatch-Selektoren, Byte-Offsets, abgeleitete Argumente und Veränderbarkeit des Zustands | Lokale Eingabe mit Rohbytes oder Hexdarstellung unter Linux x64; [Offline-EVM-Leitfaden](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Aufgezeichnete Linux-Abstürze | Unverarbeitete Note-Einträge, Register/Signale aller aufgezeichneten Threads und optionale Zuordnungskandidaten | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; optional GDB/pwndbg; [aufgezeichnete Abstürze](../recorded-crashes.md) |
|
||||
| Desktop-Installationen | Laufzeitkandidaten, Quelldateien, Komponenten und nächste Schritte | Begrenzte Verzeichnis- und Headerprüfung;[Desktop-Anwendungen](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Module, Imports, Source Maps, Routen, IPC und Beziehungen zu nativen Add-ons | Node.js und npm; [Anwendungsanalyse](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -149,6 +149,9 @@ npx rea-agents@latest setup
|
||||
| Δομή ELF χωρίς εκτέλεση | Ενότητες, τμήματα, αρχικά σύμβολα/εγγραφές μετεγκατάστασης και ενδείξεις πιθανών μέτρων προστασίας από στατική ανάλυση | pwntools που παρέχει ο χρήστης σε Linux x64· [διαγνωστικά δυαδικών αρχείων](../binary-diagnostics.md) |
|
||||
| Δυαδικά αρχεία Go | Ενσωματωμένη έκδοση μεταγλωττιστή, modules, αντικαταστάσεις και ρυθμίσεις μεταγλώττισης | Ενσωματωμένη στατική εξέταση ELF/PE/thin Mach-O· δεν απαιτείται εγκατάσταση του Go |
|
||||
| Δυαδικά αρχεία Rust | Ενσωματωμένα ονόματα, εκδόσεις, πηγές και σχέσεις εξάρτησης των crates από το cargo-auditable | Ενσωματωμένη στατική εξέταση ELF/PE/thin Mach-O· [δυαδικά αρχεία Rust](../rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Αρχείο WASM | Έλεγχο εγκυρότητας, ενότητες, ακριβείς μορφές εισαγωγών/εξαγωγών σε WAT και αποκωδικοποιημένο WAT | WABT 1.0.42 που παρέχει ο χρήστης· `inspect_wasm_artifact` |
|
||||
| Bytecode EVM | Selectors επιλογής συνάρτησης, μετατοπίσεις σε bytes, εκτιμώμενα ορίσματα και δυνατότητα μεταβολής της κατάστασης | Τοπική είσοδος σε ακατέργαστα bytes ή δεκαεξαδική μορφή σε Linux x64· [οδηγός EVM χωρίς εκτέλεση](../evm-bytecode.md) |
|
||||
| Βάσεις δεδομένων SQLite | Πίνακες, στήλες, ευρετήρια, όψεις, triggers και επιλεγμένες γραμμές κανονικών πινάκων | Τοπικό στιγμιότυπο βάσης δεδομένων/WAL· Node.js 24.x >=24.15 ή 26+ με εγγενή όρια SQLite |
|
||||
|
||||
@@ -143,8 +143,11 @@ REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las h
|
||||
| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Binarios nativos | Pseudocódigo, ensamblador, cadenas, símbolos, llamadas y referencias | Hopper, Ghidra o IDA; [análisis nativo](../native-investigation.md) |
|
||||
| Estructura ELF sin ejecutar | Secciones, segmentos, símbolos/reubicaciones originales y posibles medidas de mitigación detectadas mediante análisis estático | pwntools proporcionado por quien llama, en Linux x64; [diagnóstico de binarios](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| Bytecode EVM | Selectores de despacho, desplazamientos en bytes, argumentos inferidos y mutabilidad | Entrada local con bytes sin procesar o representación hexadecimal, en Linux x64; [guía de EVM sin ejecutar](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Fallos de Linux registrados | Registros note sin procesar, registros/señales de cada hilo registrado y posibles correspondencias de memoria opcionales | pwntools proporcionado por quien llama, en Linux x64; GDB/pwndbg opcionales; [fallos registrados](../recorded-crashes.md) |
|
||||
| Instalaciones de escritorio | Posibles entornos de ejecución, archivos de evidencia, componentes y siguientes pasos | Inspección acotada de directorios y cabeceras;[Aplicaciones de escritorio](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Módulos, importaciones, mapas de código fuente, rutas, IPC y relaciones con complementos nativos | Node.js y npm; [análisis de aplicaciones](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -135,8 +135,11 @@ REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، بههمراه np
|
||||
| ---------------------- | ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| باینریهای بومی | شبهکد، اسمبلی، رشتهها، نمادها، فراخوانیها و ارجاعات | Hopper، Ghidra یا IDA؛ [تحلیل بومی](../native-investigation.md) |
|
||||
| ساختار ELF آفلاین | بخشها، سگمنتها، نمادها و relocationهای اصلی و گزینههای احتمالی کاهش آسیبپذیری ایستا | pwntools تأمینشده توسط فراخواننده روی Linux x64؛ [تشخیص باینری](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| بایتکد EVM | selectorهای dispatch، offsetهای بایتی، آرگومانهای استنباطشده و قابلیت تغییر وضعیت | دادهٔ خام/hex محلی روی Linux x64؛ [راهنمای EVM آفلاین](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| کرشهای ثبتشدهٔ لینوکس | یادداشتهای خام، رجیسترها/سیگنالهای تمام threadهای ثبتشده و گزینههای احتمالی نگاشت | pwntools تأمینشده توسط فراخواننده روی Linux x64؛ GDB/pwndbg اختیاری؛ [کرشهای ثبتشده](../recorded-crashes.md) |
|
||||
| نصبهای دسکتاپ | نامزدهای زمان اجرا، فایلهای شاهد، نشانههای مؤلفهها و مراحل بعدی | بررسی محدود پوشهها و سرآیندها;[برنامههای دسکتاپ](../desktop-applications.md) |
|
||||
| JavaScript / Electron | ماژولها، importها، source mapها، مسیرها، IPC و ارتباط افزونههای بومی | Node.js و npm؛ [تحلیل برنامه](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -146,9 +146,12 @@ REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les
|
||||
| ---------------------------- | ----------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Binaires natifs | Pseudo-code, assembleur, chaînes, symboles, appels et références | Hopper, Ghidra ou IDA ; [analyse native](../native-investigation.md) |
|
||||
| Structure ELF hors ligne | Sections, segments, symboles et relocations d'origine, et protections statiques candidates | pwntools fourni par l'utilisateur sous Linux x64 ; [diagnostics binaires](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| Artefact WASM | Validation, sections, formes WAT exactes d'import/export et WAT décodé | WABT 1.0.42 fourni par l'utilisateur ; `inspect_wasm_artifact` |
|
||||
| Bytecode EVM | Sélecteurs de dispatch, offsets en octets, arguments inférés et mutabilité | Fichier local brut ou hexadécimal sous Linux x64 ; [guide EVM hors ligne](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Crashs Linux enregistrés | Notes brutes, registres et signaux de chaque thread enregistré, et correspondances de mappings facultatives | pwntools fourni par l'utilisateur sous Linux x64 ; GDB/pwndbg en option ; [crashs enregistrés](../recorded-crashes.md) |
|
||||
| Installations de bureau | Environnements candidats, fichiers justificatifs, composants et étapes suivantes | Inspection bornée des répertoires et en-têtes;[Applications de bureau](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC et relations avec les modules natifs | Node.js et npm ; [analyse d'applications](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -149,9 +149,12 @@ Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis:
|
||||
| -------------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Biner native | Pseudocode, assembly, string, simbol, pemanggilan, dan referensi | Hopper, Ghidra, atau IDA; [analisis native](../native-investigation.md) |
|
||||
| Tata letak ELF offline | Bagian, segmen, simbol/relokasi asli, dan kandidat mitigasi statis | pwntools yang disediakan pemanggil pada Linux x64; [diagnostik biner](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| Artefak WASM | Validasi, bagian, bentuk WAT import/ekspor yang tepat, dan WAT yang didekode | WABT 1.0.42 yang disediakan pemanggil; `inspect_wasm_artifact` |
|
||||
| Bytecode EVM | Selector dispatch, offset byte, argumen yang disimpulkan, dan sifat mutabilitas | Pembawa data raw/hex lokal; [panduan EVM offline](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Crash Linux yang direkam | Catatan mentah, register/sinyal setiap thread yang direkam, dan kandidat pemetaan opsional | pwntools yang disediakan pemanggil; GDB/pwndbg opsional; [crash yang direkam](../recorded-crashes.md) |
|
||||
| Instalasi desktop | Kandidat runtime, berkas pendukung, penanda komponen, dan langkah berikutnya | Pemeriksaan direktori dan header terbatas;[Aplikasi desktop](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -146,9 +146,12 @@ REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strument
|
||||
| ---------------------------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Binari nativi | Pseudocodice, assembly, stringhe, simboli, chiamate e riferimenti | Hopper, Ghidra o IDA; [analisi nativa](../native-investigation.md) |
|
||||
| Layout ELF offline | Sezioni, segmenti, simboli/relocazioni originali e candidati mitigazioni statiche | pwntools fornito dal chiamante su Linux x64; [diagnostica binaria](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| Artefatto WASM | Convalida, sezioni, forme WAT di import/export esatte e WAT decodificato | WABT 1.0.42 fornito dal chiamante; `inspect_wasm_artifact` |
|
||||
| Bytecode EVM | Selettori di dispatch, offset dei byte, argomenti inferiti e mutabilità | Carrier raw/hex locale; [guida EVM offline](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Crash Linux registrati | Note grezze, registri/segnai di ogni thread registrato e candidati mapping opzionali | pwntools fornito dal chiamante; GDB/pwndbg opzionale; [crash registrati](../recorded-crashes.md) |
|
||||
| Installazioni desktop | Runtime candidati, file di supporto, componenti e passi successivi | Ispezione limitata di directory e intestazioni;[Applicazioni desktop](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Moduli, import, source map, route, IPC e relazioni con add-on nativi | Node.js e npm; [analisi delle applicazioni](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -143,8 +143,11 @@ REA には Node.js 22.x(>=22.19)、24.x(>=24.11)、または 26+ と npm
|
||||
| -------------------------------- | ------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ネイティブバイナリ | 疑似コード、アセンブリ、文字列、シンボル、呼び出し、参照 | Hopper、Ghidra、または IDA;[ネイティブ分析](../native-investigation.md) |
|
||||
| オフライン ELF レイアウト | セクション、セグメント、元のシンボル/再配置情報、静的解析によるセキュリティ対策の候補 | Linux x64 で利用者が用意する pwntools;[バイナリ診断](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| EVM バイトコード | ディスパッチセレクター、バイトオフセット、推定された引数と状態変更の可否 | Linux x64 上のローカルの生バイト/16 進数の入力ファイル;[オフライン EVM ガイド](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| 記録済みの Linux クラッシュ | 生の note レコード、記録された全スレッドのレジスター/シグナル、任意のマッピング候補 | Linux x64 で利用者が用意する pwntools;GDB/pwndbg は任意;[記録済みクラッシュ](../recorded-crashes.md) |
|
||||
| デスクトップのインストール先 | ランタイム候補、根拠ファイル、コンポーネントの目印、次の調査手順 | 上限付きのディレクトリ・ヘッダー検査;[デスクトップアプリ](../desktop-applications.md) |
|
||||
| JavaScript / Electron | モジュール、インポート、ソースマップ、ルート、IPC、ネイティブアドオンの関係 | Node.js と npm;[アプリケーション分析](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -153,6 +153,9 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
|
||||
| 오프라인 ELF 레이아웃 | 섹션, 세그먼트, 원본 심볼/재배치 정보, 정적 분석으로 얻은 보안 완화 기법 후보 | Linux x64에서 호출자가 제공하는 pwntools; [바이너리 진단](../binary-diagnostics.md) |
|
||||
| Go 바이너리 | 내장된 컴파일러 버전, 모듈, `replace` 지시어, 빌드 설정 | REA 내장 ELF/PE/thin Mach-O 정적 검사; Go 설치 불필요 |
|
||||
| Rust 바이너리 | cargo-auditable이 내장한 크레이트 이름, 버전, 소스, 의존성 관계 | REA 내장 ELF/PE/thin Mach-O 정적 검사; [Rust 바이너리](../rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| WASM 아티팩트 | 유효성 검사, 섹션, import/export의 정확한 WAT 형식, 디코딩된 WAT | 호출자 제공 WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM 바이트코드 | 디스패치 선택자, 바이트 오프셋, 추론한 인자와 상태 변경 가능 여부 | Linux x64의 로컬 원시 바이트/16진수 입력 파일; [오프라인 EVM 가이드](../evm-bytecode.md) |
|
||||
| SQLite 데이터베이스 | 테이블, 열, 인덱스, 뷰, 트리거, 선택한 일반 테이블의 행 | 로컬 데이터베이스/WAL 스냅샷; Node.js 24.15 이상의 24.x 또는 26 이상(네이티브 SQLite 한도 적용) |
|
||||
|
||||
@@ -151,6 +151,9 @@ REA-യ്ക്ക് Node.js 22.x (>=22.19), 24.x (>=24.11), അല്ലെ
|
||||
| Offline ELF layout | Sections, segments, യഥാർത്ഥ symbols/relocations, static mitigation candidates | Linux x64-ൽ caller നൽകുന്ന pwntools; [binary diagnostics](../binary-diagnostics.md) |
|
||||
| Go binaries | ഉൾച്ചേർത്ത compiler version, modules, replacements, build settings | Built-in static ELF/PE/thin Mach-O പരിശോധന; Go installation ആവശ്യമില്ല |
|
||||
| Rust binaries | ഉൾച്ചേർത്ത cargo-auditable crate പേരുകൾ, versions, sources, dependency edges | Built-in static ELF/PE/thin Mach-O പരിശോധന; [Rust binaries](../rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| WASM artifact | Validation, sections, കൃത്യമായ import/export WAT രൂപങ്ങൾ, decoded WAT | Caller നൽകുന്ന WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, അനുമാനിച്ച arguments, mutability | Linux x64-ൽ local raw/hex carrier; [offline EVM guide](../evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers, തിരഞ്ഞെടുത്ത സാധാരണ table rows | Local database/WAL snapshot; native SQLite പരിമിതികളോടെ Node.js 24.x >=24.15 അല്ലെങ്കിൽ 26+ |
|
||||
|
||||
@@ -143,8 +143,11 @@ REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe na
|
||||
| --------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| Natywne pliki binarne | Pseudokod, kod asemblera, ciągi znaków, symbole, wywołania i odwołania | Hopper, Ghidra lub IDA; [analiza kodu natywnego](../native-investigation.md) |
|
||||
| Układ pliku ELF w trybie offline | Sekcje, segmenty, oryginalne symbole/relokacje i potencjalne mechanizmy zabezpieczeń wskazane przez analizę statyczną | pwntools dostarczone przez użytkownika w systemie Linux x64; [diagnostyka plików binarnych](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| Kod bajtowy EVM | Selektory wywołań, przesunięcia bajtowe, wywnioskowane argumenty i mutowalność | Lokalny plik z surowymi bajtami lub zapisem szesnastkowym w systemie Linux x64; [przewodnik EVM offline](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Zarejestrowane awarie systemu Linux | Surowe rekordy note, rejestry/sygnały każdego zarejestrowanego wątku i opcjonalni kandydaci na mapowania | pwntools dostarczone przez użytkownika w systemie Linux x64; opcjonalnie GDB/pwndbg; [zarejestrowane awarie](../recorded-crashes.md) |
|
||||
| Instalacje desktopowe | Kandydaci środowisk uruchomieniowych, pliki źródłowe dowodów, komponenty i dalsze kroki | Ograniczona inspekcja katalogów i nagłówków;[Aplikacje desktopowe](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Moduły, importy, mapy źródeł, trasy, IPC i powiązania z natywnymi dodatkami | Node.js i npm; [analiza aplikacji](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -143,8 +143,11 @@ O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As fer
|
||||
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Binários nativos | Pseudocódigo, assembly, strings, símbolos, chamadas e referências | Hopper, Ghidra ou IDA; [análise nativa](../native-investigation.md) |
|
||||
| Estrutura ELF sem execução | Seções, segmentos, símbolos/relocações originais e possíveis mecanismos de mitigação identificados por análise estática | pwntools fornecido pelo chamador no Linux x64; [diagnóstico de binários](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| Bytecode EVM | Seletores de despacho, offsets de bytes, argumentos inferidos e mutabilidade | Entrada local com bytes brutos ou representação hexadecimal no Linux x64; [guia de EVM sem execução](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Falhas do Linux registradas | Registros note brutos, registradores/sinais de cada thread registrada e candidatos opcionais a mapeamentos | pwntools fornecido pelo chamador no Linux x64; GDB/pwndbg opcionais; [falhas registradas](../recorded-crashes.md) |
|
||||
| Instalações desktop | Ambientes candidatos, arquivos de evidência, componentes e próximos passos | Inspeção limitada de diretórios e cabeçalhos;[Aplicações desktop](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Módulos, importações, source maps, rotas, IPC e relações com complementos nativos | Node.js e npm; [análise de aplicativos](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -145,6 +145,9 @@ REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а так
|
||||
| Структура ELF без запуска | Секции, сегменты, исходные символы/релокации и возможные механизмы защиты по данным статического анализа | pwntools, предоставленный вызывающей стороной, в Linux x64; [диагностика бинарных файлов](../binary-diagnostics.md) |
|
||||
| Байт-код EVM | Селекторы диспетчеризации, смещения байтов, предполагаемые аргументы и изменяемость состояния | Локальные необработанные байты или шестнадцатеричное представление в Linux x64; [руководство по EVM без запуска](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Записанные сбои Linux | Необработанные записи note, регистры/сигналы каждого записанного потока и необязательные кандидаты сопоставления с памятью | pwntools, предоставленный вызывающей стороной, в Linux x64; необязательные GDB/pwndbg; [записанные сбои](../recorded-crashes.md) |
|
||||
| Установленные настольные приложения | Возможные среды выполнения, файлы с доказательствами, компоненты и дальнейшие шаги | Ограниченная проверка каталогов и заголовков;[Настольные приложения](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Модули, импорты, карты исходного кода, маршруты, IPC и связи с нативными дополнениями | Node.js и npm; [анализ приложений](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -143,8 +143,11 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26
|
||||
| ---------------------------- | ------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ไบนารีเนทีฟ | โค้ดเทียม แอสเซมบลี สตริง สัญลักษณ์ การเรียก และการอ้างอิง | Hopper, Ghidra หรือ IDA; [การวิเคราะห์เนทีฟ](../native-investigation.md) |
|
||||
| โครงสร้าง ELF แบบออฟไลน์ | เซกชัน เซกเมนต์ สัญลักษณ์/รีโลเคชันต้นฉบับ และตัวเลือกกลไกป้องกันที่พบจากการวิเคราะห์แบบสถิต | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; [การวินิจฉัยไบนารี](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| ไบต์โค้ด EVM | ตัวเลือกฟังก์ชันสำหรับการเรียก ออฟเซ็ตไบต์ อาร์กิวเมนต์และความสามารถในการเปลี่ยนสถานะที่อนุมานได้ | ไฟล์บนเครื่องที่มีไบต์ดิบหรือข้อมูลเลขฐานสิบหกบน Linux x64; [คู่มือ EVM แบบออฟไลน์](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| ข้อมูลการแครชของ Linux ที่บันทึกไว้ | ระเบียน note ดิบ รีจิสเตอร์/สัญญาณของทุกเธรดที่บันทึกไว้ และตัวเลือกการแมปเพิ่มเติม | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; เลือกใช้ GDB/pwndbg เพิ่มเติมได้; [การแครชที่บันทึกไว้](../recorded-crashes.md) |
|
||||
| ไดเรกทอรีติดตั้งแอปเดสก์ท็อป | รันไทม์ที่เป็นไปได้ ไฟล์หลักฐาน ตัวบ่งชี้ส่วนประกอบ และขั้นตอนถัดไป | ตรวจสอบไดเรกทอรีและส่วนหัวโดยจำกัดขอบเขต;[แอปเดสก์ท็อป](../desktop-applications.md) |
|
||||
| JavaScript / Electron | โมดูล การนำเข้า ซอร์สแมป เส้นทาง IPC และความสัมพันธ์กับส่วนเสริมเนทีฟ | Node.js และ npm; [การวิเคราะห์แอปพลิเคชัน](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -147,6 +147,9 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç
|
||||
| Çevrimdışı ELF yapısı | Bölümler, segmentler, özgün semboller/yeniden konumlandırmalar ve statik analizden olası güvenlik önlemleri | Linux x64 üzerinde çağıranın sağladığı pwntools; [ikili dosya tanılama](../binary-diagnostics.md) |
|
||||
| Go ikili dosyaları | Gömülü compiler sürümü, modüller, replacement'lar ve build ayarları | Yerleşik statik ELF/PE/thin Mach-O incelemesi; Go kurulumu gerekmez |
|
||||
| Rust ikili dosyaları | Gömülü cargo-auditable crate adları, sürümleri, kaynakları ve bağımlılık ilişkileri | Yerleşik statik ELF/PE/thin Mach-O incelemesi; [Rust ikili dosyaları](../rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| WASM artefaktı | Doğrulama, bölümler, tam import/export WAT biçimleri ve decode edilmiş WAT | Çağıranın sağladığı WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bayt kodu | Yönlendirme seçicileri, bayt ofsetleri, çıkarımla belirlenen argümanlar ve durum değiştirilebilirliği | Linux x64 üzerinde yerel ham bayt/onaltılık girdi; [çevrimdışı EVM rehberi](../evm-bytecode.md) |
|
||||
| SQLite veritabanları | Tablolar, sütunlar, index'ler, view'lar, trigger'lar ve seçilen sıradan tablo satırları | Yerel database/WAL snapshot'ı; native SQLite sınırlarıyla Node.js 24.x >=24.15 veya 26+ |
|
||||
|
||||
@@ -143,8 +143,11 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та
|
||||
| ------------------------------------ | ----------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Нативні бінарні файли | Псевдокод, асемблерний код, рядки, символи, виклики й посилання | Hopper, Ghidra або IDA; [нативний аналіз](../native-investigation.md) |
|
||||
| Структура ELF без запуску | Секції, сегменти, початкові символи/релокації та можливі захисні механізми за даними статичного аналізу | pwntools, наданий стороною, що викликає інструмент, у Linux x64; [діагностика бінарних файлів](../binary-diagnostics.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| Байт-код EVM | Селектори диспетчеризації, зміщення байтів, виведені припущення про аргументи та змінюваність стану | Локальні необроблені байти або шістнадцяткове представлення у Linux x64; [посібник EVM без запуску](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Записані аварійні завершення в Linux | Необроблені записи note, регістри/сигнали кожного записаного потоку й необов'язкові кандидати зіставлення пам'яті | pwntools, наданий стороною, що викликає інструмент, у Linux x64; необов'язкові GDB/pwndbg; [записані аварійні завершення](../recorded-crashes.md) |
|
||||
| Встановлені настільні програми | Можливі середовища виконання, файли доказів, компоненти та наступні кроки | Обмежена перевірка каталогів і заголовків;[Настільні програми](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Модулі, імпорти, карти вихідного коду, маршрути, IPC і зв'язки з нативними доповненнями | Node.js і npm; [аналіз застосунків](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -145,6 +145,9 @@ REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với np
|
||||
| Bố cục ELF ngoại tuyến | Các section, segment, ký hiệu/thông tin tái định vị gốc và các biện pháp giảm thiểu có thể có từ phân tích tĩnh | pwntools do bên gọi cung cấp trên Linux x64; [chẩn đoán tệp nhị phân](../binary-diagnostics.md) |
|
||||
| Bytecode EVM | Bộ chọn điều phối, độ lệch byte, tham số suy luận và khả năng thay đổi trạng thái | Đầu vào cục bộ chứa byte thô hoặc dạng thập lục phân trên Linux x64; [hướng dẫn EVM ngoại tuyến](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| Sự cố Linux đã ghi lại | Bản ghi note thô, thanh ghi/tín hiệu của mọi luồng được ghi lại và các ứng viên ánh xạ tùy chọn | pwntools do bên gọi cung cấp trên Linux x64; GDB/pwndbg tùy chọn; [sự cố đã ghi lại](../recorded-crashes.md) |
|
||||
| Thư mục cài đặt ứng dụng máy tính | Các runtime có khả năng được dùng, tệp bằng chứng, dấu hiệu thành phần và bước tiếp theo | Kiểm tra thư mục và phần đầu tệp có giới hạn;[Ứng dụng máy tính](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](../javascript-application-workflows.md) |
|
||||
|
||||
+29
-26
@@ -141,32 +141,35 @@ AI 代理通過 MCP 呼叫 REA,檢查目標並追蹤相關程式碼。REA 返
|
||||
|
||||
REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。額外工具和主機支援取決於分析目標:
|
||||
|
||||
| 目標 | REA 返回的內容 | 要求與指南 |
|
||||
| --------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| 原生二進位檔案 | 偽程式碼、組合語言、字串、符號、呼叫和引用 | Hopper、Ghidra 或 IDA;[原生分析](../native-investigation.md) |
|
||||
| 離線 ELF 佈局 | 節、段、原始符號/重定位資訊及靜態防護機制候選項 | Linux x64 上由呼叫方提供的 pwntools;[二進位診斷](../binary-diagnostics.md) |
|
||||
| Go 二進位檔案 | 內嵌編譯器版本、模組、替換項和建置設定 | 內建靜態 ELF/PE/thin Mach-O 檢查;無需安裝 Go |
|
||||
| WASM 工件 | 驗證、節、精確的匯入/匯出 WAT 形式及解碼後的 WAT | 呼叫方提供的 WABT 1.0.42;`inspect_wasm_artifact` |
|
||||
| EVM 位元組碼 | 分派選擇器、位元組偏移、推斷的參數和狀態可變性 | Linux x64 上的本地原始位元組/十六進位輸入載體;[離線 EVM 指南](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| 已記錄的 Linux 崩潰 | 原始 note 記錄、每個已記錄執行緒的暫存器/訊號及可選的對映候選項 | Linux x64 上由呼叫方提供的 pwntools;可選的 GDB/pwndbg;[已記錄崩潰](../recorded-crashes.md) |
|
||||
| 桌面安裝目錄 | 執行階段候選、支持檔案、元件標記與後續操作 | 有界的目錄與檔頭檢查;[桌面應用](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm;[應用分析](../javascript-application-workflows.md) |
|
||||
| 網站 | 頁面結構、腳本、網路觀察結果和按請求獲取的螢幕擷取畫面 | Chrome 系瀏覽器;[瀏覽器分析](../browser-observation.md) |
|
||||
| 已保存的網路擷取 | 請求、回應、可訪問的酬載和來源位置 | HAR;原生 mitmproxy 擷取需要 Linux 上的 mitmdump;[擷取指南](../web-network-captures.md) |
|
||||
| 封包擷取 | 端點、DNS、TLS SNI、通訊協定分佈和 HTTP 請求 | 呼叫端提供的 tshark 和 capinfos;[pcap 指南](../pcap-analysis.md) |
|
||||
| .NET 程式集 | 中繼資料、CIL 指令、聲明的原生依賴和建置對比 | 靜態檢查;[托管程式碼指南](../managed-code-analysis.md) |
|
||||
| PyInstaller | CArchive, PYZ, Python bytecode | Static inspection; [Python packaged applications](../python-packaged-applications.md) |
|
||||
| Android APK | 清單聲明、類別、反編譯的方法和引用 | Linux/macOS/Windows x64 上的無界面 JADX 和完整 JDK;[Android 指南](../android-analysis.md) |
|
||||
| Android 裝置 | 裝置、套件、程序、日誌、傾印、螢幕擷取以及 APK/檔案傳輸 | 呼叫端提供的 adb(模擬器或實體裝置);[ADB 指南](../adb-device-analysis.md) |
|
||||
| JEB 專案 | 引擎支援的專案單元、成品摘要和型別/方法的虛擬碼 | 由呼叫端啟動、提供 MCP 的 JEB 用戶端,透過 REA_JEB_MCP_URL 選擇;[JEB 指南](../jeb-analysis.md) |
|
||||
| Android 資源 | 解碼後的清單、字串表、語系和版本資訊 | 呼叫端提供的 Apktool 啟動器;[資源指南](../apktool-resource-analysis.md) |
|
||||
| Flutter 應用 | Dart 快照雜湊、依 ABI 的程式庫雜湊及引擎建置來源 | 無(靜態 APK 解析);[Flutter 指南](../flutter-analysis.md) |
|
||||
| 韌體 | 區域、提取結果和轉交原生分析的內容 | Linux 上的 Binwalk / Unblob;[韌體指南](../firmware-analysis.md) |
|
||||
| 封包記錄 | 指定欄位、重複值、篩選條件與解碼規則 | Linux: TShark 4.x + prlimit; [封包分析指南](../packet-captures.md) |
|
||||
| 錄製的邏輯訊號 | 解碼註解、樣本範圍與診斷資訊 | Linux: sigrok-cli 0.7.2 + prlimit; [訊號記錄指南](../signal-recordings.md) |
|
||||
| 軟體包與資源 | 檔案清單、摘要、plist、Apple bundle 結構和提取的資源 | [製品與 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 應用](../apple-application-analysis.md) |
|
||||
| 處理程序行為 | 終端輸出、互動、退出和檔案系統觀察結果,以及執行對比 | 支援原生 PTY 的 Linux/macOS;[處理程序擷取](../process-capture.md) |
|
||||
| 目標 | REA 返回的內容 | 要求與指南 |
|
||||
| --------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| 原生二進位檔案 | 偽程式碼、組合語言、字串、符號、呼叫和引用 | Hopper、Ghidra 或 IDA;[原生分析](../native-investigation.md) |
|
||||
| 離線 ELF 佈局 | 節、段、原始符號/重定位資訊及靜態防護機制候選項 | Linux x64 上由呼叫方提供的 pwntools;[二進位診斷](../binary-diagnostics.md) |
|
||||
| Go 二進位檔案 | 內嵌編譯器版本、模組、替換項和建置設定 | 內建靜態 ELF/PE/thin Mach-O 檢查;無需安裝 Go |
|
||||
| WASM 工件 | 驗證、節、精確的匯入/匯出 WAT 形式及解碼後的 WAT | 呼叫方提供的 WABT 1.0.42;`inspect_wasm_artifact` |
|
||||
| EVM 位元組碼 | 分派選擇器、位元組偏移、推斷的參數和狀態可變性 | Linux x64 上的本地原始位元組/十六進位輸入載體;[離線 EVM 指南](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| 已記錄的 Linux 崩潰 | 原始 note 記錄、每個已記錄執行緒的暫存器/訊號及可選的對映候選項 | Linux x64 上由呼叫方提供的 pwntools;可選的 GDB/pwndbg;[已記錄崩潰](../recorded-crashes.md) |
|
||||
| 桌面安裝目錄 | 執行階段候選、支持檔案、元件標記與後續操作 | 有界的目錄與檔頭檢查;[桌面應用](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm;[應用分析](../javascript-application-workflows.md) |
|
||||
| 網站 | 頁面結構、腳本、網路觀察結果和按請求獲取的螢幕擷取畫面 | Chrome 系瀏覽器;[瀏覽器分析](../browser-observation.md) |
|
||||
| 已保存的網路擷取 | 請求、回應、可訪問的酬載和來源位置 | HAR;原生 mitmproxy 擷取需要 Linux 上的 mitmdump;[擷取指南](../web-network-captures.md) |
|
||||
| 封包擷取 | 端點、DNS、TLS SNI、通訊協定分佈和 HTTP 請求 | 呼叫端提供的 tshark 和 capinfos;[pcap 指南](../pcap-analysis.md) |
|
||||
| .NET 程式集 | 中繼資料、CIL 指令、聲明的原生依賴和建置對比 | 靜態檢查;[托管程式碼指南](../managed-code-analysis.md) |
|
||||
| PyInstaller | CArchive, PYZ, Python bytecode | Static inspection; [Python packaged applications](../python-packaged-applications.md) |
|
||||
| Android APK | 清單聲明、類別、反編譯的方法和引用 | Linux/macOS/Windows x64 上的無界面 JADX 和完整 JDK;[Android 指南](../android-analysis.md) |
|
||||
| Android 裝置 | 裝置、套件、程序、日誌、傾印、螢幕擷取以及 APK/檔案傳輸 | 呼叫端提供的 adb(模擬器或實體裝置);[ADB 指南](../adb-device-analysis.md) |
|
||||
| JEB 專案 | 引擎支援的專案單元、成品摘要和型別/方法的虛擬碼 | 由呼叫端啟動、提供 MCP 的 JEB 用戶端,透過 REA_JEB_MCP_URL 選擇;[JEB 指南](../jeb-analysis.md) |
|
||||
| Android 資源 | 解碼後的清單、字串表、語系和版本資訊 | 呼叫端提供的 Apktool 啟動器;[資源指南](../apktool-resource-analysis.md) |
|
||||
| Flutter 應用 | Dart 快照雜湊、依 ABI 的程式庫雜湊及引擎建置來源 | 無(靜態 APK 解析);[Flutter 指南](../flutter-analysis.md) |
|
||||
| 韌體 | 區域、提取結果和轉交原生分析的內容 | Linux 上的 Binwalk / Unblob;[韌體指南](../firmware-analysis.md) |
|
||||
| 封包記錄 | 指定欄位、重複值、篩選條件與解碼規則 | Linux: TShark 4.x + prlimit; [封包分析指南](../packet-captures.md) |
|
||||
| 錄製的邏輯訊號 | 解碼註解、樣本範圍與診斷資訊 | Linux: sigrok-cli 0.7.2 + prlimit; [訊號記錄指南](../signal-recordings.md) |
|
||||
| 軟體包與資源 | 檔案清單、摘要、plist、Apple bundle 結構和提取的資源 | [製品與 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 應用](../apple-application-analysis.md) |
|
||||
| 處理程序行為 | 終端輸出、互動、退出和檔案系統觀察結果,以及執行對比 | 支援原生 PTY 的 Linux/macOS;[處理程序擷取](../process-capture.md) |
|
||||
|
||||
靜態 JavaScript 和 .NET 檢查讀取提供的檔案,不執行應用。執行階段擷取會以你的使用者權限執行目標或與之互動;各執行階段指南說明瞭具體影響。
|
||||
|
||||
|
||||
+29
-26
@@ -141,32 +141,35 @@ npx rea-agents@latest setup
|
||||
|
||||
REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。额外工具和主机支持取决于分析目标:
|
||||
|
||||
| 目标 | REA 返回的内容 | 要求与指南 |
|
||||
| --------------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| 原生二进制文件 | 伪代码、汇编、字符串、符号、调用和引用 | Hopper、Ghidra 或 IDA;[原生分析](../native-investigation.md) |
|
||||
| 离线 ELF 布局 | 节、段、原始符号/重定位信息及静态防护机制候选项 | Linux x64 上由调用方提供的 pwntools;[二进制诊断](../binary-diagnostics.md) |
|
||||
| Go 二进制文件 | 内嵌编译器版本、模块、替换项和构建设置 | 内置静态 ELF/PE/thin Mach-O 检查;无需安装 Go |
|
||||
| WASM 工件 | 验证、节、精确的导入/导出 WAT 形式及解码后的 WAT | 调用方提供的 WABT 1.0.42;`inspect_wasm_artifact` |
|
||||
| EVM 字节码 | 分派选择器、字节偏移、推断的参数和状态可变性 | Linux x64 上的本地原始字节/十六进制输入载体;[离线 EVM 指南](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| 已记录的 Linux 崩溃 | 原始 note 记录、每个已记录线程的寄存器/信号及可选的映射候选项 | Linux x64 上由调用方提供的 pwntools;可选的 GDB/pwndbg;[已记录崩溃](../recorded-crashes.md) |
|
||||
| 桌面安装目录 | 运行时候选、支持文件、组件标记和后续操作 | 有界的目录与文件头检查;[桌面应用](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm;[应用分析](../javascript-application-workflows.md) |
|
||||
| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器;[浏览器分析](../browser-observation.md) |
|
||||
| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR;原生 mitmproxy 捕获需要 Linux 上的 mitmdump;[捕获指南](../web-network-captures.md) |
|
||||
| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos;[pcap 指南](../pcap-analysis.md) |
|
||||
| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查;[托管代码指南](../managed-code-analysis.md) |
|
||||
| PyInstaller | CArchive, PYZ, Python bytecode | Static inspection; [Python packaged applications](../python-packaged-applications.md) |
|
||||
| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK;[Android 指南](../android-analysis.md) |
|
||||
| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb(模拟器或实体设备);[ADB 指南](../adb-device-analysis.md) |
|
||||
| JEB 项目 | 引擎支持的项目单元、工件摘要和类型/方法的伪代码 | 由调用方启动、提供 MCP 的 JEB 客户端,通过 REA_JEB_MCP_URL 选择;[JEB 指南](../jeb-analysis.md) |
|
||||
| Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器;[资源指南](../apktool-resource-analysis.md) |
|
||||
| Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无(静态 APK 解析);[Flutter 指南](../flutter-analysis.md) |
|
||||
| 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob;[固件指南](../firmware-analysis.md) |
|
||||
| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](../packet-captures.md) |
|
||||
| 录制的逻辑信号 | 解码注释、样本范围和诊断信息 | Linux: sigrok-cli 0.7.2 + prlimit; [信号记录指南](../signal-recordings.md) |
|
||||
| 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 应用](../apple-application-analysis.md) |
|
||||
| 进程行为 | 终端输出、交互、退出和文件系统观察结果,以及运行对比 | 支持原生 PTY 的 Linux/macOS;[进程捕获](../process-capture.md) |
|
||||
| 目标 | REA 返回的内容 | 要求与指南 |
|
||||
| --------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| 原生二进制文件 | 伪代码、汇编、字符串、符号、调用和引用 | Hopper、Ghidra 或 IDA;[原生分析](../native-investigation.md) |
|
||||
| 离线 ELF 布局 | 节、段、原始符号/重定位信息及静态防护机制候选项 | Linux x64 上由调用方提供的 pwntools;[二进制诊断](../binary-diagnostics.md) |
|
||||
| Go 二进制文件 | 内嵌编译器版本、模块、替换项和构建设置 | 内置静态 ELF/PE/thin Mach-O 检查;无需安装 Go |
|
||||
| WASM 工件 | 验证、节、精确的导入/导出 WAT 形式及解码后的 WAT | 调用方提供的 WABT 1.0.42;`inspect_wasm_artifact` |
|
||||
| EVM 字节码 | 分派选择器、字节偏移、推断的参数和状态可变性 | Linux x64 上的本地原始字节/十六进制输入载体;[离线 EVM 指南](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| Erlang BEAM | IFF chunks, atoms, exports, and tables | Built-in static BEAM inspection; [Erlang BEAM](../erlang-beam.md) |
|
||||
| PE certificates | Authenticode WIN_CERTIFICATE entries and PKCS#7 signer/X.509 identity fields | Built-in static PE inspection; [PE certificates](../pe-certificates.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| 已记录的 Linux 崩溃 | 原始 note 记录、每个已记录线程的寄存器/信号及可选的映射候选项 | Linux x64 上由调用方提供的 pwntools;可选的 GDB/pwndbg;[已记录崩溃](../recorded-crashes.md) |
|
||||
| 桌面安装目录 | 运行时候选、支持文件、组件标记和后续操作 | 有界的目录与文件头检查;[桌面应用](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm;[应用分析](../javascript-application-workflows.md) |
|
||||
| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器;[浏览器分析](../browser-observation.md) |
|
||||
| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR;原生 mitmproxy 捕获需要 Linux 上的 mitmdump;[捕获指南](../web-network-captures.md) |
|
||||
| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos;[pcap 指南](../pcap-analysis.md) |
|
||||
| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查;[托管代码指南](../managed-code-analysis.md) |
|
||||
| PyInstaller | CArchive, PYZ, Python bytecode | Static inspection; [Python packaged applications](../python-packaged-applications.md) |
|
||||
| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK;[Android 指南](../android-analysis.md) |
|
||||
| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb(模拟器或实体设备);[ADB 指南](../adb-device-analysis.md) |
|
||||
| JEB 项目 | 引擎支持的项目单元、工件摘要和类型/方法的伪代码 | 由调用方启动、提供 MCP 的 JEB 客户端,通过 REA_JEB_MCP_URL 选择;[JEB 指南](../jeb-analysis.md) |
|
||||
| Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器;[资源指南](../apktool-resource-analysis.md) |
|
||||
| Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无(静态 APK 解析);[Flutter 指南](../flutter-analysis.md) |
|
||||
| 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob;[固件指南](../firmware-analysis.md) |
|
||||
| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](../packet-captures.md) |
|
||||
| 录制的逻辑信号 | 解码注释、样本范围和诊断信息 | Linux: sigrok-cli 0.7.2 + prlimit; [信号记录指南](../signal-recordings.md) |
|
||||
| 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 应用](../apple-application-analysis.md) |
|
||||
| 进程行为 | 终端输出、交互、退出和文件系统观察结果,以及运行对比 | 支持原生 PTY 的 Linux/macOS;[进程捕获](../process-capture.md) |
|
||||
|
||||
静态 JavaScript 和 .NET 检查读取提供的文件,不运行应用。运行时捕获会以你的用户权限运行目标或与之交互;各运行时指南说明了具体影响。
|
||||
|
||||
|
||||
@@ -71,7 +71,11 @@ Cutter commands use the local REA Python bridge and authenticated loopback messa
|
||||
The client and plugin must support the current v2 handshake; older peers are
|
||||
rejected before dispatch because they cannot preserve cross-caller recovery proof.
|
||||
The client checks the final encoded request against the bridge's 16 MiB frame
|
||||
limit before dispatch; an oversized command does not retain a cleanup owner.
|
||||
limit before dispatch; the 16 MiB budget applies to request payload bytes and
|
||||
excludes the terminating LF. The bridge accepts that full payload size and
|
||||
rejects larger or incomplete frames while bounding the read; an oversized
|
||||
command does not retain a cleanup owner. Hopper uses the same request payload
|
||||
budget.
|
||||
Each bridge request waits 25000 ms. Discovery admits at most 256 descriptor
|
||||
candidates and reads and probes at most eight concurrently. A larger directory
|
||||
reports `bridge_directory_capacity_exceeded`. This release has no runtime
|
||||
@@ -88,7 +92,11 @@ identify `cutter_command` and retain `status` in
|
||||
`details.diagnostics.auxiliary_operation`; malformed proof still blocks dispatch.
|
||||
Descriptor failures likewise retain the requesting operation through cleanup.
|
||||
Command admission reports typed cancellation and treats the caller's abort reason
|
||||
as opaque metadata, without inferring cleanup ownership from it.
|
||||
as opaque metadata, without inferring cleanup ownership from it. A generation
|
||||
mismatch is an input correction only when the authenticated reply confirms
|
||||
`not_started`. Missing or ambiguous execution state retains the command's
|
||||
completion uncertainty until terminal proof arrives, even if the reply names a
|
||||
generation error.
|
||||
|
||||
## Configuration and coverage
|
||||
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
# Plan
|
||||
|
||||
1. Export the capture parser from `environmentSource.ts` so a probe loads like a capture.
|
||||
2. `environmentCompareMatch.ts`: a probe request to a tool and its arguments.
|
||||
3. `environmentCompareIds.ts`: the id map, natural-key pairing, learning from two answers, translating arguments.
|
||||
4. `environmentCompareDiff.ts`: normalisation and categorised differences.
|
||||
5. `environmentCompare.ts`: per probe, reset the state, answer each request, compare and learn; the report.
|
||||
6. `rea environment-compare` in `src/cli/environmentCommands.ts`, documented in `docs/agent-environments.md` and
|
||||
`docs/cli.md`.
|
||||
7. Tests: matching, pairing, normalisation and categories as unit tests; the command over a fixture capture and probes.
|
||||
8. Verification on VMs: six applications brought up and seeded as for their capture, reset from a snapshot per sequence,
|
||||
driven by a script that records each sequence as a HAR; baseline reports per environment.
|
||||
@@ -0,0 +1,86 @@
|
||||
# Compare an environment with the application it came from
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
An environment is only as useful as it is faithful: an agent trained or graded on it should meet the behaviour of the
|
||||
application it was captured from. REA checks an environment only against its own capture, so nothing says how far it
|
||||
drifts from the application on calls the capture did not make, after writes, or on errors.
|
||||
|
||||
`rea environment-compare` measures that. It takes an environment (a manifest and its capture Evidence) and one or more
|
||||
**probe captures**, each a recording of one call sequence made against the live application, and reports how many of the
|
||||
probe's calls the environment answers as the application did, which tools disagree and how.
|
||||
|
||||
The command only reads Evidence: it makes no network call. Whoever records the probes drives the application with its own
|
||||
credentials and resets it between sequences.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Inputs: the manifest Evidence, its capture Evidence (`--evidence`), and one or more probe capture Evidence files
|
||||
(`--probe`, repeatable), each `inspect-web-network-capture` or `capture-browser-scenario` Evidence of one sequence.
|
||||
- Each probe is one sequence run against a freshly reset application. Before each probe the environment's state is reset,
|
||||
so sequences are independent on both sides.
|
||||
- Requests are taken in the probe's order (start time, then ordinal; ordinal alone when any record lacks a start
|
||||
time, as state extraction orders them). A request is matched to the manifest tool of the
|
||||
same method whose path template it fits with the most literal segments; format-rule parameters must fit their format
|
||||
once percent-decoded, as replay reads them.
|
||||
Query values the tool takes and a JSON body become the tool's arguments. A path argument is the raw segment: the
|
||||
environment decodes it once, so a segment encoded twice is not read as a second decoding. Replay matches that raw
|
||||
argument against a recorded segment that equals it, equals its one decoding, or decodes once to it.
|
||||
- A record without a usable request line (a redacted URL, a transaction the capture cannot attribute, an unparseable
|
||||
URL or no method) is not replayed, nor is a request to another origin than the probe's application (`other-origin`),
|
||||
which is the origin with the most JSON-like records, as the manifest chooses its own. A request with no recorded
|
||||
answer is made, since it may have changed the application, but not compared. Each is listed per probe with its
|
||||
reason and left out of agreement.
|
||||
- A request no tool matches is reported `no-tool`; a matched tool the environment does not offer, such as a write it
|
||||
withholds, is reported `not-served`. Neither is answered.
|
||||
- Ids differ between a re-seeded application and the capture. The comparison maps them:
|
||||
- seeded entities pair across the two answers by a natural key: a field present on every element of both arrays whose
|
||||
string values are unique on each side and shared between them;
|
||||
- identifier fields (`id`, `uuid`, `guid`, and names ending in `_id`, `Id`, `ID`, `_uuid` or `Uuid`) of paired objects,
|
||||
and of the two answers to the same call, map the live value to the environment's value; a create's two answers map
|
||||
the new ids, including a response that is only that id at the write plan's identity location. The `id`, `uuid` or
|
||||
`guid` of the answer's own entities (at the read view's entity or list location, a write's response location, or
|
||||
else the root) also maps within the collection the call names, since two collections can reuse a live id for
|
||||
different entities. An equal pair is kept in that collection, so another collection's mapping of the same live id
|
||||
does not replace it. A collection's map is a bijection: the first pairing of either side wins. Outside identifier
|
||||
fields, a string maps when the environment's state holds it as an id, and the keys of two objects keyed by ids map
|
||||
when their values pair (keys that partly overlap only when they are digits, UUIDs or their entries' identities, and
|
||||
every value pairs);
|
||||
- ids are learned only from an answer the environment gave this call, from its state or from a recording of exactly
|
||||
this call, its whole request line and its request body: a recording of another call that replay answers with
|
||||
instead holds other entities. A tool that takes a body does not match a recording whose body the probe did not send;
|
||||
- path, query and body values equal to a mapped live id are translated before the environment answers, a path
|
||||
parameter by the collection it names, except query values that page a list (`limit`, `offset`, `page` and the
|
||||
like), which count items rather than name one. A literal route segment is not translated. A nested collection is
|
||||
the live path's concrete parent values, each parameter decoded once, so two parents do not share one child map and
|
||||
spellings replay treats as equal name one. A second encoding stays a different parent. Learning and translation use
|
||||
that same live path.
|
||||
A path value the map does not know, and the environment's state does not hold, is reported `unmapped-id`.
|
||||
- A call is checked against the served tool's input schema first, as the MCP server checks it; a call it refuses is
|
||||
reported `refused`. A browser probe keeps its request bodies; one captured without response bodies is refused, since
|
||||
only statuses would compare, and so is one that makes POST, PUT or PATCH calls without recorded request bodies, since
|
||||
those calls would be made empty.
|
||||
- Answers compare after normalisation: empty response text equals no body (a JSON empty string stays one); mapped ids
|
||||
compare equal, in scalar sets as well, an entity's own identity within the call's collection and any other id by the
|
||||
probe's mapping, and an entity's own identity both sides hold verbatim differs when its collection's mapping gives it
|
||||
another environment value (a reference keeps raw equality, as the probe's mapping cannot tell which collection it
|
||||
names); an object's members compare under the environment's mapped keys only when it is keyed by learned ids as
|
||||
learning reads one, and any other object compares field by field; two clock values (ISO-8601 strings, or epoch numbers under
|
||||
time-named fields) compare by format only; two different generated values (both UUIDs, or both of one length and the
|
||||
same character classes with a digit) that neither side maps compare equal and are counted. A body over the replay
|
||||
limit, text or binary, compares by status only, as does a live body the probe did not record, unless the answer has
|
||||
none by definition (204, 205, 304, or an answer to `HEAD`). Both are counted.
|
||||
- A step agrees when the statuses are equal and the normalised bodies are. Differences are categorised: `status`,
|
||||
`missing-field` (the application had it), `extra-field`, `value`, `length`, `order`, `clock-format`, `refused` (the
|
||||
environment refused the call), `not-served`, `no-tool` and `unmapped-id`, each with a JSON pointer and short values.
|
||||
- The report gives agreement overall, per probe and per tool, agreement ignoring item order beside it, the counts per
|
||||
category, the ids mapped and the values normalised, the records left uncompared, and each step's differences (at most
|
||||
20 per step).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No network access, no credentials, no reset of the application: probes are recorded outside REA.
|
||||
- No generation of call sequences; a later change generates them where the environment disagrees.
|
||||
- No change to how environments serve.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the outcome, requirements and non-goals.
|
||||
- [x] Load probes as captures.
|
||||
- [x] Match probe requests to tools.
|
||||
- [x] Map ids and translate arguments.
|
||||
- [x] Normalise and categorise differences.
|
||||
- [x] Compare per probe and report.
|
||||
- [x] Add the command and its documentation.
|
||||
- [x] Add unit and command tests.
|
||||
- [x] Run the gates.
|
||||
- [x] Measure six environments against their live applications and record the results.
|
||||
@@ -0,0 +1,146 @@
|
||||
# Verification
|
||||
|
||||
Verified commit: 235b22f2 on the write-plan branch at 7056f5ff. Executed on macOS with Node v22.23.1 (the repository
|
||||
pins 24.18.0 in `.nvmrc`, which was not installed); the live applications ran on six disposable Linux VMs.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run build:cached`, `npm run check:fast` (typecheck and lint), `npm run format:check`, `npx knip`,
|
||||
`npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery` and `npm run mcp-catalog:check`: passed.
|
||||
- `npm run jscpd`: 3.46% duplicated lines over 3609 files, under the 3.5% gate.
|
||||
- `npx vitest run --project domain --project services`: 7487 passed, 5 pre-existing skips.
|
||||
- `npm run test:prepare`, then `npm run test:focused --` the environment-compare, export and serve CLI boundaries, the
|
||||
serve MCP boundary and the tool schema validity boundary: 5 files, 30 passed.
|
||||
|
||||
New tests: matching, id mapping and translation, pairing guards, normalisation and every category as unit tests
|
||||
(`environmentCompareIds`, `environmentCompareDiff`, `environmentCompare`, 19 tests), and the command through the built
|
||||
CLI (`environmentCompareCli`, 2 tests): a probe of the books application re-seeded under other ids agrees on every step
|
||||
with 2 ids mapped, and a probe that is not capture Evidence is refused with its index.
|
||||
|
||||
## Six environments against their live applications
|
||||
|
||||
Six published environments were compared with the application each was captured from: NetBox 4.7.3, Komga 1.29.1,
|
||||
Mealie 3.28.0, n8n 1.123.85, Shlink 5.1.7 and self-hosted Sentry 26.9.0, the same images, brought up and seeded by the
|
||||
same scripts as for their captures, on loopback. Each was snapshotted after seeding and restored from that snapshot
|
||||
before every sequence (7 to 80 s), so every sequence met the same application.
|
||||
|
||||
A driver outside REA ran three kinds of sequence per application, each starting with the capture's read-only prefix:
|
||||
the capture re-run in order; per served write type, a list, create, read, update, read, delete, read and list, then a
|
||||
second create with changed free text; and a read of every seeded entity through its item tool. It translated the
|
||||
environment's ids to the application's as it learned them, signed in with the application's own credentials, and wrote
|
||||
each sequence as a HAR without headers or cookies, which `inspect-web-network-capture` turned into probe Evidence. The
|
||||
environments' manifests use `prefix-variance`, so the comparison ran on a local build of this branch with that rule's
|
||||
branch merged.
|
||||
|
||||
| Environment | Steps | Agreement | Ignoring order | Against its own capture |
|
||||
| ----------- | ----: | --------: | -------------: | ----------------------: |
|
||||
| NetBox | 370 | 99.2% | 99.2% | 100% |
|
||||
| Komga | 356 | 93.8% | 93.8% | 93.1% |
|
||||
| Mealie | 301 | 96.0% | 97.0% | 97.1% |
|
||||
| Shlink | 133 | 89.5% | 92.5% | 92.4% |
|
||||
| n8n | 202 | 74.8% | 87.6% | 87.5% |
|
||||
| Sentry | 230 | 23.9% | 46.5% | 84.7% |
|
||||
|
||||
The last column compares each environment with its own capture as the probe: what it cannot reproduce even of the
|
||||
session it was built from. The disagreements fall into four groups.
|
||||
|
||||
- Environment gaps, the ones to fix:
|
||||
- write tools the environment withholds, and the reads after them: readlists and collections (Komga), recipes (Mealie), short URLs
|
||||
(Shlink), workflow and tag writes (n8n);
|
||||
- preconditions: n8n refuses to delete a workflow that is not archived and the environment deletes it;
|
||||
- lists: a create does not appear in a list that keeps its recorded members (Komga), a filtered list is answered
|
||||
without the filter (Shlink), a replayed search ignores its body (Komga);
|
||||
- a domain-scoped short URL answered without its `domain` parameter, where Shlink answers 404;
|
||||
- state rendering: a project field two recorded views disagree on is rendered with one value, and an organisation
|
||||
read whose shape depends on a query parameter is answered the same either way (Sentry);
|
||||
- a tag read refused for want of the fields its item view always carried (Mealie);
|
||||
- time: per-hour event counts move with the current time, while the environment's clock is frozen at the capture
|
||||
(Sentry);
|
||||
- a field the capture's scrubbing replaced, answered as `"<scrubbed>"` (Sentry).
|
||||
- The application differing from itself between seedings: Sentry returns permission lists in no fixed order and
|
||||
reseeding changed release first-event times, feature lists and profile flags; n8n's executions and NetBox's running
|
||||
worker count differ from the captured instance.
|
||||
- The probe observing itself: Komga and Sentry report the driver's own sign-in user agent.
|
||||
- Driver gaps: ids the driver never mapped (Mealie units, Sentry events by hash) were sent as the environment's ids.
|
||||
|
||||
## What the runs fixed in the comparison
|
||||
|
||||
Each live run found a way the comparison itself was wrong, fixed with a test before the next run: requests are matched
|
||||
to tools in the environment's terms, so a tool whose path holds a seeded id literally matches; ids are learned outside
|
||||
identifier fields only from strings, after NetBox's worker count of 1 mapped a live 0 to entity 1; items paired only by
|
||||
position teach ids only when no key is shared and the lengths agree, after a filtered Shlink list mapped one short URL to
|
||||
another; objects keyed by ids pair their entries (n8n data tables); reordered scalar sets differ in order only and epoch
|
||||
numbers in arrays are clocks (Sentry); and bodies over the replay limit compare by status.
|
||||
|
||||
## After the review round
|
||||
|
||||
The branch merged the write-plan branch at b6c3b168 (four serving and gate fixes its review found), then changed the
|
||||
comparison in nine places: ids are learned only from answers to the call itself and also per collection, paging values
|
||||
are never translated, calls go through the served tools' input schemas, browser probes keep their request bodies, empty
|
||||
bodies and large binary bodies compare as the server answers them, scalar sets compare after mapping, and id-keyed
|
||||
objects pair when some keys overlap. Each new test fails on the previous head. The gates above passed again (domain and
|
||||
services 7504 passed with 5 pre-existing skips; the focused boundaries 80; conformance; `jscpd` 3.46%). Re-run on the
|
||||
same probes with this branch and the template rule's merged, the six environments compare exactly as in the table above,
|
||||
step for step, with the same ids mapped and the same values normalised.
|
||||
|
||||
A second round, on Greptile's review of those changes, learned ids from a replay only when the whole recorded line and
|
||||
request body match the call, compared an entity's own identity within the call's collection and other ids by the probe's
|
||||
mapping, read overlapping keys as ids only when they evidently are ids, and kept a JSON empty string apart from empty
|
||||
text. Its four tests fail on the previous head; the six environments again compare exactly as in the table.
|
||||
|
||||
A third round located the answer's own entities where the served environment does, at the read view's entity or list
|
||||
pointer or the write's response pointer, so an entity inside an envelope keeps its collection's mapping. Its test fails
|
||||
on the previous head; the gates passed again (domain and services 7509, boundaries 80, conformance, `jscpd` 3.46%) and
|
||||
the six environments again compare exactly as in the table.
|
||||
|
||||
A fourth round, on Greptile's next review, kept what a probe did not observe out of the agreement figures. Records
|
||||
without a usable request line, and requests without a recorded answer, are listed per probe with the reason and left
|
||||
uncompared. A live body the probe did not record compares by status only and is counted. A browser probe captured
|
||||
without response bodies is refused. A format-rule path parameter fits by its decoded value, as replay reads it. The
|
||||
branch also merged the write-plan branch at 1d1c4787, whose gate no longer confirms a write by a read of a state that a
|
||||
later write replaced. Each new test fails on the previous head. The gates passed again (domain and services
|
||||
7647, 5 pre-existing skips; the compare, export, serve and analysis-view boundaries 69; conformance 279; `jscpd` 3.44%).
|
||||
|
||||
On the same probes, the six environments have no unobserved records or bodies, and five compare exactly as before.
|
||||
Komga now withholds its collection create and patch. The only read the gate could check them by is the 404 after the
|
||||
collection is deleted, because the read after the patch is the patch's own observation. Its agreement falls from
|
||||
96.9% to 93.8%: the three creates and two patches the probes make are not served, and the six reads and deletes of
|
||||
those collections that follow fail. Against its own capture it falls from 96.6% to 93.1%. The table shows the new
|
||||
figures.
|
||||
|
||||
A fifth round merged the write-plan branch at dc5e2aad, which weighs an update's derived changes when a later write
|
||||
replaces it and resets collection spellings on reset or load. It also makes a request whose answer the probe lost,
|
||||
still uncompared, so that a later read sees its effect: a lost delete followed by a 404 now agrees. Its test fails on
|
||||
the previous head. The six environments compare as in the table.
|
||||
|
||||
A sixth round merged the write-plan branch at fe8ab584, whose gate erases the fields later writes and recorded
|
||||
outcomes overwrite from earlier write effects. The six environments again compare exactly as in the table.
|
||||
|
||||
A seventh round merged the write-plan branch at 3580d85c and main d8a9673f. That brought the gate's fix for overwritten
|
||||
document writes, which Greptile raised here against the write-plan code, among others. It also fixed two findings of
|
||||
its own:
|
||||
|
||||
- a probe with any undated record keeps its recorded order, as state extraction does;
|
||||
- a browser probe that makes POST, PUT or PATCH calls without recorded request bodies is refused.
|
||||
|
||||
Both tests fail on the previous head. The six environments compare exactly as in the table, on the probes and against
|
||||
their own captures.
|
||||
|
||||
An eighth round merged the write-plan branch at 43d98ea9 and 264ce410. They bring the gate's fixes for three findings
|
||||
Greptile raised here against the write-plan code: a passing read that shows nothing of a write leaves it pending, an
|
||||
update or delete finds its entity as an item read does, and a built document plan writes later calls at its own
|
||||
object. It also fixed one finding of its own: a request to another origin than the probe's application is neither
|
||||
made nor compared, so another server's `DELETE /api/books/2` no longer deletes the environment's book. Its test fails
|
||||
on the previous head. The six environments compare exactly as in the table, on the probes and against their own
|
||||
captures, with no unobserved records.
|
||||
|
||||
It then merged the write-plan branch at 0ae64400 and 6257de93: an update's identity member never takes a rule, a page
|
||||
size of zero bounds nothing, and the paging rules, with this branch's `isPagingParameter`, move to
|
||||
`environmentStatePaging.ts`. The six environments again compare exactly as in the table.
|
||||
|
||||
A ninth round merged the write-plan branch at 466ed8b4, which fixes two findings Greptile raised here against its
|
||||
code (entity fields that look like pointers match by name; a zero page size bounds nothing only where a recording
|
||||
listed items under it) among others. The six environments compare exactly as in the table, on the probes and against
|
||||
their own captures. It then merged 7cef2fb2, whose fixes settle three more findings Greptile raised here against that branch's code:
|
||||
zero page sizes erasing another bound, partial write responses taken for removals, and interference timed by a tool's
|
||||
call elsewhere. The six environments again compare exactly as in the table.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Implementation plan
|
||||
|
||||
1. **Gate.** Judge each latest list recording on its own for paging-only query, totals, next page and state order;
|
||||
keep `all` when every line passes and otherwise name the passing lines in `all_record_ordinals`; reproduce each
|
||||
line with its own membership.
|
||||
2. **Counters.** Count `number`-of-items members as item totals.
|
||||
3. **Consumers.** One helper says whether a view lists all on a line; serving, the write gate's list checks and the
|
||||
insert retry use it, and an argument-less call may answer on any such line.
|
||||
4. **Absences.** When a read shows an entity absent, drop it from every write's claim on its collection, so a write
|
||||
left with none no longer counts there.
|
||||
5. **Tests and docs.** Domain and serving tests as the acceptance lists; the membership row of the environment-state
|
||||
docs and the list answer row of the agent-environment docs.
|
||||
@@ -0,0 +1,49 @@
|
||||
# Decide a list's whole-collection membership per line
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
A list view reproduces its lines from the state with one of two memberships: `all`, every entity of the collection in
|
||||
state order, or `recorded`, the entities each recording listed. The view was `all` only when every checked line listed
|
||||
its whole collection. A list tool often also has a filtered line (`?library_id=…`, `?name=…`), and that one line kept
|
||||
every line of the tool to its recorded members. An entity the environment created then never appeared in a list,
|
||||
though the unfiltered line of the same tool had shown the application listing every entity.
|
||||
|
||||
Each checked line is now judged on its own. A recorded-membership view names, in `all_record_ordinals`, the checked
|
||||
lines that each meet the conditions `all` asks of every line: the line has no query or only paging parameters starting
|
||||
at zero, lists
|
||||
exactly the entities of its collection in state order, and shows no total that differs from the number listed and no
|
||||
next page. A call answered on such a line lists every entity of the collection, the write gate checks a read on it
|
||||
against the whole collection, and a create's insert rule may be retried on it, as for an `all` view. Other lines keep
|
||||
their recorded members. Without such a line, or when every line is one, the view is unchanged and the member is absent,
|
||||
so existing states keep their values.
|
||||
|
||||
Positive or malformed `offset`, `start`, `_start`, and `page` positions do not establish whole membership, even for an
|
||||
empty collection. Positive page numbers are ambiguous without a recorded zero-/one-based convention. These lines keep
|
||||
recorded membership rather than inventing earlier items after a write or `data/add`; all paging values must be safe
|
||||
nonnegative integers. Repeated positions must all be zero.
|
||||
|
||||
A whole line's read checks the collection, so it exercises every write that changed the collection since it was last
|
||||
read. A write whose entity a read has since shown absent, for example a create an unmodelled delete removed and a 404
|
||||
then showed gone, no longer counts there, as when a modelled delete replaces it. The same holds when a failed item check
|
||||
takes the read's answer and that answer holds no entity. Otherwise a whole list taken after the entity was gone would
|
||||
confirm a create it cannot show.
|
||||
|
||||
`numberOfElements` and similar number-of-items members (`number` with `elements`, `entries`, `items` or `results`)
|
||||
count the listed entities, so a whole line's counter follows the state as `totalElements` does. A view with whole lines
|
||||
takes the counters that equal the number listed on each of them, since a partial page's total counts more than the page;
|
||||
a view without one keeps those that do on every line. A line is answered with each counter set to the number listed
|
||||
only when its own recording showed that counter equal to the number it listed, so a partial page keeps its recorded
|
||||
total.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Domain: a tool with a whole line and a filtered line names the whole one; a create the whole line later lists is
|
||||
confirmed; a tool whose lines are all whole, or none, names none; a whole list taken after a read showed the created
|
||||
entity gone confirms no create.
|
||||
- Serving: after a served create, the whole line lists the created entity with its counters, and the filtered line
|
||||
keeps its recorded members; beside a partial page whose total counts the collection, the whole line's total follows
|
||||
the state and the page keeps its recorded total.
|
||||
- Measured: the six published environments compared with the application before and after, on the probes of the
|
||||
fidelity comparison and the held-out sessions recorded since.
|
||||
@@ -0,0 +1,10 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Judge whole-collection membership per checked line.
|
||||
- [x] Count number-of-items members.
|
||||
- [x] Serve, check and place creates per line.
|
||||
- [x] Drop a write from its collection once a read shows its entity absent.
|
||||
- [x] Add domain and serving tests.
|
||||
- [x] Document per-line membership.
|
||||
- [x] Measure the six published environments before and after.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
@@ -0,0 +1,103 @@
|
||||
# Verification
|
||||
|
||||
Verified commit: 40800631 (this branch with the write-plan branch's head `dc5e2aad` merged in). Executed on macOS with
|
||||
Node v22.23.1 (the repository pins 24.18.0 in `.nvmrc`, which was not installed); the probes were recorded earlier
|
||||
against the live applications on six disposable Linux VMs.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run build:cached`, `npm run check:fast` (typecheck and lint), `npm run format:check`, `npx knip`,
|
||||
`npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery` and `npm run mcp-catalog:check`: passed.
|
||||
- `npm run jscpd`: 3.45% duplicated lines over 3646 files, under the 3.5% gate.
|
||||
- `npx vitest run --project domain --project services`: 7616 passed, 5 pre-existing skips.
|
||||
- `npm run test:prepare`, then `npm run test:focused --` the environment serve and export CLI boundaries, the serve
|
||||
and served-writes MCP boundaries, the analysis-view CLI and MCP boundaries, the compact input schema boundary and
|
||||
tool schema validity: 8 files, 78 passed.
|
||||
|
||||
New tests: three domain tests in `environmentStateWrites` (a tool with a whole line and a filtered line names the whole
|
||||
one, with `numberOfElements` and `totalElements` as counters, and a create the whole line lists is confirmed; a whole
|
||||
list taken after a read showed the created entity gone confirms no create, which fails without the absence rule; a
|
||||
tool whose lines are all whole, or none, names none) and one serving test in `environmentStateWriteAnswer` (after a served
|
||||
create, the whole line lists the created book with its counters, and the filtered line keeps its recorded member).
|
||||
|
||||
## Fifty published captures
|
||||
|
||||
The environment-state view ran over the 50 published captures with each manifest's template rules, on builds with and
|
||||
without this change. Both merged the prefix-variance branch, which some manifests need.
|
||||
|
||||
- 47 list views in 20 environments gain a whole line, among them Komga's books, series, collections and read lists,
|
||||
BookStack's books, chapters and pages, Mealie's cookbooks, shopping lists, categories and tags, WordPress's posts and
|
||||
comments, Vikunja's labels and Paperless's groups. On that line a call now lists the state's entities, including
|
||||
those a task loads with `data/add` and those served writes create or delete, where it listed only the recorded
|
||||
members. Three of them belong to a type with a served write: Airbyte's tags, BookStack's books and Linkding's tags.
|
||||
- Served write tools stay at 31, with two changes:
|
||||
- Airbyte's tag delete is now served. A whole list after the delete shows the tag gone.
|
||||
- Sentry's release create is now unconfirmed, in the pool's earlier manifest, where the release's update and delete
|
||||
are literal tools and not modelled. Its only confirmation had been a whole list taken after the delete and a 404
|
||||
showed the release gone. The write-plan branch already refuses such a list after a modelled delete; this change
|
||||
refuses it after an observed absence too.
|
||||
- Komga's list views also count `numberOfElements`. No other view or write plan changed.
|
||||
- Every write verdict is unchanged for the six environments republished with newer manifests (NetBox, Komga, Mealie,
|
||||
n8n, Shlink, Sentry). Komga gains 6 whole-line lists, Mealie 4 and Sentry 1.
|
||||
|
||||
## Six environments against their applications
|
||||
|
||||
The six environments of the fidelity comparison (NetBox, Komga, Mealie, n8n, Shlink, self-hosted Sentry) were compared
|
||||
with their applications by `environment-compare` with and without this change. The probes were the comparison's capture
|
||||
re-run, create/read/update/delete scenario and read of every seeded entity, and held-out sessions recorded later with
|
||||
values no capture sent. Both builds merged the environment-compare and prefix-variance branches.
|
||||
|
||||
| Environment | Steps | Before | After | Ignoring order, before | After |
|
||||
| ----------- | ----: | -----: | ----: | ---------------------: | ----: |
|
||||
| NetBox | 479 | 98.5% | 98.5% | 98.5% | 98.5% |
|
||||
| Komga | 600 | 91.5% | 91.5% | 91.5% | 91.5% |
|
||||
| Mealie | 449 | 94.0% | 94.0% | 94.9% | 94.9% |
|
||||
| n8n | 278 | 71.6% | 71.6% | 84.2% | 84.2% |
|
||||
| Shlink | 202 | 87.1% | 87.1% | 90.1% | 90.1% |
|
||||
| Sentry | 372 | 24.5% | 24.5% | 47.6% | 47.6% |
|
||||
|
||||
No step changed. This Komga environment's capture creates, updates and deletes a collection before any whole list, so
|
||||
with the write-plan branch's head its create and update are unconfirmed and serve nothing a list could show. Before
|
||||
that head (at `f2b95351`), when a list taken after the delete still confirmed them, the same comparison moved Komga
|
||||
from 94.7% to 95.2%: three list reads after creates agreed and two more differed only in where the created collection
|
||||
was placed. None of the other five environments has a list tool with both kinds of line and a served write on its type.
|
||||
|
||||
## Review round
|
||||
|
||||
The branch merged the write-plan branch at 73c0e9b5 and 6d28aa78 and main at 4dfa1c67. Six of Greptile's eight
|
||||
findings were in the write-plan branch's code and were fixed there. This branch fixed the other two:
|
||||
|
||||
- a view with whole lines takes its counters from them, and each line updates only the counters its own recording
|
||||
counted, so a partial page no longer drops a whole line's total;
|
||||
- a failed item check that takes an answer with no entity drops the writes that claimed it from its collection, as an
|
||||
absence does, so a whole list cannot confirm the create.
|
||||
|
||||
Both new tests fail on the previous head. Over the 50 published captures, the merges changed no view or write plan,
|
||||
and these two fixes add one counter each to BookStack's pages (`total`) and Linkding's tags (`count`), both views with
|
||||
a whole line beside partial ones. Served write tools stay at 31. The six environments compare exactly as in the table
|
||||
above, step for step. The gates passed again.
|
||||
|
||||
A second round merged the write-plan branch at 43d98ea9, which carries main d8a9673f and that branch's fixes for the
|
||||
four findings Greptile raised here against its code. Its page cap applies on every line that lists its whole
|
||||
collection: a call naming such a page that the collection has outgrown answers from its recording, and the gate skips
|
||||
such reads. Over the 50 captures no view or write plan changed and served write tools stay at 31; the six environments
|
||||
compare exactly as before.
|
||||
|
||||
A third round merged the write-plan branch at 264ce410. Greptile's finding here, that a `_start`–`_end` range escaped
|
||||
the page cap, was in that branch's code and was fixed there: the range's width bounds a page as `limit` does, so a
|
||||
whole line naming a range the collection has outgrown answers from its recording, and the gate checks nothing by it.
|
||||
The merge also brings that branch's other fixes: unseen writes stay pending, later creates follow their sequences, and
|
||||
later document writes stay at their object. Over the 50 captures no view or write plan changed and served write tools
|
||||
stay at 31; the six environments compare exactly as before.
|
||||
|
||||
It then merged that branch at 0ae64400 and 6257de93: an update's identity member never takes a rule, a page size of
|
||||
zero bounds nothing, and the paging rules move to `environmentStatePaging.ts`. Over the 50 captures again no view or
|
||||
write plan changed, and the six environments compare exactly as before.
|
||||
|
||||
A fourth round merged the write-plan branch at 466ed8b4, which fixes four findings Greptile raised here against its
|
||||
code: an update whose observation drops a field is refused, entities are named as serving names them, a later
|
||||
create's generated values must keep their forms, and a derived state too large to load back is not served. The page
|
||||
cap here now uses that branch's `pageBound`. Over the 50 captures no view or write plan changed and served write tools
|
||||
stay at 31; the six environments compare exactly as before. It then merged 7cef2fb2, whose fixes settle two more findings Greptile raised here against that branch's code: zero
|
||||
page sizes erasing positive bounds, and interference timed by a tool's call on another collection. Over the 50
|
||||
captures again no view or write plan changed, and the six environments compare exactly as before.
|
||||
@@ -0,0 +1,14 @@
|
||||
# Implementation plan
|
||||
|
||||
1. **Query module.** A domain module holds the roles, explains one line (trying roles by parameter name, filters and
|
||||
inert for any parameter, at most 256 combinations), merges lines, evaluates a query over entities, and renders an
|
||||
answer with counters and totals.
|
||||
2. **Gate.** The list plan learns the query from the lines that are not whole, reading each body once for its listed
|
||||
identities, role choices and counts; demotes a line whose envelope or fields do not reproduce; and adds `query` to
|
||||
the view.
|
||||
3. **Write gate.** A read on a query line is checked against the query over the replayed state, covering the
|
||||
collection only for sort- and page-only answers that hold every match.
|
||||
4. **Serving.** Line selection moves to its own module and gains the query line; list answers choose whole,
|
||||
query or recorded members and render totals.
|
||||
5. **Tests and docs.** Unit, domain and serving tests; the environment-state and agent-environment docs and the
|
||||
analysis-view tool description.
|
||||
@@ -0,0 +1,86 @@
|
||||
# Answer list filters, search, sorting and paging from the state
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
Agents list, filter, search and sort constantly. A list view answered a recorded request line from the state only when
|
||||
that line listed the whole collection; any other line, such as `?shelf=a`, `?q=tide` or `?orderBy=name`, kept the
|
||||
members it recorded, and a call with a value no line recorded replayed whichever recording was closest. A filtered
|
||||
list therefore never showed what a write created or changed, and a search for anything new answered with an
|
||||
unrelated recording.
|
||||
|
||||
A list view now learns what each of its query parameters does, from the lines it recorded, and answers those lines,
|
||||
and calls with new values, by evaluating the query over the live state.
|
||||
|
||||
## Learning
|
||||
|
||||
For each checked line that does not list the whole collection, the first combination of roles that lists exactly what
|
||||
the line recorded, from the collection's initial entities, explains it:
|
||||
|
||||
- `filter` on a field every listed item holds the value in: equal as text, or in an array directly or as an object's
|
||||
`id`, `slug` or `name`;
|
||||
- `search` for plain substring terms over the string fields the term matched in the listed items, ignoring case (URL fields only for an item
|
||||
nothing else matched); an empty term matches everything, and a term that matched no field leaves the parameter inert
|
||||
until another line's term does;
|
||||
- `sort` by the field the value names, with `-` or an `asc`/`desc` suffix, and `direction` for a separate `asc`/`desc`
|
||||
parameter: numbers by value, text ignoring case, null last ascending, ties (case-only differences included) in state
|
||||
order; an empty selection needs no sort;
|
||||
- `limit`, `offset` and `page` (counted from 1 or 0 only when the line distinguishes them) for paging; zero or a negative limit
|
||||
lists every match;
|
||||
- `inert` for a parameter that changes nothing, valid only for the values it recorded.
|
||||
|
||||
Search values resembling operators, field syntax, tags, quotes or wildcards replay rather than asserting a literal-substring result.
|
||||
Paging values must be safe integers; malformed positions are not learned as inert flags. Identical decoded repeats
|
||||
of a query value impose one constraint; conflicting repeated values remain unknown. Counter presence belongs to
|
||||
each response envelope: page counters update where observed, and serving never adds a query-only total to another envelope.
|
||||
Query totals retain their numeric or canonical decimal-string type. Unknown or differently formatted strings
|
||||
(such as padded `"03"`) are not assigned an unverified formatting rule and stay as recorded.
|
||||
`state.source` distinguishes a computed response from recording replay; `replay.exact_match` describes only the
|
||||
retained source envelope and can be false for a correctly computed new query.
|
||||
|
||||
Parameter names decide which roles are tried (`sort` for `orderBy`, `search` for `q`), and any parameter may be a
|
||||
filter or inert. No two parameters of one call take the same `sort`, `direction` or paging role: the answer would
|
||||
depend on which applied last, and serving supplies a call's values in name order rather than as recorded, so such a
|
||||
combination explains no line and such a call replays its recording. A line with more than 12 query parameters is left
|
||||
unexplained, which bounds the search; names such as `__proto__` are kept as the query's own members. The lines' roles
|
||||
merge: lines that disagree drop the parameter, inert values gather, search fields unite. The merged query keeps the
|
||||
lines it reproduces, envelope and fields included; any other line keeps its recorded members. The view's `query` names
|
||||
the roles, those lines, and the envelope members that count every match before paging (`totals`); a `number`-of-items
|
||||
member such as `numberOfElements` counts the page.
|
||||
|
||||
## Serving and the write gate
|
||||
|
||||
- A call that names a query line exactly, or whose every query argument has a role (an argument-less call keeps
|
||||
answering on a whole line), is answered by evaluating the query over the state, in the envelope of a query or whole
|
||||
line of its collection, with counters at the number listed and totals at the number matched. The answer's state
|
||||
provenance says `membership: "query"`.
|
||||
- A value an inert parameter never recorded, or a parameter without a role, replays its recording as before.
|
||||
- The write gate checks a read on a query line, or one whose every query parameter has a role on a line it did not
|
||||
name, against the query over its copy of the state, as serving answers it: on the line replay ranks first, by the
|
||||
most verbatim path values, then the most query values the read shares, then recorded order. Only a line whose
|
||||
parameters all sort or page, and whose answer holds every match, covers the collection; any other covers the
|
||||
entities it lists and those the line or the read listed that the state no longer holds, so a filtered list shows
|
||||
whether a delete or create took effect without speaking for the rest of the collection. A failed read also covers
|
||||
the held entities it listed that the answer leaves out, as a write may have moved them out of it. A read on a line
|
||||
it did not name counts as gone only what it listed itself, since the line lists another call's members. A read with
|
||||
no query answered on a query line, as serving does when no whole line fits, is checked against that line's recorded
|
||||
members, as serving lists them. A read on a line that keeps its recorded members is compared with the totals it
|
||||
recorded, as serving answers it. A read on a line it did not name is compared in its own envelope: that line's
|
||||
envelope, such as a `next` link naming the line's own values, comes from another call, so it says nothing of a
|
||||
write. A read whose query applies a sort shows where a create put its entity only when the sort leaves that entity
|
||||
tied with another, and an unsorted one only when more than its entity matches, so only then does it fix the create's
|
||||
insert position.
|
||||
|
||||
The roles are inferences from recorded behaviour and are stated as such. Nothing changes for a list whose lines all
|
||||
list the whole collection, or that no role explains.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Unit tests: each role found from a line, including filters on arrays and object ids, search ignoring URLs, sort
|
||||
spellings and nulls, paging by offset and by page, inert values; merging lines; values a role cannot apply.
|
||||
- Domain: a tool recorded whole, filtered, searched, sorted and with a flag learns all four, and a create the filter
|
||||
later lists is confirmed.
|
||||
- Serving: after a served create, new filter, search and sort values list it; an unrecorded inert value replays.
|
||||
- Measured: the fifty published captures' learned queries and write verdicts, and the six environments compared with
|
||||
their applications, before and after.
|
||||
@@ -0,0 +1,11 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Explain recorded list lines with filter, search, sort, direction, paging and inert roles.
|
||||
- [x] Merge roles across lines and keep the lines the query reproduces.
|
||||
- [x] Count matches before paging in the envelope's totals.
|
||||
- [x] Check reads on query lines in the write gate.
|
||||
- [x] Serve query lines and calls with new values from the state.
|
||||
- [x] Add unit, domain and serving tests.
|
||||
- [x] Document list queries.
|
||||
- [x] Measure the published captures and the six environments before and after.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
@@ -0,0 +1,233 @@
|
||||
# Verification
|
||||
|
||||
## Maintainer integration, 2026-10-22
|
||||
|
||||
Review of `62a9adf6c` found one attributable retry-coverage defect: a successful
|
||||
insertion-order retry returned only a boolean, so confirmation and pending-write
|
||||
cleanup still used the failed query page's covered entities. Two deterministic
|
||||
guards failed (2 failures/82 passes): an unseen update was falsely confirmed,
|
||||
and a later actual item read could no longer confirm it. The retry now returns
|
||||
its successful `CheckOutcome`; guards, targets, exercised writes, interference,
|
||||
placement and cleanup use that result. No duplicate post-retry analysis is added.
|
||||
All 623 adjacent source tests, three 161-test stability runs, 8 compiled served
|
||||
workflow boundaries, pinned static/build/docs/9 README checks and native public
|
||||
pagination pass. Independent targeted Codex source review reports no definite
|
||||
remaining defect. Public sample: 20.19s wall/28.63s CPU/490.31MiB sampled family
|
||||
RSS, no observed descendants left. Earlier CI `39776472358` passed on `62a9adf6c`,
|
||||
but cannot replace CI for the subsequent correction. Live application-write
|
||||
fidelity remains unverified; the failed-candidate policy already in main is
|
||||
unchanged, not a newly qualified rollback guarantee.
|
||||
|
||||
Representation cross-check found and reproduced one further stale counter on
|
||||
`1c9bb2e7`: canonical string total `"3"` remained `"3"` after removing an entity
|
||||
(one failure, 42 passes). Query-total interpretation now has one shared owner for
|
||||
inference and rendering, preserving number versus canonical decimal-string type.
|
||||
Unknown, padded, nonintegral or unsafe representations stay unmodeled/recorded;
|
||||
no display convention is guessed. The updated 621-test source batch and pinned
|
||||
typecheck/lint pass. The 102 query/write tests pass three consecutive runs, as do
|
||||
build, docs/9 README, format and native held-out pagination. Independent targeted
|
||||
Codex source review found no definite remaining format defect. This later public
|
||||
run took 20.13s wall/28.72s CPU/492.53MiB sampled family RSS, all observed children
|
||||
exited; it must not be hidden by the earlier faster runs or called a speedup.
|
||||
Historical counts below are pinned to their stated heads.
|
||||
|
||||
Latest-main integration: main advanced to `e93e3b1e7` (APK signing declarations).
|
||||
Merged it without dropping that feature; the sole conflict was the shared tool
|
||||
description, resolved from main with only the existing list-query clause added.
|
||||
The integrated tree passes all 619 adjacent source tests, pinned typecheck/lint,
|
||||
build, docs/9 README checks and format validation. All 80 CLI/MCP/schema boundaries
|
||||
also passed on `02ff6304d` before this main merge. An interleaved pre-merge docs
|
||||
attempt saw stale generated skill output, and format saw the temporary conflict;
|
||||
both attempts are retained and the completed integration checks pass. They are
|
||||
not attributed to a defect in main. Fresh-head CI and compiled schema verification
|
||||
remain necessary. Earlier CI run `39773075218` succeeded on `665ec74c6`, not on
|
||||
the subsequent corrections or integration.
|
||||
|
||||
Independent read-only Codex review of `665ec74c6` found one additional omission
|
||||
case: intersecting query-total names discarded a valid total when another envelope
|
||||
omitted it. A new service guard failed on that head (one failure, 41 passes).
|
||||
The producer now gathers supported totals and rejects actual contradictory counts,
|
||||
not absent members; the existing projection guard preserves omission. The updated
|
||||
527-test adjacent batch passes, including deleting a locally loaded entity and
|
||||
observing the remaining total, plus an explicit contradictory-count control.
|
||||
This review was source-derived until the regression was run; no live application
|
||||
mutation was used. The original 618-test/80-boundary counts below describe the
|
||||
prior reviewed head, not a newly invented final-head run.
|
||||
|
||||
The historical contributor runs below are not newly executed checks. This integration
|
||||
uses main `2c3dd946d` and contributor head `30e90fc64`, retaining main's zero-position
|
||||
whole-list guard and shared path decoder. New deterministic guards reproduced five
|
||||
failures before correction: unsupported search syntax, identical repeated constraints,
|
||||
stale query-only counts, and ambiguous empty-page membership. They now pass. Related
|
||||
cleanup gives decoded single-valued queries one owner, preserves per-envelope counter
|
||||
omission, and uses safe numeric precision instead of a nine-digit paging limit.
|
||||
|
||||
Executed with pinned Node 24.18.0/npm 11.16.0 on Linux: 526 adjacent source tests,
|
||||
92 catalog/schema/server source tests, 80 public CLI/MCP/schema boundary tests, and 9 README
|
||||
checks; typecheck/lint, runtime build, docs and format checks passed. Compilation is
|
||||
not evidence of live deep-provider behavior. The 99 query/write regressions also
|
||||
passed three consecutive runs; reordered constraints and equivalent encodings are
|
||||
covered without weakening conflicting-value or whole-page assertions.
|
||||
|
||||
Real-data oracle: five read-only GitHub labels observations, including a repeated
|
||||
complete collection to check snapshot stability. Only three requests enter the HAR;
|
||||
the native `per_page=3&page=2` response is held out. CLI and MCP views agree; serving
|
||||
returns the exact three native label facts with Evidence, and reordered query names
|
||||
and three warm reads agree. Main needs replay plus a full-list read/client slice;
|
||||
the query model needs one read. Structured replies total 13,282 versus 1,383 bytes;
|
||||
catalog discovery is 1,057 versus 1,090 bytes after provenance clarification.
|
||||
|
||||
| Completed task | Wall seconds | CPU seconds | Sampled family RSS MiB |
|
||||
| ------------------------------------------------------- | -----------: | ----------: | ---------------------: |
|
||||
| Main public workflow | 10.01 | 15.46 | 490.16 |
|
||||
| Candidate public workflow, before catalog clarification | 10.33 | 16.19 | 495.90 |
|
||||
| Main plus actual Codex task | 31.55 | 22.56 | 958.68 |
|
||||
| Candidate plus Codex, original catalog | 31.09 | 18.77 | 1,006.82 |
|
||||
| Candidate plus Codex, clarified catalog | 27.65 | 19.56 | 952.25 |
|
||||
| 526-test source batch | 8.75 | 20.64 | 1,513.68 |
|
||||
|
||||
RSS is a 500 ms PID/start-identity process-family sample, not a kernel peak or hard
|
||||
cap. Fresh runtime processes are separate from three later warm calls; OS/dependency
|
||||
caches and host load are uncontrolled. No speedup or cache claim follows from these
|
||||
single runs. All observed descendants exited. Node heap allowance was 2 GiB, 1 GiB
|
||||
for runtime/model children; no ambient environment was persisted.
|
||||
|
||||
Actual Codex CLI (`gpt-6.1-sol`) returned correct IDs, names, Evidence and local/not-live
|
||||
semantics in all three sessions. Main used two calls and 94,649 input tokens
|
||||
(66,816 cached), 246 output. Candidate initially still made two calls: 79,752 input
|
||||
(59,904 cached), 284 output. After clarifying that `state.source`, not
|
||||
`replay.exact_match`, identifies computed answers, the unchanged prompt used one
|
||||
call: 53,308 input (46,848 cached), 205 output. These are whole-session usage,
|
||||
including common runner context, not reply-byte token estimates or a universal
|
||||
agent-efficiency guarantee. Raw prompts, traces, exits and measurements were retained
|
||||
locally, including intermediate failed test/typecheck attempts.
|
||||
|
||||
No GitHub mutation, live-application write fidelity, full fifty-capture census,
|
||||
exhaustive search-language support or real deep-provider qualification was performed
|
||||
in this integration. Unchanged-main create-response and ignored-write-query findings
|
||||
are tracked separately in #6262 and are not claimed repaired by this PR.
|
||||
|
||||
Verified commit: 6db41d29 (this branch before this file, with #5787 and the write-plan branch's head `fe8ab584` merged in). Executed on macOS with Node v22.23.1 (the repository pins 24.18.0 in `.nvmrc`, which was
|
||||
not installed); the probes were recorded against the live applications on six disposable Linux VMs.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run build:cached`, `npm run check:fast` (typecheck and lint), `npm run format:check`, `npx knip`,
|
||||
`npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery` and `npm run mcp-catalog:check`: passed.
|
||||
- `npm run jscpd`: 3.44% duplicated lines over 3650 files, under the 3.5% gate.
|
||||
- `npx vitest run --project domain --project services`: 7630 passed, 5 pre-existing skips.
|
||||
- `npm run test:prepare`, then `npm run test:focused --` the environment serve and export CLI boundaries, the serve
|
||||
and served-writes MCP boundaries, the analysis-view CLI and MCP boundaries, the compact input schema boundary and
|
||||
tool schema validity: 8 files, 78 passed.
|
||||
|
||||
New tests: `environmentStateListQuery` (9 unit tests: each role found from a line, including filters on arrays and
|
||||
object ids, search that leaves out URLs, sort spellings and nulls, paging by offset and by page, inert values, a filter
|
||||
that excludes nothing kept inert, merging lines and refusals), one domain test in `environmentStateWrites` (a tool
|
||||
recorded whole, filtered, searched, sorted and with a flag learns all four, and a create the filter later lists is
|
||||
confirmed) and one serving test in `environmentStateWriteAnswer` (after a served create, new filter, search and sort
|
||||
values list it; an unrecorded inert value replays).
|
||||
|
||||
## Fifty published captures
|
||||
|
||||
The environment-state view ran over the 50 published captures with each manifest's template rules, on builds with and
|
||||
without this change (both with the prefix-variance branch merged, which some manifests need).
|
||||
|
||||
- 161 list views in 37 environments learn a query, and 223 checked lines are now answered by evaluating it. Roles
|
||||
learned: 27 filters, 37 searches, 35 sorts, 26 directions, 66 limits, 32 offsets, 25 page parameters and 220 inert
|
||||
parameters (each valid only for its recorded values).
|
||||
- No other view changed shape and every write verdict is unchanged (31 served write tools).
|
||||
|
||||
## Six environments against their applications
|
||||
|
||||
The six environments of the fidelity comparison (NetBox, Komga, Mealie, n8n, Shlink, self-hosted Sentry) were compared
|
||||
with their applications by `environment-compare`, with and without this change, on the comparison's probes and the
|
||||
held-out sessions recorded since: NetBox 98.5%, Komga 91.5%, Mealie 94.0%, n8n 71.6%, Shlink 87.1% and Sentry 24.5%,
|
||||
unchanged. Those probes repeat the capture's own query values, which recorded members already answer, so they cannot
|
||||
show this change.
|
||||
|
||||
New held-out probes were therefore recorded against Komga and Mealie, the two applications whose list views learned
|
||||
queries a new value can exercise, each restored from its snapshot first. After the capture's read-only prefix (so the
|
||||
driver learns the application's ids), each learned query was called with values no recording used: other search terms
|
||||
and filter values drawn from the seeded entities, the other sort direction, a page size of 1 and the second page.
|
||||
|
||||
| Application | Query calls | Exact before | Exact after | Right entities in the right order, before | After |
|
||||
| ----------- | ----------: | -----------: | ----------: | ----------------------------------------: | ----: |
|
||||
| Mealie | 38 | 2 | 17 | 2 | 38 |
|
||||
| Komga | 9 | 0 | 0 | 0 | 8 |
|
||||
|
||||
Every Mealie call now lists the entities the application lists, in its order, where 36 of them listed the recorded
|
||||
members before: sorts in the other direction (9) and new recipe searches and category filters (6) now match exactly.
|
||||
The calls that still differ, and 8 of Komga's 9, differ only in paging members of the envelope: `per_page`, `page`,
|
||||
`total_pages` and `next`/`previous` (Komga's `pageable` and `totalPages`) keep the base line's values when a call asks
|
||||
for another page size or page. One Komga search term matched more series than the learned `name`/`url` fields, since
|
||||
Komga also searches fields the list does not show. Over the whole probe Mealie went from 133 to 148 of 171 steps.
|
||||
|
||||
## What a run found
|
||||
|
||||
The first census learned Komga's `unpaged=false` as a filter on a field that is false for every entity, so `true` would
|
||||
have emptied the list. A filter that excludes nothing is now inert, and a parameter filters only where a line shows it
|
||||
excluding something: 37 of the census's 64 filters became inert values this way. A `-name` sort value matched the field loosely before its sign was read, so it sorted ascending;
|
||||
the sign is now read first.
|
||||
|
||||
## Review round
|
||||
|
||||
The branch merged the list-lines branch at de13b0c5, which carries the write-plan branch through 43d98ea9 and main
|
||||
d8a9673f. The gate's line selection, moved here to `environmentStateCheckedLines.ts`, takes the newer rules there, and
|
||||
serving answers a whole line's outgrown page from its recording before the query selection. It also fixed seven
|
||||
findings Greptile raised on this branch:
|
||||
|
||||
- a sort of an emptied collection answers no entities;
|
||||
- an empty or unmatched search term stays inert;
|
||||
- the gate checks a read with new query values as serving answers it;
|
||||
- a recorded-member line keeps its recorded totals in the gate;
|
||||
- a line with more than 12 query parameters is left unexplained, bounding the search;
|
||||
- `__proto__` and similar names stay the query's own members;
|
||||
- case-only ties sort in state order.
|
||||
|
||||
Each new test fails on the previous head. The partial-page counter test from the list-lines branch now expects the
|
||||
page, a query line here, to answer from the state with the state's total.
|
||||
|
||||
The census on this head, with the fidelity branch's compare command merged in as before, at first served 29 of 183
|
||||
writes, two fewer than at d5a07e7a. Airbyte's `GET /sources?workspaceIds=…` names a workspace no line recorded, so the
|
||||
gate now checked it on the other workspace's line, whose `next` link names that workspace. The read listed the right
|
||||
sources, but with no write exercised the failure marked the create and patch before it as side effects, and tainted
|
||||
the created source so the read after its delete went unchecked. A read on a line it did not name is now compared in
|
||||
its own envelope, which a test that fails on the previous head covers. The census again serves 31 of 183, every
|
||||
verdict as at d5a07e7a. Its 160 query views (161 before) differ from d5a07e7a only as the findings intend:
|
||||
|
||||
- empty search terms are inert in Keycloak, PhotoPrism's albums and labels, Tandoor's keywords and Umami;
|
||||
- Gitea, Grafana and Tandoor's units drop a search term that matched no shown field, with the line that carried it
|
||||
(Tandoor's units keep no query);
|
||||
- PhotoPrism's photo query leaves line 199 and its 17 parameters unexplained, and learns lines 225 and 226 instead;
|
||||
- the list-lines merge adds Bookstack's `total` and Linkding's `count` counters.
|
||||
|
||||
The six environments compare as before: NetBox 472 of 479 steps, Komga 549/600, Mealie 422/449, n8n 199/278, Shlink
|
||||
176/202 and Sentry 91/372.
|
||||
|
||||
A second review round merged the list-lines branch at c68c63be, which carries the write-plan branch through 6257de93:
|
||||
an update's identity member never takes a rule and a page size of zero bounds nothing, fixing two findings Greptile
|
||||
raised here against that branch's code, and the paging rules move to `environmentStatePaging.ts`, which this branch's
|
||||
line selection now imports. It also fixed three findings in this branch's gate:
|
||||
|
||||
- a read with no query answered on a query line is checked against that line's recorded members, as serving lists
|
||||
them, rather than against every entity;
|
||||
- a query read on a line it did not name counts as gone only what it listed, so it cannot confirm a delete of a book
|
||||
that line listed;
|
||||
- a read whose query applies a sort no longer fixes a create's insert position, so a later whole list can still try
|
||||
another rule.
|
||||
|
||||
Each new test fails on the previous head. Over the 50 captures every view and write verdict is as on the previous head,
|
||||
and the six environments compare exactly as before.
|
||||
|
||||
It then merged the list-lines branch at ed4b6dbb, which carries the write-plan branch through 466ed8b4: entities named
|
||||
as serving names them, interference weighed per write, a zero page size bounding nothing only where a recording listed
|
||||
items under it, and the other fixes of that branch's fifteenth round. Over the 50 captures every view and write verdict
|
||||
is again as before, and the six environments compare exactly as before.
|
||||
|
||||
A third review round merged the list-lines branch at 8eea9f6d, which carries the write-plan branch through 7cef2fb2,
|
||||
and fixed Greptile's finding on the second round's sort rule: a sorted read whose values tie shows the state's order
|
||||
among them, so it still fixes a create's insert position, and a later whole list can no longer move the create where
|
||||
that read showed otherwise. Its test fails on the previous head. Over the 50 captures every view and write verdict is
|
||||
as before, and the six environments compare exactly as before. A follow-up narrowed it per entity: a sorted read fixes a create's position only when the
|
||||
sort leaves the created entity itself tied, so a tie between other books no longer locks it. It also merged
|
||||
the list-lines branch at 27dd6062, carrying the write-plan branch through 45a5c8b9.
|
||||
@@ -37,7 +37,11 @@ manifest are later increments.
|
||||
value anywhere in the result.
|
||||
- List records as pointers (`ordinal`, `method`, `path`, `status`,
|
||||
`content_type`) and excluded groups with reason `other-origin`,
|
||||
`ui-page-or-asset` or `non-json`.
|
||||
`ui-page-or-asset` or `non-json`. A write group (`POST`, `PUT`, `PATCH`,
|
||||
`DELETE`) whose every response had no body (a 204 or 205, an empty text or a
|
||||
recorded size of zero) and at least one succeeded is kept whatever its content
|
||||
type: the empty reply is its answer. A missing body text alone does not count,
|
||||
as it may be a body the capture did not keep.
|
||||
- Mark the Evidence inferred whenever any tool or excluded group is templated,
|
||||
derived otherwise; keep the parent's authority.
|
||||
- Admit the capture clone, the inline envelope, every retained pointer, group,
|
||||
|
||||
@@ -29,6 +29,17 @@ later increments.
|
||||
- Prove an item type's identity field from the requested path value its
|
||||
responses repeat; otherwise infer it from a fixed list of identity names that
|
||||
every listed item carries distinctly, which needs at least one listed item.
|
||||
An item read that asked for some fields only (`fields`, `fields[...]`,
|
||||
`select`, `$select`) and does not return the item is left out of the proof
|
||||
and of the view's checks; the view lists it in `projected_record_ordinals`,
|
||||
and serving replays any call to that tool that names a projection parameter
|
||||
for an entity the state holds. The write gate also leaves out any read that
|
||||
names one and leaves out a field its view requires.
|
||||
- A field every recording of a view carried is required: serving refuses an
|
||||
entity lacking it. Once a successful read the write gate checks, naming no
|
||||
projection parameter, shows the application leaves the field out of an entity
|
||||
the state holds (APIs that omit empty or false members), no view of the type
|
||||
requires it.
|
||||
Never treat an envelope that is itself an entity as a list.
|
||||
- Merge observations into one entity per collection and identity: item views'
|
||||
latest values win over list views', differing views keep their own values,
|
||||
@@ -36,6 +47,15 @@ later increments.
|
||||
write is volatile and keeps the latest; a change across a write under another
|
||||
root, including between two tools of the type, leaves every tool of the type
|
||||
recorded. Compare list membership, including empty lists, and field presence before selecting each request line's latest recording; a change across a write also leaves the type recorded. Different query strings remain distinct; omitted fields do not erase their earlier observations on other request lines.
|
||||
- An item or list view whose recordings always spell a clock field's instant
|
||||
otherwise than the entity's stored value (`Z` against `+00:00`, another
|
||||
separator or fraction width) names that spelling in `clock_formats`; a field
|
||||
it also shows as stored, or spells two ways (counting values at another
|
||||
instant), gets none. A stored value the view never recorded, such as one a
|
||||
write made, is rendered in that spelling: its wall clock in the view's zone,
|
||||
under the view's separator, with its own fraction digits padded with zeros to
|
||||
the view's width and none dropped; an epoch spelling is computed from the
|
||||
instant.
|
||||
- A list has `all` membership only when no item-total envelope member (`total`,
|
||||
`count` and their compounds, not page counts) differs from the number of
|
||||
entities it lists and no `has_more`, `next` or next-page token is set.
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# Implementation plan
|
||||
|
||||
The branch builds on the state-serving and coverage branches (merged into it).
|
||||
|
||||
1. **Write plan types and request bodies.** Add the plan, rule and reason types
|
||||
beside the state rules, and have the state accumulator lend extraction
|
||||
parsed request bodies under the same budget, admitted like response bodies.
|
||||
2. **Field rules.** Add a module that explains an observed entity against the
|
||||
call that wrote it (echo, copy, path, slug, clock, sequence, unique,
|
||||
template and constant) and that recognises and formats clock values,
|
||||
UUIDs and tokens.
|
||||
3. **Effects.** Add one apply module that both the gate and the server use. It
|
||||
applies a create, update, delete or document write to a writable state
|
||||
interface from a plan, a call and a value source: the recorded values in
|
||||
the gate, the store's generators when served.
|
||||
4. **Gate.** Add a simulation that:
|
||||
- classifies every write tool;
|
||||
- builds each plan from its exemplar;
|
||||
- walks the timeline over a copy of the initial state;
|
||||
- checks state-answered reads;
|
||||
- attributes passes and mismatches;
|
||||
- learns the insert rule;
|
||||
- records verdicts.
|
||||
|
||||
Extraction runs it after the read gate and returns `writes` and
|
||||
`write_clock`; the view schema and its unknowns state the write model.
|
||||
|
||||
5. **Serving.**
|
||||
- The loaded state carries the plans and the clock.
|
||||
- The store gains atomic mutations, the clock and the counter (all reset by
|
||||
`data/reset` and `data/add`).
|
||||
- A write answer module validates the body, applies the effect and renders
|
||||
the response.
|
||||
- Item reads of missing entities answer recorded 404s.
|
||||
- The served catalog adds served write tools, with descriptions charged to
|
||||
the discovery budget.
|
||||
6. **Export.** Count served write tools in the bundle plan and README.
|
||||
7. **Tests.** Domain, services, adapters and MCP-boundary coverage as the
|
||||
spec's acceptance lists.
|
||||
8. **Documentation.** Document the write model in the environment-state and
|
||||
serving sections, and record the real-capture results, the AgentEnv run
|
||||
and the live comparison in `verification.md`.
|
||||
|
||||
There are no dependency, provider or REA tool contract changes beyond the
|
||||
`environment-state` result members, and no new network activity.
|
||||
@@ -0,0 +1,468 @@
|
||||
# Apply write effects to the served state
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
An environment served from its state also serves the write tools whose effects
|
||||
the capture itself confirms:
|
||||
|
||||
- a create adds an entity;
|
||||
- an update changes the fields the call sends;
|
||||
- a delete removes the entity;
|
||||
- a document write changes a document.
|
||||
|
||||
Later reads, list membership and `data/get` show the change, and `data/reset`
|
||||
undoes it.
|
||||
|
||||
The `environment-state` view decides which write tools are served. It replays
|
||||
the capture's own timeline on the extracted initial state: each recorded write
|
||||
applies its modelled effect, and each later read that the state answers is
|
||||
rendered and compared with the recorded response. A write tool is served only
|
||||
when at least one later read reproduces after its effect and none contradicts
|
||||
it. Every other write tool stays unserved, listed with the reason.
|
||||
|
||||
Values the application generates are deterministic: identities, sequences and
|
||||
clock values follow the formats the capture recorded, so the same calls after
|
||||
a reset give the same state.
|
||||
|
||||
## Write plans
|
||||
|
||||
The view adds two members to its result:
|
||||
|
||||
- `writes`: one plan per manifest tool whose method may change state
|
||||
(`changesState`);
|
||||
- `write_clock`: the instant the served clock starts from.
|
||||
|
||||
**Effects.** Each plan classifies its tool:
|
||||
|
||||
| Effect | Tool | Target |
|
||||
| ---------- | --------------------------------------------------------------------- | ------------------------- |
|
||||
| `create` | `POST` to a state type's `collection_template` | an entity of that type |
|
||||
| `update` | `PUT`, `PATCH` or `POST` to that template plus one trailing parameter | the entity the path names |
|
||||
| `delete` | `DELETE` to that same item template | the entity the path names |
|
||||
| `document` | `PUT`, `PATCH` or `POST` to a document view's path | that document |
|
||||
| `none` | anything else | reason `no-state-target` |
|
||||
|
||||
**Exemplar.** A plan is built from its first successful (2xx) recorded call,
|
||||
the exemplar.
|
||||
|
||||
- **Request body.** The exemplar's request body must be a JSON object that
|
||||
could enter the state: a JSON-like media type, complete UTF-8 text within
|
||||
256 KiB, exact numbers, no `__proto__` member. Otherwise the reason is
|
||||
`body-unusable`. A `DELETE` takes no body.
|
||||
- **The written entity.** The entity is observed in the exemplar's response,
|
||||
located by its identity value at the root, in one member, or in a
|
||||
one-element array, as item views locate it. When the response does not carry
|
||||
it, the entity is observed in the first later read that shows it: an item
|
||||
read of that entity, or for a create, the one entity a collection read newly
|
||||
lists. A create whose response names its id alone (a scalar, the identity
|
||||
field, or one member) is observed only through a read of that id. With
|
||||
neither, the reason is `no-observation`.
|
||||
|
||||
**Field rules for creates.** Each field of the observed entity gets the first
|
||||
rule that explains its value. Rules that read the call come first, so an
|
||||
identity the client sends is an echo:
|
||||
|
||||
1. **Echo.** A body member of the same name with an equal value.
|
||||
2. **Copy.** A body string of 8 or more characters, found at a JSON pointer of
|
||||
the body (the shallowest match).
|
||||
3. **Path.** The value of a path parameter of the call, percent-decoded.
|
||||
Serving decodes it from the collection the call names, whichever spelling
|
||||
the call uses, and the gate fills a recorded call's path fields from that
|
||||
collection too.
|
||||
4. **Slug.** The slug of a body string member: lower case, runs of other
|
||||
characters joined by `-`.
|
||||
5. **Identity.** The type's `id_field` is generated. A sequence when it is an
|
||||
integer (or a string of digits); otherwise a UUID when it has the
|
||||
`8-4-4-4-12` hex shape; otherwise a token of the same length and character
|
||||
classes.
|
||||
6. **Clock.** A timestamp within 10 minutes of the exemplar's start time. The
|
||||
format is kept: ISO-8601 with its separator, fraction digits and zone, or
|
||||
epoch seconds or milliseconds.
|
||||
7. **Sequence.** An integer one greater than the largest value of the field in
|
||||
the collection, else in the type. The integer may be written as `-`-separated
|
||||
digit groups of one width of at least two digits (an asset number `000-005`;
|
||||
not a date, version or address); a served value keeps the recorded value's
|
||||
groups and zero padding, and digits that outgrow them lead. A later call must
|
||||
record the value spelled that way.
|
||||
8. **Unique.** A UUID-shaped string that no other entity of the type shares.
|
||||
9. **Template.** A string containing a generated value, or else a copied
|
||||
value, as a delimited token (for example a URL that ends in the new id or in
|
||||
a slug the call sent under another name). It is rebuilt around the new
|
||||
value. An echoed value is not a token, nor is a copy of one: it may recur in
|
||||
unrelated text. A copied member whose value also sits in text no template
|
||||
rebuilds around it (beside another source in one template, or in a constant)
|
||||
is accepted only with its recorded value.
|
||||
10. **Constant.** Otherwise the exemplar's value.
|
||||
|
||||
When a constant, or the text a template keeps, contains a path value of the
|
||||
exemplar's call as a delimited token, the create is served only for calls
|
||||
naming the exemplar's collection.
|
||||
|
||||
**Body members of a create.** A member is either accepted or not:
|
||||
|
||||
- **Accepted.** An echoed member, a member copied whole into a field of another
|
||||
name and stored under none of its own, or the identity field's value. A
|
||||
member copied whole must have the JSON type of the field it is copied into.
|
||||
Every member the exemplar sent is required.
|
||||
- **Not accepted.** A member the application stored with a different value
|
||||
(transformed), or one it did not show. A call that sends such a member is a
|
||||
tool error naming the accepted members.
|
||||
|
||||
**Updates.** Body members follow the same accepted and not-accepted rules,
|
||||
except that a member the application stored differently is accepted only with
|
||||
the value a recorded call sent it with, and then stores the value that call
|
||||
recorded (its `constant` rule carries it). The type's identity field never
|
||||
takes a rule: sent as the path's id it is the identity member, sent as another
|
||||
value the application ignored it is accepted with that value and stores
|
||||
nothing, and a stored id that changed is `unexplained-change`. A plan whose
|
||||
calls use a member both as the identity and as a fixed member, or fix it at
|
||||
different values, is `unexplained-change`, as serving could not take every
|
||||
recorded call. So is an update a later read shows dropped a field the
|
||||
entity held, since serving keeps every field an update does not write; a write's
|
||||
own response may leave fields out. A member the application replaced
|
||||
with a fresh UUID (`unique`, below) is accepted with any value; a UUID stored
|
||||
as sent in another letter case is the sent value, not a fresh one. A member a later call is the first to store,
|
||||
which the plan's observation does not show, keeps the JSON type that call stored
|
||||
it as (the body's `kinds`), as an exemplar's member keeps the observation's.
|
||||
`PUT` requires every member the exemplar sent; `PATCH` and `POST` require at
|
||||
least one. A field the exemplar's update changed without sending it is
|
||||
explained as one of:
|
||||
|
||||
- clock;
|
||||
- an increment by one;
|
||||
- a slug of an echoed member;
|
||||
- `unique`: a UUID in place of another UUID, that no other entity of the type
|
||||
holds and the call sent nowhere in its body (a version the application assigns
|
||||
on every update);
|
||||
- `renewed`: an array or object whose only changes are UUIDs replaced by ones
|
||||
that neither it nor another entity of the type held, as child rows an update
|
||||
rewrites are. The rule names the pointers it renewed at (`*` for any array
|
||||
index): every UUID at them is new, each old UUID by a single new one, and
|
||||
UUIDs elsewhere, such as a row's references, are kept. One call renews a
|
||||
UUID alike in every field and in either letter case, and never two old UUIDs
|
||||
into one new one.
|
||||
|
||||
A field the entity lacked before is a change too when the view that observed
|
||||
the update had shown the entity, so a field the application fills in on update
|
||||
follows its rule. Any other change is `unexplained-change`. An increment or a slug applies to
|
||||
every call that could take it, and `unique` and `renewed` to every call, so the
|
||||
plan is also `unexplained-change` when such a rule would give a call of the
|
||||
tool another value than the one it recorded, a recorded call kept a `unique` or
|
||||
`renewed` field, or a later call treats a member otherwise than the plan: one
|
||||
call echoes it and another stores something else, or the two rule it
|
||||
differently.
|
||||
|
||||
**Deletes.** A delete removes the entity. Its response is the exemplar's
|
||||
response, with the entity re-rendered when the response carries it.
|
||||
|
||||
**Documents.** A document write merges the body's members (replaces them, for
|
||||
`PUT`) into the document object that holds most of them: the root or one
|
||||
member. Later calls write at the exemplar's object. Document reads answer only on their recorded line, so a write is
|
||||
served only at the path the state holds the document at, each segment verbatim
|
||||
or either spelling once percent-decoded, and the gate treats a recorded write
|
||||
at another path as an unmodelled write. A recorded write whose
|
||||
body serving would refuse (a member of another JSON type than the document
|
||||
holds) is `unexplained-change`.
|
||||
|
||||
**Responses.** A write answers with the status and content type of one
|
||||
recorded response:
|
||||
|
||||
| Response | Body |
|
||||
| ---------- | --------------------------------------------------------------------------------------------------------- |
|
||||
| `entity` | the written entity, rendered into the recorded envelope at the plan's pointer through the recorded fields |
|
||||
| `identity` | the recorded body with the new identity at the plan's pointer |
|
||||
| `empty` | no body |
|
||||
| `recorded` | the recorded body |
|
||||
|
||||
A rendered body missing a field the recorded response carried is a tool error,
|
||||
and the state is left unchanged.
|
||||
|
||||
**List position.** A create's `insert` rule places the new entity in its
|
||||
collection's order: sorted by the identity field or a clock field (ascending
|
||||
or descending), `first` or `last`. The rule is the first candidate consistent
|
||||
with both the initial `all`-membership recordings and the position a later
|
||||
recorded list shows. With no evidence, the rule is `last`.
|
||||
|
||||
## The gate
|
||||
|
||||
The view replays every primary-origin record in timeline order on a copy of
|
||||
the initial state.
|
||||
|
||||
**Writes.** Only a 2xx write is modelled. A modelled write applies its effect, using the values it recorded
|
||||
for generated fields and the stored values for fields it transformed. A
|
||||
modelled write whose call the plan does not explain brings its entity to the
|
||||
recorded outcome instead, without attribution. So does an update of an unknown
|
||||
entity (below): its call cannot be checked, so it adds nothing to a built plan.
|
||||
A later create whose generated
|
||||
values take another form than serving gives them (a sequence value out of
|
||||
order, a clock in another format, a non-UUID unique value) makes the plan
|
||||
`unexplained-change`; integer identities are left to the id mapping. A later
|
||||
create outside the collection a tied plan permits is unmodelled, as serving
|
||||
refuses that call, so a read of what it recorded does not confirm the plan. The
|
||||
gate picks that collection among equal spellings in serving's order (below). An update or
|
||||
delete names its entity as an item read does (below). A later update that does
|
||||
not answer with its entity is observed only by a read whose view shows every
|
||||
clock the plan learned, since serving stamps them; without one it is unmodelled.
|
||||
|
||||
Any other write that may have changed state, including a redirected or
|
||||
unanswered one, is unmodelled, and what it may have changed
|
||||
depends on where its path lies relative to each type's collection paths
|
||||
(segment-prefix related, parameters matching any segment):
|
||||
|
||||
- **Below an item path:** the named entity is unknown until a read shows it
|
||||
again. When the path names no held entity, every entity of that collection
|
||||
is unknown.
|
||||
- **At a collection path:** that collection is not checked again.
|
||||
- **Above collection paths:** everything under the path is not checked again.
|
||||
|
||||
The path names its entity as an update's does and its collection as a create's
|
||||
does, so a plain spelling reaches a collection held under an encoded one; a
|
||||
path above collections covers each collection whose segments are its own
|
||||
verbatim or once percent-decoded.
|
||||
|
||||
An item read of an unknown entity shows it again: the copy takes its recorded
|
||||
values. A failed read that is no absence shows nothing, and the entity stays
|
||||
unknown. Writes that changed it since its last check are checked only on the
|
||||
members they sent, less those the view treats as volatile, which show nothing
|
||||
of a write.
|
||||
|
||||
**Reads.** A read is checked when the served environment would answer it
|
||||
from the state:
|
||||
|
||||
- an item read of a state-backed item view, of the entity serving would name:
|
||||
in a held collection the read's path matches, each segment verbatim or either
|
||||
spelling once percent-decoded and verbatim spellings first, the id
|
||||
percent-decoded once, then as written;
|
||||
- a list or document read on one of its view's checked request lines,
|
||||
chosen as replay chooses the line for the read's call: each path parameter
|
||||
the read's value verbatim or either spelling once percent-decoded, preferring verbatim
|
||||
spellings, then recorded order, and the query values decoded, in any order.
|
||||
A read with no query on a whole list falls back, as serving does, on a
|
||||
checked line that only pages it. A line whose query repeats a name with
|
||||
different values is never chosen, since no call's arguments can name it; one
|
||||
repeating a name with one value is, as a call naming it matches both. A list
|
||||
is compared as served from the collection of that line. A read on a page
|
||||
(`limit`, `size`, `per_page` and the like, or a `_start`–`_end` range; a size
|
||||
of zero is no bound on a line whose recording listed items under it, while its
|
||||
other sizes and range still bound the page) of a
|
||||
whole list checks nothing once the collection holds more entities than the
|
||||
page, which serving then answers from its recording. A line without proof of
|
||||
whole membership lists its recorded members still held, unless a learned
|
||||
query evaluates the read over the state. An empty later page therefore does
|
||||
not contradict a create outside that page.
|
||||
- A read made while a modelled write to its entity or collection was still
|
||||
running, starting before that write ended, checks nothing: it may show the
|
||||
state before or after the write. Nor does such a read show an unknown entity
|
||||
again.
|
||||
- A read a write's effect was learned from checks nothing of that write, but
|
||||
still contradicts the other writes it exercises.
|
||||
|
||||
**Comparison.** The read's answer is the full response serving would give from
|
||||
the copy: an item read's entity rendered into its item view's base recording,
|
||||
or a list read's members as serving would list them (so a member the copy no
|
||||
longer holds fails the check) rendered into its line's base recording. It is
|
||||
compared with the recording, as the read gate compares, with volatile fields taking the
|
||||
observed values. A recorded 400, 404 or 410, or a 2xx with a JSON `null`
|
||||
body, for an entity the copy does not hold agrees with it. A successful read
|
||||
whose usable body no longer holds the entity or the list fails, as does one of
|
||||
an entity lacking a field every recording of its view carried, which serving
|
||||
refuses to answer with.
|
||||
|
||||
**Attribution.**
|
||||
|
||||
- A check exercises the write tools that changed its entity or collection
|
||||
since the previous check of it, less those whose effect later writes erased.
|
||||
An update's effect is the fields it sent or changed, derived ones included.
|
||||
A later update overwrites the fields it sets (an increment adds to its field
|
||||
instead), a recorded outcome every field, and a create or delete the whole
|
||||
entity. A document write's effect is the members it merged at its
|
||||
object, or everything under the object it replaces, the whole document at the
|
||||
root; an entity's fields match by name only, whatever their names look like. A write with nothing left on an entity no
|
||||
longer counts there, nor in a collection where no entity it changed keeps any
|
||||
of its effect. A mismatch contradicts them. A pass confirms those it shows
|
||||
something of: a create's or delete's membership, or a changed field the
|
||||
read's view shows and does not treat as volatile; a read that shows none of
|
||||
an update's changes confirms nothing of it, and leaves it for a later check
|
||||
of that entity or collection.
|
||||
- A mismatch contradicts each exercised write that no unmodelled write came
|
||||
after its latest call on any of the read's entities or collections where its
|
||||
effect survives, whatever earlier write is still pending. For a failed list read the unmodelled write must
|
||||
also follow the last check and pending writes of each member that differs, so an older change elsewhere
|
||||
does not excuse a newer write the list shows undone. An item read checks the writes an unmodelled
|
||||
write did follow only on the members they sent, and a list with such a write
|
||||
is not checked again: the unmodelled write may explain the rest.
|
||||
- A mismatch on an entity or collection that no write changed contradicts,
|
||||
as `side-effect`, the modelled writes since that previous check whose paths
|
||||
are segment-prefix related to the read's path.
|
||||
- After a failed item check, the copy takes the read's recorded values (or
|
||||
drops the entity on a recorded absence). A list whose check failed is not
|
||||
checked again.
|
||||
- A list that fails only for where one create placed its entity passes when
|
||||
another of that create's insert candidates reproduces it, which then
|
||||
becomes the plan's rule, unless an earlier list check relied on the rule
|
||||
(an item read's confirmation does not fix it).
|
||||
- The read that supplied a create's or update's observation does not check
|
||||
it, but checks the other writes it exercises.
|
||||
|
||||
**Verdicts.** A plan is served when at least one check confirms it and none
|
||||
contradicts it. Otherwise the reason, in this precedence, is:
|
||||
|
||||
1. the first build reason;
|
||||
2. `contradicted`, with `contradicted_by` naming the first contradicting read;
|
||||
3. `side-effect`;
|
||||
4. `unconfirmed`;
|
||||
5. `incomplete-entities`: an update or delete that answers with the entity,
|
||||
when no initial entity holds every field its recorded answer shows (for
|
||||
example a delete that answers with an edit view of the entity the reads
|
||||
never showed), so it could answer only for entities the environment itself
|
||||
created. An update's answer renders the entity it wrote, so a field one of
|
||||
its rules writes counts as held, except an increment's, which adds only to
|
||||
a held value.
|
||||
|
||||
`checked_record_ordinals` lists the confirming reads.
|
||||
|
||||
## Serving
|
||||
|
||||
**Catalog.** In state mode, the catalog adds the write tools whose plan is
|
||||
served. Their descriptions say the tool:
|
||||
|
||||
- applies its effect to the served state;
|
||||
- names its target type;
|
||||
- lists its required and accepted body members, quoted and sampled within the
|
||||
description budget as response fields are.
|
||||
|
||||
The discovery budget charges them. `--replay-only` and the replay fallback are
|
||||
unchanged.
|
||||
|
||||
**Calls.** A served write answers synchronously against the live store and
|
||||
changes it atomically, so every MCP session and `data/get` see one state.
|
||||
|
||||
**Refusals.** These are tool errors that leave the state unchanged:
|
||||
|
||||
- a body that is not an object;
|
||||
- a missing required member;
|
||||
- a member that is not accepted;
|
||||
- a value of another JSON type than the exemplar's (for a member only a later
|
||||
update stored, that call's), for documents as well as entities;
|
||||
- an update or delete of an entity the state does not hold;
|
||||
- a create in another collection when the plan is tied to its own;
|
||||
- a document write at a path other than the one the state holds it at;
|
||||
- a body member or a resulting value `data/add` would refuse (a number not
|
||||
exact as `data/get` writes it, a `__proto__` member, nesting past the
|
||||
limit), increments included, so `data/get` always loads back;
|
||||
- a create, update or document write that would take what `data/get` reports
|
||||
past the 32 MiB `data/add` intake limit, as a `resource_constraint` error;
|
||||
- a rendered response missing a recorded field.
|
||||
|
||||
When the tool, or else an item view of the type, recorded a 400, 404 or 410
|
||||
response, an update or delete of a missing entity answers with the latest such
|
||||
recording instead.
|
||||
|
||||
A create in a collection held under more than one spelling (`acme1` and
|
||||
`%61cme1`) goes to the one the call spells verbatim, as item calls choose their
|
||||
entity; a collection keeps its spelling after its last entity is deleted. A
|
||||
reset or load knows the spellings of the Evidence's state, of the state it
|
||||
installs and of the type's checked list lines (so a collection the capture
|
||||
listed empty keeps its spelling), never those of a state it replaced. When two
|
||||
spellings fit a call equally, the one holding entities wins, the earlier first
|
||||
entity in state order among held ones and the earlier known among emptied ones. `data/add` refuses a document that `data/get` would write back past the
|
||||
32 MiB intake limit once it adds the members a document may omit, such as
|
||||
`types` and the state Evidence id. A derived state `data/get` would write
|
||||
past that limit is not served: the environment serves replay and reports the
|
||||
state `unavailable`.
|
||||
|
||||
**Clock and generators.** The store's clock starts at `write_clock` and
|
||||
advances one second before each served write; clock fields take its value in
|
||||
their recorded format.
|
||||
|
||||
- **Sequences:** one more than the largest current value in their scope, the
|
||||
largest the state started with at its last reset or load, and the largest
|
||||
value they generated, so a deleted entity's value is not reused, as database
|
||||
sequences do not reuse values.
|
||||
- **UUIDs and tokens:** derived from a SHA-256 of the state Evidence id, the
|
||||
tool, the field and a write counter, skipping values the type already holds.
|
||||
- **Renewed fields:** the entity's current value with each UUID at the rule's
|
||||
pointers replaced by one derived from a SHA-256 of the state Evidence id, the
|
||||
tool, the write counter and the UUID's order in the call, the same
|
||||
replacement wherever the call renews that UUID in either letter case, spelled
|
||||
in each place's case; held values are not skipped.
|
||||
An entity without the field is left without it.
|
||||
|
||||
`data/reset` and `data/add` restart the clock, the counter and the
|
||||
sequences.
|
||||
|
||||
**Missing entities.** An item read of an entity the state does not hold
|
||||
answers with the tool's latest recorded 400, 404 or 410 response, or 2xx JSON
|
||||
`null` response, the absences the gate accepts, when it has one, labelled
|
||||
`state.view: "missing"`. Otherwise it stays a tool error.
|
||||
|
||||
**Lists of a whole collection.** Written entities matter only if lists show
|
||||
them, so the read gate's `membership: "all"` also covers a line that only pages
|
||||
its list (`limit`, `offset`, `page`, `per_page`, `perpage`, `page_size`,
|
||||
`pagesize`, `size`, `start`, `_start`, `_end`) when its recording listed the
|
||||
whole collection at a proved zero start. Positive `page` values do not prove
|
||||
the beginning, because their base is unknown. Whole membership is per line:
|
||||
`all_record_ordinals` can name whole lines within a recorded-membership view.
|
||||
A learned query can filter, search, sort and page the state; otherwise a later
|
||||
page lists its recorded members still held. A list call with no query argument
|
||||
answers from a whole line of the collection its path arguments name, with
|
||||
`replay.exact_match` false; a collection with no such line answers from its
|
||||
recording. A call that asks for a page evaluates a learned query when one
|
||||
covers its arguments; otherwise it keeps the recorded page members.
|
||||
|
||||
**Provenance.** Write answers carry `state: {source: "state", view: "create" |
|
||||
"update" | "delete", type, collection, id}`, or `{source: "state", view:
|
||||
"document-write", path}`.
|
||||
|
||||
**Data plane and export.**
|
||||
|
||||
- `data/get` reports written entities without record citations; they load
|
||||
back through `data/add` unchanged.
|
||||
- `environment-export` counts served write tools in the bundle's tool count
|
||||
and README. Its check stays a read tool.
|
||||
|
||||
## Not modelled
|
||||
|
||||
These stay unserved, or are labelled in the description:
|
||||
|
||||
- side effects on counters, relations and copies of entities in other
|
||||
collections;
|
||||
- inputs the application transforms;
|
||||
- members the capture never sent;
|
||||
- reordering after an update;
|
||||
- membership of filtered or paged lists, which answer their recorded members;
|
||||
- write paths the manifest left literal;
|
||||
- writes whose request body the capture did not retain;
|
||||
- referential rules on delete: a delete removes the entity alone, so an
|
||||
application that refuses to delete a referenced entity, or deletes its
|
||||
dependents with it, is not modelled;
|
||||
- authorization.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- **Domain tests:** classification, every field rule, request-body admission,
|
||||
observation from a later read, the insert rules, every verdict and reason,
|
||||
attribution and resynchronisation, and taint by an unmodelled write.
|
||||
- **Services tests:**
|
||||
- store mutations and their atomicity;
|
||||
- every refusal;
|
||||
- generated values and their determinism across reset;
|
||||
- `data/get` round trips of written entities;
|
||||
- the recorded 404 for missing entities;
|
||||
- the catalog and descriptions in each mode.
|
||||
- **Boundary test:** a write tool changes the next read over MCP.
|
||||
|
||||
**Real captures.**
|
||||
|
||||
- The view runs over every showcase capture within its budget, and the
|
||||
verification reports, per capture, the write tools served and the reasons
|
||||
for the rest.
|
||||
- One environment, chosen at random among those serving a write tool, is
|
||||
exported, run in the AgentEnv framework, and graded from `data/get` after an
|
||||
agent's create, update and delete.
|
||||
- The served model is compared with that application running live on the same
|
||||
calls.
|
||||
|
||||
No test reaches an external network.
|
||||
@@ -0,0 +1,22 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the write model, the gate and the serving contract.
|
||||
- [x] Admit request bodies and add the plan types.
|
||||
- [x] Explain created and updated entities with field rules.
|
||||
- [x] Apply create, update, delete and document effects in one shared module.
|
||||
- [x] Simulate the capture's timeline and decide each plan's verdict.
|
||||
- [x] Leave entities below an unmodelled write unknown until a read shows them
|
||||
again, and check writes such a write follows only on the members they
|
||||
sent.
|
||||
- [x] Serve gated write tools from the live store with deterministic values.
|
||||
- [x] Answer recorded 400, 404 and 410 responses for missing entities.
|
||||
- [x] Let lines that only page a whole collection back `membership: "all"`,
|
||||
and answer argument-less list calls from them.
|
||||
- [x] Count served write tools in exports and describe them in bundle READMEs.
|
||||
- [x] Add domain, services, adapters and boundary coverage.
|
||||
- [x] Document the write model and serving.
|
||||
- [x] Run the view over every showcase capture and report per-capture verdicts.
|
||||
- [x] Export one randomly chosen environment, run write tasks in the AgentEnv
|
||||
framework and grade them from `data/get`.
|
||||
- [x] Compare the served model with the live application on the same calls.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
@@ -0,0 +1,376 @@
|
||||
# Verification
|
||||
|
||||
Verified base: cbb7b613 (this branch before this file). Executed on macOS with
|
||||
Node v22.23.1 (the repository pins 24.18.0 in `.nvmrc`, which was not
|
||||
installed); the AgentEnv runs and the live comparison ran on a disposable Linux
|
||||
VM. Every check below ran as written after the last source change.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run build:cached`, `npm run check:fast` (typecheck and lint),
|
||||
`npm run format:check`, `npx knip`, `npm run scan:todos`, `npm run docs:check`
|
||||
and `npm run verify:test-discovery`: passed.
|
||||
- `npm run jscpd`: 3.43% duplicated lines over 3538 files, under the 3.5% gate.
|
||||
- `npx vitest run --project domain --project services`: 7407 passed, 5
|
||||
pre-existing skips.
|
||||
- `npm run test:prepare`, then `npm run test:focused --` the new write MCP
|
||||
boundary, the serve MCP, CLI and process boundaries, the export CLI
|
||||
boundary, the analysis-view CLI and MCP boundaries, the compact input schema
|
||||
boundary and the contract description tests: 10 files, 72 passed.
|
||||
- `npm run mcp-catalog:check`: passed.
|
||||
|
||||
New tests: `environmentStateWriteValues` (94), `environmentStateWriteExplain`
|
||||
(101), `environmentStateWriteApply` (52), `environmentStateWrites` (16, the gate
|
||||
through the view), `environmentStateWriteAnswer` (14, serving) and
|
||||
`environmentServeWritesMcp` (1, the built CLI over MCP). The replay card pinned
|
||||
by the serve MCP boundary changed with the response summaries tool descriptions
|
||||
carry; its digest now matches the combined branch.
|
||||
|
||||
## Real captures
|
||||
|
||||
The `environment-state` view ran over the 50 showcase captures (45 HAR
|
||||
inspections and 5 `capture_browser_scenario` captures), each within its budget in
|
||||
8–110 ms. It served 33 write tools in 16 captures (11 creates, 12 updates, 10
|
||||
deletes). Of the other 150 state-changing tools, 102 name no state type or
|
||||
document (most write to literal single-entity paths the manifest left
|
||||
untemplated, or to collections whose reads are not state-backed), and 48 are
|
||||
modelled but not served:
|
||||
|
||||
- 15 unconfirmed (no later read checked them);
|
||||
- 13 no-observation;
|
||||
- 8 contradicted, for example a deleted Airbyte source and EspoCRM task still
|
||||
readable, Immich and BookStack item reads showing fields the create
|
||||
response lacked, and Sonarr's tag list ordering;
|
||||
- 7 unexplained-change, such as BookStack's description_html and Paperless's
|
||||
derived text colour;
|
||||
- 4 side-effect, for example a Sonarr tag also listed by `/tag/detail`;
|
||||
- 1 incomplete-entities: WordPress answers a delete with the post in its edit
|
||||
context, which the reads never showed for seeded posts.
|
||||
|
||||
| Capture | Served (create / update / delete) | Not served |
|
||||
| ---------------- | --------------------------------- | ----------------------------------------------------------------------------------------------- |
|
||||
| airbyte | 2 / 2 / 0 | create contradicted, delete contradicted, delete unconfirmed |
|
||||
| baserow | none | no-state-target ×3 |
|
||||
| bookstack | 0 / 0 / 1 | create contradicted, create side-effect, update unexplained-change |
|
||||
| directus | none | no-state-target ×2 |
|
||||
| espocrm | none | create unconfirmed, delete contradicted, update unexplained-change |
|
||||
| ghost | none | no-state-target ×2 |
|
||||
| gitea | 1 / 1 / 0 | no-state-target ×2 |
|
||||
| grafana | none | create unconfirmed, no-state-target ×5 |
|
||||
| grist | none | create no-observation, no-state-target ×5 |
|
||||
| grocy | none | create no-observation, delete no-observation, update no-observation, no-state-target ×2 |
|
||||
| healthchecks | none | create no-observation, delete no-observation, update no-observation, no-state-target ×2 |
|
||||
| homebox | none | document no-observation, no-state-target ×3 |
|
||||
| immich | 0 / 0 / 1 | create contradicted, update contradicted |
|
||||
| jellyfin | none | no-state-target ×2 |
|
||||
| keycloak | none | no-state-target ×1 |
|
||||
| komga | none | create unconfirmed ×2, no-state-target ×4 |
|
||||
| linkding | 2 / 2 / 2 | create unconfirmed, delete unconfirmed, update unconfirmed, no-state-target ×2 |
|
||||
| listmonk | 1 / 1 / 1 | — |
|
||||
| lychee | none | no-state-target ×4 |
|
||||
| mailpit | none | document no-observation, no-state-target ×5 |
|
||||
| mattermost | 1 / 1 / 1 | no-state-target ×3 |
|
||||
| mealie | none | create no-observation, no-state-target ×2 |
|
||||
| memos | none | no-state-target ×3 |
|
||||
| metabase | 0 / 0 / 1 | create side-effect, update unexplained-change |
|
||||
| miniflux | 0 / 1 / 0 | no-state-target ×3 |
|
||||
| n8n | none | create unconfirmed, no-state-target ×6 |
|
||||
| navidrome | none | create no-observation, no-state-target ×2 |
|
||||
| netbox | 1 / 1 / 1 | — |
|
||||
| nocodb | none | create no-observation, no-state-target ×4 |
|
||||
| nomad | none | no-state-target ×4 |
|
||||
| paperless | 0 / 0 / 1 | create unconfirmed, update unexplained-change, no-state-target ×3 |
|
||||
| photoprism | 1 / 1 / 0 | delete unconfirmed |
|
||||
| pinry | none | create unconfirmed, delete unconfirmed, update unexplained-change |
|
||||
| pocketbase | none | no-state-target ×4 |
|
||||
| redpanda-console | none | document no-observation, no-state-target ×5 |
|
||||
| sentry | 1 / 0 / 0 | update unexplained-change, no-state-target ×3 |
|
||||
| shlink | none | create unconfirmed, no-state-target ×1 |
|
||||
| sonarr | 0 / 1 / 0 | create contradicted, create side-effect, update contradicted |
|
||||
| tandoor | none | no-state-target ×4 |
|
||||
| umami | 1 / 1 / 1 | — |
|
||||
| vault | none | no-state-target ×7 |
|
||||
| vikunja | none | no-state-target ×4 |
|
||||
| wordpress | none | create side-effect, delete incomplete-entities, document unconfirmed, update unexplained-change |
|
||||
|
||||
An in-process smoke derived each capture's served state and called every served
|
||||
write tool once with a varied body: all 33 applied (status equal to the
|
||||
recording), and every created or updated entity read back from the state;
|
||||
deleted entities answered the recorded 400 or 404 where one was recorded. Eight
|
||||
older published manifests no longer match the tool names the current view
|
||||
derives, so their state is unavailable and they serve replay, as designed;
|
||||
none of them has a served write tool.
|
||||
|
||||
## End to end on a published environment
|
||||
|
||||
Five published environments serve a create, an update and a delete (Linkding,
|
||||
Listmonk, Mattermost, NetBox, Umami); `random.SystemRandom().choice` picked
|
||||
NetBox (NetBoxEnv-REA, NetBox 4.7.3).
|
||||
|
||||
- `rea environment-export` (packed from this branch) wrote the bundle; its
|
||||
smoke task passed in AgentEnv framework 0.9.1334 with 42 served tools (39
|
||||
reads and `post_dcim_sites`, `patch_dcim_sites_by_id`,
|
||||
`delete_dcim_sites_by_id`).
|
||||
- Four write tasks, each graded by an `env_outcome_verifier` that reads the
|
||||
environment's state through `/agentenv` `data/get` (not the agent's reply),
|
||||
with the replay-qa agent over the Hugging Face router, one run per task and
|
||||
model:
|
||||
|
||||
| Task | gpt-oss-120b | GLM-5.3-Flash | Tool calls (gpt-oss) |
|
||||
| ----------------------------------------------- | ------------ | ------------- | ------------------------------------------------------------ |
|
||||
| Create a site in the Copper Valley region | 1.0 | 1.0 | get_dcim_regions, post_dcim_sites |
|
||||
| Update a seeded site's facility and description | 1.0 | 1.0 | get_dcim_sites, patch_dcim_sites_by_id |
|
||||
| Delete one seeded site and only it | 1.0 | 1.0 | get_dcim_sites, delete_dcim_sites_by_id |
|
||||
| Create a site, then set its facility | 1.0 | 1.0 | get_dcim_regions, post_dcim_sites, patch_dcim_sites_by_id ×2 |
|
||||
|
||||
- The bundle's 12 retrieval tasks (one per seeded object) passed 12/12 for both
|
||||
models, as they did on the published replay bundle.
|
||||
|
||||
## The served model against the live application
|
||||
|
||||
The same NetBox 4.7.3 image ran on the VM, seeded by the capture's own seed
|
||||
script (the same ids), and one scenario ran against both the live API and the
|
||||
environment served from the capture. Only `created` and `last_updated` were
|
||||
normalised; every other member had to be equal:
|
||||
|
||||
| Step | Live | Environment | Equal |
|
||||
| ----------------------------------------- | ---- | ----------- | ---------------------------- |
|
||||
| List sites | 200 | 200 | yes |
|
||||
| Create Tide Gauge Annex (id 5) | 201 | 201 | yes |
|
||||
| Read it | 200 | 200 | yes |
|
||||
| List after the create (sorted by name) | 200 | 200 | yes |
|
||||
| Patch its description and facility | 200 | 200 | yes |
|
||||
| Read after the patch | 200 | 200 | yes |
|
||||
| Create Granite Spur (id 6) | 201 | 201 | yes |
|
||||
| Patch a seeded site | 200 | 200 | yes |
|
||||
| Delete Tide Gauge Annex | 204 | 204 | yes |
|
||||
| Read the deleted site | 404 | 404 | yes |
|
||||
| Create Ledger Point (id 7, not reusing 5) | 201 | 201 | yes |
|
||||
| Delete a seeded site with devices | 409 | 204 | no |
|
||||
| Final list | 200 | 200 | no (follows from the delete) |
|
||||
|
||||
11 of 13 steps were equal. The difference is NetBox's referential protection:
|
||||
it refuses to delete a site that devices reference, while the environment's
|
||||
delete removes the entity alone. The capture only deleted a scratch site with
|
||||
no dependents, so the gate could not see the rule; it is listed under what is
|
||||
not modelled.
|
||||
|
||||
## Not covered
|
||||
|
||||
- A run of the hosted `real-agentenv` lane in CI (the AgentEnv runs here used
|
||||
the published framework release on a VM).
|
||||
- Document writes on a real capture: the four modelled ones are unserved.
|
||||
- Collections tied to a create's own collection on a real capture, and UUID
|
||||
or token identities through an AgentEnv run (both are covered by tests).
|
||||
- Agent token usage and cost.
|
||||
|
||||
## After merging the base branches and main
|
||||
|
||||
The branch then merged the serving branch at 5a61853a, the coverage branch at
|
||||
b4b7ebc7 (their review fixes) and main at edbd291a, then main at 17f581ff
|
||||
(the analysis-view tool description string, which main changes often). The gates above ran again
|
||||
and passed: `npm run jscpd` 3.44% over 3579 files; domain and services 7442
|
||||
passed with 5 pre-existing skips; the focused boundary set 72 passed. The
|
||||
write verdicts over the 50 captures were unchanged plan by plan, and the
|
||||
in-process smoke again applied all 33 served write tools. The AgentEnv runs
|
||||
and the live comparison above ran on cbb7b613, before these merges.
|
||||
After the second main merge: `npm run check:fast`, `npm run format:check`,
|
||||
`npm run docs:check` and `npm run mcp-catalog:check` passed; the contracts,
|
||||
analysis-view and environment tests 857 passed with 3 pre-existing skips; the
|
||||
analysis-view and environment MCP and CLI boundaries 53 passed.
|
||||
|
||||
## After the review round
|
||||
|
||||
The branch then merged the serving branch at c00fde41, the coverage branch at
|
||||
f6c756aa and main at 1629266e, and took the review fixes (82556cd5, 138ea4b1,
|
||||
6d21b62e). The coverage branch quotes field names in response summaries, so the
|
||||
replay card pinned by the serve MCP boundary is now 12,818 bytes; removing
|
||||
those quotes gives back the previous 12,618-byte digest exactly. The gates ran
|
||||
again and passed: `npm run check:fast`, `npm run format:check`, `npx knip`,
|
||||
`npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery`,
|
||||
`npm run verify:module-boundaries` and `npm run mcp-catalog:check`;
|
||||
`npm run jscpd` 3.47% over 3601 files; domain and services 7463 passed with 5
|
||||
pre-existing skips; the focused boundary set 75 passed; conformance 279
|
||||
passed. The gate is stricter, but the write verdicts over the 50 captures were
|
||||
unchanged plan by plan, and the in-process smoke again applied all 33 served
|
||||
write tools. The AgentEnv runs and the live comparison above ran on cbb7b613,
|
||||
before these merges and fixes.
|
||||
|
||||
A second review round then made the gate and serving agree in five more places
|
||||
(reads matched to checked lines as serving matches calls, increments and slugs
|
||||
held for every call, insert rules still learned after an item read, the
|
||||
document body check in the gate, and decoded parent path values). Each new test
|
||||
fails on the previous head and passes here. The same gates passed again
|
||||
(domain and services 7468 passed with 5 pre-existing skips, the focused
|
||||
boundary set 75, conformance 279, `jscpd` 3.47%), the write verdicts over the 50
|
||||
captures were again unchanged plan by plan, and the smoke applied all 33 served
|
||||
write tools.
|
||||
|
||||
A third round kept a checked line's path spelling (replay prefers a verbatim
|
||||
one, so two spellings of one parent stay separate lines) and compared a later
|
||||
slug rule with the values earlier calls recorded rather than only their member
|
||||
names; both new tests fail on the previous head. A fourth round chose a read's
|
||||
line as replay does, so a read spelled plainly is checked against the encoded
|
||||
line that answers it, and its list against that line's collection; its test
|
||||
fails on the previous head.
|
||||
|
||||
A review of the fidelity branch built on this one found four more places: a
|
||||
document write at another path is now unmodelled in the gate, a create picks
|
||||
the collection the call spells verbatim, a missing entity is answered with a
|
||||
recorded 2xx `null` the gate accepted as an absence, and `data/add` refuses a
|
||||
document whose `data/get` form would exceed the intake limit. Each new test
|
||||
fails on the previous head. The gates passed again (domain and services 7475
|
||||
with 5 pre-existing skips, boundaries 75, conformance, `jscpd` 3.46%), the
|
||||
verdicts over the 50 captures were unchanged plan by plan, and the smoke
|
||||
applied all 33 served write tools.
|
||||
|
||||
A sixth round, after merging the serving and coverage branches and main
|
||||
335c042d, settled five more findings: a read whose usable body no longer holds
|
||||
the entity or list fails its check, an entity lacking a field its view always
|
||||
carried confirms nothing, a write a later write replaced is not confirmed by
|
||||
reads of the replacement, sequences never reuse a value the state started
|
||||
with, and a collection keeps its spelling after it empties. Each new test fails
|
||||
on the previous head. Over the 50 captures one capture's verdicts change:
|
||||
Linkding's bundle create and patch, confirmed only by a list read after the
|
||||
same bundle was deleted, are now unconfirmed, so 31 write tools are served (10
|
||||
creates, 11 updates, 10 deletes) and the in-process smoke applies all 31. The
|
||||
gates passed again (domain and services, boundaries, conformance and `jscpd`).
|
||||
|
||||
A seventh round settled two more findings. An update keeps its confirmation
|
||||
when a later write overwrites only part of what it changed: a derived change,
|
||||
such as an incremented version, counts as part of the update's effect, and a
|
||||
later increment adds to the field rather than replacing it. A reset or load
|
||||
forgets the collection spellings of the state it replaced. Both new tests fail
|
||||
on the previous head. The verdicts over the 50 captures are unchanged plan by
|
||||
plan, the in-process smoke applies all 31 served write tools, and the gates
|
||||
passed again.
|
||||
|
||||
An eighth round settled three more. A later write erases the fields of an
|
||||
earlier effect it overwrites, so two writes that together overwrite all of an
|
||||
update's changes leave it nothing to be confirmed by. A recorded outcome erases
|
||||
what earlier writes changed on its entity. A load keeps the Evidence's own
|
||||
collection spellings, so a create in a collection the loaded state does not
|
||||
hold keeps its recorded spelling. Each new test fails on the previous head. The
|
||||
verdicts over the 50 captures are unchanged plan by plan, the smoke applies all
|
||||
31 served write tools, and the gates passed again.
|
||||
|
||||
A ninth round made a create prefer a collection the state holds over an equally
|
||||
fitting spelling known only from the Evidence, so a create after a load lands in
|
||||
the loaded collection and reads back. Its test fails on the previous head; the
|
||||
verdicts and the smoke are unchanged and the gates passed again.
|
||||
|
||||
A tenth round settled six findings Greptile raised on the list-lines branch
|
||||
against this branch's code:
|
||||
|
||||
- a read with no query on a whole list is checked on the paging line serving
|
||||
answers it from, and a line whose query repeats a name is never checked;
|
||||
- a read made while a write to its targets was still running checks nothing;
|
||||
- a later call that stores a member otherwise than the plan refuses the plan;
|
||||
- a create keeps the spelling of a collection the capture listed empty;
|
||||
- a wide array no longer overflows the stack while checking path-dependent
|
||||
values;
|
||||
- the guide counts served write tools among the catalog's.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures
|
||||
are unchanged plan by plan, the smoke applies all 31 served write tools, and
|
||||
the gates passed again.
|
||||
|
||||
An eleventh round settled six more findings, five raised on this branch and one on the fidelity branch:
|
||||
|
||||
- a passing read confirms only the writes it shows something of;
|
||||
- an item read names its entity as serving does, percent-decoded spellings included;
|
||||
- a read a write's effect was learned from still contradicts the other writes;
|
||||
- a line repeating a query name with one value stays checkable;
|
||||
- document writes join the effect tracking, so a replacement overwrites an earlier merge;
|
||||
- a call naming a page that the collection has outgrown answers from its recording (`page-exceeded`), and the gate does not check such reads.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A twelfth round settled four findings Greptile raised on the list-lines branch against this branch's code:
|
||||
|
||||
- a create that answers with its id alone is observed only through a read of that id;
|
||||
- a later call that is the first to send the path's id in its body adds the identity member to the plan;
|
||||
- the copy search walks a request body with an index, so a wide body no longer takes quadratic time (500,000 zeros: 63 s
|
||||
before, 74 ms after);
|
||||
- the sent values of fixed members a plan keeps are counted against the view's allowance.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A thirteenth round settled seven findings Greptile raised on this branch, the fidelity branch and the list-lines branch:
|
||||
|
||||
- a passing read that shows nothing of a write leaves it for a later check, instead of clearing it;
|
||||
- a read begun while a write to an unknown entity still ran neither checks nor shows the entity again;
|
||||
- a later create whose sequence fields do not follow the learned sequence refuses the plan;
|
||||
- a document replacement below the root overwrites only what it replaced;
|
||||
- a built document plan writes later calls at its own object, even one sharing no member with the document;
|
||||
- an update or delete finds its entity as an item read does, percent-decoded spellings included;
|
||||
- a `_start`–`_end` range bounds a page as `limit` does, in serving and in the gate.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A fourteenth round settled two findings Greptile raised on the list-queries branch against this branch's code:
|
||||
|
||||
- the type's identity field never becomes a constant: an update whose id member the application ignored no longer
|
||||
writes the exemplar's id into the entity another call names;
|
||||
- a page size of zero bounds nothing, so a whole line recorded with `limit=0` answers from the state.
|
||||
|
||||
Both tests fail on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke applies
|
||||
all 31 served write tools, and the gates passed again.
|
||||
|
||||
A fifteenth round settled nine findings Greptile raised on this branch, the fidelity branch and the list-lines branch,
|
||||
five of them on the previous rounds' own changes:
|
||||
|
||||
- an entity field whose name looks like a pointer is matched by name; only a document's pointers cover the members
|
||||
under them;
|
||||
- a failed read contradicts each exercised write no unmodelled write came after, rather than skipping all of them
|
||||
when the oldest pending write came before one;
|
||||
- an item read, update or delete names its entity as serving does, decoding the id once, so `book%2531` no longer
|
||||
reaches `book1`;
|
||||
- a plan whose calls use a member both as the identity and as a fixed member, or fix it at different values, is
|
||||
refused;
|
||||
- a size of zero bounds nothing only on a line whose recording listed items under it;
|
||||
- an update whose observation drops a field the entity held is refused;
|
||||
- a later create whose clock takes another format, or whose unique value is not a UUID, is refused;
|
||||
- a derived state `data/get` would write past 32 MiB is not served.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A sixteenth round settled four findings Greptile raised on the fifteenth round's changes, here and on the fidelity,
|
||||
list-lines and list-queries branches:
|
||||
|
||||
- a size of zero is no bound only by itself: a range or another size on the same line still bounds the page;
|
||||
- an update is refused for a dropped field only when a later read shows it dropped, since a write's own response may
|
||||
leave fields out;
|
||||
- interference is weighed against a tool's latest call on the read's entity or collection, not its latest call
|
||||
anywhere.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A seventeenth round settled two more findings Greptile raised on the list-queries branch against this branch's
|
||||
interference rule: an unmodelled write counts against a tool when it followed the tool's call on any target the read
|
||||
covers, so a later call on another entity does not hide it; and a failed item read contradicts the writes no
|
||||
unmodelled write followed, checking only the others on their members. Both tests fail on the previous head. The
|
||||
verdicts over the 50 captures are unchanged plan by plan, the smoke applies all 31 served write tools, and the gates
|
||||
passed again. A follow-up weighs interference only on targets where the tool's effect survives, so a call
|
||||
whose effect a later write overwrote no longer excuses a later mismatch; its test fails on the previous head.
|
||||
|
||||
## Collection-history accounting check
|
||||
|
||||
On 2026-10-22, Node 24.18.0, I compared the source before the write-history
|
||||
change (`cfb42ddde`) with the bounded-history implementation. Each operation
|
||||
inserted an entity at a new collection path with a 128-character repeated
|
||||
suffix and immediately deleted it. Across five fresh Vitest worker processes,
|
||||
the median for 1,000 completed cycles was 14.57 ms wall / 23,867 us CPU before
|
||||
and 9.98 ms / 16,914 us after. The process RSS samples were 381.1–387.9 MiB
|
||||
before and 380.9–387.6 MiB after. This fixture benchmark measures one matched
|
||||
completed trace, not a representative user task or process-family peak memory.
|
||||
The values include Vitest and V8 and are noisy; they do not establish a memory
|
||||
bound. The 1 MiB policy is a deterministic accounting allowance for
|
||||
additional identifier history, set to 1/32 of the separate 32 MiB state-intake
|
||||
ceiling; it is not derived from RSS samples and does not bound total store or
|
||||
process memory.
|
||||
@@ -0,0 +1,8 @@
|
||||
# Implementation plan
|
||||
|
||||
1. **Re-spelling.** A values helper spells a clock in another format: its wall clock in the format's zone, with its own
|
||||
fraction digits padded to the format's width.
|
||||
2. **Learning.** The gate notes, per view, the spelling of each clock field that differs from the stored value, while it
|
||||
checks the view's recordings; the view gains `clock_formats` when any field always differs.
|
||||
3. **Rendering.** Entity rendering spells a stored value in the view's spelling when the view has no value of its own.
|
||||
4. **Schema, tests and docs.** The view schema, gate, serving and unit tests, and the state-view docs.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Spell a written clock as each view spells it
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
Some applications spell one instant differently by view. Mealie's recipe reads end timestamps in `Z`
|
||||
(`2026-10-21T17:06:13.710410Z`) and its recipe list in `+00:00`. The state keeps each view's own values for the entities
|
||||
it recorded, so those render as recorded, but a created or updated entity has only the values the write showed. A list
|
||||
read after a create therefore rendered the new recipe's clocks in the item spelling. The write gate saw a contradiction
|
||||
and left the create unserved, and serving would have listed the wrong spelling.
|
||||
|
||||
## Behaviour
|
||||
|
||||
- **Learning.** While checking that an item or list view reproduces its recordings, the gate compares each recorded
|
||||
object's clock fields with the entity's stored values. A field whose recorded value always names the same instant in
|
||||
one other spelling gets that spelling in the view's optional `clock_formats`. A field the view also shows as stored, or
|
||||
spells two ways, counting values that name another instant, gets none. A spelling has at most nine fraction digits.
|
||||
Instant equality includes all recorded fraction digits, not just milliseconds; offset changes and trailing zeros
|
||||
alone do not change the instant. The clock parser owns this precision and rendering reuses its parsed facts.
|
||||
- **Bounds.** Each field retains at most one matching and one other-instant spelling, or a permanent mixed marker
|
||||
for either category. Rejected formats do not accumulate. The existing projection budget covers the learner's maps,
|
||||
keys, formats and returned view members before retention; exhaustion keeps the existing typed resource refusal.
|
||||
Preliminary list-query probes do not learn or retain clock state; only the final admitted view does.
|
||||
- **Rendering.** A stored value the view never recorded, such as one a write made, is rendered in the view's spelling:
|
||||
its wall clock in the view's zone, under the view's separator, with its own fraction digits padded with zeros to the
|
||||
view's width. No digit is dropped, so a width learned from whole seconds does not cut a later value's fraction. An
|
||||
epoch spelling is computed from the instant.
|
||||
|
||||
The gate and serving render through the same function, so a served create lists with the list's spelling and reads back
|
||||
with the item's. `data/get` returns the stored values.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- The pilot recording's Mealie create is served on the list-query stack.
|
||||
- Over the 50 published captures, one view learns a spelling (Mealie's recipe list), and every write verdict is unchanged.
|
||||
- Every served write tool of the 50 captures still applies.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Re-spell a clock in another format, keeping digits past milliseconds.
|
||||
- [x] Learn each view's clock spellings while checking its recordings.
|
||||
- [x] Render stored values a view never recorded in its spelling.
|
||||
- [x] Add gate, serving and unit tests, failing on the previous head.
|
||||
- [x] Document the view member.
|
||||
- [x] Measure the pilot recordings, the 50 published captures and the served smoke.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Verification
|
||||
|
||||
Executed on macOS with Node v22.23.1 on this branch with the write-plan branch's head `45a5c8b9` merged in. The pilot recordings
|
||||
were made against live applications on six disposable Linux VMs, each restored from a snapshot first.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run build:cached`, `npm run check:fast` (typecheck and lint, no warnings in the changed files),
|
||||
`npm run format:check`, `npx knip`, `npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery` and
|
||||
`npm run mcp-catalog:check`: passed.
|
||||
- `npm run jscpd`: 3.43% duplicated lines over 3753 files.
|
||||
- `npx vitest run --project domain --project services`: 7755 passed, 5 pre-existing skips.
|
||||
- `npm run test:prepare`, then `npm run test:focused --` the environment serve and export CLI boundaries, the serve
|
||||
and served-writes MCP boundaries, the analysis-view CLI and MCP boundaries, the compact input schema boundary and
|
||||
tool schema validity: 8 files, 79 passed.
|
||||
|
||||
## Tests
|
||||
|
||||
- Gate: a list that spells clocks `+00:00` where item reads spell `Z` learns both spellings, the item view learns none,
|
||||
and a create read back by both is served; a create whose clock has a fraction is served where the list's spelling was
|
||||
learned from whole seconds; a list that spells a field both ways, at the stored instant or another, learns nothing.
|
||||
- Serving: a served create lists with the list's spelling and reads back with the item's.
|
||||
- Unit: re-spelling keeps a value's fraction digits at any offset and across a date change, pads them to the format's
|
||||
width without dropping any, and formats an epoch spelling from the instant; an epoch value's milliseconds become
|
||||
fraction digits.
|
||||
- Schema: a view clock spelling wider than nine fraction digits is refused.
|
||||
- The gate and serving tests fail on `264ce410`, the write-plan head they were written on.
|
||||
|
||||
## Pilot recordings
|
||||
|
||||
Mealie's recipe reads spell clocks `Z` and its recipe list `+00:00`. On this branch alone the list line the create is
|
||||
read back on keeps its recorded members, so the create stays unserved; per-line list membership and list queries
|
||||
(#5787, #5800) answer that line from the state. Built on the list-query stack (#5800 with #5699 merged):
|
||||
|
||||
- Mealie's `post_recipes` goes from `contradicted` to served, and Mealie's held-out sessions agree on 298 of 310 steps,
|
||||
up from 294.
|
||||
- With the assigned-values branch (#6136) too, all three Mealie write tools are served, at 302 of 310.
|
||||
- The other five applications are unchanged.
|
||||
|
||||
## Published captures
|
||||
|
||||
- Over the 50 published captures, one view learns a spelling: Mealie's recipe list (`dateUpdated`, `createdAt`,
|
||||
`updatedAt`). Every write verdict is unchanged.
|
||||
- The served smoke applies all 31 served write tools.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Implementation plan
|
||||
|
||||
1. **Explanation.** Creates accept members a whole-member copy stores and template text around copied values.
|
||||
Updates rule a fresh UUID `unique`, accepting a sent member so replaced, and child rows with only renewed UUIDs
|
||||
`renewed`; the update reads the type's other entities to tell a fresh UUID from a reference.
|
||||
2. **Values.** Helpers find the UUIDs a value renewed and renew the UUIDs of a value.
|
||||
3. **Apply and serve.** Updates take `unique` and `renewed` from the generator; serving renews the current entity's
|
||||
UUIDs.
|
||||
4. **Schema, tests and docs.** The view's rule schema, unit, gate and serving tests, and the write specs.
|
||||
@@ -0,0 +1,52 @@
|
||||
# Serve writes whose values the application renames or regenerates
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
Recordings made to show each write's effect, with every write read back at once and two scratch objects per tool,
|
||||
still left writes unserved where the application stored a value in a way the write rules did not explain:
|
||||
|
||||
- a create's member stored under another name: Shlink stores `customSlug` as `shortCode`, and its `shortUrl` ends in
|
||||
it, so every slug but the recorded one was refused;
|
||||
- an update that gives a field a fresh UUID each time: n8n's `versionId`, which the client sends back with the
|
||||
version it read;
|
||||
- an update that rewrites child rows with new UUIDs: Mealie gives a recipe's instructions new ids on every update.
|
||||
|
||||
These writes are now explained, and served where every recorded call agrees with the explanation.
|
||||
|
||||
## Rules
|
||||
|
||||
- **Creates.** A member copied whole into a field of another name, and stored under none of its own, is accepted with
|
||||
any value of that field's JSON type. A string that contains a copied value as a delimited token is a template of
|
||||
the copying field and is rebuilt around the new value; the copied value has the copy rule's 8 characters at least.
|
||||
Echoed values, and copies of them, make no templates, since an echoed name may recur in unrelated text.
|
||||
Each text containing a copied value must rebuild that copy; rebuilding it in one text does not make occurrences
|
||||
in other templates safe. Otherwise the member remains fixed to its recorded value.
|
||||
- **Updates.**
|
||||
- `unique`: a field that held a UUID and now holds another, which the call did not send (in any letter case) and
|
||||
no other entity of the type holds. A sent member replaced this way is accepted with any value.
|
||||
A value shared with another output field is not independent generation: scalar and nested generators cannot
|
||||
reproduce that relation, so the update remains unexplained rather than serving disconnected UUIDs.
|
||||
- `renewed`: an array or object whose only changes are UUIDs replaced by ones neither it nor another entity of the
|
||||
type held, at the pointers the rule names (`*` for any array index): every UUID there is new, each old UUID by a
|
||||
single new one, and UUIDs elsewhere are kept. A lone UUID is never renewed: it is either `unique` or unexplained.
|
||||
Renewal identity is canonical and case-insensitive, including repeated mixed-case occurrences within one field;
|
||||
source spelling remains in the retained entity and is preserved by serving. Fresh values must not be sent in the
|
||||
request, owned by an earlier or another entity, or shared with non-renewed output fields. Such relations remain
|
||||
unexplained until a generation rule can reproduce them. Nested renewed fields may still share one fresh replacement.
|
||||
- **Serving.** A `unique` field gets a UUID derived as for creates. A `renewed` field keeps the entity's current
|
||||
value with each UUID at the rule's pointers replaced by one seeded by the call, the same replacement wherever the
|
||||
call renews that UUID; an entity without the field stays without it.
|
||||
|
||||
The gate takes the recorded values for both, as for other generated fields. A later call must rule a field the same
|
||||
way, and no recorded call may keep a `unique` or `renewed` field, since serving changes it on every call.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- The pilot recordings' Shlink create and n8n update are served. Mealie's update stays unserved: one recorded call keeps
|
||||
the ingredient ids the gate's copy of the recipe shows renewed, and no other verdict changes.
|
||||
- The 50 published captures keep every verdict; only Shlink's unserved create plan changes, accepting
|
||||
`customSlug` and templating `shortUrl`.
|
||||
- Every served write tool of the 50 captures still applies.
|
||||
- A unit test shows a UUID another entity holds is not `unique`, and a child row whose text changed is not `renewed`.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Accept a create member stored under another name, and template text around copied values.
|
||||
- [x] Rule fresh UUIDs on update as `unique`, and rewritten child rows as `renewed`.
|
||||
- [x] Apply and serve both rules.
|
||||
- [x] Add unit, gate and serving tests, each failing on the previous head.
|
||||
- [x] Document the rules.
|
||||
- [x] Measure the pilot recordings, the 50 published captures and the served smoke.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Verification
|
||||
|
||||
Executed on macOS with Node v22.23.1 on this branch with the write-plan branch's head `45a5c8b9` merged in. The pilot recordings
|
||||
were made against live applications on six disposable Linux VMs, each restored from a snapshot first.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run build:cached`, `npm run check:fast` (typecheck and lint, no warnings in the changed files),
|
||||
`npm run format:check`, `npx knip`, `npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery` and
|
||||
`npm run mcp-catalog:check`: passed.
|
||||
- `npm run jscpd`: 3.43% duplicated lines over 3754 files.
|
||||
- `npx vitest run --project domain --project services`: 7769 passed, 5 pre-existing skips.
|
||||
- `npm run test:prepare`, then `npm run test:focused --` the environment serve and export CLI boundaries, the serve
|
||||
and served-writes MCP boundaries, the analysis-view CLI and MCP boundaries, the compact input schema boundary and
|
||||
tool schema validity: 8 files, 79 passed.
|
||||
|
||||
## Tests
|
||||
|
||||
- Gate: a renamed book renewing a sent `version_id` and its chapters' ids is served with `unique` and `renewed`
|
||||
rules; two creates sending a `handle` stored as `code` and embedded in `link` are served with `copy` and
|
||||
`template` rules.
|
||||
- Serving: each served update gets a fresh version and new chapter ids that keep their titles; a served create with
|
||||
a new handle stores it as the code and rebuilds the link.
|
||||
- Unit: the explanation, including a UUID another entity holds (not `unique`), a child row whose text changed (not
|
||||
`renewed`), and a member copied only in part (still fixed); the renewal helpers; applying both update rules.
|
||||
- The four gate and serving tests fail on `43d98ea9`.
|
||||
- Review round (on `a477b21d`, where each fails): serving renews only the UUIDs at the pointers the recording renewed,
|
||||
keeps an author id each chapter names, gives one UUID the same replacement in every renewed field, and gives no rows
|
||||
to an entity that holds none; a recorded call that keeps a renewed value refuses the plan; a renewal that keeps,
|
||||
moves or borrows a UUID is not one; a copied member is type-checked against its stored field; a member stored
|
||||
otherwise under its own name is not accepted; an echoed value's copy is no template source; a sent UUID stored in
|
||||
another letter case is not a generated one.
|
||||
|
||||
## Pilot recordings
|
||||
|
||||
Each application was reset, then recorded with the capture's reads and, per write target, two scratch objects whose
|
||||
every write was read back at once (item and list), deletes read back as 404.
|
||||
|
||||
| Application | Write | Before | After |
|
||||
| ----------- | ------------------------------------- | -------------------- | -------------------- |
|
||||
| Shlink | `post_rest_short_urls` | `unexplained-change` | served |
|
||||
| n8n | `patch_rest_workflows_by_workflow_id` | `unexplained-change` | served |
|
||||
| Mealie | `patch_recipes_by_recipe_id` | `unexplained-change` | `unexplained-change` |
|
||||
|
||||
No other verdict of the six recordings changed. Mealie's update was served by an earlier version of this branch with
|
||||
`recipeIngredient` renewed. Each of its two recorded updates keeps the ingredient ids; the gate's copy of the second
|
||||
recipe held the first recipe's ids, because the read that showed it was the create's observation and was not adopted.
|
||||
Each renewed field now needs every recorded call to renew it, so the plan is refused rather than served regenerating
|
||||
ingredient ids. Built on the list-query stack (#5800 with #5699 merged), the held-out sessions agreed on two more
|
||||
steps each for n8n (127 to 129 of 143) and Shlink (126 to 128 of 137) with the earlier version; Komga, NetBox and
|
||||
Sentry were unchanged.
|
||||
|
||||
## Published captures
|
||||
|
||||
- Over the 50 published captures, every write verdict is unchanged: 31 served write tools. The only plan that
|
||||
changes is Shlink's unserved create, which now accepts `customSlug` and templates `shortUrl`.
|
||||
- An earlier version also made echoed values template sources. It templated Grafana's folder `url` around `uid`
|
||||
(the URL also holds a slug of the title) and BookStack's page HTML around its markdown. Echoed values are no
|
||||
longer sources.
|
||||
- The served smoke applies all 31 served write tools, as on `43d98ea9`.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Implementation plan
|
||||
|
||||
1. **Bodiless writes.** The manifest accumulator tracks, per write group, whether every response was empty, and keeps
|
||||
the group when one succeeded.
|
||||
2. **Omitted fields.** The write simulation records required fields its successful reads show missing, and the
|
||||
extracted state's views drop them.
|
||||
3. **Projected reads.** Item planning sets aside unlocated projected reads and the view records their ordinals; the gate
|
||||
skips them, and serving replays projected calls to such a view.
|
||||
4. **Sequences.** A values helper reads a digit-group value as an integer and fills the next one into its mask; the
|
||||
write rules, the gate and serving number sequences through it.
|
||||
5. **Schema, tests and docs.** The view schema, manifest, gate, serving and unit tests, and the manifest, capture,
|
||||
state-view and state-writes docs.
|
||||
@@ -0,0 +1,49 @@
|
||||
# Wider write coverage
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
Four recording shapes kept write tools out of the manifest or left their writes unserved, although the recordings showed
|
||||
what the writes did:
|
||||
|
||||
- **Bodiless replies.** A write answered `204` with no body (Jellyfin's and Shlink's deletes, Miniflux's and Keycloak's
|
||||
updates) had no JSON-like content type, so its group was excluded as `non-json` and never became a tool.
|
||||
- **Omitted fields.** A field every recording of a view carried was required, so an application that leaves empty or
|
||||
false members out of some entities (Go's `omitempty`) made the write gate refuse its own read-backs.
|
||||
- **Projected reads.** An item read asking for some fields only (Directus's `/items/provisions/2?fields=title`) does not
|
||||
repeat the requested id, so it blocked the identity proof of the whole item view, and the view's writes had no state
|
||||
target.
|
||||
- **Digit-group sequences.** A sequence spelled in digit groups (Homebox's asset number `000-005`) was not an integer, so
|
||||
a create that numbers it had no rule for it and was contradicted.
|
||||
|
||||
## Behaviour
|
||||
|
||||
- **Manifest.** A write group (`POST`, `PUT`, `PATCH`, `DELETE`) is kept as a tool when every response had no body (a
|
||||
`204` or `205`, an empty text, or a recorded size of zero) and one succeeded. A missing text alone does not count.
|
||||
- **Omitted fields.** The write gate notes required fields missing from the entities the state holds that successful
|
||||
reads it checks return, when the read names no projection parameter: the item whose id the path names, or the listed
|
||||
items. No view of the type requires them any more. A read that checks nothing, such as one begun while a modelled
|
||||
write to its targets still ran or a page the collection has outgrown, notes none.
|
||||
Established omissions apply when classifying subsequent reads and choosing later write observations, not just
|
||||
when serializing final views. A complete `fields=*` read is not partial solely for lacking a known optional field.
|
||||
- **Projected reads.** An item read naming `fields`, `fields[...]`, `select` or `$select` that does not return the item
|
||||
is left out of the identity proof and the view's checks. The view lists it in `projected_record_ordinals`, and serving
|
||||
replays a call to that tool naming such a parameter, for an entity the state holds, with the reason `projection`;
|
||||
the replay shows the recording, not served writes. The write gate neither checks nor learns a write's effect from a
|
||||
successful read that names such a parameter and leaves out a field its view requires; a 400, 404 or 410 such a read
|
||||
answers is still the absence it checks.
|
||||
Identity-free projection evidence after the initial-state cutoff is retained too, without seeding later values
|
||||
into initial entities. Failed responses are classified before the projection-replay shortcut, so a projected
|
||||
absence can confirm a delete or contradict an unexplained disappearance.
|
||||
- **Sequences.** A sequence value may be `-`-separated digit groups of one width of at least two digits, not a date,
|
||||
version or address. The next value keeps the exemplar's groups and zero padding; digits that outgrow them lead. A
|
||||
later create must record its value spelled that way. A held value whose digits outgrew the mask (`100-00` after
|
||||
`99-99`) counts only when it is the exemplar's spelling of its number, so a reloaded state numbers on from it.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- Over the 50 published captures, no write group is excluded as `non-json`, served write tools rise from 35 to 39, and
|
||||
no write tool is served on the base and unserved here.
|
||||
- The re-recorded Homebox capture's create is served.
|
||||
- Every served write tool of those captures applies.
|
||||
@@ -0,0 +1,10 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Keep write groups answered with no body as tools.
|
||||
- [x] Stop requiring fields a successful read shows the application leaves out.
|
||||
- [x] Set projected item reads aside from the identity proof and replay projected calls.
|
||||
- [x] Number sequences spelled in digit groups.
|
||||
- [x] Add manifest, gate, serving and unit tests, failing on the previous head.
|
||||
- [x] Document the manifest rule, the view member, the replay reason and the sequence spelling.
|
||||
- [x] Measure the 50 published captures, the re-recorded Homebox capture and the served smoke.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
@@ -0,0 +1,74 @@
|
||||
# Verification
|
||||
|
||||
Executed on macOS with Node v22.23.1 on this branch, whose base is the write-plan branch's head `ad8f674d`.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run build:cached`, `npm run check:fast` (typecheck and lint, no new warnings in the changed files),
|
||||
`npm run format:check`, `npx knip`, `npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery` and
|
||||
`npm run mcp-catalog:check`: passed.
|
||||
- `npm run jscpd`: 3.45% duplicated lines over 3776 files.
|
||||
- `npx vitest run --project domain --project services`: 7808 passed, 5 pre-existing skips. Two unrelated heavy tests
|
||||
(keyed-archive traversal, a 125,000-dependent Evidence graph) passed only with a longer timeout on a loaded machine.
|
||||
- `npm run test:prepare`, then `npm run test:focused --` the environment serve and export CLI boundaries, the serve
|
||||
and served-writes MCP boundaries, the analysis-view CLI and MCP boundaries, the compact input schema boundary and
|
||||
tool schema validity: 8 files, 79 passed; the analysis-view MCP file needed a longer timeout on the loaded machine.
|
||||
|
||||
## Tests
|
||||
|
||||
- Manifest: a `204` write with no body is kept as a tool whatever its content type, and a write only redirected is not.
|
||||
- Gate:
|
||||
- a field a read-back leaves out stops being required, and a create lacking it is served only when its own read
|
||||
leaves it out too;
|
||||
- an item view proves its identity past a read that asked for some fields only, which is left to replay;
|
||||
- creates numbered in digit groups are served.
|
||||
- Serving: a created entity is served without the omitted field while seeded ones keep it; a call naming `fields`
|
||||
replays with the reason `projection` while other calls answer from the state; creates take the next number in the
|
||||
held value's digit groups.
|
||||
- Unit: `-`-separated digit groups of one width read as a sequence, and a date, version, address or decimal does not;
|
||||
the next value keeps the groups, overflow leading.
|
||||
- The gate, serving and manifest tests fail on `ad8f674d`.
|
||||
|
||||
Review fixes, each with a test that fails on `a4c4a23b`:
|
||||
|
||||
- Gate:
|
||||
- a read naming `fields`, or of a book the state no longer holds, notes no omission, and a create read back only
|
||||
through `?fields=id,name` is not served;
|
||||
- an omission an item read shows stops every view of the type requiring the field;
|
||||
- a read naming `fields` that leaves a required field out neither resynchronises an entity nor serves as a write's
|
||||
observation, so a `204` update read back first through `?fields=id,name` learns its effect from the whole read;
|
||||
- a later create whose sequence value serving would spell otherwise (`00-04` or `4` after `000-003`) withholds the
|
||||
plan.
|
||||
- Serving: an empty projection argument replays; a projection of a deleted book answers as missing; a created book
|
||||
without the omitted field lists.
|
||||
|
||||
## Published captures
|
||||
|
||||
Over the 50 published captures, against `ad8f674d`:
|
||||
|
||||
- Write groups excluded as `non-json` go from 13 in five captures to none. Directus, Jellyfin, Keycloak, Miniflux and
|
||||
Shlink gain those tools, so their manifests change.
|
||||
- Served write tools go from 35 to 39; none is lost:
|
||||
- Directus's provisions item view proves its identity past `GET /items/provisions/2?fields=title`, so its create and
|
||||
update are served;
|
||||
- Jellyfin's `DELETE /Items/{id}` and Shlink's `DELETE /rest/v3/short-urls/{slug}` are served.
|
||||
- Airbyte's job item view stops requiring two fields its reads leave out; its verdicts are unchanged.
|
||||
- Six states change: those five captures and Airbyte.
|
||||
- A served smoke with manifests rebuilt by each tree applies all 39 served write tools here and all 35 on the base.
|
||||
|
||||
## Re-recorded captures
|
||||
|
||||
Captures re-recorded with a read-back after each first write and two scratch objects per write tool:
|
||||
|
||||
- A Homebox pilot recording numbers assets in digit groups (`000-001`, `000-002`, …). Its create goes from
|
||||
`contradicted` to served with a `sequence` rule on `assetId`, and the smoke applies it and the delete.
|
||||
- Five published environments rebuilt from such recordings (Navidrome, Lychee, Homebox, Directus, Linkding) go from 9
|
||||
served write tools to 11, and the smoke applies all 11:
|
||||
- Navidrome's playlist views stop requiring `imageAbsent`, which its new playlists leave out, so its playlist update is
|
||||
served;
|
||||
- Homebox's create is served, as in the pilot recording.
|
||||
- Jellyfin and Memos, re-recorded and rebuilt end to end with this change (manifest, bundle, smoke and both evaluation
|
||||
models on 12 retrieval tasks each), serve no new write:
|
||||
- Jellyfin's four bodiless write groups become tools, all withheld. Its playlists are created through
|
||||
`POST /Playlists` but read back only through `/Items/{id}`, which stops that item view rendering from the state.
|
||||
- Memos has none of the four shapes; it names items `memos/<id>`, which the identity proof does not match.
|
||||
@@ -0,0 +1,113 @@
|
||||
# Inspect APK signature schemes (v1–v3.1)
|
||||
|
||||
## Outcome and scope
|
||||
|
||||
Answer, statically, **who signed a selected local APK, under which schemes, and
|
||||
whether the declared content digests match the current bytes**. This closes a gap REA's own
|
||||
Android inventory documents: "ZIP inventory cannot see the APK Signing Block, so
|
||||
an empty signing array does not mean the APK is unsigned"
|
||||
(`src/domain/android/androidApplication.ts`).
|
||||
|
||||
Static format inspection within existing offline-artifact authority: no Android
|
||||
code executes, no device or `adb`, no acquisition, no mutation.
|
||||
|
||||
## Inspection contract
|
||||
|
||||
- v1 (JAR) signing detected from the central directory as apksig selects it:
|
||||
`META-INF/` entries at any depth ending in `.SF` or `.RSA`/`.DSA`/`.EC`
|
||||
(case-sensitive), present only when a block has its matching `.SF`.
|
||||
- Each paired v1 block's PKCS#7 `SignedData` read with bounded DER (BER
|
||||
indefinite lengths accepted): every carried certificate, and per
|
||||
`SignerInfo` the certificate it names by `issuerAndSerialNumber` or
|
||||
`subjectKeyIdentifier` plus its digest/signature algorithm OIDs (apksig
|
||||
resolves only `issuerAndSerialNumber`, so a `subjectKeyIdentifier` signer is
|
||||
reported as declared, with a limitation, and is unknown to the posture's v1
|
||||
comparison). The signer
|
||||
`certificate` goes through the same X.509 owner as v2/v3; it is `null` when
|
||||
there is not exactly one `SignerInfo` or it names no carried certificate. A
|
||||
block that cannot be read (including a local header that disagrees with
|
||||
the central directory, as apksig checks) carries
|
||||
`decode_error`, never an absent signer; so does a signer whose block,
|
||||
`.SF` or `META-INF/MANIFEST.MF` name is duplicated. Duplicate names are
|
||||
found among all central-directory entries, as apksig rejects the v1
|
||||
signature for any of them ("Duplicate entry"), and a limitation names the
|
||||
first few (JSON-quoted, cut at 256 bytes with the stored length) and counts
|
||||
the rest. Names are compared as apksig's Java UTF-8 decoding makes them, so
|
||||
malformed bytes that decode to the same string are duplicates; where Zip64
|
||||
declares more records than the classic end-of-central-directory count that
|
||||
apksig reads, duplicates only past those records are named separately. As
|
||||
apksig does, the ZIP encryption flag is
|
||||
ignored: the block is read and its decoding decides, with a limitation.
|
||||
- The APK Signing Block located from the ZIP end-of-central-directory (Zip64
|
||||
followed) and framed into id-value pairs; v2, v3 and v3.1 signers decoded:
|
||||
digests by signature-algorithm id, X.509 certificate DER, v3 min/max SDK,
|
||||
additional attributes.
|
||||
- Certificate identity as SHA-256 over the raw embedded DER (as apksig
|
||||
hashes it); subject, issuer, serial, ISO 8601 validity and key algorithm via
|
||||
the platform X.509 parser, each `null` when unreadable rather than guessed.
|
||||
- The v3/v3.1 proof-of-rotation lineage (version 1) decoded into nodes:
|
||||
certificate and the capability flags retained by that node's certificate.
|
||||
Node signatures are not verified: the lineage is what the APK declares. A
|
||||
lineage that cannot be decoded is reported with `decode_error`.
|
||||
- Source-stamp presence and the declared stamp certificate (V2 block
|
||||
preferred), or a `decode_error` when it cannot be read.
|
||||
- Declared digest consistency: the v2/v3 content digest is recomputed for
|
||||
SHA-256/SHA-512 and compared with each declared digest (`matches_apk`),
|
||||
rolled up into `posture.self_consistent`. This is not a signature
|
||||
verification: an attacker who changes the bytes can recompute and replace
|
||||
the declared digest. Verity digests are not recomputed (`null`).
|
||||
- A derived posture that keeps observation and declaration apart:
|
||||
`schemes_agree_on_signer` compares each block scheme's signer certificate
|
||||
_set_; `signer_change_declared_by_rotation` says whether every differing
|
||||
signer appears in a declared lineage (`null` when identity or a needed
|
||||
lineage is unknown); `v1_signers_match_block_signers` compares the v1
|
||||
signer set with what the lowest block scheme that has signers (v2, else
|
||||
v3, else v3.1) expects: its signers, or the first (original) certificate
|
||||
of a v3/v3.1 signer's declared lineage, as apksig requires; `null` when
|
||||
either side's identity is unknown, with a limitation when the lineage
|
||||
origin is the unknown side. Also `v1_only` (which cannot distinguish a v1-only APK
|
||||
from one whose v2/v3 block was removed, since `X-Android-APK-Signed` is not
|
||||
read), strongest scheme, rotation and source-stamp presence.
|
||||
- Unknown is never reported as absent: a central directory that cannot be
|
||||
walked, or whose record count does not consume its declared extent, is an
|
||||
input error (v1 presence would otherwise be guessed); a signer without a
|
||||
certificate makes agreement `null`; scheme errors name the scheme and reason.
|
||||
|
||||
## Authority
|
||||
|
||||
Declared identity and digest consistency, never trust: no cryptographic signature
|
||||
verification, no certificate-chain or revocation evaluation, no authenticity
|
||||
claim. A full verifier would be a separate scope decision.
|
||||
|
||||
## Resource admission
|
||||
|
||||
One bounded read of the archive (512 MiB cap, no-follow, change-detected).
|
||||
Every ZIP-tail and signing-block size/offset is checked against the buffer.
|
||||
Signers per scheme, v1 signers and lineage nodes are capped at 32
|
||||
(format-justified: one node per rotation, one v3 signer per SDK range; apksig
|
||||
accepts at most 10 v1 signers); pairs and per-signer elements at 4096. Each v1
|
||||
block inflates to at most 1 MiB (bounded by the inflater, not by the declared
|
||||
size), and its DER is read within 32 nesting levels and 65,536 elements, with
|
||||
at most 32 certificates and 32 `SignerInfo`s. The result size is projected from decoded counts before
|
||||
materialisation (an upper bound), and the exact serialized size of the result and its limitations is checked against
|
||||
the 8 MiB output budget (authoritative); over-budget is a capacity refusal, not
|
||||
truncation. The content digest is
|
||||
computed at most once per algorithm in 1 MiB chunks. Peak memory is about
|
||||
2.5 times the file size (the shared stable read keeps its chunks and then
|
||||
concatenates them; 1.25 GiB measured for a 500 MiB APK, about 1.3 GB at the
|
||||
cap) plus bounded decoding overhead; the digest passes run synchronously (a streamed positioned-read design is
|
||||
a follow-up).
|
||||
|
||||
## Verification
|
||||
|
||||
`npm run verify:apk-signature` (Android SDK `apksigner` + `aapt2`, JDK
|
||||
`keytool`): builds a real minimal APK, signs default v2+v3, v1-only (RSA and
|
||||
EC), v1+v2+v3 on a legacy target, a rotated lineage with and without v1, rotation carried by v3.1, a source stamp and a copy with one changed byte, and cross-checks REA against `apksigner verify`,
|
||||
`apksigner lineage` and `keytool -printcert -jarfile` through CLI and MCP in Evidence parity, with the advertised
|
||||
schema validated. SDK-free unit and adapter tests cover framing (including Zip64
|
||||
and damaged archives), signer and lineage decoding with caps, the digest
|
||||
algorithm (including a pooled-Buffer regression), set-based posture, the output
|
||||
budget, and the provider's error surfaces. Independent review before handoff.
|
||||
|
||||
Format reference: APK Signature Scheme v2/v3 —
|
||||
https://source.android.com/docs/security/features/apksigning
|
||||
+54
-27
@@ -26,6 +26,14 @@ behavior, and complete CI evidence before merging.
|
||||
| `npm run check:pr` | Opt-in complete local deterministic gate and generated-file checks |
|
||||
| `npm run docs:check` | Generated-document validation and fresh onboarding example consistency checks |
|
||||
|
||||
The general source suite above runs on Linux in CI, and Windows CI runs a
|
||||
curated lane ([Native platform baseline in CI](#native-platform-baseline-in-ci)).
|
||||
A Windows source checkout can fail source tests for reasons unrelated to the
|
||||
change, such as POSIX fixture paths, `#!/bin/sh` tool stubs, or POSIX rename
|
||||
semantics. Compare failures against the base revision, or run the suite on
|
||||
Linux or WSL, before attributing them to a change. See `CONTRIBUTING.md` →
|
||||
Windows source checkouts.
|
||||
|
||||
For example:
|
||||
|
||||
```bash
|
||||
@@ -258,6 +266,13 @@ cases. Share setup without hiding inputs or expected evidence. For successful
|
||||
`.ok` assertion adds no coverage. Protocol fixtures must reject unmodeled
|
||||
commands and preserve actual producer reply shapes.
|
||||
|
||||
Packed bridge validation uses Python's standard-library AST and tar readers
|
||||
with `-I -S`: caller-local modules, `PYTHONPATH`, and site startup hooks must
|
||||
not replace those readers or execute during validation. Controlled `ast.py`
|
||||
and `sitecustomize.py` fixtures must leave their markers absent and still
|
||||
reject malformed packed Python with `SyntaxError`. This checks static package
|
||||
verification, not real Hopper or Ghidra behavior.
|
||||
|
||||
Stateless source-analysis matrices may share a file-owned stdio MCP connection
|
||||
and retain representative compiled CLI parity checks. Every distinct source
|
||||
scenario still runs through the public workflow; session-lifecycle scenarios
|
||||
@@ -707,13 +722,18 @@ miss. Native
|
||||
artifacts, dependency installs, package archives, and runtime verification remain
|
||||
host-local. Keep platform-dependent outputs out of this shared cache.
|
||||
|
||||
The Windows native verifier's private-runtime ACL observer reads SID rules once
|
||||
for both trustee validation and counting. It deliberately rejects the convenience
|
||||
`Access` property, which resolves account names. Its `private-runtime-acl` stderr
|
||||
markers report completed phases and elapsed time; inspect these on a timeout
|
||||
before attributing it to startup, ACL reads, or identity lookup. The original
|
||||
15-second deadline and all ownership, DACL, and readonly assertions remain in
|
||||
force. A passing retry alone does not explain a previous timeout.
|
||||
The Windows native verifier's private-runtime ACL observer reads the owner and
|
||||
access rules only in SID form, checks that every identity it compares is a
|
||||
`SecurityIdentifier`, and never uses `Get-Acl` or the convenience `Access`
|
||||
property, which resolves account names. Its Windows PowerShell observers call
|
||||
.NET members directly and build their JSON without cmdlets, pipelines, module
|
||||
autoloading, or type-table updates. The `private-runtime-acl` stderr markers
|
||||
report the powershell.exe process uptime and each completed phase; inspect them
|
||||
on a timeout before attributing it to startup, ACL reads, or identity lookup.
|
||||
The 60-second observer deadline bounds a hung observer while covering a cold
|
||||
powershell.exe start on a starved hosted runner. All ownership, DACL, and
|
||||
readonly assertions remain in force. A passing retry alone does not explain a
|
||||
previous timeout.
|
||||
|
||||
Record cold and warm preparation separately from fresh test execution when
|
||||
benchmarking the optional `npm run check:pr` gate. Compare multiple runs on the
|
||||
@@ -748,16 +768,16 @@ installation (`application.java.min` through `application.java.max`). Current
|
||||
12.1 releases require JDK 21 or newer and set no maximum. The lanes below still
|
||||
prove behavior on the verified Ghidra 12.1.4 and JDK 21 build.
|
||||
|
||||
| Ghidra lane | Supported runner/target | Additional local tools |
|
||||
| ------------------------------------------ | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| `npm run verify:ghidra` | Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:seeds` | POSIX; raw x86 fixture; seed ordering, BSS labels, analyzer readback and unknown-name rejection | Ghidra 12.1.4 and full JDK 21 |
|
||||
| `npm run verify:ghidra:swift` | macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:riscv` | Linux/macOS x64/arm64 host; source-owned RV32/RV64 ELF through CLI and MCP | RISC-V Clang, LLD, LLVM readelf, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:switch` | Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:aarch64-jump-table` | Any supported Ghidra host; AArch64 ELF; signed/unsigned byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:cross-format` | Any supported Ghidra host; also analyzes AArch64 ELF, x86-64 PE, and x86-64 Mach-O | `clang`, LLD, and `lld-link` in addition to host-lane prerequisites |
|
||||
| `npm run verify:ghidra:windows` | Controlled Windows x64 with native x86-64 PE; `-- --x86` selects native x86 PE | Ghidra 12.1.4, full JDK 21, and the matching native artifact |
|
||||
| Ghidra lane | Supported runner/target | Additional local tools |
|
||||
| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
|
||||
| `npm run verify:ghidra` | Linux x64/arm64 ELF or macOS x64/arm64 Mach-O | Host C compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:seeds` | POSIX; raw x86 fixture; seed ordering, BSS labels, analyzer readback, not-applicable and unknown analyzer names | Ghidra 12.1.4 and full JDK 21 |
|
||||
| `npm run verify:ghidra:swift` | macOS x64/arm64 Mach-O | Host Swift compiler, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:riscv` | Linux/macOS x64/arm64 host; source-owned RV32/RV64 ELF through CLI and MCP | RISC-V Clang, LLD, LLVM readelf, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:switch` | Linux x64 ELF; GCC/Clang optimized and stripped switch fixtures | GCC, Clang, GNU nm/objdump/strip, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:aarch64-jump-table` | Any supported Ghidra host; AArch64 ELF; signed/unsigned byte/halfword tables; host ARM64 Mach-O | Clang with AArch64 target support, Ghidra 12.1.4, and full JDK 21 |
|
||||
| `npm run verify:ghidra:cross-format` | Any supported Ghidra host; also analyzes AArch64 ELF, x86-64 PE, and x86-64 Mach-O | `clang`, LLD, and `lld-link` in addition to host-lane prerequisites |
|
||||
| `npm run verify:ghidra:windows` | Controlled Windows x64 with native x86-64 PE; `-- --x86` selects native x86 PE | Ghidra 12.1.4, full JDK 21, and the matching native artifact |
|
||||
|
||||
The optional Swift lane compiles a source-owned fixture and checks modern and
|
||||
provider-demangled procedure names through real stdio MCP, with and without the
|
||||
@@ -1243,7 +1263,9 @@ host behavior or native process-supervision controls.
|
||||
real `.nib` with Xcode `ibtool`, wraps it in a temporary app bundle, and runs
|
||||
the compiled CLI and stdio MCP server with production providers. It compares
|
||||
their decoded results and checks the view hierarchy, outlet, action, evidence coverage, and truncation
|
||||
status. Storyboard compilation additionally requires an installed iOS platform.
|
||||
status. It also recompiles the XIB with toggled control booleans and requires
|
||||
both decodes to match `ibtool --export` for each source. Storyboard compilation
|
||||
additionally requires an installed iOS platform.
|
||||
|
||||
Keep the provider-specific acceptance path independent from optional
|
||||
cross-compilers. Cross-format failures belong to the cross-format lane and must
|
||||
@@ -1479,14 +1501,19 @@ fallback weight for new files, rather than dividing only by file count.
|
||||
Each file keeps its full duration on one worker. Extra workers overlap other
|
||||
files in the same sequence group, and `process-boundary` runs after that
|
||||
group. History published before the competing-loader split still names only
|
||||
`javascriptLiteralPercentPaths.test.ts`. Until that artifact records one of
|
||||
the three new files, the sequencer assigns that recorded duration by the
|
||||
pre-split case counts: 54/192 to the remaining file and 46/192 to each new
|
||||
file. Every discovered file still belongs to exactly one shard; worker limits
|
||||
and project ordering remain unchanged. Fast qualification keeps eight built
|
||||
shards without coverage. The merge job rejects successful reports with missing,
|
||||
unexpected, or duplicate portable files. Coverage instrumentation and aggregate
|
||||
thresholds remain enforced in full qualification.
|
||||
`javascriptLiteralPercentPaths.test.ts`. Until an artifact records
|
||||
`javascriptLiteralLoaderOwnership.test.ts`, the sequencer assigns that
|
||||
recorded duration by the pre-split case counts: 54/192 stays with the
|
||||
percent-path file and 138/192 goes to the consolidated loader file. History
|
||||
that names the retired Early, Middle, and Late shards sums those durations
|
||||
onto the consolidated file instead, and scales a partial set as three equal
|
||||
shares. That weight is the old dual-surface cost. The demoted file runs every
|
||||
loader once through the shared stdio MCP session, so the first schedule
|
||||
overestimates it. Every discovered file still belongs to exactly one shard;
|
||||
worker limits and project ordering remain unchanged. Fast qualification keeps
|
||||
eight built shards without coverage. The merge job rejects successful reports
|
||||
with missing, unexpected, or duplicate portable files. Coverage instrumentation
|
||||
and aggregate thresholds remain enforced in full qualification.
|
||||
Missing or unusable history falls back to Vitest's default assignment. Only
|
||||
timing data is reused; live provider results are never cached. Turbo outputs are
|
||||
restored by task hash, with trusted default-branch runs as cache writers and PRs
|
||||
@@ -1818,7 +1845,7 @@ the built CLI, packs the checkout with `npm pack`, exports the bundle with
|
||||
registry package, and runs `agent-env run` on the bundle. It checks that the
|
||||
export reports the planned file set with the inputs copied byte-exact and a
|
||||
bundle served from the derived state, which serves fewer tools than the
|
||||
manifest because write tools are not served, that the runner exits `0` with
|
||||
manifest because write tools the capture did not confirm are not served, that the runner exits `0` with
|
||||
`tasks/smoke.json` reported as passed and the verifier's description naming
|
||||
that served tool count, and that no container exists
|
||||
after the run that did not exist before; the container inventory runs even when
|
||||
|
||||
@@ -17,7 +17,7 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
|
||||
13,src/domain/errors.ts:EvidenceIntegrityError,Evidence validation fails,Raw constructor message,May expose validation internals and lacks recovery,"Evidence is invalid or has changed. Recreate or re-import it, then try again.",implemented,focused assertions passed; full `npm run check` passed
|
||||
14,src/domain/errors.ts:UnknownRegistryError,Evidence-state mutation fails,"Residual unknown registry mutation failed: {reason}",Exposes internal subsystem and state token,"Missing IDs identify the session and retry; concurrent revision conflicts refresh current state.",implemented,focused assertions passed; full `npm run check` passed
|
||||
15,src/domain/errors.ts:ProcessCaptureError,Process capture fails,Raw constructor message,May expose process and cleanup internals,"Process capture could not complete. Check command access and limits, then try again.",implemented,focused assertions passed; full `npm run check` passed
|
||||
16,src/domain/errors.ts:AnalysisTimeoutError,Analysis deadline expires,"REA's {operation} operation reached its {timeoutMs}ms server/provider deadline; extending only the client deadline does not change this budget",Returned server deadline is distinct from a host timeout that discards the response,"Inspect partial Evidence and reported cleanup state; diagnose the provider or reduce supported work without assuming adjustable limits. JavaScript specifically names analysis_timeout_ms and exact source_paths; omit an unneeded filter rather than passing an empty array.",implemented,projection and public CLI/MCP timeout regression coverage; client recovery also taught during initialization and in the shipped skill
|
||||
16,src/domain/errors.ts:AnalysisTimeoutError,Analysis deadline expires,"REA's {operation} operation reached its {timeoutMs}ms server/provider deadline; extending only the client deadline does not change this budget",Returned server deadline is distinct from a host timeout that discards the response,"Inspect partial Evidence, provider diagnostics, and reported cleanup, then follow the returned remediation. Change a request field or REA_* setting only when the returned remediation explicitly recommends that change. Restart REA when that text says the value is read at startup. A mention that a control does not raise the deadline is not a recommendation to change it. When it names no setting, reduce supported work; do not invent one. JavaScript names analysis_timeout_ms and exact source_paths; omit an unneeded filter rather than passing an empty array.",implemented,generic projection does not invent a setting; producer remediation names analysis_timeout_ms or the applicable REA_* value; initialization and the shipped skill teach the same split
|
||||
17,src/domain/errors.ts:HopperTimeoutError,Provider deadline expires,"Hopper did not respond within {timeoutMs}ms",Exposes provider and internal deadline,"Analysis took too long. Try a smaller request, then run `rea doctor` if it continues.",implemented,focused assertions passed; full `npm run check` passed
|
||||
18,src/domain/errors.ts:ProviderSelectionError,Deep-provider selection fails,"Analysis provider selection failed ({reason}) for {requestedProviderId}; candidates: {candidateIds}",Retains non-secret local selection context that needs a safe structured projection,Reason-specific guidance with stable candidate IDs and actionable rejection diagnostics,implemented,focused registry/CLI/MCP assertions passed; full `npm run check` passed
|
||||
20,src/domain/errors.ts:AnalysisInputError,Parsed analysis input invalid,"Invalid analysis input for {operation}",Exposes operation identifier and lacks correction,"Analysis input is invalid. Check the arguments and try again.",implemented,focused assertions passed; full `npm run check` passed
|
||||
@@ -45,10 +45,10 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
|
||||
33,src/application/Setup.ts:configureDetectedClients(write),Agent configuration write fails,"{clientName} configuration write verification failed; no successful configuration was reported.",Exposes machine client key and gives no recovery,"Agent configuration could not be updated. Check file permissions, then rerun setup.",implemented,focused assertions passed
|
||||
34,src/application/Setup.ts:configureDetectedClients(readback),Written agent configuration cannot be verified,"{clientName} configuration readback verification failed; no successful configuration was reported.",Exposes machine client key and omits backup recovery,"Agent configuration could not be verified after writing. The original was restored and its backup at {backupPath} was retained. Repair the configuration file or restore that backup, then rerun setup. Without a backup: Agent configuration could not be verified. Repair the configuration file, then rerun setup.",implemented,focused assertions passed
|
||||
35,src/application/Setup.ts:installSkill,Skill install or semantic readback fails,Agent skill installation or readback failed.,No useful recovery action,"REA analysis skill could not be installed or verified. Check permissions for `~/.agents/skills`, then rerun setup.",implemented,focused assertions passed
|
||||
36,src/main.ts:serveStdio.onerror,Established MCP client connection reports an error,MCP stdio transport error,Exposes transport implementation and gives no recovery,REA lost its MCP client connection. Restart REA from your MCP client.,implemented,verified by injected transport onerror runtime test
|
||||
37,src/main.ts:serveStdio throw,MCP connection cannot be created,Failed to start MCP stdio transport,Exposes transport implementation and gives no recovery,REA could not start its MCP connection. Restart REA from your MCP client; run `rea doctor` if it fails again.,implemented,verified by injected transport startup-throw runtime test
|
||||
36,src/main.ts:serveStdio.onerror,Established MCP client connection reports an error,MCP stdio transport error,Exposes transport implementation and gives no recovery,"Retry the tool call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or it fails again, tell the user REA lost its MCP client connection and ask them to restart the MCP client.",implemented,verified by injected transport onerror runtime test
|
||||
37,src/main.ts:serveStdio throw,MCP connection cannot be created,Failed to start MCP stdio transport,Exposes transport implementation and gives no recovery,"Retry once. If it fails again, tell the user REA could not start its MCP connection and ask them to restart the MCP client, then run `rea doctor`.",implemented,verified by injected transport startup-throw runtime test
|
||||
38,src/main.ts:requestShutdown catch,Connection or analysis session close rejects,Failed to shut down cleanly,Does not explain duplicate-process risk or recovery,REA could not close cleanly. End the REA process before starting it again.,implemented,verified by injected handle-close rejection runtime test
|
||||
39,src/main.ts:entrypoint catch,Unhandled rejection escapes run(),Unexpected server startup failure,Vague and lacks diagnostic recovery,"REA could not start. Run `rea doctor`, then restart REA from your MCP client.",implemented,verified by injected entrypoint rejection test
|
||||
39,src/main.ts:entrypoint catch,Unhandled rejection escapes run(),Unexpected server startup failure,Vague and lacks diagnostic recovery,"Do not repeat this startup. Tell the user REA could not start and ask them to restart the MCP client, then run `rea doctor`.",implemented,verified by injected entrypoint rejection test
|
||||
40,src/application/Upgrade.ts:unknown-install-method,Running package is not owned by detectable global npm installation,Update manually with: npm install --global rea-agents@latest,Already gives exact safe recovery,Keep current copy,unchanged,existing focused assertion passed
|
||||
41,src/application/Upgrade.ts:install,npm global install subprocess exits unsuccessfully,npm install --global rea-agents@latest failed.,Does not identify bounded likely checks before retry,"REA could not update through npm. Check npm registry access and global install permissions, then run: npm install --global rea-agents@latest",implemented,focused assertion passed
|
||||
42,src/application/Uninstall.ts:removeClient read,Existing agent configuration cannot be read,Configuration could not be read.,No recovery step,"Configuration could not be read. Check file permissions, then rerun uninstall.",implemented,verified by injected filesystem read failure test
|
||||
@@ -76,7 +76,7 @@ harm_rank,location,trigger,current_copy,user_risk,proposed_replacement,implement
|
||||
63,src/application/process/ProcessCli.ts:readJson parse,Process scenario or evidence file is malformed JSON,Raw JSON parser exception through CLI framework,Can expose parser implementation and lacks recovery,"Process input file is not valid JSON. Repair the file, then try again.",implemented,verified by focused CLI test
|
||||
64,src/application/process/ProcessCli.ts:capture scenario parse,Scenario fails Process Capture schema,Raw Zod validation exception through CLI framework,Exposes schema internals and field paths without stable recovery,"Process scenario is invalid. Check its required fields and limits, then try again.",implemented,verified by focused CLI test
|
||||
66,src/application/process/ProcessCli.ts:wrong evidence,Comparison receives valid Evidence from another workflow,Expected REA process capture Evidence,Uses internal schema name and does not explain recovery,"Capture evidence is not from the current process-capture workflow. Create new capture evidence, then try again.",implemented,verified by focused CLI test
|
||||
67,src/application/process/ProcessCli.ts:unexpected,Unexpected parser/comparison/canonicalization failure escapes expected classifications,Raw thrown exception through CLI framework,Can expose stack or internal invariant,"Process command could not complete. Check the input files and run `rea doctor`, then try again.",implemented,verified by direct unexpected-cause projection test
|
||||
67,src/application/process/ProcessCli.ts:unexpected,Unexpected parser/comparison/canonicalization failure escapes expected classifications,Raw thrown exception through CLI framework,Can expose stack or internal invariant,"Message keeps the sanitized cause. Remediation: Do not repeat this command. Tell the user the process command could not complete and ask them to check the input files, then run `rea doctor`.",implemented,verified by direct unexpected-cause projection test
|
||||
68,src/server/registerEvidenceTools.ts:object guard,Validated provider-tool input is unexpectedly not an object,Provider tool input was not an object,Exposes provider architecture and implies caller can fix impossible state,Remove the impossible duplicate guard and parse through the declared JSON-object schema.,removed,verified by typecheck and MCP integration tests
|
||||
69,src/server/registerEnhancedTools.ts:object guard,Validated enhanced-tool input is unexpectedly not an object,Validated enhanced input was not a JSON object,Exposes validation/JSON implementation detail,Remove the impossible duplicate guard and parse through the declared JSON-object schema.,removed,verified by typecheck and enhanced MCP integration tests
|
||||
70,src/server/registerOfficialTools.ts:object guard,Validated direct-tool input is unexpectedly not an object,Validated MCP tool input was not a JSON object,Exposes MCP validation implementation detail,Remove the impossible duplicate guard and parse through the declared JSON-object schema.,removed,verified by typecheck and official MCP integration tests
|
||||
@@ -116,3 +116,5 @@ Then rerun this installer.",implemented,verified by shell branch inspection
|
||||
90,install.sh:version mismatch,Installed rea reports a different version,installed version ACTUAL does not match EXPECTED.,Low risk; gives exact integrity mismatch,Keep current copy,unchanged,verified by exact installer test
|
||||
90.5,@modelcontextprotocol/server:tool input validation,Caller sends arguments that violate a declared tool schema,"Canonical invalid_request with the SDK diagnostic plus recovery.kind=inspect_input_contract and a bounded instruction to read the advertised schema or call tools/list",Keeps field/type correction while telling the agent how to recover; diagnostics over 4 KiB stay on the existing retained-error path,Normalize ordinary validation failures and retain oversized diagnostics,verified by real in-memory and stdio MCP client/server calls with wrong argument types
|
||||
90.6,incur:unknown CLI command,Caller enters an unknown REA command,"'COMMAND' is not a command for 'rea'. Suggested command: rea --help",Low risk; identifies input and exact discovery action,Keep framework-generated copy,unchanged,verified by real packaged dispatcher invocation
|
||||
91,src/application/binaryDiagnostics/ByteEntropyService.ts,Invalid file or window,"Expected a regular file.; Selected file could not be found.; Selected byte window extends beyond the end of the file.; File size exceeds exact numeric offset representation.",Invalid selection cannot produce complete entropy Evidence,Choose a readable regular file and a window within its byte size,implemented,entropy service and MCP boundary tests
|
||||
92,src/domain/analysisView/byteEntropyView.ts,Non-summary entropy view,Byte entropy Evidence supports the summary view; retrieve parent Evidence for the complete block series.,Unsupported view selection,Use the summary view or retrieve complete parent Evidence,implemented,typed input rejection
|
||||
|
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,188 @@
|
||||
# APK signing verification
|
||||
|
||||
## Current posture qualification
|
||||
|
||||
Runtime `4fcbdf6fb78873b47f424c5e15ebda6af715befe`, tree `5cadce8a73107560d77045de481559b6eb6ef666`,
|
||||
integrates main `2c3dd946d59c870388b03c3417eeb5218af96eeb`. The latest review exposed two additional
|
||||
derived-fact errors: unchanged multi-signer sets could look rotated, and an
|
||||
unknown current signer could be replaced with its readable lineage origin.
|
||||
Three regressions failed against the previous published owner, with14 controls.
|
||||
The owner now shares one signer-set comparison and preserves missing identity.
|
||||
|
||||
- **225 source tests /14 suites**, **11 public CLI/MCP tests /3 suites**,
|
||||
**23 real SDK cases**, uncached typecheck/lint and CI static passed.
|
||||
- The new public regression proves the missing-identity result through real
|
||||
CLI, SDK summary and retained lineage reads after source deletion. The
|
||||
earlier416-test combined registration/schema matrix stays pinned to8f8541c;
|
||||
contracts, raw framing and process-wide admission have not changed here.
|
||||
- Current complete versus selected F-Droid task, same answer and complete
|
||||
parent: **5,246 → 4,230 tokenx estimates**,
|
||||
**3.850 → 3.591s** elapsed,
|
||||
**4.732 → 4.666 CPU s**,
|
||||
**694,124,544 → 689,983,488 bytes** peak family RSS.
|
||||
|
||||
JSON `latest_posture_fix` records exact profiles and scope. The prior
|
||||
three-provider comparison remains pinned to35a9b13. These are fresh-process,
|
||||
warm-cache observations; actual model tokens, statistical speedups,
|
||||
cryptographic authenticity and independent Windows/macOS behavior are unverified.
|
||||
Current-head CI/review follows publication.
|
||||
|
||||
## Integration qualification at8f8541c
|
||||
|
||||
Runtime `8f8541c39586d78911e4743d6ec07f75c449a0d5`, tree `50b93dccf15c40f9067030614d743d59e0aa334c`,
|
||||
integrates main `2c3dd946d59c870388b03c3417eeb5218af96eeb`. The previous published head passed all
|
||||
technical CI and5/5 review before new upstream Python pyc registrations caused
|
||||
three additive conflicts. Documentation links, test adapter coverage and named
|
||||
tool effects now preserve both capabilities. The APK provider/domain/view and
|
||||
stable-file reader are byte-identical to35a9b13.
|
||||
|
||||
- **222 source tests /14 suites**, **416 compiled public tests /15 suites**
|
||||
(including actual advertised schemas and combined CLI/MCP registrations),
|
||||
**23 real SDK cases**, uncached typecheck/lint and exact CI static passed.
|
||||
- Complete versus selected F-Droid task, same answer and complete parent:
|
||||
**5,243 → 4,227 tokenx estimates**,
|
||||
**3.616 → 3.799s** elapsed,
|
||||
**4.389 → 4.825 CPU s**,
|
||||
**709,554,176 → 717,103,104 bytes** peak family RSS.
|
||||
These fresh-process observations use warm OS caches; used input declarations
|
||||
and every call/reply/export are counted, with full-catalog estimates in JSON.
|
||||
- The earlier three-provider memory comparison remains pinned to35a9b13;
|
||||
it is not a new measurement of this integration. No selection bypasses hashing,
|
||||
and no cryptographic authenticity or statistical speedup is claimed.
|
||||
|
||||
JSON `latest_main_integration` records exact commands and input receipts.
|
||||
The earlier sections retain their original commit-specific evidence.
|
||||
Current-head CI and review follow publication.
|
||||
|
||||
## Review follow-up qualification at35a9b13
|
||||
|
||||
That review runtime is `35a9b13adaaccff109125664898d99886d76d959`, tree `db41f5fbee2d1dca69544c221324daa8238b8e8b`.
|
||||
The original qualification below remains historical at `f85a887`; JSON
|
||||
`review_followup` records the new commands, outcomes and costs.
|
||||
|
||||
- **216 source tests /13 suites**, **394 compiled boundary tests /13 suites**,
|
||||
**23 real SDK cases**, uncached typecheck/lint and exact CI static passed.
|
||||
The unchanged schema contracts retain the earlier15-check qualification.
|
||||
- Corrected regressions reproduce **13 assertion failures and67 passing controls**
|
||||
against the published producer source. Required lineage and CMS signature
|
||||
framing, present empty signer sets, full ZIP/Zip64 directory extents and
|
||||
pre-aborted cancellation reasons are now handled at their owning boundaries.
|
||||
- A shared FIFO admits one active APK scan and eight pending requests across
|
||||
provider instances. Pending scans hold no APK bytes. Capacity, cancellation,
|
||||
release recovery and close/drain have deterministic tests; production-boundary
|
||||
tests use real files with a controlled descriptor pause.
|
||||
- Three simultaneous scans of the same generated/re-signed180MB input:
|
||||
**981,049,344 → 448,929,792 bytes**
|
||||
peak process-family RSS, **2.355 → 2.257s** elapsed,
|
||||
**3.116 → 2.491 CPU s**. Every result matches the
|
||||
same source SHA, SDK signer fingerprints and declared content digests.
|
||||
- Current complete versus selected F-Droid SDK task, same answer and complete
|
||||
parent: **5,243 → 4,227 tokenx estimates**,
|
||||
**3.564 → 3.824s** elapsed,
|
||||
**4.289 → 4.271 CPU s**,
|
||||
**754,368,512 → 762,630,144 bytes** peak RSS.
|
||||
|
||||
These are single fresh-process observations with warm OS caches. Queueing bounds
|
||||
live materialization; it does not make selection skip hashing or establish a
|
||||
statistical speedup. Earlier failed fixture probes, lint and library preparation
|
||||
checks are retained in the JSON record. Current-head CI/review follows publication.
|
||||
|
||||
## Original qualification
|
||||
|
||||
Issue [#4537](https://github.com/morluto/rea/issues/4537), existing PR
|
||||
[#4158](https://github.com/morluto/rea/pull/4158). Runtime `f85a887a05fa62d0675139746dbce6eba7548aee`,
|
||||
tree `ff1f42613847e198988bf84dfb2d50d578084bdb`, integrates main `a4a548718d696d7329ffc7b9caa64ee72a215461` while retaining
|
||||
contributor head `76445a83c8e6542131f7b38290ec749b3a199ba4`. The [JSON record](apk-signature-2026-10-22.json)
|
||||
pins commands, inputs, resource profiles, failed controls and limits.
|
||||
|
||||
The selected-file provider owns ZIP/signing/PKCS#7 parsing and digest work.
|
||||
The shared application workflow owns identity validation, cancellation and
|
||||
Evidence retention. CLI/MCP use the same summary and signer/digest/lineage
|
||||
projections; immutable report caches recheck parent subject identity. Selected
|
||||
pages preserve identity links, source scope and unknowns after source deletion.
|
||||
Complete source Evidence remains accessible and CLI compact delivery exports it.
|
||||
|
||||
## Executed checks
|
||||
|
||||
- **201 source tests /12 suites passed**, and uncached typecheck/lint passed.
|
||||
Tests ran on the exact staged tree; commit hooks left that tree identical.
|
||||
- **408 compiled public tests /14 suites passed**, including15 advertised
|
||||
schema checks, both SDK input profiles, CLI/MCP parity, retained reads after
|
||||
deletion, batch/field/page semantics, partial v1/lineage facts, cached subject
|
||||
identity, cancellation/cleanup, stable readers and CI selection/required gates.
|
||||
- The original real-toolchain verifier passed before and after: **23
|
||||
verifier-reported CLI/SDK cases**, including v1 RSA/EC, legacy v1+v2+v3,
|
||||
rotation, v3.1, source stamp, tampered bytes and forged declared digests.
|
||||
Forged declarations can match bytes while apksigner rejects their signature;
|
||||
REA preserves that distinction. Build Tools36.1.0/platform36 were downloaded
|
||||
from Google, checked against repository metadata and pinned SHA-256.
|
||||
- Exact CI static checks passed. Documentation validation follows this generated
|
||||
report; current-head GitHub CI and review are checked separately.
|
||||
|
||||
The initial matrix had four failures and28 controls. Two missing/truncated
|
||||
framing cases and one lost algorithm-field case were genuine; its universal
|
||||
suffix-refusal assertion was incorrect and is withdrawn. The owner now checks complete
|
||||
signer framing, matching signed/outer v3 SDK bounds and exact algorithm records.
|
||||
Reports retain raw unsigned algorithm IDs, declared digest hex, signature lengths
|
||||
and unknown algorithms. Real SDK signing exposed an extra empty signed-data
|
||||
field accepted by Android readers. Scheme/signer/signed-data/record suffixes
|
||||
now retain explicit uninterpreted counts instead of being mislabeled malformed;
|
||||
required lengths and repeated v3 SDK bounds remain checked. Representation expansion is charged before report
|
||||
allocation. Shared fixtures now produce full framing. No cryptographic signature
|
||||
verification is claimed.
|
||||
|
||||
The larger public matrix exposed two old30s test timeouts. The SDK matrix now
|
||||
passes its actual advertised definition; malformed stateless inputs share one
|
||||
production stdio session while representative CLI cases remain. All original
|
||||
assertions and deadlines are retained. Failed preparations and superseded runs
|
||||
are explicitly recorded in JSON.
|
||||
|
||||
## Public artifact and resource evidence
|
||||
|
||||
The unchanged official [F-Droid APK](https://f-droid.org/F-Droid.apk) is
|
||||
12,569,951 bytes, SHA-256
|
||||
`19d51fad482239f6df6a99120e34513667e9a86d136dafdc4a211a68c3327b52`. SDK `apksigner verify -v --print-certs`
|
||||
verified v3 and identified `43238d512c1e5eb2d6569f4a3afbf5523418b82e0a3ed1552770abb9a9c9ccab`; every task reproduces that
|
||||
fingerprint and exports the complete parent report. URL, response metadata and
|
||||
input identity are retained. Android code was never executed.
|
||||
|
||||
A distinct generated stress input adds a160MiB stored asset to that APK and
|
||||
re-signs it with a generated fixture key. It is
|
||||
180,350,384 bytes, SHA-256
|
||||
`cf3a3936f90bb79dabe2598dc4290f8c47d10680993c054fe8cac9d391fadaaf`; apksigner verifies it. Three provider
|
||||
calls per input check exact source identity, signer and content-digest equality.
|
||||
Their first/repeated in-process timings are recorded separately. An additional
|
||||
512MiB+1 sparse file is refused before whole-file allocation.
|
||||
|
||||
Whole resource probe: **2.047 s / 2.393 CPU s / 449,503,232 bytes peak sampled family RSS**. The whole-file
|
||||
reader retains chunks and a concatenated buffer; its512MiB input cap is not a
|
||||
512MiB process-memory claim. Selection reduces delivery and does not skip file
|
||||
SHA-256 or supported digest analysis.
|
||||
|
||||
## Equivalent completed SDK task
|
||||
|
||||
Task: identify every declared v3 signer fingerprint in the unchanged APK and
|
||||
export complete Evidence. The current complete and selected tasks export the
|
||||
same parent Evidence identity. Earlier normalized facts differ because the
|
||||
original report omitted raw algorithm details; all three answers and source
|
||||
SHA-256 agree. The original baseline predates the later main integration;
|
||||
whole-catalog changes include other upstream tools. Only the two current tasks
|
||||
share identical advertised definitions and a complete parent identity.
|
||||
|
||||
| Delivery | Used full input declarations + all calls/replies, tokenx | Entire catalog + all calls/replies, tokenx | Task resources |
|
||||
| ----------------- | -------------------------------------------------------: | -----------------------------------------: | ----------------------------------------------------------------- |
|
||||
| Original complete | 4,156 | 1,140,214 | 3.282 s / 4.083 CPU s / 751,788,032 bytes peak sampled family RSS |
|
||||
| Current complete | 5,246 | 1,157,394 | 3.612 s / 4.339 CPU s / 698,650,624 bytes peak sampled family RSS |
|
||||
| Current selected | 4,230 | 1,156,378 | 3.508 s / 4.384 CPU s / 700,706,816 bytes peak sampled family RSS |
|
||||
|
||||
These are estimates, not actual model tokens. Counts include actual advertisements,
|
||||
arguments, complete replies and exports; initialization and JSON-RPC envelopes
|
||||
are unmeasured. Processes/sessions are fresh with warm OS caches. No disk-cold
|
||||
or statistically stable timing claim is made. CPU uses payload-family wait-rusage
|
||||
delta; RSS samples every20ms and sums descendant processes. Heavy tasks run
|
||||
serially on two CPUs with8GiB/420s guards; no guard fired.
|
||||
|
||||
The added GitHub lane requires pinned SDK oracle tools and has no silent skip.
|
||||
Local results establish Linux static-reader behavior. Windows/macOS APK behavior,
|
||||
platform installation, authenticity, cryptographic signer/lineage validity and
|
||||
v1 entry coverage remain unverified.
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,228 @@
|
||||
# Static JavaScript route inventory verification
|
||||
|
||||
Issue [#4772](https://github.com/morluto/rea/issues/4772). Qualified runtime
|
||||
`b49ee40bf330b7bf1c5180b14dcc104c1c613a8e`, tree `0e5f8fe2feae1796c340c07c26e0e28e5d4328d8`,
|
||||
integrates main `c8ed32853881b08aa83459d4da55a28a4e09b0c8`. Recorded at
|
||||
`2026-10-22T08:30:58.269Z`. Subsequent verification-only
|
||||
commits change these reports. The [JSON record](javascript-route-inventory-2026-10-21.json)
|
||||
pins original inputs, exact commands, profiles, failed controls and historical revisions.
|
||||
Current-head GitHub CI and automated review must be checked separately.
|
||||
|
||||
The latest main contracts, selected environment and ASAR admission remain shared
|
||||
by CLI and MCP. The staged-code hook runs lint fixes before formatting, resolving the
|
||||
producing order behind the earlier CI format failure.
|
||||
|
||||
## Current implementation and executed checks
|
||||
|
||||
Shared source analysis produces stable route, occurrence, source and gap facts.
|
||||
CLI and MCP use the same retained Evidence views, with filtering, selected fields,
|
||||
batch identities and pagination; complete source Evidence remains accessible.
|
||||
Coverage and explicit unknowns survive compact delivery and source deletion.
|
||||
Selected CLI delivery requires a complete Evidence export; MCP retention is
|
||||
checked before analysis. Immutable snapshots use weak identity caching with
|
||||
parent identity checks; live target state is not cached.
|
||||
|
||||
Route scanning reuses semantic bindings, mutation, literal-field, reference and
|
||||
truthiness owners. Navigation data ownership excludes query/search/hash values,
|
||||
callee/projection/predicate edges and definitely inactive branches while keeping
|
||||
independent lexical routes. Registration patterns preserve optional syntax and
|
||||
initializer ranges. Unrecognized array/string methods do not become navigation.
|
||||
Rest-copy exclusions, unresolved projections, mutations and cycles retain their
|
||||
appropriate independent facts or unknowns. Next page classification shares one
|
||||
owner, including private folders, encoded public underscores and unresolved
|
||||
interception context. Source ranges use one-based lines, zero-based UTF-16
|
||||
columns and exclusive ends.
|
||||
|
||||
Source/fact and inventory budgets are 8 MiB; optional facts fit the actual
|
||||
remaining combined-result budget. Selected pages include the sealing digest
|
||||
within their 1 MiB bound. Cancellation preserves earlier facts and Evidence;
|
||||
worker cleanup and capacity failures retain typed outcomes.
|
||||
|
||||
The current integration uses main's canonical Evidence publication/cleanup owner
|
||||
and deferred MCP runtime/catalog. CLI delivery explicitly produces JsonValue and
|
||||
projects the two publication receipt scalars without cloning complete Evidence.
|
||||
The canonical web-bundle state and coordinate owners replace duplicate types.
|
||||
The coordinator parser-import boundary remains covered.
|
||||
|
||||
**3847 source checks and 116 compiled public checks passed**,
|
||||
including 245 scanner regressions, actual full/compact SDK profiles, both JSON
|
||||
Schema dialects, source selection/deletion, retained reads, cancellation,
|
||||
recovery, real workers and CLI/browser adapters. The current public selection additionally covers ASAR admission, selected
|
||||
environment, CLI resource closing and Electron file scope; earlier startup/
|
||||
publication checks remain pinned to their recorded runtime.
|
||||
Selected-path identity runs separately with its original 30-second deadline.
|
||||
Typecheck/lint passed; documentation checks are recorded in JSON after generation.
|
||||
The main-baseline service-worker deadline correction [#6186](https://github.com/morluto/rea/pull/6186)
|
||||
is included; it retains the original stalled-handshake assertions and test guard.
|
||||
|
||||
A valid review case exposed overwritten query fields hiding independent route
|
||||
initializers: seven corrected regressions failed before the fix, while182 tests
|
||||
passed, including four new effective/uncertain-data controls. Ownership now
|
||||
consults the existing cached effective-field reader. A known final slot or method
|
||||
excludes discarded references; unknown spreads/computed keys remain conservative.
|
||||
Full/compact SDK reads after source deletion and both browser adapters retain
|
||||
the independent route. The public SDK assertion requires the exact expected count
|
||||
and every expected path. The record preserves failed validation and a corrected
|
||||
initial test-harness field name; no invalid harness failure is counted as a bug.
|
||||
|
||||
A second review case exposed selected member query data becoming page routes.
|
||||
Six direct/computed/optional, alias, nested and array-member cases failed while 189
|
||||
controls passed. The existing static member-chain helper now lives in the shared
|
||||
AST owner; both require consumers use it and its old export is removed. Optional
|
||||
admission checks charge each path step before retention. Navigation source
|
||||
selection reuses canonical possible-value sources at each read and one selected-slot
|
||||
walk shared with copied initializers, without claiming whole receivers or callee
|
||||
namespaces. Current full/compact retained SDK queries and both browser adapters
|
||||
exclude the selected query object while preserving its independent sibling.
|
||||
|
||||
A previous runtime's public batch hit two original 30-second schema deadlines.
|
||||
That failed attempt, successful isolated rerun and their combined resource cost
|
||||
remain pinned in JSON. Current execution runs schema validity alone, 18 other
|
||||
suites together and selected-path identity alone, retaining original assertions
|
||||
and deadlines.
|
||||
|
||||
Five prototype/own-property cases failed before the latest fix while 198 passed.
|
||||
The field owner now reuses the existing AST prototype-setter predicate. Setters
|
||||
retain unresolved inheritance but cannot overwrite computed/shorthand own keys;
|
||||
setter data references are not discarded as own-key writes. Direct, member and
|
||||
destructured query selection share this owner. Full/compact historical SDK reads
|
||||
and both browser adapters exclude the prototype-shaped query payload.
|
||||
The runtime retains main's integer-schema allocation optimization and integrates
|
||||
the latest main contracts, shared keyed ordering and retained-parent parser. The
|
||||
only merge conflict was the analysis-view description: the complete upstream
|
||||
text plus the exact route-view sentence were retained. Current catalog changes
|
||||
and fresh client/schema checks are recorded separately in JSON.
|
||||
|
||||
A later review found that lifetime identity eligibility retained fields proved
|
||||
replaced before a query read. Eight cases failed while 206 controls passed. Both
|
||||
member reads and direct identifiers now use canonical value-flow sources or
|
||||
binding snapshots for their read site. Unknown order keeps possible sources;
|
||||
known replacements exclude stale ones. Binding-once deduplication is removed
|
||||
because separate reads may select different values; syntax-cycle and resource
|
||||
bounds remain. Both SDK profiles after deletion and both browser adapters preserve
|
||||
the independent route through member and direct-variable replacements, while
|
||||
actual replacement payloads and earlier/possible query data remain excluded.
|
||||
|
||||
A further review reproduced four failures through conditional aliases, unknown
|
||||
array members and nested receiver aliases, while 215 controls passed. The shared
|
||||
mutation producer now distinguishes proved receiver identity from possible
|
||||
aliasing. Canonical read-time selection requires both identity and ordering
|
||||
before discarding original values or superseded prefixes. Deferred reference
|
||||
paths retain this proof and assigned-value provenance; deduplication includes
|
||||
certainty. Direct writes through the read's own alias still prove replacement.
|
||||
Both public SDK profiles after deletion and both browser adapters cover a
|
||||
distinct uncertain-alias payload without claiming the entire receiver.
|
||||
|
||||
CI also caught our integration regression in the unrelated Dart ABI advertisement:
|
||||
lint removed a slash escape, making the pattern invalid in JavaScript `v` mode.
|
||||
The ABI schema now encodes the slash as a hex code point, consistent with existing
|
||||
portable patterns and stable under lint fixes. The original advertised-pattern
|
||||
client test is included in this qualification. The prior inference that escaped
|
||||
and unescaped character classes were equivalent across clients is withdrawn;
|
||||
no Dart-provider execution is claimed. The interrupted superseded local run and
|
||||
failed CI are retained in the JSON record.
|
||||
|
||||
Two later review cases reproduced eleven failures while 222 controls passed:
|
||||
possible query fields cleared by spreads became pages, and selected member
|
||||
configurations claimed unrelated siblings. The selected-slot walk now uses the
|
||||
canonical possible object/array reference owners; exact fields still prove
|
||||
positive destinations. Configuration ownership preserves member and copied
|
||||
initializer paths through the same walk. Conditional-container selection reuses
|
||||
existing value-edge/truthiness rules. Normal paths advance offsets; exceptional
|
||||
path expansion is charged before allocation and node/path visits bound cycles.
|
||||
|
||||
This also exposed a canonical computed-key overwrite defect: an independent
|
||||
semantic-analysis regression failed with unknown coverage before the owner fix
|
||||
and retains its original literal afterward. Earlier unknown computed values are
|
||||
excluded by later definite slots; a getter paired with only a later setter stays
|
||||
possible. Accessor and conditional-container controls pass. Both SDK profiles
|
||||
after source deletion and both browser adapters exclude a distinct possible-query
|
||||
payload and preserve the selected configuration's independent sibling with exact
|
||||
counts. Unsupported projected configurations keep explicit gaps. Actual SDK
|
||||
tool definitions remain structurally identical to the preceding qualified runtime.
|
||||
|
||||
The next review reproduced two growing getter projections exhausting the 8 MiB
|
||||
route budget and hiding an independent route. An existing direct reader golden
|
||||
also retained overwritten computed-key alternatives; the earlier selected test
|
||||
matrix had omitted that producer suite. Three cases failed and 241 controls
|
||||
passed, including a finite repeated-getter control. The source-order expectations
|
||||
now match the shared owner's definite overwrite rule.
|
||||
|
||||
A subsequent review showed that suffix lengths are insufficient recursion proof:
|
||||
two finite getter chains were marked cyclic, leaking query data as page routes;
|
||||
243 controls passed. That inference is withdrawn. Selected-value traversal now
|
||||
uses value/lookup/finish frames: evaluate each canonical returned-reference path,
|
||||
clear its unfinished receiver/key, then resume the caller's member path. Scoped
|
||||
visited maps preserve independent result sinks. The suffix-length heuristic and
|
||||
suffix concatenation have been removed. Three finite/nested cases pass alongside
|
||||
genuine direct/mutual recursive getters and finite repeated selections. Both
|
||||
retained SDK profiles after deletion and both browser adapters exclude distinct
|
||||
finite-getter query payloads while retaining exact independent routes and actual
|
||||
cyclic gaps.
|
||||
Current source qualification covers **all 133 JavaScript domain suites plus eight
|
||||
related suites**. It ran on the exact staged source tree; commit hooks left that
|
||||
tree identical, so its source tests were not needlessly repeated after commit.
|
||||
New build-dependent public and real-artifact workflows ran on the frozen runtime.
|
||||
|
||||
## Real production inputs and oracle
|
||||
|
||||
Every one of the 67 saved production scripts was rehashed and freshly analyzed
|
||||
through production isolated workers, with one semantic projection node, 512 MiB
|
||||
worker old-space and the existing 15-second source deadline. Application sources
|
||||
were read as data. No source facts were reused for this qualification.
|
||||
|
||||
| Homepage capture | Scripts | Bytes | Completed | Route checkpoints | Rooted patterns |
|
||||
| ---------------- | ------: | --------: | --------: | ----------------: | --------------: |
|
||||
| nextjs-org | 29 | 1,533,098 | 28 | 28 | 0 |
|
||||
| shadcn-ui | 38 | 2,465,078 | 36 | 37 | 18 |
|
||||
|
||||
The full task cost **207.348 s / 313.817 CPU s / 502,669,312 bytes peak family RSS**. The previous pinned runtime `972c5ad86` full-worker run
|
||||
took 277.184 s/409.684 CPU s. These are individual observations, not a timing
|
||||
distribution. Earlier paired source elapsed observations
|
||||
remain pinned to their measured revisions in JSON. There is no controlled causal ablation or repeated timing
|
||||
study, so neither the source correction nor the main integration is assigned
|
||||
a performance cause. Configured deadlines and partial/unavailable
|
||||
facts are explicit. Zero supported rooted Next patterns does not establish zero
|
||||
application routes. Lazy/server-only code, reachability and the full application
|
||||
route denominator remain unknown.
|
||||
|
||||
Independent direct-byte inspection of the
|
||||
[86,218-byte shadcn production chunk](https://ui.shadcn.com/_next/static/immutable/chunks/0ftd4gqztonqp.js)
|
||||
finds exactly 18 quoted `/docs` patterns, including 12 direct href properties.
|
||||
SHA-256: `45c6ead1e08cc578464e7a0c6b2e6764d7af83f99039e291f288a971b988b3fc`.
|
||||
Every fresh SDK and browser verifier requires the entire sorted 18-pattern oracle,
|
||||
original source identity, provenance and complete retained Evidence.
|
||||
|
||||
An owned Chromium 156.0.8078.4 compiled this source via
|
||||
Runtime.compileScript/persistScript on a blank local page without running it.
|
||||
Actual CLI and official SDK analyze_web_bundle calls passively captured it via
|
||||
CDP and returned all 18 patterns, original URL and SHA-256. Source parsing was
|
||||
complete, enumeration partial and application coverage unknown. Owned process
|
||||
and temporary-profile cleanup succeeded. Cost: **10.379 s / 13.481 CPU s / 2,091,114,496 bytes peak family RSS**.
|
||||
|
||||
## Equivalent SDK task costs
|
||||
|
||||
Both fresh sessions list `/docs` patterns with confidence, mechanism and source
|
||||
provenance, preserve complete Evidence, perform three retained follow-up reads
|
||||
and export Evidence. They assert the same complete parent Evidence identity.
|
||||
Measurements include the instrumentation and use full input advertisements.
|
||||
|
||||
| Delivery | Used declarations + all calls/replies, tokenx | Entire catalog + all calls/replies, tokenx | Task resources |
|
||||
| -------- | --------------------------------------------: | -----------------------------------------: | ------------------------------------------------------------- |
|
||||
| Complete | 1,934,534 | 3,035,567 | 13.785 s / 18.104 CPU s / 1,356,595,200 bytes peak family RSS |
|
||||
| Selected | 58,902 | 1,159,935 | 12.243 s / 15.909 CPU s / 1,303,572,480 bytes peak family RSS |
|
||||
|
||||
Tokenx counts actual advertised schemas, arguments and full SDK replies, including
|
||||
exports and follow-ups; it estimates tokens rather than actual model usage.
|
||||
Initialization, JSON-RPC envelopes and separate token-count instrumentation costs
|
||||
remain unmeasured. Processes/sessions are fresh with warm OS caches; there is no
|
||||
filesystem-cache flush, repeated statistical timing study or claim of stable
|
||||
latency/analysis speedup. Default compact-advertisement costs are not measured.
|
||||
CPU is payload process-family wait-rusage delta and RSS samples all descendant
|
||||
tasks every 20 ms. Heavy tasks run sequentially on two CPUs with an 8 GiB family
|
||||
RSS/420-second guard. No qualification guard fired.
|
||||
|
||||
The JSON preserves prior failures, source-owner corrections and their exact
|
||||
before/after revisions. Historical CI/review does not establish the current head.
|
||||
Private issue samples, complete application routes, live route execution,
|
||||
external cloud clients and Windows/macOS provider execution remain unverified.
|
||||
Loaded 100 of 858 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user