Compare commits

...
Author SHA1 Message Date
Kaoru0822 198aa6e4aa Merge branch 'feat/4057-native-export-inventory' into feat/4057-verify-native-exports 2026-10-22 08:10:02 +00:00
Kaoru0822andClaude Opus 5.5 871613860d fix(native): bound export inventories and read header-backed PE tables
Address review of the export-table reader:

- Charge every retained row and its name, forwarder and version text
  against shared 262144-row and 16 MiB allowances before storing it, so
  repeated long names or PE aliases cannot exhaust memory.
- Map PE RVAs through the headers as well as sections; the PE image
  reader now exposes its SizeOfHeaders mapping.
- Keep ELF exports with version "unknown" when .gnu.version lies outside
  the file, and report an empty version-definition name as unresolved.
- Reject a .dynsym without its mandatory null entry as malformed.
- Decode the ELF section directory once.
- Correct the docs on which gaps drop rows and which keep them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 08:08:30 +00:00
Kaoru0822andClaude Opus 5.5 6be1c9e272 feat(managed): verify P/Invokes against portable native export Evidence
verify_managed_native_boundaries accepted only inspect_macho exports and
one analyze_function dossier per function, so checking a library's
P/Invokes on a non-macOS host took one engine call per declaration.

It now also accepts inspect_native_exports Evidence: one record supplies
a whole PE export directory or ELF dynamic symbol table. An exact name
verifies only when a by-name lookup binds it; a PE forwarder or an ELF
symbol with only a hidden or unreadable version is inferred with basis
qualified-export-candidate and a limitation naming the reason.
Ordinal-only PE exports match no declaration.

Fixes #4057

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 07:31:34 +00:00
Kaoru0822andClaude Opus 5.5 65a3128873 Merge origin/main into feat/4057-native-export-inventory
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 07:27:01 +00:00
Kaoru0822andClaude Opus 5.5 c083fedaf4 fix(cli): exclude declared sensitive values from parser diagnostics (#6258)
* fix(cli): exclude declared sensitive values from parser diagnostics

Incur rejects malformed command tokens and output-flag values before the
web capture service applies --sensitive-value declarations, so its
diagnostics repeated declared literals.

Read declarations from the tokens Incur leaves for command parsing,
independently of parse success. Commands accepting the option get an
outermost middleware that excludes declarations from parse, validation
and thrown failures while keeping type, code and exit status. Rejected
--format/--token-limit/--token-offset values are replaced by a
literal-free value Incur still rejects. The option definition and the
Incur builtin-flag rules now have one owner each.

Fixes #6252

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): redact whole rejected output-flag diagnostics

Incur builds its rejected --format/--token-limit/--token-offset message
from fixed text and the value, outside any middleware. Substituting only
the value left declarations that match the fixed text, such as "token"
in "--token-limit", in the diagnostic.

Report that first rejection at the serve boundary instead, redacting the
whole message as parser failures are, while keeping the UNKNOWN code and
Incur's toon or TTY rendering. Incur's message is now mirrored beside its
acceptance rule, and the value-substitution path is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 00:22:16 -07:00
Kaoru0822andClaude Opus 5.5 acc2940ed8 docs(native): describe portable PE and ELF export inventories
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 07:22:06 +00:00
Kaoru0822andClaude Opus 5.5 0c7e8a6e17 feat(native): inspect native export tables through CLI and MCP
Expose the reader as inspect_native_exports / inspect-native-exports:
one stable, digest-bound read of an explicit file with a caller-selected
input budget, typed failures for malformed, unsupported, oversized and
cancelled reads, and observed Evidence bound to the file path and
SHA-256. Mach-O inputs point to inspect_macho.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 07:22:06 +00:00
Kaoru0822andClaude Opus 5.5 c02945bc44 feat(native): decode Swift nominal type declarations from __swift5_types (#6257)
* feat(native): decode Swift nominal type declarations from __swift5_types

Fill swift_decls from the nominal type context descriptors that survive
stripping: kind (class, struct, enum, actor), own and qualified name,
module, generic flag, anonymous-context flag, metadata accessor and
descriptor location, with evidence citing the type record. Unresolved
parent contexts keep qualified_name null and the entry partial.

One shared reader walks each context once; class vtable recovery now
consumes its descriptors instead of re-reading the records, so a type
record that cannot be read keeps both facets partial. Remove the
never-populated swift_decls fields that descriptors do not carry.

Fixes #5341

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(native): bound Swift context walks and decode vtables in record order

Address review of the nominal-type reader:

- Cache intrinsic context depth (distance to the module) so a cached
  parent cannot bypass the 32-context bound, and never cache a stop that
  depends on where the walk began. Bound qualified names to 4096
  characters so nesting cannot multiply name storage.
- Visit each descriptor as soon as its kind and name are read, so a
  class vtable spends the record budget in record order instead of after
  every later declaration.
- Report an accessor outside executable mapped code as null and keep the
  declaration partial.
- Forward a name failure to vtable coverage only when the record is a
  class or its kind is unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 00:19:15 -07:00
Matt Van Horn 2edf3b23c1 feat(elf): inspect PT_NOTE and SHT_NOTE tables (#4837)
Inspect selected ELF32/ELF64 PT_NOTE and SHT_NOTE declarations without executing
the artifact, retaining independent table identities, source ranges, descriptor
digests, GNU build-id/ABI fields and explicit coverage limitations.

Integrate mvanhorn's implementation and hardening with current main while
preserving byte-entropy registration and the Windows curated regression suite.
The maintainer follow-up uses ordinary merges/pushes, not history rewriting.

Executed validation: 96 focused/shared/docs tests, complete static/docs checks,
eleven real CLI/MCP pairs including GCC ELF32 and Clang big-endian PowerPC64,
and two actual Codex task traces checked against their Evidence. Full costs,
grader correction and verification limits are recorded in PR #4837.

Refs #6262.
2026-10-22 15:02:23 +08:00
Kaoru0822andClaude Opus 5.5 2dee400f93 feat(native): read PE and ELF export tables
Add a host-independent export-table reader on the shared native
container layer. PE images report each used export-address-table slot
with its ordinal, one row per name, its address or forwarder text, and
the export directory identity; the PE image reader now exposes its data
directories. ELF images report defined default/protected global, weak
and unique .dynsym symbols with type, binding and .gnu.version_d
version (hidden versions kept distinct, unreadable version tables
"unknown"). Unreadable entries are counted in issues and make coverage
partial; an ELF without section headers is unsupported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 06:58:12 +00:00
Kaoru0822andClaude Opus 5.5 c8ed328538 fix(call-path): count only expanded edges as explored (#6256)
build_call_path summed the out-degree of every reached function, so a
terminal goal's callees counted as explored even though the forward
search never expands the goal. Count explored edges and depth at the
expansion boundary inside the search, drop the post-hoc summary and the
now-unused out-degree helper, and reuse the schema-owned explored type.

Fixes #5906

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 23:32:35 -07:00
Kaoru0822andClaude Opus 5.5 ef5ab1f1f2 fix(apple): decode NIBArchive booleans with AppKit's polarity (#6255)
The NIBArchive value decoder read type tag 4 as true and 5 as false, the
reverse of what ibtool writes and AppKit's decoder returns, so every
compiled-nib boolean in decode-interface-builder output was inverted. The
test fixture encoder shared the reversed mapping, hiding it in round trips.

Name the value-type tags once and use them in both the read and skip
paths, fix the fixture encoder, pin raw tags in a unit test, and make
verify:interface-builder compare both polarities with ibtool --export.

Fixes #5541

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 23:32:21 -07:00
Kaoru0822andClaude Opus 5.5 14877a069b fix(nativeaot): preserve COPY bytes when hydration output grows (#6254)
The DEHYDRATED_DATA COPY writer evaluated the output buffer before the
reservation that could replace it, so a COPY that grew the hydration
buffer wrote its payload into the superseded buffer. The overlay kept the
advanced length and hashed uninitialized bytes while the report claimed
complete recovery.

Reservation now returns a view of the retained buffer, and every writer
(COPY, zero fill, relative and absolute fixups) writes only through that
view, so no caller can hold a stale buffer across growth.

Fixes #5882

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 23:32:07 -07:00
7aa1b51282 feat(diagnostics): report a byte entropy profile for one local file (#2331)
* feat(diagnostics): report a byte entropy profile for one local file

The diff adds the MCP tool inspect_byte_entropy and the CLI command
inspect-byte-entropy, both implemented by ByteEntropyService. One
absolute regular file is read offline into observed Evidence: file-wide
SHA-256 and Shannon entropy, then fixed blocks and adjacent runs for an
optional window, using a power-of-two block size from 256 to 65536
(default 4096), with high at 7.2 or above and low at 1.0 or below. A
window past EOF, a missing path, or a non-file is an input error; a size
or identity change during the read is an artifact-changed error; empty
input reports entropy 0; cancellation is honored before and during the
read. Summary detail retains the complete series and returns file
entropy, counts, and the ten longest runs for inspect_analysis_view,
while the registered effect remains read-only Evidence and the
diagnostics docs record the operation and its errors.

* fix(diagnostics): flag zero-size files as not shown empty

Generated files such as Linux /proc entries report size 0 while holding
content, so inspect_byte_entropy read nothing and returned an empty-looking
report. A file that reports size 0 now carries a limitation that the result
does not show the file is empty. The result stays a success because the
tool reads the declared size, which the file-stability checks depend on.

* fix(mcp): reject unsupported diagnostic summaries before inspection

* fix(ci): repair integrated checks and Hopper test baseline

* test(entropy): use shared MCP schema verification after merge

* fix(catalog): list inspect_byte_entropy under its own provider

Adding inspect_byte_entropy to BINARY_DIAGNOSTICS_TOOL_CONTRACTS put it
under pwntools-elf in the generated product catalog, because the
pwntools group took every binary diagnostics tool except
inspect_pe_resources. The service reports provider byte-entropy.

Add a BYTE_ENTROPY_PROVIDER identity beside PE_RESOURCES_PROVIDER, use
it in ByteEntropyService, give it its own catalog group, and name the
pwntools tools (inspect_binary_layout, inspect_recorded_crash)
explicitly so a new diagnostics tool is not attributed to pwntools by
default. The catalog test pins both groups.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KFTE4BHbGYzgf8Uhzcppdh

* fix(cli): resolve the byte-entropy path with absoluteCallerPath

main replaced the node:path resolve import in binaryDiagnosticsCommands.ts
with absoluteCallerPath (#4067), so the byte-entropy command's
resolve(args.path) no longer type-checks once the PR is merged with main.
Use absoluteCallerPath like the other commands, and add
src/filesystem/callerPath.ts matching main so the branch builds on its own.

* fix(diagnostics): retain entropy read cleanup ownership

* docs(diagnostics): explain entropy output capacity

* fix(diagnostics): preserve entropy cancellation outcomes

* fix(diagnostics): close the entropy input and warn on large reports

Public inspect_byte_entropy input now uses publicInputStrictObject before
detail is added, so unknown keys stay rejected on the advertised schema.
The tool description warns that complete output can become very large and
points callers to a shorter length or a larger block_size.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(diagnostics): keep byte-entropy registration mergeable with main

Register inspect_byte_entropy beside layout inspection and reject
unretained summaries before the file read, without editing workflows.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(diagnostics): register byte entropy with the shared tool options

Use toolRegistrationOptions so the handler typechecks on this branch and
on main, and still refuse unretained summaries before the file read.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-22 14:19:12 +08:00
538017d1d2 feat(environment): apply evidence-gated write effects to the served state (#5600)
* feat(environment): render more read tools from the state and describe responses

Keep main's environment serving and fold in coverage for non-entity reads,
response summaries, and MCP tool annotations.

Squashed rebase of #5352 onto main.

* fix(environment): keep rendered documents and quoted samples inside their budgets

A recorded body of exponential numbers can render past 256 KiB and turn a
working read into a state size error, so that document stays a recorded replay.
Description samples are measured after JSON quoting, and the replay-fallback
catalog check uses a tool count the longer response descriptions still separate.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* feat(environment): apply evidence-gated write effects to the served state

Serve write tools whose later reads confirm the effect, and keep main's
state-file ownership and part-wise reply measurement.

Squashed rebase of #5600 onto the rebased #5352 tip.

* fix(environment): bound and validate write state history

* feat(environment): render more read tools from the state and describe responses

Keep main's environment serving and fold in coverage for non-entity reads,
response summaries, and MCP tool annotations.

Squashed rebase of #5352 onto main.

* fix(environment): keep rendered documents and quoted samples inside their budgets

A recorded body of exponential numbers can render past 256 KiB and turn a
working read into a state size error, so that document stays a recorded replay.
Description samples are measured after JSON quoting, and the replay-fallback
catalog check uses a tool count the longer response descriptions still separate.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* feat(environment): apply evidence-gated write effects to the served state

Serve write tools whose later reads confirm the effect, and keep main's
state-file ownership and part-wise reply measurement.

Squashed rebase of #5600 onto the rebased #5352 tip.

* fix(environment): bound and validate write state history

* fix(environment): bound additional collection history

* fix(environment): credit sent fields, negative sequences, and encoded targets

A later overwrite copied surviving derived fields into the sent-member
claim, so a members-only check rejected an update whose sent field still
matched. Serving floored sequences at zero and stored missing history as
zero, so a collection sequence learned from negative ranks skipped the
next value. An absence after a delete forgot the resolved collection, and
an encoded read could not observe an update spelled with another
collection encoding.

Regression tests cover each case.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(environment): cover resolved encoded update identities

* fix: validate timed update observations

* fix(environment): share write overlap and check clocks on earlier calls

Observation search skips a read begun before the write ended. That
comparison now uses the same overlap as the check path.

A clock learned on a later call was not checked against earlier calls
that kept the field. Serving would stamp those calls too, so the plan
stays unserved. A clock learned earlier is still refused when a later
call does not advance it.

Regression tests cover an overlapping empty update and a published_at
clock that one recorded call leaves unchanged.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix: preserve partial update evidence

* fix(environment): let an unmodelled write explain only the list members that differ

A failed whole-list read now reports which members alone account for the
difference. An unmodelled write excuses the read's writes only when it came
after each such member's last check and pending writes, so an older change
elsewhere no longer hides a newer write the list shows undone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 7b6aab9d7191e600051693f2c192afcfc743d083)

* fix(environment): explain added fields, tie creates to serving's collection, and bound write descriptions

An update explains a field the entity lacked before when the observing
view had shown the entity, so a server-filled field is not dropped. A
create is tied to the held collection serving picks for its path. Write
descriptions and body refusals quote member names and sample them within
a byte bound. The request body is indexed once per explanation, so copied
and slugged fields no longer rescan it.

Replays 07aca54e onto the restacked branch. Its rule that a read begun
before a write ended teaches nothing is already here (fb6785dc), so only
its test is kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): state how list reads weigh unmodelled writes, added update fields and quoted members

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(environment): keep the observing view helpers with the observations

shownBefore and the item-or-list fields check move to the observe module,
keeping the write effects within the line limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): resolve unmodelled writes onto the held collection

An unmodelled path kept the request's collection spelling. An archive under
the plain spelling of a collection held encoded marked only that plain
collection unknown, so a later update was blamed for the archive's fields
and dropped. Name that entity the way a modelled write does.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(environment): confirm no write through volatile fields or overwritten replacements, and bound the served clock

A members-only check no longer compares volatile fields, which render
from the read itself. A document replacement credits the members it
wrote or removed at its object, so a later merge that overwrites them
takes its confirmation away. A recorded time past the range of a date
gives no served clock instead of throwing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): leave unknown entities unchecked by failed reads, volatile fields and unchecked calls

A failed item read that is no absence leaves an unknown entity unknown, so
a later usable read still checks its writes. A read showing an unknown
entity again checks no write on a volatile field. An update of an unknown
entity takes its recorded outcome and adds nothing to a built plan. A later
update that does not answer with its entity is observed only by a read
whose view shows every learned clock; without one it is unmodelled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): fill a create's path fields from the collection serving selects

The gate built a later create's path fields from the recorded request path,
while serving fills them from the held collection it selects for that path,
so a read showing the other spelling's parent could confirm a create serving
cannot reproduce. Rules still learn from the recorded path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): name unmodelled writes as modelled ones, count fields updates write, and keep later members' types

An unmodelled write below an item path names its entity through itemTarget,
and one at or above a collection resolves the held collection, as modelled
writes do, so an encoded spelling no longer hides the entity it changed.
An update's answer counts the fields its rules write, other than increments,
as available, so a field only the update writes no longer makes it
incomplete-entities. A member a later call first stores keeps that call's
JSON type in the body's kinds, which the body check falls back to.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): let a document replacement overwrite everything under its object

A replacement claims only the members it wrote or removed, but it now
overwrites every earlier effect under its object, so a merge that wrote
the object it replaces loses its confirmation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): start the served clock from the records whose times a date can hold

A record timed past the range of a date no longer leaves the served clock
without a start, which the store read as zero and stamped writes in 1970;
the clock starts after the latest record a date can hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 14:15:35 +08:00
morluto b8c1b45316 fix(views): centralize retained-result validation errors 2026-10-22 14:14:35 +08:00
morluto 407a718050 refactor(browser): share scenario transaction identity indexes 2026-10-22 14:04:28 +08:00
morluto 51be954c4b refactor(domain): reuse canonical ordering and digest owners 2026-10-22 14:04:28 +08:00
Kaoru0822andClaude Opus 5.5 c537c52c96 fix(environment): report capture payload failures at their field paths (#6247)
Authenticated capture Evidence whose operation is supported but whose
normalized_result fails its capture schema fell through to a generic
evidence.operation refusal, discarding the schema issues. Each recognized
operation now refuses with the projected schema issues beneath
evidence.normalized_result and retains the schema failure as the cause;
only an operation no capture schema owns blames evidence.operation.

The view Evidence parser had the same conflation: a wrong operation, a
wrong kind and a malformed payload of the requested kind all blamed
normalized_result.kind. It now reports role.operation, the kind, or the
payload's own field paths respectively.

Fixes #6119

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 22:22:45 -07:00
Kaoru0822andClaude Opus 5.5 6f96d7a3fa fix(capture): report scenario path search denial as access denied (#6244)
Electron and process scenario path admission caught every realpath or
stat failure as an invalid_value input issue, so a correct path beneath a
directory without search permission returned invalid_request with
schema-correction recovery instead of a host access denial.

Admission now classifies EACCES/EPERM from path resolution and metadata
reads as the existing AnalysisAccessDeniedError (selected path,
system_code, filesystem-read boundary). Missing, looping or otherwise
unresolvable paths and reachable files without execute permission keep
their input-error semantics. Electron reports invalid selections before a
denial on another field. Process PATH search still continues past a
denied directory; only an explicitly selected executable reports denial.

Fixes #6117

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 22:22:15 -07:00
郭立 Guo Li 7590b50801 perf(schema): reduce retained integer validator allocations (#4360)
Construct integer formats directly while preserving bounds, refinements, metadata and current shared schema owners. Keep Incur-dependent CLI numeric coercion declarations unchanged.

Verify canonical advertisements, 126 numeric boundary cases, 974 actual CLI parser cases, broad source tests and real public-artifact SDK workflows. Record 72 normal tasks and 12 separate owned-inspector heap diagnostics with whole-family resource bounds; distinguish SDK spans and asymmetric audit overhead. Generate numerical report sections from receipts and reject stale tables through documentation conformance checks.

Closes #6236. Current-head CI and Greptile pass; actionable review findings are addressed. Codex review was unavailable due repository usage limits.
2026-10-22 05:11:08 +00:00
Kaoru0822andClaude Opus 5.5 bd5d189a8f docs(javascript): state that duplicate unmapped content stays ambiguous (#6238)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:20:02 -07:00
Kaoru0822andClaude Opus 5.5 89268f4a45 fix(ci): run the lockfile-pinned Skills CLI in package verification (#6235)
* fix(ci): run the lockfile-pinned Skills CLI in package verification

The repository skill stage fetched skills@1.7.2 and its unlocked
dependencies from the npm registry on every run, taking 5-120 s and
intermittently exceeding the 120 s stage timeout on main and PRs.
Pin skills as a devDependency and run its locked bin directly.

Fixes #6234

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ci): lock the Skills CLI outside the root dependency tree

skills 1.6+ requires Node >=22.20.0, so pinning it as a root
devDependency made `npm ci` fail under engine-strict on the supported
Node 22.19.x. Keep the root install unchanged and give the verifier its
own package.json and lockfile, installed with `npm ci --prefer-offline`
in the verification workspace. The lockfile still removes all registry
metadata resolution and pins every transitive dependency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:19:45 -07:00
Kaoru0822andClaude Opus 5.5 3637205622 feat(native): let callers raise the Apple dispatch metadata input budget (#6233)
* feat(native): let callers raise the Apple dispatch metadata input budget

inspect_native_dispatch_metadata refused any Mach-O over a fixed 64 MiB
before decoding, and its recovery said there was no override. Most large
macOS binaries exceed it.

max_input_bytes (CLI --max-input-bytes) now selects the budget, default
64 MiB and at most 1 GiB, through the shared input contract, the enhanced
workflow and the native provider. Results record input_admission with the
effective budget, its source and the retained bytes. A refusal names the
observed size, the budget and its source, and the budget to select or, past
1 GiB, a deep analysis provider.

Fixes #3917

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): describe --max-input-bytes in inspect-native-dispatch-metadata help

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(native): cover max_input_bytes forwarding through CLI and MCP

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:19:33 -07:00
Kaoru0822andClaude Opus 5.5 b4f9f0deac fix(pcap): report workspace allocation failures by their system cause (#6229)
* fix(pcap): report workspace allocation failures by their system cause

Pcap workspace allocation and snapshot writes mapped every filesystem
error, including ENOTDIR and EACCES from mkdtemp, to a file-size resource
constraint with null limits and worker file-size advice. Flutter's owned
snapshot storage had the same errno-blind mapping, while TShark and sigrok
carried duplicate copies of the correct split.

Move the private-workspace failure projection into one shared owner:
ENOSPC/EDQUOT/EFBIG stay file-size resource constraints with their system
code and path; descriptor exhaustion and every other failure become
provider workspace failures carrying stage, workspace path, failing path,
system code and cause-specific recovery.

Fixes #6123

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(flutter): locate owned storage failures by their workspace

mapOwnedIoFailure always passed a null workspace path, so a descriptor
write or read failure without its own errno path lost every location.
Thread the owned root through: allocation keeps null (no workspace yet),
snapshot failures use the allocated root, and package-reader system
errors propagate to the request owner, which maps them against the
snapshot root instead of mapping early without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:19:18 -07:00
Kaoru0822andClaude Opus 5.5 ec0e663cca fix(browser): keep target detail when attributing CDP failures (#6223)
Authorize CDP targets with the caller's operation instead of a fixed
inspect_web_page label, and reuse the authorization owner's derived
origin scope. When a browser observation failure is attributed to a
different operation, keep its producer detail, remediation, cleanup,
diagnostics, partial observation and original cause rather than
rebuilding it from the reason alone. Both the CDP browser and CDP
Electron providers use the same domain-owned relabel.

Fixes #6122

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:19:00 -07:00
Kaoru0822andClaude Opus 5.5 26356dc34a fix(process): record withheld unrelated Linux processes as unverified (#6220)
* fix(process): record withheld unrelated Linux processes as unverified

The token sweep after an owned command (for example the Ghidra Java probe
run by setup and doctor) failed cleanup when an unrelated same-UID process
started during the sweep and Linux denied its /proc/<pid>/environ. Setgid or
setuid programs (mismatched real/saved IDs) and non-dumpable processes are
withheld by the kernel exactly like another account's processes, which were
already recorded as unverified.

Derive the kernel restriction from the same stat-bracketed procfs read that
already supplies the effective UID: all Uid/Gid IDs from status, and the
root ownership procfs reports for a non-dumpable process's environ. Keep that
fact through exit settlement. Related candidates still fail closed.

Fixes #6204

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(process): define caller credentials on hosts without geteuid

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:18:47 -07:00
Kaoru0822andClaude Opus 5.5 b928963675 feat(native): inspect a universal Mach-O slice in Go and Rust metadata readers (#6219)
* feat(native): inspect a universal Mach-O slice in Go and Rust metadata readers

inspect_go_binary and inspect_rust_binary refused universal Mach-O files and
offered no slice selection, so hosts without lipo could not inspect them.

Both now accept architecture (x86, x86_64, arm, arm64). One selector reads
REA's own FAT table, validates the slice range, alignment and header CPU,
and returns the thin image with its macho-selected-image-v1 identity. The
universal file stays the artifact; results add selected_slice and keep
every source range a file offset. Missing, absent, duplicate or misplaced
selections fail on architecture and list the slices present. Subject
format mach-o-universal now has one owner shared by the services and the
retained Go/Rust views.

Fixes #4102

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(native): reject damaged universal files and self-contradicting slice reports

A FAT table whose slice could not be read was treated as a non-universal
file, so callers were told to omit architecture. Its reason now fails on
path. Go and Rust reports with a selected slice must be Mach-O and name the
slice's CPU family, through one shared agreement check. The CLI
architecture flag is described in help.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:18:35 -07:00
Kaoru0822andClaude Opus 5.5 8451b961ad feat(artifacts): make ASAR header admission configurable (#6209)
* feat(artifacts): make ASAR header admission configurable

ASAR header/index admission was a fixed 16 MiB with no override, so
operators with large Electron indexes could only extract the archive.

REA_ASAR_MAX_HEADER_BYTES now sets it (default 16777216, maximum
268435456). One owner resolves the effective admission for the
classification probe, inventory, ASAR reader and Electron file scope,
including the Electron header cache budget. Startup and doctor validate
the setting; admission refuses an invalid value instead of falling back.
Recovery and error details name the effective value and its source.

Fixes #6176

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(artifacts): resolve ASAR header admission from the caller's environment

The admission read process.env, so composition roots that select their own
environment (createDirectAnalysis, createServer, CLI, optional observation
factories) could not raise or lower it, and an ambient value leaked into
callers that left it unset.

Nothing reads process.env now. ArtifactProvider and extraction resolve the
admission from their environment; JavaScript analysis parses it from the
environment its server or CLI caller passes and refuses an invalid value;
the Electron and V8 providers receive it from their factories. Inventory,
nested ASAR readers, classification probes and Electron page authorization
carry that one value. A header cached by Electron under a larger admission
is refused for a smaller one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 21:18:22 -07:00
Kaoru0822andClaude Opus 5.5 4a4bbab3c3 fix(cli): keep committed evidence bundle receipts after cancellation (#6221)
Summary delivery with --evidence_output wrote the bundle, then let the
command's close/cancellation check replace the result. A SIGINT/SIGTERM
during staging cleanup after commit returned typed cancellation without
the receipt, and a persistent staging cleanup failure after commit was
rewrapped as a provider cleanup error that dropped published_path.

Every --evidence_output command now selects its view first and publishes
through publishEvidenceDelivery, the owner inspect-managed-members already
used. Post-commit cancellation keeps {path, bytes} under
details.diagnostics.evidence_bundle; incomplete cleanup keeps
details.published_path and bytes; a cleanup retry that succeeds returns
the normal receipt; pre-commit cancellation creates nothing and names
the command's operation.

Fixes #6095

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:32:26 -07:00
Kaoru0822andClaude Opus 5.5 e5d2805b42 fix(javascript): keep unmapped captured runtime scripts unknown (#6214)
A captured script whose bytes matched no static digest was reported as
high-confidence unmatched even when no automatic or caller-declared
mapping applied to its location in any static layer. The matcher now
computes mapping presence once while locating candidates and shares one
no-candidate classification between the captured-source and source-free
branches, so an unmapped location stays unknown with
no-authorized-location-mapping and is not counted as unmatched.

Fixes #6061.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:32:10 -07:00
morluto 6f28b3633b Merge remote-tracking branch 'origin/main' into codex/legacy-cleanup-followup-20261022 2026-10-22 11:11:42 +08:00
morluto c0eb612780 refactor(web): share retained text artifact metadata 2026-10-22 11:11:42 +08:00
morluto 27dc9735e4 refactor(managed): reuse the member comparison input contract 2026-10-22 11:11:42 +08:00
morluto 51e0df97e9 refactor(javascript): own application node merging in the graph domain 2026-10-22 11:11:42 +08:00
Kaoru0822andClaude Opus 5.5 292fd532c9 fix(ghidra): retry retained language catalog closes before admission (#6210)
The language catalog kept failed .ldefs closes in a module-global set that
only a later catalog read retried, and the provider returned a cleanup error
bound to no owner. Registry admission retries provider close before profile
resolution, so the retained handle was never retried and the original error
stayed incomplete forever.

Read definitions through the provider's caller-owned inspection scope with the
shared regular-file owner used by the seed and extension readers. A failed
close is retained as an exact lease in that scope, provider close and the next
catalog admission retry it, and the translated error is bound to the lease so
confirmed release reopens the same registry.

Fixes #6133

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:10:25 -07:00
Kaoru0822andClaude Opus 5.5 aa7d960ef5 fix(process): settle owned cleanup with the launcher's own exit proof (#6205)
* fix(process): settle owned cleanup with the launcher's own exit proof

When owned process-group verification fails while the supervised
launcher is still mid-exit, the supervisor now waits up to the kill grace
for its own child-exit observation and verifies once more after it
arrives. Verification is not repeated when the exit was already observed
before cleanup started, so there is no new evidence and the original
failure is kept. Nothing is signaled based on the wait.

Fixes #6134

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(process): bound Hopper session cleanup attempts by exit evidence

The fake Hopper launcher exits after its shutdown acknowledgement. When
the supervisor first observes that exit during a failed verification, it
verifies once more, so the first close makes one or two attempts depending
on timing. The test now asserts that bound. Later session retries, made
after the exit was already observed, make exactly one attempt each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 20:10:07 -07:00
郭立 Guo Li 0b7213c890 perf(mcp): defer analysis runtime after catalog discovery (#4424)
Serve build-recorded default compact and explicit full catalogs, then materialize canonical analysis handlers on first use. Keep transport, Evidence delivery, profile identity and caller choices consistent. Share sealed MCP server ownership between eager and deferred lifecycle paths.

Validate 125 focused cases, schema/contract repeats, negative controls, builds/static/docs checks and latest-main integration. Record 72 real SDK task trials with independent Rust metadata oracles and whole-family resource bounds; disclose shifted first-RPC cost and unchanged catalog tokens. Qualify the real Rizin small-target debug lifecycle separately. Automated review findings addressed; Codex review unavailable due repository usage limits.
2026-10-22 03:07:31 +00:00
morluto 63fe0eb5db refactor(electron): reuse retention and capture contracts 2026-10-22 10:57:27 +08:00
morluto ddce4bb446 refactor(javascript): centralize source coordinate operations 2026-10-22 10:57:27 +08:00
morluto dd3497e133 refactor(javascript): reuse argument binding lookup 2026-10-22 10:46:25 +08:00
morluto 4f54379831 refactor(json): reuse shared parser at CLI and shape boundaries 2026-10-22 10:46:25 +08:00
morluto d77bc0e8a0 refactor(cli): share analysis snapshot option contract 2026-10-22 10:33:38 +08:00
morluto 527ade245a refactor(views): derive layout summary fields from source contract 2026-10-22 10:33:29 +08:00
morluto dbcec4cf56 refactor(javascript): share result binding projection and facts 2026-10-22 10:33:18 +08:00
morluto 34fd7bdf7d refactor(reference): share resource failure projection 2026-10-22 10:19:28 +08:00
morluto 81ab0ddc95 refactor(managed): derive verification declaration facts from their owner 2026-10-22 10:19:16 +08:00
morluto 306ca984bc refactor(electron): share observed process and window schemas 2026-10-22 10:19:07 +08:00
morluto dd46a13dec refactor(config): share HTTP listener port contract 2026-10-22 10:18:57 +08:00
morluto ce5bc8cb01 refactor(domain): consolidate projection and verification digests 2026-10-22 10:11:02 +08:00
morluto b88bdfccd2 refactor(evidence): share profile and completion digest owner 2026-10-22 10:10:58 +08:00
morluto 34aa22d0aa refactor(javascript): use shared canonical identity primitives 2026-10-22 10:10:55 +08:00
morluto 719d56ebfe fix(evidence): preserve provider execution provenance across workflows 2026-10-22 10:02:31 +08:00
morluto 87c8086fc1 refactor(managed): reuse canonical JSON digest owner 2026-10-22 09:52:58 +08:00
morluto eab4e47916 refactor(ghidra): share session startup observation checks 2026-10-22 09:52:49 +08:00
morluto db87917daa refactor(native): use canonical deadline defaults 2026-10-22 09:52:40 +08:00
morluto 183b4d4e7e refactor(managed): share metadata token schema 2026-10-22 09:33:58 +08:00
morluto 00819258a7 refactor(browser): separate bundle state from AST inspection 2026-10-22 09:33:58 +08:00
morluto c465ff5a81 refactor(evidence): share canonical ASCII ordering rules 2026-10-22 09:21:54 +08:00
morluto 1ac7ce3f3b refactor(native): share call observation outcome schema 2026-10-22 09:21:54 +08:00
morluto 55f5c8db69 refactor(kuna): centralize installation cancellation projection 2026-10-22 09:21:54 +08:00
morluto 5b0373db11 refactor(browser): map artifact failures at one boundary 2026-10-22 09:21:54 +08:00
morluto 73be86c39e fix(android): decode guardian output as UTF-8 streams 2026-10-22 09:11:44 +08:00
morluto 8dbe92a4a3 refactor(reference): centralize source inventory rules 2026-10-22 09:11:44 +08:00
morluto 5a28a147f1 fix(hopper): preserve streamed Unicode lease owner identities 2026-10-22 09:11:44 +08:00
morluto 13a8640204 fix(browser): observe fetch settlement after synchronous abort 2026-10-22 09:03:51 +08:00
morluto 09b29436f4 refactor(snapshot): share persisted execution projection 2026-10-22 09:03:51 +08:00
morluto 421d527bfe Merge remote-tracking branch 'origin/main' into codex/legacy-cleanup-followup-20261022 2026-10-22 08:52:56 +08:00
morluto 24bd2b989a refactor(browser): derive comparability once per capture pair 2026-10-22 08:52:56 +08:00
morluto b2fad362f2 fix(browser): stop expired work and release deadline resources 2026-10-22 08:52:56 +08:00
morluto 247c8f4eab refactor(javascript): own property capture translation 2026-10-22 08:52:56 +08:00
morluto adfde012ec refactor(readiness): share canonical stage validation 2026-10-22 08:52:56 +08:00
Jay-Alexander Elliot 57fd3ea5d8 feat(cli): summarize managed members with durable Evidence paging (#5167)
Fixes #5148.

Add opt-in durable managed member summaries and authenticated paging while preserving complete default output, original Evidence, cleanup ownership and queued cancellation.

Verified with bounded real Newtonsoft/Redux/Rust tasks, independent oracles, 556 focused tests and the expanded domain gate; paired token estimates and latency tradeoffs are recorded in experiments/analysis-efficiency/managed-cli.md.
2026-10-22 00:43:30 +00:00
morluto d8948efb81 refactor(mcp): bind Electron provider failures at registration 2026-10-22 08:40:36 +08:00
morluto 3966c92c71 refactor(ghidra): derive reference facts from shared schema 2026-10-22 08:40:36 +08:00
morluto 1fbfbf74ef refactor(gdb): own command record snapshots and stream groups 2026-10-22 08:40:36 +08:00
morluto 2604059961 refactor(process): share decoded NDJSON framing 2026-10-22 08:35:20 +08:00
morluto 6b44e35bec refactor(managed): own successful PE inspection identity 2026-10-22 08:35:20 +08:00
morluto 20bf7d957f refactor(setup): reuse desired registration per document snapshot 2026-10-22 08:35:20 +08:00
morluto c9f95eaf90 refactor(javascript): own graph Evidence envelope validation 2026-10-22 08:27:39 +08:00
morluto a2805bc934 refactor(managed): share artifact and token identity schemas 2026-10-22 08:27:39 +08:00
morluto b7529304f0 refactor(browser): centralize shared CDP observation facts 2026-10-22 08:27:39 +08:00
morluto 22c4b528d2 Merge remote-tracking branch 'origin/main' into codex/legacy-cleanup-followup-20261022 2026-10-22 08:20:30 +08:00
morluto 1158dc2a94 refactor(process): own worker frame wait lifecycle 2026-10-22 08:20:30 +08:00
morluto 2765fc12bb refactor(browser): share composed artifact cleanup ownership 2026-10-22 08:20:30 +08:00
gensokyo-dev 18669641e9 test(browser): select a bounded service-worker setup deadline (#6186) 2026-10-22 08:16:34 +08:00
morluto cdd718ffb0 refactor(inputs): own absolute path validation in shared contracts 2026-10-22 08:13:46 +08:00
morluto c5c5ff45aa refactor: import shared limits and projection helpers from their owners 2026-10-22 08:13:46 +08:00
morluto 5da3b7c393 refactor(asar): centralize read failure and cancellation semantics 2026-10-22 08:13:46 +08:00
morluto 2451e187be test(prompts): isolate capture admission regression 2026-10-22 07:51:42 +08:00
morluto 5e6529521c Merge remote-tracking branch 'origin/main' into codex/legacy-cleanup-followup-20261022 2026-10-22 07:49:01 +08:00
morluto 7cec547ecc fix(prompts): suggest only admitted process capture evidence 2026-10-22 07:49:00 +08:00
morluto c1a0561bf3 refactor(mobile): retain inventory sources at the projection boundary 2026-10-22 07:48:39 +08:00
morluto fafc1881b5 refactor(managed): return singular admitted evidence records 2026-10-22 07:48:39 +08:00
morluto a59abe932e refactor(evidence): centralize comparison identities and source recording 2026-10-22 07:45:47 +08:00
morluto 452e3ccad1 refactor(javascript): own source range validation once 2026-10-22 07:45:47 +08:00
8413ec08f7 fix(process): report the effective regex heap and uncertain signal retry (#6099)
* fix(process): report the effective regex worker heap

Node gives --max-old-space-size, including NODE_OPTIONS, precedence over
the worker's 128 MiB resourceLimits request. Name that effective
old-generation heap and stop saying the request has no runtime override.
The matching deadline stays a fixed cap.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): keep SIGXCPU and SIGXFSZ retries uncertain

The signal does not prove the retained rlimit was exhausted, so an
unchanged retry may succeed. Project that retry as possible instead of
saying it fails the same way.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave percentage heap overrides unestablished

--max-old-space-size-percentage takes precedence over the absolute flag
in either order. Node computes that MiB from startup memory, which a
later read does not recover, so the effective heap stays unknown instead
of the 128 MiB request or an absolute flag.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): read launch NODE_OPTIONS, not process.env

A preload can rewrite process.env.NODE_OPTIONS after V8 has applied the
original flags, and those flags are not copied into execArgv. Use the
initial environment block when it can be read, and leave the effective
heap unestablished when that provenance cannot be recovered.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave env-file heap overrides unestablished

--env-file and --env-file-if-exists can supply NODE_OPTIONS when that
variable was absent from the initial environment. The file is not in
/proc/self/environ, so a missing entry is not the 128 MiB request.
An initial NODE_OPTIONS is still parsed, because Node does not let the
file replace it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave --max-heap-size old generation unestablished

A positive --max-heap-size or --max_heap_size sets the total heap, not the
old generation, and can stay below the 128 MiB worker request when an
old-space flag is also present. The effective old generation stays
unestablished while that value is in effect. Zero or empty clears it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave config-file heap options unestablished

--experimental-config-file and --experimental-default-config-file can
apply a heap percentage from a file that is not launch provenance. The
effective old generation stays unestablished when either flag is present,
including beside an absolute --max-old-space-size.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): read launch exec argv from the command line

process.execArgv can be cleared after V8 applies --max-old-space-size.
Linux recovers those tokens from /proc/self/cmdline and leaves the heap
unestablished when that vector cannot be separated from the program entry.
Heap probes compare pinned absolute limits instead of a percentage of host memory.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(process): leave a title-overwritten command line unestablished

process.title and --title replace /proc/self/cmdline with the title.
A buffer without a program entry is not an empty option list, so the
heap stays unknown instead of the 128 MiB request. The command-line
probe does not read /proc off Linux.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-22 07:40:14 +08:00
morluto 603a7d022c refactor(process): share capture evidence admission 2026-10-22 07:37:21 +08:00
morluto b2dc45006d refactor(cli): share registered command traversal 2026-10-22 07:37:21 +08:00
morluto 1828cf87a0 refactor: remove redundant contract and syntax definitions 2026-10-22 07:37:21 +08:00
morluto a73eac190b refactor: remove obsolete forwarding APIs and unify CI contracts (#6181)
* refactor: remove obsolete internal forwarding APIs

* refactor(ci): centralize failure classification contracts
2026-10-22 07:30:54 +08:00
0e5c511c01 fix(electron): follow absolute symlinks that stay inside the framework (#3587)
* fix(electron): follow absolute symlinks that stay inside the framework

An absolute symlink target is now passed to relativeInside(root,
target), and a null result still ends the walk as an escape. When the
target lies inside the root, resolved is cleared and the root-relative
pieces are queued on pending, which replaces the path walked so far with
that absolute target. Later hops stay under the hop limit and the
existing lstat containment checks. Provider tests now cover an absolute
Versions/Current that stays on in-framework version B and matches the
relative bundle's resolved path and SHA-256, an absolute Current outside
the framework, and an absolute in-framework link whose next hop leaves
the framework; those last two still surface an escape error.

Fixes #3380

* test(electron): cover absolute framework symlink cycles

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-22 07:21:37 +08:00
2e6b38cef0 feat(environment): render more read tools from the state and describe responses (#5352)
* feat(environment): render more read tools from the state and describe responses

Keep main's environment serving and fold in coverage for non-entity reads,
response summaries, and MCP tool annotations.

Squashed rebase of #5352 onto main.

* fix(environment): keep rendered documents and quoted samples inside their budgets

A recorded body of exponential numbers can render past 256 KiB and turn a
working read into a state size error, so that document stays a recorded replay.
Description samples are measured after JSON quoting, and the replay-fallback
catalog check uses a tool count the longer response descriptions still separate.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(environment): pin the replay card this branch serves

The replay-only card test pinned main's 11,131-byte card. This branch's descriptions name each tool's recorded responses, so the card it serves is 12,818 bytes; the pin and its name follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 07:08:44 +08:00
d7ca93cc2c fix(artifacts): name the ASAR header admission budget (#6172)
Keep the 16 MiB ASAR header/index ceiling fixed and report that budget,
the lack of a runtime override, and that source selection does not bypass
header admission.

Refs #5910

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-22 06:33:09 +08:00
Afloat 82e4e032df fix(artifacts): complete positional ZIP reads (#4414)
fix(artifacts): complete ZIP positional reads
2026-10-22 06:13:24 +08:00
Abdul Samad Siddiquiandmorluto ca2ca16146 fix(dotnet): reject unfinished CIL prefix chains (#4416)
* fix(dotnet): reject unfinished CIL prefix chains

* test(dotnet): cover unfinished prefixes in fat method bodies

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
2026-10-22 06:13:10 +08:00
xjh 14c4c2a1d7 fix(managed): preserve observed method header differences (#3814) 2026-10-22 06:12:55 +08:00
Hemang VarshneyandHemang Varshney 4842f23765 fix(source): retain CommonJS requires spelled as exact no-substitution templates (#4511)
* fix(source): retain CommonJS requires spelled as exact no-substitution templates

Admit expression-free template arguments at the CommonJS require gate and
read their single cooked quasi through the shared exact-literal reader.
Scope is the require path only: static ESM template declarations keep their
parse failures and dynamic import() keeps the explicit unknown/partial
<dynamic-import> relationship with no invented target.

Refs: #4382

* fix(source): admit TypeScript-wrapped exact require arguments via the shared literal reader

Address review: the require gate matched the outer AST node, so
runtime-transparent TS wrappers (as const, !, satisfies) around exact
strings or expression-free templates were still silently omitted even
though readExactJavaScriptLiteral unwraps them. Remove the gate and let
the shared exact-literal reader own argument exactness; computed
arguments remain silent and dynamic import() behavior is unchanged.

Refs: #4382

---------

Co-authored-by: Hemang Varshney <218273197+hemv-857@users.noreply.github.com>
2026-10-22 06:12:42 +08:00
7721d59e1a fix(adb): normalize absent installer package sentinels (#4073)
* fix(adb): report a null installer declaration as an absent installer

Package Manager prints its String fields by concatenation, so a package
without a recorded installer shows `installerPackageName=null` in
`dumpsys package`. The projection treated only an empty value as absent
and forwarded the text "null" as a present installer identity, which a
consumer cannot tell apart from a package literally named "null" (#3943).
`versionName` is printed the same way and had the same gap.

The parser now maps the empty value and the `null` sentinel of both
String fields to the nullable result's null at the producing boundary;
a reported installer such as com.android.vending is kept unchanged.

Validation: parser unit tests for the sentinel, empty and missing
installer lines and the absent version name, plus a provider-level
regression through the inspect_adb_package operation; three of them fail
on main and pass here. The issue's four CLI controls through
`rea inspect-adb-package` against a stub adb return JSON null for the
sentinel, empty and missing cases and keep com.android.vending. Typecheck,
lint and module boundaries pass.

Closes #3943

* fix(adb): keep the null sentinel handling to the installer field

Review feedback on #4073: a manifest can declare the literal string
"null" as android:versionName, and dumpsys prints it exactly like an
absent value, so mapping versionName to null would turn a real version
into absence. Only installerPackageName, where no package can carry that
name, treats the printed null as the absent installer. The test for the
version name now pins the retained text.

* test(adb): establish permission inventory in null-installer fixture

Main now reports partial coverage when the requested-permission
inventory is not established (9534e42ec). Give the null-installer
fixture a requested permissions section so it still checks a
complete projection.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 06:11:49 +08:00
76dbc32f39 fix(environment): bound state reads and retain cleanup ownership (#6158)
* feat(environment): extract the initial state a capture's reads observed

Add the environment-state analysis view. From the parents the environment
manifest accepts, it extracts the data the capture's reads observed before
the first related write: entity types with an identity field (proved by a
path parameter or inferred from an identity name), entities with their
recorded values and provenance, documents for written non-entity paths, and
a render plan per read tool that is kept only when it reproduces the tool's
latest initial recordings from the state; every other read tool is a
recorded view with the reason. The Evidence is always inferred.

Response-body decoding moves to a domain owner shared with replay, exact
JSON parsing moves to the owner of the exact-number policy, the manifest and
state views share parent admission and record grouping, and
projectAnalysisView takes its accumulators as one object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): bound state at redacted writes; infer identity only from items

A scenario write whose URL was redacted has no request line, so it was
dropped and reads after it counted as initial state. Such a successful
write now joins the timeline with an empty path, whose root relates to
every type, so it ends every type's state; a limitation counts them.

A list whose recordings were all empty passed the "every listed item
carries it" check vacuously and published an identity field nothing
showed; it now needs at least one item, else the view stays recorded
as no-identity.

A value that differed between two request lines of one item tool, such
as a query variant, was called volatile, and the gate then substituted
each recording's own value; only repeats of one request line now show
volatility, so differing lines go through fields-differ.

List membership is compared only while a list can still be the whole
collection, and merged entities are indexed by type and collection once,
so paged captures no longer cost pages times entities (8,000 pages of
40,000 entities: 5.7 s to 0.23 s).

Docs and the tool contract say scenario state keeps the capture's
redaction markers and how redacted writes bound it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep first sightings, one list pointer and linear field unions

Entities took the order of the first plan observed, and item plans run
before list plans, so a complete list read before its detail reads came
out reordered and lost "all" membership; each entity now keeps its
earliest recorded sighting.

A list plan reused the first recording's pointer for later recordings,
so a later response that was itself an entity wrapping a list was read
as that list; every recording must now locate the same list.

The field union scanned its ordered list for every key, quadratic in an
object's width; a set now answers membership.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep post-write values and partial lists out of the initial state

A value that changed between repeats of one request line was named
volatile whenever no write under the type's own root lay between, so a
write under another root (for example POST /api/dashboards/db before a
second GET /api/search) left its effect in the initial state. Any
recorded write between the two observations now leaves every tool of the
type recorded with reason fields-differ instead.

A list was published with membership "all" even when a total or count
envelope member exceeded the number it listed. Such a counter now makes
the membership "recorded".

Also drop a stale doc comment on ObservedEntity.order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): scope item-shaped and deleted written paths out of documents

State documents are held only for list-shaped paths a POST, PUT or PATCH
tool writes. Reads of item-shaped written paths and of paths that are only
deleted stay recorded views; the guide now says so and the spec defers them
to a later increment of #4933.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): count writes in flight when a value changes between reads

The volatility check compared write start positions with the two reads,
so a write already in flight when the earlier read started was missed and
its effect was named volatile. A write now counts when its interval
overlaps the span from the earlier read's start to the later read's end.

The check also ran once per changed field and scanned every write. It now
runs once per pair of observations, answered by a binary search over the
writes in timeline order and the latest end among each prefix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): catch cross-tool changes across writes; read only item totals

A value that changed across a write was caught only between repeats of
one tool's request line, so a list read before a write and an item read
after it put the post-write value into the initial state. Changes are
now classified per entity in timeline order after observation: across
any recorded write, a change between repeats of a line or between two
lines of the type rewrites the type (every tool recorded as
fields-differ); with no write between, a repeat's change is volatile.
The classification lives in environmentStateMerge.ts.

Any numeric member whose name contained "total" or "count" counted as
an item total, so total_pages, account_id or page_count made complete
lists partial. Item totals are now members whose name tokens include
total or count and none of page, pages or per; a string of digits counts
as a number; and a set has_more, next or next-page token member also
marks the list partial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep post-write observations out of initial state

* fix(environment): retain field provenance and bound state inference work

* feat(environment): serve tool calls from the environment state

`environment-serve --state <state.json>` serves an environment from the
initial state an environment-state view extracted. Read tools whose views
are state-backed answer from a live state store: item views render the held
entity into the recorded envelope, list views render the collection or the
recorded members, and document views return the held value; a missing entity
or one lacking a required field is refused, and requests the state cannot
answer fall back to their recordings with the reason. The /agentenv data
plane implements data/reset, data/add (a rea-state/v1 document as a data
part, base64 bytes, a file:// path or an https URL, validated and replaced
atomically) and data/get, and the card advertises those operations and an
intake declaration. Stage 1 is read-only: write tools are not served in
state mode. Without --state nothing changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(environment): record verification after merging the state view's review fixes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(environment)!: serve the environment state by default

environment-serve derives the environment state at startup by running the
environment-state view in-process on the capture with the manifest's
template rules, so an environment comes with its state and a user supplies
nothing extra. The derived Evidence is the same one inspect-analysis-view
produces by hand. --state is removed; --replay-only serves exactly the
stateless replay of main. When the state cannot be derived, or its catalog
does not fit the discovery budget, the server starts in replay mode, logs
the reason and reports it in the stop summary's `state`. Exported bundles
serve the state by default too: export derives it the same way, so the
verifier's tool count and checked answer match what is served, and the
README describes the served mode.

BREAKING CHANGE: environment-serve answers read tools from the derived
state and does not serve write tools unless --replay-only is given; the
stop summary reports `state` instead of `state_evidence_id`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): bound data-plane replies, keep batch order, cancel intake on stop

- One /agentenv response is held to 64 MiB: replies are measured as they
  are produced and one past the budget becomes a -32000 error stating it.
- Every message of a batch enters the arrival-order queue on arrival, so
  another request's reset never lands between them.
- Stopping the server, or a caller leaving, cancels the active download
  and answers queued calls `cancelled` without running them.
- file:// URIs name only this host: another host or a network share is
  refused before anything is opened.
- A loaded view value may not give an entity another identity.
- List answers refuse a member lacking a field every recording carried,
  as item answers do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep bodies too deep for the state's nesting out of the state

A recorded body at the JSON depth limit was accepted by extraction, then
nested a few levels deeper as a document or entity value, so sealing the
state Evidence threw and environment-serve and environment-export stopped
instead of falling back. stateBodyJson, the one rule extraction and
serving share, now treats a body nesting deeper than the limit minus 16
levels as unusable, so its tool stays a recorded view.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): measure data-plane replies before building them

83fcc739 bounded one /agentenv response to 64 MiB, but it still
serialized each reply whole before measuring it, so a batch of data/get
calls on a large state built every refused reply in full only to drop
it. Each reply is now measured in bounded JSON parts (the shared
jsonParts encoder), its parts kept only while it fits; a reply past the
budget is counted for its exact reply_bytes but never assembled. A test
checks that a refused read is never handed to JSON.stringify whole and
that its reply_bytes is exact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): release batch replies as they are encoded; judge exact numbers by value

answerBatch kept every settled reply in its pending array until the
batch ended, so a batch alternating data/add and data/get held each
loaded state alive; each reply now leaves the queue as it is encoded.

The exact-number policy judged a number by its recorded spelling, so a
state loaded with 1e20 came back from data/get as
100000000000000000000, which data/add refused, breaking the promise
that data/get output loads back unchanged. A number is exact only when
both its recorded spelling and the one serialization writes are safe,
so integers from 2^53 below 1e21 are refused in any spelling; replay
keeps such bodies as recorded text. A domain test checks that every
accepted number is accepted again once written back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep room for refusals and framing in the reply budget

The response budget counted accepted replies only: a budget error that
replaced a reply, and a batch's brackets and commas, were not charged,
so near 64 MiB later refusals pushed the body past the advertised
limit. Every reply still to come now keeps room for its own budget
error, and a batch keeps room for its framing, so a refusal always fits
and the body never passes the budget. Reply ids and which messages are
answered come from one place shared by the replies and the reserve.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): refuse a loaded state that data/get would write past the intake limit

data/add measured only the document it was given, while data/get
writes the state back with the members a document may omit (types, the
state Evidence id, view fields and record citations). A document just
under 32 MiB could therefore load and be reported larger than any intake
accepts, breaking the data/get then data/add round trip. replace now
measures the state as data/get writes it, part by part, and refuses it
past the limit, leaving the state unchanged. The limit moves to the
state document module, so the application store can read it; the
intake re-exports it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): keep merged export checks on the shared test graph

Main's export tests installed module mocks for the whole services
project, which reuses one module graph. A tarball read failure then
missed its importer, and a one-shot heap mock could be taken by another
file. Read those through bindings the tests replace for one case.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(environment): keep a state file whose close failed owned until it closes

A file:// data/add that could not close its descriptor dropped the handle,
and an admission failure whose cleanup close failed dropped its owner, so
nothing could retry the close or report it.

The read now runs through withOwnedRegularFile, and a failed close is
retained in a server-lifetime ArtifactResourceScope. The next file read
retries it first and answers -32000 while it is still open; inline and
https documents still load. Closing the served environment retries it again
and rejects with the unfinished close.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(environment): attempt all state cleanup on close

* fix(environment): bound wide state JSON traversal

* fix(environment): scope cached state reply sizes to requests

* fix(domain): preserve JSON depth diagnostic guidance

---------

Co-authored-by: Edgar Arakelyan <edgar.arakelyan@scale.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <gensokyo-dev@users.noreply.github.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-22 06:06:36 +08:00
Mohith DK f23066f0da fix(reconstruction): fall back to a kind-based title for blank graph labels (#6137)
Fall back to a kind-based obligation title when a graph label is blank

A whitespace-only application observation or semantic node label passed through ?? as the obligation title, and the ledger schema (trimmed, min length 1) then rejected the whole ledger. Use trim() || so a blank label takes the generated title.

Closes #6126
2026-10-22 05:53:42 +08:00
Abel YagubyanandAbelo9996 cbbd317fb5 fix(process): index late scenario events in validation issues (#6160)
Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com>
2026-10-22 05:53:22 +08:00
Abel YagubyanandAbelo9996 d54c4345d3 fix(browser): report zero decoded bytes for undecodable gRPC-web text (#6159)
Co-authored-by: Abelo9996 <Abelo9996@users.noreply.github.com>
2026-10-22 05:53:13 +08:00
bdf0be87c6 fix(mcp): name response-byte and startup ceilings in recovery (#6005)
* fix(mcp): name response-byte and startup ceilings in recovery

Name REA_MCP_MAX_RESPONSE_BYTES, its effective value, the 10485760 byte
floor, and process restart when the stdio response budget is hit, and
state that the setting does not raise Node's string ceiling. Name fixed
MCP startup phase budgets when those deadlines fire, and keep Codex's
30 second startup figure as client guidance.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(mcp): distinguish string ceilings and keep dossier recovery

Name REA's reserved string limit separately from Node's ceiling, restore
detail:summary discovery for an oversized complete evidence bundle, and
apply the doctor deadline sentence when binary_session itself times out.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(mcp): keep retained SDK errors inside the text budget

Quoted SDK validation diagnostics expand when recovery JSON is escaped
into MCP text. Shorten only that preview so the retained-evidence
reference still arrives as parseable error content.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(mcp): keep export advice when a summary exceeds the response budget

An oversized get_evidence_bundle summary no longer repeats detail: summary.
export_evidence_bundle still writes the retained bundle outside one MCP frame,
including when a larger byte budget cannot raise the string ceiling.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* style(mcp): format the SDK diagnostic preview return

oxfmt rejected the fitted-preview ternary in EvidenceMcpServer.ts, which failed Static checks and the CI required gate.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-22 05:48:56 +08:00
0155b89953 fix(browser): name web-bundle budgets and recover web deadlines (#6036)
* fix(browser): name web-bundle budgets and recover web deadlines

Name the existing heap, per-script, and total web-bundle phases, and reject a per-script deadline above the total. Promote capture, CDP command, source-map decode, and source-map fetch deadlines at today's defaults. Name DevTools timeoutMs, and state that discovery, cleanup grace, byte budgets, and Electron hook retention have no runtime override.

Refs #5910

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): apply selected web deadlines to the owning phase

Keep HTTP and browser-socket discovery on the fixed discovery deadline so a
short command_timeout_ms starts at the handshake. Apply scenario
command_timeout_ms to launch startup and the primary connect session, and
keep service-worker restoration on the fixed 5000 ms grace. Omit
command_timeout_ms from recovery text when the operation does not accept it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): retain scenario commands that finish after their deadline

A timed-out scenario command stays owned so cleanup can close a late
browser or context and restore a mutation that lands after the deadline.
Child storage replies use that deadline, scenario recovery names
command_timeout_ms in the capture JSON, and the session test connection
implements beginCleanupGrace.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): keep child command timeouts as the public failure

A nested command such as Page.enable that is delivered but never answered
was stored as protocol_error, hiding the command name, selected deadline,
and recovery. Store the typed timeout and assert that public error.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): keep runtime cleanup on the fixed grace

Switch runtime restoration to the cleanup grace before the first send so a
short command_timeout_ms cannot fail an otherwise successful observation.
Stop advertising command_timeout_ms on list_browser_targets, which never
bounds a post-discovery command.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(browser): prove a short command deadline cannot fail runtime cleanup

Drive restoration through the runtime fixture. A cleanup command that
outlives command_timeout_ms still succeeds, and a stalled restoration
names the fixed 5000 ms grace.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): name the effective source-map decode deadline

codecBudgetLimitation reported a fixed 20-second deadline after a
caller selected decode_timeout_ms. Result and Evidence limitations
now name that effective millisecond deadline.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): separate handshake and target-attach deadlines

A browser-level Target.attachToTarget no longer shares the WebSocket
handshake budget. Blocked service-worker attachment keeps discovery on
its fixed deadline and applies command_timeout_ms to the later handshake
and attach.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): name the web-bundle resource each runtime exhausts

Bun ignores the V8 old-space flag. A Windows Job reports the process
commit limit, and other Bun hosts no longer suggest raising max_heap_mb.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): cite the browser-socket discovery message budget

WebSocket discovery enforces the 64 MiB protocol budget. Timeout
recovery no longer advertises the 8 MiB HTTP document budget.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): bound the blocked service-worker check before discovery

The first registration check has its own fixed 10s deadline. Discovery,
handshake, and Target.attachToTarget stay on their separate deadlines.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(browser): name the operand that stopped web-bundle admission

Host heap pressure no longer claims the fixed result budget is exhausted.
A Windows Bun allocation before the commit limit still names that limit.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-22 05:47:38 +08:00
882 changed files with 62897 additions and 7988 deletions

No files matched your search

@@ -219,6 +219,12 @@ explicit path or endpoint and do not need it.
delivery requires `--evidence_output`.
See the
[Rust binary guide](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
file. This built-in target-free inspection requires no disassembler. Preserve
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
build-id or ABI-tag fields. A missing table is observed absence of those
declarations, not proof the image never carried notes. See the
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
with Node.js 24.x >=24.15 or 26+ providing native SQLite limits. Read schema
@@ -241,6 +247,8 @@ explicit path or endpoint and do not need it.
- PE resource type/name/language trees and static icon group references:
`inspect_pe_resources` with an explicit path. This portable operation requires
no active target or engine; opaque payloads retain file ranges and SHA-256.
- PE export directories and ELF dynamic-symbol exports: `inspect_native_exports`
with an explicit path; portable, no active target or engine.
- Native executable, library, or analysis database: `open_binary`, then
use focused analysis tools directly; call `binary_overview` when metadata or
inventory context is useful and available from the selected provider.
@@ -93,10 +93,14 @@ Use `view: {kind: "environment-state"}` with the manifest's `template_rules` to
extract the data those reads observed instead: entity types and identity
fields, entities with recorded values, documents, and per-tool render plans
checked against the recordings. The state comes only from reads before the
first related write; types, identity and list membership are inferences.
first related write; types, identity and list membership are inferences. The
view also gives each state-changing tool a write plan, served only when a later
recorded read confirms its effect and none contradicts it.
`rea environment-serve manifest.json --evidence capture.json` derives that
state at startup, answers read tools from it and serves `data/reset`,
`data/add` and `data/get` on `/agentenv`; write tools are not served. It
replays the capture instead with `--replay-only` or when the state is
unavailable (not derivable, or a catalog over the discovery budget), and its
stopped document's `state` says which applied.
state at startup, answers read tools from it, serves the write tools whose plan
is served, each applying its effect to that state, and serves `data/reset`,
`data/add` and `data/get` on `/agentenv`; other write tools are not served, and
each plan states why in its `reason`. It replays the capture instead with
`--replay-only` or when the state is unavailable (not derivable, or a catalog
over the discovery budget), and its stopped document's `state` says which
applied.
@@ -22,6 +22,11 @@ requires complete application analysis; `source_paths: []` is invalid, not an
unfiltered request. `detail: "summary"` reduces delivery
and does not by itself reduce analysis work.
ASAR header/index admission defaults to 16 MiB / 16777216 bytes;
`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB, set where REA starts) changes it.
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise it. Or
analyze an extracted directory.
Exact `#alias` specifiers resolve through the importer's
enclosing package `imports` map to package-local files; nested `node_modules`
packages use their own map. Wildcard, recursive, and external-package import
@@ -45,7 +45,8 @@ See [desktop installations](https://github.com/morluto/rea/blob/main/docs/deskto
## Go build metadata
Use `inspect_go_binary` for an explicit local ELF, PE or thin Mach-O when the
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
files need `architecture`) when the
question concerns the embedded compiler version, module/dependency versions,
local replacements or build settings. No `open_binary`, Go installation or
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
@@ -84,7 +85,8 @@ See [IL2CPP metadata](https://github.com/morluto/rea/blob/main/docs/il2cpp-metad
## Rust dependency metadata
Use `inspect_rust_binary` for an explicit local ELF, PE, or thin Mach-O when
Use `inspect_rust_binary` for an explicit local ELF, PE, or Mach-O (universal
files need `architecture`) when
the question concerns the cargo-auditable crate list embedded in `.dep-v0`
(Mach-O: `__DATA,.dep-v0`). No `open_binary`, Rust installation, or target
execution is required. The CLI equivalent is `inspect-rust-binary PATH`.
@@ -94,6 +96,17 @@ dependencies. An omitted package kind stays null. A rustc commit hash is
inferred from a read-only `/rustc/<40-hex>/` path and does not by itself
identify the compiler. See [Rust binaries](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
## ELF note tables
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
question concerns PT_NOTE/SHT_NOTE owners, GNU build-id, or GNU ABI-tag
fields. No `open_binary` or target execution is required. The CLI equivalent
is `inspect-elf-notes PATH`. Preserve artifact SHA-256, table file ranges,
descriptor hashes, and original owner spellings. A missing table is observed
absence of those declarations, not proof the image never carried notes in
another container. See
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
## SQLite snapshots
Use `inspect_sqlite_database` with an explicit local database snapshot for its
@@ -366,6 +379,8 @@ For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
`arm`, `arm64`, `6502`, or `m68k`), and `entry_point` (`0x` hex). File offset 0 is mapped at
@@ -394,6 +409,16 @@ Other payloads are opaque, and this operation does not decode images or simulate
Windows resource loading. Consult the budgets and supported tree profile in
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
## Offline PE and ELF export tables
Use `inspect_native_exports` with an absolute `path`, or
`rea inspect-native-exports ./libexample.so --json`, to read a PE export
directory or ELF dynamic symbol table on any host without opening a native
target. Ordinal-only, forwarded and hidden-version exports stay distinct;
partial or unsupported coverage means the inventory is incomplete, not that
exports are absent. Use `inspect_macho` for Mach-O. See
[offline binary diagnostics](https://github.com/morluto/rea/blob/main/docs/binary-diagnostics.md).
For exact core-module import/export differences, use `compare_wasm_interfaces`
with two complete inspection Evidence records (or current-session retained
references) and explicit scope. Preserve unknown types/duplicate names; static
+1
View File
@@ -191,6 +191,7 @@ Additional tools and host support depend on the target:
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
+1
View File
@@ -32,6 +32,7 @@ export default defineConfig({
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
{ text: "Model artifacts", link: "/model-artifacts" },
{ text: "Rust dependencies", link: "/rust-binaries" },
{ text: "ELF notes", link: "/elf-notes" },
{
text: "JavaScript and Electron",
link: "/javascript-artifact-reconstruction",
+128 -45
View File
@@ -5,7 +5,8 @@ environment: an MCP server whose tools answer from what one retained web
network capture observed, plus the environment card the agentenv framework
reads to discover it. By default the server derives the
[state the capture observed](#the-state-a-capture-observed) at startup and
answers read tools from it; `--replay-only` replays the capture's records
answers read tools from it, and the write tools whose effect the capture
confirmed by changing it; `--replay-only` replays the capture's records
instead. Nothing is fetched from the recorded application; every answer comes
from retained Evidence. `rea environment-export` writes the same pair as a
bundle folder `agent-env run` builds and runs; see
@@ -234,11 +235,12 @@ moment of the capture recorded them. The
[environment state](web-network-captures.md#environment-state) view extracts
instead the data the capture's reads observed before the first related write:
entity types with an identity field, entities with their recorded values,
documents for written paths, and a render plan per read tool, kept only where
it reproduces that tool's recordings from the state. It is the starting state
an environment holds, loads and reports: `environment-serve` derives it from
the capture at startup; see [Serve from the state](#serve-from-the-state).
With `--replay-only`, replay is unchanged.
documents for reads that are not entities, and a render plan per read tool,
kept only where it reproduces that tool's recordings from the state. It is the
starting state an environment holds, loads and reports: `environment-serve`
derives it from the capture at startup; see
[Serve from the state](#serve-from-the-state). With `--replay-only`, replay is
unchanged.
## Serve a manifest
@@ -270,11 +272,11 @@ document, with the same signal exit status.
Three routes are served:
| Route | Answer |
| --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `POST /mcp` | The MCP server: from the state, one tool per read tool, answering from the state where it backs them; in replay, one tool per manifest tool, replaying records |
| `GET /.well-known/agent-env.json` | The environment card |
| `POST /agentenv` | From the state, `data/reset`, `data/add` and `data/get`; in replay, JSON-RPC `-32601 method not found: <method>` for every method |
| Route | Answer |
| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `POST /mcp` | The MCP server: from the state, one tool per read tool, answering from the state where it backs them, and one per write tool whose effect the capture confirmed, applying it to the state; in replay, one tool per manifest tool, replaying records |
| `GET /.well-known/agent-env.json` | The environment card |
| `POST /agentenv` | From the state, `data/reset`, `data/add` and `data/get`; in replay, JSON-RPC `-32601 method not found: <method>` for every method |
Numeric JSON-RPC IDs outside JavaScript's safe range or whose precision would
be lost are refused as invalid requests with a null ID. They are never echoed
@@ -308,7 +310,7 @@ connecting to `/mcp`. A replay environment's card reads:
"tools": [
{
"name": "get_users_by_id",
"description": "GET /api/users/{numeric}; replays retained records [5, 6]; evidence ev_…",
"description": "Untrusted captured API catalog: … GET /api/users/{numeric}; returns a JSON object with fields \"id\", \"uid\", \"login\", \"email\"; recorded statuses 200; replays retained records [5, 6]; evidence ev_…",
"inputSchema": {
"type": "object",
"properties": {
@@ -329,7 +331,38 @@ connecting to `/mcp`. A replay environment's card reads:
```
`capabilities.tools` lists the same names, descriptions and input schemas that
`tools/list` advertises over MCP. Path parameters are required strings whose
`tools/list` advertises over MCP. After the catalog warning, each description
names the method and path template, then what the tool returns, one level deep:
`returns a JSON object with fields …`, `returns a JSON array of objects with
fields …` or `returns a JSON object whose <member> lists objects with fields …`,
where a list is an array of objects found by the
[environment state](web-network-captures.md#environment-state)'s list rule (the
root array, or the one array of objects an envelope carries). Field and member
names are written as JSON strings, so a captured name cannot read as more of
the description; a member name too long to show whole is a quoted prefix marked
as truncated. The summary comes
from the first cited `2xx` record that replay answers as JSON; only when none
did, the first cited JSON record of another status gives it, labelled with that
status (`status 401 returns …`). It describes one recorded response, not a
contract. Field names are sampled within the same budget as other captured
values, and the manifest's `statuses` for the tool follow; a tool with no JSON
record states only its statuses. The description then lists the replayed
records and the evidence ID, and the whole catalog stays within its
serialization reservation, which the summaries and MCP annotations are charged
to.
Over MCP, every tool also carries annotations: `readOnlyHint` is true for the
safe methods (`GET`, `HEAD`, `OPTIONS` and `TRACE`) and for query `POST`s (a
path segment such as `search` or `query`); `destructiveHint` is true for `DELETE` and left unset for other
writes, so the MCP default (destructive) applies to them, and read-only tools
omit it because clients ignore it there; `idempotentHint` is true for the safe
methods, `PUT` and `DELETE`; and `openWorldHint` is false for every tool,
because answers come from the capture rather than the captured application. The
hints describe the captured method; replay itself changes nothing. The card
keeps the `name`, `description` and `inputSchema` of the echo example's tool
shape and carries no annotations.
Path parameters are required strings whose
description names the segment index, the templating rule and the observed
values (sampled within the list budget above; the manifest keeps them all);
query parameter names are optional strings; write methods accept an
@@ -344,8 +377,11 @@ In replay, `extensions`, `operations` and `children_environments` are fixed to
`null`, `[]` and `null`: a replay environment declares no extensions, no
data-plane operations and no children. Served
[from the state](#serve-from-the-state), the card declares the three data-plane
operations and one intake extension instead, leaves write tools out and
describes each tool as answering from the state where the state backs it.
operations and one intake extension instead, and the catalog holds the read
tools and only the write tools whose effect the capture confirmed: a read tool
is described as answering from the state where the state backs it, and a
served write tool by the effect it applies to the state and the body members it
accepts.
## Serve from the state
@@ -368,7 +404,8 @@ A state that is unavailable does not stop the server: it logs a warning naming
the error `code` and message before it binds, then serves replay. The state is
unavailable when it cannot be derived, such as from a
`capture_browser_scenario` capture that selected header values but no response
bodies or over the view's budget, and when its catalog under the served name
bodies or over the view's budget, when `data/get` would write it past the 32 MiB
`data/add` intake limit, and when its catalog under the served name
exceeds the [discovery reservation](#what-a-manifest-is). That check runs
first, so no state is derived for it, and replay is then admitted exactly as
with `--replay-only`. `--replay-only`
@@ -385,24 +422,55 @@ member reports which applied:
Each read tool answers according to its view in the state:
| View | Answer |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold, or one lacking a field every recording carried, is a tool error; a body is never invented. |
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"`, or the recorded members still held. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
| `recorded` | The recording, as replay answers it. |
| View | Answer |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error; a body is never invented. |
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"`, or the recorded members still held. A call with no query argument to a `membership: "all"` view answers from a checked line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
| `recorded` | The recording, as replay answers it. |
A list or document call that names no checked line, including a line recorded
after a write or with other query values, answers from the recording. Every
answer has the replay shape plus `state`: `{"source": "state", "view": …}`
naming the entity, collection or document path, or `{"source": "recording",
"reason": …}` with the view's reason, `no-view` or `unrecorded-request`. A
"reason": …}` with the view's reason, `no-view`, `unrecorded-request` or `page-exceeded` (a call naming a page of a whole list that the collection has outgrown). A
state answer carries the base record's status, content type and `replay`
provenance with `truncated: false`; one whose JSON would exceed 256 KiB is
refused as a `resource_constraint` tool error rather than cut. Serving is
read-only: `POST`, `PUT`, `PATCH` and `DELETE` tools, other than query `POST`s,
are not served, so nothing a call does changes the state. Query `POST`s stay
served and answer from their recordings with the reason `query-body`.
refused as a `resource_constraint` tool error rather than cut. Extraction does
not state-back a document whose rendered JSON exceeds 256 KiB; that read stays
recorded (`rendered-oversize`) and replays the captured body. Query `POST`s
stay served and answer from their recordings with the reason `query-body`.
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
[write plan](web-network-captures.md#environment-state) for them is served:
the capture's later reads confirmed the effect the plan models. Other write
tools are not served, so no call claims a change it does not make; each one's
plan states why in its `reason`. A served write changes the live state, which
every MCP session and `data/get` then see:
| Effect | Call |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, increment and slug changes the plan models. |
| `delete` | Removes the entity the path names. |
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
Its answer has the status and content type of the plan's recorded response,
with the written entity or its identity rendered in, no body, or the recorded
body. A call whose body is not an object, sends a member the capture did not
show stored, sends a fixed member with another value or a member of another
JSON type, or lacks a required member is a tool error naming the accepted
members; so is a rendered answer missing a field the recorded response
carried, and an update or delete of an entity the state does not hold answers
the recorded 400, 404 or 410 of the tool or of an item view of the type, if
any. A refused call leaves the state unchanged. Values a write generates are
deterministic: the clock starts at `write_clock` and advances one second per
served write, sequences take one past both the largest held value and the
largest they generated, and UUIDs and tokens derive from the state Evidence
id, tool, field and write count, skipping held values. `data/reset` and
`data/add` restart the clock, counter and sequences, so the same calls after a
reset give the same state. Answers carry `state: {"source": "state", "view":
"create" | "update" | "delete", …}` or `"document-write"`.
The data plane serves three JSON-RPC methods:
@@ -436,18 +504,29 @@ names its category in `error.data.reason`: `not-found`, `permission`,
downloads, and `cancelled` for a download the server's stop or its caller's
departure ended.
Served creates also bound additional collection history since the last reset
or `data/add` to 1 MiB of deterministic accounting: UTF-8 bytes for each new
collection spelling and its possible sequence keys, plus a 64-byte allowance
per collection and potential sequence-key record. The score bounds identifier
payload and a fixed record allowance; it is not a measurement or guarantee of
process memory. Collections
already present in the Evidence or loaded state are part of the state itself;
the store indexes their unique spellings so they remain known after deletion,
but they do not consume this additional-history budget. A create that would
exceed the budget is refused before the state changes. `data/reset` or `data/add`
clears generated history; if the same calls would hit the limit again, use
fewer distinct collection paths. The 1 MiB score ceiling is 1/32 of the
separate 32 MiB state-intake ceiling, reserving a small, explicit allowance
for identifiers retained beyond the loaded state. A loaded-state path that is
later deleted and then introduced again may consume the additional-history
budget when the store must retain it beyond the source entity.
Stopping the server cancels every pending call, and a caller that leaves before
its reply cancels its own: a download in progress ends, and calls not yet run
are answered `-32000` with `error.data.reason` `cancelled` without running. A
cancelled call leaves the state unchanged, and shutdown never waits for a
download.
A `file://` state file whose close fails stays owned by the server, and
the `data/add` gets `-32000`. The next `file://` load retries that close first
and, while it still fails, answers `-32000` without reading another file;
inline and `https://` documents still load. Closing the server retries it once
more and fails with the unfinished close rather than dropping the descriptor.
One `/agentenv` response carries at most 64 MiB of replies, twice the 32 MiB
a loaded document may hold: room for a `data/get` of a loaded state, while a
batch of reads cannot multiply a large state into one body. Replies are
@@ -531,14 +610,17 @@ The card declares the operations and what `data/add` accepts:
}
```
The state is a projection of recorded reads. List and document calls answer
from the state only for the lines their views checked, and every recorded view
answers from recordings; item calls answer for any held entity but render only
the envelope one recording carried. Values copied outside entity objects,
derived values and counters other than those that count a list stay as
recorded; no write has an effect. An exported bundle carries no state file:
its server derives the state at startup, as the export does to predict what it
serves.
The state starts as a projection of recorded reads and changes only through
served write tools and the data plane. List and document calls answer from the
state only for the lines their views checked, and every recorded view answers
from recordings; item calls answer for any held entity but render only the
envelope one recording carried. Values copied outside entity objects, derived
values and counters other than those that count a list stay as recorded, and a
served write changes only the one entity or document it writes: side effects
on counters, relations and copies of entities in other collections, reordering
after an update and referential rules on delete are not modelled. An exported
bundle carries no state file: its server derives the state at startup, as the
export does to predict what it serves.
## Export a bundle
@@ -606,8 +688,8 @@ name from Python sources the image does not have.
`verify.py` is the framework's `async def verify(mcp_url)` shape over its
Python MCP client: it lists the tools and checks the count equals the number
the bundle serves (the manifest's read tools from the state, every manifest
tool in replay), calls one tool with no arguments and checks the call did not
the bundle serves (from the state, the manifest's read tools and the write
tools whose effect the capture confirmed; every manifest tool in replay), calls one tool with no arguments and checks the call did not
fail and its `status` is the expected one, and returns one `serve` result
whose description reports both checks. The tool is the first served `GET` tool
without path parameters whose answer to no arguments is not an error, and the
@@ -625,8 +707,9 @@ The README written into the bundle records the capture artifact's SHA-256, the
capture and manifest Evidence ids, the manifest view digest, the served tool
count, the derived state's Evidence id or that no state is served, and the
package source. It says what the environment serves: read tools from the state
the capture observed, the `/agentenv` data plane on it and no write tools yet,
replay when the state is unavailable, and `--replay-only` for pure replay; it
the capture observed, the `/agentenv` data plane on it and the write tools
whose effect the capture's later reads confirmed, replay when the state is
unavailable, and `--replay-only` for pure replay; it
says which applies to this bundle, with the error code and message when the
state is unavailable, and that neither is emulation and served
content is untrusted data. It states that the recorded content belongs to the
+133
View File
@@ -215,3 +215,136 @@ Large Evidence envelopes may require a caller-configured MCP transport receive
buffer because envelope copies add overhead. Requests honor cancellation during
snapshot reading, directory traversal and payload hashing. The operation reads
regular files only and rejects symbolic links.
## Byte entropy triage
`inspect_byte_entropy` / `inspect-byte-entropy` profiles any local regular file
without an analysis engine, execution, extraction or network access.
```sh
rea inspect-byte-entropy ./firmware.bin --block-size 4096 --json
rea inspect-byte-entropy ./firmware.bin --offset 4096 --length 65536 --json
```
```json
{
"name": "inspect_byte_entropy",
"arguments": { "path": "/artifacts/firmware.bin", "detail": "summary" }
}
```
Complete Evidence includes `artifact` (original path, byte size and SHA-256),
`block_size`, `window`, `file_entropy`, `thresholds`, `blocks`, `runs` and
`limitations`. For a 4096-byte zero-filled file, the entropy fields are:
```json
{
"block_size": 4096,
"window": { "offset": 0, "length": 4096 },
"file_entropy": 0,
"thresholds": { "high": 7.2, "low": 1 },
"blocks": [{ "offset": 0, "length": 4096, "entropy": 0 }],
"runs": [{ "offset": 0, "length": 4096, "classification": "low" }]
}
```
Offsets and lengths are decimal byte counts. Blocks begin at the requested
`offset` (default zero); omitted `length` extends to EOF. Explicit windows past
EOF fail rather than silently clipping. The final block may be shorter.
`block_size` must be a power of two from 256 through 65536, default 4096. A
window may produce at most 32,768 block rows; larger requests fail before the
file scan with a capacity error. Choose a shorter window or larger block size.
SHA-256 and `file_entropy` always cover the entire file, including when a window
is selected. Empty files and empty windows are accepted; empty entropy is zero
by convention. The tool reads the size the file reports. A file that reports size
0, such as a Linux `/proc` entry, returns no blocks with a limitation that the
result does not show the file is empty. File identity, size and timestamps are checked across the read;
detected changes produce an error rather than Evidence.
Shannon entropy is reported in bits per byte from 0 to 8, rounded to four decimal
places. Adjacent blocks with reported entropy at least 7.2 form `high` runs;
blocks at most 1.0 form `low` runs. Other blocks interrupt runs. Entropy does not
prove encryption, and this measure cannot distinguish encrypted from compressed
data. Small blocks can underestimate entropy; the result is a triage clue.
MCP `detail: "summary"` retains complete Evidence in the session and returns
file entropy, block/run counts, thresholds, the selected window, and the ten
longest runs (ties ordered by file offset). Use its `parent_evidence_id` to
retrieve complete Evidence, or request its summary with `inspect_analysis_view`.
The CLI returns complete Evidence through the same service. Summary detail
requires a session capable of retaining Evidence.
Reads use one block-sized buffer and a 256-entry histogram. Complete output
can become very large because it includes one row per selected block, and
summary delivery still retains that full list. Use a shorter length or a
larger block size to reduce it. Full Evidence and its serialization require
memory proportional to the number of selected blocks. The 32,768-row ceiling bounds the actual retained block objects and
their serialized representation: a cap-sized run below produced 1,514,877
serialized bytes and increased V8 heap use by 21.2 MiB on the measured host.
The cap therefore bounds this output independently of file size; it is not a
portable process-memory guarantee. Narrow the window or increase block size to
reduce retained rows further.
Local service measurement (macOS arm64, Node 24.18.0, 100,000,000-byte file,
4096-byte blocks, file already in the OS cache): first process run took 1.113 s
wall / 1.148 s CPU with 145.3 MiB peak RSS; a second run in the same process took
1.015 s / 1.035 s CPU, with cumulative peak RSS 155.8 MiB. Each read 100,000,000
bytes and produced 24,415 blocks. Measurements include Evidence construction,
summary projection and JSON serialization, with no child processes. Complete
Evidence serialized to 1,122,303 bytes and its summary to 2,705 bytes. A
cap-sized 134,217,728-byte sparse file with 4096-byte blocks produced 32,768
rows and 1,514,877 serialized bytes; V8 heap use increased by 21.2 MiB and RSS
by 18.1 MiB in that process. This separate measurement used the same macOS
arm64 host and Node version. Cold disk I/O, token counts and an agent
investigation were not measured; there was no previous equivalent tool to
benchmark. These figures do not establish a portable peak-memory bound,
constant memory for retained Evidence or faster agent task completion.
## Portable native exports
`inspect_native_exports` / `inspect-native-exports` reads the export table of an
explicit PE or ELF image on any supported host, without an active target,
native engine, `objdump` or Windows SDK:
```sh
rea inspect-native-exports ./runtimes/linux-x64/native/libexample.so --json
```
```json
{
"name": "inspect_native_exports",
"arguments": { "path": "/artifacts/native/example.dll" }
}
```
The complete Evidence identifies the stable artifact snapshot by path, byte
count and SHA-256.
- PE rows reproduce the export directory: each used export-address-table slot
with its `ordinal`, one row per name (aliases share an ordinal), `address`
as the preferred-base virtual address, or `forwarder` text when the slot
forwards to another DLL. Ordinal-only exports have a null `name`. The
directory's DLL name, ordinal base and timestamp are reported, and an image
without an export directory is a complete empty inventory.
- ELF rows list defined `.dynsym` symbols with global, weak or GNU-unique
binding and default or protected visibility. Each row keeps `value`
(`st_value` as stored; bit 0 marks Thumb code on 32-bit ARM), `size`, `type`
and `binding`. `version` is null for an unversioned symbol, the
`.gnu.version_d` index, name and `hidden` flag for a versioned one, or
`"unknown"` when the version table cannot be read. A hidden version is not
the default version that a by-name `dlsym` lookup selects.
Unreadable names, out-of-range name ordinals and unreadable forwarders are
listed in `issues`, their entries are not reported, and coverage is `partial`.
Unresolved symbol versions keep their rows: the version name is null for an
index no definition names, and `version` is `"unknown"` when the version table
cannot be read; coverage is likewise `partial`. A dynamic symbol table without
its null entry is malformed. An ELF without section headers is `unsupported`:
enumerating dynamic symbols from the dynamic segment is not
implemented. Mach-O exports are reported by `inspect_macho`.
`max_file_bytes` defaults to 256 MiB (maximum 512 MiB; CLI `--max-file-bytes`).
Export directories in PE headers are read like those in sections. An export or
dynamic symbol table above 262144 entries, more than 262144 rows, or more than
16 MiB of retained names fails without partial success. Export rows do not establish that a symbol is reachable, safe to call
or loaded at runtime.
+40 -19
View File
@@ -385,7 +385,8 @@ section explicitly truncated.
Disallowed-origin entries are filtered before retention.
Selected script-source collection shares a 32 MiB aggregate UTF-8 retention
budget and a 20-second deadline across the capture. An oversized source is
budget and the `command_timeout_ms` deadline (default 20000 ms) across the
capture. The byte budget has no runtime override. An oversized source is
omitted while later smaller sources may still fit. When the deadline expires,
remaining source reads are skipped. Script metadata, per-item omission reasons,
and `script_sources` completeness remain available. A single CDP response can
@@ -393,19 +394,25 @@ still be materialized up to the separate 64 MiB transport limit before source
admission; the retention budget does not bound that transient allocation.
Bundle parsing and semantic inspection run sequentially in an owned child,
without executing captured code. `max_heap_mb` defaults to 1024 MiB V8
old-space; `analysis_timeout_ms` defaults to 30000 per script, including startup
and transfer. A failed or timed-out script becomes an explicit unknown while
completed findings and later scripts remain available. `total_analysis_timeout_ms`
defaults to 300000 across script analysis; expiry returns a typed timeout with
partial Evidence and identifies unstarted scripts. These controls are also
available as CLI flags with hyphenated names. Capture and optional source-map
fetching have separate budgets.
without executing captured code. These are separate phases. `max_heap_mb`
defaults to 1024 MiB V8 old-space (range 128..16384). `analysis_timeout_ms`
defaults to 30000 per script, including startup and transfer.
`total_analysis_timeout_ms` defaults to 300000 across script analysis. A
per-script deadline above the effective total is rejected. A failed or
timed-out script becomes an explicit unknown while completed findings and later
scripts remain available. Total-deadline expiry returns a typed timeout with
partial Evidence and identifies unstarted scripts. `source_map_fetch_timeout_ms`
defaults to 30000 and is an item limitation, not that analysis timeout. These
controls are also available as CLI flags with hyphenated names. Worker protocol
and result byte budgets have no runtime override.
One web-bundle capture/analysis is admitted per REA process at a time. Completed
script results share an 8 MiB serialized transfer budget, with host heap
reservation checked before decoding and validating another result.
Exhausted result capacity stops further script launches and returns a typed
When the host reservation is what reaches zero, the failure names that heap
headroom and the remaining fixed budget. When the 8 MiB allowance itself is
exhausted, the failure names that fixed budget and says it has no runtime
override. Either stop ends further script launches and returns a typed
resource failure with completed facts and explicit unstarted-script unknowns.
Worker control frames are limited to 64 KiB. Cleanup must confirm child termination
before releasing its private files; failed cleanup retains its exact owner and
@@ -415,8 +422,10 @@ facts, including cancellation during final cleanup.
The worker heap setting is not a process-family RSS limit. Node enforces V8
old-space flags; Bun ignores them. Windows Bun instead uses an owned process
commit limit with a 512 MiB runtime reserve; other Bun hosts have no enforced
memory limit. These bounds do not establish an aggregate host-memory limit
across other REA operations or external processes.
memory limit. An allocation failure before that Windows limit is reported as
reached still names the configured limit and says it was not reached. These
bounds do not establish an aggregate host-memory limit across other REA
operations or external processes.
Resource-tree inventories distinguish a valid empty `resources` array from a
missing or malformed collection. Invalid resource entries are omitted with
@@ -430,12 +439,24 @@ an invalid supplied size remains unknown with incomplete resource coverage while
the resource's other facts are retained. Zero and nonnegative fractional sizes
are preserved.
CDP discovery has a 10-second deadline. Each WebSocket handshake and command
response has its own 20-second deadline, separate from the caller-selected
observation window. A stalled handshake or command reports `provider_timeout`;
caller cancellation still reports cancellation. WebSocket messages remain subject
to the [64 MiB limit](#request-boundary). REA does not impose a message-count or
pending-command quota. On cancellation, it closes only its own WebSocket,
CDP discovery has a 10-second response deadline. HTTP discovery documents
use an 8 MiB budget. Browser-socket discovery (`Browser.getVersion`,
`Target.getTargets`) uses the 64 MiB protocol message budget. Neither budget
nor the discovery deadline has a runtime override. `list_browser_targets` accepts no
`command_timeout_ms`: HTTP discovery and browser-socket discovery
(`Browser.getVersion`, `Target.getTargets`) stay on that fixed deadline, and
the socket closes without a later command. Other CDP tools use
`command_timeout_ms` (default 20000) for each WebSocket handshake and command
response, separate from the caller-selected observation window, discovery, and
the 5000 ms cleanup grace. A browser-level socket's handshake and the following
`Target.attachToTarget` each receive that full deadline; they do not share one
budget. A stalled handshake or command reports
`provider_timeout` and names `command_timeout_ms` only when that operation
accepts it; caller cancellation still reports cancellation. WebSocket messages
remain subject to the [64 MiB limit](#request-boundary), which has no runtime
override. REA does
not impose a message-count or pending-command quota. On cancellation, it closes
only its own WebSocket,
waiting up to one second before terminating it. Malformed unsolicited events
and malformed replies to pending commands poison the connection so later
commands fail closed. A correlated reply's session ID must match the command's
@@ -444,7 +465,7 @@ rejects it as unknown; other command rejections retain their reported failure.
Network and console coverage starts only after REA attaches and enables the relevant CDP domains. `prior_activity_available` is always `false`; absence from these arrays is not evidence that an event never occurred. Scripts are different: scripts the page parsed before REA attached are still reported under `scripts`. A page that loaded before attach can therefore list its scripts while `network.requests` lacks the requests that fetched them. If CDP reuses a request ID without supplying redirect metadata, REA retains the predecessor observation and marks network coverage incomplete.
Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Each fetch operation has a 30-second deadline and a 64 MiB aggregate streamed-response budget across redirects. A deadline or byte-limit failure remains explicitly unavailable and makes bundle completeness partial. Source-map decoding also bounds raw resolved identities to 64 MiB and aggregate inline evidence to 32 MiB. Failed maps preserve their failure context when it fits; if another context cannot fit, collection stops with the full requested count, the returned processed count, and an explicit unknown-results limitation. Retained mappings and original sources are returned inline. See [source-location limits](web-source-location.md) for the structural decoder bounds. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
Source maps are not fetched by inspection. Bundle analysis fetches them only when `fetch_source_maps` is selected, with exact-origin redirect checks and no credentials, cookies, or referrer. Each fetch operation uses `source_map_fetch_timeout_ms` (default 30000) and a 64 MiB aggregate streamed-response budget across redirects. The byte budget has no runtime override. A fetch overrun is an item limitation, not a thrown analysis timeout. A deadline or byte-limit failure remains explicitly unavailable and makes bundle completeness partial. Source-map decoding also bounds raw resolved identities to 64 MiB and aggregate inline evidence to 32 MiB. Failed maps preserve their failure context when it fits; if another context cannot fit, collection stops with the full requested count, the returned processed count, and an explicit unknown-results limitation. Retained mappings and original sources are returned inline. See [source-location limits](web-source-location.md) for the structural decoder bounds. Original source artifacts with `.ts`, `.tsx`, `.mts`, or `.cts` suffixes carry `text/typescript`; other source names retain `text/javascript`.
source-map items retain the script's `declared_url` separately from the observed
`effective_url`. successful maps use the latter as their source-resolution base.
+15 -3
View File
@@ -23,9 +23,21 @@ that rejection does not roll back browser effects. REA removes its owned
future-document scripts, restores live frame methods, then disconnects.
Add secret declarations, storage seeds, and additional artifact or event
capture only when needed. Action, secret, and storage counts are not capped.
Duration, action, and navigation timeouts remain fixed provider-owned liveness
limits. `capture.limits` controls retained event and network-content budgets;
it does not change those liveness deadlines.
Duration, action, and navigation timeouts remain separate from CDP command
deadlines. `command_timeout_ms` (default 20000; omit it to keep that default
and leave it out of Evidence) bounds launch startup, the connect-over-CDP
handshake, target lookup, environment commands, and storage CDP commands,
including replies from attached child targets. Set it in the scenario JSON.
`capture-browser-scenario` has no `--command-timeout-ms` option.
It does not change action `timeout_ms`, `wait_for_timeout.duration_ms`, or
navigation. Service-worker restoration keeps a fixed 5000 ms grace and does
not inherit a raised `command_timeout_ms`. CDP discovery stays on its own
fixed deadline. When service workers are blocked, the registration check
before discovery has its own fixed 10000 ms deadline; `command_timeout_ms`
does not apply because discovery has not started. The discovery deadline
covers endpoint discovery only. The following WebSocket handshake and
`Target.attachToTarget` each use `command_timeout_ms`. `capture.limits` controls retained event and network-content
budgets; it does not change those liveness deadlines.
## Minimal launch example
+36 -2
View File
@@ -63,6 +63,16 @@ or repeat it to select several types.
Selection limits decoding to the selected types; member references remain an
assembly-wide inventory, so the output can still be large.
Select `--detail summary --evidence-bundle /absolute/path/members.json` to
receive identity, selection, coverage, limitations and collection counts while
saving the complete original Evidence in a new bundle. Summary requires that
explicit destination; existing files are preserved. Later
`inspect-analysis-view` calls can page the exact parent from the saved bundle
without reopening the assembly. See
[CLI summary and saved member rows](managed-code-analysis.md#cli-summary-and-saved-member-rows)
for the complete command sequence. Omitted or `--detail complete` retains the
full inline response.
Choose the workflow from the artifact's contents rather than its `.exe` or
`.dll` extension. The managed parser requires PE/CLI metadata; it does not
unpack .NET single-file hosts or decode IL2CPP. For the image, type and method
@@ -145,6 +155,16 @@ dependency edges, and the section's file range. A missing `.dep-v0` section is
null metadata, not proof that the binary has no dependencies. See
[Rust binaries](rust-binaries.md).
Inspect ELF PT_NOTE and SHT_NOTE tables without executing the image:
```bash
rea inspect-elf-notes /absolute/path/to/program --json
```
The workflow reports note owners, types, descriptor hashes, and GNU build-id
or ABI-tag fields when those descriptors match. A missing note table is
observed absence of those declarations. See [ELF note tables](elf-notes.md).
Configure [Hopper or Ghidra](installation.md#hopper), the
[IDA adapter](ida-provider.md), or [Kuna](kuna-provider.md) before analyzing a
native target. Substitute
@@ -279,7 +299,12 @@ Exports preserve an existing destination unless `--overwrite` is explicit.
For application traces and version comparisons, see
[JavaScript application workflows](javascript-application-workflows.md).
MCP [retained Evidence references](mcp-contracts.md#retained-application-evidence-inputs)
belong to one connection. Separate CLI calls consume full saved records.
belong to one connection. Separate CLI calls can consume full saved records.
`inspect-analysis-view REQUEST.json --evidence-bundle BUNDLE.json` resolves
the request's `retained-evidence` source ID from that authenticated bundle,
including bundles exported from an MCP session. The same view JSON works for
every supported parent kind; no target is reopened and no record is selected
by file order. Omit `--evidence-bundle` when the request supplies inline Evidence.
## Import historical source
@@ -399,7 +424,8 @@ evidence file's `evidence_id`; otherwise the command refuses with
`invalid_request` before binding. Before binding it derives the
`environment-state` view of the capture with the manifest's `template_rules`,
as `inspect-analysis-view` would, and serves from it: read tools whose views
are state-backed answer from a live state, write tools are not served, and
are state-backed answer from a live state, write tools are served only where
the capture's later reads confirmed their effect, and
`/agentenv` serves `data/reset`, `data/add` and `data/get`. When the state
cannot be derived, or its catalog exceeds the discovery reservation, it logs a
warning with the error code and message and serves replay; `--replay-only` serves replay without deriving. In replay every
@@ -536,6 +562,14 @@ command, objdump, Rizin and GDB, and the agent environment server
signal-derived status. This describes the current source routing; it is not a
claim that every command or provider runtime has been independently exercised.
Commands with `--evidence_output` select their delivered view, then publish the
complete Evidence bundle atomically. Cancellation before publication leaves no
new file. A publication that has committed can win a later cancellation; the
`cancelled` result then keeps the written path and byte count under
`details.diagnostics.evidence_bundle`. If staging cleanup stays incomplete
after commit, the `cleanup_incomplete` error reports the same file as
`details.published_path` and `details.bytes`.
When the reader of piped output closes early, as `rea ... | head` does, the
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
SIGPIPE-terminated writer). The operation itself may have completed; only its
+7 -1
View File
@@ -245,4 +245,10 @@ application path and capture outcome to the static graph, while frames, scripts,
workers, and execution claims remain unavailable until a separate passive runtime
capture provides them.
ASAR observation reads the header through one open file handle, validates its declared lengths against the archive size, and limits encoded JSON headers to 16 MiB before parsing. The header cache also retains at most 16 MiB of encoded headers across eight archives. Oversized headers produce a scope refusal.
ASAR observation reads the header through one open file handle and validates its
declared lengths against the archive size. Header/index admission is the same
budget used by static ASAR reads: 16 MiB / 16777216 bytes by default, or
`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB) when set. An oversized header is
not parsed; selecting a member path does not bypass admission, and the page
stays unauthorized. The header cache separately retains at most that many bytes
of already-admitted encoded headers across eight archives. Evicting that cache does not raise admission.
+53
View File
@@ -0,0 +1,53 @@
# ELF note tables
`inspect_elf_notes` / `inspect-elf-notes` answers whether one selected local
ELF32/ELF64 image declares `PT_NOTE` program headers or `SHT_NOTE` sections,
which note owners and types those tables store, and which GNU build-id or
ABI-tag fields those notes carry. It reads the file without executing it and
does not require a disassembler, debug package, or package catalog.
```bash
rea inspect-elf-notes /absolute/path/to/program --json
```
The CLI resolves relative paths against the caller's working directory. MCP
requires an absolute filesystem path and preserves that exact selection:
```json
{
"name": "inspect_elf_notes",
"arguments": { "path": "/artifacts/program" }
}
```
## Observations
The inline Evidence result contains the artifact path, SHA-256, and byte length.
`tables` lists each declared note program header or section, including the
section name when a string table is present. Each `entries[]` item reports owner,
type, descriptor SHA-256, and a hex preview for descriptors of 64 bytes or less.
For GNU notes, `gnu.build_id_hex`, `gnu.abi_tag`, or `gnu.gold_version` are
filled when the descriptor matches that type. Other owners keep `gnu` null.
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
appear twice when both tables declare it.
A missing table list is observed absence of those declarations. It is not proof
the image never carried notes in another container or overlay.
## Limits and failures
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
32 note entries are examined by default across every table. Retained note
names, section names, and GNU descriptor text share that file's byte budget,
and repeated table ranges count again. Each descriptor range is hashed once,
and each GNU version's trailing NUL run is scanned once; neither walk may
exceed that byte ceiling. Malformed ELF headers, truncated header
tables, declared note tables that extend past the file, and truncated
note entries fail without partial success. ELF extended
numbering (`PN_XNUM`, including a section count stored when `e_shnum` is 0) is
unsupported.
This workflow does not execute the image or verify GNU build-id uniqueness
against package catalogs.
+15 -5
View File
@@ -109,11 +109,21 @@ module record and no dependency records has an observed empty dependency list.
## Supported boundary
The built-in TypeScript reader supports ELF, PE and thin Mach-O images with Go's
inline or historical pointer build-info encoding. Source ranges refer to the
original stable file snapshot, including pointer-backed strings. It does not
scan arbitrary byte strings as compiler metadata or interpret a fat Mach-O
container as one selected architecture.
The built-in TypeScript reader supports ELF, PE and thin Mach-O images, and one
selected slice of a universal Mach-O, with Go's inline or historical pointer
build-info encoding. Source ranges refer to the original stable file snapshot,
including pointer-backed strings. It does not scan arbitrary byte strings as
compiler metadata.
A universal (FAT) Mach-O needs `architecture` (`x86`, `x86_64`, `arm`
or `arm64`; CLI `--architecture`). REA reads its own FAT table, without `lipo`,
and validates the selected slice's range, alignment and Mach-O header CPU
against the table. The result keeps the universal file as `artifact` (its
SHA-256 and size), adds `selected_slice` with the slice offset, length, raw CPU
type/subtype and the FAT table digest, and reports every source range as a
file offset inside that slice. Without `architecture`, or when the file has no
such slice or several, the request fails on `architecture` and lists the slices
present. `architecture` on any other image is rejected rather than ignored.
For ELF images without a named build-info section, discovery checks each
nonempty writable, non-executable load segment in program-header order. This
+1 -1
View File
@@ -50,7 +50,7 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
| Task | Guides |
| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [ELF notes](elf-notes.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
+14 -1
View File
@@ -106,6 +106,18 @@ only reduces delivery; source selection reduces text acquisition and analysis.
Explicit native provider or snapshot routing rejects `--source-path` rather
than ignoring it.
ASAR header/index admission defaults to 16 MiB / 16777216 bytes. Set
`REA_ASAR_MAX_HEADER_BYTES` in the environment that starts REA to a decimal
byte count of at most 268435456 (256 MiB) to raise or lower it; restart REA
after changing it. Invalid values are configuration errors, never a silent
fallback. The archive index is read before any member can be selected, so
`source_paths`, `--source-path`, `detail: "summary"`, `max_heap_mb`, and
available host memory do not bypass or raise it. When the header exceeds the
effective allowance, the failure names it, whether it came from the default or
the setting, and the setting's limits. Alternatively, analyze an extracted
application directory, which does not admit an ASAR header. Repeating the unchanged
request fails the same way.
## Read the analysis
`analyze_javascript_application` returns the complete analysis Evidence by
@@ -172,7 +184,8 @@ for the tool-result contract.
Use MCP `detail: "summary"` for a large application, then read selected views of
the retained analysis. Summary changes delivery only. To reduce source parsing,
select exact [source paths](#select-sources-before-analysis).
select exact [source paths](#select-sources-before-analysis). Source selection
does not bypass ASAR header/index admission.
If a complete response exceeds the transport budget, use its retained Evidence
ID for a summary, module page, or item, or export the bundle. See
+6 -1
View File
@@ -106,7 +106,12 @@ nodes. REA evaluates compatible static candidates conservatively:
that disagreement. If a mapped candidate's digest is unavailable and no
mapped content match is known, the comparison remains `unknown`.
6. Missing authorization mappings and incomplete or truncated input coverage
remain unknown or truncated rather than becoming absence claims.
remain unknown or truncated rather than becoming absence claims. A runtime
location with no automatic artifact-root or caller-declared mapping into
any layer, and no content match, is `unknown` with
`no-authorized-location-mapping`, with or without captured bytes; several
content matches remain `ambiguous` under rule 4. Only a mapped location can
be `unmatched` with `no-static-content-match`.
Every successful cross-layer `observed_as` edge has
`authority: cross-layer-reconciliation` and `state: inferred`. The underlying
+1 -1
View File
@@ -68,7 +68,7 @@ The background export belongs to the session. `close_binary`, or the end of a CL
The assembly label index is scanned once per export reader. Cancelling a request stops its wait without interrupting other readers; closing the session aborts and drains the shared scan before releasing its files.
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a five-second matching deadline and a 128 MiB worker heap. Literal matching remains the default. Cancellation or a matching deadline leaves the session available. This release has no runtime override for that deadline or heap.
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a five-second matching deadline and a worker old-generation heap. Literal matching remains the default. Cancellation or a matching deadline leaves the session available. This release has no runtime override for that deadline. The worker requests 128 MiB. A positive Node.js `--max-old-space-size` on the process that starts REA, including `NODE_OPTIONS`, overrides that request and sets the effective heap. `--max-old-space-size-percentage` takes precedence over that flag in either order. REA does not turn the percentage into a MiB, so the effective heap stays unestablished instead of the 128 MiB request or the absolute flag. The reported `NODE_OPTIONS` value is the process initial environment when that block can be read. A later edit to the environment is not the value Node applied, and an unreadable initial value leaves the effective heap unestablished. The launch Node options are the initial command line on Linux when that buffer still has a program entry, not a later edit to `process.execArgv`. A preload can clear that array after V8 has applied the flags. Setting `process.title` or `--title` overwrites the command line, so a buffer without a program entry leaves the effective heap unestablished. `--env-file` or `--env-file-if-exists` can supply `NODE_OPTIONS` when the initial environment has none. That file is not part of the initial environment block, so the effective heap stays unestablished instead of the 128 MiB request. An initial `NODE_OPTIONS` is still used, because Node does not let the file replace it. A positive `--max-heap-size`, including the underscore form and launch `NODE_OPTIONS` on Node versions that allow it, sets the total heap rather than the old generation and can be lower than the 128 MiB request. The effective old generation stays unestablished while that value is in effect. A zero or empty value clears it. `--experimental-config-file` or `--experimental-default-config-file` can set `nodeOptions`, including a heap percentage, from a file that is not read back. The effective heap stays unestablished when either flag is present, including when an absolute `--max-old-space-size` is also on the command line or in the recovered `NODE_OPTIONS`. Node accepts the config path as `=` or the next argument.
Single-function subprocess output and each cached source facet share a 16 MiB buffered-text capacity. Cached spans are checked before allocation; larger text remains in the local export and the error identifies the file and range. `analyze_function` initially returns the complete dossier, which can still be large; later retained views do not remove that first transfer. Prefer the individual text tools for a focused question.
+59 -4
View File
@@ -85,7 +85,7 @@ rejects selected summary before analysis; complete delivery remains available.
Complete `inspect_managed_members` delivery always includes the assembly-wide
`member_refs` and `exported_types`, which `type_names` does not narrow, so even
one selected or unmatched type can return megabytes. Select
one selected or unmatched type can return megabytes. In MCP, select
`{"path":"/absolute/path/Assembly.dll","type_names":["Namespace.Type"],"detail":"summary"}`
to keep the complete source in the session and receive identity, metadata
status, the selection with its unmatched names, coverage/issues, the separate
@@ -115,6 +115,49 @@ literal pool's own coverage. Older Evidence that never examined exported types
or user strings reports a null count and refuses those pages rather than
returning an empty one.
#### CLI summary and saved member rows
The CLI accepts the same `complete` and `summary` detail choices. Omitted or
`--detail complete` returns complete Evidence. To receive a summary, choose a
new Evidence bundle destination so its original complete parent remains
available after the command exits:
```bash
rea inspect-managed-members /absolute/path/Assembly.dll --type Namespace.Type \
--detail summary --evidence-bundle /absolute/path/evidence/members.json --json
```
`--evidence-bundle` writes the complete original Evidence in REA's canonical
bundle format. Its parent directory must exist, and it never replaces an
existing path. Relative paths resolve against the command's working directory.
The option also works with complete delivery when a saved bundle is useful.
Summary without a destination is rejected before analysis. A publication failure
returns a typed error with the completed Evidence under
`details.partial_observation`; it does not report successful summary delivery.
Cancellation before publication prevents the new output. Once atomic
publication has started, it can complete despite cancellation; a cancelled
delivery then retains the committed path and byte count under
`details.diagnostics.evidence_bundle`.
Copy `normalized_result.parent_evidence_id` from the summary into the
`source.evidence_id` field of the `managed-members` request above, then save that
small request as `members-page.json` and run:
```bash
rea inspect-analysis-view /absolute/path/evidence/members-page.json \
--evidence-bundle /absolute/path/evidence/members.json --json
```
The view command authenticates the selected bundle and resolves that exact
parent ID. It does not reopen the assembly, so the original target may have
been moved or removed. Subsequent calls can select another collection or use
the returned `coverage.next_offset` to continue the same collection. An inline
source already includes its parent and cannot also select `--evidence-bundle`.
Summary changes delivery, while type selection still controls decoding and the
same inspection resource limits apply. Each separate CLI page reads and
authenticates the saved bundle; the full source is retained on disk.
## Shipped scope
The canonical parser admits PE/CLI bytes and their metadata/CIL without loading
@@ -372,6 +415,15 @@ function implements a specific managed method.
`verify_managed_native_boundaries` compares declared P/Invoke module and entry
point identities with supplied native export or function evidence. Name/module
matches establish declaration links; they do not establish that a call executed.
Native observations may be `inspect_macho`, `analyze_function`, or
`inspect_native_exports` Evidence. One `inspect_native_exports` record supplies
a whole PE export directory or ELF dynamic symbol table, on any host. An exact
name verifies only when it is what a by-name lookup binds: a PE export that
forwards to another DLL, or an ELF symbol whose only version is hidden or whose
version could not be read, is `inferred` with basis
`qualified-export-candidate` and a limitation naming the reason. Ordinal-only
PE exports have no name and match no declaration.
When either side lacks a usable module identity, a matching name remains
unresolved and retains the candidate evidence; REA reports a module mismatch
only when both identities are available and differ.
@@ -622,6 +674,9 @@ which managed methods executed or how the CLR resolved them.
Managed member comparisons report a `metadata` dimension when an observed
MethodDef flags or implementation flags value differs, including accessibility
or synchronization changes. This is separate from the CIL/signature matching
tiers: identical instructions do not establish unchanged method metadata. The
reported difference is static metadata evidence, not proof of runtime behavior.
or synchronization changes. Observed method-body `init_locals` and `max_stack`
differences also contribute to `metadata`, even when CIL decoding is incomplete.
An unavailable header value keeps body coverage unknown rather than establishing
equality or a difference. These comparisons are separate from the CIL/signature
matching tiers: identical instructions do not establish unchanged method metadata.
The reported difference is static metadata evidence, not proof of runtime behavior.
+9 -6
View File
@@ -234,11 +234,14 @@ The destination changes only when a complete export is atomically published.
Clients that explicitly configure a larger receive buffer can set the REA
server's `REA_MCP_MAX_RESPONSE_BYTES` environment variable to the same byte
count. This setting must be a safe decimal integer at least 10485760; REA
reserves 1024 bytes for the envelope. Raising it restores complete inline
delivery for responses that fit that buffer and Node's single-string limit.
It does not change the client's buffer, analysis coverage, or retained content.
Ordinary responses keep their existing complete result contract.
count. This setting must be a safe decimal integer at least 10485760 (10 MiB);
REA reserves 1024 bytes for the envelope. The value is read when the REA
process starts, so a change requires a restart and a matching client receive
buffer. Raising it restores complete inline delivery for responses that fit
that buffer. It does not raise REA's reserved single-string limit, which is
1024 UTF-16 code units below Node's single-string ceiling, and it does not
raise that Node ceiling. It does not change analysis coverage or retained
content. Ordinary responses keep their existing complete result contract.
### Error recovery and deadline ownership
@@ -329,7 +332,7 @@ See [environment manifests](web-network-captures.md#environment-manifest).
`inspect_analysis_view` also accepts `environment-state` for the same parents,
extracting the initial state the recorded reads observed: entity types with an
identity field, entities, documents for written non-entity paths, and a render
identity field, entities, documents for non-entity reads, and a render
plan per read tool that reproduces its recordings from the state or names why it
cannot. Types and identity are inferences, so the Evidence is always inferred.
See [environment state](web-network-captures.md#environment-state).
+56 -6
View File
@@ -202,10 +202,38 @@ Function dossiers retain this same string evidence. Native call edges retain
Hopper's partial `CallReference` classification and exact endpoints; detailed
reference flags remain unavailable rather than being invented.
Regex searches use ECMAScript Unicode syntax in a cancellable worker with a
five-second matching deadline and a 128 MiB worker heap. Deadline or
five-second matching deadline and a worker old-generation heap. Deadline or
cancellation stops matching while leaving the Hopper API available. This
release has no runtime override for that deadline or heap. Literal mode
retains Unicode casefold matching.
release has no runtime override for that deadline. The worker requests 128
MiB. A positive Node.js `--max-old-space-size` on the process that starts
REA, including `NODE_OPTIONS`, overrides that request and sets the effective
heap. `--max-old-space-size-percentage` takes precedence over that flag in
either order. REA does not turn the percentage into a MiB, so the effective
heap stays unestablished instead of the 128 MiB request or the absolute flag.
The reported `NODE_OPTIONS` value is the process initial environment when that
block can be read. A later edit to the environment is not the value Node
applied, and an unreadable initial value leaves the effective heap
unestablished. The launch Node options are the initial command line on Linux
when that buffer still has a program entry, not a later edit to
`process.execArgv`. A preload can clear that array after V8 has applied the
flags. Setting `process.title` or `--title` overwrites the command line, so a
buffer without a program entry leaves the effective heap unestablished.
`--env-file` or `--env-file-if-exists` can supply `NODE_OPTIONS`
when the initial environment has none. That file is not part of the initial
environment block, so the effective heap stays unestablished instead of the
128 MiB request. An initial `NODE_OPTIONS` is still used, because Node does
not let the file replace it. A positive `--max-heap-size`, including the
underscore form and launch `NODE_OPTIONS` on Node versions that allow it,
sets the total heap rather than the old generation and can be lower than the
128 MiB request. The effective old generation stays unestablished while that
value is in effect. A zero or empty value clears it.
`--experimental-config-file` or `--experimental-default-config-file` can set
`nodeOptions`, including a heap percentage, from a file that is not read
back. The effective heap stays unestablished when either flag is present,
including when an absolute `--max-old-space-size` is also on the command
line or in the recovered `NODE_OPTIONS`. Node accepts the config path as
`=` or the next argument.
Literal mode retains Unicode casefold matching.
## Bounded static traces
@@ -434,11 +462,33 @@ complete observation.
`rea inspect-native-dispatch-metadata <app-or-binary>` prefers a validated macOS Mach-O byte reader. It decodes
64-bit little-endian Objective-C class/metaclass records, superclass pointers,
absolute/relative method entries, ivar offsets/sizes/alignment and protocol
declarations. It also decodes simple Swift conformances, static synchronous
witness slots, signed-relative pointers and non-generic, non-resilient class
vtable descriptors. Records include exact virtual addresses/file offsets and
declarations. It also decodes Swift nominal type declarations, simple Swift
conformances, static synchronous witness slots, signed-relative pointers and
non-generic, non-resilient class vtable descriptors. Records include exact virtual addresses/file offsets and
artifact evidence. Vtable indexes are metadata word offsets; witness indexes
start after the conformance header. Names unavailable in metadata remain null.
The byte reader holds the whole selected file in memory, so it admits files up
to 64 MiB / 67108864 bytes by default. Select a larger budget with
`max_input_bytes` (CLI `--max-input-bytes`), at most 1 GiB / 1073741824
bytes. The whole file counts, including every slice of a universal Mach-O.
Results record `input_admission` with the effective budget, its source
(`default` or `caller`) and the bytes retained. A refusal names the observed
file size, the effective budget and its source, and either the budget to
select or, above 1 GiB, a deep analysis provider.
`swift_decls` lists each `__swift5_types` nominal type descriptor, which
survives stripping: `kind` (`class`, `struct`, `enum`, or `actor` from the
class actor flag), the descriptor's own `name`, `module`, `qualified_name`
(`Outer.Inner`), `generic`, the metadata `accessor_address`, the descriptor
`location`, and evidence citing its `__swift5_types` record.
Private and local types set `anonymous_context` because their anonymous
context is omitted from `qualified_name`. When a parent context cannot be
resolved, such as an extension of a type named only by a mangled name, a
missing module, or a cyclic chain, `qualified_name` is null and the entry is
partial with a reason. Access level, superclass, conformances, and fields are
not reported by this facet. The `swift_declarations` coverage counters count
examined type records and fully decoded declarations. Class vtable recovery
reads the same descriptors, so an unreadable type record also keeps
`swift_class_vtable_descriptors` partial.
The `swift_class_vtable_descriptors` coverage counters count examined vtable
slots and successfully decoded slots, rather than type-section entries.
The `objc_class_method_ivar_metadata` counters count admitted class and
+1
View File
@@ -145,6 +145,7 @@ npx rea-agents@latest setup
| بنية ELF دون تشغيل | الأقسام والمقاطع والرموز ومعلومات إعادة التموضع الأصلية، وآليات الحماية المحتملة المستدلّ عليها بالفحص الثابت | pwntools يوفّره المستدعي على Linux x64؛ [تشخيص الملفات الثنائية](../binary-diagnostics.md) |
| شيفرة EVM البايتية | محددات التوجيه، وإزاحات البايتات، والمعاملات المستنتجة، وقابلية تغيير الحالة | مدخل محلي يحمل بايتات خامًا أو تمثيلًا سداسيًا عشريًا على Linux x64؛ [دليل EVM دون تشغيل](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| أعطال Linux المسجّلة | سجلات note الخام، ومسجّلات المعالج/الإشارات لكل خيط مسجّل، وترشيحات اختيارية لخرائط الذاكرة | pwntools يوفّره المستدعي على Linux x64؛ GDB/pwndbg اختياريان؛ [الأعطال المسجّلة](../recorded-crashes.md) |
| تثبيتات سطح المكتب | عائلات وقت التشغيل، والملفات الداعمة، وعلامات المكونات، والخطوات التالية | فحص محدود للملفات والترويسات;[تطبيقات سطح المكتب](../desktop-applications.md) |
| JavaScript / Electron | الوحدات، وعمليات الاستيراد، وخرائط المصدر، والمسارات، وIPC، وعلاقات إضافات الشيفرة الأصلية | Node.js وnpm؛ [تحليل التطبيقات](../javascript-application-workflows.md) |
+1
View File
@@ -145,6 +145,7 @@ REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätz
| Offline-ELF-Struktur | Sektionen, Segmente, ursprüngliche Symbole/Relokationen und mögliche Schutzmaßnahmen aus statischer Analyse | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; [Binärdiagnostik](../binary-diagnostics.md) |
| EVM-Bytecode | Dispatch-Selektoren, Byte-Offsets, abgeleitete Argumente und Veränderbarkeit des Zustands | Lokale Eingabe mit Rohbytes oder Hexdarstellung unter Linux x64; [Offline-EVM-Leitfaden](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Aufgezeichnete Linux-Abstürze | Unverarbeitete Note-Einträge, Register/Signale aller aufgezeichneten Threads und optionale Zuordnungskandidaten | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; optional GDB/pwndbg; [aufgezeichnete Abstürze](../recorded-crashes.md) |
| Desktop-Installationen | Laufzeitkandidaten, Quelldateien, Komponenten und nächste Schritte | Begrenzte Verzeichnis- und Headerprüfung;[Desktop-Anwendungen](../desktop-applications.md) |
| JavaScript / Electron | Module, Imports, Source Maps, Routen, IPC und Beziehungen zu nativen Add-ons | Node.js und npm; [Anwendungsanalyse](../javascript-application-workflows.md) |
+1
View File
@@ -149,6 +149,7 @@ npx rea-agents@latest setup
| Δομή ELF χωρίς εκτέλεση | Ενότητες, τμήματα, αρχικά σύμβολα/εγγραφές μετεγκατάστασης και ενδείξεις πιθανών μέτρων προστασίας από στατική ανάλυση | pwntools που παρέχει ο χρήστης σε Linux x64· [διαγνωστικά δυαδικών αρχείων](../binary-diagnostics.md) |
| Δυαδικά αρχεία Go | Ενσωματωμένη έκδοση μεταγλωττιστή, modules, αντικαταστάσεις και ρυθμίσεις μεταγλώττισης | Ενσωματωμένη στατική εξέταση ELF/PE/thin Mach-O· δεν απαιτείται εγκατάσταση του Go |
| Δυαδικά αρχεία Rust | Ενσωματωμένα ονόματα, εκδόσεις, πηγές και σχέσεις εξάρτησης των crates από το cargo-auditable | Ενσωματωμένη στατική εξέταση ELF/PE/thin Mach-O· [δυαδικά αρχεία Rust](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Αρχείο WASM | Έλεγχο εγκυρότητας, ενότητες, ακριβείς μορφές εισαγωγών/εξαγωγών σε WAT και αποκωδικοποιημένο WAT | WABT 1.0.42 που παρέχει ο χρήστης· `inspect_wasm_artifact` |
| Bytecode EVM | Selectors επιλογής συνάρτησης, μετατοπίσεις σε bytes, εκτιμώμενα ορίσματα και δυνατότητα μεταβολής της κατάστασης | Τοπική είσοδος σε ακατέργαστα bytes ή δεκαεξαδική μορφή σε Linux x64· [οδηγός EVM χωρίς εκτέλεση](../evm-bytecode.md) |
| Βάσεις δεδομένων SQLite | Πίνακες, στήλες, ευρετήρια, όψεις, triggers και επιλεγμένες γραμμές κανονικών πινάκων | Τοπικό στιγμιότυπο βάσης δεδομένων/WAL· Node.js 24.x >=24.15 ή 26+ με εγγενή όρια SQLite |
+1
View File
@@ -145,6 +145,7 @@ REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las h
| Estructura ELF sin ejecutar | Secciones, segmentos, símbolos/reubicaciones originales y posibles medidas de mitigación detectadas mediante análisis estático | pwntools proporcionado por quien llama, en Linux x64; [diagnóstico de binarios](../binary-diagnostics.md) |
| Bytecode EVM | Selectores de despacho, desplazamientos en bytes, argumentos inferidos y mutabilidad | Entrada local con bytes sin procesar o representación hexadecimal, en Linux x64; [guía de EVM sin ejecutar](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Fallos de Linux registrados | Registros note sin procesar, registros/señales de cada hilo registrado y posibles correspondencias de memoria opcionales | pwntools proporcionado por quien llama, en Linux x64; GDB/pwndbg opcionales; [fallos registrados](../recorded-crashes.md) |
| Instalaciones de escritorio | Posibles entornos de ejecución, archivos de evidencia, componentes y siguientes pasos | Inspección acotada de directorios y cabeceras;[Aplicaciones de escritorio](../desktop-applications.md) |
| JavaScript / Electron | Módulos, importaciones, mapas de código fuente, rutas, IPC y relaciones con complementos nativos | Node.js y npm; [análisis de aplicaciones](../javascript-application-workflows.md) |
+1
View File
@@ -137,6 +137,7 @@ REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، به‌همراه np
| ساختار ELF آفلاین | بخش‌ها، سگمنت‌ها، نمادها و relocationهای اصلی و گزینه‌های احتمالی کاهش آسیب‌پذیری ایستا | pwntools تأمین‌شده توسط فراخواننده روی Linux x64؛ [تشخیص باینری](../binary-diagnostics.md) |
| بایت‌کد EVM | selectorهای dispatch، offsetهای بایتی، آرگومان‌های استنباط‌شده و قابلیت تغییر وضعیت | دادهٔ خام/hex محلی روی Linux x64؛ [راهنمای EVM آفلاین](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| کرش‌های ثبت‌شدهٔ لینوکس | یادداشت‌های خام، رجیسترها/سیگنال‌های تمام threadهای ثبت‌شده و گزینه‌های احتمالی نگاشت | pwntools تأمین‌شده توسط فراخواننده روی Linux x64؛ GDB/pwndbg اختیاری؛ [کرش‌های ثبت‌شده](../recorded-crashes.md) |
| نصب‌های دسکتاپ | نامزدهای زمان اجرا، فایل‌های شاهد، نشانه‌های مؤلفه‌ها و مراحل بعدی | بررسی محدود پوشه‌ها و سرآیندها;[برنامه‌های دسکتاپ](../desktop-applications.md) |
| JavaScript / Electron | ماژول‌ها، importها، source mapها، مسیرها، IPC و ارتباط افزونه‌های بومی | Node.js و npm؛ [تحلیل برنامه](../javascript-application-workflows.md) |
+1
View File
@@ -149,6 +149,7 @@ REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les
| Artefact WASM | Validation, sections, formes WAT exactes d'import/export et WAT décodé | WABT 1.0.42 fourni par l'utilisateur ; `inspect_wasm_artifact` |
| Bytecode EVM | Sélecteurs de dispatch, offsets en octets, arguments inférés et mutabilité | Fichier local brut ou hexadécimal sous Linux x64 ; [guide EVM hors ligne](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Crashs Linux enregistrés | Notes brutes, registres et signaux de chaque thread enregistré, et correspondances de mappings facultatives | pwntools fourni par l'utilisateur sous Linux x64 ; GDB/pwndbg en option ; [crashs enregistrés](../recorded-crashes.md) |
| Installations de bureau | Environnements candidats, fichiers justificatifs, composants et étapes suivantes | Inspection bornée des répertoires et en-têtes;[Applications de bureau](../desktop-applications.md) |
| JavaScript / Electron | Modules, imports, source maps, routes, IPC et relations avec les modules natifs | Node.js et npm ; [analyse d'applications](../javascript-application-workflows.md) |
+1
View File
@@ -152,6 +152,7 @@ Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis:
| Artefak WASM | Validasi, bagian, bentuk WAT import/ekspor yang tepat, dan WAT yang didekode | WABT 1.0.42 yang disediakan pemanggil; `inspect_wasm_artifact` |
| Bytecode EVM | Selector dispatch, offset byte, argumen yang disimpulkan, dan sifat mutabilitas | Pembawa data raw/hex lokal; [panduan EVM offline](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Crash Linux yang direkam | Catatan mentah, register/sinyal setiap thread yang direkam, dan kandidat pemetaan opsional | pwntools yang disediakan pemanggil; GDB/pwndbg opsional; [crash yang direkam](../recorded-crashes.md) |
| Instalasi desktop | Kandidat runtime, berkas pendukung, penanda komponen, dan langkah berikutnya | Pemeriksaan direktori dan header terbatas;[Aplikasi desktop](../desktop-applications.md) |
| JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](../javascript-application-workflows.md) |
+1
View File
@@ -149,6 +149,7 @@ REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strument
| Artefatto WASM | Convalida, sezioni, forme WAT di import/export esatte e WAT decodificato | WABT 1.0.42 fornito dal chiamante; `inspect_wasm_artifact` |
| Bytecode EVM | Selettori di dispatch, offset dei byte, argomenti inferiti e mutabilità | Carrier raw/hex locale; [guida EVM offline](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Crash Linux registrati | Note grezze, registri/segnai di ogni thread registrato e candidati mapping opzionali | pwntools fornito dal chiamante; GDB/pwndbg opzionale; [crash registrati](../recorded-crashes.md) |
| Installazioni desktop | Runtime candidati, file di supporto, componenti e passi successivi | Ispezione limitata di directory e intestazioni;[Applicazioni desktop](../desktop-applications.md) |
| JavaScript / Electron | Moduli, import, source map, route, IPC e relazioni con add-on nativi | Node.js e npm; [analisi delle applicazioni](../javascript-application-workflows.md) |
+1
View File
@@ -145,6 +145,7 @@ REA には Node.js 22.x(>=22.19)、24.x(>=24.11)、または 26+ と npm
| オフライン ELF レイアウト | セクション、セグメント、元のシンボル/再配置情報、静的解析によるセキュリティ対策の候補 | Linux x64 で利用者が用意する pwntools;[バイナリ診断](../binary-diagnostics.md) |
| EVM バイトコード | ディスパッチセレクター、バイトオフセット、推定された引数と状態変更の可否 | Linux x64 上のローカルの生バイト/16 進数の入力ファイル;[オフライン EVM ガイド](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| 記録済みの Linux クラッシュ | 生の note レコード、記録された全スレッドのレジスター/シグナル、任意のマッピング候補 | Linux x64 で利用者が用意する pwntools;GDB/pwndbg は任意;[記録済みクラッシュ](../recorded-crashes.md) |
| デスクトップのインストール先 | ランタイム候補、根拠ファイル、コンポーネントの目印、次の調査手順 | 上限付きのディレクトリ・ヘッダー検査;[デスクトップアプリ](../desktop-applications.md) |
| JavaScript / Electron | モジュール、インポート、ソースマップ、ルート、IPC、ネイティブアドオンの関係 | Node.js と npm;[アプリケーション分析](../javascript-application-workflows.md) |
+1
View File
@@ -153,6 +153,7 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
| 오프라인 ELF 레이아웃 | 섹션, 세그먼트, 원본 심볼/재배치 정보, 정적 분석으로 얻은 보안 완화 기법 후보 | Linux x64에서 호출자가 제공하는 pwntools; [바이너리 진단](../binary-diagnostics.md) |
| Go 바이너리 | 내장된 컴파일러 버전, 모듈, `replace` 지시어, 빌드 설정 | REA 내장 ELF/PE/thin Mach-O 정적 검사; Go 설치 불필요 |
| Rust 바이너리 | cargo-auditable이 내장한 크레이트 이름, 버전, 소스, 의존성 관계 | REA 내장 ELF/PE/thin Mach-O 정적 검사; [Rust 바이너리](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| WASM 아티팩트 | 유효성 검사, 섹션, import/export의 정확한 WAT 형식, 디코딩된 WAT | 호출자 제공 WABT 1.0.42; `inspect_wasm_artifact` |
| EVM 바이트코드 | 디스패치 선택자, 바이트 오프셋, 추론한 인자와 상태 변경 가능 여부 | Linux x64의 로컬 원시 바이트/16진수 입력 파일; [오프라인 EVM 가이드](../evm-bytecode.md) |
| SQLite 데이터베이스 | 테이블, 열, 인덱스, 뷰, 트리거, 선택한 일반 테이블의 행 | 로컬 데이터베이스/WAL 스냅샷; Node.js 24.15 이상의 24.x 또는 26 이상(네이티브 SQLite 한도 적용) |
+1
View File
@@ -151,6 +151,7 @@ REA-യ്ക്ക് Node.js 22.x (>=22.19), 24.x (>=24.11), അല്ലെ
| Offline ELF layout | Sections, segments, യഥാർത്ഥ symbols/relocations, static mitigation candidates | Linux x64-ൽ caller നൽകുന്ന pwntools; [binary diagnostics](../binary-diagnostics.md) |
| Go binaries | ഉൾച്ചേർത്ത compiler version, modules, replacements, build settings | Built-in static ELF/PE/thin Mach-O പരിശോധന; Go installation ആവശ്യമില്ല |
| Rust binaries | ഉൾച്ചേർത്ത cargo-auditable crate പേരുകൾ, versions, sources, dependency edges | Built-in static ELF/PE/thin Mach-O പരിശോധന; [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| WASM artifact | Validation, sections, കൃത്യമായ import/export WAT രൂപങ്ങൾ, decoded WAT | Caller നൽകുന്ന WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bytecode | Dispatch selectors, byte offsets, അനുമാനിച്ച arguments, mutability | Linux x64-ൽ local raw/hex carrier; [offline EVM guide](../evm-bytecode.md) |
| SQLite databases | Tables, columns, indexes, views, triggers, തിരഞ്ഞെടുത്ത സാധാരണ table rows | Local database/WAL snapshot; native SQLite പരിമിതികളോടെ Node.js 24.x >=24.15 അല്ലെങ്കിൽ 26+ |
+1
View File
@@ -145,6 +145,7 @@ REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe na
| Układ pliku ELF w trybie offline | Sekcje, segmenty, oryginalne symbole/relokacje i potencjalne mechanizmy zabezpieczeń wskazane przez analizę statyczną | pwntools dostarczone przez użytkownika w systemie Linux x64; [diagnostyka plików binarnych](../binary-diagnostics.md) |
| Kod bajtowy EVM | Selektory wywołań, przesunięcia bajtowe, wywnioskowane argumenty i mutowalność | Lokalny plik z surowymi bajtami lub zapisem szesnastkowym w systemie Linux x64; [przewodnik EVM offline](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Zarejestrowane awarie systemu Linux | Surowe rekordy note, rejestry/sygnały każdego zarejestrowanego wątku i opcjonalni kandydaci na mapowania | pwntools dostarczone przez użytkownika w systemie Linux x64; opcjonalnie GDB/pwndbg; [zarejestrowane awarie](../recorded-crashes.md) |
| Instalacje desktopowe | Kandydaci środowisk uruchomieniowych, pliki źródłowe dowodów, komponenty i dalsze kroki | Ograniczona inspekcja katalogów i nagłówków;[Aplikacje desktopowe](../desktop-applications.md) |
| JavaScript / Electron | Moduły, importy, mapy źródeł, trasy, IPC i powiązania z natywnymi dodatkami | Node.js i npm; [analiza aplikacji](../javascript-application-workflows.md) |
+1
View File
@@ -145,6 +145,7 @@ O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As fer
| Estrutura ELF sem execução | Seções, segmentos, símbolos/relocações originais e possíveis mecanismos de mitigação identificados por análise estática | pwntools fornecido pelo chamador no Linux x64; [diagnóstico de binários](../binary-diagnostics.md) |
| Bytecode EVM | Seletores de despacho, offsets de bytes, argumentos inferidos e mutabilidade | Entrada local com bytes brutos ou representação hexadecimal no Linux x64; [guia de EVM sem execução](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Falhas do Linux registradas | Registros note brutos, registradores/sinais de cada thread registrada e candidatos opcionais a mapeamentos | pwntools fornecido pelo chamador no Linux x64; GDB/pwndbg opcionais; [falhas registradas](../recorded-crashes.md) |
| Instalações desktop | Ambientes candidatos, arquivos de evidência, componentes e próximos passos | Inspeção limitada de diretórios e cabeçalhos;[Aplicações desktop](../desktop-applications.md) |
| JavaScript / Electron | Módulos, importações, source maps, rotas, IPC e relações com complementos nativos | Node.js e npm; [análise de aplicativos](../javascript-application-workflows.md) |
+1
View File
@@ -145,6 +145,7 @@ REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а так
| Структура ELF без запуска | Секции, сегменты, исходные символы/релокации и возможные механизмы защиты по данным статического анализа | pwntools, предоставленный вызывающей стороной, в Linux x64; [диагностика бинарных файлов](../binary-diagnostics.md) |
| Байт-код EVM | Селекторы диспетчеризации, смещения байтов, предполагаемые аргументы и изменяемость состояния | Локальные необработанные байты или шестнадцатеричное представление в Linux x64; [руководство по EVM без запуска](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Записанные сбои Linux | Необработанные записи note, регистры/сигналы каждого записанного потока и необязательные кандидаты сопоставления с памятью | pwntools, предоставленный вызывающей стороной, в Linux x64; необязательные GDB/pwndbg; [записанные сбои](../recorded-crashes.md) |
| Установленные настольные приложения | Возможные среды выполнения, файлы с доказательствами, компоненты и дальнейшие шаги | Ограниченная проверка каталогов и заголовков;[Настольные приложения](../desktop-applications.md) |
| JavaScript / Electron | Модули, импорты, карты исходного кода, маршруты, IPC и связи с нативными дополнениями | Node.js и npm; [анализ приложений](../javascript-application-workflows.md) |
+1
View File
@@ -145,6 +145,7 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26
| โครงสร้าง ELF แบบออฟไลน์ | เซกชัน เซกเมนต์ สัญลักษณ์/รีโลเคชันต้นฉบับ และตัวเลือกกลไกป้องกันที่พบจากการวิเคราะห์แบบสถิต | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; [การวินิจฉัยไบนารี](../binary-diagnostics.md) |
| ไบต์โค้ด EVM | ตัวเลือกฟังก์ชันสำหรับการเรียก ออฟเซ็ตไบต์ อาร์กิวเมนต์และความสามารถในการเปลี่ยนสถานะที่อนุมานได้ | ไฟล์บนเครื่องที่มีไบต์ดิบหรือข้อมูลเลขฐานสิบหกบน Linux x64; [คู่มือ EVM แบบออฟไลน์](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| ข้อมูลการแครชของ Linux ที่บันทึกไว้ | ระเบียน note ดิบ รีจิสเตอร์/สัญญาณของทุกเธรดที่บันทึกไว้ และตัวเลือกการแมปเพิ่มเติม | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; เลือกใช้ GDB/pwndbg เพิ่มเติมได้; [การแครชที่บันทึกไว้](../recorded-crashes.md) |
| ไดเรกทอรีติดตั้งแอปเดสก์ท็อป | รันไทม์ที่เป็นไปได้ ไฟล์หลักฐาน ตัวบ่งชี้ส่วนประกอบ และขั้นตอนถัดไป | ตรวจสอบไดเรกทอรีและส่วนหัวโดยจำกัดขอบเขต;[แอปเดสก์ท็อป](../desktop-applications.md) |
| JavaScript / Electron | โมดูล การนำเข้า ซอร์สแมป เส้นทาง IPC และความสัมพันธ์กับส่วนเสริมเนทีฟ | Node.js และ npm; [การวิเคราะห์แอปพลิเคชัน](../javascript-application-workflows.md) |
+1
View File
@@ -147,6 +147,7 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç
| Çevrimdışı ELF yapısı | Bölümler, segmentler, özgün semboller/yeniden konumlandırmalar ve statik analizden olası güvenlik önlemleri | Linux x64 üzerinde çağıranın sağladığı pwntools; [ikili dosya tanılama](../binary-diagnostics.md) |
| Go ikili dosyaları | Gömülü compiler sürümü, modüller, replacement'lar ve build ayarları | Yerleşik statik ELF/PE/thin Mach-O incelemesi; Go kurulumu gerekmez |
| Rust ikili dosyaları | Gömülü cargo-auditable crate adları, sürümleri, kaynakları ve bağımlılık ilişkileri | Yerleşik statik ELF/PE/thin Mach-O incelemesi; [Rust ikili dosyaları](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| WASM artefaktı | Doğrulama, bölümler, tam import/export WAT biçimleri ve decode edilmiş WAT | Çağıranın sağladığı WABT 1.0.42; `inspect_wasm_artifact` |
| EVM bayt kodu | Yönlendirme seçicileri, bayt ofsetleri, çıkarımla belirlenen argümanlar ve durum değiştirilebilirliği | Linux x64 üzerinde yerel ham bayt/onaltılık girdi; [çevrimdışı EVM rehberi](../evm-bytecode.md) |
| SQLite veritabanları | Tablolar, sütunlar, index'ler, view'lar, trigger'lar ve seçilen sıradan tablo satırları | Yerel database/WAL snapshot'ı; native SQLite sınırlarıyla Node.js 24.x >=24.15 veya 26+ |
+1
View File
@@ -145,6 +145,7 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та
| Структура ELF без запуску | Секції, сегменти, початкові символи/релокації та можливі захисні механізми за даними статичного аналізу | pwntools, наданий стороною, що викликає інструмент, у Linux x64; [діагностика бінарних файлів](../binary-diagnostics.md) |
| Байт-код EVM | Селектори диспетчеризації, зміщення байтів, виведені припущення про аргументи та змінюваність стану | Локальні необроблені байти або шістнадцяткове представлення у Linux x64; [посібник EVM без запуску](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Записані аварійні завершення в Linux | Необроблені записи note, регістри/сигнали кожного записаного потоку й необов'язкові кандидати зіставлення пам'яті | pwntools, наданий стороною, що викликає інструмент, у Linux x64; необов'язкові GDB/pwndbg; [записані аварійні завершення](../recorded-crashes.md) |
| Встановлені настільні програми | Можливі середовища виконання, файли доказів, компоненти та наступні кроки | Обмежена перевірка каталогів і заголовків;[Настільні програми](../desktop-applications.md) |
| JavaScript / Electron | Модулі, імпорти, карти вихідного коду, маршрути, IPC і зв'язки з нативними доповненнями | Node.js і npm; [аналіз застосунків](../javascript-application-workflows.md) |
+1
View File
@@ -145,6 +145,7 @@ REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với np
| Bố cục ELF ngoại tuyến | Các section, segment, ký hiệu/thông tin tái định vị gốc và các biện pháp giảm thiểu có thể có từ phân tích tĩnh | pwntools do bên gọi cung cấp trên Linux x64; [chẩn đoán tệp nhị phân](../binary-diagnostics.md) |
| Bytecode EVM | Bộ chọn điều phối, độ lệch byte, tham số suy luận và khả năng thay đổi trạng thái | Đầu vào cục bộ chứa byte thô hoặc dạng thập lục phân trên Linux x64; [hướng dẫn EVM ngoại tuyến](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| Sự cố Linux đã ghi lại | Bản ghi note thô, thanh ghi/tín hiệu của mọi luồng được ghi lại và các ứng viên ánh xạ tùy chọn | pwntools do bên gọi cung cấp trên Linux x64; GDB/pwndbg tùy chọn; [sự cố đã ghi lại](../recorded-crashes.md) |
| Thư mục cài đặt ứng dụng máy tính | Các runtime có khả năng được dùng, tệp bằng chứng, dấu hiệu thành phần và bước tiếp theo | Kiểm tra thư mục và phần đầu tệp có giới hạn;[Ứng dụng máy tính](../desktop-applications.md) |
| JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](../javascript-application-workflows.md) |
+1
View File
@@ -149,6 +149,7 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
| WASM 工件 | 驗證、節、精確的匯入/匯出 WAT 形式及解碼後的 WAT | 呼叫方提供的 WABT 1.0.42;`inspect_wasm_artifact` |
| EVM 位元組碼 | 分派選擇器、位元組偏移、推斷的參數和狀態可變性 | Linux x64 上的本地原始位元組/十六進位輸入載體;[離線 EVM 指南](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| 已記錄的 Linux 崩潰 | 原始 note 記錄、每個已記錄執行緒的暫存器/訊號及可選的對映候選項 | Linux x64 上由呼叫方提供的 pwntools;可選的 GDB/pwndbg;[已記錄崩潰](../recorded-crashes.md) |
| 桌面安裝目錄 | 執行階段候選、支持檔案、元件標記與後續操作 | 有界的目錄與檔頭檢查;[桌面應用](../desktop-applications.md) |
| JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm;[應用分析](../javascript-application-workflows.md) |
+27 -26
View File
@@ -141,32 +141,33 @@ npx rea-agents@latest setup
REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。额外工具和主机支持取决于分析目标:
| 目标 | REA 返回的内容 | 要求与指南 |
| --------------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| 原生二进制文件 | 伪代码、汇编、字符串、符号、调用和引用 | Hopper、Ghidra 或 IDA;[原生分析](../native-investigation.md) |
| 离线 ELF 布局 | 节、段、原始符号/重定位信息及静态防护机制候选项 | Linux x64 上由调用方提供的 pwntools;[二进制诊断](../binary-diagnostics.md) |
| Go 二进制文件 | 内嵌编译器版本、模块、替换项和构建设置 | 内置静态 ELF/PE/thin Mach-O 检查;无需安装 Go |
| WASM 工件 | 验证、节、精确的导入/导出 WAT 形式及解码后的 WAT | 调用方提供的 WABT 1.0.42;`inspect_wasm_artifact` |
| EVM 字节码 | 分派选择器、字节偏移、推断的参数和状态可变性 | Linux x64 上的本地原始字节/十六进制输入载体;[离线 EVM 指南](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| 已记录的 Linux 崩溃 | 原始 note 记录、每个已记录线程的寄存器/信号及可选的映射候选项 | Linux x64 上由调用方提供的 pwntools;可选的 GDB/pwndbg;[已记录崩溃](../recorded-crashes.md) |
| 桌面安装目录 | 运行时候选、支持文件、组件标记和后续操作 | 有界的目录与文件头检查;[桌面应用](../desktop-applications.md) |
| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm;[应用分析](../javascript-application-workflows.md) |
| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器;[浏览器分析](../browser-observation.md) |
| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR;原生 mitmproxy 捕获需要 Linux 上的 mitmdump;[捕获指南](../web-network-captures.md) |
| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos;[pcap 指南](../pcap-analysis.md) |
| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查;[托管代码指南](../managed-code-analysis.md) |
| PyInstaller | CArchive, PYZ, Python bytecode | Static inspection; [Python packaged applications](../python-packaged-applications.md) |
| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK;[Android 指南](../android-analysis.md) |
| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb(模拟器或实体设备);[ADB 指南](../adb-device-analysis.md) |
| JEB 项目 | 引擎支持的项目单元、工件摘要和类型/方法的伪代码 | 由调用方启动、提供 MCP 的 JEB 客户端,通过 REA_JEB_MCP_URL 选择;[JEB 指南](../jeb-analysis.md) |
| Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器;[资源指南](../apktool-resource-analysis.md) |
| Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无(静态 APK 解析);[Flutter 指南](../flutter-analysis.md) |
| 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob;[固件指南](../firmware-analysis.md) |
| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](../packet-captures.md) |
| 录制的逻辑信号 | 解码注释、样本范围和诊断信息 | Linux: sigrok-cli 0.7.2 + prlimit; [信号记录指南](../signal-recordings.md) |
| 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 应用](../apple-application-analysis.md) |
| 进程行为 | 终端输出、交互、退出和文件系统观察结果,以及运行对比 | 支持原生 PTY 的 Linux/macOS;[进程捕获](../process-capture.md) |
| 目标 | REA 返回的内容 | 要求与指南 |
| --------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
| 原生二进制文件 | 伪代码、汇编、字符串、符号、调用和引用 | Hopper、Ghidra 或 IDA;[原生分析](../native-investigation.md) |
| 离线 ELF 布局 | 节、段、原始符号/重定位信息及静态防护机制候选项 | Linux x64 上由调用方提供的 pwntools;[二进制诊断](../binary-diagnostics.md) |
| Go 二进制文件 | 内嵌编译器版本、模块、替换项和构建设置 | 内置静态 ELF/PE/thin Mach-O 检查;无需安装 Go |
| WASM 工件 | 验证、节、精确的导入/导出 WAT 形式及解码后的 WAT | 调用方提供的 WABT 1.0.42;`inspect_wasm_artifact` |
| EVM 字节码 | 分派选择器、字节偏移、推断的参数和状态可变性 | Linux x64 上的本地原始字节/十六进制输入载体;[离线 EVM 指南](../evm-bytecode.md) |
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
| 已记录的 Linux 崩溃 | 原始 note 记录、每个已记录线程的寄存器/信号及可选的映射候选项 | Linux x64 上由调用方提供的 pwntools;可选的 GDB/pwndbg;[已记录崩溃](../recorded-crashes.md) |
| 桌面安装目录 | 运行时候选、支持文件、组件标记和后续操作 | 有界的目录与文件头检查;[桌面应用](../desktop-applications.md) |
| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm;[应用分析](../javascript-application-workflows.md) |
| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器;[浏览器分析](../browser-observation.md) |
| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR;原生 mitmproxy 捕获需要 Linux 上的 mitmdump;[捕获指南](../web-network-captures.md) |
| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos;[pcap 指南](../pcap-analysis.md) |
| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查;[托管代码指南](../managed-code-analysis.md) |
| PyInstaller | CArchive, PYZ, Python bytecode | Static inspection; [Python packaged applications](../python-packaged-applications.md) |
| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK;[Android 指南](../android-analysis.md) |
| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb(模拟器或实体设备);[ADB 指南](../adb-device-analysis.md) |
| JEB 项目 | 引擎支持的项目单元、工件摘要和类型/方法的伪代码 | 由调用方启动、提供 MCP 的 JEB 客户端,通过 REA_JEB_MCP_URL 选择;[JEB 指南](../jeb-analysis.md) |
| Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器;[资源指南](../apktool-resource-analysis.md) |
| Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无(静态 APK 解析);[Flutter 指南](../flutter-analysis.md) |
| 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob;[固件指南](../firmware-analysis.md) |
| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](../packet-captures.md) |
| 录制的逻辑信号 | 解码注释、样本范围和诊断信息 | Linux: sigrok-cli 0.7.2 + prlimit; [信号记录指南](../signal-recordings.md) |
| 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 应用](../apple-application-analysis.md) |
| 进程行为 | 终端输出、交互、退出和文件系统观察结果,以及运行对比 | 支持原生 PTY 的 Linux/macOS;[进程捕获](../process-capture.md) |
静态 JavaScript 和 .NET 检查读取提供的文件,不运行应用。运行时捕获会以你的用户权限运行目标或与之交互;各运行时指南说明了具体影响。
+12 -3
View File
@@ -86,7 +86,7 @@ The inline Evidence result contains the artifact path, SHA-256, byte length,
container format, architecture, word size, and byte order. When a `.dep-v0`
section is present, `auditable_metadata` contains:
- `section_name`, `.dep-v0` for ELF and PE or `__DATA,.dep-v0` for thin Mach-O;
- `section_name`, `.dep-v0` for ELF and PE or `__DATA,.dep-v0` for Mach-O;
- `file_offset` and `section_bytes`, the exact file range of the compressed section;
- `compressed_bytes` and `uncompressed_bytes`;
- `format_version`, the embedded integer, or `null` when the field is omitted;
@@ -124,8 +124,17 @@ verified. An omitted revision remains `null` and follows revision 0 semantics.
Embedded names and versions are file content. They do not authenticate the
packages or the compiler.
Universal Mach-O containers are unsupported. Select a thin slice first. ELF,
PE, and thin Mach-O are the supported containers.
ELF, PE, thin Mach-O and one selected slice of a universal Mach-O are the
supported containers.
A universal (FAT) Mach-O needs `architecture` (`x86`, `x86_64`, `arm`
or `arm64`; CLI `--architecture`). REA reads its own FAT table, without `lipo`,
and validates the selected slice's range, alignment and Mach-O header CPU
against the table. The result keeps the universal file as `artifact` (its
SHA-256 and size), adds `selected_slice` with the slice offset, length, raw CPU
type/subtype and the FAT table digest, and reports the section and rustc-path offsets as a
file offset inside that slice. Without `architecture`, or when the file has no
such slice or several, the request fails on `architecture` and lists the slices
present. `architecture` on any other image is rejected rather than ignored.
Package and dependency counts are checked against the minimum required JSON
representation before graph allocation or schema cloning. Oversized graphs
@@ -0,0 +1,32 @@
# Implementation plan
1. In `environmentStateRules.ts`, parse pointers into tokens and let `at` and
`replaceAt` read and replace element 0 of a one-element array at the root
or in one member; extend `locateItem` with those forms after the object
forms, requiring an identity name there; add `identityCandidates`, the
identity names then suffix-named fields, and let `locateList` prefer arrays
identified by name, then arrays of two or more items identified by suffix.
2. In `EnvironmentStateExtraction.ts`, plan lists with the shared candidates,
treat an empty recording as unconstraining, accept a suffix-named field only
once a recording shows it distinct across two or more items, leave reads
without a list or identity to documents, drop the written-path requirement
and the `not-entity` fallback, record `path-varies` apart from
`query-varies`, and keep each entity's earliest sighting;
`environmentStateMerge.ts` cites records in timeline order. Remove
`not-entity` from the reason type and result schema, add `path-varies`, and
describe the new pointer forms and `field-name` basis.
3. Move the classification of retained bodies (none, base64, cut, JSON, text)
into `replayedBody` in `environmentSource.ts`, which owns recorded
exchanges; replay renders from it and tool descriptions read its JSON case.
4. Make `environmentToolDefinitions` take the `EnvironmentSource`; describe the
first `2xx` JSON record, else the first JSON record labelled with its status,
one level deep with the state's list rule and the existing sample budget;
list the manifest tool's statuses; and charge the whole description and the
tool's MCP annotations to the catalog reservation. Update the serve and
bundle callers.
5. Let the tool definitions own the MCP annotations derived from the method and
the query-`POST` rule; `serveEnvironment.ts` registers them and the card
leaves them out.
6. Cover domain, services, adapters and the compiled MCP boundary; document the
rules in the capture guide, descriptions and hints in the agent environments
guide, and the contract text.
@@ -0,0 +1,99 @@
# Wider environment state coverage and tool descriptions
Issue: https://github.com/morluto/rea/issues/4933
## Outcome
The `environment-state` view backs more read tools with state, and a served
environment tells agents more about each tool. Read-only paths whose responses
are not entities become documents, listed items may be identified by an
`…Id` or `…_id` field, an item wrapped in a one-element array is located
inside it, and reads that vary by path get their own recorded reason. Each tool description says what the tool returns and which statuses
were recorded, and every MCP tool carries method hints. This increment adds no
tool, view kind, provider, network activity or write effect.
## Requirements
- A `GET` list tool (its template does not end in a parameter) that yields no
item or list plan holds its latest initial value as a `document` when its
initial reads have one request line, whether or not the capture writes its
path, unless its value changed between two of those reads across any
recorded write (even under another root), which leaves it `fields-differ`. A
value whose rendered JSON exceeds 256 KiB stays `rendered-oversize` and is
replayed from the recorded body, which itself fit. More request lines give
`path-varies` when they differ by path (a template with a parameter before
its end) and `query-varies` when they share one path. With every such read
now a document, `path-varies`, `query-varies`, `body-unusable` or
`rendered-oversize`, the unreachable `not-entity` reason leaves the result
schema, and `no-identity` names item tools only. A document holds the value
of its one `path`.
- When no path parameter proves an identity and no identity name identifies
the listed items, the `field-name` identity is the first key of the first
listed item, in its order, whose name ends in `Id` or `_id`, that every item
carries as a distinct string or number, and that at least one initial
recording shows distinct across two or more items. The last condition goes
beyond "every item carries it distinctly": one item satisfies that
vacuously, and a lone item's suffix-named fields are often foreign keys (a
reaction keyed by its user, a comment by its post), so a list that only ever
held one item stays a document. Empty recordings constrain no field.
Identity names keep precedence, also when choosing among an envelope's
arrays, where a suffix-named field identifies only an array of two or more
items.
- An item response that is a root array of exactly one object, or an object
with one member holding such an array, locates its entity at `/0` or
`/<member>/0` when one of the identity names in that object repeats the path
value. Objects at the root or in one member keep precedence. `at` and
`replaceAt` read and replace exactly these forms besides the root and one
member, so the gate and renderers run unchanged.
- An entity's state order is its earliest sighting on the timeline and it cites
its records in timeline order, whichever plan observed it first.
- Each tool description reads `METHOD template; <returns>; recorded statuses
…; replays retained records […]; evidence ev_…` after the untrusted-catalog
warning. `<returns>` describes one level deep the first cited `2xx` record
replay answers as JSON: an object's fields, an array of objects' fields, or
an envelope whose member lists objects, where a list is an array of objects
found by the state's list rule; an empty field set reads `with no fields`.
Only when no `2xx` record replays as JSON does the first cited JSON record of
another status give it, labelled `status <code> returns …`; with no JSON
record it is omitted. The statuses are the manifest tool's `statuses`, the
one owner of that set (`no recorded status` when it is empty). Field names
and statuses are sampled within the description sample budget and charged
with the whole description to the catalog reservation before it is retained.
One owner in the environment source classifies each retained body as none,
base64, cut, JSON or text; replay answers from that classification and
descriptions read only its JSON case.
- MCP tools carry `readOnlyHint` (true for the safe methods `GET`, `HEAD`,
`OPTIONS` and `TRACE`, and for query `POST`s),
`destructiveHint: true` for `DELETE` (unset for other writes, so the MCP
default applies, and for reads, where clients ignore it), `idempotentHint`
(true for the safe methods, `PUT` and `DELETE`) and `openWorldHint: false`. The
tool definitions own the hints and charge them to the catalog reservation;
the environment card keeps its `name`, `description`, `inputSchema` tool
shape.
## Acceptance and limits
Synthetic captures cover read-only documents, `path-varies` and
`query-varies`, suffix-named identities with their key order, their
two-item proof, empty first recordings and their precedence (including the
choice between an envelope's arrays), a Ghost-like `{"posts":[...]}` item under
a trailing-slash path merging with its list view, a root one-element array, a
one-element filtered list that stays unidentified, and the pointer forms. The
Grafana-shaped fixture covers documents for its read-only paths (its one-member
team list among them) and object, array and enveloped-list descriptions;
synthetic sources cover the `2xx` preference over an earlier error body, the
first of several JSON records, the labelled fallback, the manifest's statuses,
empty, scalar and fieldless bodies, a truncated member name, a name whose
JSON quoting exceeds the sample, a recorded body under 256 KiB whose rendered
JSON exceeds it, non-JSON bodies and the catalog budget; the in-process and
compiled MCP servers advertise the
hints. A filtered list may still answer with one element whose identity name
repeats the requested value; list filters, pagination, lossless big integers
and cross-endpoint entity linking remain out of scope. Documents inline whole
recorded values and enlarge the state result; agent tokens, CPU time and peak
memory of the state and description build were not measured.
## Deferred issue criteria
Serving from the state, the AgentEnv data plane and write effects remain later
increments of #4933. This PR must reference rather than close the issue.
@@ -0,0 +1,16 @@
# Tasks
- [x] Record the coverage and description contract and its limits.
- [x] Extend pointers, item location and identity candidates.
- [x] Require two distinct items before a suffix-named field identifies a list.
- [x] Hold read-only non-entity reads as documents, retire `not-entity` and
separate `path-varies` from `query-varies`.
- [x] Order entities by earliest sighting and cite records in timeline order.
- [x] Classify retained bodies in one owner and describe tool responses,
preferring `2xx` records, with the manifest's statuses.
- [x] Advertise MCP tool annotations, charged to the catalog, without changing
the card.
- [x] Add domain, services, adapters and compiled-boundary coverage.
- [x] Document the rules, descriptions and hints; update the contract text.
- [x] Run static, formatting, duplication, documentation and test gates and the
real-capture measurement.
@@ -0,0 +1,123 @@
# Verification
Verified base: 7a668b73f4783b42eb0e8ce1eeff079f86d53e22 (the environment-state
view branch).
Executed on the implementation worktree with its existing locked installation
under Node.js v22.23.1 (the repository pins 24.18.0, which was not available);
every check below ran as written in this pass, after the last source change.
- `npm run format` then `npm run format:check`: passed.
- `npm run check:fast`: typecheck and lint passed (3959 files, no errors). The
touched files carry two lint warnings, both present on the base: `max-params`
in `environmentReplay.ts` and an unused `_` in `environmentCapacity.test.ts`.
- `npx knip`: passed (exit 0).
- `npm run jscpd`: 3.40% duplicated lines over 3348 files, under the 3.5%
gate. The three clones that touch changed test files pair the same blocks as
on the base; none involves code this change adds.
- `npm run scan:todos`: no markers.
- `npm run docs:check`: the built documentation checks and the README
consistency conformance test (5 tests) passed.
- `npx vitest run --project domain --project services`: 341 files, 5859 tests
passed, 5 pre-existing skips.
- `npm run build:cached`: passed.
- `npm run test:focused --` the environment serve adapters and capacity tests,
the environment serve MCP boundary, the environment serve and export CLI
boundaries, the environment serve shutdown process boundary, the
analysis-view MCP and CLI boundaries, the compact input-schema contract and
MCP boundary, the tool-schema validity boundary, the analysis-detail and
advertised-input-example contracts and the qlog acceptance test: 14 files,
113 tests passed, 1 pre-existing skip.
Real captures (outside the test suite): the built view ran through the
application service on the same 44 recorded captures (39 HAR, 5 browser
scenario) with all three template rules, in 7 to 50 ms each. 42 produced a
state, and the same two browser-scenario captures are refused with
`resource_constraint` as on the base. The base column is the brief's baseline,
which the same script reproduced on the base build in an earlier pass; this
pass did not rebuild the base.
| Measure | Base | This change |
| ---------------------------------- | ----- | ----------- |
| Types | 568 | 576 |
| Entities | 2,213 | 2,246 |
| Documents | 2 | 299 |
| Item views | 211 | 213 |
| List views | 497 | 504 |
| Document views | 2 | 299 |
| State-backed read tools (of 1,258) | 710 | 1,016 |
| Recorded: `not-entity` | 306 | (retired) |
| Recorded: `no-identity` | 129 | 32 |
| Recorded: `path-varies` | (new) | 32 |
| Recorded: `query-varies` | 5 | 69 |
| Recorded: `body-unusable` | 32 | 32 |
| Recorded: `list-differs` | 31 | 31 |
| Recorded: `no-init-recording` | 26 | 26 |
| Recorded: `query-body` | 12 | 12 |
| Recorded: `fields-differ` | 3 | 4 |
| Recorded: `envelope-varies` | 4 | 4 |
Seven list types gained a suffix-named identity (`permissionId`, `product_id`,
`ItemId`, `user_id`, `channel_id`, `spl_id` and `eventId`). Seven more had one
before the two-item proof and lost it, because every initial response listed a
single item: an Airbyte organization, a Mattermost reaction and team
membership, and a Redpanda broker and schema reference became documents; a
NOAA station read is `query-varies` and a Redpanda consumer read
`path-varies`. Two Ghost item tools render from `/members/0` and `/tags/0`. A
third Ghost item tool is located at `/posts/0` but stays `fields-differ`: its
recordings differ by an `include` query.
State result size, as the JSON length of each Evidence summed over the 42
states: 2,677 KiB on the base and 3,196 KiB now (+19%), measured by the same
script. The largest growth per capture is Metabase (133 to 236 KiB), Directus
(21 to 93), Redpanda Console (26 to 68) and Vault (9 to 43). Documents take
492,516 of the 3,274,243 UTF-8 bytes of the results now; the base held two.
Agent tokens, CPU time and peak memory of the state view were not measured.
Tool descriptions on the 44 captures' manifests: all 44 catalogs were admitted
within their reservation. 1,336 of 1,400 tools state what they return: 1,333
from a `2xx` record and three labelled with another status (Miniflux and
NocoDB 404, Shlink 501), because none of their records succeeded as JSON. The
summaries add 261,502 bytes to a 1,185,893-byte catalog of names, descriptions
and input schemas (median 136, 95th percentile 554, maximum 1,133 bytes per
tool); five reached the field sample bound (one Keycloak, one Lychee and three
Sentry tools with 46 to 105 fields). The MCP annotations add 113,182 bytes to
`tools/list` and none to the card. Building a catalog took a median 1.6 ms and
at most 7.6 ms (Metabase), after the source was loaded. Agent tokens, CPU time
and peak memory of the description build were not measured.
After merging the environment state view's review fixes (its branch at
6ad63bcc, which also brings main's evidence-file sources), every gate ran
again on the merged tree: `npm run check:fast`, `npm run format:check`,
`npx knip`, `npm run scan:todos` and `npm run docs:check` passed; `npm run
jscpd`: 3.39% duplicated lines over 3362 files; `npx vitest run --project
domain --project services`: 344 files, 5914 tests passed, 5 pre-existing
skips; `npm run build:cached`, then `npm run test:focused --` the serve and
tool-definition adapters, the serve MCP and CLI boundaries, the export CLI
boundary, the analysis-view MCP and CLI boundaries, the compact input-schema
budget and the analysis-detail and advertised-input-example contracts: 10
files, 98 tests passed.
The real-capture comparison was repeated on the merged code, with the base
measured on the merged state view without this change: all 44 captures now
produce a state, and 1,011 of 1,277 read tools are state-backed against 625
on the base (216 item, 416 list and 379 document views against 214, 409 and
2). Recorded reasons: `no-identity` 216 to 32 and `not-entity` 310 to none;
`query-varies` 5 to 76 and `path-varies` 36 are document reads under more
than one query or path; `list-differs` 35, `body-unusable` 33,
`no-init-recording` 28, `query-body` 12, `envelope-varies` 4 are unchanged,
and `fields-differ` is 9 to 10.
Not covered here: serving from the state, the AgentEnv data plane and write
effects, which are later increments.
## After the squash onto main
The branch was squashed onto main once the state-serve branch merged (166fb082). Against a merge of the previous head
into that main, the squash keeps both sides of every conflict and drops nothing of this branch. Two tests then failed
on it and were refitted with the oversized-document fix in 2a926a0a: the bundle test of a state catalog over the
discovery budget uses 300 read tools, inside the window where the state's catalog is refused and replay's, with its
response summaries, still fits (280–320 by a probe), and the rea-state validation test names the pushed document by
the count of documents the state holds. The replay-only card test pinned main's 11,131-byte card; it now pins this
branch's 12,818-byte card, whose descriptions name each tool's recorded responses. The branch then merged main at
d7ca93cc without conflicts.
@@ -0,0 +1,45 @@
# Implementation plan
The branch builds on the state-serving and coverage branches (merged into it).
1. **Write plan types and request bodies.** Add the plan, rule and reason types
beside the state rules, and have the state accumulator lend extraction
parsed request bodies under the same budget, admitted like response bodies.
2. **Field rules.** Add a module that explains an observed entity against the
call that wrote it (echo, copy, path, slug, clock, sequence, unique,
template and constant) and that recognises and formats clock values,
UUIDs and tokens.
3. **Effects.** Add one apply module that both the gate and the server use. It
applies a create, update, delete or document write to a writable state
interface from a plan, a call and a value source: the recorded values in
the gate, the store's generators when served.
4. **Gate.** Add a simulation that:
- classifies every write tool;
- builds each plan from its exemplar;
- walks the timeline over a copy of the initial state;
- checks state-answered reads;
- attributes passes and mismatches;
- learns the insert rule;
- records verdicts.
Extraction runs it after the read gate and returns `writes` and
`write_clock`; the view schema and its unknowns state the write model.
5. **Serving.**
- The loaded state carries the plans and the clock.
- The store gains atomic mutations, the clock and the counter (all reset by
`data/reset` and `data/add`).
- A write answer module validates the body, applies the effect and renders
the response.
- Item reads of missing entities answer recorded 404s.
- The served catalog adds served write tools, with descriptions charged to
the discovery budget.
6. **Export.** Count served write tools in the bundle plan and README.
7. **Tests.** Domain, services, adapters and MCP-boundary coverage as the
spec's acceptance lists.
8. **Documentation.** Document the write model in the environment-state and
serving sections, and record the real-capture results, the AgentEnv run
and the live comparison in `verification.md`.
There are no dependency, provider or REA tool contract changes beyond the
`environment-state` result members, and no new network activity.
@@ -0,0 +1,427 @@
# Apply write effects to the served state
Issue: https://github.com/morluto/rea/issues/4933
## Outcome
An environment served from its state also serves the write tools whose effects
the capture itself confirms:
- a create adds an entity;
- an update changes the fields the call sends;
- a delete removes the entity;
- a document write changes a document.
Later reads, list membership and `data/get` show the change, and `data/reset`
undoes it.
The `environment-state` view decides which write tools are served. It replays
the capture's own timeline on the extracted initial state: each recorded write
applies its modelled effect, and each later read that the state answers is
rendered and compared with the recorded response. A write tool is served only
when at least one later read reproduces after its effect and none contradicts
it. Every other write tool stays unserved, listed with the reason.
Values the application generates are deterministic: identities, sequences and
clock values follow the formats the capture recorded, so the same calls after
a reset give the same state.
## Write plans
The view adds two members to its result:
- `writes`: one plan per manifest tool whose method may change state
(`changesState`);
- `write_clock`: the instant the served clock starts from.
**Effects.** Each plan classifies its tool:
| Effect | Tool | Target |
| ---------- | --------------------------------------------------------------------- | ------------------------- |
| `create` | `POST` to a state type's `collection_template` | an entity of that type |
| `update` | `PUT`, `PATCH` or `POST` to that template plus one trailing parameter | the entity the path names |
| `delete` | `DELETE` to that same item template | the entity the path names |
| `document` | `PUT`, `PATCH` or `POST` to a document view's path | that document |
| `none` | anything else | reason `no-state-target` |
**Exemplar.** A plan is built from its first successful (2xx) recorded call,
the exemplar.
- **Request body.** The exemplar's request body must be a JSON object that
could enter the state: a JSON-like media type, complete UTF-8 text within
256 KiB, exact numbers, no `__proto__` member. Otherwise the reason is
`body-unusable`. A `DELETE` takes no body.
- **The written entity.** The entity is observed in the exemplar's response,
located by its identity value at the root, in one member, or in a
one-element array, as item views locate it. When the response does not carry
it, the entity is observed in the first later read that shows it: an item
read of that entity, or for a create, the one entity a collection read newly
lists. A create whose response names its id alone (a scalar, the identity
field, or one member) is observed only through a read of that id. With
neither, the reason is `no-observation`.
**Field rules for creates.** Each field of the observed entity gets the first
rule that explains its value. Rules that read the call come first, so an
identity the client sends is an echo:
1. **Echo.** A body member of the same name with an equal value.
2. **Copy.** A body string of 8 or more characters, found at a JSON pointer of
the body (the shallowest match).
3. **Path.** The value of a path parameter of the call, percent-decoded.
Serving decodes it from the collection the call names, whichever spelling
the call uses, and the gate fills a recorded call's path fields from that
collection too.
4. **Slug.** The slug of a body string member: lower case, runs of other
characters joined by `-`.
5. **Identity.** The type's `id_field` is generated. A sequence when it is an
integer (or a string of digits); otherwise a UUID when it has the
`8-4-4-4-12` hex shape; otherwise a token of the same length and character
classes.
6. **Clock.** A timestamp within 10 minutes of the exemplar's start time. The
format is kept: ISO-8601 with its separator, fraction digits and zone, or
epoch seconds or milliseconds.
7. **Sequence.** An integer one greater than the largest value of the field in
the collection, else in the type.
8. **Unique.** A UUID-shaped string that no other entity of the type shares.
9. **Template.** A string containing a generated value as a delimited token
(for example a URL that ends in the new id). It is rebuilt around the new
value.
10. **Constant.** Otherwise the exemplar's value.
When a constant, or the text a template keeps, contains a path value of the
exemplar's call as a delimited token, the create is served only for calls
naming the exemplar's collection.
**Body members of a create.** A member is either accepted or not:
- **Accepted.** An echoed member, or the identity field's value. Every member
the exemplar sent is required.
- **Not accepted.** A member the application stored with a different value
(transformed), or one it did not show. A call that sends such a member is a
tool error naming the accepted members.
**Updates.** Body members follow the same accepted and not-accepted rules,
except that a member the application stored differently is accepted only with
the value a recorded call sent it with, and then stores the value that call
recorded (its `constant` rule carries it). The type's identity field never
takes a rule: sent as the path's id it is the identity member, sent as another
value the application ignored it is accepted with that value and stores
nothing, and a stored id that changed is `unexplained-change`. A plan whose
calls use a member both as the identity and as a fixed member, or fix it at
different values, is `unexplained-change`, as serving could not take every
recorded call. So is an update a later read shows dropped a field the
entity held, since serving keeps every field an update does not write; a write's
own response may leave fields out. A member a later call is the first to store,
which the plan's observation does not show, keeps the JSON type that call stored
it as (the body's `kinds`), as an exemplar's member keeps the observation's.
`PUT` requires every member the exemplar sent; `PATCH` and `POST` require at
least one. A field the exemplar's update changed without sending it is
explained as one of:
- clock;
- an increment by one;
- a slug of an echoed member.
A field the entity lacked before is a change too when the view that observed
the update had shown the entity, so a field the application fills in on update
follows its rule. Any other change is `unexplained-change`. An increment or a slug applies to
every call that could take it, so the plan is also `unexplained-change` when
such a rule would give a call of the tool another value than the one it
recorded, or when a later call treats a member otherwise than the plan: one
call echoes it and another stores something else, or the two rule it
differently.
**Deletes.** A delete removes the entity. Its response is the exemplar's
response, with the entity re-rendered when the response carries it.
**Documents.** A document write merges the body's members (replaces them, for
`PUT`) into the document object that holds most of them: the root or one
member. Later calls write at the exemplar's object. Document reads answer only on their recorded line, so a write is
served only at the path the state holds the document at, and the gate treats a
recorded write at another path as an unmodelled write. A recorded write whose
body serving would refuse (a member of another JSON type than the document
holds) is `unexplained-change`.
**Responses.** A write answers with the status and content type of one
recorded response:
| Response | Body |
| ---------- | --------------------------------------------------------------------------------------------------------- |
| `entity` | the written entity, rendered into the recorded envelope at the plan's pointer through the recorded fields |
| `identity` | the recorded body with the new identity at the plan's pointer |
| `empty` | no body |
| `recorded` | the recorded body |
A rendered body missing a field the recorded response carried is a tool error,
and the state is left unchanged.
**List position.** A create's `insert` rule places the new entity in its
collection's order: sorted by the identity field or a clock field (ascending
or descending), `first` or `last`. The rule is the first candidate consistent
with both the initial `all`-membership recordings and the position a later
recorded list shows. With no evidence, the rule is `last`.
## The gate
The view replays every primary-origin record in timeline order on a copy of
the initial state.
**Writes.** Only a 2xx write is modelled. A modelled write applies its effect, using the values it recorded
for generated fields and the stored values for fields it transformed. A
modelled write whose call the plan does not explain brings its entity to the
recorded outcome instead, without attribution. So does an update of an unknown
entity (below): its call cannot be checked, so it adds nothing to a built plan.
A later create whose generated
values take another form than serving gives them (a sequence value out of
order, a clock in another format, a non-UUID unique value) makes the plan
`unexplained-change`; integer identities are left to the id mapping. An update or
delete names its entity as an item read does (below). A later update that does
not answer with its entity is observed only by a read whose view shows every
clock the plan learned, since serving stamps them; without one it is unmodelled.
Any other write that may have changed state, including a redirected or
unanswered one, is unmodelled, and what it may have changed
depends on where its path lies relative to each type's collection paths
(segment-prefix related, parameters matching any segment):
- **Below an item path:** the named entity is unknown until a read shows it
again. When the path names no held entity, every entity of that collection
is unknown.
- **At a collection path:** that collection is not checked again.
- **Above collection paths:** everything under the path is not checked again.
The path names its entity as an update's does and its collection as a create's
does, so a plain spelling reaches a collection held under an encoded one; a
path above collections covers each collection whose segments are its own
verbatim or once percent-decoded.
An item read of an unknown entity shows it again: the copy takes its recorded
values. A failed read that is no absence shows nothing, and the entity stays
unknown. Writes that changed it since its last check are checked only on the
members they sent, less those the view treats as volatile, which show nothing
of a write.
**Reads.** A read is checked when the served environment would answer it
from the state:
- an item read of a state-backed item view, of the entity serving would name:
in a held collection the read's path matches, each segment verbatim or once
percent-decoded and verbatim spellings first, the id percent-decoded once,
then as written;
- a list or document read on one of its view's checked request lines,
chosen as replay chooses the line for the read's call: each path parameter
the read's value verbatim or once percent-decoded, preferring verbatim
spellings, then recorded order, and the query values decoded, in any order.
A read with no query on a whole list falls back, as serving does, on a
checked line that only pages it. A line whose query repeats a name with
different values is never chosen, since no call's arguments can name it; one
repeating a name with one value is, as a call naming it matches both. A list
is compared as served from the collection of that line. A read on a page
(`limit`, `size`, `per_page` and the like, or a `_start`–`_end` range; a size
of zero is no bound on a line whose recording listed items under it, while its
other sizes and range still bound the page) of a
whole list checks nothing once the collection holds more entities than the
page, which serving then answers from its recording.
- A read made while a modelled write to its entity or collection was still
running, starting before that write ended, checks nothing: it may show the
state before or after the write. Nor does such a read show an unknown entity
again.
- A read a write's effect was learned from checks nothing of that write, but
still contradicts the other writes it exercises.
**Comparison.** The read's answer is the full response serving would give from
the copy: an item read's entity rendered into its item view's base recording,
or a list read's members as serving would list them (so a member the copy no
longer holds fails the check) rendered into its line's base recording. It is
compared with the recording, as the read gate compares, with volatile fields taking the
observed values. A recorded 400, 404 or 410, or a 2xx with a JSON `null`
body, for an entity the copy does not hold agrees with it. A successful read
whose usable body no longer holds the entity or the list fails, as does one of
an entity lacking a field every recording of its view carried, which serving
refuses to answer with.
**Attribution.**
- A check exercises the write tools that changed its entity or collection
since the previous check of it, less those whose effect later writes erased.
An update's effect is the fields it sent or changed, derived ones included.
A later update overwrites the fields it sets (an increment adds to its field
instead), a recorded outcome every field, and a create or delete the whole
entity. A document write's effect is the members it merged at its
object, or everything under the object it replaces, the whole document at the
root; an entity's fields match by name only, whatever their names look like. A write with nothing left on an entity no
longer counts there, nor in a collection where no entity it changed keeps any
of its effect. A mismatch contradicts them. A pass confirms those it shows
something of: a create's or delete's membership, or a changed field the
read's view shows and does not treat as volatile; a read that shows none of
an update's changes confirms nothing of it, and leaves it for a later check
of that entity or collection.
- A mismatch contradicts each exercised write that no unmodelled write came
after its latest call on any of the read's entities or collections where its
effect survives, whatever earlier write is still pending. For a failed list read the unmodelled write must
also follow the last check and pending writes of each member that differs, so an older change elsewhere
does not excuse a newer write the list shows undone. An item read checks the writes an unmodelled
write did follow only on the members they sent, and a list with such a write
is not checked again: the unmodelled write may explain the rest.
- A mismatch on an entity or collection that no write changed contradicts,
as `side-effect`, the modelled writes since that previous check whose paths
are segment-prefix related to the read's path.
- After a failed item check, the copy takes the read's recorded values (or
drops the entity on a recorded absence). A list whose check failed is not
checked again.
- A list that fails only for where one create placed its entity passes when
another of that create's insert candidates reproduces it, which then
becomes the plan's rule, unless an earlier list check relied on the rule
(an item read's confirmation does not fix it).
- The read that supplied a create's or update's observation does not check
it, but checks the other writes it exercises.
**Verdicts.** A plan is served when at least one check confirms it and none
contradicts it. Otherwise the reason, in this precedence, is:
1. the first build reason;
2. `contradicted`, with `contradicted_by` naming the first contradicting read;
3. `side-effect`;
4. `unconfirmed`;
5. `incomplete-entities`: an update or delete that answers with the entity,
when no initial entity holds every field its recorded answer shows (for
example a delete that answers with an edit view of the entity the reads
never showed), so it could answer only for entities the environment itself
created. An update's answer renders the entity it wrote, so a field one of
its rules writes counts as held, except an increment's, which adds only to
a held value.
`checked_record_ordinals` lists the confirming reads.
## Serving
**Catalog.** In state mode, the catalog adds the write tools whose plan is
served. Their descriptions say the tool:
- applies its effect to the served state;
- names its target type;
- lists its required and accepted body members, quoted and sampled within the
description budget as response fields are.
The discovery budget charges them. `--replay-only` and the replay fallback are
unchanged.
**Calls.** A served write answers synchronously against the live store and
changes it atomically, so every MCP session and `data/get` see one state.
**Refusals.** These are tool errors that leave the state unchanged:
- a body that is not an object;
- a missing required member;
- a member that is not accepted;
- a value of another JSON type than the exemplar's (for a member only a later
update stored, that call's), for documents as well as entities;
- an update or delete of an entity the state does not hold;
- a create in another collection when the plan is tied to its own;
- a document write at a path other than the one the state holds it at;
- a body member or a resulting value `data/add` would refuse (a number not
exact as `data/get` writes it, a `__proto__` member, nesting past the
limit), increments included, so `data/get` always loads back;
- a create, update or document write that would take what `data/get` reports
past the 32 MiB `data/add` intake limit, as a `resource_constraint` error;
- a rendered response missing a recorded field.
When the tool, or else an item view of the type, recorded a 400, 404 or 410
response, an update or delete of a missing entity answers with the latest such
recording instead.
A create in a collection held under more than one spelling (`acme1` and
`%61cme1`) goes to the one the call spells verbatim, as item calls choose their
entity; a collection keeps its spelling after its last entity is deleted. A
reset or load knows the spellings of the Evidence's state, of the state it
installs and of the type's checked list lines (so a collection the capture
listed empty keeps its spelling), never those of a state it replaced. When two
spellings fit a call equally, the one holding entities wins. `data/add` refuses a document that `data/get` would write back past the
32 MiB intake limit once it adds the members a document may omit, such as
`types` and the state Evidence id. A derived state `data/get` would write
past that limit is not served: the environment serves replay and reports the
state `unavailable`.
**Clock and generators.** The store's clock starts at `write_clock` and
advances one second before each served write; clock fields take its value in
their recorded format.
- **Sequences:** one more than the largest current value in their scope, the
largest the state started with at its last reset or load, and the largest
value they generated, so a deleted entity's value is not reused, as database
sequences do not reuse values.
- **UUIDs and tokens:** derived from a SHA-256 of the state Evidence id, the
tool, the field and a write counter, skipping values the type already holds.
`data/reset` and `data/add` restart the clock, the counter and the
sequences.
**Missing entities.** An item read of an entity the state does not hold
answers with the tool's latest recorded 400, 404 or 410 response, or 2xx JSON
`null` response, the absences the gate accepts, when it has one, labelled
`state.view: "missing"`. Otherwise it stays a tool error.
**Lists of a whole collection.** Written entities matter only if lists show
them, so the read gate's `membership: "all"` also covers a line that only pages
its list (`limit`, `offset`, `page`, `per_page`, `perpage`, `page_size`,
`pagesize`, `size`, `start`, `_start`, `_end`) when its recording listed the
whole collection. A list call with no query argument answers from such a line
of an `all` view of the collection its path arguments name, with
`replay.exact_match` false; a collection with no such line answers from its
recording. A call that asks for a page
still answers from its recording.
**Provenance.** Write answers carry `state: {source: "state", view: "create" |
"update" | "delete", type, collection, id}`, or `{source: "state", view:
"document-write", path}`.
**Data plane and export.**
- `data/get` reports written entities without record citations; they load
back through `data/add` unchanged.
- `environment-export` counts served write tools in the bundle's tool count
and README. Its check stays a read tool.
## Not modelled
These stay unserved, or are labelled in the description:
- side effects on counters, relations and copies of entities in other
collections;
- inputs the application transforms;
- members the capture never sent;
- reordering after an update;
- membership of filtered or paged lists, which answer their recorded members;
- write paths the manifest left literal;
- writes whose request body the capture did not retain;
- referential rules on delete: a delete removes the entity alone, so an
application that refuses to delete a referenced entity, or deletes its
dependents with it, is not modelled;
- authorization.
## Acceptance
- **Domain tests:** classification, every field rule, request-body admission,
observation from a later read, the insert rules, every verdict and reason,
attribution and resynchronisation, and taint by an unmodelled write.
- **Services tests:**
- store mutations and their atomicity;
- every refusal;
- generated values and their determinism across reset;
- `data/get` round trips of written entities;
- the recorded 404 for missing entities;
- the catalog and descriptions in each mode.
- **Boundary test:** a write tool changes the next read over MCP.
**Real captures.**
- The view runs over every showcase capture within its budget, and the
verification reports, per capture, the write tools served and the reasons
for the rest.
- One environment, chosen at random among those serving a write tool, is
exported, run in the AgentEnv framework, and graded from `data/get` after an
agent's create, update and delete.
- The served model is compared with that application running live on the same
calls.
No test reaches an external network.
@@ -0,0 +1,22 @@
# Tasks
- [x] Record the write model, the gate and the serving contract.
- [x] Admit request bodies and add the plan types.
- [x] Explain created and updated entities with field rules.
- [x] Apply create, update, delete and document effects in one shared module.
- [x] Simulate the capture's timeline and decide each plan's verdict.
- [x] Leave entities below an unmodelled write unknown until a read shows them
again, and check writes such a write follows only on the members they
sent.
- [x] Serve gated write tools from the live store with deterministic values.
- [x] Answer recorded 400, 404 and 410 responses for missing entities.
- [x] Let lines that only page a whole collection back `membership: "all"`,
and answer argument-less list calls from them.
- [x] Count served write tools in exports and describe them in bundle READMEs.
- [x] Add domain, services, adapters and boundary coverage.
- [x] Document the write model and serving.
- [x] Run the view over every showcase capture and report per-capture verdicts.
- [x] Export one randomly chosen environment, run write tasks in the AgentEnv
framework and grade them from `data/get`.
- [x] Compare the served model with the live application on the same calls.
- [x] Run static, formatting, duplication, documentation and test gates.
@@ -0,0 +1,376 @@
# Verification
Verified base: cbb7b613 (this branch before this file). Executed on macOS with
Node v22.23.1 (the repository pins 24.18.0 in `.nvmrc`, which was not
installed); the AgentEnv runs and the live comparison ran on a disposable Linux
VM. Every check below ran as written after the last source change.
## Gates
- `npm run build:cached`, `npm run check:fast` (typecheck and lint),
`npm run format:check`, `npx knip`, `npm run scan:todos`, `npm run docs:check`
and `npm run verify:test-discovery`: passed.
- `npm run jscpd`: 3.43% duplicated lines over 3538 files, under the 3.5% gate.
- `npx vitest run --project domain --project services`: 7407 passed, 5
pre-existing skips.
- `npm run test:prepare`, then `npm run test:focused --` the new write MCP
boundary, the serve MCP, CLI and process boundaries, the export CLI
boundary, the analysis-view CLI and MCP boundaries, the compact input schema
boundary and the contract description tests: 10 files, 72 passed.
- `npm run mcp-catalog:check`: passed.
New tests: `environmentStateWriteValues` (94), `environmentStateWriteExplain`
(101), `environmentStateWriteApply` (52), `environmentStateWrites` (16, the gate
through the view), `environmentStateWriteAnswer` (14, serving) and
`environmentServeWritesMcp` (1, the built CLI over MCP). The replay card pinned
by the serve MCP boundary changed with the response summaries tool descriptions
carry; its digest now matches the combined branch.
## Real captures
The `environment-state` view ran over the 50 showcase captures (45 HAR
inspections and 5 `capture_browser_scenario` captures), each within its budget in
8–110 ms. It served 33 write tools in 16 captures (11 creates, 12 updates, 10
deletes). Of the other 150 state-changing tools, 102 name no state type or
document (most write to literal single-entity paths the manifest left
untemplated, or to collections whose reads are not state-backed), and 48 are
modelled but not served:
- 15 unconfirmed (no later read checked them);
- 13 no-observation;
- 8 contradicted, for example a deleted Airbyte source and EspoCRM task still
readable, Immich and BookStack item reads showing fields the create
response lacked, and Sonarr's tag list ordering;
- 7 unexplained-change, such as BookStack's description_html and Paperless's
derived text colour;
- 4 side-effect, for example a Sonarr tag also listed by `/tag/detail`;
- 1 incomplete-entities: WordPress answers a delete with the post in its edit
context, which the reads never showed for seeded posts.
| Capture | Served (create / update / delete) | Not served |
| ---------------- | --------------------------------- | ----------------------------------------------------------------------------------------------- |
| airbyte | 2 / 2 / 0 | create contradicted, delete contradicted, delete unconfirmed |
| baserow | none | no-state-target ×3 |
| bookstack | 0 / 0 / 1 | create contradicted, create side-effect, update unexplained-change |
| directus | none | no-state-target ×2 |
| espocrm | none | create unconfirmed, delete contradicted, update unexplained-change |
| ghost | none | no-state-target ×2 |
| gitea | 1 / 1 / 0 | no-state-target ×2 |
| grafana | none | create unconfirmed, no-state-target ×5 |
| grist | none | create no-observation, no-state-target ×5 |
| grocy | none | create no-observation, delete no-observation, update no-observation, no-state-target ×2 |
| healthchecks | none | create no-observation, delete no-observation, update no-observation, no-state-target ×2 |
| homebox | none | document no-observation, no-state-target ×3 |
| immich | 0 / 0 / 1 | create contradicted, update contradicted |
| jellyfin | none | no-state-target ×2 |
| keycloak | none | no-state-target ×1 |
| komga | none | create unconfirmed ×2, no-state-target ×4 |
| linkding | 2 / 2 / 2 | create unconfirmed, delete unconfirmed, update unconfirmed, no-state-target ×2 |
| listmonk | 1 / 1 / 1 | — |
| lychee | none | no-state-target ×4 |
| mailpit | none | document no-observation, no-state-target ×5 |
| mattermost | 1 / 1 / 1 | no-state-target ×3 |
| mealie | none | create no-observation, no-state-target ×2 |
| memos | none | no-state-target ×3 |
| metabase | 0 / 0 / 1 | create side-effect, update unexplained-change |
| miniflux | 0 / 1 / 0 | no-state-target ×3 |
| n8n | none | create unconfirmed, no-state-target ×6 |
| navidrome | none | create no-observation, no-state-target ×2 |
| netbox | 1 / 1 / 1 | — |
| nocodb | none | create no-observation, no-state-target ×4 |
| nomad | none | no-state-target ×4 |
| paperless | 0 / 0 / 1 | create unconfirmed, update unexplained-change, no-state-target ×3 |
| photoprism | 1 / 1 / 0 | delete unconfirmed |
| pinry | none | create unconfirmed, delete unconfirmed, update unexplained-change |
| pocketbase | none | no-state-target ×4 |
| redpanda-console | none | document no-observation, no-state-target ×5 |
| sentry | 1 / 0 / 0 | update unexplained-change, no-state-target ×3 |
| shlink | none | create unconfirmed, no-state-target ×1 |
| sonarr | 0 / 1 / 0 | create contradicted, create side-effect, update contradicted |
| tandoor | none | no-state-target ×4 |
| umami | 1 / 1 / 1 | — |
| vault | none | no-state-target ×7 |
| vikunja | none | no-state-target ×4 |
| wordpress | none | create side-effect, delete incomplete-entities, document unconfirmed, update unexplained-change |
An in-process smoke derived each capture's served state and called every served
write tool once with a varied body: all 33 applied (status equal to the
recording), and every created or updated entity read back from the state;
deleted entities answered the recorded 400 or 404 where one was recorded. Eight
older published manifests no longer match the tool names the current view
derives, so their state is unavailable and they serve replay, as designed;
none of them has a served write tool.
## End to end on a published environment
Five published environments serve a create, an update and a delete (Linkding,
Listmonk, Mattermost, NetBox, Umami); `random.SystemRandom().choice` picked
NetBox (NetBoxEnv-REA, NetBox 4.7.3).
- `rea environment-export` (packed from this branch) wrote the bundle; its
smoke task passed in AgentEnv framework 0.9.1334 with 42 served tools (39
reads and `post_dcim_sites`, `patch_dcim_sites_by_id`,
`delete_dcim_sites_by_id`).
- Four write tasks, each graded by an `env_outcome_verifier` that reads the
environment's state through `/agentenv` `data/get` (not the agent's reply),
with the replay-qa agent over the Hugging Face router, one run per task and
model:
| Task | gpt-oss-120b | GLM-5.3-Flash | Tool calls (gpt-oss) |
| ----------------------------------------------- | ------------ | ------------- | ------------------------------------------------------------ |
| Create a site in the Copper Valley region | 1.0 | 1.0 | get_dcim_regions, post_dcim_sites |
| Update a seeded site's facility and description | 1.0 | 1.0 | get_dcim_sites, patch_dcim_sites_by_id |
| Delete one seeded site and only it | 1.0 | 1.0 | get_dcim_sites, delete_dcim_sites_by_id |
| Create a site, then set its facility | 1.0 | 1.0 | get_dcim_regions, post_dcim_sites, patch_dcim_sites_by_id ×2 |
- The bundle's 12 retrieval tasks (one per seeded object) passed 12/12 for both
models, as they did on the published replay bundle.
## The served model against the live application
The same NetBox 4.7.3 image ran on the VM, seeded by the capture's own seed
script (the same ids), and one scenario ran against both the live API and the
environment served from the capture. Only `created` and `last_updated` were
normalised; every other member had to be equal:
| Step | Live | Environment | Equal |
| ----------------------------------------- | ---- | ----------- | ---------------------------- |
| List sites | 200 | 200 | yes |
| Create Tide Gauge Annex (id 5) | 201 | 201 | yes |
| Read it | 200 | 200 | yes |
| List after the create (sorted by name) | 200 | 200 | yes |
| Patch its description and facility | 200 | 200 | yes |
| Read after the patch | 200 | 200 | yes |
| Create Granite Spur (id 6) | 201 | 201 | yes |
| Patch a seeded site | 200 | 200 | yes |
| Delete Tide Gauge Annex | 204 | 204 | yes |
| Read the deleted site | 404 | 404 | yes |
| Create Ledger Point (id 7, not reusing 5) | 201 | 201 | yes |
| Delete a seeded site with devices | 409 | 204 | no |
| Final list | 200 | 200 | no (follows from the delete) |
11 of 13 steps were equal. The difference is NetBox's referential protection:
it refuses to delete a site that devices reference, while the environment's
delete removes the entity alone. The capture only deleted a scratch site with
no dependents, so the gate could not see the rule; it is listed under what is
not modelled.
## Not covered
- A run of the hosted `real-agentenv` lane in CI (the AgentEnv runs here used
the published framework release on a VM).
- Document writes on a real capture: the four modelled ones are unserved.
- Collections tied to a create's own collection on a real capture, and UUID
or token identities through an AgentEnv run (both are covered by tests).
- Agent token usage and cost.
## After merging the base branches and main
The branch then merged the serving branch at 5a61853a, the coverage branch at
b4b7ebc7 (their review fixes) and main at edbd291a, then main at 17f581ff
(the analysis-view tool description string, which main changes often). The gates above ran again
and passed: `npm run jscpd` 3.44% over 3579 files; domain and services 7442
passed with 5 pre-existing skips; the focused boundary set 72 passed. The
write verdicts over the 50 captures were unchanged plan by plan, and the
in-process smoke again applied all 33 served write tools. The AgentEnv runs
and the live comparison above ran on cbb7b613, before these merges.
After the second main merge: `npm run check:fast`, `npm run format:check`,
`npm run docs:check` and `npm run mcp-catalog:check` passed; the contracts,
analysis-view and environment tests 857 passed with 3 pre-existing skips; the
analysis-view and environment MCP and CLI boundaries 53 passed.
## After the review round
The branch then merged the serving branch at c00fde41, the coverage branch at
f6c756aa and main at 1629266e, and took the review fixes (82556cd5, 138ea4b1,
6d21b62e). The coverage branch quotes field names in response summaries, so the
replay card pinned by the serve MCP boundary is now 12,818 bytes; removing
those quotes gives back the previous 12,618-byte digest exactly. The gates ran
again and passed: `npm run check:fast`, `npm run format:check`, `npx knip`,
`npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery`,
`npm run verify:module-boundaries` and `npm run mcp-catalog:check`;
`npm run jscpd` 3.47% over 3601 files; domain and services 7463 passed with 5
pre-existing skips; the focused boundary set 75 passed; conformance 279
passed. The gate is stricter, but the write verdicts over the 50 captures were
unchanged plan by plan, and the in-process smoke again applied all 33 served
write tools. The AgentEnv runs and the live comparison above ran on cbb7b613,
before these merges and fixes.
A second review round then made the gate and serving agree in five more places
(reads matched to checked lines as serving matches calls, increments and slugs
held for every call, insert rules still learned after an item read, the
document body check in the gate, and decoded parent path values). Each new test
fails on the previous head and passes here. The same gates passed again
(domain and services 7468 passed with 5 pre-existing skips, the focused
boundary set 75, conformance 279, `jscpd` 3.47%), the write verdicts over the 50
captures were again unchanged plan by plan, and the smoke applied all 33 served
write tools.
A third round kept a checked line's path spelling (replay prefers a verbatim
one, so two spellings of one parent stay separate lines) and compared a later
slug rule with the values earlier calls recorded rather than only their member
names; both new tests fail on the previous head. A fourth round chose a read's
line as replay does, so a read spelled plainly is checked against the encoded
line that answers it, and its list against that line's collection; its test
fails on the previous head.
A review of the fidelity branch built on this one found four more places: a
document write at another path is now unmodelled in the gate, a create picks
the collection the call spells verbatim, a missing entity is answered with a
recorded 2xx `null` the gate accepted as an absence, and `data/add` refuses a
document whose `data/get` form would exceed the intake limit. Each new test
fails on the previous head. The gates passed again (domain and services 7475
with 5 pre-existing skips, boundaries 75, conformance, `jscpd` 3.46%), the
verdicts over the 50 captures were unchanged plan by plan, and the smoke
applied all 33 served write tools.
A sixth round, after merging the serving and coverage branches and main
335c042d, settled five more findings: a read whose usable body no longer holds
the entity or list fails its check, an entity lacking a field its view always
carried confirms nothing, a write a later write replaced is not confirmed by
reads of the replacement, sequences never reuse a value the state started
with, and a collection keeps its spelling after it empties. Each new test fails
on the previous head. Over the 50 captures one capture's verdicts change:
Linkding's bundle create and patch, confirmed only by a list read after the
same bundle was deleted, are now unconfirmed, so 31 write tools are served (10
creates, 11 updates, 10 deletes) and the in-process smoke applies all 31. The
gates passed again (domain and services, boundaries, conformance and `jscpd`).
A seventh round settled two more findings. An update keeps its confirmation
when a later write overwrites only part of what it changed: a derived change,
such as an incremented version, counts as part of the update's effect, and a
later increment adds to the field rather than replacing it. A reset or load
forgets the collection spellings of the state it replaced. Both new tests fail
on the previous head. The verdicts over the 50 captures are unchanged plan by
plan, the in-process smoke applies all 31 served write tools, and the gates
passed again.
An eighth round settled three more. A later write erases the fields of an
earlier effect it overwrites, so two writes that together overwrite all of an
update's changes leave it nothing to be confirmed by. A recorded outcome erases
what earlier writes changed on its entity. A load keeps the Evidence's own
collection spellings, so a create in a collection the loaded state does not
hold keeps its recorded spelling. Each new test fails on the previous head. The
verdicts over the 50 captures are unchanged plan by plan, the smoke applies all
31 served write tools, and the gates passed again.
A ninth round made a create prefer a collection the state holds over an equally
fitting spelling known only from the Evidence, so a create after a load lands in
the loaded collection and reads back. Its test fails on the previous head; the
verdicts and the smoke are unchanged and the gates passed again.
A tenth round settled six findings Greptile raised on the list-lines branch
against this branch's code:
- a read with no query on a whole list is checked on the paging line serving
answers it from, and a line whose query repeats a name is never checked;
- a read made while a write to its targets was still running checks nothing;
- a later call that stores a member otherwise than the plan refuses the plan;
- a create keeps the spelling of a collection the capture listed empty;
- a wide array no longer overflows the stack while checking path-dependent
values;
- the guide counts served write tools among the catalog's.
Each new test fails on the previous head. The verdicts over the 50 captures
are unchanged plan by plan, the smoke applies all 31 served write tools, and
the gates passed again.
An eleventh round settled six more findings, five raised on this branch and one on the fidelity branch:
- a passing read confirms only the writes it shows something of;
- an item read names its entity as serving does, percent-decoded spellings included;
- a read a write's effect was learned from still contradicts the other writes;
- a line repeating a query name with one value stays checkable;
- document writes join the effect tracking, so a replacement overwrites an earlier merge;
- a call naming a page that the collection has outgrown answers from its recording (`page-exceeded`), and the gate does not check such reads.
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
applies all 31 served write tools, and the gates passed again.
A twelfth round settled four findings Greptile raised on the list-lines branch against this branch's code:
- a create that answers with its id alone is observed only through a read of that id;
- a later call that is the first to send the path's id in its body adds the identity member to the plan;
- the copy search walks a request body with an index, so a wide body no longer takes quadratic time (500,000 zeros: 63 s
before, 74 ms after);
- the sent values of fixed members a plan keeps are counted against the view's allowance.
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
applies all 31 served write tools, and the gates passed again.
A thirteenth round settled seven findings Greptile raised on this branch, the fidelity branch and the list-lines branch:
- a passing read that shows nothing of a write leaves it for a later check, instead of clearing it;
- a read begun while a write to an unknown entity still ran neither checks nor shows the entity again;
- a later create whose sequence fields do not follow the learned sequence refuses the plan;
- a document replacement below the root overwrites only what it replaced;
- a built document plan writes later calls at its own object, even one sharing no member with the document;
- an update or delete finds its entity as an item read does, percent-decoded spellings included;
- a `_start`–`_end` range bounds a page as `limit` does, in serving and in the gate.
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
applies all 31 served write tools, and the gates passed again.
A fourteenth round settled two findings Greptile raised on the list-queries branch against this branch's code:
- the type's identity field never becomes a constant: an update whose id member the application ignored no longer
writes the exemplar's id into the entity another call names;
- a page size of zero bounds nothing, so a whole line recorded with `limit=0` answers from the state.
Both tests fail on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke applies
all 31 served write tools, and the gates passed again.
A fifteenth round settled nine findings Greptile raised on this branch, the fidelity branch and the list-lines branch,
five of them on the previous rounds' own changes:
- an entity field whose name looks like a pointer is matched by name; only a document's pointers cover the members
under them;
- a failed read contradicts each exercised write no unmodelled write came after, rather than skipping all of them
when the oldest pending write came before one;
- an item read, update or delete names its entity as serving does, decoding the id once, so `book%2531` no longer
reaches `book1`;
- a plan whose calls use a member both as the identity and as a fixed member, or fix it at different values, is
refused;
- a size of zero bounds nothing only on a line whose recording listed items under it;
- an update whose observation drops a field the entity held is refused;
- a later create whose clock takes another format, or whose unique value is not a UUID, is refused;
- a derived state `data/get` would write past 32 MiB is not served.
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
applies all 31 served write tools, and the gates passed again.
A sixteenth round settled four findings Greptile raised on the fifteenth round's changes, here and on the fidelity,
list-lines and list-queries branches:
- a size of zero is no bound only by itself: a range or another size on the same line still bounds the page;
- an update is refused for a dropped field only when a later read shows it dropped, since a write's own response may
leave fields out;
- interference is weighed against a tool's latest call on the read's entity or collection, not its latest call
anywhere.
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
applies all 31 served write tools, and the gates passed again.
A seventeenth round settled two more findings Greptile raised on the list-queries branch against this branch's
interference rule: an unmodelled write counts against a tool when it followed the tool's call on any target the read
covers, so a later call on another entity does not hide it; and a failed item read contradicts the writes no
unmodelled write followed, checking only the others on their members. Both tests fail on the previous head. The
verdicts over the 50 captures are unchanged plan by plan, the smoke applies all 31 served write tools, and the gates
passed again. A follow-up weighs interference only on targets where the tool's effect survives, so a call
whose effect a later write overwrote no longer excuses a later mismatch; its test fails on the previous head.
## Collection-history accounting check
On 2026-10-22, Node 24.18.0, I compared the source before the write-history
change (`cfb42ddde`) with the bounded-history implementation. Each operation
inserted an entity at a new collection path with a 128-character repeated
suffix and immediately deleted it. Across five fresh Vitest worker processes,
the median for 1,000 completed cycles was 14.57 ms wall / 23,867 us CPU before
and 9.98 ms / 16,914 us after. The process RSS samples were 381.1–387.9 MiB
before and 380.9–387.6 MiB after. This fixture benchmark measures one matched
completed trace, not a representative user task or process-family peak memory.
The values include Vitest and V8 and are noisy; they do not establish a memory
bound. The 1 MiB policy is a deterministic accounting allowance for
additional identifier history, set to 1/32 of the separate 32 MiB state-intake
ceiling; it is not derived from RSS samples and does not bound total store or
process memory.
+16 -2
View File
@@ -65,6 +65,18 @@ API dependency, and independent fixture compiler oracles keep their existing
TypeScript versions.
`npm run build:cached` compiles the CLI/MCP runtime and its packaged skill.
The build also records `dist/mcp-catalog.json` (compact inputs) and
`dist/mcp-catalog-full.json` from the canonical SDK registrations and binds
each to the runtime with its own SHA-256 digest. These are
runtime artifacts, separate from the test catalog below. Default target-free
MCP startup serves its schemas without constructing analysis validators;
the first tool or guided-workflow request materializes the canonical runtime
on that same SDK server. Explicit compact/full input selections preserve
their advertised schemas throughout materialization. Initial-target,
HTTP and output-schema-off startup keep the eager path; source runs without
recorded artifacts also use eager composition. Compare memory both after discovery and after a completed
request; report diagnostic GC separately from normal resident memory.
Turbo's strict task environment preserves caller-selected `GOMAXPROCS`,
`GOMEMLIMIT`, and `UV_THREADPOOL_SIZE` for resource-constrained builds and
checks. These execution controls do not change artifact content, so changing
@@ -1231,7 +1243,9 @@ host behavior or native process-supervision controls.
real `.nib` with Xcode `ibtool`, wraps it in a temporary app bundle, and runs
the compiled CLI and stdio MCP server with production providers. It compares
their decoded results and checks the view hierarchy, outlet, action, evidence coverage, and truncation
status. Storyboard compilation additionally requires an installed iOS platform.
status. It also recompiles the XIB with toggled control booleans and requires
both decodes to match `ibtool --export` for each source. Storyboard compilation
additionally requires an installed iOS platform.
Keep the provider-specific acceptance path independent from optional
cross-compilers. Cross-format failures belong to the cross-format lane and must
@@ -1806,7 +1820,7 @@ the built CLI, packs the checkout with `npm pack`, exports the bundle with
registry package, and runs `agent-env run` on the bundle. It checks that the
export reports the planned file set with the inputs copied byte-exact and a
bundle served from the derived state, which serves fewer tools than the
manifest because write tools are not served, that the runner exits `0` with
manifest because write tools the capture did not confirm are not served, that the runner exits `0` with
`tasks/smoke.json` reported as passed and the verifier's description naming
that served tool count, and that no container exists
after the run that did not exist before; the container inventory runs even when
+2
View File
@@ -116,3 +116,5 @@ Then rerun this installer.",implemented,verified by shell branch inspection
90,install.sh:version mismatch,Installed rea reports a different version,installed version ACTUAL does not match EXPECTED.,Low risk; gives exact integrity mismatch,Keep current copy,unchanged,verified by exact installer test
90.5,@modelcontextprotocol/server:tool input validation,Caller sends arguments that violate a declared tool schema,"Canonical invalid_request with the SDK diagnostic plus recovery.kind=inspect_input_contract and a bounded instruction to read the advertised schema or call tools/list",Keeps field/type correction while telling the agent how to recover; diagnostics over 4 KiB stay on the existing retained-error path,Normalize ordinary validation failures and retain oversized diagnostics,verified by real in-memory and stdio MCP client/server calls with wrong argument types
90.6,incur:unknown CLI command,Caller enters an unknown REA command,"'COMMAND' is not a command for 'rea'. Suggested command: rea --help",Low risk; identifies input and exact discovery action,Keep framework-generated copy,unchanged,verified by real packaged dispatcher invocation
91,src/application/binaryDiagnostics/ByteEntropyService.ts,Invalid file or window,"Expected a regular file.; Selected file could not be found.; Selected byte window extends beyond the end of the file.; File size exceeds exact numeric offset representation.",Invalid selection cannot produce complete entropy Evidence,Choose a readable regular file and a window within its byte size,implemented,entropy service and MCP boundary tests
92,src/domain/analysisView/byteEntropyView.ts,Non-summary entropy view,Byte entropy Evidence supports the summary view; retrieve parent Evidence for the complete block series.,Unsupported view selection,Use the summary view or retrieve complete parent Evidence,implemented,typed input rejection
1 harm_rank location trigger current_copy user_risk proposed_replacement implementation_status verification_result
116
117
118
119
120
+91 -27
View File
@@ -522,9 +522,9 @@ Pass the same parent to `inspect_analysis_view` with
`view: {"kind":"environment-state"}` to extract the initial state the
capture's reads observed, and a render plan per read tool that reproduces those
reads from it. Where the environment manifest names an application's tools, the
state holds its data: entities by type and identity, plus documents for written
paths whose reads are not entities. Use the `template_rules` of the manifest the
state accompanies so tool names match; the view groups records the same way.
state holds its data: entities by type and identity, plus documents for reads
that are not entities. Use the `template_rules` of the manifest the state
accompanies so tool names match; the view groups records the same way.
```json
{
@@ -560,13 +560,27 @@ refused for them, as the manifest is not.
Each `GET` tool whose template ends in a parameter is an item tool, and its
type's `collection_template` is the template without that parameter; any other
`GET` tool is a list tool of its own template. An item tool proves its type's
`GET` tool is a list tool of its own template. Empty segments are ignored, so a
trailing slash names the same collection. An item tool proves its type's
`id_field` when every initial response carries the requested path value in the
same field, at the root or in one wrapper member (`id_basis: "path-parameter"`).
Otherwise the field is the first of `id`, `uid`, `uuid`, `_id`, `ID`, `UID`,
`Id`, `guid`, `key`, `slug` and `name` that every listed item carries as a
distinct string or number (`id_basis: "field-name"`). A list is the response
array, or the one array of objects in an envelope that is not itself an entity.
same field of one entity object (`id_basis: "path-parameter"`). The object is
the response itself or one wrapper member; failing both, it is the only element
of a one-element array at the root (pointer `/0`) or in one member
(`/<member>/0`), as in `{"posts":[{...}]}` or `{"data":[{...}]}`, provided one
of the identity names below repeats the value, since a filtered list may also
answer with one element that repeats its parent's value. Otherwise the field is
the first of `id`, `uid`, `uuid`, `_id`, `ID`, `UID`, `Id`, `guid`, `key`,
`slug` and `name` that every listed item carries as a distinct string or
number; when none does, it is the first key of the first listed item, in its
order, whose name ends in `Id` or `_id` (such as `orgId` or `user_id`), that
every item carries as a distinct string or number, and that at least one
initial response shows distinct across two or more items
(`id_basis: "field-name"`). A single listed item is distinct from nothing, and
its suffix-named fields are often foreign keys, so a list that only ever held
one item is left to a document. Empty initial lists constrain no field. A list
is the response array, or the one array of objects in an envelope that is not
itself an entity, preferring an array whose items an identity name identifies,
then an array of two or more items a suffix-named field identifies.
`entities` merge every initial observation of one identity within one
collection path: an item view's latest values win over a list view's, a view
@@ -581,12 +595,12 @@ Every entity cites its `record_ordinals`.
`views` lists one plan per read tool in tool order:
| Shape | Plan |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `item` | The entity object at `pointer` (the root or one member), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares |
| `list` | The array at `pointer`, its `fields`, `membership` (`all`: every entity of the collection in state order, when each recording listed exactly that, had no query, no item-total member (`total`, `count`, `totalCount`, `item_count`, `totalcount`, `itemcount`, but not `total_pages` or `per_page`; a number or a string of digits) differing from the number listed, and no `has_more`, `next` or next-page token member set; `recorded`: the entities each recording listed) and `counters` that equal the number listed |
| `document` | The path's latest initial value, held in `documents` |
| `recorded` | A `reason` the tool replays its recordings instead |
| Shape | Plan |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `item` | The entity object at `pointer` (the root, one member, or `/0` or `/<member>/0`), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares |
| `list` | The array at `pointer`, its `fields`, `membership` (`all`: every entity of the collection in state order, when each recording listed exactly that, had no query or only paging parameters (`limit`, `offset`, `page`, `per_page`, `perpage`, `page_size`, `pagesize`, `size`, `start`, `_start`, `_end`), no item-total member (`total`, `count`, `totalCount`, `item_count`, `totalcount`, `itemcount`, but not `total_pages` or `per_page`; a number or a string of digits) differing from the number listed, and no `has_more`, `next` or next-page token member set; `recorded`: the entities each recording listed) and `counters` that equal the number listed |
| `document` | The latest initial value of a list tool whose responses hold no identified list, when its initial reads share one request line; held in `documents` with that `path`, whether or not the capture writes it, unless its value changed across any recorded write between those reads (`fields-differ`) |
| `recorded` | A `reason` the tool replays its recordings instead |
An item or list plan is kept only when it reproduces the latest initial
recording of each request line from the state, comparing JSON values without
@@ -594,38 +608,88 @@ regard to member order; a volatile field compares its recorded value. Each
plan lists the `checked_record_ordinals`. Otherwise the tool is a recorded view
with reason `envelope-varies`, `fields-differ` or `list-differs`; the other
reasons are `query-body` (a query `POST`), `no-init-recording`,
`body-unusable`, `not-entity`, `no-identity` and `query-varies` (a document
read under several queries). Documents are held only for list-shaped paths
that a `POST`, `PUT` or `PATCH` tool writes: a non-entity read of an item path
such as `/api/files/{id}`, or of a path that is only deleted, stays recorded
with `no-identity` or `not-entity`. A body is usable when its content type is
`body-unusable`, `no-identity` (an item read without a proved identity),
`path-varies` (another read under more than one path, as a template with a
parameter before its end), `query-varies` (another read of one path under
more than one query) and `rendered-oversize` (a document value whose JSON
renders to more than 256 KiB). A document holds the value of its one `path`, so for such
a template it says nothing about the template's other paths. A body is usable
when its content type is
JSON-like, it is UTF-8 text of at most 256 KiB, every number survives native
conversion exactly, and no member is named `__proto__`.
`writes` lists one plan per tool whose request may change state, in tool
order. Its `effect` is `create` for a `POST` to a type's
`collection_template`, `update` for a `PUT`, `PATCH` or `POST` to that template
plus one trailing parameter, `delete` for a `DELETE` to it, `document` for a
`PUT`, `PATCH` or `POST` to a document's path, and otherwise `none` with reason
`no-state-target`. A plan is built from the tool's first successful call: its
request body must be a usable JSON object (as response bodies are usable), and
the written entity must show in that call's response or in a later read before
the next write. Each field of a created entity gets the first rule that
explains it: generated `identity` (a sequence, UUID or token, by the recorded
value's shape), `echo` of a body member, `copy` of a body string of 8 or more
characters, a `path` value, a `slug` of a body member, a `clock` value within 10
minutes of the request in its recorded format, a `sequence` one past the
largest value of its collection or type, a `unique` UUID, or a `template` around
a generated value; other fields keep the observed value. An update's members
echo when stored as sent; a member stored differently is accepted only with its
recorded value (`body.fixed`) and stores the value the call that showed it
recorded (the `value` of its `constant` rule), and a field it changed without
sending must be a clock, an increment or a slug, or the plan is
`unexplained-change`; an increment or slug must hold for every call of the
tool. Every member the first call sent is required, except by `PATCH` and
`POST` updates.
The view then replays the timeline on a copy of the initial state. Modelled
writes apply their effect; only a `2xx` write is modelled or an example, and
any other write (a redirect, an unanswered or unmodelled one) leaves the entity
below whose item path it wrote unknown until a read shows it again (a whole
collection when the path names no held entity), and taints what it wrote at or
above. A read the served environment would answer from the state is compared
with its recording as the read plans are, as the whole response it would serve:
the entity or list rendered into the envelope of the item view's or the line's
base recording, with the members the state still holds. A 400, 404 or 410 (or a
`null` body) agrees with an entity the copy does not hold. A match confirms
the writes that changed its entity or collection since it was last read; a
mismatch contradicts them, or, when an unmodelled write came after them, is
checked only on the members they sent; a mismatch no write explains marks the related writes since as
`side-effect`. A plan is `served` when a read confirmed it and none
contradicted it; otherwise its `reason` is the first of its build reason
(`no-call`, `body-unusable`, `no-observation`, `unexplained-change`),
`contradicted` (with `contradicted_by`), `side-effect`, `unconfirmed` and `incomplete-entities` (an update or delete that answers with the entity, when no initial entity holds every field its recorded answer shows, counting for an update the fields its rules write, so it could answer only for entities the environment created).
`checked_record_ordinals` lists the confirming reads, `insert` places created
entities (sorted by a field the collection's recorded order shows, else last),
and `write_clock` is the end of the last timed record, to the next second.
Values in the state are recorded data, so the view refuses the same parents as
the manifest, including those inspected with `sensitive_values`, and a
`capture_browser_scenario` parent must have selected response bodies. Types and
identity are inferences, so the Evidence is always inferred. `unknowns` states
that the state establishes no authorization model, no way the application
computes responses or derived values, and no effects of writes. The state
computes responses or derived values, and write effects only as modelled from
recorded calls. The state
accumulator charges the manifest's admission and grouping, every record line,
each body as temporary bytes before it is decoded and parsed, and every value
it keeps, within the same 96 MiB budget. The view performs no network activity
and serves nothing; `rea environment-serve` derives it from the capture at
startup, serves read tools from it and loads replacement states through the
AgentEnv data plane, as
it keeps, within the same 96 MiB budget. Documents hold whole recorded response
values whose recorded text and rendered JSON are each at most 256 KiB, so a
capture with many read-only paths yields a correspondingly larger state result.
The view performs no network activity and serves nothing; `rea environment-serve`
derives it from the capture at startup, serves read tools and the served write
plans from it and loads replacement states through the AgentEnv data plane, as
[serve from the state](agent-environments.md#serve-from-the-state) describes.
## Resource and lifecycle boundaries
Complete evidence has a 32 MiB input, 96 MiB decoder reply and 64-level nesting
budget. Exceeding a budget returns no partial success.
budget. Those budgets have no runtime override. `capture_timeout_ms` does not
raise them. Exceeding a budget returns no partial success.
Oversized selected captures return an input error identifying `capture_path`
and the byte budget; owned reply/output failures remain output or provider errors.
HAR, SAZ and qlog run in an owned
192 MiB Node old-generation heap with one V8 worker. Mitmproxy decoding applies
Linux limits of 768 MiB address space, 30 seconds CPU and 96 MiB file output;
both commands have a 30-second wall deadline and independently supervised
both commands use `capture_timeout_ms` (default 30000) and independently supervised
process cleanup. Owned decoder diagnostics retain at most 1 MiB across stdout
and stderr; the collector drops overflow as it arrives, reports the limit
failure and verifies cleanup. Heap limits alone do not establish aggregate RSS limits.
+1 -1
View File
@@ -52,7 +52,7 @@ The result contains directly registered listener types/flags, callback coordinat
## Bounds and verification
Each protocol message has a 64 MiB budget; script/context metadata and request metadata each have an 8 MiB retained budget. Complete retained source text has a 32 MiB budget. Oversized or malformed evidence returns no partial successful result. Each command and the complete inspection/source-join phase have a 20-second deadline; cleanup has a separate 5-second command deadline plus transport closure. The selected observation duration must fit the host timer's positive 32-bit millisecond range. REA does not control the externally owned browser's CPU or memory.
Each protocol message has a 64 MiB budget; script/context metadata and request metadata each have an 8 MiB retained budget. Complete retained source text has a 32 MiB budget. Oversized or malformed evidence returns no partial successful result. Each command and the complete inspection/source-join phase use `command_timeout_ms` (default 20000). Cleanup uses a separate 5000 ms grace; `command_timeout_ms` does not raise or shorten it, and this release has no runtime override for that grace. Transport closure follows the grace. The selected observation duration must fit the host timer's positive 32-bit millisecond range. REA does not control the externally owned browser's CPU or memory.
`npm run verify:browser:runtime` uses caller-supplied `REA_BROWSER_EXECUTABLE` and the native `ps` command on Linux/macOS against an owned synthetic fixture, through real public CLI and stdio MCP. It checks exact Unicode/CRLF text/digests, same-URL identities, same-document navigation, inert declared names, executed and zero-count branches, function-only unknowns on repeated observation, listener selection, initiator association, actual arming and preservation of the external page. A separate fixture page closes during an armed MCP window to verify prompt termination and inline cleanup diagnostics. The optional positional entrypoint to `scripts/verify-browser-runtime.mjs` permits the same checks against an isolated installed tarball.
+3 -2
View File
@@ -71,7 +71,7 @@ implement every current source-map extension.
One owned Node subprocess decodes the map. Its private request directory is
removed before success is returned. Only this child's inherited `NODE_OPTIONS`
is removed; explicit V8 flags set a 192 MiB old-generation and 8 MiB semi-space
budget, one V8 worker and a 20-second deadline. Reported actual V8 heap limits
budget, one V8 worker and `decode_timeout_ms` (default 20000). The codec heap, map budget, and reply budget have no runtime override. Reported actual V8 heap limits
are checked. These are heap limits, not an aggregate process RSS limit. When
REA runs on Bun, the subprocess is Bun, which ignores these V8 flags: `runtime`
records `id: "bun"`, the Bun version, its Node.js compatibility version and a
@@ -86,7 +86,8 @@ cleanup overrides success with `cleanup_incomplete` and the owning
operation/resources.
Complete evidence is bounded by a 4 MiB map, 32 MiB codec reply, 262144 decoded
rows/segments and 64 nested section levels. Exceeding a budget returns an explicit
rows/segments and 64 nested section levels. Those structural budgets and the
192 MiB codec heap have no runtime override. Exceeding a budget returns an explicit
failure, without partial results. The existing 32 MiB manifest and 16 MiB selected
script boundaries also apply. No global toolchain or browser configuration changes
are made.
@@ -0,0 +1,84 @@
"""Summarize completed CLI trials; token counts are estimates, not model usage."""
import argparse
import hashlib
import json
import statistics
from pathlib import Path
import tiktoken
parser = argparse.ArgumentParser()
parser.add_argument('root', type=Path)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
root = args.root
inputs = json.loads((root / 'measurement-inputs.json').read_text())
expected = inputs['expected_runtime_bindings']
assert expected['complete']['git_revision'] == inputs['baseline_revision']
assert expected['selected']['git_revision'] == inputs['feature_parent_revision']
assert expected['complete']['root'] != expected['selected']['root']
assert expected['complete']['compiled_cli_sha256'] != expected['selected']['compiled_cli_sha256']
assert expected['complete']['source_diff_sha256'] == hashlib.sha256(b'').hexdigest()
assert expected['complete']['untracked_source_sha256'] == hashlib.sha256(b'[]').hexdigest()
enc = tiktoken.get_encoding('cl100k_base')
count = lambda text: len(enc.encode(text, disallowed_special=()))
compact = lambda obj: json.dumps(obj, ensure_ascii=False, separators=(',', ':'))
runs = []
for case in sorted(root.glob('pair-*')):
if not case.is_dir():
continue
report = json.loads((case / 'report.json').read_text())
assert report['status'] == 'passed' and report['cleanup_verified']
assert report['runtime_binding_verified']
assert report['runtime_binding'] == expected[report['mode']], case.name
assert str(Path(report['runtime']).resolve()) == report['runtime_binding']['root']
receipt = next(json.loads(line) for line in reversed(case.with_name(case.name + '.log').read_text().splitlines()) if line.startswith('{"resource_scope":'))
obs = receipt['payload_resource_observation']
assert receipt['abort'] is None and receipt['payload_resource_observation_verified']
assert obs['payload_exit_code'] == 0 and obs['payload_cleanup_verified']
assert obs['resource_events']['memory.events']['oom_kill'] == 0
tokens = {'question_and_recipe': count(report['question'] + '\n' + report['recipe']),
'arguments': sum(count(compact(call['arguments'])) for call in report['calls']),
'help_and_replies': sum(count((case / call['stdout_file']).read_text()) for call in report['calls'])}
tokens['whole_task'] = sum(tokens.values())
runs.append({'case': case.name, 'mode': report['mode'], 'cache': report['cache'], 'runtime': report['runtime'], 'runtime_binding': report['runtime_binding'], 'runtime_binding_audit_ms': report['runtime_binding_audit_ms'],
'wall_seconds': receipt['elapsed_seconds'], 'cpu_seconds': obs['cpu_stat']['usage_usec'] / 1e6,
'peak_process_family_bytes_including_page_cache': obs['memory_peak_bytes'],
'tokens_estimated': tokens, 'calls': report['calls'],
'parent_evidence_id': report['parent_evidence_id'], 'parent_semantic_sha256': report['parent_semantic_sha256'],
'answer_sha256': report['answer_sha256'], 'oracle_verified': report['oracle_verified'],
'cleanup_verified': True, 'resource_events': obs['resource_events']})
assert len(runs) == 12
for key in ['answer_sha256', 'parent_semantic_sha256', 'parent_evidence_id']:
assert len({row[key] for row in runs}) == 1, key
summary = {}
for cache in ['cold', 'warm']:
summary[cache] = {}
for mode in ['complete', 'selected']:
rows = [r for r in runs if r['mode'] == mode and r['cache'] == cache]
assert len(rows) == 3
summary[cache][mode] = {key: {'median': statistics.median(r[key] for r in rows), 'min': min(r[key] for r in rows), 'max': max(r[key] for r in rows)} for key in ['wall_seconds', 'cpu_seconds', 'peak_process_family_bytes_including_page_cache']}
summary[cache][mode]['whole_task_tokens_estimated'] = [r['tokens_estimated']['whole_task'] for r in rows]
result = {'status': 'passed', 'tokenizer': 'tiktoken==0.12.0/cl100k_base; estimates, not model usage',
'context_scope': 'Question, supplied command recipe, actual selected-command help, all logical CLI argv and all stdout. No model invoked; reasoning, framing, irrelevant tool discovery and actual native-agent usage unmeasured.',
'cache_scope': 'cold disables Node compile cache; warm primes the identical workflow once per runtime/mode before timing. Fresh processes and isolated HOME each trial; OS page cache uncontrolled.',
'resource_scope': 'Whole task driver plus all CLI children; includes pre/post source-and-compiled-runtime binding verification, bundle publication/authentication, help, extra calls and local answer/oracle checking. Kernel cgroup peak includes file cache; not sampled RSS. Download/build/oracle preparation and priming excluded.',
'acquisition': json.loads((root / 'acquisition.json').read_text()), 'measurement_inputs': inputs,
'summary': summary, 'runs': runs}
native = root / 'native-control'
if native.exists():
binding = json.loads((native / 'runtime-binding.json').read_text())
assert binding['runtime_binding_verified'] and binding['runtime_binding'] == expected['complete']
assert str(Path(binding['runtime']).resolve()) == binding['runtime_binding']['root']
receipt = next(json.loads(line) for line in reversed((root / 'native-control.log').read_text().splitlines()) if line.startswith('{"resource_scope":'))
obs = receipt['payload_resource_observation']
assert receipt['abort'] is None and receipt['payload_resource_observation_verified'] and obs['payload_exit_code'] == 0 and obs['payload_cleanup_verified']
baseline_case = root / next(run['case'] for run in runs if run['mode'] == 'complete')
assert json.loads((native / 'answer.json').read_text()) == json.loads((baseline_case / 'answer.json').read_text())
control_source = Path(__file__).with_name('managed-cli-local-control.py').read_text()
result['existing_native_control'] = {'runtime': binding['runtime'], 'runtime_binding': binding['runtime_binding'], 'runtime_binding_audit_ms': binding['runtime_binding_audit_ms'], 'repetitions': 1,
'tokens_estimated_script_plus_help_plus_answer': count(control_source) + count((native / 'help.txt').read_text()) + count((native / 'answer.json').read_text()),
'token_scope': 'Projection source, original CLI help and returned answer; excludes shared question and launcher argv. No actual model usage measured.',
'wall_seconds': receipt['elapsed_seconds'], 'cpu_seconds': obs['cpu_stat']['usage_usec'] / 1e6,
'peak_process_family_bytes_including_page_cache': obs['memory_peak_bytes'], 'answer_matches': True, 'cleanup_verified': True}
args.output.write_text(json.dumps(result, indent=2) + '\n')
print(json.dumps(summary, indent=2))
@@ -0,0 +1,44 @@
"""Existing native code-composition control: save complete stdout, return selected facts."""
import argparse
import json
import subprocess
import time
from pathlib import Path
parser = argparse.ArgumentParser()
parser.add_argument('runtime', type=Path)
parser.add_argument('target', type=Path)
parser.add_argument('output', type=Path)
parser.add_argument('--measurement-inputs', type=Path, required=True)
args = parser.parse_args()
args.output.mkdir()
node = 'node'
inputs = json.loads(args.measurement_inputs.read_text())
expected = inputs['expected_runtime_bindings']['complete']
assert expected['git_revision'] == inputs['baseline_revision']
helper = Path(__file__).with_name('managed-cli-runtime.mjs')
def capture_binding():
return json.loads(subprocess.check_output([node, str(helper), str(args.runtime)], text=True))
binding_started = time.monotonic()
binding = capture_binding()
binding_audit_ms = {'before': (time.monotonic() - binding_started) * 1000}
assert binding == expected
entry = str(args.runtime / 'scripts/rea.mjs')
help_result = subprocess.run([node, entry, 'inspect-managed-members', '--help'], check=True, capture_output=True, text=True)
(args.output / 'help.txt').write_text(help_result.stdout)
with (args.output / 'parent.json').open('w') as file:
subprocess.run([node, entry, 'inspect-managed-members', str(args.target), '--type', 'Newtonsoft.Json.JsonConvert', '--json'], check=True, stdout=file)
parent = json.loads((args.output / 'parent.json').read_text())
data = parent['normalized_result']
answer = {'parent_evidence_id': parent['evidence_id'], 'artifact_sha256': data['artifact']['sha256'],
'selection': data['selection'], 'source_coverage': data['coverage'],
'total_methods': len(data['methods']), 'total_member_refs': len(data['member_refs']),
'methods': data['methods'][:8], 'member_refs': data['member_refs'][:16],
'limitation': 'MemberRef rows are assembly-wide; methods are selected-type rows. Page exhaustion is not an absence claim.'}
(args.output / 'answer.json').write_text(json.dumps(answer))
binding_finished = time.monotonic()
assert capture_binding() == binding
binding_audit_ms['after'] = (time.monotonic() - binding_finished) * 1000
(args.output / 'runtime-binding.json').write_text(json.dumps({'runtime': str(args.runtime), 'runtime_binding': binding, 'runtime_binding_verified': True, 'runtime_binding_audit_ms': binding_audit_ms}))
print(json.dumps(answer))
@@ -0,0 +1,18 @@
import json,hashlib,argparse
from pathlib import Path
import dnfile
parser=argparse.ArgumentParser();parser.add_argument('root',type=Path);root=parser.parse_args().root;path=root/'Newtonsoft.Json.dll'
pe=dnfile.dnPE(str(path));assert pe.net is not None
counts={name:len(getattr(pe.net.mdtables,name).rows) if getattr(pe.net.mdtables,name,None) else 0 for name in ['TypeDef','MethodDef','MemberRef','Field','Property','Event','Param','StandAloneSig','TypeSpec','ExportedType']}
selected=[]
for index,row in enumerate(pe.net.mdtables.TypeDef.rows,1):
name=(str(row.TypeNamespace)+'.' if str(row.TypeNamespace) else '')+str(row.TypeName)
if name!='Newtonsoft.Json.JsonConvert':continue
selected.append({'token':f'0x{0x02000000|index:08x}','name':name,'methods':[{'token':f'0x{0x06000000|m.row_index:08x}','name':str(m.row.Name),'rva':m.row.Rva,'signature_hex':m.row.Signature.value.hex()} for m in row.MethodList]})
assert len(selected)==1
refs=[{'token':f'0x{0x0a000000|i:08x}','name':str(row.Name),'signature_hex':row.Signature.value.hex()} for i,row in enumerate(pe.net.mdtables.MemberRef.rows[:16],1)]
assembly=pe.net.mdtables.Assembly.rows[0]
oracle={'decoder':'dnfile==0.18.0 (independent Python PE/CLI parser)','assembly_sha256':hashlib.sha256(path.read_bytes()).hexdigest(),'assembly_name':str(assembly.Name),'assembly_version':'.'.join(str(getattr(assembly,k)) for k in ['MajorVersion','MinorVersion','BuildNumber','RevisionNumber']),'row_counts':counts,'selected_types':selected,'member_refs_first_16':refs,'target_execution':False}
(root/'oracle.json').write_text(json.dumps(oracle,indent=2)+'\n')
print(json.dumps({'assembly':oracle['assembly_name'],'version':oracle['assembly_version'],'row_counts':counts,'selected_type':selected[0]['name'],'selected_methods':len(selected[0]['methods']),'target_execution':False}))
pe.close()
@@ -0,0 +1,106 @@
import assert from "node:assert/strict";
import { execFileSync } from "node:child_process";
import { createHash } from "node:crypto";
import { readFile, readdir, realpath } from "node:fs/promises";
import { join, relative } from "node:path";
import { fileURLToPath } from "node:url";
const sha = (value) => createHash("sha256").update(value).digest("hex");
/** Bind a trial to its source checkout and compiled runtime, including deferred assets. */
export const captureRuntimeBinding = async (runtime, kind = "cli") => {
assert.ok(kind === "cli" || kind === "mcp", "Unknown runtime entry");
const root = await realpath(runtime);
const git = (args) =>
execFileSync("git", args, { cwd: root, maxBuffer: 16 * 1024 * 1024 });
const gitRevision = git(["rev-parse", "HEAD"]).toString().trim();
const sourcePaths = ["src", "scripts", "package.json", "package-lock.json"];
const sourceDiff = git([
"diff",
"--no-ext-diff",
"--no-textconv",
"--binary",
"HEAD",
"--",
...sourcePaths,
]);
const untracked = git([
"ls-files",
"--others",
"--exclude-standard",
"-z",
"--",
...sourcePaths,
])
.toString()
.split("\0")
.filter(Boolean)
.sort();
const untrackedEntries = [];
for (const name of untracked)
untrackedEntries.push([name, sha(await readFile(join(root, name)))]);
const files = ["scripts/rea.mjs", "package.json", "package-lock.json"];
const visit = async (directory) => {
const entries = await readdir(directory, { withFileTypes: true });
for (const entry of entries.sort((a, b) =>
a.name < b.name ? -1 : a.name > b.name ? 1 : 0,
)) {
const path = join(directory, entry.name);
assert.ok(!entry.isSymbolicLink(), `Unbound compiled CLI link: ${path}`);
if (entry.isDirectory()) await visit(path);
else if (entry.isFile() && /\.(?:[cm]?js|json)$/.test(entry.name))
files.push(relative(root, path));
}
};
// MCP materializes additional workers and reads build-owned JSON after
// discovery. Bind every compiled JS/JSON asset, rather than only its entry.
await visit(join(root, kind === "mcp" ? "dist" : "dist/cli-runtime"));
assert.ok(
files.includes(
kind === "mcp"
? join("dist", "mcp", "main.js")
: join("dist", "cli-runtime", "cli.js"),
),
"Missing compiled runtime entry",
);
const hash = createHash("sha256");
let bytes = 0;
for (const name of files.sort()) {
const value = await readFile(join(root, name));
bytes += value.length;
hash.update(JSON.stringify([name, value.length, sha(value)]) + "\n");
}
return {
binding_version: kind === "mcp" ? 2 : 1,
root,
node: process.version,
git_revision: gitRevision,
source_diff_sha256: sha(sourceDiff),
untracked_source_sha256: sha(JSON.stringify(untrackedEntries)),
...(kind === "mcp"
? {
compiled_runtime_sha256: hash.digest("hex"),
compiled_runtime_files: files.length,
compiled_runtime_bytes: bytes,
}
: {
compiled_cli_sha256: hash.digest("hex"),
compiled_cli_files: files.length,
compiled_cli_bytes: bytes,
}),
};
};
export const assertRuntimeBinding = (actual, expected, mode) => {
assert.ok(expected, `Missing expected runtime binding for ${mode}`);
assert.deepEqual(actual, expected, `Runtime binding differs for ${mode}`);
};
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
console.log(
JSON.stringify(
await captureRuntimeBinding(process.argv[2], process.argv[3]),
null,
2,
),
);
}
@@ -0,0 +1,72 @@
{
"status": "passed",
"scope": "Single completed SDK regression after shared JSON reader/admission changes; not a new paired source-to-bundle performance comparison. Existing semantic-node cap one answers static correspondence only; no equal semantic-graph coverage claim. Corpus/reference preparation performed earlier and excluded.",
"artifact": {
"name": "Redux Toolkit 2.2.7",
"tarball_sha256": "f2517c466b50424b661a27e48039f15db7783e5822f46b5c7d39f20104ac67a9"
},
"runtime_parent_revision": "4fab2dfda16236dc407aa179920780d6a970dda3",
"production_manifest": "managed-cli.json#measurement_inputs.production_sources_sha256",
"input_schema_profile": "default compact",
"output_schema_profile": "full",
"tokens_estimated": {
"input_catalog_once": 49054,
"arguments": 307,
"text_replies": 7017,
"whole_task": 56378
},
"token_scope": "tiktoken==0.12.0/cl100k_base; one name/description/input-schema catalog plus actual arguments and MCP text replies; excludes output schemas, framing, model prompts/reasoning and actual model usage. Not actual native-agent token usage.",
"actual_model_usage": null,
"wall_seconds": 21.38,
"cpu_seconds": 21.022683,
"peak_process_family_bytes_including_page_cache": 974041088,
"calls": [
{
"name": "analyze_javascript_application",
"elapsed_ms": 9837.265845000002,
"is_error": false,
"reply_bytes": 25941
},
{
"name": "compare_source_to_bundle",
"elapsed_ms": 256.3669919999993,
"is_error": false,
"reply_bytes": 13199
},
{
"name": "inspect_analysis_view",
"elapsed_ms": 3012.057904000001,
"is_error": false,
"reply_bytes": 19393
},
{
"name": "export_evidence_bundle",
"elapsed_ms": 148.9011160000009,
"is_error": false,
"reply_bytes": 364
}
],
"independent_proof": {
"status": "passed",
"method": "Independent Python JSON/SHA-256/POSIX path and basename joins; no REA parser or matcher.",
"source_files": 180,
"embedded_nodes": 99,
"candidate_relations": 210,
"strong_location_rows": 33,
"limitation": "Static multiple-location projections; no semantic-equivalence or removal claim."
},
"resource_events": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"cleanup_verified": true
}
@@ -0,0 +1,266 @@
import {
captureRuntimeBinding,
assertRuntimeBinding,
} from "./managed-cli-runtime.mjs";
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { promisify } from "node:util";
import { createHash } from "node:crypto";
import { readFile, writeFile, mkdir } from "node:fs/promises";
import { join, resolve } from "node:path";
const execute = promisify(execFile);
const [runtime, root, output, mode, cache = "cold"] = process.argv.slice(2);
for (const value of [runtime, root, output])
assert.equal(resolve(value), value);
assert.ok(["complete", "selected"].includes(mode));
assert.ok(["cold", "warm"].includes(cache));
await mkdir(output);
await mkdir(join(output, "tmp"));
const oracle = JSON.parse(await readFile(join(root, "oracle.json"), "utf8"));
const target = join(root, "Newtonsoft.Json.dll");
const hash = (value) => createHash("sha256").update(value).digest("hex");
assert.equal(hash(await readFile(target)), oracle.assembly_sha256);
const stable = (value) =>
Array.isArray(value)
? value.map(stable)
: value !== null && typeof value === "object"
? Object.fromEntries(
Object.keys(value)
.sort()
.map((key) => [key, stable(value[key])]),
)
: value;
const report = {
status: "incomplete",
runtime,
mode,
cache,
node: process.version,
question:
"Report identity/selection/source coverage for JsonConvert, its total method count and the first eight complete method records, plus the first sixteen assembly-wide MemberRef records and continuation coverage.",
recipe:
mode === "selected"
? "Use --detail summary --evidence-bundle FILE; pass the exact parent_evidence_id to inspect-analysis-view --evidence-bundle FILE with source kind retained-evidence and view {kind:managed-members,collection:methods|member_refs,offset:0,limit:8|16}."
: "Use inspect-managed-members --type Newtonsoft.Json.JsonConvert; the completed full Evidence supplies the requested rows and remains accessible.",
calls: [],
cleanup_verified: false,
};
const began = performance.now();
const environment = {
PATH: process.env.PATH,
HOME: output,
TMPDIR: join(output, "tmp"),
LANG: "C.UTF-8",
REA_LOG_LEVEL: "silent",
UV_THREADPOOL_SIZE: "1",
NODE_OPTIONS: "--max-old-space-size=2048",
...(cache === "cold"
? { NODE_DISABLE_COMPILE_CACHE: "1" }
: { NODE_COMPILE_CACHE: join(root, `compile-cache-${mode}`) }),
};
const call = async (label, args, json = true) => {
const started = performance.now();
const result = await execute(
process.execPath,
[join(runtime, "scripts/rea.mjs"), ...args],
{
cwd: output,
env: environment,
maxBuffer: 16 * 1024 * 1024,
timeout: 120000,
},
);
await writeFile(join(output, label + ".stdout"), result.stdout);
await writeFile(join(output, label + ".stderr"), result.stderr);
report.calls.push({
label,
arguments: ["rea", ...args],
elapsed_ms: performance.now() - started,
stdout_bytes: Buffer.byteLength(result.stdout),
stdout_file: label + ".stdout",
});
return json ? JSON.parse(result.stdout) : result.stdout;
};
let parent, summary, methods, refs;
try {
const inputs = JSON.parse(
await readFile(join(root, "measurement-inputs.json"), "utf8"),
);
const expected = inputs.expected_runtime_bindings?.[mode];
assert.ok(expected, `Missing expected runtime for ${mode}`);
assert.equal(
expected.git_revision,
mode === "complete"
? inputs.baseline_revision
: inputs.feature_parent_revision,
);
assert.notEqual(
inputs.expected_runtime_bindings.complete.root,
inputs.expected_runtime_bindings.selected.root,
);
assert.notEqual(
inputs.expected_runtime_bindings.complete.compiled_cli_sha256,
inputs.expected_runtime_bindings.selected.compiled_cli_sha256,
);
const bindingStarted = performance.now();
report.runtime_binding = await captureRuntimeBinding(runtime);
report.runtime_binding_audit_ms = {
before: performance.now() - bindingStarted,
};
assertRuntimeBinding(report.runtime_binding, expected, mode);
await call("help-members", ["inspect-managed-members", "--help"], false);
if (mode === "complete") {
parent = await call("complete", [
"inspect-managed-members",
target,
"--type",
"Newtonsoft.Json.JsonConvert",
"--json",
]);
methods = parent.normalized_result.methods.slice(0, 8);
refs = parent.normalized_result.member_refs.slice(0, 16);
} else {
await call("help-view", ["inspect-analysis-view", "--help"], false);
const bundlePath = join(output, "parent.json");
summary = await call("summary", [
"inspect-managed-members",
target,
"--type",
"Newtonsoft.Json.JsonConvert",
"--detail",
"summary",
"--evidence-bundle",
bundlePath,
"--json",
]);
const id = summary.normalized_result.parent_evidence_id;
assert.ok(id);
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
assert.equal(bundle.records.length, 1);
parent = bundle.records[0];
assert.equal(parent.evidence_id, id);
for (const [collection, limit] of [
["methods", 8],
["member_refs", 16],
]) {
const value = await call(collection, [
"inspect-analysis-view",
JSON.stringify({
source: { kind: "retained-evidence", evidence_id: id },
view: { kind: "managed-members", collection, offset: 0, limit },
}),
"--evidence-bundle",
bundlePath,
"--json",
]);
const page = value.normalized_result;
assert.equal(page.parent_evidence_id, id);
assert.deepEqual(page.source_coverage, parent.normalized_result.coverage);
assert.deepEqual(
page.page.rows,
parent.normalized_result[collection].slice(0, limit),
);
assert.equal(
page.coverage.total,
parent.normalized_result[collection].length,
);
assert.equal(page.coverage.next_offset, limit);
assert.equal(page.coverage.exhausted, false);
if (collection === "methods") methods = page.page.rows;
else refs = page.page.rows;
}
}
const data = parent.normalized_result;
assert.equal(data.artifact.sha256, oracle.assembly_sha256);
assert.equal(data.methods.length, 72);
assert.equal(data.member_refs.length, 2149);
assert.equal(data.types.length, 1);
assert.equal(data.types[0].token, oracle.selected_types[0].token);
for (let i = 0; i < data.methods.length; i++) {
const row = data.methods[i],
wanted = oracle.selected_types[0].methods[i];
assert.equal(row.token, wanted.token);
assert.equal(row.name, wanted.name);
assert.equal(row.rva, wanted.rva);
assert.equal(
row.signature.raw_sha256,
hash(Buffer.from(wanted.signature_hex, "hex")),
);
}
for (let i = 0; i < refs.length; i++) {
const row = refs[i],
wanted = oracle.member_refs_first_16[i];
assert.equal(row.token, wanted.token);
assert.equal(row.name, wanted.name);
assert.equal(
row.signature.raw_sha256,
hash(Buffer.from(wanted.signature_hex, "hex")),
);
}
assert.equal(methods.length, 8);
assert.equal(refs.length, 16);
if (summary) {
assert.equal(summary.normalized_result.summary.reported_counts.methods, 72);
assert.equal(
summary.normalized_result.summary.reported_counts.member_refs,
2149,
);
assert.deepEqual(summary.normalized_result.summary.coverage, data.coverage);
assert.deepEqual(
summary.normalized_result.summary.selection,
data.selection,
);
}
const answer = {
parent_evidence_id: parent.evidence_id,
artifact_sha256: oracle.assembly_sha256,
selection: data.selection,
source_coverage: data.coverage,
total_methods: data.methods.length,
total_member_refs: data.member_refs.length,
methods,
member_refs: refs,
limitation:
"MemberRef rows are assembly-wide; methods are selected-type rows. Page exhaustion is not an absence claim.",
};
report.parent_evidence_id = parent.evidence_id;
report.parent_semantic_sha256 = hash(JSON.stringify(stable(parent)));
report.answer_sha256 = hash(JSON.stringify(stable(answer)));
report.oracle_verified = {
all_selected_method_metadata: 72,
returned_methods: 8,
returned_member_refs: 16,
};
const bindingFinished = performance.now();
assertRuntimeBinding(
await captureRuntimeBinding(runtime),
report.runtime_binding,
mode,
);
report.runtime_binding_audit_ms.after = performance.now() - bindingFinished;
report.runtime_binding_verified = true;
report.status = "passed";
await writeFile(join(output, "parent-evidence.json"), JSON.stringify(parent));
await writeFile(join(output, "answer.json"), JSON.stringify(answer));
} finally {
report.elapsed_ms = performance.now() - began;
report.cleanup_verified = true;
await writeFile(
join(output, "report.json"),
JSON.stringify(report, null, 2) + "\n",
);
}
console.log(
JSON.stringify({
status: report.status,
mode,
cache,
parent: report.parent_evidence_id,
answer: report.answer_sha256,
calls: report.calls.map((c) => ({
label: c.label,
elapsed_ms: c.elapsed_ms,
bytes: c.stdout_bytes,
})),
}),
);
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,219 @@
# Portable managed CLI member views
Issue #5148 / PR #5167. Baseline `main@3dbe314ada9f6dcc994ca4655dc0faf082503847`;
feature integrates the original `7b3ed4d025edd24d062cde0327a53bd311e174fc`.
The measured production-file SHA-256 manifest is in [managed-cli.json](managed-cli.json).
These are working-tree measurements. Each trial verifies both source and the
complete compiled CLI graph before and after execution against its explicitly
selected build. The two roles have distinct checkout roots and compiled digests.
Identity-audit time is included in whole-task costs and recorded separately.
The audit reads compiled files before timing individual CLI calls, warming those
filesystem pages; cold means disabled Node compile cache, not disk-cold startup.
## Question and independent checks
Report identity, selected type, source coverage, the total method count, the first
eight complete method records, and the first sixteen assembly-wide MemberRef
records, including continuation coverage. Keep complete original Evidence available.
The question deliberately does not ask for every method or reference.
Public artifact: [Newtonsoft.Json 13.0.3](https://www.nuget.org/packages/Newtonsoft.Json/13.0.3),
from the NuGet flat-container archive. The 2,441,966-byte archive SHA-256 is
`872fc189e638ab1056555b03aaa38f68bcb54286e221aa646eb1129babf63c77`.
Selected member `lib/net6.0/Newtonsoft.Json.dll`, 712,464 bytes, SHA-256
`22c649f75fce5be7c7ccda8880473b634ef69ecf33f5d1ab8ad892caf47d5a07`.
Package version 13.0.3 is distinct from metadata assembly version 13.0.0.0.
No library code is executed; REA parses PE/CLI statically.
An independent Python oracle uses `dnfile==0.18.0` and `pefile==2024.8.26`.
It finds 494 TypeDefs, 4,208 MethodDefs and 2,149 MemberRefs. The selected
`Newtonsoft.Json.JsonConvert` has 72 methods. Every trial checks all 72 method
tokens, names, RVAs and original signature hashes, plus the sixteen returned
MemberRef tokens, names and signature hashes, against this oracle. This does
not independently establish complete CIL semantics or opcode correctness.
The full parent Evidence and requested answers are identical across all twelve
trials; returned rows equal the exact parent slices. Selection and source
coverage remain distinct from page exhaustion. MemberRefs remain assembly-wide.
## Completed-task cost
Linux x64, Node 24.18.0/npm 11.16.0. Three alternating pairs per cache condition,
serial, one CPU; 4 GiB hard / 3 GiB early-stop family guard, no swap, 192-task
ceiling. Every task includes actual selected-command help, all CLI calls,
provider parsing, bundle publication/authentication, local answer extraction
and oracle comparisons, plus both runtime-identity audits. Build, download, oracle preparation and warm priming
are excluded. OS page cache is uncontrolled. No build/test overlapped timing.
Complete mode starts two CLI processes (help plus analysis). Selected mode starts
five (two help commands, summary/publication, methods page, MemberRef page).
Cold disables the Node compile cache; warm primes the exact workflow once per
runtime/mode, then still starts fresh processes with isolated HOME each trial.
All trials complete with verified process cleanup and no OOM/resource-stop event.
| Whole task, median | Complete | Summary + two pages |
| ----------------------------------------------- | -------: | ------------------: |
| Estimated context tokens, both cache conditions | 451,373 | 13,078 |
| Cold wall seconds | 9.969 | 14.801 |
| Cold family CPU seconds | 9.743 | 14.555 |
| Cold kernel family peak MiB | 393.281 | 400.098 |
| Warm wall seconds | 10.339 | 24.141 |
| Warm family CPU seconds | 10.097 | 23.842 |
| Warm kernel family peak MiB | 401.809 | 407.164 |
Context estimates include the question, supplied command recipe, all logical
argv, actual help and stdout. Tokenizer `tiktoken==0.12.0`, `cl100k_base`.
The reduction is 97.1% for this fixed task. There is no model invocation;
actual billed/native-agent usage, reasoning and transport framing are unmeasured.
The memory counter is the kernel cgroup process-family peak including file
cache, not sampled RSS. Ranges and individual receipts are in the JSON.
This is an opt-in context/portability improvement with extra latency and CPU,
not a speed or memory optimization. Complete provider parsing and full authentication of file-supplied bundles
remain. Fresh managed Evidence is sealed at its producing factory and reused
without another payload clone, including low-headroom error projection. Complete output remains the default and is preferable
for small outputs or questions requiring every row. No cache or parser cap
was introduced.
## Existing native code composition
The unchanged baseline can already redirect full stdout to a file and select
rows in code. [The measured control](managed-cli-local-control.py) returns the
identical answer and retains the complete original Evidence. One cold run:
11.249 s wall, 11.000 s family CPU,
375.3 MiB kernel peak. Projection source,
original help and returned answer together estimate 10,555 tokens; the shared
question and launcher argv are excluded. This single-run control does not
establish stable performance or actual native-agent usage. It does demonstrate
that raw CLI reply savings are not universal savings over code composition.
The additive interface supplies authenticated reusable projections through the
same application/domain owner as MCP, without caller-maintained JSON slicing.
## Correctness and resource boundaries
The public CLI regression pages all eight managed collections after process
exit, target deletion and a working-directory change. Complete parent identity,
original row order, signatures, source/page coverage and unknowns survive.
Existing outputs are not replaced; failed publication retains completed facts.
The Evidence factory parses mutable caller data into independent trees, computes
identity, then seals its fresh owned snapshot with active ancestry/iterators.
It does not retain a set/array of every completed object. Existing sealed roots
can be shared; mutable input aliases are cloned independently. General graph
sealing keeps its separate cycle, sharing, hidden-field and atomic-failure rules.
Managed and derived-view factories seal their own completed Evidence, so CLI
error projection can borrow it without reauthentication or another payload copy.
The view owner retains strict JSON normalization for optional-field contracts;
the CLI no longer copies its already validated completed projection again.
Shared heap admission traverses by depth rather than queueing every report row
or allocating an entries array. It reserves parsed working copies and actual
escaping expansion before cloning/hashing, reporting UTF-16 storage explicitly.
Canonical bundle equality streams existing ordinary JSON encoder parts, preserving
native key order, omission, negative zero and Unicode semantics without two
whole-document strings. All production bundle readers explicitly pass their
operation: CLI import/export/compare/inspection, retained views, MCP import/
comparison and both native-matching read passes. Errors retain the original
cause, cleanup primary and lease through retry.
One process primitive dispatches queued callbacks and then their OS signal
notifications. The managed producer checks it before accepting complete Evidence.
Publication/read stages use it before their final checks. Inline, file and bundle
CLI views share the existing command finalizer and actual resource scope; the
finalizer checks interruption after close while preserving any primary/cleanup
failure. Existing inline error labels remain intact. Synchronous analysis is
cooperatively cancellable between stages, not preemptible.
Four compiled publication cases (SIGINT/SIGTERM × event/actual POSIX signal)
fail without their checkpoint and pass restored. Eight additional compiled
completion cases cover default complete delivery and inline projection; all fail
with only their new completion checkpoints removed and pass restored. Fixture
hooks match the selected target path or exact parent ID. Two real Newtonsoft
publication runs and four real default/inline completion runs retain the identical
parent or independently checked selected summary, exit 130/143 and preserve
committed receipts where applicable. No target library code is executed.
All 556 focused tests across 36 files pass, including direct
finalizer/cleanup ownership, WASM/Android cancellation and native matching.
All six `npm run check` tasks plus architecture guards, and all six
`npm run docs:check` tasks plus five README tests pass. The expanded domain
project passes 5,617 tests in 287 files with
3 skipped cases; this overlaps the focused gate. Its first
batched shard hit two existing 30-second timeouts in unchanged large synthetic
tests. Both passed isolated and in the fresh shard rerun under the same timeout;
the cause of the batch variation is unverified. No production/timeout change
was made for those cases. An earlier unsplit expansion was manually stopped
before its heap ceiling; that incomplete attempt is not counted as a pass.
Eight producing/operation negative controls also catch graph-sized sealing/
admission scratch and wrong operation attribution, including native reload.
Native built-in instrumentation is confined to the isolated process-global lane.
Initial test-only spy/probe/schema-import issues were corrected before these
passing gates. No actual heap exhaustion is induced. Simulated providers do not
qualify foreign engines; Windows OS signals remain unverified locally.
## Shared-reader real-artifact regression
The existing Redux Toolkit 2.2.7 source/bundle task also completes through the
updated reader with default full output schemas and compact input schemas.
An independent Python SHA-256/path oracle verifies 180 historical files,
99 embedded source nodes and 210 candidate relations (33 strong-location rows,
147 unknown). Original signals and partial source coverage remain visible.
This uses the existing explicit semantic-node cap of one for static correspondence;
it does not establish equivalent semantic-graph coverage or behavioral equivalence.
One completed task: 21.380 s wall,
21.023 s family CPU,
928.9 MiB kernel family peak,
56,378 estimated context tokens (one input catalog,
all arguments and MCP text replies; excludes output schemas, framing and model
reasoning). Corpus/reference preparation was performed earlier and excluded.
This is a regression verification with resource observation, not a new paired
performance comparison. Actual model usage remains unmeasured. Details:
[managed-cli-shared-read.json](managed-cli-shared-read.json).
## Reproduction
Extract the selected DLL to a private lab as `Newtonsoft.Json.dll`. Create an
`acquisition.json` containing the URL, hashes, sizes and selected ZIP member
shown above; never commit the DLL. Prepare the independent oracle:
```bash
uv run --no-project --with dnfile==0.18.0 --with pefile==2024.8.26 python \
experiments/analysis-efficiency/managed-cli-oracle.py /absolute/lab
```
Build baseline and feature separately with the pinned toolchain. Run each under
the shared resource guard, alternating complete/selected, selected/complete,
complete/selected for cold and warm. Use a new output directory for every run:
```bash
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
node experiments/analysis-efficiency/managed-cli-task.mjs \
/absolute/runtime /absolute/lab /absolute/lab/pair-cold-1-selected selected cold
```
Save each guard output to the adjacent `pair-*.log`. Warm mode uses the lab's
mode-specific compile-cache directory; prime each identical workflow once first.
Supply a `measurement-inputs.json` with exact revisions and production hashes,
plus `expected_runtime_bindings.complete` and `.selected`. Capture each with
`node experiments/analysis-efficiency/managed-cli-runtime.mjs /absolute/runtime`
after building. Bind complete to the clean baseline revision, selected to the
feature parent revision plus tracked/untracked source digests, with different
roots and compiled-graph digests. The collector rejects a wrong build before any
CLI call, rechecks it afterward, and aggregation independently enforces identity.
A real wrong-runtime attempt is rejected before analysis. A copied trial with a
tampered revision fails aggregation; removing only that consumer guard accepts it.
The twelve-trial aggregation reproduces without private pilots. The native
control also requires `--measurement-inputs /absolute/lab/measurement-inputs.json`
and performs the same before/after identity checks.
Then aggregate with:
```bash
uv run --no-project --with tiktoken==0.12.0 python \
experiments/analysis-efficiency/managed-cli-costs.py /absolute/lab \
--output /absolute/results.json
```
The original issue reporter's executable, foreign operating systems and actual
model sessions remain unverified. Setup pilots and earlier unbound cohorts, including one failed setup run
while compiled artifacts were absent after a typecheck failure, are excluded.
@@ -226,3 +226,8 @@ keeps default valid calls and typed refusals while checking full producer
constraints separately. Five isolated suites passed 14 tests across call paths,
ADB start, browser scenarios, source selection and managed analysis. The first
browser attempt failed and remains recorded; no performance cohort changed.
The subsequent [startup audit](mcp-startup.md) extends this harness with
source/bundle bindings and a post-materialization directory check. Its new
measurements are a separate cohort; reproduce this report at its recorded
revision when comparing the original timing and model-session receipts.
@@ -7,6 +7,11 @@ import { parseArgs } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { Ajv2020 } from "ajv/dist/2020.js";
import { sampleRetainedHeap } from "./mcp-retained-heap.mjs";
import {
captureRuntimeBinding,
assertRuntimeBinding,
} from "./managed-cli-runtime.mjs";
const { values } = parseArgs({
options: {
@@ -15,54 +20,135 @@ const { values } = parseArgs({
oracle: { type: "string" },
output: { type: "string" },
profile: { type: "string", default: "default" },
phase: { type: "string", default: "analysis" },
cache: { type: "string", default: "cold" },
"cache-root": { type: "string" },
bindings: { type: "string" },
role: { type: "string" },
"diagnostic-heap": { type: "boolean", default: false },
},
});
for (const key of ["runtime", "artifact", "oracle", "output"])
assert.ok(["analysis", "discovery"].includes(values.phase));
assert.ok(["cold", "warm"].includes(values.cache));
assert.ok(!values["diagnostic-heap"] || values.phase === "analysis");
for (const key of values.phase === "analysis"
? ["runtime", "artifact", "oracle", "output"]
: ["runtime", "output"])
assert.ok(
values[key] && resolve(values[key]) === values[key],
`${key} must be absolute`,
);
assert.ok(["default", "full", "compact"].includes(values.profile));
if (values.cache === "warm")
assert.ok(
values["cache-root"] &&
resolve(values["cache-root"]) === values["cache-root"],
);
await mkdir(values.output, { recursive: false });
const hash = (value) => createHash("sha256").update(value).digest("hex");
const oracle = JSON.parse(await readFile(values.oracle, "utf8"));
const artifactSha = hash(await readFile(values.artifact));
assert.equal(artifactSha, oracle.artifact_sha256);
let oracle;
let artifactSha;
if (values.phase === "analysis") {
oracle = JSON.parse(await readFile(values.oracle, "utf8"));
artifactSha = hash(await readFile(values.artifact));
assert.equal(artifactSha, oracle.artifact_sha256);
}
let expectedBinding;
let bindingBefore;
let auditBeforeMs;
let measurementInputsSha256;
const producerSources = {};
if (values.bindings !== undefined) {
assert.ok(["baseline", "candidate"].includes(values.role));
const inputBytes = await readFile(values.bindings);
measurementInputsSha256 = hash(inputBytes);
const inputs = JSON.parse(inputBytes.toString("utf8"));
for (const name of [
"mcp-input-catalog.mjs",
"managed-cli-runtime.mjs",
"mcp-retained-heap.mjs",
]) {
producerSources[name] = hash(
await readFile(new URL(name, import.meta.url)),
);
assert.equal(producerSources[name], inputs.harness_sources_sha256[name]);
}
expectedBinding = inputs.runtime_bindings[values.role];
const start = performance.now();
bindingBefore = await captureRuntimeBinding(values.runtime, "mcp");
auditBeforeMs = performance.now() - start;
assertRuntimeBinding(bindingBefore, expectedBinding, values.role);
}
const environment = {
PATH: process.env.PATH,
HOME: values.output,
REA_LOG_LEVEL: "silent",
UV_THREADPOOL_SIZE: "1",
NODE_OPTIONS: "--max-old-space-size=2048",
NODE_DISABLE_COMPILE_CACHE: "1",
...(values.cache === "cold"
? { NODE_DISABLE_COMPILE_CACHE: "1" }
: { NODE_COMPILE_CACHE: values["cache-root"] }),
...(values.profile === "default"
? {}
: { REA_MCP_INPUT_SCHEMA_PROFILE: values.profile }),
};
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
args: [
...(values["diagnostic-heap"] ? ["--inspect=127.0.0.1:0"] : []),
join(values.runtime, "scripts/rea.mjs"),
"mcp",
],
cwd: values.output,
env: environment,
stderr: "pipe",
});
const protocolRequests = {};
const send = transport.send.bind(transport);
transport.send = (...args) => {
const method = args[0].method;
if (typeof method === "string")
protocolRequests[method] = (protocolRequests[method] ?? 0) + 1;
return send(...args);
};
const client = new Client({ name: "rea-catalog-task", version: "1" });
const report = {
profile: values.profile,
phase: values.phase,
cache: values.cache,
cache_root: values["cache-root"] ?? null,
role: values.role,
measurement_inputs_sha256: measurementInputsSha256,
producer_sources_sha256: producerSources,
artifact_sha256: artifactSha,
calls: [],
checks: [],
status: "incomplete",
cleanup_verified: false,
protocol_requests: protocolRequests,
};
const transcript = [];
let stderr = "";
let inspectorEndpoint;
transport.stderr?.on("data", (value) => {
stderr += value;
if (values["diagnostic-heap"] && inspectorEndpoint === undefined)
inspectorEndpoint = stderr.match(
/Debugger listening on (ws:\/\/127\.0\.0\.1:\d+\/[^\s]+)/u,
)?.[1];
});
const began = performance.now();
try {
await client.connect(transport);
report.protocol_era = client.getProtocolEra();
await writeFile(
join(values.output, "server.json"),
JSON.stringify({
info: client.getServerVersion(),
capabilities: client.getServerCapabilities(),
instructions: client.getInstructions(),
}),
);
const catalog = await client.listTools();
assert.equal(catalog.nextCursor, undefined);
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
@@ -73,97 +159,154 @@ try {
ajv.validateSchema(tool.inputSchema),
`Invalid schema: ${tool.name}`,
);
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
{
const encoded = JSON.stringify(catalog);
await writeFile(join(values.output, "catalog.json"), encoded);
report.catalog_sha256 = hash(encoded);
}
report.tool_names = [...definitions.keys()];
const call = async (name, args) => {
const definition = definitions.get(name);
assert.ok(definition, `Missing named tool ${name}`);
const start = performance.now();
const reply = await client.callTool(
{ name, arguments: args },
{ toolDefinition: definition },
if (values.phase === "discovery") {
const prompts = await client.listPrompts();
assert.equal(prompts.nextCursor, undefined);
await writeFile(
join(values.output, "prompts.json"),
JSON.stringify(prompts),
);
report.calls.push({
name,
elapsed_ms: performance.now() - start,
is_error: reply.isError === true,
});
transcript.push({ name, arguments: args, reply });
return reply;
};
const input = {
path: values.artifact,
view: {
collection: "packages",
names: ["serde"],
fields: ["name", "version", "kind"],
},
};
const first = await call("inspect_rust_binary", input);
assert.notEqual(first.isError, true);
const selected = first.structuredContent;
assert.ok(selected?.normalized_result?.page);
assert.deepEqual(selected.normalized_result.page.rows, oracle.serde_rows);
assert.equal(selected.subject.digest.sha256, artifactSha);
const parentId = selected.normalized_result.parent_evidence_id;
for (let index = 0; index < 2; index++) {
const reply = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: parentId },
report.prompts_sha256 = hash(JSON.stringify(prompts));
report.answer = {
tools: report.tool_names,
prompts: prompts.prompts.map((prompt) => prompt.name),
};
report.status = "passed";
} else {
const call = async (name, args) => {
const definition = definitions.get(name);
assert.ok(definition, `Missing named tool ${name}`);
const start = performance.now();
const reply = await client.callTool(
{ name, arguments: args },
{ toolDefinition: definition },
);
report.calls.push({
name,
elapsed_ms: performance.now() - start,
is_error: reply.isError === true,
});
transcript.push({ name, arguments: args, reply });
return reply;
};
const input = {
path: values.artifact,
view: {
kind: "rust",
collection: "packages",
names: ["serde"],
fields: ["name", "version", "kind"],
},
};
const first = await call("inspect_rust_binary", input);
assert.notEqual(first.isError, true);
const selected = first.structuredContent;
assert.ok(selected?.normalized_result?.page);
assert.deepEqual(selected.normalized_result.page.rows, oracle.serde_rows);
assert.equal(selected.subject.digest.sha256, artifactSha);
const parentId = selected.normalized_result.parent_evidence_id;
for (let index = 0; index < 2; index++) {
const reply = await call("inspect_analysis_view", {
source: { kind: "retained-evidence", evidence_id: parentId },
view: {
kind: "rust",
collection: "packages",
names: ["serde"],
fields: ["name", "version", "kind"],
},
});
assert.notEqual(reply.isError, true);
assert.deepEqual(
reply.structuredContent.normalized_result.page.rows,
oracle.serde_rows,
);
assert.equal(
reply.structuredContent.normalized_result.parent_evidence_id,
parentId,
);
}
const malformed = await call("inspect_rust_binary", {
path: values.artifact,
view: { collection: "packages", offset: -1 },
});
assert.notEqual(reply.isError, true);
assert.equal(malformed.isError, true);
report.checks.push(
"Independent package rows and digest; repeated retained identity; canonical negative-offset refusal",
);
const exported = await call("export_evidence_bundle", {
path: join(values.output, "evidence.json"),
});
assert.notEqual(exported.isError, true);
const bundle = JSON.parse(
await readFile(join(values.output, "evidence.json"), "utf8"),
);
const parent = bundle.records.find(
(record) => record.evidence_id === parentId,
);
assert.ok(parent, "Complete original Evidence is retained");
assert.equal(parent.subject.digest.sha256, artifactSha);
assert.deepEqual(
reply.structuredContent.normalized_result.page.rows,
oracle.serde_rows,
parent.normalized_result.auditable_metadata.packages,
oracle.packages,
);
assert.equal(
reply.structuredContent.normalized_result.parent_evidence_id,
parentId,
assert.deepEqual(
parent.normalized_result.auditable_metadata.dependency_edges,
oracle.edges,
);
report.parent_evidence_id = parentId;
report.parent_semantic_sha256 = hash(JSON.stringify(parent));
report.answer = oracle.serde_rows;
assert.deepEqual(
await client.listTools(),
catalog,
"Advertisement changed after canonical materialization",
);
report.checks.push(
"Catalog remains identical after canonical materialization",
);
report.status = "passed";
}
if (values["diagnostic-heap"]) {
report.status = "incomplete";
assert.ok(inspectorEndpoint, "Missing explicitly launched inspector");
report.diagnostic_heap = await sampleRetainedHeap(
inspectorEndpoint,
transport.pid,
);
report.status = "passed";
}
const malformed = await call("inspect_rust_binary", {
path: values.artifact,
view: { collection: "packages", offset: -1 },
});
assert.equal(malformed.isError, true);
report.checks.push(
"Independent package rows and digest; repeated retained identity; canonical negative-offset refusal",
);
const exported = await call("export_evidence_bundle", {
path: join(values.output, "evidence.json"),
});
assert.notEqual(exported.isError, true);
const bundle = JSON.parse(
await readFile(join(values.output, "evidence.json"), "utf8"),
);
const parent = bundle.records.find(
(record) => record.evidence_id === parentId,
);
assert.ok(parent, "Complete original Evidence is retained");
assert.equal(parent.subject.digest.sha256, artifactSha);
assert.deepEqual(
parent.normalized_result.auditable_metadata.packages,
oracle.packages,
);
assert.deepEqual(
parent.normalized_result.auditable_metadata.dependency_edges,
oracle.edges,
);
report.parent_evidence_id = parentId;
report.parent_semantic_sha256 = hash(JSON.stringify(parent));
report.answer = oracle.serde_rows;
report.status = "passed";
} finally {
await client.close();
await transport.close();
report.cleanup_verified = transport.pid === null;
assert.ok(report.cleanup_verified);
report.elapsed_ms = performance.now() - began;
if (expectedBinding !== undefined) {
const start = performance.now();
const after = await captureRuntimeBinding(values.runtime, "mcp");
report.runtime_audit_ms = {
before: auditBeforeMs,
after: performance.now() - start,
};
assertRuntimeBinding(after, expectedBinding, values.role);
assert.deepEqual(after, bindingBefore);
assert.equal(
hash(await readFile(values.bindings)),
measurementInputsSha256,
);
for (const [name, digest] of Object.entries(producerSources))
assert.equal(
hash(await readFile(new URL(name, import.meta.url))),
digest,
);
report.runtime_binding = after;
report.runtime_binding_verified = true;
}
await writeFile(
join(values.output, "report.json"),
JSON.stringify(report, null, 2) + "\n",
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,143 @@
# Integer validator allocation after MCP materialization
Issue #6236; PR #4360. Linux x64, 2026-10-22.
<!-- BEGIN:allocation-report:provenance -->
Baseline `main@4a4bbab3c337701febeb6b532ace49041c4876e0`; candidate `461f342c71a72ef2be60912391df2393aa4e2ea3`. This is a pinned comparison.
In separate inspector/forced-GC diagnostics, retained REA V8 heap decreases by about 9.6 MiB. The three-pair ranges do not overlap in either profile. Inspector perturbation is included. Normal task timing, CPU and peak memory are reported separately; these observations do not establish a general latency or peak-memory reduction.
<!-- END:allocation-report:provenance -->
Directory discovery uses recorded advertisements after #4424; this study separates it from completed tasks that materialize canonical handlers. Context is unchanged.
## Change and compatibility
Replace 1,247 argument-free `z.number().int()` constructions with `z.int()` across 175 existing source files. Keep current shared schema owners, refinements, bounds, defaults and metadata. AST-token comparison against the baseline, after normalizing this construction and formatting, is equal for every touched source file. No new tools, provider effects, caches or response contracts.
The locked Zod 4.4.3 chained form retains a `ZodNumber` with a separate `ZodNumberFormat` check; direct construction uses the format itself. Incur 0.6.1 coerces argv strings only for `ZodNumber`. Existing CLI declarations and shared-input exceptions therefore retain the chained form. The actual compiled CLI registers 160 leaf commands; 104 numeric argument/option fields have identical results/errors in 974 installed-Incur parser cases across both runtimes, each field with an accepted probe. Fields are isolated from unrelated required options and no handlers are invoked. This is consumer-boundary coverage, not a real-engine claim.
Seven dependency-boundary variants match parsed values and complete error issues for 126 cases, including fractions, nonfinite values, strings/null/undefined and unsafe integers; 14 JSON Schema projections match in draft-07/draft-2020-12. Actual SDK advertisements match byte-for-byte in both profiles before/after materialization and across roles.
## Tasks, artifacts and bounds
Reuse the shared SDK task driver and independent ELF/section/zlib/JSON oracles from [the catalog study](mcp-input-catalog.md): official Alpine v3.21 x86_64 ripgrep 14.1.1-r0 and Zellij 0.40.0-r0. All 42/422 packages and 88/1,063 edges match the independent oracles. Select serde name/version/kind, repeat exact retained-parent reads twice, verify a canonical negative-offset refusal, export/check complete Evidence and confirm the unchanged directory. `kind: null` stays unknown. Targets are not executed; dependency presence does not establish runtime use. Acquisition URLs, archive/executable/oracle hashes and complete answers are in [the receipts](mcp-integer-allocation.json). Previously acquired public bytes are rehashed, not redownloaded.
Normal cohort: three alternating baseline/candidate pairs for each task × cold/warm server compile cache × default compact/full inputs: 72 trials, plus four excluded full-task warm priming runs. Separate diagnostic cohort: three alternating pairs per profile, cold Zellij tasks, 12 trials. All pass, with identical complete parent IDs/hashes and answers.
Node 24.18.0/npm 11.16.0; fresh client/server and HOME per trial, driver compile cache disabled. Each trial is serial, pinned to one CPU, 4 GiB hard/3 GiB early stop, no swap, at most 192 tasks. Before/after checks bind clean source, all compiled JS/JSON assets (1,990 baseline / 1,990 candidate files), scripts and manifests; producer and resource-observer hashes are also bound. No OOM, resource stop or leftover payload processes. Whole-family measurements include SDK validation, extra calls, source/compiled audits and oracle checking; kernel peak includes file cache. Build/acquisition/oracle generation and priming are excluded. Warm uses a populated role/profile-specific Node cache; hit rate and OS page cache are unmeasured, and audits warm compiled-file pages.
## Normal task costs
Medians, baseline → candidate; all ranges are in the JSON. No inspector or diagnostic GC runs in this cohort.
<!-- BEGIN:allocation-report:normal -->
| Task | Cache | Input profile | Wall s | Family CPU s | Family peak MiB |
| --------- | ----- | ------------- | --------------: | --------------: | --------------: |
| directory | cold | default | 2.739 → 2.239 | 2.553 → 2.075 | 247.4 → 248.4 |
| directory | cold | full | 2.940 → 2.437 | 2.682 → 2.217 | 348.0 → 346.8 |
| directory | warm | default | 2.639 → 2.135 | 2.461 → 1.980 | 251.8 → 246.2 |
| directory | warm | full | 2.840 → 2.338 | 2.666 → 2.144 | 348.6 → 347.2 |
| ripgrep | cold | default | 7.571 → 7.169 | 7.372 → 6.918 | 798.8 → 791.1 |
| zellij | cold | default | 7.876 → 7.373 | 7.623 → 7.134 | 812.2 → 810.5 |
| ripgrep | cold | full | 7.876 → 7.475 | 7.710 → 7.295 | 803.7 → 801.2 |
| zellij | cold | full | 8.079 → 7.573 | 7.852 → 7.378 | 815.7 → 819.1 |
| ripgrep | warm | default | 7.575 → 6.977 | 7.352 → 6.810 | 815.0 → 791.6 |
| zellij | warm | default | 7.774 → 7.178 | 7.581 → 6.981 | 820.9 → 822.0 |
| ripgrep | warm | full | 7.976 → 7.875 | 7.746 → 7.679 | 812.7 → 789.3 |
| zellij | warm | full | 14.995 → 13.978 | 14.742 → 13.718 | 818.5 → 811.5 |
<!-- END:allocation-report:normal -->
### SDK span and audit separation
<!-- BEGIN:allocation-report:sdk -->
Cold/default directory audit wall medians are 1.273 s baseline / 0.782 s candidate; SDK spans are 0.944 → 0.948 s. Whole-family costs include those experimental audits. Their cause and per-span CPU are unmeasured.
SDK span from client connect through task checks and client/server close. Excludes pre/post runtime binding and preflight artifact/oracle loading. Diagnostic spans include inspector and forced GC. CPU is measured for the whole family only; per-span CPU is unmeasured.
| Task | Cache | Inputs | SDK span s, baseline → candidate |
| --------- | ----- | ------- | -------------------------------: |
| directory | cold | default | 0.944 → 0.948 |
| directory | cold | full | 1.067 → 1.102 |
| directory | warm | default | 0.888 → 0.878 |
| directory | warm | full | 1.038 → 1.042 |
| ripgrep | cold | default | 5.778 → 5.752 |
| zellij | cold | default | 5.998 → 5.991 |
| ripgrep | cold | full | 6.077 → 6.119 |
| zellij | cold | full | 6.228 → 6.209 |
| ripgrep | warm | default | 5.743 → 5.656 |
| zellij | warm | default | 5.965 → 5.788 |
| ripgrep | warm | full | 6.100 → 6.492 |
| zellij | warm | full | 11.776 → 11.529 |
<!-- END:allocation-report:sdk -->
Full-profile warm Zellij is slower than its cold condition in both roles in this cohort. Cache hit rate and the cause of this observation are unmeasured, and cold/warm groups run sequentially. These timings support neither a general cache speedup nor a causal regression attribution.
## Separate retained-heap diagnostics
The same completed SDK task launches only its owned REA process with `--inspect=127.0.0.1:0`. Four CDP calls verify the owned transport PID, read heap, force GC and read heap again. The endpoint comes from that child’s stderr; the PID is a cross-check of the owned endpoint, not sole ownership proof. The inspector socket closes before MCP shutdown. These V8 values include inspector perturbation and exclude the client; diagnostic wall/CPU/family peaks include inspector and forced-GC overhead and are not normal-task timing results.
<!-- BEGIN:allocation-report:heap -->
| Inputs | REA V8 retained MiB | Baseline range | Candidate range | Diagnostic wall s | CPU s | Family peak MiB |
| ------- | ------------------: | --------------: | --------------: | ----------------: | --------------: | --------------: |
| default | 189.835 → 180.242 | 189.823–189.855 | 180.208–180.366 | 15.490 → 14.291 | 15.246 → 14.123 | 814.1 → 786.3 |
| full | 191.260 → 181.693 | 191.165–191.304 | 181.588–181.848 | 8.979 → 8.377 | 8.769 → 8.142 | 810.3 → 824.9 |
<!-- END:allocation-report:heap -->
A mismatched PID is rejected; removing only that guard makes the negative oracle fail. Both connections in that check use the same owned REA endpoint, with no foreign-process attachment. Aggregation rejects diagnostic GC in a normal trial. Historical #4424 receipt reaggregation preserves previous metrics and adds the separately reported SDK-span fields; its aggregator source hash changes.
## Context accounting
<!-- BEGIN:allocation-report:context -->
tiktoken==0.14.0/cl100k_base; estimates, not model usage. Actual model usage is unmeasured.
| Profile | Input directory once | Directory task | ripgrep task | Zellij task |
| ------- | -------------------: | -------------: | -----------: | ----------: |
| default | 49,670 | 50,840 | 54,029 | 53,949 |
| full | 205,744 | 206,914 | 210,103 | 210,023 |
Raw SDK directory costs per list: 756,165/912,436 tokens and 2,935,148/3,587,792 bytes for default/full.
Question, explicit recipe, server instructions, one complete name/description/input-schema directory, every tool argument and text reply, plus prompt directory for discovery. This projection assumes the client retains a single unchanged directory. Raw catalog result costs include every observed tools/list result separately, with output schemas and without JSON-RPC envelopes. SDK initialization/framing, reasoning and actual model usage are unmeasured. The analysis task includes retained-read, rejection, export/oracle and post-materialization directory checks.
<!-- END:allocation-report:context -->
## Validation and reproduction
- Both runtime builds and all six static checks plus architecture guards pass. Before the final main synchronization, 5,972 source/contract/public CLI/MCP tests pass across 323 files; four skip for unavailable/guarded coverage. After synchronization, both builds/static checks repeat successfully, 77 affected producer and CLI/MCP tests pass, and all 974 registered numeric-parameter parser cases match again. Includes real SQLite CLI/MCP record/column selection and JavaScript heap/deadline preservation. A first direct schema-validity invocation lacked generated catalog metadata; the supported focused entry generates it and all 15 cases pass. A later affected-test invocation selected two nonexistent paths; the corrected three-file producer/consumer run is the 77-case result above. Neither setup failure was counted as a passing test, and no production or timeout change was needed.
- All 84 measured tasks and four warm priming tasks pass independent oracles, canonical refusal, parent/directory identity and owned cleanup checks. This does not verify external Hopper/Ghidra/IDA, Android devices, all providers or foreign OS behavior. The earlier macOS numbers in #4360 describe a different revision/platform and are not combined with these results.
Build the pinned baseline/candidate in separate clean worktrees with the locked toolchain. Use the candidate’s producer scripts for both roles. `measurement_inputs` in the JSON records the complete matrix, questions/recipes, runtime bindings and producer hashes; extract it unchanged as the bindings file. Historical receipts require their matching producer revision. One normal task:
```bash
NODE_DISABLE_COMPILE_CACHE=1 node experiments/analysis-efficiency/mcp-input-catalog.mjs \
--runtime "$CANDIDATE_RUNTIME" --bindings "$MEASUREMENT_INPUTS" --role candidate \
--artifact "$RIPGREP_BINARY" --oracle "$RIPGREP_ORACLE" \
--output "$NEW_CASE_DIRECTORY" --profile default --phase analysis --cache cold
```
Run each task through `resource-run.py --observe-payload-exit --` to obtain whole-family counters. For warm runs replace `--cache cold` with `--cache warm --cache-root <role/profile cache>` after one complete priming task. For the separate Zellij diagnostic tasks add `--diagnostic-heap` and use cold cache. Keep their case names/logs aligned with `measurement_inputs`; aggregate both cohorts with:
```bash
python experiments/analysis-efficiency/mcp-startup-costs.py "$RECEIPT_ROOT" \
--output mcp-integer-allocation.json
```
The aggregator requires the recorded tiktoken version, exact catalog/parent/runtime/producer/guard identities, clean source, successful payload cleanup and matching matrix coverage. Default driver behavior continues without an inspector; the heap flag is confined to the experiment.
Regenerate or verify every numerical section directly from the committed JSON:
```bash
node experiments/analysis-efficiency/mcp-integer-report.mjs
node experiments/analysis-efficiency/mcp-integer-report.mjs --check
```
The documentation conformance gate checks current receipts, rejects stale normal/heap tables, and refuses missing producer markers without writing.
@@ -0,0 +1,212 @@
// One producer for the numerical sections of the committed allocation report.
import assert from "node:assert/strict";
import { readFile, writeFile } from "node:fs/promises";
import { parseArgs } from "node:util";
const { values } = parseArgs({
options: {
data: { type: "string" },
markdown: { type: "string" },
check: { type: "boolean", default: false },
},
});
const dataPath =
values.data ?? new URL("mcp-integer-allocation.json", import.meta.url);
const markdownPath =
values.markdown ?? new URL("mcp-integer-allocation.md", import.meta.url);
const data = JSON.parse(await readFile(dataPath, "utf8"));
assert.equal(data.status, "passed");
const original = await readFile(markdownPath, "utf8");
const fixed = (value, digits = 3) => {
assert.ok(Number.isFinite(value));
return value.toFixed(digits);
};
const mib = (value, digits = 1) => fixed(value / 2 ** 20, digits);
const median = (values) => {
assert.ok(values.length > 0);
const sorted = [...values].sort((a, b) => a - b);
const middle = Math.floor(sorted.length / 2);
return sorted.length % 2
? sorted[middle]
: (sorted[middle - 1] + sorted[middle]) / 2;
};
const table = (headers, rows, right = []) =>
[
`| ${headers.join(" | ")} |`,
`| ${headers.map((_, index) => (right.includes(index) ? "---:" : "---")).join(" | ")} |`,
...rows.map((row) => `| ${row.join(" | ")} |`),
].join("\n");
const pair = (item, key, format = fixed) =>
`${format(item.baseline[key].median)} → ${format(item.candidate[key].median)}`;
const heapMetric = "retained_v8_heap_bytes_after_gc";
const heaps = data.heap_diagnostics.summary;
const changes = heaps.map(
(item) =>
(item.baseline[heapMetric].median - item.candidate[heapMetric].median) /
2 ** 20,
);
const changesText = [...new Set(changes.map((value) => fixed(value, 1)))].join(
"–",
);
const separated = heaps.every(
(item) => item.baseline[heapMetric].min > item.candidate[heapMetric].max,
);
const provenance = `Baseline \`main@${data.measurement_inputs.baseline_revision}\`; candidate \`${data.measurement_inputs.candidate_revision}\`. This is a pinned comparison.\n\nIn separate inspector/forced-GC diagnostics, retained REA V8 heap decreases by about ${changesText} MiB. ${separated ? "The three-pair ranges do not overlap in either profile." : "Ranges overlap; inspect the observations below."} Inspector perturbation is included. Normal task timing, CPU and peak memory are reported separately; these observations do not establish a general latency or peak-memory reduction.`;
const normal = table(
[
"Task",
"Cache",
"Input profile",
"Wall s",
"Family CPU s",
"Family peak MiB",
],
data.summary.map((item) => [
item.artifact ?? "directory",
item.cache,
item.profile,
pair(item, "wall_seconds"),
pair(item, "cpu_seconds"),
pair(item, "kernel_family_peak_bytes_including_page_cache", mib),
]),
[3, 4, 5],
);
const discovery = data.summary.find(
(item) =>
item.phase === "discovery" &&
item.cache === "cold" &&
item.profile === "default",
);
assert.ok(discovery);
const audit = (role) =>
median(
data.records
.filter(
(row) =>
row.phase === "discovery" &&
row.cache === "cold" &&
row.profile === "default" &&
row.role === role,
)
.map(
(row) =>
(row.runtime_audit_ms.before + row.runtime_audit_ms.after) / 1000,
),
);
const sdk =
`Cold/default directory audit wall medians are ${fixed(audit("baseline"))} s baseline / ${fixed(audit("candidate"))} s candidate; SDK spans are ${pair(discovery, "sdk_task_elapsed_seconds")} s. Whole-family costs include those experimental audits. Their cause and per-span CPU are unmeasured.\n\n${data.sdk_task_elapsed_scope}\n\n` +
table(
["Task", "Cache", "Inputs", "SDK span s, baseline → candidate"],
data.summary.map((item) => [
item.artifact ?? "directory",
item.cache,
item.profile,
pair(item, "sdk_task_elapsed_seconds"),
]),
[3],
);
const heap = table(
[
"Inputs",
"REA V8 retained MiB",
"Baseline range",
"Candidate range",
"Diagnostic wall s",
"CPU s",
"Family peak MiB",
],
heaps.map((item) => [
item.profile,
pair(item, heapMetric, (value) => mib(value, 3)),
...["baseline", "candidate"].map(
(role) =>
`${mib(item[role][heapMetric].min, 3)}–${mib(item[role][heapMetric].max, 3)}`,
),
pair(item, "wall_seconds"),
pair(item, "cpu_seconds"),
pair(item, "kernel_family_peak_bytes_including_page_cache", mib),
]),
[1, 2, 3, 4, 5, 6],
);
const integer = (value) => new Intl.NumberFormat("en-US").format(value);
const tokens = (profile, phase, artifact) => {
const values = data.records
.filter(
(row) =>
row.profile === profile &&
row.phase === phase &&
row.artifact === artifact,
)
.map((row) => row.tokens_estimated.whole_task);
assert.ok(values.length > 0);
return [...new Set([Math.min(...values), Math.max(...values)])]
.map(integer)
.join("–");
};
const catalogs = data.catalogs;
const context =
`${data.tokenizer}. Actual model usage is unmeasured.\n\n` +
table(
[
"Profile",
"Input directory once",
"Directory task",
"ripgrep task",
"Zellij task",
],
["default", "full"].map((profile) => [
profile,
integer(catalogs[profile].input_catalog_tokens),
tokens(profile, "discovery", null),
tokens(profile, "analysis", "ripgrep"),
tokens(profile, "analysis", "zellij"),
]),
[1, 2, 3, 4],
) +
`\n\nRaw SDK directory costs per list: ${integer(catalogs.default.raw_catalog_tokens)}/${integer(catalogs.full.raw_catalog_tokens)} tokens and ${integer(catalogs.default.raw_catalog_bytes)}/${integer(catalogs.full.raw_catalog_bytes)} bytes for default/full.\n\n${data.context_scope}`;
const sections = { provenance, normal, sdk, heap, context };
// oxfmt may pad table cells and separator dashes; neither changes report facts.
const normalized = (value) =>
value
.trim()
.replace(/\s+/gu, " ")
.replace(/\s*\|\s*/gu, "|")
.replace(
/(?<=\|)(:)?-{3,}(:)?(?=\|)/gu,
(_, left, right) => `${left ?? ""}---${right ?? ""}`,
);
let rendered = original;
const changed = [];
for (const [name, content] of Object.entries(sections)) {
const begin = `<!-- BEGIN:allocation-report:${name} -->`;
const end = `<!-- END:allocation-report:${name} -->`;
assert.equal(
rendered.split(begin).length,
2,
`Expected one opening report marker: ${name}`,
);
assert.equal(
rendered.split(end).length,
2,
`Expected one closing report marker: ${name}`,
);
const start = rendered.indexOf(begin) + begin.length;
const finish = rendered.indexOf(end);
assert.ok(finish > start, `Invalid report marker order: ${name}`);
if (normalized(rendered.slice(start, finish)) !== normalized(content)) {
changed.push(name);
rendered =
rendered.slice(0, start) + `\n\n${content}\n\n` + rendered.slice(finish);
}
}
if (values.check) {
assert.equal(
changed.length,
0,
`Generated allocation report differs: ${changed.join(", ")}`,
);
console.log("Allocation report matches committed receipt data");
} else if (changed.length) {
await writeFile(markdownPath, rendered);
console.log(`Updated allocation report sections: ${changed.join(", ")}`);
}
@@ -0,0 +1,107 @@
// Diagnostic only: inspect the explicitly launched REA process after its task.
import assert from "node:assert/strict";
export const sampleRetainedHeap = async (endpoint, expectedPid) => {
const url = new URL(endpoint);
assert.equal(url.protocol, "ws:");
assert.equal(url.hostname, "127.0.0.1");
assert.ok(Number.isSafeInteger(expectedPid) && expectedPid > 0);
const socket = new WebSocket(url);
let sequence = 0;
const pending = new Map();
const closed = new Promise((resolve) => {
socket.addEventListener("close", resolve, { once: true });
});
const rejectPending = (cause) => {
for (const request of pending.values()) {
clearTimeout(request.timer);
request.reject(cause);
}
pending.clear();
};
socket.addEventListener("error", () =>
rejectPending(new Error("Owned REA inspector connection failed")),
);
socket.addEventListener("close", () =>
rejectPending(new Error("Owned REA inspector connection closed")),
);
socket.addEventListener("message", (event) => {
try {
const reply = JSON.parse(event.data);
const request = pending.get(reply.id);
if (request === undefined) return;
pending.delete(reply.id);
clearTimeout(request.timer);
if (reply.error !== undefined)
request.reject(new Error(JSON.stringify(reply.error)));
else request.resolve(reply.result);
} catch (cause) {
rejectPending(cause);
}
});
const withinDeadline = async (promise) => {
let timer;
try {
return await Promise.race([
promise,
new Promise((_, reject) => {
timer = setTimeout(
() => reject(new Error("Owned REA inspector deadline exceeded")),
10_000,
);
}),
]);
} finally {
clearTimeout(timer);
}
};
const call = (method, params = {}) =>
new Promise((resolve, reject) => {
const id = ++sequence;
const timer = setTimeout(() => {
pending.delete(id);
reject(new Error(`Owned REA inspector timed out: ${method}`));
}, 10_000);
pending.set(id, { resolve, reject, timer });
try {
socket.send(JSON.stringify({ id, method, params }));
} catch (cause) {
pending.delete(id);
clearTimeout(timer);
reject(cause);
}
});
try {
await withinDeadline(
new Promise((resolve, reject) => {
socket.addEventListener("open", resolve, { once: true });
socket.addEventListener("error", reject, { once: true });
socket.addEventListener("close", reject, { once: true });
}),
);
const identity = await call("Runtime.evaluate", {
expression: "process.pid",
returnByValue: true,
});
assert.equal(identity.result.value, expectedPid);
const before = await call("Runtime.getHeapUsage");
await call("HeapProfiler.collectGarbage");
const after = await call("Runtime.getHeapUsage");
for (const sample of [before, after]) {
assert.ok(Number.isSafeInteger(sample.usedSize) && sample.usedSize >= 0);
assert.ok(Number.isSafeInteger(sample.totalSize));
assert.ok(sample.totalSize >= sample.usedSize);
}
return {
scope:
"Owned REA V8 heap after the complete SDK task; inspector and forced GC are diagnostic overhead",
pid_verified: true,
before_gc: before,
after_gc: after,
};
} finally {
rejectPending(new Error("Owned REA inspector diagnostic finished"));
socket.close();
await withinDeadline(closed);
}
};
@@ -0,0 +1,151 @@
"""Verify paired SDK startup/analysis receipts and account for the whole task."""
import argparse
import hashlib
import importlib.util
import json
import math
import statistics
from pathlib import Path
import tiktoken
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('root', type=Path)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
root = args.root
input_bytes = (root / 'measurement-inputs.json').read_bytes()
inputs = json.loads(input_bytes)
expected = inputs['runtime_bindings']
sha = lambda value: hashlib.sha256(value).hexdigest()
compact = lambda value: json.dumps(value, ensure_ascii=False, separators=(',', ':'), allow_nan=False)
encoding = tiktoken.get_encoding('cl100k_base')
count = lambda value: len(encoding.encode(value, disallowed_special=()))
for role in ['baseline', 'candidate']:
assert expected[role]['git_revision'] == inputs[f'{role}_revision']
assert expected[role]['source_diff_sha256'] == sha(b'')
assert expected[role]['untracked_source_sha256'] == sha(b'[]')
assert expected['baseline']['root'] != expected['candidate']['root']
assert expected['baseline']['compiled_runtime_sha256'] != expected['candidate']['compiled_runtime_sha256']
spec = importlib.util.spec_from_file_location('catalog_costs', Path(__file__).with_name('mcp-input-catalog-costs.py'))
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
catalogs = {}
identities = {}
rows = []
heap_rows = []
for case in inputs['cases'] + inputs.get('diagnostic_cases', []):
directory = root / case['case']
report = json.loads((directory / 'report.json').read_text())
assert report['status'] == 'passed' and report['cleanup_verified']
assert report['runtime_binding_verified']
assert report['measurement_inputs_sha256'] == sha(input_bytes)
for name, producer_digest in report['producer_sources_sha256'].items():
assert producer_digest == inputs['harness_sources_sha256'][name]
assert report['runtime_binding'] == expected[case['role']]
assert type(report['elapsed_ms']) in [int, float] and math.isfinite(report['elapsed_ms']) and report['elapsed_ms'] >= 0
for key in ['role', 'cache', 'profile', 'phase']:
assert report[key] == case[key]
raw = (directory / 'catalog.json').read_bytes()
catalog_digest = sha(raw)
assert catalog_digest == report['catalog_sha256']
profile = case['profile']
if profile not in catalogs:
catalog = json.loads(raw)
definitions = [{key: tool[key] for key in ['name', 'description', 'inputSchema']} for tool in catalog['tools']]
catalogs[profile] = {'sha256': catalog_digest, 'tool_count': len(catalog['tools']),
'input_catalog_tokens': count(compact(definitions)),
'raw_catalog_tokens': count(raw.decode()), 'raw_catalog_bytes': len(raw)}
assert catalog_digest == catalogs[profile]['sha256'], 'Directory differs across roles or phases'
transcript = json.loads((directory / 'transcript.json').read_text())
server = json.loads((directory / 'server.json').read_text())
tokens = {'question_and_recipe': count(inputs['questions'][case['phase']] + '\n' + inputs['recipes'][case['phase']]),
'server_instructions': count(server.get('instructions') or ''),
'input_catalog_once': catalogs[profile]['input_catalog_tokens'],
'arguments': sum(count(compact({'name': call['name'], 'arguments': call['arguments']})) for call in transcript),
'text_replies': sum(count(block['text']) for call in transcript for block in call['reply']['content'] if block['type'] == 'text')}
requests = report['protocol_requests']
assert requests['initialize'] == 1
if case['phase'] == 'analysis':
assert len(transcript) == 5 and requests['tools/call'] == 5 and requests['tools/list'] == 2
assert report['artifact_sha256'] == inputs['artifacts'][case['artifact']]['sha256']
identity = (report['parent_evidence_id'], report['parent_semantic_sha256'], sha(compact(report['answer']).encode()))
assert identities.setdefault(case['artifact'], identity) == identity
assert sum(call['is_error'] for call in report['calls']) == 1
else:
assert not transcript and requests.get('tools/call', 0) == 0
assert requests['tools/list'] == 1 and requests['prompts/list'] == 1
prompts = (directory / 'prompts.json').read_bytes()
assert sha(prompts) == report['prompts_sha256']
assert identities.setdefault('prompts', report['prompts_sha256']) == report['prompts_sha256']
tokens['prompt_directory'] = count(prompts.decode())
tokens['whole_task'] = sum(tokens.values())
resource = module.resource(directory)
for name, guard_digest in resource['guard_sources_sha256'].items():
assert guard_digest == inputs['harness_sources_sha256'][name]
del resource['guard_sources_sha256']
row = {**case, 'runtime_binding_sha256': sha(compact(report['runtime_binding']).encode()), 'runtime_audit_ms': report['runtime_audit_ms'],
'sdk_task_elapsed_seconds': report['elapsed_ms'] / 1000,
'catalog_sha256': catalog_digest, 'parent_evidence_id': report.get('parent_evidence_id'),
'parent_semantic_sha256': report.get('parent_semantic_sha256'),
'answer_sha256': sha(compact(report['answer']).encode()), 'tokens_estimated': tokens,
'raw_catalog_result_tokens_all_lists': catalogs[profile]['raw_catalog_tokens'] * requests['tools/list'],
'raw_catalog_result_bytes_all_lists': catalogs[profile]['raw_catalog_bytes'] * requests['tools/list'],
'actual_model_usage': None, 'protocol_era': report['protocol_era'], 'protocol_requests': requests, 'calls': report['calls'],
**resource,
'receipt_sha256': {name: sha((directory / name).read_bytes()) for name in ['report.json', 'catalog.json', 'server.json', 'transcript.json']}}
assert row['catalog_sha256'] == row['receipt_sha256']['catalog.json'] == catalogs[profile]['sha256']
heap = report.get('diagnostic_heap')
if case.get('diagnostic_heap'):
assert case['phase'] == 'analysis' and case['cache'] == 'cold' and case['artifact'] == 'zellij'
assert heap is not None and heap['pid_verified'] is True
for sample in [heap['before_gc'], heap['after_gc']]:
assert type(sample['usedSize']) is int and sample['usedSize'] >= 0
assert type(sample['totalSize']) is int and sample['totalSize'] >= sample['usedSize']
row['heap_observation'] = heap
row['retained_v8_heap_bytes_after_gc'] = heap['after_gc']['usedSize']
heap_rows.append(row)
else:
assert heap is None, 'Diagnostic GC contaminated a normal task trial'
rows.append(row)
assert len(rows) == 72
summary = []
for phase in ['discovery', 'analysis']:
for cache in ['cold', 'warm']:
for profile in ['default', 'full']:
for artifact in ([None] if phase == 'discovery' else ['ripgrep', 'zellij']):
item = {'phase': phase, 'cache': cache, 'profile': profile, 'artifact': artifact}
for role in ['baseline', 'candidate']:
selected = [row for row in rows if all(row[key] == item[key] for key in ['phase', 'cache', 'profile', 'artifact']) and row['role'] == role]
assert len(selected) == 3
item[role] = {metric: {'median': statistics.median(row[metric] for row in selected),
'min': min(row[metric] for row in selected), 'max': max(row[metric] for row in selected)}
for metric in ['wall_seconds', 'cpu_seconds', 'kernel_family_peak_bytes_including_page_cache', 'sdk_task_elapsed_seconds']}
item[role]['tokens_estimated'] = [row['tokens_estimated']['whole_task'] for row in selected]
summary.append(item)
result = {'status': 'passed', 'tokenizer': f'tiktoken=={tiktoken.__version__}/cl100k_base; estimates, not model usage',
'aggregation_sources_sha256': {path.name: sha(path.read_bytes()) for path in [Path(__file__), Path(spec.origin)]},
'context_scope': 'Question, explicit recipe, server instructions, one complete name/description/input-schema directory, every tool argument and text reply, plus prompt directory for discovery. This projection assumes the client retains a single unchanged directory. Raw catalog result costs include every observed tools/list result separately, with output schemas and without JSON-RPC envelopes. SDK initialization/framing, reasoning and actual model usage are unmeasured. The analysis task includes retained-read, rejection, export/oracle and post-materialization directory checks.',
'cache_scope': 'Cold disables server Node compile cache. Warm enables a prepopulated role/profile-specific cache after one complete analysis priming run. Fresh server/client processes and HOME each trial. Driver compile cache disabled in both cohorts. Cache hit rate and OS page cache are unmeasured; source/runtime audits warm compiled-file pages before startup.',
'resource_scope': 'Whole SDK driver, resource observer and owned REA process family. Includes before/after source-and-all-compiled-JS/JSON binding, SDK schema checks, extra validation calls, bundle export and independent oracle checking. Kernel cgroup peak includes file cache. Build/download/oracle preparation and cache priming excluded. Retained V8 heap and forced-GC diagnostics were not separately measured.',
'measurement_inputs': inputs, 'catalogs': catalogs, 'summary': summary, 'records': rows,
'sdk_task_elapsed_scope': 'SDK span from client connect through task checks and client/server close. Excludes pre/post runtime binding and preflight artifact/oracle loading. Diagnostic spans include inspector and forced GC. CPU is measured for the whole family only; per-span CPU is unmeasured.',
'warmup_receipts': {name: sha((root / name / 'report.json').read_bytes()) for name in [f'warmup-{profile}-{role}' for profile in ['default', 'full'] for role in ['baseline', 'candidate']]}}
if heap_rows:
assert len(heap_rows) == 12
heap_summary = []
for profile in ['default', 'full']:
item = {'profile': profile}
for role in ['baseline', 'candidate']:
selected = [row for row in heap_rows if row['profile'] == profile and row['role'] == role]
assert len(selected) == 3
item[role] = {metric: {'median': statistics.median(row[metric] for row in selected),
'min': min(row[metric] for row in selected), 'max': max(row[metric] for row in selected)}
for metric in ['retained_v8_heap_bytes_after_gc', 'wall_seconds', 'cpu_seconds', 'sdk_task_elapsed_seconds',
'kernel_family_peak_bytes_including_page_cache']}
heap_summary.append(item)
result['heap_diagnostics'] = {
'scope': 'Separate three-pair Zellij tasks per profile, with an explicitly launched loopback inspector. Four CDP calls verify the owned REA PID, read heap, force GC and read heap again after the completed SDK task. These whole-family costs include inspector and diagnostic GC overhead; they are excluded from the normal timing cohort. Heap values describe REA V8 only, not the client or the whole process family.',
'summary': heap_summary, 'records': heap_rows}
args.output.write_text(json.dumps(result, indent=2) + '\n')
print(json.dumps(summary, indent=2))
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,61 @@
# MCP discovery and complete Rust tasks
Target-free stdio discovery previously constructed the canonical analysis runtime before the client could inspect its directory. The candidate records SDK advertisements at build time for default/explicit compact and explicit full inputs. Its first tool, prompt or completion request materializes canonical validators and handlers on the same SDK server. HTTP, output-schema-off, initial-target and unrecorded source runs retain eager composition.
Pinned baseline: [`main@c465ff5a8`](https://github.com/morluto/rea/commit/c465ff5a814b8e1a386fc547cbcd94df5c35f086). Measured candidate: [`04900abbc`](https://github.com/leeguooooo/rea/commit/04900abbcd3c3438fce4af932ff28400fe744a7d). Both use the same main base and locked dependencies. Before/after source and all compiled JS/JSON bindings identify the exact runtime used in every trial. The input manifest and producer scripts are also hashed and checked.
## Real tasks and independent facts
Official Alpine v3.21 packages supply ripgrep 14.1.1-r0 (3,416,168-byte executable) and Zellij 0.40.0-r0 (17,151,864 bytes). Their download URLs, archive/executable digests and aports revisions are retained in [the measurement data](mcp-startup.json). The independent Python ELF/section/zlib/JSON oracle checks 42 packages and 88 dependency edges for ripgrep, and 422 packages and 1,063 edges for Zellij. Serde versions are 1.0.210 and 1.0.137 respectively; the selected package kind remains explicit unknown (`null`). Verification covers embedded static build metadata; execution and runtime reachability remain unverified.
Each analysis task initializes the actual SDK client, lists and validates the input directory, selects serde name/version/kind, reads the exact retained parent twice, verifies canonical rejection of a negative offset, exports and independently checks the complete package/edge inventory, confirms unchanged advertisements after materialization, then closes. All five tool calls and both directory reads are included in the measured task. Discovery tasks initialize, list tools/prompts, validate the input directory and close without an analysis call.
All 72 paired trials and four priming runs passed. Each artifact has identical answers, parent Evidence IDs and complete semantic hashes across roles, profiles and cache modes. Both profiles retain byte-identical directories across baseline/candidate and before/after canonical materialization. The directory contains 213 tools. Complete Evidence is retained and exported through the existing shared workflow.
## Measured cost
Three alternating pairs per condition; table values are medians. Min/max ranges and individual receipts are in [mcp-startup.json](mcp-startup.json). Each cell is baseline → candidate. Kernel process-family peak includes file cache, SDK driver, observer and REA children.
| Task | Cache | Inputs | Wall seconds | Family CPU seconds | Family peak MiB |
| --------- | ----- | ----------------- | ------------: | -----------------: | --------------: |
| Directory | cold | compact (default) | 5.661 → 2.236 | 5.403 → 2.050 | 559.3 → 248.6 |
| Directory | cold | full | 5.256 → 2.337 | 5.040 → 2.175 | 650.9 → 349.1 |
| Directory | warm | compact (default) | 5.056 → 2.236 | 4.800 → 2.015 | 572.8 → 251.4 |
| Directory | warm | full | 4.957 → 2.338 | 4.754 → 2.139 | 672.7 → 349.6 |
| ripgrep | cold | compact (default) | 8.177 → 8.078 | 7.945 → 7.899 | 849.7 → 792.5 |
| zellij | cold | compact (default) | 8.276 → 7.276 | 8.041 → 7.105 | 846.2 → 819.0 |
| ripgrep | cold | full | 8.180 → 7.483 | 8.000 → 7.312 | 846.5 → 794.4 |
| zellij | cold | full | 8.484 → 7.776 | 8.284 → 7.514 | 850.4 → 813.1 |
| ripgrep | warm | compact (default) | 7.976 → 6.879 | 7.739 → 6.678 | 826.8 → 802.6 |
| zellij | warm | compact (default) | 8.285 → 7.472 | 8.019 → 7.254 | 882.1 → 811.4 |
| ripgrep | warm | full | 8.074 → 7.476 | 7.890 → 7.265 | 852.1 → 803.6 |
| zellij | warm | full | 8.382 → 7.479 | 8.151 → 7.293 | 854.3 → 830.0 |
Discovery improves clearly in this cohort. Completed-task changes depend on the workload: default cold ripgrep is approximately unchanged. Initialization moves into the first actual analysis request; its latency must be assessed with the full workflow.
| Default ripgrep first RPC | Baseline ms | Candidate ms |
| ------------------------- | ----------: | -----------: |
| cold | 231.5 | 2532.1 |
| warm | 227.1 | 2118.2 |
Within each profile, advertised context remains identical. Input-directory estimates are 49,670 tokens for default compact and 205,744 for full. Whole-task estimates include the supplied question/recipe, server instructions, directory once, every tool argument and text reply, and the prompt directory in discovery tasks. This projection assumes the client retains one unchanged directory; raw catalog result costs count every observed list result separately, including output schemas. These use tiktoken 0.14.0/cl100k_base; actual model usage, reasoning and SDK initialization/framing are unmeasured.
Linux x64 / AMD EPYC 7B12 / kernel 6.1.0-54-cloud-amd64 / Node 24.18.0, npm 11.16.0. Timed SDK tasks negotiated the legacy protocol era; auto/modern correctness is covered separately by compiled boundary tests. Trials run serially under a 1-CPU quota pinned to CPU 15, 4 GiB hard memory / 3 GiB early stop, no swap and 192 tasks. No resource stop, OOM or remaining payload process was observed. Cold disables server Node compile cache; warm uses a role/profile-specific cache populated by one complete task. Fresh processes and HOME are used each trial. Cache hit rate and OS page cache are unmeasured. Runtime audits warm compiled-file pages and are included in whole-task costs. Builds, downloads, oracle generation and cache priming are preparation outside those costs. Retained V8 heap and forced-GC measurements were not separately collected.
## Ownership and validation
`OwnedMcpServers` seals publication before taking its close snapshot, keeps failed exact close capabilities for retry and serves both eager and deferred paths. Deferred startup still joins servers published before an asynchronous serving handle settles. Post-snapshot and post-close factory calls are refused before allocation. Both production-boundary regressions fail with only the shared guard removed and pass restored.
The moved transport/delivery implementation preserves current complete-error retention, escaped-preview fitting, response/string-limit recovery, target-content provenance and native Evidence continuations. Canonical parsers remain authoritative after lazy binding. Both protocol eras, input/output profile choices, invalid input/output, completion cancellation, partial startup and cleanup retry are exercised.
Aggregation keeps catalog, producer-script and guard-script digests in distinct variables. Before publishing a trial it verifies that the catalog digest matches both its catalog-file receipt and the shared profile identity. Deliberately selecting the guard digest for that field is rejected. Regeneration preserves all measured timing, CPU, memory, context and parent-Evidence values.
Executed local gates: 125 focused tests across 14 files; a repeated 22-case schema/profile subset after the pinned main merge; all six static-check tasks plus architecture guards; builds for both pinned runtimes; six documentation checks and five README consistency tests. HTTP coverage includes the real compiled CLI session/replacement/Host tests. Runtime-binding admission rejects a mismatched compiled graph before MCP launch; aggregation rejects a tampered trial binding. The raw receipts remain in `/tmp/rea-perf-mcp-startup-20261022`, identified by hashes in the data. Real initial-target providers, foreign OS runtime behavior and other providers remain outside this measurement.
## Reproduction
Use the pinned Node/npm toolchain and separate clean baseline/candidate checkouts. Build each with `npm ci` and `npm run build:cached` (the measured workspace shared one locked dependency installation). Acquire and verify the official packages listed in the data, extract the named executables and generate independent oracles with `mcp-input-catalog-oracle.py`.
Capture each checkout with `node experiments/analysis-efficiency/managed-cli-runtime.mjs CHECKOUT mcp`. Construct `measurement-inputs.json` using the recorded schema, updating absolute checkout/artifact/oracle paths and their independently verified bindings. Keep it immutable through a cohort; producer source digests cover `mcp-input-catalog.mjs`, `managed-cli-runtime.mjs`, `resource-run.py` and `resource_payload.py`.
Run `mcp-input-catalog.mjs` through `resource-run.py --observe-payload-exit` with `--runtime`, `--output`, `--bindings`, `--role baseline|candidate`, `--profile default|full`, `--phase discovery|analysis` and `--cache cold|warm`. Analysis also takes `--artifact` and `--oracle`; warm requires a role/profile-specific `--cache-root`. Disable the driver compile cache for both cohorts. Prime each warm role/profile once with the complete ripgrep task. Follow the 72-case order in the data, reversing role order in every second pair. Aggregate with `python mcp-startup-costs.py LAB --output mcp-startup.json` using tiktoken 0.14.0. The aggregator verifies every input/runtime/guard binding and completed receipt before producing summaries.
+2 -2
View File
@@ -305,8 +305,8 @@
"verify:solidity:metadata": "npm run build:cached && node scripts/verify-solidity-metadata.mjs",
"verify:recorded:crash": "npm run build:cached && node scripts/verify-recorded-crash.mjs",
"mcp-catalog:check:built": "node scripts/generate-mcp-tool-catalog.mjs --check",
"compile": "node scripts/clean-build-output.mjs && node scripts/run-typescript.mjs -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/bundle-runtime.mjs",
"compile:termux": "node scripts/clean-build-output.mjs && node --max-old-space-size=3072 node_modules/typescript-termux/lib/tsc.js -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/bundle-runtime.mjs",
"compile": "node scripts/clean-build-output.mjs && node scripts/run-typescript.mjs -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/record-mcp-catalog.mjs && node scripts/bundle-runtime.mjs",
"compile:termux": "node scripts/clean-build-output.mjs && node --max-old-space-size=3072 node_modules/typescript-termux/lib/tsc.js -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/record-mcp-catalog.mjs && node scripts/bundle-runtime.mjs",
"artifacts:mcp-catalog": "node scripts/generate-mcp-tool-catalog.mjs",
"artifacts:skills": "node scripts/generate-skill-metadata.mjs",
"artifacts:product-catalog": "node scripts/generate-product-catalog.mjs",
+40 -14
View File
@@ -1,6 +1,31 @@
import type { NormalizedWorkflowRun } from "./workflow-run.mjs";
import type { StaticTaskContext } from "./static-task-context.mjs";
export type { StaticTaskContext } from "./static-task-context.mjs";
export interface CiRun {
readonly id?: string | number;
readonly run_attempt?: number;
readonly status?: string;
readonly conclusion?: string | null;
readonly event?: string;
readonly path?: string;
readonly head_sha?: string;
readonly head_branch?: string;
readonly repository?: { readonly full_name?: string };
readonly pull_requests?: readonly {
readonly base?: {
readonly sha?: string;
readonly repo?: { readonly full_name?: string };
};
}[];
}
export interface CiJob {
readonly name?: string;
readonly status?: string;
readonly conclusion?: string | null;
readonly run_id?: number;
readonly run_attempt?: number;
}
export interface StaticDiagnostic {
readonly kind: "lint" | "jscpd";
@@ -11,15 +36,20 @@ export interface StaticDiagnostic {
readonly severity?: string;
}
export interface StaticEvidence {
readonly context: StaticTaskContext;
readonly diagnostics: readonly StaticDiagnostic[];
}
export function classifyFailure(input: {
repository: string;
defaultBranch: string;
run: Record<string, unknown> & NormalizedWorkflowRun;
job: Record<string, unknown>;
run: CiRun & NormalizedWorkflowRun;
job: CiJob;
diagnostics: readonly StaticDiagnostic[];
context: StaticTaskContext;
mainRun: Record<string, unknown> & NormalizedWorkflowRun;
mainJob: Record<string, unknown>;
mainRun: CiRun & NormalizedWorkflowRun;
mainJob: CiJob;
mainDiagnostics: readonly StaticDiagnostic[];
mainContext: StaticTaskContext;
executionSourcesVerified: boolean;
@@ -47,7 +77,7 @@ export function safeSha(value: unknown): string | null;
export function validRepoPath(value: unknown): value is string;
export function selectStaticCheckArtifact(
page: unknown,
run: { id?: string | number; head_sha?: string },
run: Pick<CiRun, "id" | "head_sha">,
runAttempt: number,
name?: string,
): unknown | null;
@@ -57,12 +87,8 @@ export function readStaticCheckArtifact(options: {
artifactId: number | string;
token: string;
fetchImpl?: typeof fetch;
}): Promise<{
context: StaticTaskContext;
diagnostics: readonly StaticDiagnostic[];
}>;
}): Promise<StaticEvidence>;
export function parseStaticArchive(archivePath: string): Promise<{
context: StaticTaskContext;
diagnostics: readonly StaticDiagnostic[];
}>;
export function parseStaticArchive(
archivePath: string,
): Promise<StaticEvidence>;
+1 -53
View File
@@ -1,38 +1,5 @@
import type { NormalizedWorkflowRun } from "./workflow-run.mjs";
import type { StaticDiagnostic } from "./artifacts.mjs";
export type { StaticDiagnostic } from "./artifacts.mjs";
import type { CiRun, CiJob, StaticEvidence } from "./artifacts.mjs";
import type { StaticTaskContext } from "./static-task-context.mjs";
export type { StaticTaskContext } from "./static-task-context.mjs";
export interface CiRun {
readonly id?: string | number;
readonly run_attempt?: number;
readonly status?: string;
readonly conclusion?: string | null;
readonly event?: string;
readonly path?: string;
readonly head_sha?: string;
readonly head_branch?: string;
readonly repository?: { readonly full_name?: string };
readonly pull_requests?: readonly {
readonly base?: {
readonly sha?: string;
readonly repo?: { readonly full_name?: string };
};
}[];
}
export interface CiJob {
readonly name?: string;
readonly status?: string;
readonly conclusion?: string | null;
readonly run_id?: number;
readonly run_attempt?: number;
}
export interface StaticEvidence {
readonly context: StaticTaskContext;
readonly diagnostics: readonly StaticDiagnostic[];
}
export interface CiFailureReport {
readonly version: 1;
@@ -99,25 +66,6 @@ export interface FailureReportApi {
): Promise<Readonly<Record<string, boolean>>>;
}
export function classifyFailure(input: {
repository: string;
defaultBranch: string;
run: CiRun & NormalizedWorkflowRun;
job: CiJob;
diagnostics: readonly StaticDiagnostic[];
context: StaticTaskContext;
mainRun: CiRun & NormalizedWorkflowRun;
mainJob: CiJob;
mainDiagnostics: readonly StaticDiagnostic[];
mainContext: StaticTaskContext;
executionSourcesVerified: boolean;
ancestry: { status: string; mergeBaseSha: string | null };
blobs: Readonly<Record<string, boolean>>;
}): {
classification: "matching-main-diagnostics" | "unknown";
reason: string;
};
export function reportCiFailures(options: {
repository: string;
defaultBranch: string;
-2
View File
@@ -26,8 +26,6 @@ const MAX_MAIN_RUN_LOOKUPS = 100;
const MAX_PATHS_PER_COMPARISON = 50;
const MAX_API_REQUESTS = 250;
export { classifyFailure } from "./artifacts.mjs";
/** Uses only failed CI candidates present in the supplied bounded timing manifest. */
export async function reportCiFailures({
repository,
@@ -142,6 +142,9 @@ export const verifyScenarioPolicySetup = async (
page,
) => {
const result = {};
// Exercise a caller-selected handshake deadline within the existing teardown
// guard; the default command deadline may legitimately outlast that guard.
const setupCommandTimeoutMs = 2_000;
for (const mode of ["http-websocket", "http-discovery", "deadline"]) {
const proxy = await startProxy(endpoint, mode);
const provider = createBrowserScenarioProvider(process.env);
@@ -152,6 +155,9 @@ export const verifyScenarioPolicySetup = async (
target_id: targetId,
},
start_url: { url: origin },
...(mode === "deadline"
? { command_timeout_ms: setupCommandTimeoutMs }
: {}),
environment: { service_workers: "block" },
actions: [
{ step_id: "settle", action: "wait_for_timeout", duration_ms: 1 },
@@ -189,7 +195,9 @@ export const verifyScenarioPolicySetup = async (
if (!next.ok) throw next.error;
assert.equal(next.value.browser.cleanup, "disconnected-external");
result[mode] = {
...(mode === "deadline" ? { timed_out: true } : { cancelled: true }),
...(mode === "deadline"
? { timed_out: true, command_timeout_ms: setupCommandTimeoutMs }
: { cancelled: true }),
socket_closed: true,
target_reused: true,
};
+24 -1
View File
@@ -238,7 +238,9 @@ export const providerCatalog = (sources) => {
identity: sources.pwntoolsProvider.PWNTOOLS_PROVIDER_IDENTITY,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name !== "inspect_pe_resources",
({ name }) =>
name === "inspect_binary_layout" ||
name === "inspect_recorded_crash",
),
},
{
@@ -248,6 +250,27 @@ export const providerCatalog = (sources) => {
({ name }) => name === "inspect_pe_resources",
),
},
{
identity: sources.artifactProviders.ELF_NOTES_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_elf_notes",
),
},
{
identity: sources.artifactProviders.BYTE_ENTROPY_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_byte_entropy",
),
},
{
identity: sources.artifactProviders.NATIVE_EXPORTS_PROVIDER,
contracts:
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
({ name }) => name === "inspect_native_exports",
),
},
{
identity: sources.captureProvider.TSHARK_PROVIDER_IDENTITY,
contracts: sources.captureContracts.CAPTURE_TOOL_CONTRACTS,
+1
View File
@@ -39,6 +39,7 @@ export const parseDevelopmentTestRequest = (mode, arguments_) => {
// runtime or typecheck prerequisites. Add a consumer here when it reads one.
const artifactConsumers = {
"artifacts:mcp-catalog": [
"tests/boundary/mcp/deferredRuntime.test.ts",
"tests/boundary/mcp/contractPresentation.test.ts",
"tests/boundary/mcp/toolSchemaValidity.test.ts",
],
+7
View File
@@ -47,6 +47,13 @@ export function profileMcpModuleLoading(
/** Parse Linux procfs VmRSS text into bytes. */
export function parseLinuxVmRssBytes(status: string): number | null;
/** Name a fixed MCP startup phase budget. Mirrors `fixedMcpPhaseBudgetRecovery`. */
export function fixedMcpStartupPhaseBudgetRecovery(
phase: string,
value: number,
unit: "ms" | "bytes",
): string;
/** Collect unique file-backed ESM module URLs from Node diagnostics. */
export function createEsmModuleProfileCollector(packageName?: string): {
append(chunk: string): void;
+71 -14
View File
@@ -35,11 +35,22 @@ export async function measureMcpStartup({
const startedAt = performance.now();
rss.start();
try {
await client.connect(transport, { timeout: policy.initializeBudgetMs });
await withFixedPhaseBudget(
"MCP initialize phase budget",
policy.initializeBudgetMs,
"ms",
() => client.connect(transport, { timeout: policy.initializeBudgetMs }),
);
const initializedAt = performance.now();
const tools = await client.listTools(undefined, {
timeout: policy.toolsListBudgetMs,
});
const tools = await withFixedPhaseBudget(
"MCP tools/list phase budget",
policy.toolsListBudgetMs,
"ms",
() =>
client.listTools(undefined, {
timeout: policy.toolsListBudgetMs,
}),
);
const listedAt = performance.now();
rss.stop();
const measurement = {
@@ -52,7 +63,11 @@ export async function measureMcpStartup({
assertStartupBudgets(measurement, policy);
return measurement;
} catch (cause) {
throw new Error(`packaged MCP startup probe failed: ${stderr}`, { cause });
const detail = cause instanceof Error ? cause.message : String(cause);
throw new Error(
`packaged MCP startup probe failed: ${detail}${stderr.length === 0 ? "" : `\n${stderr}`}`,
{ cause },
);
} finally {
rss.stop();
await Promise.allSettled([client.close(), transport.close()]);
@@ -82,12 +97,24 @@ export async function profileMcpModuleLoading({
version: "1.0.0",
});
try {
await client.connect(transport, {
timeout: policy.doctorDeadlineMs,
});
await client.listTools(undefined, {
timeout: policy.toolsListBudgetMs,
});
await withFixedPhaseBudget(
"MCP doctor startup deadline",
policy.doctorDeadlineMs,
"ms",
() =>
client.connect(transport, {
timeout: policy.doctorDeadlineMs,
}),
);
await withFixedPhaseBudget(
"MCP tools/list phase budget",
policy.toolsListBudgetMs,
"ms",
() =>
client.listTools(undefined, {
timeout: policy.toolsListBudgetMs,
}),
);
return profile.result();
} finally {
await Promise.allSettled([client.close(), transport.close()]);
@@ -197,16 +224,46 @@ const createLinuxRssSampler = (pid) => {
};
};
/** Same sentence as `fixedMcpPhaseBudgetRecovery`; this script cannot import that TypeScript module. */
export const fixedMcpStartupPhaseBudgetRecovery = (phase, value, unit) =>
`REA's fixed ${phase} is ${String(value)} ${unit} and this release has no runtime override.`;
const phaseTimeout = (cause) => {
const message = cause instanceof Error ? cause.message : String(cause);
return (
message.includes("Request timed out") ||
message.includes("Maximum total timeout exceeded") ||
message.includes("timed out")
);
};
const withFixedPhaseBudget = async (phase, value, unit, operation) => {
try {
return await operation();
} catch (cause) {
if (!phaseTimeout(cause)) throw cause;
const detail = cause instanceof Error ? cause.message : String(cause);
throw new Error(
`${detail} ${fixedMcpStartupPhaseBudgetRecovery(phase, value, unit)}`,
{ cause },
);
}
};
const assertStartupBudgets = (measurement, policy) => {
if (measurement.process_started_to_initialized_ms > policy.initializeBudgetMs)
throw new Error("packaged MCP exceeded the initialize regression budget");
throw new Error(
`packaged MCP exceeded the initialize phase budget. ${fixedMcpStartupPhaseBudgetRecovery("MCP initialize phase budget", policy.initializeBudgetMs, "ms")}`,
);
if (measurement.initialized_to_tools_list_ms > policy.toolsListBudgetMs)
throw new Error("packaged MCP exceeded the tools/list regression budget");
throw new Error(
`packaged MCP exceeded the tools/list phase budget. ${fixedMcpStartupPhaseBudgetRecovery("MCP tools/list phase budget", policy.toolsListBudgetMs, "ms")}`,
);
if (
measurement.process_started_to_tools_list_ms > policy.firstCatalogBudgetMs
)
throw new Error(
"packaged MCP exceeded the first-catalog regression budget",
`packaged MCP exceeded the first-catalog phase budget. ${fixedMcpStartupPhaseBudgetRecovery("MCP first-catalog phase budget", policy.firstCatalogBudgetMs, "ms")}`,
);
};
+119
View File
@@ -0,0 +1,119 @@
import { createHash } from "node:crypto";
import { mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
// Runtime discovery is generated from the same canonical registrations, after
// default identity has been recorded and before bundling. Test catalogs stay separate.
const compiled = resolve(dirname(fileURLToPath(import.meta.url)), "../dist");
const read = (name) => import(pathToFileURL(join(compiled, name)).href);
const [
{ createServer },
{ createBinarySession },
{ parseConfig },
{ silentLogger },
{ PROMPT_CONTRACTS },
] = await Promise.all([
read("server/createServer.js"),
read("composition/binary.js"),
read("config/parseConfig.js"),
read("logger.js"),
read("contracts/promptContracts.js"),
]);
const recordCatalog = async (profile) => {
// Catalog generation is independent of the host account and user configuration.
const home = await mkdtemp(join(tmpdir(), "rea-mcp-catalog-"));
const environment = {
REA_MCP_INPUT_SCHEMA_PROFILE: profile,
HOME: home,
USERPROFILE: home,
XDG_CONFIG_HOME: join(home, "config"),
XDG_CACHE_HOME: join(home, "cache"),
XDG_STATE_HOME: join(home, "state"),
};
const cleanup = [];
let producerFailure;
try {
const config = parseConfig(environment);
if (!config.ok) throw config.error;
const session = createBinarySession(
config.value,
silentLogger,
environment,
);
cleanup.push(async () => {
const closed = await session.close({ retainProviderDocuments: true });
if (!closed.ok) throw closed.error;
});
const server = createServer({ kind: "session", session }, { environment });
cleanup.push(() => server.close());
const client = new Client({ name: "rea-catalog-recorder", version: "1" });
cleanup.push(() => client.close());
const [clientTransport, serverTransport] =
InMemoryTransport.createLinkedPair();
await server.connect(serverTransport);
await client.connect(clientTransport);
const tools = (await client.listTools()).tools;
const prompts = (await client.listPrompts()).prompts.map((prompt) => {
const contract = PROMPT_CONTRACTS.find(
(candidate) => candidate.name === prompt.name,
);
if (contract === undefined)
throw new Error(`Missing prompt ${prompt.name}`);
return { ...prompt, parameters: contract.arguments };
});
const catalog = {
version: 1,
info: client.getServerVersion(),
capabilities: client.getServerCapabilities(),
instructions: client.getInstructions(),
identity: server.catalogIdentity,
tools,
prompts,
};
const encoded = JSON.stringify(catalog);
const digest = createHash("sha256").update(encoded).digest("hex");
await writeFile(
join(
compiled,
profile === "full" ? "mcp-catalog-full.json" : "mcp-catalog.json",
),
encoded,
);
return digest;
} catch (cause) {
producerFailure = { cause };
throw cause;
} finally {
const results = await Promise.allSettled(
cleanup.reverse().map((close) => Promise.resolve().then(close)),
);
const failures = results.flatMap((result) =>
result.status === "rejected" ? [result.reason] : [],
);
if (failures.length === 0) {
try {
await rm(home, { recursive: true, force: true });
} catch (cause) {
failures.push(cause);
}
}
if (failures.length > 0)
throw new AggregateError(
producerFailure === undefined
? failures
: [producerFailure.cause, ...failures],
`Catalog cleanup failed; temporary home retained at ${home}`,
);
}
};
const digests = {};
for (const profile of ["compact", "full"])
digests[profile] = await recordCatalog(profile);
await writeFile(
join(compiled, "recordedMcpCatalog.js"),
`/** Recorded from the compiled runtime; do not edit. */\nexport const RECORDED_MCP_CATALOG_SHA256 = ${JSON.stringify(digests)};\n`,
);
+27 -8
View File
@@ -3,6 +3,26 @@ import { dirname, join, relative } from "node:path";
import { exec, execNpm, pathExists } from "./lib/verify-package-core.mjs";
/** Install the separately locked Skills CLI so verification resolves no registry metadata. */
const lockedSkillsCli = async (root, workspace) => {
const tool = join(workspace, "skills-cli");
await mkdir(tool, { recursive: true });
for (const file of ["package.json", "package-lock.json"])
await cp(join(root, "scripts/verify-tools/skills", file), join(tool, file));
await execNpm(
["ci", "--prefer-offline", "--ignore-scripts", "--no-audit", "--no-fund"],
{ cwd: tool, timeout: 120_000 },
);
const installed = join(tool, "node_modules/skills");
const manifest = JSON.parse(
await readFile(join(installed, "package.json"), "utf8"),
);
return {
version: manifest.version,
entry: join(installed, manifest.bin.skills),
};
};
/** Exercise Skills' default root discovery without ignored build outputs. */
export async function verifyRepositorySkill({ root, workspace }) {
const snapshot = join(workspace, "repository");
@@ -33,15 +53,14 @@ export async function verifyRepositorySkill({ root, workspace }) {
throw new Error("repository skill check must run without generated skills");
await mkdir(consumer, { recursive: true });
await mkdir(home, { recursive: true });
const skillsVersion = "1.7.2";
await execNpm(
const { version: skillsVersion, entry } = await lockedSkillsCli(
root,
workspace,
);
await exec(
process.execPath,
[
"exec",
"--yes",
"--package",
`skills@${skillsVersion}`,
"--",
"skills",
entry,
"add",
snapshot,
"--skill",
+112
View File
@@ -0,0 +1,112 @@
{
"name": "rea-verify-skills-cli",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "rea-verify-skills-cli",
"dependencies": {
"skills": "1.7.2"
}
},
"node_modules/@isaacs/fs-minipass": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
"integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==",
"license": "ISC",
"dependencies": {
"minipass": "^7.0.4"
},
"engines": {
"node": ">=18.0.0"
}
},
"node_modules/chownr": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
"integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/minipass": {
"version": "7.1.3",
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz",
"integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=16 || 14 >=14.17"
}
},
"node_modules/minizlib": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.1.0.tgz",
"integrity": "sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==",
"license": "MIT",
"dependencies": {
"minipass": "^7.1.2"
},
"engines": {
"node": ">= 18"
}
},
"node_modules/skills": {
"version": "1.7.2",
"resolved": "https://registry.npmjs.org/skills/-/skills-1.7.2.tgz",
"integrity": "sha512-j+GzyF/oNahsjjjup6y3lJpFkC1lQjIkf4kAMXICA1hboBqi0fAGXAGoE4x9FRZiP1br/YE9+L3/Tqq6PIAe0A==",
"license": "MIT",
"dependencies": {
"tar": "^7.5.20",
"yaml": "^2.8.3"
},
"bin": {
"add-skill": "bin/cli.mjs",
"skills": "bin/cli.mjs"
},
"engines": {
"node": ">=22.20.0"
}
},
"node_modules/tar": {
"version": "7.5.22",
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
"license": "BlueOak-1.0.0",
"dependencies": {
"@isaacs/fs-minipass": "^4.0.0",
"chownr": "^3.0.0",
"minipass": "^7.1.2",
"minizlib": "^3.1.0",
"yallist": "^5.0.0"
},
"engines": {
"node": ">=18"
}
},
"node_modules/yallist": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz",
"integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==",
"license": "BlueOak-1.0.0",
"engines": {
"node": ">=18"
}
},
"node_modules/yaml": {
"version": "2.9.1",
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.1.tgz",
"integrity": "sha512-3NxN8+78OdzbT7C/WjGsyfPAtJaN3FNDsWxv7Y7mcDsT/oOmgW8BpyQQFFBnvZE3j9Y2Sdz1ULFLezL7Eb2yFw==",
"license": "ISC",
"bin": {
"yaml": "bin.mjs"
},
"engines": {
"node": ">= 14.6"
},
"funding": {
"url": "https://github.com/sponsors/eemeli"
}
}
}
}
+8
View File
@@ -0,0 +1,8 @@
{
"name": "rea-verify-skills-cli",
"private": true,
"description": "Lockfile-pinned Skills CLI for the repository skill check in verify:package.",
"dependencies": {
"skills": "1.7.2"
}
}
+73
View File
@@ -195,6 +195,72 @@ const checkSwift = (metadata, mode) => {
);
};
/** Declarations from dispatch.swift that survive stripping unchanged. */
const EXPECTED_SWIFT_DECLS = [
["struct", "ReaWitnessFixture", false],
["class", "ReaVtableFixture", false],
["enum", "ReaDeclFixtureEnum", false],
["struct", "ReaDeclFixtureOuter", false],
["struct", "ReaDeclFixtureOuter.Inner", false],
["struct", "ReaDeclFixtureGeneric", true],
["actor", "ReaDeclFixtureActor", false],
];
/** Stable declaration facts compared across the symbols and stripped builds. */
const swiftDeclarations = (metadata) =>
metadata.swift_decls.map((item) => ({
kind: item.kind,
qualified_name: item.qualified_name,
module: item.module,
generic: item.generic,
descriptor: item.location.address,
accessor: item.accessor_address,
status: item.decode.status,
}));
const checkSwiftDeclarations = (metadata, mode) => {
const declarations = swiftDeclarations(metadata);
for (const [kind, qualifiedName, generic] of EXPECTED_SWIFT_DECLS)
assert.ok(
declarations.some(
(item) =>
item.kind === kind &&
item.qualified_name === qualifiedName &&
item.module === "ReaWitnessFixture" &&
item.generic === generic &&
item.accessor !== null &&
item.status === "decoded",
),
`Swift declaration ${qualifiedName} missing (${mode.mode}): ${JSON.stringify(declarations)}`,
);
assert.equal(
metadata.coverage.find(({ facet }) => facet === "swift_declarations")
?.status,
"complete",
);
return declarations;
};
/** Descriptor and accessor addresses must be the compiler's Mn/Ma symbols. */
const checkSwiftSymbolAddresses = async (path, declarations) => {
const { stdout } = await exec("/usr/bin/nm", ["--defined-only", path]);
const symbols = (suffix) =>
new Set(
stdout.split("\n").flatMap((line) => {
const [address, , name] = line.trim().split(/\s+/u);
return name?.endsWith(suffix) === true
? [`0x${BigInt(`0x${address}`).toString(16)}`]
: [];
}),
);
const descriptors = symbols("Mn");
const accessors = symbols("Ma");
for (const item of declarations) {
assert.ok(descriptors.has(item.descriptor), JSON.stringify(item));
assert.ok(accessors.has(item.accessor), JSON.stringify(item));
}
};
const root = await mkdtemp(join(tmpdir(), "rea-dispatch-"));
let report;
try {
@@ -259,14 +325,21 @@ try {
"-o",
swiftPath,
]);
let unstrippedDeclarations;
for (const variant of ["symbols", "stripped"]) {
if (variant === "stripped")
await exec("/usr/bin/strip", ["-x", swiftPath]);
const { target, metadata } = await inspect(swiftPath);
checkSwift(metadata, mode);
const declarations = checkSwiftDeclarations(metadata, mode);
if (variant === "symbols") {
await checkSwiftSymbolAddresses(swiftPath, declarations);
unstrippedDeclarations = declarations;
} else assert.deepEqual(declarations, unstrippedDeclarations);
results.push({
variant: `swift-${mode.mode}-${variant}`,
target_sha256: target.sha256,
declarations: declarations.length,
conformances: metadata.swift_conformances.length,
slots: metadata.swift_dispatch_slots.length,
fixups: fixupCoverage(metadata)?.reason,
+100 -8
View File
@@ -10,6 +10,7 @@ import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import {
artifactCli,
artifactCliEvidence,
artifactMcpResult,
withArtifactMcp,
@@ -25,6 +26,56 @@ const nibPath = join(resourcesPath, "Main.nib");
const sourceRoot = fileURLToPath(
new URL("../../../tests/conformance/interface-builder/", import.meta.url),
);
const mainSource = join(sourceRoot, "Main.xib");
// ibtool property names mapped to the keys AppKit archives for NSControl.
const buttonBooleanKeys = {
continuous: "NSControlContinuous",
refusesFirstResponder: "NSControlRefusesFirstResponder",
};
const plistDocument = (body) =>
`<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">${body}</plist>\n`;
const compileNib = (source) =>
exec("xcrun", [
"ibtool",
"--errors",
"--warnings",
"--compile",
nibPath,
source,
]);
/** Apple's source-level property values: the oracle for compiled booleans. */
const exportedButtonBooleans = async (source) => {
const exported = join(fixtureRoot, "export.plist");
const { stdout } = await exec("xcrun", [
"ibtool",
"--export",
join(fixtureRoot, "export-request.plist"),
source,
]);
await writeFile(exported, stdout);
const { stdout: json } = await exec("plutil", [
"-convert",
"json",
"-o",
"-",
exported,
]);
return JSON.parse(json)["com.apple.ibtool.document.export"].button;
};
const decodedButtonBooleans = (result) => {
const button = result.graph.nodes.find(
(node) => node.kind === "control" && node.name === "NSButton",
);
assert.ok(button, "compiled button control was not decoded");
return Object.fromEntries(
Object.entries(buttonBooleanKeys).map(([property, key]) => [
property,
button.attributes.nibValues?.[key],
]),
);
};
let report;
try {
@@ -36,14 +87,7 @@ try {
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0"><dict><key>CFBundleExecutable</key><string>Fixture</string><key>CFBundleIdentifier</key><string>dev.rea.interface-builder-fixture</string></dict></plist>\n`,
);
await exec("xcrun", [
"ibtool",
"--errors",
"--warnings",
"--compile",
nibPath,
join(sourceRoot, "Main.xib"),
]);
await compileNib(mainSource);
await exec("cc", [join(sourceRoot, "fixture.c"), "-o", executablePath]);
const evidence = await artifactCliEvidence(
@@ -110,6 +154,53 @@ try {
);
assert.equal(decoded.graph.truncated, false);
// Compiled booleans must keep their source polarity in both directions.
const properties = Object.keys(buttonBooleanKeys);
await writeFile(
join(fixtureRoot, "export-request.plist"),
plistDocument(
`<dict><key>NSControl</key><array>${properties.map((property) => `<string>${property}</string>`).join("")}</array></dict>`,
),
);
await writeFile(
join(fixtureRoot, "toggle.plist"),
plistDocument(
`<dict><key>button</key><dict>${properties.map((property) => `<key>${property}</key><true/>`).join("")}</dict></dict>`,
),
);
const toggledSource = join(fixtureRoot, "Toggled.xib");
await exec("xcrun", [
"ibtool",
"--import",
join(fixtureRoot, "toggle.plist"),
"--write",
toggledSource,
mainSource,
]);
const mainBooleans = await exportedButtonBooleans(mainSource);
const toggledBooleans = await exportedButtonBooleans(toggledSource);
assert.deepEqual(mainBooleans, {
continuous: false,
refusesFirstResponder: false,
});
assert.deepEqual(toggledBooleans, {
continuous: true,
refusesFirstResponder: true,
});
assert.deepEqual(
decodedButtonBooleans(decoded),
mainBooleans,
"compiled false booleans differ from the source export",
);
await compileNib(toggledSource);
assert.deepEqual(
decodedButtonBooleans(
await artifactCli("decode-interface-builder", appPath),
),
toggledBooleans,
"compiled true booleans differ from the source export",
);
report = {
ok: true,
fixture: "compiled-appkit-nib",
@@ -122,6 +213,7 @@ try {
connections: decoded.documents[0].connection_count,
recovered_control: button.name,
recovered_action: action.name,
button_booleans: { main: mainBooleans, toggled: toggledBooleans },
hierarchy_complete: decoded.documents[0].hierarchy_complete,
truncated: decoded.graph.truncated,
};
+27
View File
@@ -784,6 +784,33 @@ it("projects package details from dumpsys package output", async () => {
});
});
// Package Manager prints an absent installer as the text "null"; the
// normalized result must report the absence, not a package named "null"
// (morluto/rea#3943).
it("reports a null installer declaration as an absent installer", async () => {
const { binary } = await writeAdbStub(
shellStub({
"dumpsys:package": `Packages:
Package [com.example] (abcdef):
versionName=1.2.3
installerPackageName=null
requested permissions:
android.permission.INTERNET
`,
}),
);
const result = await evidenceResult(providerOver(binary), {
operation: "inspect_adb_package",
input: { serial: "emulator-5554", package: "com.example" },
});
expect(result).toMatchObject({
version_name: "1.2.3",
installer_package_name: null,
requested_permissions_count: 1,
coverage: "complete",
});
});
// PackageInfo.getLongVersionCode() combines versionCodeMajor and versionCode
// into a signed 64-bit long, which JSON numbers cannot hold exactly.
it.each(["9007199254740993", "-4294967296"])(
+46
View File
@@ -7,6 +7,52 @@ import {
parseWindowFocusOutput,
} from "./AdbShellOutput.js";
const packageDump = (installerLine: string | undefined): string =>
[
"Packages:",
" Package [com.example] (abcdef):",
" versionName=1.2.3",
...(installerLine === undefined ? [] : [` ${installerLine}`]),
" pkgFlags=[ HAS_CODE ]",
"",
].join("\n");
describe("parseDumpsysPackageOutput", () => {
it("keeps a reported installer package name", () => {
expect(
parseDumpsysPackageOutput(
packageDump("installerPackageName=com.android.vending"),
).installerPackageName,
).toBe("com.android.vending");
});
// Package Manager prints an absent String field as the text "null"; a
// missing or empty declaration means the same absence (morluto/rea#3943).
it.each([
["the literal null sentinel", "installerPackageName=null"],
["an empty value", "installerPackageName="],
["a missing line", undefined],
])("reports an absent installer as null for %s", (_case, line) => {
expect(parseDumpsysPackageOutput(packageDump(line))).toMatchObject({
versionName: "1.2.3",
installerPackageName: null,
});
});
// A manifest may declare the literal string "null" as its version name and
// dumpsys prints it the same way as an absent one, so the text is kept.
it("keeps a version name printed as the text null", () => {
expect(
parseDumpsysPackageOutput(
packageDump("installerPackageName=null").replace(
"versionName=1.2.3",
"versionName=null",
),
).versionName,
).toBe("null");
});
});
describe("parseWindowFocusOutput display contexts", () => {
it.each([false, true])(
"retains both display owners independent of block order: %s",
+12 -1
View File
@@ -275,6 +275,17 @@ export interface AdbPackageDetails {
const DETAIL_LINE = /^\s+([A-Za-z][A-Za-z0-9_]*)=(.*)$/u;
/**
* Package Manager prints its `String` fields by concatenation, so an absent
* `installerPackageName` appears as the literal text `null`
* (morluto/rea#3943). No installer can be named "null", so an empty value or
* that sentinel is the absence of the fact and becomes the nullable result's
* null. `versionName` keeps the printed text: a manifest may declare the
* string "null" on purpose, and the parser cannot tell the two apart.
*/
const absentInstallerOrName = (value: string): string | null =>
value === "" || value === "null" ? null : value;
/**
* Project `dumpsys package PACKAGE` output. Only the listed fields are
* extracted; the input is version-dependent and everything else is ignored
@@ -340,7 +351,7 @@ export const parseDumpsysPackageOutput = (
key === "installerPackageName" &&
details.installerPackageName === null
)
details.installerPackageName = value === "" ? null : value;
details.installerPackageName = absentInstallerOrName(value);
else if (key === "userId" && details.userId === null) {
const parsed = /^(\d+)/u.exec(value);
if (parsed !== null) details.userId = Number(parsed[1]!);
+7 -7
View File
@@ -4,14 +4,14 @@ import { jsonValueSchema } from "../domain/jsonValue.js";
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
import { dexDescriptorSchema } from "../domain/android/androidAnalysis.js";
const count = z.number().int().nonnegative();
const count = z.int().nonnegative();
/** Typed allocation failures authored before upstream erases the JVM exception. */
const jadxMemoryFailureSchema = z.strictObject({
kind: z.literal("jvm-memory-failure"),
memory_area: z.enum(["heap", "unknown"]),
exception_type: z.literal("java.lang.OutOfMemoryError"),
message: z.string().optional(),
max_heap_bytes: z.number().int().positive().safe(),
max_heap_bytes: z.int().positive().safe(),
});
/** Selector failures authored by REA's bridge, separate from arbitrary engine errors. */
const jadxInputFailureSchema = z.strictObject({
@@ -40,8 +40,8 @@ export const parseJadxFailure = (
/** Values observed inside the exact JVM producing the analysis. */
export const jadxRuntimeSchema = z.object({
engine_reported_version: z.string().min(1),
max_heap_bytes: z.number().int().positive().safe(),
available_processors: z.number().int().positive(),
max_heap_bytes: z.int().positive().safe(),
available_processors: z.int().positive(),
java_version: z.string().min(1),
metadata_scope: z.literal("parsed_members"),
});
@@ -81,11 +81,11 @@ export const jadxClassSchema = z.object({
signature: z.string(),
dex_descriptor: dexDescriptorSchema,
is_constructor: z.boolean(),
def_pos: z.number().int(),
def_pos: z.int(),
}),
),
fields: z.array(
z.object({ name: z.string(), type: z.string(), def_pos: z.number().int() }),
z.object({ name: z.string(), type: z.string(), def_pos: z.int() }),
),
inner_classes: z.array(z.string()),
});
@@ -115,7 +115,7 @@ export const jadxXrefsSchema = z.object({
full_name: z.string(),
containing_class: z.string(),
top_class: z.string(),
def_pos: z.number().int(),
def_pos: z.int(),
direct_invokes_exhaustive: z.boolean().optional(),
direct_invokes: z
.array(
+89
View File
@@ -0,0 +1,89 @@
import { EventEmitter } from "node:events";
import { win32 } from "node:path";
import { PassThrough } from "node:stream";
import { afterEach, expect, it, vi } from "vitest";
const { spawnMock, tmpdirMock, lstatMock } = vi.hoisted(() => ({
spawnMock: vi.fn(),
tmpdirMock: vi.fn(),
lstatMock: vi.fn(),
}));
vi.mock("node:child_process", async (importOriginal) => ({
...(await importOriginal<typeof import("node:child_process")>()),
spawn: spawnMock,
}));
vi.mock("node:os", async (importOriginal) => ({
...(await importOriginal<typeof import("node:os")>()),
tmpdir: tmpdirMock,
}));
vi.mock("node:fs/promises", async (importOriginal) => ({
...(await importOriginal<typeof import("node:fs/promises")>()),
lstat: lstatMock,
}));
import { createJadxWorkspaceRoot } from "./JadxWorkspaceRoot.js";
afterEach(() => {
vi.restoreAllMocks();
vi.clearAllMocks();
});
it("decodes a Windows guardian path across split UTF-8 stdout writes", async () => {
const previousPlatform = process.platform;
Object.defineProperty(process, "platform", {
configurable: true,
value: "win32",
});
const parent = "C:\\Users\\研究者\\AppData\\Local\\Temp";
const path = win32.join(parent, `rea-android-${"a".repeat(32)}`);
const child = Object.assign(new EventEmitter(), {
stdin: new PassThrough(),
stdout: new PassThrough(),
stderr: new PassThrough(),
unref: vi.fn(),
});
child.stdin.once("finish", () => {
child.stdout.end();
child.stderr.end();
child.emit("close", 0, null);
});
tmpdirMock.mockReturnValue(parent);
const missingPath = Object.assign(new Error("missing after cleanup"), {
code: "ENOENT",
});
lstatMock.mockRejectedValue(missingPath);
spawnMock.mockReturnValue(child);
const ready = Buffer.from(
`${JSON.stringify({
path,
volumeSerial: "0123456789abcdef",
fileId: "0123456789abcdef0123456789abcdef",
privateDacl: true,
})}\n`,
);
const split = ready.indexOf(Buffer.from("研")) + 1;
try {
const start = createJadxWorkspaceRoot();
setImmediate(() => {
child.stdout.write(ready.subarray(0, split), () => {
setImmediate(() => child.stdout.write(ready.subarray(split)));
});
});
const root = await start;
try {
expect(root.path).toBe(path);
} finally {
await expect(root.close()).resolves.toBeUndefined();
}
expect(lstatMock).toHaveBeenCalledWith(path);
} finally {
Object.defineProperty(process, "platform", {
configurable: true,
value: previousPlatform,
});
}
});
+6 -4
View File
@@ -105,8 +105,9 @@ class WindowsJadxWorkspaceRoot implements JadxWorkspaceRoot {
},
});
let diagnostics = "";
child.stderr.on("data", (chunk: Buffer) => {
diagnostics = (diagnostics + chunk.toString()).slice(-8192);
child.stderr.setEncoding("utf8");
child.stderr.on("data", (chunk: string) => {
diagnostics = (diagnostics + chunk).slice(-8192);
});
const exit = new Promise<GuardianExit>((resolve) => {
child.once("error", (error: Error) =>
@@ -116,8 +117,9 @@ class WindowsJadxWorkspaceRoot implements JadxWorkspaceRoot {
});
const ready = new Promise<string>((resolve, reject) => {
let frame = "";
child.stdout.on("data", (chunk: Buffer) => {
frame += chunk.toString();
child.stdout.setEncoding("utf8");
child.stdout.on("data", (chunk: string) => {
frame += chunk;
if (frame.length > 4096) {
reject(
new Error("Android workspace guardian readiness exceeded 4 KiB"),
+3
View File
@@ -1,3 +1,4 @@
import type { EvidenceCreationOptions } from "../domain/evidence.js";
import type { ProviderInspectionScope } from "../providers/ProviderInspectionScope.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import type { OfficialToolName } from "../contracts/officialToolContracts.js";
@@ -146,6 +147,7 @@ export const createAnalysisExecutionEvidence = (
input: unknown,
execution: AnalysisExecution,
confidence: "observed" | "derived",
options?: EvidenceCreationOptions,
): Evidence =>
createEvidence(
execution.subject === undefined
@@ -167,6 +169,7 @@ export const createAnalysisExecutionEvidence = (
locations: execution.locations,
confidence,
},
options,
);
/** Provider-neutral application capability for one analysis operation. */
+8
View File
@@ -138,6 +138,9 @@ const browserParameters = (
: "target_id" in input && observedOrigin !== undefined
? [observedOrigin]
: [],
...("command_timeout_ms" in input && input.command_timeout_ms !== undefined
? { command_timeout_ms: input.command_timeout_ms }
: {}),
};
if (!("target_id" in input)) return scope;
return {
@@ -171,6 +174,11 @@ const browserParameters = (
...(input.total_analysis_timeout_ms === undefined
? {}
: { total_analysis_timeout_ms: input.total_analysis_timeout_ms }),
...(input.source_map_fetch_timeout_ms === undefined
? {}
: {
source_map_fetch_timeout_ms: input.source_map_fetch_timeout_ms,
}),
}
: {}),
};
+4 -1
View File
@@ -350,6 +350,7 @@ export const compactToolAvailability = (
const availabilityFor = (context: AvailabilityContext): Availability => {
// These built-in workflows accept a caller-selected endpoint; they do not load the optional passive provider.
if (
context.name === "inspect_byte_entropy" ||
context.name === "observe_web_execution" ||
context.name === "inspect_web_event_listeners"
)
@@ -497,7 +498,9 @@ const workflowAvailabilityFor = ({
remediation:
"Supply WABT 1.0.42 via absolute REA_WABT_BIN_DIRECTORY. No active binary target is required; tools are checked at execution.",
};
if (name === "inspect_pe_resources")
if (name === "inspect_pe_resources" || name === "inspect_native_exports")
return { reason: "available", remediation: null };
if (name === "inspect_elf_notes")
return { reason: "available", remediation: null };
if (name === "inspect_evm_interface") {
if (policy.evmInterfaceEnabled !== true)
+90 -23
View File
@@ -1,4 +1,5 @@
import type { ExecutableFormatHint } from "../domain/dosCom.js";
import { dispatchQueuedInterrupts } from "../process/queuedInterrupts.js";
import type { BinaryArchitecture } from "../domain/binaryTargetTypes.js";
import type { JsonValue } from "../domain/jsonValue.js";
import type { DirectAnalysisDependencies } from "./DirectAnalysisDependencies.js";
@@ -59,7 +60,11 @@ import {
import type { AnalysisProviderSelector } from "../contracts/providerSelection.js";
import { artifactInspectionResultSchema } from "../domain/artifactInspection.js";
import { resolveXrefsAddress } from "./XrefsAddressResolution.js";
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
import {
AnalysisCancelledError,
AnalysisInputError,
} from "../domain/analysisErrorCore.js";
import { deliverAnalysisEvidence } from "./EvidenceBundleCommands.js";
import type { BinaryTargetError } from "../domain/configurationErrors.js";
import {
directAnalysisCleanupFailure,
@@ -146,24 +151,81 @@ export const runDirectAnalysis = async (
/** Execute one provider-native semantic operation with atomic provenance. */
export const runProviderAnalysis = async (
dependencies: DirectAnalysisDependencies,
...[path, tool, arguments_, logger = silentLogger, signal]: readonly [
path: string,
tool: NativeToolName | ArtifactAnalysisOperation | ManagedToolName,
arguments_: Readonly<Record<string, JsonValue>>,
logger?: Logger,
signal?: AbortSignal,
]
...[path, tool, arguments_, logger = silentLogger, signal, delivery]:
| readonly [
path: string,
tool: NativeToolName | ArtifactAnalysisOperation | ManagedToolName,
arguments_: Readonly<Record<string, JsonValue>>,
logger?: Logger | undefined,
signal?: AbortSignal | undefined,
]
| readonly [
path: string,
tool: "inspect_managed_members",
arguments_: Readonly<Record<string, JsonValue>>,
logger: Logger | undefined,
signal: AbortSignal | undefined,
delivery: {
readonly detail?: "complete" | "summary";
readonly evidenceBundlePath?: string;
},
]
): Promise<JsonValue> => {
if (isManagedToolName(tool))
return runManagedProviderAnalysis(
dependencies,
path,
tool,
arguments_,
signal,
if (delivery !== undefined && tool !== "inspect_managed_members")
return cliError(
new AnalysisInputError(tool, undefined, [
{
path: ["delivery"],
reason: "invalid_value",
message:
"Evidence delivery options are only supported by inspect_managed_members",
},
]),
);
return withProcessCancellation(signal, (operationSignal) =>
runAnalysis(dependencies, path, tool, arguments_, {
if (
delivery?.detail === "summary" &&
delivery.evidenceBundlePath === undefined
)
return cliError(
new AnalysisInputError(tool, undefined, [
{
path: ["evidence_bundle"],
reason: "missing_argument",
message:
"summary detail requires --evidence-bundle so complete Evidence remains available to later CLI commands",
},
]),
);
return withProcessCancellation(signal, async (operationSignal) => {
if (isManagedToolName(tool)) {
const original = await runManagedProviderAnalysis(
dependencies,
path,
tool,
arguments_,
operationSignal,
);
await dispatchQueuedInterrupts();
if (!original.ok) return cliError(original.error);
if (operationSignal.aborted)
return cliError(
new AnalysisCancelledError(tool, {
cause: operationSignal.reason,
partialObservation: original.value,
}),
);
const result =
delivery?.evidenceBundlePath === undefined
? original
: await deliverAnalysisEvidence(
original.value,
delivery.evidenceBundlePath,
delivery.detail,
operationSignal,
);
return result.ok ? result.value : cliError(result.error);
}
return runAnalysis(dependencies, path, tool, arguments_, {
logger,
snapshotPath: undefined,
signal: operationSignal,
@@ -172,8 +234,8 @@ export const runProviderAnalysis = async (
: NATIVE_INSPECTION_OPERATIONS.has(tool)
? { createSession: dependencies.createNativeBinarySession }
: {}),
}),
);
});
});
};
/** Execute managed metadata inspection in an isolated managed-only session. */
@@ -199,13 +261,14 @@ export const runManagedProviderExecution = async (
);
});
/** Preserve typed managed Evidence before selecting its CLI delivery. */
const runManagedProviderAnalysis = async (
dependencies: DirectAnalysisDependencies,
dependencies: Pick<DirectAnalysisDependencies, "createManagedBinarySession">,
path: string,
tool: ManagedToolName,
arguments_: Readonly<Record<string, JsonValue>>,
signal?: AbortSignal,
): Promise<JsonValue> => {
): Promise<Result<Evidence, AnalysisError>> => {
const execution = await runManagedProviderExecution(
dependencies,
path,
@@ -213,9 +276,13 @@ const runManagedProviderAnalysis = async (
signal,
arguments_,
);
if (!execution.ok) return cliError(execution.error);
if (!execution.ok) return execution;
const value = execution.value;
return createAnalysisExecutionEvidence(tool, arguments_, value, "observed");
return ok(
createAnalysisExecutionEvidence(tool, arguments_, value, "observed", {
immutable: true,
}),
);
};
const runAnalysis = async (
+6 -20
View File
@@ -198,12 +198,13 @@ export class EnhancedTools {
): EnhancedResult {
switch (call.name) {
case "inspect_native_dispatch_metadata":
return this.#inspectNativeDispatchMetadata(
call.input.max_records,
return inspectNativeDispatch(this.analysis, call.input.max_records, {
signal,
call.input.objc_class_import_addresses,
call.input.objc_class_import_addresses_complete,
);
objcClassImportAddresses: call.input.objc_class_import_addresses,
objcClassImportAddressesComplete:
call.input.objc_class_import_addresses_complete,
maxInputBytes: call.input.max_input_bytes,
});
case "get_objc_classes":
return this.#objcClasses(call.input.pattern, signal);
case "get_objc_protocols":
@@ -420,21 +421,6 @@ export class EnhancedTools {
);
}
#inspectNativeDispatchMetadata(
maxRecords: number,
signal?: AbortSignal,
objcClassImportAddresses?: readonly string[],
objcClassImportAddressesComplete?: boolean,
): EnhancedResult {
return inspectNativeDispatch(
this.analysis,
maxRecords,
signal,
objcClassImportAddresses,
objcClassImportAddressesComplete,
);
}
async #objcProtocols(signal?: AbortSignal): EnhancedResult {
const names = await this.#allAddressed("list_names", signal);
return names.ok ? ok(discoverObjcProtocols(names.value)) : names;
+193 -12
View File
@@ -1,9 +1,23 @@
import { dispatchQueuedInterrupts } from "../process/queuedInterrupts.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { requireAuthenticationHeadroom } from "./JsonAuthenticationHeadroom.js";
import {
EvidenceFileError,
EvidenceIntegrityError,
type PublishedEvidenceFile,
} from "../domain/evidenceErrors.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import type { EvidenceBundle } from "../domain/evidenceBundle.js";
import {
AnalysisCancelledError,
AnalysisInputError,
} from "../domain/analysisErrorCore.js";
import type { Evidence } from "../domain/evidence.js";
import {
createImmutableEvidenceBundle,
type EvidenceBundle,
} from "../domain/evidenceBundle.js";
import type { InspectAnalysisViewInput } from "../domain/analysisView/analysisView.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
@@ -14,12 +28,18 @@ import {
readImmutableEvidenceBundle,
writeEvidenceBundle,
} from "./EvidenceBundleFiles.js";
import { inspectAnalysisViewValidated } from "./analysisView/AnalysisViewService.js";
/** Validate a standalone bundle and report its complete record/revision counts. */
export const importEvidenceBundleCommand = async (
path: string,
): Promise<Result<JsonValue, AnalysisError>> => {
const loaded = await readEvidenceBundle(path);
const loaded = await readEvidenceBundle(
path,
undefined,
undefined,
"import_evidence_bundle",
);
if (!loaded.ok) return loaded;
return ok({
imported: loaded.value.records.length,
@@ -35,16 +55,170 @@ export const exportEvidenceBundleCommand = async (
overwrite: boolean,
): Promise<Result<JsonValue, AnalysisError>> => {
// The sealed snapshot lets the writer reuse this authentication.
const loaded = await readImmutableEvidenceBundle(sourcePath);
const loaded = await readImmutableEvidenceBundle(
sourcePath,
undefined,
undefined,
"export_evidence_bundle",
);
if (!loaded.ok) return loaded;
return projectWrite(
loaded.value,
await persistEvidenceBundleCommand(loaded.value, outputPath, overwrite),
);
};
/** Publish complete parent Evidence before delivering it or its linked summary. */
export const deliverAnalysisEvidence = async (
evidence: Evidence,
outputPath: string,
detail: "complete" | "summary" = "complete",
signal?: AbortSignal,
): Promise<Result<Evidence, AnalysisError>> => {
const headroom = requireAuthenticationHeadroom(evidence, {
operation: evidence.operation,
path: absoluteCallerPath(outputPath),
description:
"Insufficient heap headroom to authenticate the complete parent Evidence",
});
if (!headroom.ok)
return err(headroom.error.retainPartialObservation(evidence));
const bundle = createImmutableEvidenceBundle([evidence]);
const delivered =
detail === "summary"
? inspectAnalysisViewValidated(
{
source: {
kind: "retained-evidence",
evidence_id: evidence.evidence_id,
},
view: { kind: "summary" },
},
(id) => bundle.records.find((record) => record.evidence_id === id),
)
: ok(evidence);
if (!delivered.ok) return delivered;
const published = await publishEvidenceDelivery({
bundle,
complete: evidence,
delivered: delivered.value,
outputPath,
operation: evidence.operation,
signal,
});
return published.ok ? ok(published.value.delivered) : published;
};
/** Selected Evidence delivered after its complete bundle was committed. */
export interface PublishedEvidenceDelivery {
readonly delivered: Evidence;
readonly evidenceBundle: PublishedEvidenceFile;
}
/**
* Publish a complete bundle, then deliver Evidence already projected from it.
* Cancellation observed before commit leaves no output and retains the
* complete Evidence. A committed publication wins a later cancellation race;
* its receipt stays in the typed cancellation as diagnostics.evidence_bundle,
* and a cleanup failure after commit keeps its own published receipt.
*/
export const publishEvidenceDelivery = async (input: {
readonly bundle: EvidenceBundle;
readonly complete: Evidence;
readonly delivered: Evidence;
readonly outputPath: string;
readonly operation: string;
readonly signal?: AbortSignal | undefined;
}): Promise<Result<PublishedEvidenceDelivery, AnalysisError>> => {
const { signal, operation } = input;
const written = await persistEvidenceBundleCommand(
input.bundle,
input.outputPath,
false,
signal,
operation,
);
// Deliver interrupts queued during final publication cleanup before the
// command's signal owner accepts completion and removes its listeners.
await dispatchQueuedInterrupts();
if (!written.ok)
return err(written.error.retainPartialObservation(input.complete));
if (signal?.aborted)
return err(
new AnalysisCancelledError(operation, {
cause: signal.reason,
partialObservation: input.delivered,
diagnostics: { evidence_bundle: { ...written.value } },
}),
);
return ok({ delivered: input.delivered, evidenceBundle: written.value });
};
/** Resolve the caller's exact retained parent from an authenticated portable bundle. */
export const inspectAnalysisViewFromBundle = async (
input: InspectAnalysisViewInput,
bundlePath: string,
resources: ArtifactResourceScope,
signal?: AbortSignal,
): Promise<Result<Evidence, AnalysisError>> => {
if (input.source.kind !== "retained-evidence")
return err(
new AnalysisInputError("inspect_analysis_view", undefined, [
{
path: ["source", "kind"],
reason: "invalid_value",
message:
"--evidence-bundle requires a retained-evidence source with the exact parent evidence_id; inline Evidence already supplies its source",
},
]),
);
const loaded = await readImmutableEvidenceBundle(
bundlePath,
signal,
resources,
"inspect_analysis_view",
);
// Authentication and projection are synchronous; dispatch queued interrupts before accepting either result.
await dispatchQueuedInterrupts();
if (!loaded.ok) return loaded;
if (signal?.aborted)
return err(
new AnalysisCancelledError("inspect_analysis_view", {
cause: signal.reason,
}),
);
const projected = inspectAnalysisViewValidated(input, (id) =>
loaded.value.records.find((record) => record.evidence_id === id),
);
await dispatchQueuedInterrupts();
if (projected.ok && signal?.aborted)
return err(
new AnalysisCancelledError("inspect_analysis_view", {
cause: signal.reason,
partialObservation: projected.value,
}),
);
return projected;
};
/** Publish a completed bundle and retry its exact staged-output cleanup owner. */
const persistEvidenceBundleCommand = async (
bundle: EvidenceBundle,
outputPath: string,
overwrite: boolean,
signal?: AbortSignal,
operation?: string,
): Promise<Result<PublishedEvidenceFile, AnalysisError>> => {
const resources = new ArtifactResourceScope();
const written = await writeEvidenceBundle(
loaded.value,
bundle,
outputPath,
overwrite,
resources,
signal,
operation,
);
if (written.ok) return projectWrite(loaded.value, written);
if (written.ok) return written;
// Confirmed release resolves the bound error to its primary outcome; a
// failed retry leaves that error incomplete with its exact retained owner.
@@ -54,10 +228,7 @@ export const exportEvidenceBundleCommand = async (
written.error instanceof EvidenceFileError &&
written.error.published
) {
return projectWrite(loaded.value, {
ok: true,
value: written.error.published,
});
return ok(written.error.published);
}
return written;
};
@@ -70,8 +241,18 @@ export const compareEvidenceBundlesCommand = async (input: {
// Sealed snapshots let comparison and its bundle digests reuse this
// authentication instead of re-validating every record twice more.
const [left, right] = await Promise.all([
readImmutableEvidenceBundle(input.leftPath),
readImmutableEvidenceBundle(input.rightPath),
readImmutableEvidenceBundle(
input.leftPath,
undefined,
undefined,
"compare_bundles",
),
readImmutableEvidenceBundle(
input.rightPath,
undefined,
undefined,
"compare_bundles",
),
]);
if (!left.ok) return left;
if (!right.ok) return right;
@@ -88,7 +269,7 @@ export const compareEvidenceBundlesCommand = async (input: {
const projectWrite = (
bundle: EvidenceBundle,
written: Awaited<ReturnType<typeof writeEvidenceBundle>>,
written: Result<PublishedEvidenceFile, AnalysisError>,
): Result<JsonValue, AnalysisError> =>
written.ok
? ok({
+24 -18
View File
@@ -1,3 +1,4 @@
import { requireAuthenticationHeadroom } from "./JsonAuthenticationHeadroom.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import {
@@ -39,45 +40,47 @@ export const readEvidenceBundle = (
path: string,
signal?: AbortSignal,
resources?: ArtifactResourceScope,
operation = "import_evidence_bundle",
): Promise<Result<EvidenceBundle, EvidenceReadFailure>> =>
readBundle(path, parseEvidenceBundle, signal, resources);
readBundle(path, parseEvidenceBundle, signal, resources, operation);
/** Read an authenticated immutable snapshot for a subsequent session import. */
export const readImmutableEvidenceBundle = (
path: string,
signal?: AbortSignal,
resources?: ArtifactResourceScope,
operation = "import_evidence_bundle",
): Promise<Result<EvidenceBundle, EvidenceReadFailure>> =>
readBundle(path, parseImmutableEvidenceBundle, signal, resources);
readBundle(path, parseImmutableEvidenceBundle, signal, resources, operation);
const readBundle = async (
path: string,
parseBundle: (input: unknown) => EvidenceBundle,
signal?: AbortSignal,
resources?: ArtifactResourceScope,
operation = "import_evidence_bundle",
): Promise<Result<EvidenceBundle, EvidenceReadFailure>> => {
if (signal?.aborted)
return err(
new AnalysisCancelledError("import_evidence_bundle", {
new AnalysisCancelledError(operation, {
cause: signal.reason,
}),
);
const loaded = await readJsonFile(path, signal, resources);
if (!loaded.ok) {
if (loaded.error instanceof AnalysisCancelledError)
return err(
new AnalysisCancelledError("import_evidence_bundle", {
cause: loaded.error.cause,
}),
);
return loaded;
}
const loaded = await readJsonFile(path, signal, resources, operation);
if (!loaded.ok) return loaded;
if (signal?.aborted)
return err(
new AnalysisCancelledError("import_evidence_bundle", {
new AnalysisCancelledError(operation, {
cause: signal.reason,
}),
);
const headroom = requireAuthenticationHeadroom(loaded.value, {
operation,
path: absoluteCallerPath(path),
description:
"Insufficient heap headroom to authenticate the Evidence bundle",
});
if (!headroom.ok) return headroom;
let bundle: EvidenceBundle;
try {
bundle = parseBundle(loaded.value);
@@ -121,6 +124,7 @@ export const writeEvidenceBundle = async (
overwrite: boolean,
resources: ArtifactResourceScope,
signal?: AbortSignal,
operation = "export_evidence_bundle",
): Promise<
Result<
{ readonly path: string; readonly bytes: number },
@@ -128,7 +132,11 @@ export const writeEvidenceBundle = async (
>
> => {
if (signal?.aborted === true)
return err(new AnalysisCancelledError("export_evidence_bundle"));
return err(
new AnalysisCancelledError(operation, {
cause: signal.reason,
}),
);
let checked: EvidenceBundle;
try {
checked = parseEvidenceBundle(bundle);
@@ -144,8 +152,6 @@ export const writeEvidenceBundle = async (
path,
overwrite,
resources,
signal === undefined
? undefined
: { signal, operation: "export_evidence_bundle" },
signal === undefined ? undefined : { signal, operation },
);
};
Loaded 100 of 882 files, more files were not shown because too many files have changed in this diff. Show more