Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 Kaoru0822andClaude Opus 5.5
|
c6cda443c1 | ||
|
Kaoru0822andClaude Opus 5.5
|
e308f35c43 |
No files matched your search
@@ -219,12 +219,6 @@ explicit path or endpoint and do not need it.
|
||||
delivery requires `--evidence_output`.
|
||||
See the
|
||||
[Rust binary guide](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
- ELF note tables: `inspect_elf_notes` with the explicit local ELF32/ELF64
|
||||
file. This built-in target-free inspection requires no disassembler. Preserve
|
||||
PT_NOTE and SHT_NOTE ranges, owners, types, descriptor SHA-256, and GNU
|
||||
build-id or ABI-tag fields. A missing table is observed absence of those
|
||||
declarations, not proof the image never carried notes. See the
|
||||
[ELF notes guide](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
|
||||
- Explicit local SQLite database snapshot: `inspect_sqlite_database`. Inspect
|
||||
with Node.js 24.x >=24.15 or 26+ providing native SQLite limits. Read schema
|
||||
|
||||
@@ -93,14 +93,10 @@ Use `view: {kind: "environment-state"}` with the manifest's `template_rules` to
|
||||
extract the data those reads observed instead: entity types and identity
|
||||
fields, entities with recorded values, documents, and per-tool render plans
|
||||
checked against the recordings. The state comes only from reads before the
|
||||
first related write; types, identity and list membership are inferences. The
|
||||
view also gives each state-changing tool a write plan, served only when a later
|
||||
recorded read confirms its effect and none contradicts it.
|
||||
first related write; types, identity and list membership are inferences.
|
||||
`rea environment-serve manifest.json --evidence capture.json` derives that
|
||||
state at startup, answers read tools from it, serves the write tools whose plan
|
||||
is served, each applying its effect to that state, and serves `data/reset`,
|
||||
`data/add` and `data/get` on `/agentenv`; other write tools are not served, and
|
||||
each plan states why in its `reason`. It replays the capture instead with
|
||||
`--replay-only` or when the state is unavailable (not derivable, or a catalog
|
||||
over the discovery budget), and its stopped document's `state` says which
|
||||
applied.
|
||||
state at startup, answers read tools from it and serves `data/reset`,
|
||||
`data/add` and `data/get` on `/agentenv`; write tools are not served. It
|
||||
replays the capture instead with `--replay-only` or when the state is
|
||||
unavailable (not derivable, or a catalog over the discovery budget), and its
|
||||
stopped document's `state` says which applied.
|
||||
@@ -22,10 +22,10 @@ requires complete application analysis; `source_paths: []` is invalid, not an
|
||||
unfiltered request. `detail: "summary"` reduces delivery
|
||||
and does not by itself reduce analysis work.
|
||||
|
||||
ASAR header/index admission defaults to 16 MiB / 16777216 bytes;
|
||||
`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB, set where REA starts) changes it.
|
||||
`source_paths`, `detail`, and `max_heap_mb` do not bypass or raise it. Or
|
||||
analyze an extracted directory.
|
||||
ASAR header/index admission is fixed at 16 MiB / 16777216 bytes for this
|
||||
release and has no runtime override. `source_paths`, `detail`, and
|
||||
`max_heap_mb` do not bypass or raise it. Analyze an extracted directory, or
|
||||
supply an archive whose header fits.
|
||||
|
||||
Exact `#alias` specifiers resolve through the importer's
|
||||
enclosing package `imports` map to package-local files; nested `node_modules`
|
||||
|
||||
@@ -45,8 +45,7 @@ See [desktop installations](https://github.com/morluto/rea/blob/main/docs/deskto
|
||||
|
||||
## Go build metadata
|
||||
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or Mach-O (universal
|
||||
files need `architecture`) when the
|
||||
Use `inspect_go_binary` for an explicit local ELF, PE or thin Mach-O when the
|
||||
question concerns the embedded compiler version, module/dependency versions,
|
||||
local replacements or build settings. No `open_binary`, Go installation or
|
||||
target execution is required. The CLI equivalent is `inspect-go-binary PATH`.
|
||||
@@ -85,8 +84,7 @@ See [IL2CPP metadata](https://github.com/morluto/rea/blob/main/docs/il2cpp-metad
|
||||
|
||||
## Rust dependency metadata
|
||||
|
||||
Use `inspect_rust_binary` for an explicit local ELF, PE, or Mach-O (universal
|
||||
files need `architecture`) when
|
||||
Use `inspect_rust_binary` for an explicit local ELF, PE, or thin Mach-O when
|
||||
the question concerns the cargo-auditable crate list embedded in `.dep-v0`
|
||||
(Mach-O: `__DATA,.dep-v0`). No `open_binary`, Rust installation, or target
|
||||
execution is required. The CLI equivalent is `inspect-rust-binary PATH`.
|
||||
@@ -96,17 +94,6 @@ dependencies. An omitted package kind stays null. A rustc commit hash is
|
||||
inferred from a read-only `/rustc/<40-hex>/` path and does not by itself
|
||||
identify the compiler. See [Rust binaries](https://github.com/morluto/rea/blob/main/docs/rust-binaries.md).
|
||||
|
||||
## ELF note tables
|
||||
|
||||
Use `inspect_elf_notes` for an explicit local ELF32/ELF64 image when the
|
||||
question concerns PT_NOTE/SHT_NOTE owners, GNU build-id, or GNU ABI-tag
|
||||
fields. No `open_binary` or target execution is required. The CLI equivalent
|
||||
is `inspect-elf-notes PATH`. Preserve artifact SHA-256, table file ranges,
|
||||
descriptor hashes, and original owner spellings. A missing table is observed
|
||||
absence of those declarations, not proof the image never carried notes in
|
||||
another container. See
|
||||
[ELF note tables](https://github.com/morluto/rea/blob/main/docs/elf-notes.md).
|
||||
|
||||
## SQLite snapshots
|
||||
|
||||
Use `inspect_sqlite_database` with an explicit local database snapshot for its
|
||||
@@ -379,8 +366,6 @@ For headerless DOS COM, explicitly open with `format: "dos-com"` (CLI
|
||||
analysis: the entry is `0x10100`, file offset zero, with imposed real-mode segment
|
||||
context. PSP/stack/device state remains unmodeled. See [DOS guide](https://github.com/morluto/rea/blob/main/docs/ghidra-dos.md).
|
||||
|
||||
Use `inspect_byte_entropy` to triage high- and low-entropy ranges before extraction.
|
||||
|
||||
For other headerless images (embedded firmware without ELF/Mach-O/PE/MZ
|
||||
headers), open with `format: "raw-binary"`, `architecture` (`x86`, `x86_64`,
|
||||
`arm`, `arm64`, `6502`, or `m68k`), and `entry_point` (`0x` hex). File offset 0 is mapped at
|
||||
|
||||
@@ -875,7 +875,6 @@ jobs:
|
||||
tests/boundary/process/packageNpmInvocation.test.ts
|
||||
tests/boundary/process/packedBridgeVerification.test.ts
|
||||
tests/boundary/process/windowsOutputCapture.test.ts
|
||||
tests/boundary/process/windowsPrivateRuntimeAclObserver.test.ts
|
||||
tests/acceptance/webModuleCli.test.ts
|
||||
tests/acceptance/webSourceLocationCli.test.ts
|
||||
tests/boundary/cli/referenceSourceImportStatus.test.ts
|
||||
|
||||
@@ -191,7 +191,6 @@ Additional tools and host support depend on the target:
|
||||
| IL2CPP metadata | Unity global-metadata.dat assembly images, type and method names and index ranges | Built-in static decoding of versions 24, 27, 29, 31 and 39; no Unity required |
|
||||
| Godot PCK | Archive inventory and extraction, original header/directory facts and selected content identities | Built-in static formats 2, 3 and 4; no Godot required |
|
||||
| Rust binaries | Embedded cargo-auditable crate names, versions, sources and dependency edges | Built-in static ELF/PE/thin Mach-O inspection; [Rust binaries](docs/rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](docs/elf-notes.md) |
|
||||
| WASM artifact | Validation, sections, exact import/export WAT forms and decoded WAT | Caller-supplied WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, inferred arguments and mutability | Local raw/hex carrier on Linux x64; [offline EVM guide](docs/evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers and selected ordinary-table rows | Local database/WAL snapshot; Node.js 24.x >=24.15 or 26+ with native SQLite limits |
|
||||
|
||||
@@ -32,7 +32,6 @@ export default defineConfig({
|
||||
{ text: "IL2CPP metadata", link: "/il2cpp-metadata" },
|
||||
{ text: "Model artifacts", link: "/model-artifacts" },
|
||||
{ text: "Rust dependencies", link: "/rust-binaries" },
|
||||
{ text: "ELF notes", link: "/elf-notes" },
|
||||
{
|
||||
text: "JavaScript and Electron",
|
||||
link: "/javascript-artifact-reconstruction",
|
||||
|
||||
+38
-87
@@ -5,8 +5,7 @@ environment: an MCP server whose tools answer from what one retained web
|
||||
network capture observed, plus the environment card the agentenv framework
|
||||
reads to discover it. By default the server derives the
|
||||
[state the capture observed](#the-state-a-capture-observed) at startup and
|
||||
answers read tools from it, and the write tools whose effect the capture
|
||||
confirmed by changing it; `--replay-only` replays the capture's records
|
||||
answers read tools from it; `--replay-only` replays the capture's records
|
||||
instead. Nothing is fetched from the recorded application; every answer comes
|
||||
from retained Evidence. `rea environment-export` writes the same pair as a
|
||||
bundle folder `agent-env run` builds and runs; see
|
||||
@@ -272,11 +271,11 @@ document, with the same signal exit status.
|
||||
|
||||
Three routes are served:
|
||||
|
||||
| Route | Answer |
|
||||
| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `POST /mcp` | The MCP server: from the state, one tool per read tool, answering from the state where it backs them, and one per write tool whose effect the capture confirmed, applying it to the state; in replay, one tool per manifest tool, replaying records |
|
||||
| `GET /.well-known/agent-env.json` | The environment card |
|
||||
| `POST /agentenv` | From the state, `data/reset`, `data/add` and `data/get`; in replay, JSON-RPC `-32601 method not found: <method>` for every method |
|
||||
| Route | Answer |
|
||||
| --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `POST /mcp` | The MCP server: from the state, one tool per read tool, answering from the state where it backs them; in replay, one tool per manifest tool, replaying records |
|
||||
| `GET /.well-known/agent-env.json` | The environment card |
|
||||
| `POST /agentenv` | From the state, `data/reset`, `data/add` and `data/get`; in replay, JSON-RPC `-32601 method not found: <method>` for every method |
|
||||
|
||||
Numeric JSON-RPC IDs outside JavaScript's safe range or whose precision would
|
||||
be lost are refused as invalid requests with a null ID. They are never echoed
|
||||
@@ -377,11 +376,8 @@ In replay, `extensions`, `operations` and `children_environments` are fixed to
|
||||
`null`, `[]` and `null`: a replay environment declares no extensions, no
|
||||
data-plane operations and no children. Served
|
||||
[from the state](#serve-from-the-state), the card declares the three data-plane
|
||||
operations and one intake extension instead, and the catalog holds the read
|
||||
tools and only the write tools whose effect the capture confirmed: a read tool
|
||||
is described as answering from the state where the state backs it, and a
|
||||
served write tool by the effect it applies to the state and the body members it
|
||||
accepts.
|
||||
operations and one intake extension instead, leaves write tools out and
|
||||
describes each tool as answering from the state where the state backs it.
|
||||
|
||||
## Serve from the state
|
||||
|
||||
@@ -404,8 +400,7 @@ A state that is unavailable does not stop the server: it logs a warning naming
|
||||
the error `code` and message before it binds, then serves replay. The state is
|
||||
unavailable when it cannot be derived, such as from a
|
||||
`capture_browser_scenario` capture that selected header values but no response
|
||||
bodies or over the view's budget, when `data/get` would write it past the 32 MiB
|
||||
`data/add` intake limit, and when its catalog under the served name
|
||||
bodies or over the view's budget, and when its catalog under the served name
|
||||
exceeds the [discovery reservation](#what-a-manifest-is). That check runs
|
||||
first, so no state is derived for it, and replay is then admitted exactly as
|
||||
with `--replay-only`. `--replay-only`
|
||||
@@ -422,55 +417,26 @@ member reports which applied:
|
||||
|
||||
Each read tool answers according to its view in the state:
|
||||
|
||||
| View | Answer |
|
||||
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold answers the tool's latest recorded 400, 404 or 410 response with `state.view` `missing`, or is a tool error when none was recorded; one lacking a field every recording carried is a tool error; a body is never invented. |
|
||||
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"`, or the recorded members still held. A call with no query argument to a `membership: "all"` view answers from a checked line of the collection its path arguments name that only pages its list, with `replay.exact_match` false. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
|
||||
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
|
||||
| `recorded` | The recording, as replay answers it. |
|
||||
| View | Answer |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity the path arguments name, rendered into the recorded envelope of `envelope_ordinal`. Arguments match as replay matches them: each collection segment equals the held collection's segment as written or after percent-decoding, and the identity is the last argument percent-decoded, then as written; values are never re-split on `/`. Any entity the state holds answers, including one `data/add` loaded; query arguments only decide `replay.exact_match`. An entity the state does not hold, or one lacking a field every recording carried, is a tool error; a body is never invented. |
|
||||
| `list` | When the arguments name one of the lines the view checked exactly (its latest recordings before the first related write; every path and query value equal to the recording's, and every recorded query parameter supplied), that recorded envelope with its array rendered from the state: every entity of the collection in state order for `membership: "all"`, or the recorded members still held. A member lacking a field every recording carried makes the call a tool error rather than a partial list. |
|
||||
| `document` | When the arguments name the document's checked line exactly, the state's value for it; a tool error when the state holds none. |
|
||||
| `recorded` | The recording, as replay answers it. |
|
||||
|
||||
A list or document call that names no checked line, including a line recorded
|
||||
after a write or with other query values, answers from the recording. Every
|
||||
answer has the replay shape plus `state`: `{"source": "state", "view": …}`
|
||||
naming the entity, collection or document path, or `{"source": "recording",
|
||||
"reason": …}` with the view's reason, `no-view`, `unrecorded-request` or `page-exceeded` (a call naming a page of a whole list that the collection has outgrown). A
|
||||
"reason": …}` with the view's reason, `no-view` or `unrecorded-request`. A
|
||||
state answer carries the base record's status, content type and `replay`
|
||||
provenance with `truncated: false`; one whose JSON would exceed 256 KiB is
|
||||
refused as a `resource_constraint` tool error rather than cut. Extraction does
|
||||
not state-back a document whose rendered JSON exceeds 256 KiB; that read stays
|
||||
recorded (`rendered-oversize`) and replays the captured body. Query `POST`s
|
||||
stay served and answer from their recordings with the reason `query-body`.
|
||||
|
||||
`POST`, `PUT`, `PATCH` and `DELETE` tools are served only when the state's
|
||||
[write plan](web-network-captures.md#environment-state) for them is served:
|
||||
the capture's later reads confirmed the effect the plan models. Other write
|
||||
tools are not served, so no call claims a change it does not make; each one's
|
||||
plan states why in its `reason`. A served write changes the live state, which
|
||||
every MCP session and `data/get` then see:
|
||||
|
||||
| Effect | Call |
|
||||
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `create` | Adds an entity to the collection the path names, built from the observed entity with each ruled field taken from the call or generated, at the plan's `insert` position. A plan tied to its exemplar's collection writes only there. |
|
||||
| `update` | Sets the sent members on the entity the path names, constants for members accepted only with their recorded value, and the clock, increment and slug changes the plan models. |
|
||||
| `delete` | Removes the entity the path names. |
|
||||
| `document` | Merges the body into the plan's document object, or replaces it for `PUT`. |
|
||||
|
||||
Its answer has the status and content type of the plan's recorded response,
|
||||
with the written entity or its identity rendered in, no body, or the recorded
|
||||
body. A call whose body is not an object, sends a member the capture did not
|
||||
show stored, sends a fixed member with another value or a member of another
|
||||
JSON type, or lacks a required member is a tool error naming the accepted
|
||||
members; so is a rendered answer missing a field the recorded response
|
||||
carried, and an update or delete of an entity the state does not hold answers
|
||||
the recorded 400, 404 or 410 of the tool or of an item view of the type, if
|
||||
any. A refused call leaves the state unchanged. Values a write generates are
|
||||
deterministic: the clock starts at `write_clock` and advances one second per
|
||||
served write, sequences take one past both the largest held value and the
|
||||
largest they generated, and UUIDs and tokens derive from the state Evidence
|
||||
id, tool, field and write count, skipping held values. `data/reset` and
|
||||
`data/add` restart the clock, counter and sequences, so the same calls after a
|
||||
reset give the same state. Answers carry `state: {"source": "state", "view":
|
||||
"create" | "update" | "delete", …}` or `"document-write"`.
|
||||
recorded (`rendered-oversize`) and replays the captured body. Serving is
|
||||
read-only: `POST`, `PUT`, `PATCH` and `DELETE` tools, other than query `POST`s,
|
||||
are not served, so nothing a call does changes the state. Query `POST`s stay
|
||||
served and answer from their recordings with the reason `query-body`.
|
||||
|
||||
The data plane serves three JSON-RPC methods:
|
||||
|
||||
@@ -504,29 +470,18 @@ names its category in `error.data.reason`: `not-found`, `permission`,
|
||||
downloads, and `cancelled` for a download the server's stop or its caller's
|
||||
departure ended.
|
||||
|
||||
Served creates also bound additional collection history since the last reset
|
||||
or `data/add` to 1 MiB of deterministic accounting: UTF-8 bytes for each new
|
||||
collection spelling and its possible sequence keys, plus a 64-byte allowance
|
||||
per collection and potential sequence-key record. The score bounds identifier
|
||||
payload and a fixed record allowance; it is not a measurement or guarantee of
|
||||
process memory. Collections
|
||||
already present in the Evidence or loaded state are part of the state itself;
|
||||
the store indexes their unique spellings so they remain known after deletion,
|
||||
but they do not consume this additional-history budget. A create that would
|
||||
exceed the budget is refused before the state changes. `data/reset` or `data/add`
|
||||
clears generated history; if the same calls would hit the limit again, use
|
||||
fewer distinct collection paths. The 1 MiB score ceiling is 1/32 of the
|
||||
separate 32 MiB state-intake ceiling, reserving a small, explicit allowance
|
||||
for identifiers retained beyond the loaded state. A loaded-state path that is
|
||||
later deleted and then introduced again may consume the additional-history
|
||||
budget when the store must retain it beyond the source entity.
|
||||
|
||||
Stopping the server cancels every pending call, and a caller that leaves before
|
||||
its reply cancels its own: a download in progress ends, and calls not yet run
|
||||
are answered `-32000` with `error.data.reason` `cancelled` without running. A
|
||||
cancelled call leaves the state unchanged, and shutdown never waits for a
|
||||
download.
|
||||
|
||||
A `file://` state file whose close fails stays owned by the server, and
|
||||
the `data/add` gets `-32000`. The next `file://` load retries that close first
|
||||
and, while it still fails, answers `-32000` without reading another file;
|
||||
inline and `https://` documents still load. Closing the server retries it once
|
||||
more and fails with the unfinished close rather than dropping the descriptor.
|
||||
|
||||
One `/agentenv` response carries at most 64 MiB of replies, twice the 32 MiB
|
||||
a loaded document may hold: room for a `data/get` of a loaded state, while a
|
||||
batch of reads cannot multiply a large state into one body. Replies are
|
||||
@@ -610,17 +565,14 @@ The card declares the operations and what `data/add` accepts:
|
||||
}
|
||||
```
|
||||
|
||||
The state starts as a projection of recorded reads and changes only through
|
||||
served write tools and the data plane. List and document calls answer from the
|
||||
state only for the lines their views checked, and every recorded view answers
|
||||
from recordings; item calls answer for any held entity but render only the
|
||||
envelope one recording carried. Values copied outside entity objects, derived
|
||||
values and counters other than those that count a list stay as recorded, and a
|
||||
served write changes only the one entity or document it writes: side effects
|
||||
on counters, relations and copies of entities in other collections, reordering
|
||||
after an update and referential rules on delete are not modelled. An exported
|
||||
bundle carries no state file: its server derives the state at startup, as the
|
||||
export does to predict what it serves.
|
||||
The state is a projection of recorded reads. List and document calls answer
|
||||
from the state only for the lines their views checked, and every recorded view
|
||||
answers from recordings; item calls answer for any held entity but render only
|
||||
the envelope one recording carried. Values copied outside entity objects,
|
||||
derived values and counters other than those that count a list stay as
|
||||
recorded; no write has an effect. An exported bundle carries no state file:
|
||||
its server derives the state at startup, as the export does to predict what it
|
||||
serves.
|
||||
|
||||
## Export a bundle
|
||||
|
||||
@@ -688,8 +640,8 @@ name from Python sources the image does not have.
|
||||
|
||||
`verify.py` is the framework's `async def verify(mcp_url)` shape over its
|
||||
Python MCP client: it lists the tools and checks the count equals the number
|
||||
the bundle serves (from the state, the manifest's read tools and the write
|
||||
tools whose effect the capture confirmed; every manifest tool in replay), calls one tool with no arguments and checks the call did not
|
||||
the bundle serves (the manifest's read tools from the state, every manifest
|
||||
tool in replay), calls one tool with no arguments and checks the call did not
|
||||
fail and its `status` is the expected one, and returns one `serve` result
|
||||
whose description reports both checks. The tool is the first served `GET` tool
|
||||
without path parameters whose answer to no arguments is not an error, and the
|
||||
@@ -707,9 +659,8 @@ The README written into the bundle records the capture artifact's SHA-256, the
|
||||
capture and manifest Evidence ids, the manifest view digest, the served tool
|
||||
count, the derived state's Evidence id or that no state is served, and the
|
||||
package source. It says what the environment serves: read tools from the state
|
||||
the capture observed, the `/agentenv` data plane on it and the write tools
|
||||
whose effect the capture's later reads confirmed, replay when the state is
|
||||
unavailable, and `--replay-only` for pure replay; it
|
||||
the capture observed, the `/agentenv` data plane on it and no write tools yet,
|
||||
replay when the state is unavailable, and `--replay-only` for pure replay; it
|
||||
says which applies to this bundle, with the error code and message when the
|
||||
state is unavailable, and that neither is emulation and served
|
||||
content is untrusted data. It states that the recorded content belongs to the
|
||||
|
||||
@@ -215,87 +215,3 @@ Large Evidence envelopes may require a caller-configured MCP transport receive
|
||||
buffer because envelope copies add overhead. Requests honor cancellation during
|
||||
snapshot reading, directory traversal and payload hashing. The operation reads
|
||||
regular files only and rejects symbolic links.
|
||||
|
||||
## Byte entropy triage
|
||||
|
||||
`inspect_byte_entropy` / `inspect-byte-entropy` profiles any local regular file
|
||||
without an analysis engine, execution, extraction or network access.
|
||||
|
||||
```sh
|
||||
rea inspect-byte-entropy ./firmware.bin --block-size 4096 --json
|
||||
rea inspect-byte-entropy ./firmware.bin --offset 4096 --length 65536 --json
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_byte_entropy",
|
||||
"arguments": { "path": "/artifacts/firmware.bin", "detail": "summary" }
|
||||
}
|
||||
```
|
||||
|
||||
Complete Evidence includes `artifact` (original path, byte size and SHA-256),
|
||||
`block_size`, `window`, `file_entropy`, `thresholds`, `blocks`, `runs` and
|
||||
`limitations`. For a 4096-byte zero-filled file, the entropy fields are:
|
||||
|
||||
```json
|
||||
{
|
||||
"block_size": 4096,
|
||||
"window": { "offset": 0, "length": 4096 },
|
||||
"file_entropy": 0,
|
||||
"thresholds": { "high": 7.2, "low": 1 },
|
||||
"blocks": [{ "offset": 0, "length": 4096, "entropy": 0 }],
|
||||
"runs": [{ "offset": 0, "length": 4096, "classification": "low" }]
|
||||
}
|
||||
```
|
||||
|
||||
Offsets and lengths are decimal byte counts. Blocks begin at the requested
|
||||
`offset` (default zero); omitted `length` extends to EOF. Explicit windows past
|
||||
EOF fail rather than silently clipping. The final block may be shorter.
|
||||
`block_size` must be a power of two from 256 through 65536, default 4096. A
|
||||
window may produce at most 32,768 block rows; larger requests fail before the
|
||||
file scan with a capacity error. Choose a shorter window or larger block size.
|
||||
SHA-256 and `file_entropy` always cover the entire file, including when a window
|
||||
is selected. Empty files and empty windows are accepted; empty entropy is zero
|
||||
by convention. The tool reads the size the file reports. A file that reports size
|
||||
0, such as a Linux `/proc` entry, returns no blocks with a limitation that the
|
||||
result does not show the file is empty. File identity, size and timestamps are checked across the read;
|
||||
detected changes produce an error rather than Evidence.
|
||||
|
||||
Shannon entropy is reported in bits per byte from 0 to 8, rounded to four decimal
|
||||
places. Adjacent blocks with reported entropy at least 7.2 form `high` runs;
|
||||
blocks at most 1.0 form `low` runs. Other blocks interrupt runs. Entropy does not
|
||||
prove encryption, and this measure cannot distinguish encrypted from compressed
|
||||
data. Small blocks can underestimate entropy; the result is a triage clue.
|
||||
|
||||
MCP `detail: "summary"` retains complete Evidence in the session and returns
|
||||
file entropy, block/run counts, thresholds, the selected window, and the ten
|
||||
longest runs (ties ordered by file offset). Use its `parent_evidence_id` to
|
||||
retrieve complete Evidence, or request its summary with `inspect_analysis_view`.
|
||||
The CLI returns complete Evidence through the same service. Summary detail
|
||||
requires a session capable of retaining Evidence.
|
||||
|
||||
Reads use one block-sized buffer and a 256-entry histogram. Complete output
|
||||
can become very large because it includes one row per selected block, and
|
||||
summary delivery still retains that full list. Use a shorter length or a
|
||||
larger block size to reduce it. Full Evidence and its serialization require
|
||||
memory proportional to the number of selected blocks. The 32,768-row ceiling bounds the actual retained block objects and
|
||||
their serialized representation: a cap-sized run below produced 1,514,877
|
||||
serialized bytes and increased V8 heap use by 21.2 MiB on the measured host.
|
||||
The cap therefore bounds this output independently of file size; it is not a
|
||||
portable process-memory guarantee. Narrow the window or increase block size to
|
||||
reduce retained rows further.
|
||||
|
||||
Local service measurement (macOS arm64, Node 24.18.0, 100,000,000-byte file,
|
||||
4096-byte blocks, file already in the OS cache): first process run took 1.113 s
|
||||
wall / 1.148 s CPU with 145.3 MiB peak RSS; a second run in the same process took
|
||||
1.015 s / 1.035 s CPU, with cumulative peak RSS 155.8 MiB. Each read 100,000,000
|
||||
bytes and produced 24,415 blocks. Measurements include Evidence construction,
|
||||
summary projection and JSON serialization, with no child processes. Complete
|
||||
Evidence serialized to 1,122,303 bytes and its summary to 2,705 bytes. A
|
||||
cap-sized 134,217,728-byte sparse file with 4096-byte blocks produced 32,768
|
||||
rows and 1,514,877 serialized bytes; V8 heap use increased by 21.2 MiB and RSS
|
||||
by 18.1 MiB in that process. This separate measurement used the same macOS
|
||||
arm64 host and Node version. Cold disk I/O, token counts and an agent
|
||||
investigation were not measured; there was no previous equivalent tool to
|
||||
benchmark. These figures do not establish a portable peak-memory bound,
|
||||
constant memory for retained Evidence or faster agent task completion.
|
||||
+1
-20
@@ -155,16 +155,6 @@ dependency edges, and the section's file range. A missing `.dep-v0` section is
|
||||
null metadata, not proof that the binary has no dependencies. See
|
||||
[Rust binaries](rust-binaries.md).
|
||||
|
||||
Inspect ELF PT_NOTE and SHT_NOTE tables without executing the image:
|
||||
|
||||
```bash
|
||||
rea inspect-elf-notes /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The workflow reports note owners, types, descriptor hashes, and GNU build-id
|
||||
or ABI-tag fields when those descriptors match. A missing note table is
|
||||
observed absence of those declarations. See [ELF note tables](elf-notes.md).
|
||||
|
||||
Configure [Hopper or Ghidra](installation.md#hopper), the
|
||||
[IDA adapter](ida-provider.md), or [Kuna](kuna-provider.md) before analyzing a
|
||||
native target. Substitute
|
||||
@@ -424,8 +414,7 @@ evidence file's `evidence_id`; otherwise the command refuses with
|
||||
`invalid_request` before binding. Before binding it derives the
|
||||
`environment-state` view of the capture with the manifest's `template_rules`,
|
||||
as `inspect-analysis-view` would, and serves from it: read tools whose views
|
||||
are state-backed answer from a live state, write tools are served only where
|
||||
the capture's later reads confirmed their effect, and
|
||||
are state-backed answer from a live state, write tools are not served, and
|
||||
`/agentenv` serves `data/reset`, `data/add` and `data/get`. When the state
|
||||
cannot be derived, or its catalog exceeds the discovery reservation, it logs a
|
||||
warning with the error code and message and serves replay; `--replay-only` serves replay without deriving. In replay every
|
||||
@@ -562,14 +551,6 @@ command, objdump, Rizin and GDB, and the agent environment server
|
||||
signal-derived status. This describes the current source routing; it is not a
|
||||
claim that every command or provider runtime has been independently exercised.
|
||||
|
||||
Commands with `--evidence_output` select their delivered view, then publish the
|
||||
complete Evidence bundle atomically. Cancellation before publication leaves no
|
||||
new file. A publication that has committed can win a later cancellation; the
|
||||
`cancelled` result then keeps the written path and byte count under
|
||||
`details.diagnostics.evidence_bundle`. If staging cleanup stays incomplete
|
||||
after commit, the `cleanup_incomplete` error reports the same file as
|
||||
`details.published_path` and `details.bytes`.
|
||||
|
||||
When the reader of piped output closes early, as `rea ... | head` does, the
|
||||
CLI stops writing, prints no error and exits `141` (`128 + 13`, the status of a
|
||||
SIGPIPE-terminated writer). The operation itself may have completed; only its
|
||||
|
||||
@@ -247,8 +247,8 @@ capture provides them.
|
||||
|
||||
ASAR observation reads the header through one open file handle and validates its
|
||||
declared lengths against the archive size. Header/index admission is the same
|
||||
budget used by static ASAR reads: 16 MiB / 16777216 bytes by default, or
|
||||
`REA_ASAR_MAX_HEADER_BYTES` (at most 256 MiB) when set. An oversized header is
|
||||
not parsed; selecting a member path does not bypass admission, and the page
|
||||
stays unauthorized. The header cache separately retains at most that many bytes
|
||||
of already-admitted encoded headers across eight archives. Evicting that cache does not raise admission.
|
||||
fixed 16 MiB / 16777216 byte budget used by static ASAR reads. This release has
|
||||
no runtime override. An oversized header is not parsed; selecting a member path
|
||||
does not bypass admission, and the page stays unauthorized. The header cache
|
||||
separately retains at most 16 MiB of already-admitted encoded headers across
|
||||
eight archives. Evicting that cache does not raise admission.
|
||||
@@ -1,53 +0,0 @@
|
||||
# ELF note tables
|
||||
|
||||
`inspect_elf_notes` / `inspect-elf-notes` answers whether one selected local
|
||||
ELF32/ELF64 image declares `PT_NOTE` program headers or `SHT_NOTE` sections,
|
||||
which note owners and types those tables store, and which GNU build-id or
|
||||
ABI-tag fields those notes carry. It reads the file without executing it and
|
||||
does not require a disassembler, debug package, or package catalog.
|
||||
|
||||
```bash
|
||||
rea inspect-elf-notes /absolute/path/to/program --json
|
||||
```
|
||||
|
||||
The CLI resolves relative paths against the caller's working directory. MCP
|
||||
requires an absolute filesystem path and preserves that exact selection:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "inspect_elf_notes",
|
||||
"arguments": { "path": "/artifacts/program" }
|
||||
}
|
||||
```
|
||||
|
||||
## Observations
|
||||
|
||||
The inline Evidence result contains the artifact path, SHA-256, and byte length.
|
||||
`tables` lists each declared note program header or section, including the
|
||||
section name when a string table is present. Each `entries[]` item reports owner,
|
||||
type, descriptor SHA-256, and a hex preview for descriptors of 64 bytes or less.
|
||||
|
||||
For GNU notes, `gnu.build_id_hex`, `gnu.abi_tag`, or `gnu.gold_version` are
|
||||
filled when the descriptor matches that type. Other owners keep `gnu` null.
|
||||
|
||||
`PT_NOTE` and `SHT_NOTE` are examined independently. The same file range may
|
||||
appear twice when both tables declare it.
|
||||
|
||||
A missing table list is observed absence of those declarations. It is not proof
|
||||
the image never carried notes in another container or overlay.
|
||||
|
||||
## Limits and failures
|
||||
|
||||
The selected file is limited to 64 MiB by default (512 MiB maximum). At most
|
||||
32 note entries are examined by default across every table. Retained note
|
||||
names, section names, and GNU descriptor text share that file's byte budget,
|
||||
and repeated table ranges count again. Each descriptor range is hashed once,
|
||||
and each GNU version's trailing NUL run is scanned once; neither walk may
|
||||
exceed that byte ceiling. Malformed ELF headers, truncated header
|
||||
tables, declared note tables that extend past the file, and truncated
|
||||
note entries fail without partial success. ELF extended
|
||||
numbering (`PN_XNUM`, including a section count stored when `e_shnum` is 0) is
|
||||
unsupported.
|
||||
|
||||
This workflow does not execute the image or verify GNU build-id uniqueness
|
||||
against package catalogs.
|
||||
+5
-15
@@ -109,21 +109,11 @@ module record and no dependency records has an observed empty dependency list.
|
||||
|
||||
## Supported boundary
|
||||
|
||||
The built-in TypeScript reader supports ELF, PE and thin Mach-O images, and one
|
||||
selected slice of a universal Mach-O, with Go's inline or historical pointer
|
||||
build-info encoding. Source ranges refer to the original stable file snapshot,
|
||||
including pointer-backed strings. It does not scan arbitrary byte strings as
|
||||
compiler metadata.
|
||||
|
||||
A universal (FAT) Mach-O needs `architecture` (`x86`, `x86_64`, `arm`
|
||||
or `arm64`; CLI `--architecture`). REA reads its own FAT table, without `lipo`,
|
||||
and validates the selected slice's range, alignment and Mach-O header CPU
|
||||
against the table. The result keeps the universal file as `artifact` (its
|
||||
SHA-256 and size), adds `selected_slice` with the slice offset, length, raw CPU
|
||||
type/subtype and the FAT table digest, and reports every source range as a
|
||||
file offset inside that slice. Without `architecture`, or when the file has no
|
||||
such slice or several, the request fails on `architecture` and lists the slices
|
||||
present. `architecture` on any other image is rejected rather than ignored.
|
||||
The built-in TypeScript reader supports ELF, PE and thin Mach-O images with Go's
|
||||
inline or historical pointer build-info encoding. Source ranges refer to the
|
||||
original stable file snapshot, including pointer-backed strings. It does not
|
||||
scan arbitrary byte strings as compiler metadata or interpret a fat Mach-O
|
||||
container as one selected architecture.
|
||||
|
||||
For ELF images without a named build-info section, discovery checks each
|
||||
nonempty writable, non-executable load segment in program-header order. This
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
|
||||
| Task | Guides |
|
||||
| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Identify a desktop installation | [Runtime candidates, evidence and follow-up files](desktop-applications.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [ELF notes](elf-notes.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
|
||||
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
|
||||
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
|
||||
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
|
||||
|
||||
@@ -106,16 +106,13 @@ only reduces delivery; source selection reduces text acquisition and analysis.
|
||||
Explicit native provider or snapshot routing rejects `--source-path` rather
|
||||
than ignoring it.
|
||||
|
||||
ASAR header/index admission defaults to 16 MiB / 16777216 bytes. Set
|
||||
`REA_ASAR_MAX_HEADER_BYTES` in the environment that starts REA to a decimal
|
||||
byte count of at most 268435456 (256 MiB) to raise or lower it; restart REA
|
||||
after changing it. Invalid values are configuration errors, never a silent
|
||||
fallback. The archive index is read before any member can be selected, so
|
||||
`source_paths`, `--source-path`, `detail: "summary"`, `max_heap_mb`, and
|
||||
available host memory do not bypass or raise it. When the header exceeds the
|
||||
effective allowance, the failure names it, whether it came from the default or
|
||||
the setting, and the setting's limits. Alternatively, analyze an extracted
|
||||
application directory, which does not admit an ASAR header. Repeating the unchanged
|
||||
ASAR header/index admission is a fixed 16 MiB / 16777216 bytes for this
|
||||
release. There is no runtime override. The archive index is read before any
|
||||
member can be selected, so `source_paths`, `--source-path`, `detail:
|
||||
"summary"`, `max_heap_mb`, and available host memory do not bypass or raise
|
||||
it. When the header exceeds that allowance, the failure names this budget.
|
||||
Analyze an extracted application directory, which does not admit an ASAR
|
||||
header, or supply an archive whose header fits. Repeating the unchanged
|
||||
request fails the same way.
|
||||
|
||||
## Read the analysis
|
||||
|
||||
@@ -467,14 +467,6 @@ complete observation.
|
||||
vtable descriptors. Records include exact virtual addresses/file offsets and
|
||||
artifact evidence. Vtable indexes are metadata word offsets; witness indexes
|
||||
start after the conformance header. Names unavailable in metadata remain null.
|
||||
The byte reader holds the whole selected file in memory, so it admits files up
|
||||
to 64 MiB / 67108864 bytes by default. Select a larger budget with
|
||||
`max_input_bytes` (CLI `--max-input-bytes`), at most 1 GiB / 1073741824
|
||||
bytes. The whole file counts, including every slice of a universal Mach-O.
|
||||
Results record `input_admission` with the effective budget, its source
|
||||
(`default` or `caller`) and the bytes retained. A refusal names the observed
|
||||
file size, the effective budget and its source, and either the budget to
|
||||
select or, above 1 GiB, a deep analysis provider.
|
||||
The `swift_class_vtable_descriptors` coverage counters count examined vtable
|
||||
slots and successfully decoded slots, rather than type-section entries.
|
||||
The `objc_class_method_ivar_metadata` counters count admitted class and
|
||||
|
||||
@@ -145,7 +145,6 @@ npx rea-agents@latest setup
|
||||
| بنية ELF دون تشغيل | الأقسام والمقاطع والرموز ومعلومات إعادة التموضع الأصلية، وآليات الحماية المحتملة المستدلّ عليها بالفحص الثابت | pwntools يوفّره المستدعي على Linux x64؛ [تشخيص الملفات الثنائية](../binary-diagnostics.md) |
|
||||
| شيفرة EVM البايتية | محددات التوجيه، وإزاحات البايتات، والمعاملات المستنتجة، وقابلية تغيير الحالة | مدخل محلي يحمل بايتات خامًا أو تمثيلًا سداسيًا عشريًا على Linux x64؛ [دليل EVM دون تشغيل](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| أعطال Linux المسجّلة | سجلات note الخام، ومسجّلات المعالج/الإشارات لكل خيط مسجّل، وترشيحات اختيارية لخرائط الذاكرة | pwntools يوفّره المستدعي على Linux x64؛ GDB/pwndbg اختياريان؛ [الأعطال المسجّلة](../recorded-crashes.md) |
|
||||
| تثبيتات سطح المكتب | عائلات وقت التشغيل، والملفات الداعمة، وعلامات المكونات، والخطوات التالية | فحص محدود للملفات والترويسات；[تطبيقات سطح المكتب](../desktop-applications.md) |
|
||||
| JavaScript / Electron | الوحدات، وعمليات الاستيراد، وخرائط المصدر، والمسارات، وIPC، وعلاقات إضافات الشيفرة الأصلية | Node.js وnpm؛ [تحليل التطبيقات](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -145,7 +145,6 @@ REA benötigt Node.js 22.x (>=22.19), 24.x (>=24.11) oder 26+ sowie npm. Zusätz
|
||||
| Offline-ELF-Struktur | Sektionen, Segmente, ursprüngliche Symbole/Relokationen und mögliche Schutzmaßnahmen aus statischer Analyse | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; [Binärdiagnostik](../binary-diagnostics.md) |
|
||||
| EVM-Bytecode | Dispatch-Selektoren, Byte-Offsets, abgeleitete Argumente und Veränderbarkeit des Zustands | Lokale Eingabe mit Rohbytes oder Hexdarstellung unter Linux x64; [Offline-EVM-Leitfaden](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Aufgezeichnete Linux-Abstürze | Unverarbeitete Note-Einträge, Register/Signale aller aufgezeichneten Threads und optionale Zuordnungskandidaten | Vom Aufrufer bereitgestelltes pwntools unter Linux x64; optional GDB/pwndbg; [aufgezeichnete Abstürze](../recorded-crashes.md) |
|
||||
| Desktop-Installationen | Laufzeitkandidaten, Quelldateien, Komponenten und nächste Schritte | Begrenzte Verzeichnis- und Headerprüfung；[Desktop-Anwendungen](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Module, Imports, Source Maps, Routen, IPC und Beziehungen zu nativen Add-ons | Node.js und npm; [Anwendungsanalyse](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -149,7 +149,6 @@ npx rea-agents@latest setup
|
||||
| Δομή ELF χωρίς εκτέλεση | Ενότητες, τμήματα, αρχικά σύμβολα/εγγραφές μετεγκατάστασης και ενδείξεις πιθανών μέτρων προστασίας από στατική ανάλυση | pwntools που παρέχει ο χρήστης σε Linux x64· [διαγνωστικά δυαδικών αρχείων](../binary-diagnostics.md) |
|
||||
| Δυαδικά αρχεία Go | Ενσωματωμένη έκδοση μεταγλωττιστή, modules, αντικαταστάσεις και ρυθμίσεις μεταγλώττισης | Ενσωματωμένη στατική εξέταση ELF/PE/thin Mach-O· δεν απαιτείται εγκατάσταση του Go |
|
||||
| Δυαδικά αρχεία Rust | Ενσωματωμένα ονόματα, εκδόσεις, πηγές και σχέσεις εξάρτησης των crates από το cargo-auditable | Ενσωματωμένη στατική εξέταση ELF/PE/thin Mach-O· [δυαδικά αρχεία Rust](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Αρχείο WASM | Έλεγχο εγκυρότητας, ενότητες, ακριβείς μορφές εισαγωγών/εξαγωγών σε WAT και αποκωδικοποιημένο WAT | WABT 1.0.42 που παρέχει ο χρήστης· `inspect_wasm_artifact` |
|
||||
| Bytecode EVM | Selectors επιλογής συνάρτησης, μετατοπίσεις σε bytes, εκτιμώμενα ορίσματα και δυνατότητα μεταβολής της κατάστασης | Τοπική είσοδος σε ακατέργαστα bytes ή δεκαεξαδική μορφή σε Linux x64· [οδηγός EVM χωρίς εκτέλεση](../evm-bytecode.md) |
|
||||
| Βάσεις δεδομένων SQLite | Πίνακες, στήλες, ευρετήρια, όψεις, triggers και επιλεγμένες γραμμές κανονικών πινάκων | Τοπικό στιγμιότυπο βάσης δεδομένων/WAL· Node.js 24.x >=24.15 ή 26+ με εγγενή όρια SQLite |
|
||||
|
||||
@@ -145,7 +145,6 @@ REA requiere Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, además de npm. Las h
|
||||
| Estructura ELF sin ejecutar | Secciones, segmentos, símbolos/reubicaciones originales y posibles medidas de mitigación detectadas mediante análisis estático | pwntools proporcionado por quien llama, en Linux x64; [diagnóstico de binarios](../binary-diagnostics.md) |
|
||||
| Bytecode EVM | Selectores de despacho, desplazamientos en bytes, argumentos inferidos y mutabilidad | Entrada local con bytes sin procesar o representación hexadecimal, en Linux x64; [guía de EVM sin ejecutar](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Fallos de Linux registrados | Registros note sin procesar, registros/señales de cada hilo registrado y posibles correspondencias de memoria opcionales | pwntools proporcionado por quien llama, en Linux x64; GDB/pwndbg opcionales; [fallos registrados](../recorded-crashes.md) |
|
||||
| Instalaciones de escritorio | Posibles entornos de ejecución, archivos de evidencia, componentes y siguientes pasos | Inspección acotada de directorios y cabeceras；[Aplicaciones de escritorio](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Módulos, importaciones, mapas de código fuente, rutas, IPC y relaciones con complementos nativos | Node.js y npm; [análisis de aplicaciones](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -137,7 +137,6 @@ REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، بههمراه np
|
||||
| ساختار ELF آفلاین | بخشها، سگمنتها، نمادها و relocationهای اصلی و گزینههای احتمالی کاهش آسیبپذیری ایستا | pwntools تأمینشده توسط فراخواننده روی Linux x64؛ [تشخیص باینری](../binary-diagnostics.md) |
|
||||
| بایتکد EVM | selectorهای dispatch، offsetهای بایتی، آرگومانهای استنباطشده و قابلیت تغییر وضعیت | دادهٔ خام/hex محلی روی Linux x64؛ [راهنمای EVM آفلاین](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| کرشهای ثبتشدهٔ لینوکس | یادداشتهای خام، رجیسترها/سیگنالهای تمام threadهای ثبتشده و گزینههای احتمالی نگاشت | pwntools تأمینشده توسط فراخواننده روی Linux x64؛ GDB/pwndbg اختیاری؛ [کرشهای ثبتشده](../recorded-crashes.md) |
|
||||
| نصبهای دسکتاپ | نامزدهای زمان اجرا، فایلهای شاهد، نشانههای مؤلفهها و مراحل بعدی | بررسی محدود پوشهها و سرآیندها；[برنامههای دسکتاپ](../desktop-applications.md) |
|
||||
| JavaScript / Electron | ماژولها، importها، source mapها، مسیرها، IPC و ارتباط افزونههای بومی | Node.js و npm؛ [تحلیل برنامه](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -149,7 +149,6 @@ REA nécessite Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, ainsi que npm. Les
|
||||
| Artefact WASM | Validation, sections, formes WAT exactes d'import/export et WAT décodé | WABT 1.0.42 fourni par l'utilisateur ; `inspect_wasm_artifact` |
|
||||
| Bytecode EVM | Sélecteurs de dispatch, offsets en octets, arguments inférés et mutabilité | Fichier local brut ou hexadécimal sous Linux x64 ; [guide EVM hors ligne](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Crashs Linux enregistrés | Notes brutes, registres et signaux de chaque thread enregistré, et correspondances de mappings facultatives | pwntools fourni par l'utilisateur sous Linux x64 ; GDB/pwndbg en option ; [crashs enregistrés](../recorded-crashes.md) |
|
||||
| Installations de bureau | Environnements candidats, fichiers justificatifs, composants et étapes suivantes | Inspection bornée des répertoires et en-têtes；[Applications de bureau](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Modules, imports, source maps, routes, IPC et relations avec les modules natifs | Node.js et npm ; [analyse d'applications](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -152,7 +152,6 @@ Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis:
|
||||
| Artefak WASM | Validasi, bagian, bentuk WAT import/ekspor yang tepat, dan WAT yang didekode | WABT 1.0.42 yang disediakan pemanggil; `inspect_wasm_artifact` |
|
||||
| Bytecode EVM | Selector dispatch, offset byte, argumen yang disimpulkan, dan sifat mutabilitas | Pembawa data raw/hex lokal; [panduan EVM offline](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Crash Linux yang direkam | Catatan mentah, register/sinyal setiap thread yang direkam, dan kandidat pemetaan opsional | pwntools yang disediakan pemanggil; GDB/pwndbg opsional; [crash yang direkam](../recorded-crashes.md) |
|
||||
| Instalasi desktop | Kandidat runtime, berkas pendukung, penanda komponen, dan langkah berikutnya | Pemeriksaan direktori dan header terbatas；[Aplikasi desktop](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -149,7 +149,6 @@ REA richiede Node.js 22.x (>=22.19), 24.x (>=24.11) o 26+, oltre a npm. Strument
|
||||
| Artefatto WASM | Convalida, sezioni, forme WAT di import/export esatte e WAT decodificato | WABT 1.0.42 fornito dal chiamante; `inspect_wasm_artifact` |
|
||||
| Bytecode EVM | Selettori di dispatch, offset dei byte, argomenti inferiti e mutabilità | Carrier raw/hex locale; [guida EVM offline](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Crash Linux registrati | Note grezze, registri/segnai di ogni thread registrato e candidati mapping opzionali | pwntools fornito dal chiamante; GDB/pwndbg opzionale; [crash registrati](../recorded-crashes.md) |
|
||||
| Installazioni desktop | Runtime candidati, file di supporto, componenti e passi successivi | Ispezione limitata di directory e intestazioni；[Applicazioni desktop](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Moduli, import, source map, route, IPC e relazioni con add-on nativi | Node.js e npm; [analisi delle applicazioni](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -145,7 +145,6 @@ REA には Node.js 22.x（>=22.19）、24.x（>=24.11）、または 26+ と npm
|
||||
| オフライン ELF レイアウト | セクション、セグメント、元のシンボル／再配置情報、静的解析によるセキュリティ対策の候補 | Linux x64 で利用者が用意する pwntools；[バイナリ診断](../binary-diagnostics.md) |
|
||||
| EVM バイトコード | ディスパッチセレクター、バイトオフセット、推定された引数と状態変更の可否 | Linux x64 上のローカルの生バイト／16 進数の入力ファイル；[オフライン EVM ガイド](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| 記録済みの Linux クラッシュ | 生の note レコード、記録された全スレッドのレジスター／シグナル、任意のマッピング候補 | Linux x64 で利用者が用意する pwntools；GDB/pwndbg は任意；[記録済みクラッシュ](../recorded-crashes.md) |
|
||||
| デスクトップのインストール先 | ランタイム候補、根拠ファイル、コンポーネントの目印、次の調査手順 | 上限付きのディレクトリ・ヘッダー検査；[デスクトップアプリ](../desktop-applications.md) |
|
||||
| JavaScript / Electron | モジュール、インポート、ソースマップ、ルート、IPC、ネイティブアドオンの関係 | Node.js と npm；[アプリケーション分析](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -153,7 +153,6 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
|
||||
| 오프라인 ELF 레이아웃 | 섹션, 세그먼트, 원본 심볼/재배치 정보, 정적 분석으로 얻은 보안 완화 기법 후보 | Linux x64에서 호출자가 제공하는 pwntools; [바이너리 진단](../binary-diagnostics.md) |
|
||||
| Go 바이너리 | 내장된 컴파일러 버전, 모듈, `replace` 지시어, 빌드 설정 | REA 내장 ELF/PE/thin Mach-O 정적 검사; Go 설치 불필요 |
|
||||
| Rust 바이너리 | cargo-auditable이 내장한 크레이트 이름, 버전, 소스, 의존성 관계 | REA 내장 ELF/PE/thin Mach-O 정적 검사; [Rust 바이너리](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| WASM 아티팩트 | 유효성 검사, 섹션, import/export의 정확한 WAT 형식, 디코딩된 WAT | 호출자 제공 WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM 바이트코드 | 디스패치 선택자, 바이트 오프셋, 추론한 인자와 상태 변경 가능 여부 | Linux x64의 로컬 원시 바이트/16진수 입력 파일; [오프라인 EVM 가이드](../evm-bytecode.md) |
|
||||
| SQLite 데이터베이스 | 테이블, 열, 인덱스, 뷰, 트리거, 선택한 일반 테이블의 행 | 로컬 데이터베이스/WAL 스냅샷; Node.js 24.15 이상의 24.x 또는 26 이상(네이티브 SQLite 한도 적용) |
|
||||
|
||||
@@ -151,7 +151,6 @@ REA-യ്ക്ക് Node.js 22.x (>=22.19), 24.x (>=24.11), അല്ലെ
|
||||
| Offline ELF layout | Sections, segments, യഥാർത്ഥ symbols/relocations, static mitigation candidates | Linux x64-ൽ caller നൽകുന്ന pwntools; [binary diagnostics](../binary-diagnostics.md) |
|
||||
| Go binaries | ഉൾച്ചേർത്ത compiler version, modules, replacements, build settings | Built-in static ELF/PE/thin Mach-O പരിശോധന; Go installation ആവശ്യമില്ല |
|
||||
| Rust binaries | ഉൾച്ചേർത്ത cargo-auditable crate പേരുകൾ, versions, sources, dependency edges | Built-in static ELF/PE/thin Mach-O പരിശോധന; [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| WASM artifact | Validation, sections, കൃത്യമായ import/export WAT രൂപങ്ങൾ, decoded WAT | Caller നൽകുന്ന WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bytecode | Dispatch selectors, byte offsets, അനുമാനിച്ച arguments, mutability | Linux x64-ൽ local raw/hex carrier; [offline EVM guide](../evm-bytecode.md) |
|
||||
| SQLite databases | Tables, columns, indexes, views, triggers, തിരഞ്ഞെടുത്ത സാധാരണ table rows | Local database/WAL snapshot; native SQLite പരിമിതികളോടെ Node.js 24.x >=24.15 അല്ലെങ്കിൽ 26+ |
|
||||
|
||||
@@ -145,7 +145,6 @@ REA wymaga Node.js 22.x (>=22.19), 24.x (>=24.11) lub 26+ oraz npm. Dodatkowe na
|
||||
| Układ pliku ELF w trybie offline | Sekcje, segmenty, oryginalne symbole/relokacje i potencjalne mechanizmy zabezpieczeń wskazane przez analizę statyczną | pwntools dostarczone przez użytkownika w systemie Linux x64; [diagnostyka plików binarnych](../binary-diagnostics.md) |
|
||||
| Kod bajtowy EVM | Selektory wywołań, przesunięcia bajtowe, wywnioskowane argumenty i mutowalność | Lokalny plik z surowymi bajtami lub zapisem szesnastkowym w systemie Linux x64; [przewodnik EVM offline](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Zarejestrowane awarie systemu Linux | Surowe rekordy note, rejestry/sygnały każdego zarejestrowanego wątku i opcjonalni kandydaci na mapowania | pwntools dostarczone przez użytkownika w systemie Linux x64; opcjonalnie GDB/pwndbg; [zarejestrowane awarie](../recorded-crashes.md) |
|
||||
| Instalacje desktopowe | Kandydaci środowisk uruchomieniowych, pliki źródłowe dowodów, komponenty i dalsze kroki | Ograniczona inspekcja katalogów i nagłówków；[Aplikacje desktopowe](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Moduły, importy, mapy źródeł, trasy, IPC i powiązania z natywnymi dodatkami | Node.js i npm; [analiza aplikacji](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -145,7 +145,6 @@ O REA requer Node.js 22.x (>=22.19), 24.x (>=24.11) ou 26+, além do npm. As fer
|
||||
| Estrutura ELF sem execução | Seções, segmentos, símbolos/relocações originais e possíveis mecanismos de mitigação identificados por análise estática | pwntools fornecido pelo chamador no Linux x64; [diagnóstico de binários](../binary-diagnostics.md) |
|
||||
| Bytecode EVM | Seletores de despacho, offsets de bytes, argumentos inferidos e mutabilidade | Entrada local com bytes brutos ou representação hexadecimal no Linux x64; [guia de EVM sem execução](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Falhas do Linux registradas | Registros note brutos, registradores/sinais de cada thread registrada e candidatos opcionais a mapeamentos | pwntools fornecido pelo chamador no Linux x64; GDB/pwndbg opcionais; [falhas registradas](../recorded-crashes.md) |
|
||||
| Instalações desktop | Ambientes candidatos, arquivos de evidência, componentes e próximos passos | Inspeção limitada de diretórios e cabeçalhos；[Aplicações desktop](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Módulos, importações, source maps, rotas, IPC e relações com complementos nativos | Node.js e npm; [análise de aplicativos](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -145,7 +145,6 @@ REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а так
|
||||
| Структура ELF без запуска | Секции, сегменты, исходные символы/релокации и возможные механизмы защиты по данным статического анализа | pwntools, предоставленный вызывающей стороной, в Linux x64; [диагностика бинарных файлов](../binary-diagnostics.md) |
|
||||
| Байт-код EVM | Селекторы диспетчеризации, смещения байтов, предполагаемые аргументы и изменяемость состояния | Локальные необработанные байты или шестнадцатеричное представление в Linux x64; [руководство по EVM без запуска](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Записанные сбои Linux | Необработанные записи note, регистры/сигналы каждого записанного потока и необязательные кандидаты сопоставления с памятью | pwntools, предоставленный вызывающей стороной, в Linux x64; необязательные GDB/pwndbg; [записанные сбои](../recorded-crashes.md) |
|
||||
| Установленные настольные приложения | Возможные среды выполнения, файлы с доказательствами, компоненты и дальнейшие шаги | Ограниченная проверка каталогов и заголовков；[Настольные приложения](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Модули, импорты, карты исходного кода, маршруты, IPC и связи с нативными дополнениями | Node.js и npm; [анализ приложений](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -145,7 +145,6 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26
|
||||
| โครงสร้าง ELF แบบออฟไลน์ | เซกชัน เซกเมนต์ สัญลักษณ์/รีโลเคชันต้นฉบับ และตัวเลือกกลไกป้องกันที่พบจากการวิเคราะห์แบบสถิต | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; [การวินิจฉัยไบนารี](../binary-diagnostics.md) |
|
||||
| ไบต์โค้ด EVM | ตัวเลือกฟังก์ชันสำหรับการเรียก ออฟเซ็ตไบต์ อาร์กิวเมนต์และความสามารถในการเปลี่ยนสถานะที่อนุมานได้ | ไฟล์บนเครื่องที่มีไบต์ดิบหรือข้อมูลเลขฐานสิบหกบน Linux x64; [คู่มือ EVM แบบออฟไลน์](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| ข้อมูลการแครชของ Linux ที่บันทึกไว้ | ระเบียน note ดิบ รีจิสเตอร์/สัญญาณของทุกเธรดที่บันทึกไว้ และตัวเลือกการแมปเพิ่มเติม | pwntools ที่ผู้ใช้จัดเตรียมบน Linux x64; เลือกใช้ GDB/pwndbg เพิ่มเติมได้; [การแครชที่บันทึกไว้](../recorded-crashes.md) |
|
||||
| ไดเรกทอรีติดตั้งแอปเดสก์ท็อป | รันไทม์ที่เป็นไปได้ ไฟล์หลักฐาน ตัวบ่งชี้ส่วนประกอบ และขั้นตอนถัดไป | ตรวจสอบไดเรกทอรีและส่วนหัวโดยจำกัดขอบเขต；[แอปเดสก์ท็อป](../desktop-applications.md) |
|
||||
| JavaScript / Electron | โมดูล การนำเข้า ซอร์สแมป เส้นทาง IPC และความสัมพันธ์กับส่วนเสริมเนทีฟ | Node.js และ npm; [การวิเคราะห์แอปพลิเคชัน](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -147,7 +147,6 @@ REA; Node.js 22.x (>=22.19), 24.x (>=24.11) veya 26+ ve npm gerektirir. Ek araç
|
||||
| Çevrimdışı ELF yapısı | Bölümler, segmentler, özgün semboller/yeniden konumlandırmalar ve statik analizden olası güvenlik önlemleri | Linux x64 üzerinde çağıranın sağladığı pwntools; [ikili dosya tanılama](../binary-diagnostics.md) |
|
||||
| Go ikili dosyaları | Gömülü compiler sürümü, modüller, replacement'lar ve build ayarları | Yerleşik statik ELF/PE/thin Mach-O incelemesi; Go kurulumu gerekmez |
|
||||
| Rust ikili dosyaları | Gömülü cargo-auditable crate adları, sürümleri, kaynakları ve bağımlılık ilişkileri | Yerleşik statik ELF/PE/thin Mach-O incelemesi; [Rust ikili dosyaları](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| WASM artefaktı | Doğrulama, bölümler, tam import/export WAT biçimleri ve decode edilmiş WAT | Çağıranın sağladığı WABT 1.0.42; `inspect_wasm_artifact` |
|
||||
| EVM bayt kodu | Yönlendirme seçicileri, bayt ofsetleri, çıkarımla belirlenen argümanlar ve durum değiştirilebilirliği | Linux x64 üzerinde yerel ham bayt/onaltılık girdi; [çevrimdışı EVM rehberi](../evm-bytecode.md) |
|
||||
| SQLite veritabanları | Tablolar, sütunlar, index'ler, view'lar, trigger'lar ve seçilen sıradan tablo satırları | Yerel database/WAL snapshot'ı; native SQLite sınırlarıyla Node.js 24.x >=24.15 veya 26+ |
|
||||
|
||||
@@ -145,7 +145,6 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та
|
||||
| Структура ELF без запуску | Секції, сегменти, початкові символи/релокації та можливі захисні механізми за даними статичного аналізу | pwntools, наданий стороною, що викликає інструмент, у Linux x64; [діагностика бінарних файлів](../binary-diagnostics.md) |
|
||||
| Байт-код EVM | Селектори диспетчеризації, зміщення байтів, виведені припущення про аргументи та змінюваність стану | Локальні необроблені байти або шістнадцяткове представлення у Linux x64; [посібник EVM без запуску](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Записані аварійні завершення в Linux | Необроблені записи note, регістри/сигнали кожного записаного потоку й необов'язкові кандидати зіставлення пам'яті | pwntools, наданий стороною, що викликає інструмент, у Linux x64; необов'язкові GDB/pwndbg; [записані аварійні завершення](../recorded-crashes.md) |
|
||||
| Встановлені настільні програми | Можливі середовища виконання, файли доказів, компоненти та наступні кроки | Обмежена перевірка каталогів і заголовків；[Настільні програми](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Модулі, імпорти, карти вихідного коду, маршрути, IPC і зв'язки з нативними доповненнями | Node.js і npm; [аналіз застосунків](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -145,7 +145,6 @@ REA yêu cầu Node.js 22.x (>=22.19), 24.x (>=24.11) hoặc 26+, cùng với np
|
||||
| Bố cục ELF ngoại tuyến | Các section, segment, ký hiệu/thông tin tái định vị gốc và các biện pháp giảm thiểu có thể có từ phân tích tĩnh | pwntools do bên gọi cung cấp trên Linux x64; [chẩn đoán tệp nhị phân](../binary-diagnostics.md) |
|
||||
| Bytecode EVM | Bộ chọn điều phối, độ lệch byte, tham số suy luận và khả năng thay đổi trạng thái | Đầu vào cục bộ chứa byte thô hoặc dạng thập lục phân trên Linux x64; [hướng dẫn EVM ngoại tuyến](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| Sự cố Linux đã ghi lại | Bản ghi note thô, thanh ghi/tín hiệu của mọi luồng được ghi lại và các ứng viên ánh xạ tùy chọn | pwntools do bên gọi cung cấp trên Linux x64; GDB/pwndbg tùy chọn; [sự cố đã ghi lại](../recorded-crashes.md) |
|
||||
| Thư mục cài đặt ứng dụng máy tính | Các runtime có khả năng được dùng, tệp bằng chứng, dấu hiệu thành phần và bước tiếp theo | Kiểm tra thư mục và phần đầu tệp có giới hạn；[Ứng dụng máy tính](../desktop-applications.md) |
|
||||
| JavaScript / Electron | Mô-đun, các khai báo import, source map, tuyến, IPC và quan hệ với tiện ích mở rộng mã máy | Node.js và npm; [phân tích ứng dụng](../javascript-application-workflows.md) |
|
||||
|
||||
@@ -149,7 +149,6 @@ REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。
|
||||
| WASM 工件 | 驗證、節、精確的匯入/匯出 WAT 形式及解碼後的 WAT | 呼叫方提供的 WABT 1.0.42；`inspect_wasm_artifact` |
|
||||
| EVM 位元組碼 | 分派選擇器、位元組偏移、推斷的參數和狀態可變性 | Linux x64 上的本地原始位元組/十六進位輸入載體；[離線 EVM 指南](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| 已記錄的 Linux 崩潰 | 原始 note 記錄、每個已記錄執行緒的暫存器/訊號及可選的對映候選項 | Linux x64 上由呼叫方提供的 pwntools；可選的 GDB/pwndbg；[已記錄崩潰](../recorded-crashes.md) |
|
||||
| 桌面安裝目錄 | 執行階段候選、支持檔案、元件標記與後續操作 | 有界的目錄與檔頭檢查；[桌面應用](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 模組、匯入、source map、路由、IPC 和原生附加元件關係 | Node.js 和 npm；[應用分析](../javascript-application-workflows.md) |
|
||||
|
||||
+26
-27
@@ -141,33 +141,32 @@ npx rea-agents@latest setup
|
||||
|
||||
REA 需要 Node.js 22.x（>=22.19）、24.x（>=24.11）或 26+，以及 npm。额外工具和主机支持取决于分析目标：
|
||||
|
||||
| 目标 | REA 返回的内容 | 要求与指南 |
|
||||
| --------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| 原生二进制文件 | 伪代码、汇编、字符串、符号、调用和引用 | Hopper、Ghidra 或 IDA；[原生分析](../native-investigation.md) |
|
||||
| 离线 ELF 布局 | 节、段、原始符号/重定位信息及静态防护机制候选项 | Linux x64 上由调用方提供的 pwntools；[二进制诊断](../binary-diagnostics.md) |
|
||||
| Go 二进制文件 | 内嵌编译器版本、模块、替换项和构建设置 | 内置静态 ELF/PE/thin Mach-O 检查；无需安装 Go |
|
||||
| WASM 工件 | 验证、节、精确的导入/导出 WAT 形式及解码后的 WAT | 调用方提供的 WABT 1.0.42；`inspect_wasm_artifact` |
|
||||
| EVM 字节码 | 分派选择器、字节偏移、推断的参数和状态可变性 | Linux x64 上的本地原始字节/十六进制输入载体；[离线 EVM 指南](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| ELF notes | PT_NOTE/SHT_NOTE owners, types, GNU build-id and ABI-tag fields | Built-in static ELF inspection; [ELF notes](../elf-notes.md) |
|
||||
| 已记录的 Linux 崩溃 | 原始 note 记录、每个已记录线程的寄存器/信号及可选的映射候选项 | Linux x64 上由调用方提供的 pwntools；可选的 GDB/pwndbg；[已记录崩溃](../recorded-crashes.md) |
|
||||
| 桌面安装目录 | 运行时候选、支持文件、组件标记和后续操作 | 有界的目录与文件头检查；[桌面应用](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm；[应用分析](../javascript-application-workflows.md) |
|
||||
| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器；[浏览器分析](../browser-observation.md) |
|
||||
| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR；原生 mitmproxy 捕获需要 Linux 上的 mitmdump；[捕获指南](../web-network-captures.md) |
|
||||
| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos；[pcap 指南](../pcap-analysis.md) |
|
||||
| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查；[托管代码指南](../managed-code-analysis.md) |
|
||||
| PyInstaller | CArchive, PYZ, Python bytecode | Static inspection; [Python packaged applications](../python-packaged-applications.md) |
|
||||
| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK；[Android 指南](../android-analysis.md) |
|
||||
| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb（模拟器或实体设备）；[ADB 指南](../adb-device-analysis.md) |
|
||||
| JEB 项目 | 引擎支持的项目单元、工件摘要和类型/方法的伪代码 | 由调用方启动、提供 MCP 的 JEB 客户端，通过 REA_JEB_MCP_URL 选择；[JEB 指南](../jeb-analysis.md) |
|
||||
| Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器；[资源指南](../apktool-resource-analysis.md) |
|
||||
| Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无（静态 APK 解析）；[Flutter 指南](../flutter-analysis.md) |
|
||||
| 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob；[固件指南](../firmware-analysis.md) |
|
||||
| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](../packet-captures.md) |
|
||||
| 录制的逻辑信号 | 解码注释、样本范围和诊断信息 | Linux: sigrok-cli 0.7.2 + prlimit; [信号记录指南](../signal-recordings.md) |
|
||||
| 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 应用](../apple-application-analysis.md) |
|
||||
| 进程行为 | 终端输出、交互、退出和文件系统观察结果，以及运行对比 | 支持原生 PTY 的 Linux/macOS；[进程捕获](../process-capture.md) |
|
||||
| 目标 | REA 返回的内容 | 要求与指南 |
|
||||
| --------------------- | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| 原生二进制文件 | 伪代码、汇编、字符串、符号、调用和引用 | Hopper、Ghidra 或 IDA；[原生分析](../native-investigation.md) |
|
||||
| 离线 ELF 布局 | 节、段、原始符号/重定位信息及静态防护机制候选项 | Linux x64 上由调用方提供的 pwntools；[二进制诊断](../binary-diagnostics.md) |
|
||||
| Go 二进制文件 | 内嵌编译器版本、模块、替换项和构建设置 | 内置静态 ELF/PE/thin Mach-O 检查；无需安装 Go |
|
||||
| WASM 工件 | 验证、节、精确的导入/导出 WAT 形式及解码后的 WAT | 调用方提供的 WABT 1.0.42；`inspect_wasm_artifact` |
|
||||
| EVM 字节码 | 分派选择器、字节偏移、推断的参数和状态可变性 | Linux x64 上的本地原始字节/十六进制输入载体；[离线 EVM 指南](../evm-bytecode.md) |
|
||||
| Rust binaries | Embedded cargo-auditable dependency list | [Rust binaries](../rust-binaries.md) |
|
||||
| 已记录的 Linux 崩溃 | 原始 note 记录、每个已记录线程的寄存器/信号及可选的映射候选项 | Linux x64 上由调用方提供的 pwntools；可选的 GDB/pwndbg；[已记录崩溃](../recorded-crashes.md) |
|
||||
| 桌面安装目录 | 运行时候选、支持文件、组件标记和后续操作 | 有界的目录与文件头检查；[桌面应用](../desktop-applications.md) |
|
||||
| JavaScript / Electron | 模块、导入、source map、路由、IPC 和原生扩展关系 | Node.js 和 npm；[应用分析](../javascript-application-workflows.md) |
|
||||
| 网站 | 页面结构、脚本、网络观察结果和按请求获取的截图 | Chrome 系浏览器；[浏览器分析](../browser-observation.md) |
|
||||
| 已保存的网络捕获 | 请求、响应、可访问的载荷和来源位置 | HAR；原生 mitmproxy 捕获需要 Linux 上的 mitmdump；[捕获指南](../web-network-captures.md) |
|
||||
| 数据包捕获 | 端点、DNS、TLS SNI、协议分布和 HTTP 请求 | 调用方提供的 tshark 和 capinfos；[pcap 指南](../pcap-analysis.md) |
|
||||
| .NET 程序集 | 元数据、CIL 指令、声明的原生依赖和构建对比 | 静态检查；[托管代码指南](../managed-code-analysis.md) |
|
||||
| PyInstaller | CArchive, PYZ, Python bytecode | Static inspection; [Python packaged applications](../python-packaged-applications.md) |
|
||||
| Android APK | 清单声明、类、反编译的方法和引用 | Linux/macOS/Windows x64 上的无界面 JADX 和完整 JDK；[Android 指南](../android-analysis.md) |
|
||||
| Android 设备 | 设备、软件包、进程、日志、转储、屏幕截图以及 APK/文件传输 | 调用方提供的 adb（模拟器或实体设备）；[ADB 指南](../adb-device-analysis.md) |
|
||||
| JEB 项目 | 引擎支持的项目单元、工件摘要和类型/方法的伪代码 | 由调用方启动、提供 MCP 的 JEB 客户端，通过 REA_JEB_MCP_URL 选择；[JEB 指南](../jeb-analysis.md) |
|
||||
| Android 资源 | 解码后的清单、字符串表、语言区域和版本信息 | 调用方提供的 Apktool 启动器；[资源指南](../apktool-resource-analysis.md) |
|
||||
| Flutter 应用 | Dart 快照哈希、按 ABI 的库摘要及引擎构建溯源 | 无（静态 APK 解析）；[Flutter 指南](../flutter-analysis.md) |
|
||||
| 固件 | 区域、提取结果和转交原生分析的内容 | Linux 上的 Binwalk / Unblob；[固件指南](../firmware-analysis.md) |
|
||||
| 抓包文件 | 指定字段、重复值、过滤条件与解码规则 | Linux: TShark 4.x + prlimit; [抓包分析指南](../packet-captures.md) |
|
||||
| 录制的逻辑信号 | 解码注释、样本范围和诊断信息 | Linux: sigrok-cli 0.7.2 + prlimit; [信号记录指南](../signal-recordings.md) |
|
||||
| 软件包与资源 | 文件清单、摘要、plist、Apple bundle 结构和提取的资源 | [制品与 JavaScript 指南](../javascript-artifact-reconstruction.md)、[Apple 应用](../apple-application-analysis.md) |
|
||||
| 进程行为 | 终端输出、交互、退出和文件系统观察结果，以及运行对比 | 支持原生 PTY 的 Linux/macOS；[进程捕获](../process-capture.md) |
|
||||
|
||||
静态 JavaScript 和 .NET 检查读取提供的文件，不运行应用。运行时捕获会以你的用户权限运行目标或与之交互；各运行时指南说明了具体影响。
|
||||
|
||||
|
||||
+3
-12
@@ -86,7 +86,7 @@ The inline Evidence result contains the artifact path, SHA-256, byte length,
|
||||
container format, architecture, word size, and byte order. When a `.dep-v0`
|
||||
section is present, `auditable_metadata` contains:
|
||||
|
||||
- `section_name`, `.dep-v0` for ELF and PE or `__DATA,.dep-v0` for Mach-O;
|
||||
- `section_name`, `.dep-v0` for ELF and PE or `__DATA,.dep-v0` for thin Mach-O;
|
||||
- `file_offset` and `section_bytes`, the exact file range of the compressed section;
|
||||
- `compressed_bytes` and `uncompressed_bytes`;
|
||||
- `format_version`, the embedded integer, or `null` when the field is omitted;
|
||||
@@ -124,17 +124,8 @@ verified. An omitted revision remains `null` and follows revision 0 semantics.
|
||||
Embedded names and versions are file content. They do not authenticate the
|
||||
packages or the compiler.
|
||||
|
||||
ELF, PE, thin Mach-O and one selected slice of a universal Mach-O are the
|
||||
supported containers.
|
||||
A universal (FAT) Mach-O needs `architecture` (`x86`, `x86_64`, `arm`
|
||||
or `arm64`; CLI `--architecture`). REA reads its own FAT table, without `lipo`,
|
||||
and validates the selected slice's range, alignment and Mach-O header CPU
|
||||
against the table. The result keeps the universal file as `artifact` (its
|
||||
SHA-256 and size), adds `selected_slice` with the slice offset, length, raw CPU
|
||||
type/subtype and the FAT table digest, and reports the section and rustc-path offsets as a
|
||||
file offset inside that slice. Without `architecture`, or when the file has no
|
||||
such slice or several, the request fails on `architecture` and lists the slices
|
||||
present. `architecture` on any other image is rejected rather than ignored.
|
||||
Universal Mach-O containers are unsupported. Select a thin slice first. ELF,
|
||||
PE, and thin Mach-O are the supported containers.
|
||||
|
||||
Package and dependency counts are checked against the minimum required JSON
|
||||
representation before graph allocation or schema cloning. Oversized graphs
|
||||
|
||||
@@ -1,45 +0,0 @@
|
||||
# Implementation plan
|
||||
|
||||
The branch builds on the state-serving and coverage branches (merged into it).
|
||||
|
||||
1. **Write plan types and request bodies.** Add the plan, rule and reason types
|
||||
beside the state rules, and have the state accumulator lend extraction
|
||||
parsed request bodies under the same budget, admitted like response bodies.
|
||||
2. **Field rules.** Add a module that explains an observed entity against the
|
||||
call that wrote it (echo, copy, path, slug, clock, sequence, unique,
|
||||
template and constant) and that recognises and formats clock values,
|
||||
UUIDs and tokens.
|
||||
3. **Effects.** Add one apply module that both the gate and the server use. It
|
||||
applies a create, update, delete or document write to a writable state
|
||||
interface from a plan, a call and a value source: the recorded values in
|
||||
the gate, the store's generators when served.
|
||||
4. **Gate.** Add a simulation that:
|
||||
- classifies every write tool;
|
||||
- builds each plan from its exemplar;
|
||||
- walks the timeline over a copy of the initial state;
|
||||
- checks state-answered reads;
|
||||
- attributes passes and mismatches;
|
||||
- learns the insert rule;
|
||||
- records verdicts.
|
||||
|
||||
Extraction runs it after the read gate and returns `writes` and
|
||||
`write_clock`; the view schema and its unknowns state the write model.
|
||||
|
||||
5. **Serving.**
|
||||
- The loaded state carries the plans and the clock.
|
||||
- The store gains atomic mutations, the clock and the counter (all reset by
|
||||
`data/reset` and `data/add`).
|
||||
- A write answer module validates the body, applies the effect and renders
|
||||
the response.
|
||||
- Item reads of missing entities answer recorded 404s.
|
||||
- The served catalog adds served write tools, with descriptions charged to
|
||||
the discovery budget.
|
||||
6. **Export.** Count served write tools in the bundle plan and README.
|
||||
7. **Tests.** Domain, services, adapters and MCP-boundary coverage as the
|
||||
spec's acceptance lists.
|
||||
8. **Documentation.** Document the write model in the environment-state and
|
||||
serving sections, and record the real-capture results, the AgentEnv run
|
||||
and the live comparison in `verification.md`.
|
||||
|
||||
There are no dependency, provider or REA tool contract changes beyond the
|
||||
`environment-state` result members, and no new network activity.
|
||||
@@ -1,427 +0,0 @@
|
||||
# Apply write effects to the served state
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
An environment served from its state also serves the write tools whose effects
|
||||
the capture itself confirms:
|
||||
|
||||
- a create adds an entity;
|
||||
- an update changes the fields the call sends;
|
||||
- a delete removes the entity;
|
||||
- a document write changes a document.
|
||||
|
||||
Later reads, list membership and `data/get` show the change, and `data/reset`
|
||||
undoes it.
|
||||
|
||||
The `environment-state` view decides which write tools are served. It replays
|
||||
the capture's own timeline on the extracted initial state: each recorded write
|
||||
applies its modelled effect, and each later read that the state answers is
|
||||
rendered and compared with the recorded response. A write tool is served only
|
||||
when at least one later read reproduces after its effect and none contradicts
|
||||
it. Every other write tool stays unserved, listed with the reason.
|
||||
|
||||
Values the application generates are deterministic: identities, sequences and
|
||||
clock values follow the formats the capture recorded, so the same calls after
|
||||
a reset give the same state.
|
||||
|
||||
## Write plans
|
||||
|
||||
The view adds two members to its result:
|
||||
|
||||
- `writes`: one plan per manifest tool whose method may change state
|
||||
(`changesState`);
|
||||
- `write_clock`: the instant the served clock starts from.
|
||||
|
||||
**Effects.** Each plan classifies its tool:
|
||||
|
||||
| Effect | Tool | Target |
|
||||
| ---------- | --------------------------------------------------------------------- | ------------------------- |
|
||||
| `create` | `POST` to a state type's `collection_template` | an entity of that type |
|
||||
| `update` | `PUT`, `PATCH` or `POST` to that template plus one trailing parameter | the entity the path names |
|
||||
| `delete` | `DELETE` to that same item template | the entity the path names |
|
||||
| `document` | `PUT`, `PATCH` or `POST` to a document view's path | that document |
|
||||
| `none` | anything else | reason `no-state-target` |
|
||||
|
||||
**Exemplar.** A plan is built from its first successful (2xx) recorded call,
|
||||
the exemplar.
|
||||
|
||||
- **Request body.** The exemplar's request body must be a JSON object that
|
||||
could enter the state: a JSON-like media type, complete UTF-8 text within
|
||||
256 KiB, exact numbers, no `__proto__` member. Otherwise the reason is
|
||||
`body-unusable`. A `DELETE` takes no body.
|
||||
- **The written entity.** The entity is observed in the exemplar's response,
|
||||
located by its identity value at the root, in one member, or in a
|
||||
one-element array, as item views locate it. When the response does not carry
|
||||
it, the entity is observed in the first later read that shows it: an item
|
||||
read of that entity, or for a create, the one entity a collection read newly
|
||||
lists. A create whose response names its id alone (a scalar, the identity
|
||||
field, or one member) is observed only through a read of that id. With
|
||||
neither, the reason is `no-observation`.
|
||||
|
||||
**Field rules for creates.** Each field of the observed entity gets the first
|
||||
rule that explains its value. Rules that read the call come first, so an
|
||||
identity the client sends is an echo:
|
||||
|
||||
1. **Echo.** A body member of the same name with an equal value.
|
||||
2. **Copy.** A body string of 8 or more characters, found at a JSON pointer of
|
||||
the body (the shallowest match).
|
||||
3. **Path.** The value of a path parameter of the call, percent-decoded.
|
||||
Serving decodes it from the collection the call names, whichever spelling
|
||||
the call uses, and the gate fills a recorded call's path fields from that
|
||||
collection too.
|
||||
4. **Slug.** The slug of a body string member: lower case, runs of other
|
||||
characters joined by `-`.
|
||||
5. **Identity.** The type's `id_field` is generated. A sequence when it is an
|
||||
integer (or a string of digits); otherwise a UUID when it has the
|
||||
`8-4-4-4-12` hex shape; otherwise a token of the same length and character
|
||||
classes.
|
||||
6. **Clock.** A timestamp within 10 minutes of the exemplar's start time. The
|
||||
format is kept: ISO-8601 with its separator, fraction digits and zone, or
|
||||
epoch seconds or milliseconds.
|
||||
7. **Sequence.** An integer one greater than the largest value of the field in
|
||||
the collection, else in the type.
|
||||
8. **Unique.** A UUID-shaped string that no other entity of the type shares.
|
||||
9. **Template.** A string containing a generated value as a delimited token
|
||||
(for example a URL that ends in the new id). It is rebuilt around the new
|
||||
value.
|
||||
10. **Constant.** Otherwise the exemplar's value.
|
||||
|
||||
When a constant, or the text a template keeps, contains a path value of the
|
||||
exemplar's call as a delimited token, the create is served only for calls
|
||||
naming the exemplar's collection.
|
||||
|
||||
**Body members of a create.** A member is either accepted or not:
|
||||
|
||||
- **Accepted.** An echoed member, or the identity field's value. Every member
|
||||
the exemplar sent is required.
|
||||
- **Not accepted.** A member the application stored with a different value
|
||||
(transformed), or one it did not show. A call that sends such a member is a
|
||||
tool error naming the accepted members.
|
||||
|
||||
**Updates.** Body members follow the same accepted and not-accepted rules,
|
||||
except that a member the application stored differently is accepted only with
|
||||
the value a recorded call sent it with, and then stores the value that call
|
||||
recorded (its `constant` rule carries it). The type's identity field never
|
||||
takes a rule: sent as the path's id it is the identity member, sent as another
|
||||
value the application ignored it is accepted with that value and stores
|
||||
nothing, and a stored id that changed is `unexplained-change`. A plan whose
|
||||
calls use a member both as the identity and as a fixed member, or fix it at
|
||||
different values, is `unexplained-change`, as serving could not take every
|
||||
recorded call. So is an update a later read shows dropped a field the
|
||||
entity held, since serving keeps every field an update does not write; a write's
|
||||
own response may leave fields out. A member a later call is the first to store,
|
||||
which the plan's observation does not show, keeps the JSON type that call stored
|
||||
it as (the body's `kinds`), as an exemplar's member keeps the observation's.
|
||||
`PUT` requires every member the exemplar sent; `PATCH` and `POST` require at
|
||||
least one. A field the exemplar's update changed without sending it is
|
||||
explained as one of:
|
||||
|
||||
- clock;
|
||||
- an increment by one;
|
||||
- a slug of an echoed member.
|
||||
|
||||
A field the entity lacked before is a change too when the view that observed
|
||||
the update had shown the entity, so a field the application fills in on update
|
||||
follows its rule. Any other change is `unexplained-change`. An increment or a slug applies to
|
||||
every call that could take it, so the plan is also `unexplained-change` when
|
||||
such a rule would give a call of the tool another value than the one it
|
||||
recorded, or when a later call treats a member otherwise than the plan: one
|
||||
call echoes it and another stores something else, or the two rule it
|
||||
differently.
|
||||
|
||||
**Deletes.** A delete removes the entity. Its response is the exemplar's
|
||||
response, with the entity re-rendered when the response carries it.
|
||||
|
||||
**Documents.** A document write merges the body's members (replaces them, for
|
||||
`PUT`) into the document object that holds most of them: the root or one
|
||||
member. Later calls write at the exemplar's object. Document reads answer only on their recorded line, so a write is
|
||||
served only at the path the state holds the document at, and the gate treats a
|
||||
recorded write at another path as an unmodelled write. A recorded write whose
|
||||
body serving would refuse (a member of another JSON type than the document
|
||||
holds) is `unexplained-change`.
|
||||
|
||||
**Responses.** A write answers with the status and content type of one
|
||||
recorded response:
|
||||
|
||||
| Response | Body |
|
||||
| ---------- | --------------------------------------------------------------------------------------------------------- |
|
||||
| `entity` | the written entity, rendered into the recorded envelope at the plan's pointer through the recorded fields |
|
||||
| `identity` | the recorded body with the new identity at the plan's pointer |
|
||||
| `empty` | no body |
|
||||
| `recorded` | the recorded body |
|
||||
|
||||
A rendered body missing a field the recorded response carried is a tool error,
|
||||
and the state is left unchanged.
|
||||
|
||||
**List position.** A create's `insert` rule places the new entity in its
|
||||
collection's order: sorted by the identity field or a clock field (ascending
|
||||
or descending), `first` or `last`. The rule is the first candidate consistent
|
||||
with both the initial `all`-membership recordings and the position a later
|
||||
recorded list shows. With no evidence, the rule is `last`.
|
||||
|
||||
## The gate
|
||||
|
||||
The view replays every primary-origin record in timeline order on a copy of
|
||||
the initial state.
|
||||
|
||||
**Writes.** Only a 2xx write is modelled. A modelled write applies its effect, using the values it recorded
|
||||
for generated fields and the stored values for fields it transformed. A
|
||||
modelled write whose call the plan does not explain brings its entity to the
|
||||
recorded outcome instead, without attribution. So does an update of an unknown
|
||||
entity (below): its call cannot be checked, so it adds nothing to a built plan.
|
||||
A later create whose generated
|
||||
values take another form than serving gives them (a sequence value out of
|
||||
order, a clock in another format, a non-UUID unique value) makes the plan
|
||||
`unexplained-change`; integer identities are left to the id mapping. An update or
|
||||
delete names its entity as an item read does (below). A later update that does
|
||||
not answer with its entity is observed only by a read whose view shows every
|
||||
clock the plan learned, since serving stamps them; without one it is unmodelled.
|
||||
|
||||
Any other write that may have changed state, including a redirected or
|
||||
unanswered one, is unmodelled, and what it may have changed
|
||||
depends on where its path lies relative to each type's collection paths
|
||||
(segment-prefix related, parameters matching any segment):
|
||||
|
||||
- **Below an item path:** the named entity is unknown until a read shows it
|
||||
again. When the path names no held entity, every entity of that collection
|
||||
is unknown.
|
||||
- **At a collection path:** that collection is not checked again.
|
||||
- **Above collection paths:** everything under the path is not checked again.
|
||||
|
||||
The path names its entity as an update's does and its collection as a create's
|
||||
does, so a plain spelling reaches a collection held under an encoded one; a
|
||||
path above collections covers each collection whose segments are its own
|
||||
verbatim or once percent-decoded.
|
||||
|
||||
An item read of an unknown entity shows it again: the copy takes its recorded
|
||||
values. A failed read that is no absence shows nothing, and the entity stays
|
||||
unknown. Writes that changed it since its last check are checked only on the
|
||||
members they sent, less those the view treats as volatile, which show nothing
|
||||
of a write.
|
||||
|
||||
**Reads.** A read is checked when the served environment would answer it
|
||||
from the state:
|
||||
|
||||
- an item read of a state-backed item view, of the entity serving would name:
|
||||
in a held collection the read's path matches, each segment verbatim or once
|
||||
percent-decoded and verbatim spellings first, the id percent-decoded once,
|
||||
then as written;
|
||||
- a list or document read on one of its view's checked request lines,
|
||||
chosen as replay chooses the line for the read's call: each path parameter
|
||||
the read's value verbatim or once percent-decoded, preferring verbatim
|
||||
spellings, then recorded order, and the query values decoded, in any order.
|
||||
A read with no query on a whole list falls back, as serving does, on a
|
||||
checked line that only pages it. A line whose query repeats a name with
|
||||
different values is never chosen, since no call's arguments can name it; one
|
||||
repeating a name with one value is, as a call naming it matches both. A list
|
||||
is compared as served from the collection of that line. A read on a page
|
||||
(`limit`, `size`, `per_page` and the like, or a `_start`–`_end` range; a size
|
||||
of zero is no bound on a line whose recording listed items under it, while its
|
||||
other sizes and range still bound the page) of a
|
||||
whole list checks nothing once the collection holds more entities than the
|
||||
page, which serving then answers from its recording.
|
||||
- A read made while a modelled write to its entity or collection was still
|
||||
running, starting before that write ended, checks nothing: it may show the
|
||||
state before or after the write. Nor does such a read show an unknown entity
|
||||
again.
|
||||
- A read a write's effect was learned from checks nothing of that write, but
|
||||
still contradicts the other writes it exercises.
|
||||
|
||||
**Comparison.** The read's answer is the full response serving would give from
|
||||
the copy: an item read's entity rendered into its item view's base recording,
|
||||
or a list read's members as serving would list them (so a member the copy no
|
||||
longer holds fails the check) rendered into its line's base recording. It is
|
||||
compared with the recording, as the read gate compares, with volatile fields taking the
|
||||
observed values. A recorded 400, 404 or 410, or a 2xx with a JSON `null`
|
||||
body, for an entity the copy does not hold agrees with it. A successful read
|
||||
whose usable body no longer holds the entity or the list fails, as does one of
|
||||
an entity lacking a field every recording of its view carried, which serving
|
||||
refuses to answer with.
|
||||
|
||||
**Attribution.**
|
||||
|
||||
- A check exercises the write tools that changed its entity or collection
|
||||
since the previous check of it, less those whose effect later writes erased.
|
||||
An update's effect is the fields it sent or changed, derived ones included.
|
||||
A later update overwrites the fields it sets (an increment adds to its field
|
||||
instead), a recorded outcome every field, and a create or delete the whole
|
||||
entity. A document write's effect is the members it merged at its
|
||||
object, or everything under the object it replaces, the whole document at the
|
||||
root; an entity's fields match by name only, whatever their names look like. A write with nothing left on an entity no
|
||||
longer counts there, nor in a collection where no entity it changed keeps any
|
||||
of its effect. A mismatch contradicts them. A pass confirms those it shows
|
||||
something of: a create's or delete's membership, or a changed field the
|
||||
read's view shows and does not treat as volatile; a read that shows none of
|
||||
an update's changes confirms nothing of it, and leaves it for a later check
|
||||
of that entity or collection.
|
||||
- A mismatch contradicts each exercised write that no unmodelled write came
|
||||
after its latest call on any of the read's entities or collections where its
|
||||
effect survives, whatever earlier write is still pending. For a failed list read the unmodelled write must
|
||||
also follow the last check and pending writes of each member that differs, so an older change elsewhere
|
||||
does not excuse a newer write the list shows undone. An item read checks the writes an unmodelled
|
||||
write did follow only on the members they sent, and a list with such a write
|
||||
is not checked again: the unmodelled write may explain the rest.
|
||||
- A mismatch on an entity or collection that no write changed contradicts,
|
||||
as `side-effect`, the modelled writes since that previous check whose paths
|
||||
are segment-prefix related to the read's path.
|
||||
- After a failed item check, the copy takes the read's recorded values (or
|
||||
drops the entity on a recorded absence). A list whose check failed is not
|
||||
checked again.
|
||||
- A list that fails only for where one create placed its entity passes when
|
||||
another of that create's insert candidates reproduces it, which then
|
||||
becomes the plan's rule, unless an earlier list check relied on the rule
|
||||
(an item read's confirmation does not fix it).
|
||||
- The read that supplied a create's or update's observation does not check
|
||||
it, but checks the other writes it exercises.
|
||||
|
||||
**Verdicts.** A plan is served when at least one check confirms it and none
|
||||
contradicts it. Otherwise the reason, in this precedence, is:
|
||||
|
||||
1. the first build reason;
|
||||
2. `contradicted`, with `contradicted_by` naming the first contradicting read;
|
||||
3. `side-effect`;
|
||||
4. `unconfirmed`;
|
||||
5. `incomplete-entities`: an update or delete that answers with the entity,
|
||||
when no initial entity holds every field its recorded answer shows (for
|
||||
example a delete that answers with an edit view of the entity the reads
|
||||
never showed), so it could answer only for entities the environment itself
|
||||
created. An update's answer renders the entity it wrote, so a field one of
|
||||
its rules writes counts as held, except an increment's, which adds only to
|
||||
a held value.
|
||||
|
||||
`checked_record_ordinals` lists the confirming reads.
|
||||
|
||||
## Serving
|
||||
|
||||
**Catalog.** In state mode, the catalog adds the write tools whose plan is
|
||||
served. Their descriptions say the tool:
|
||||
|
||||
- applies its effect to the served state;
|
||||
- names its target type;
|
||||
- lists its required and accepted body members, quoted and sampled within the
|
||||
description budget as response fields are.
|
||||
|
||||
The discovery budget charges them. `--replay-only` and the replay fallback are
|
||||
unchanged.
|
||||
|
||||
**Calls.** A served write answers synchronously against the live store and
|
||||
changes it atomically, so every MCP session and `data/get` see one state.
|
||||
|
||||
**Refusals.** These are tool errors that leave the state unchanged:
|
||||
|
||||
- a body that is not an object;
|
||||
- a missing required member;
|
||||
- a member that is not accepted;
|
||||
- a value of another JSON type than the exemplar's (for a member only a later
|
||||
update stored, that call's), for documents as well as entities;
|
||||
- an update or delete of an entity the state does not hold;
|
||||
- a create in another collection when the plan is tied to its own;
|
||||
- a document write at a path other than the one the state holds it at;
|
||||
- a body member or a resulting value `data/add` would refuse (a number not
|
||||
exact as `data/get` writes it, a `__proto__` member, nesting past the
|
||||
limit), increments included, so `data/get` always loads back;
|
||||
- a create, update or document write that would take what `data/get` reports
|
||||
past the 32 MiB `data/add` intake limit, as a `resource_constraint` error;
|
||||
- a rendered response missing a recorded field.
|
||||
|
||||
When the tool, or else an item view of the type, recorded a 400, 404 or 410
|
||||
response, an update or delete of a missing entity answers with the latest such
|
||||
recording instead.
|
||||
|
||||
A create in a collection held under more than one spelling (`acme1` and
|
||||
`%61cme1`) goes to the one the call spells verbatim, as item calls choose their
|
||||
entity; a collection keeps its spelling after its last entity is deleted. A
|
||||
reset or load knows the spellings of the Evidence's state, of the state it
|
||||
installs and of the type's checked list lines (so a collection the capture
|
||||
listed empty keeps its spelling), never those of a state it replaced. When two
|
||||
spellings fit a call equally, the one holding entities wins. `data/add` refuses a document that `data/get` would write back past the
|
||||
32 MiB intake limit once it adds the members a document may omit, such as
|
||||
`types` and the state Evidence id. A derived state `data/get` would write
|
||||
past that limit is not served: the environment serves replay and reports the
|
||||
state `unavailable`.
|
||||
|
||||
**Clock and generators.** The store's clock starts at `write_clock` and
|
||||
advances one second before each served write; clock fields take its value in
|
||||
their recorded format.
|
||||
|
||||
- **Sequences:** one more than the largest current value in their scope, the
|
||||
largest the state started with at its last reset or load, and the largest
|
||||
value they generated, so a deleted entity's value is not reused, as database
|
||||
sequences do not reuse values.
|
||||
- **UUIDs and tokens:** derived from a SHA-256 of the state Evidence id, the
|
||||
tool, the field and a write counter, skipping values the type already holds.
|
||||
|
||||
`data/reset` and `data/add` restart the clock, the counter and the
|
||||
sequences.
|
||||
|
||||
**Missing entities.** An item read of an entity the state does not hold
|
||||
answers with the tool's latest recorded 400, 404 or 410 response, or 2xx JSON
|
||||
`null` response, the absences the gate accepts, when it has one, labelled
|
||||
`state.view: "missing"`. Otherwise it stays a tool error.
|
||||
|
||||
**Lists of a whole collection.** Written entities matter only if lists show
|
||||
them, so the read gate's `membership: "all"` also covers a line that only pages
|
||||
its list (`limit`, `offset`, `page`, `per_page`, `perpage`, `page_size`,
|
||||
`pagesize`, `size`, `start`, `_start`, `_end`) when its recording listed the
|
||||
whole collection. A list call with no query argument answers from such a line
|
||||
of an `all` view of the collection its path arguments name, with
|
||||
`replay.exact_match` false; a collection with no such line answers from its
|
||||
recording. A call that asks for a page
|
||||
still answers from its recording.
|
||||
|
||||
**Provenance.** Write answers carry `state: {source: "state", view: "create" |
|
||||
"update" | "delete", type, collection, id}`, or `{source: "state", view:
|
||||
"document-write", path}`.
|
||||
|
||||
**Data plane and export.**
|
||||
|
||||
- `data/get` reports written entities without record citations; they load
|
||||
back through `data/add` unchanged.
|
||||
- `environment-export` counts served write tools in the bundle's tool count
|
||||
and README. Its check stays a read tool.
|
||||
|
||||
## Not modelled
|
||||
|
||||
These stay unserved, or are labelled in the description:
|
||||
|
||||
- side effects on counters, relations and copies of entities in other
|
||||
collections;
|
||||
- inputs the application transforms;
|
||||
- members the capture never sent;
|
||||
- reordering after an update;
|
||||
- membership of filtered or paged lists, which answer their recorded members;
|
||||
- write paths the manifest left literal;
|
||||
- writes whose request body the capture did not retain;
|
||||
- referential rules on delete: a delete removes the entity alone, so an
|
||||
application that refuses to delete a referenced entity, or deletes its
|
||||
dependents with it, is not modelled;
|
||||
- authorization.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- **Domain tests:** classification, every field rule, request-body admission,
|
||||
observation from a later read, the insert rules, every verdict and reason,
|
||||
attribution and resynchronisation, and taint by an unmodelled write.
|
||||
- **Services tests:**
|
||||
- store mutations and their atomicity;
|
||||
- every refusal;
|
||||
- generated values and their determinism across reset;
|
||||
- `data/get` round trips of written entities;
|
||||
- the recorded 404 for missing entities;
|
||||
- the catalog and descriptions in each mode.
|
||||
- **Boundary test:** a write tool changes the next read over MCP.
|
||||
|
||||
**Real captures.**
|
||||
|
||||
- The view runs over every showcase capture within its budget, and the
|
||||
verification reports, per capture, the write tools served and the reasons
|
||||
for the rest.
|
||||
- One environment, chosen at random among those serving a write tool, is
|
||||
exported, run in the AgentEnv framework, and graded from `data/get` after an
|
||||
agent's create, update and delete.
|
||||
- The served model is compared with that application running live on the same
|
||||
calls.
|
||||
|
||||
No test reaches an external network.
|
||||
@@ -1,22 +0,0 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the write model, the gate and the serving contract.
|
||||
- [x] Admit request bodies and add the plan types.
|
||||
- [x] Explain created and updated entities with field rules.
|
||||
- [x] Apply create, update, delete and document effects in one shared module.
|
||||
- [x] Simulate the capture's timeline and decide each plan's verdict.
|
||||
- [x] Leave entities below an unmodelled write unknown until a read shows them
|
||||
again, and check writes such a write follows only on the members they
|
||||
sent.
|
||||
- [x] Serve gated write tools from the live store with deterministic values.
|
||||
- [x] Answer recorded 400, 404 and 410 responses for missing entities.
|
||||
- [x] Let lines that only page a whole collection back `membership: "all"`,
|
||||
and answer argument-less list calls from them.
|
||||
- [x] Count served write tools in exports and describe them in bundle READMEs.
|
||||
- [x] Add domain, services, adapters and boundary coverage.
|
||||
- [x] Document the write model and serving.
|
||||
- [x] Run the view over every showcase capture and report per-capture verdicts.
|
||||
- [x] Export one randomly chosen environment, run write tasks in the AgentEnv
|
||||
framework and grade them from `data/get`.
|
||||
- [x] Compare the served model with the live application on the same calls.
|
||||
- [x] Run static, formatting, duplication, documentation and test gates.
|
||||
@@ -1,376 +0,0 @@
|
||||
# Verification
|
||||
|
||||
Verified base: cbb7b613 (this branch before this file). Executed on macOS with
|
||||
Node v22.23.1 (the repository pins 24.18.0 in `.nvmrc`, which was not
|
||||
installed); the AgentEnv runs and the live comparison ran on a disposable Linux
|
||||
VM. Every check below ran as written after the last source change.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run build:cached`, `npm run check:fast` (typecheck and lint),
|
||||
`npm run format:check`, `npx knip`, `npm run scan:todos`, `npm run docs:check`
|
||||
and `npm run verify:test-discovery`: passed.
|
||||
- `npm run jscpd`: 3.43% duplicated lines over 3538 files, under the 3.5% gate.
|
||||
- `npx vitest run --project domain --project services`: 7407 passed, 5
|
||||
pre-existing skips.
|
||||
- `npm run test:prepare`, then `npm run test:focused --` the new write MCP
|
||||
boundary, the serve MCP, CLI and process boundaries, the export CLI
|
||||
boundary, the analysis-view CLI and MCP boundaries, the compact input schema
|
||||
boundary and the contract description tests: 10 files, 72 passed.
|
||||
- `npm run mcp-catalog:check`: passed.
|
||||
|
||||
New tests: `environmentStateWriteValues` (94), `environmentStateWriteExplain`
|
||||
(101), `environmentStateWriteApply` (52), `environmentStateWrites` (16, the gate
|
||||
through the view), `environmentStateWriteAnswer` (14, serving) and
|
||||
`environmentServeWritesMcp` (1, the built CLI over MCP). The replay card pinned
|
||||
by the serve MCP boundary changed with the response summaries tool descriptions
|
||||
carry; its digest now matches the combined branch.
|
||||
|
||||
## Real captures
|
||||
|
||||
The `environment-state` view ran over the 50 showcase captures (45 HAR
|
||||
inspections and 5 `capture_browser_scenario` captures), each within its budget in
|
||||
8–110 ms. It served 33 write tools in 16 captures (11 creates, 12 updates, 10
|
||||
deletes). Of the other 150 state-changing tools, 102 name no state type or
|
||||
document (most write to literal single-entity paths the manifest left
|
||||
untemplated, or to collections whose reads are not state-backed), and 48 are
|
||||
modelled but not served:
|
||||
|
||||
- 15 unconfirmed (no later read checked them);
|
||||
- 13 no-observation;
|
||||
- 8 contradicted, for example a deleted Airbyte source and EspoCRM task still
|
||||
readable, Immich and BookStack item reads showing fields the create
|
||||
response lacked, and Sonarr's tag list ordering;
|
||||
- 7 unexplained-change, such as BookStack's description_html and Paperless's
|
||||
derived text colour;
|
||||
- 4 side-effect, for example a Sonarr tag also listed by `/tag/detail`;
|
||||
- 1 incomplete-entities: WordPress answers a delete with the post in its edit
|
||||
context, which the reads never showed for seeded posts.
|
||||
|
||||
| Capture | Served (create / update / delete) | Not served |
|
||||
| ---------------- | --------------------------------- | ----------------------------------------------------------------------------------------------- |
|
||||
| airbyte | 2 / 2 / 0 | create contradicted, delete contradicted, delete unconfirmed |
|
||||
| baserow | none | no-state-target ×3 |
|
||||
| bookstack | 0 / 0 / 1 | create contradicted, create side-effect, update unexplained-change |
|
||||
| directus | none | no-state-target ×2 |
|
||||
| espocrm | none | create unconfirmed, delete contradicted, update unexplained-change |
|
||||
| ghost | none | no-state-target ×2 |
|
||||
| gitea | 1 / 1 / 0 | no-state-target ×2 |
|
||||
| grafana | none | create unconfirmed, no-state-target ×5 |
|
||||
| grist | none | create no-observation, no-state-target ×5 |
|
||||
| grocy | none | create no-observation, delete no-observation, update no-observation, no-state-target ×2 |
|
||||
| healthchecks | none | create no-observation, delete no-observation, update no-observation, no-state-target ×2 |
|
||||
| homebox | none | document no-observation, no-state-target ×3 |
|
||||
| immich | 0 / 0 / 1 | create contradicted, update contradicted |
|
||||
| jellyfin | none | no-state-target ×2 |
|
||||
| keycloak | none | no-state-target ×1 |
|
||||
| komga | none | create unconfirmed ×2, no-state-target ×4 |
|
||||
| linkding | 2 / 2 / 2 | create unconfirmed, delete unconfirmed, update unconfirmed, no-state-target ×2 |
|
||||
| listmonk | 1 / 1 / 1 | — |
|
||||
| lychee | none | no-state-target ×4 |
|
||||
| mailpit | none | document no-observation, no-state-target ×5 |
|
||||
| mattermost | 1 / 1 / 1 | no-state-target ×3 |
|
||||
| mealie | none | create no-observation, no-state-target ×2 |
|
||||
| memos | none | no-state-target ×3 |
|
||||
| metabase | 0 / 0 / 1 | create side-effect, update unexplained-change |
|
||||
| miniflux | 0 / 1 / 0 | no-state-target ×3 |
|
||||
| n8n | none | create unconfirmed, no-state-target ×6 |
|
||||
| navidrome | none | create no-observation, no-state-target ×2 |
|
||||
| netbox | 1 / 1 / 1 | — |
|
||||
| nocodb | none | create no-observation, no-state-target ×4 |
|
||||
| nomad | none | no-state-target ×4 |
|
||||
| paperless | 0 / 0 / 1 | create unconfirmed, update unexplained-change, no-state-target ×3 |
|
||||
| photoprism | 1 / 1 / 0 | delete unconfirmed |
|
||||
| pinry | none | create unconfirmed, delete unconfirmed, update unexplained-change |
|
||||
| pocketbase | none | no-state-target ×4 |
|
||||
| redpanda-console | none | document no-observation, no-state-target ×5 |
|
||||
| sentry | 1 / 0 / 0 | update unexplained-change, no-state-target ×3 |
|
||||
| shlink | none | create unconfirmed, no-state-target ×1 |
|
||||
| sonarr | 0 / 1 / 0 | create contradicted, create side-effect, update contradicted |
|
||||
| tandoor | none | no-state-target ×4 |
|
||||
| umami | 1 / 1 / 1 | — |
|
||||
| vault | none | no-state-target ×7 |
|
||||
| vikunja | none | no-state-target ×4 |
|
||||
| wordpress | none | create side-effect, delete incomplete-entities, document unconfirmed, update unexplained-change |
|
||||
|
||||
An in-process smoke derived each capture's served state and called every served
|
||||
write tool once with a varied body: all 33 applied (status equal to the
|
||||
recording), and every created or updated entity read back from the state;
|
||||
deleted entities answered the recorded 400 or 404 where one was recorded. Eight
|
||||
older published manifests no longer match the tool names the current view
|
||||
derives, so their state is unavailable and they serve replay, as designed;
|
||||
none of them has a served write tool.
|
||||
|
||||
## End to end on a published environment
|
||||
|
||||
Five published environments serve a create, an update and a delete (Linkding,
|
||||
Listmonk, Mattermost, NetBox, Umami); `random.SystemRandom().choice` picked
|
||||
NetBox (NetBoxEnv-REA, NetBox 4.7.3).
|
||||
|
||||
- `rea environment-export` (packed from this branch) wrote the bundle; its
|
||||
smoke task passed in AgentEnv framework 0.9.1334 with 42 served tools (39
|
||||
reads and `post_dcim_sites`, `patch_dcim_sites_by_id`,
|
||||
`delete_dcim_sites_by_id`).
|
||||
- Four write tasks, each graded by an `env_outcome_verifier` that reads the
|
||||
environment's state through `/agentenv` `data/get` (not the agent's reply),
|
||||
with the replay-qa agent over the Hugging Face router, one run per task and
|
||||
model:
|
||||
|
||||
| Task | gpt-oss-120b | GLM-5.3-Flash | Tool calls (gpt-oss) |
|
||||
| ----------------------------------------------- | ------------ | ------------- | ------------------------------------------------------------ |
|
||||
| Create a site in the Copper Valley region | 1.0 | 1.0 | get_dcim_regions, post_dcim_sites |
|
||||
| Update a seeded site's facility and description | 1.0 | 1.0 | get_dcim_sites, patch_dcim_sites_by_id |
|
||||
| Delete one seeded site and only it | 1.0 | 1.0 | get_dcim_sites, delete_dcim_sites_by_id |
|
||||
| Create a site, then set its facility | 1.0 | 1.0 | get_dcim_regions, post_dcim_sites, patch_dcim_sites_by_id ×2 |
|
||||
|
||||
- The bundle's 12 retrieval tasks (one per seeded object) passed 12/12 for both
|
||||
models, as they did on the published replay bundle.
|
||||
|
||||
## The served model against the live application
|
||||
|
||||
The same NetBox 4.7.3 image ran on the VM, seeded by the capture's own seed
|
||||
script (the same ids), and one scenario ran against both the live API and the
|
||||
environment served from the capture. Only `created` and `last_updated` were
|
||||
normalised; every other member had to be equal:
|
||||
|
||||
| Step | Live | Environment | Equal |
|
||||
| ----------------------------------------- | ---- | ----------- | ---------------------------- |
|
||||
| List sites | 200 | 200 | yes |
|
||||
| Create Tide Gauge Annex (id 5) | 201 | 201 | yes |
|
||||
| Read it | 200 | 200 | yes |
|
||||
| List after the create (sorted by name) | 200 | 200 | yes |
|
||||
| Patch its description and facility | 200 | 200 | yes |
|
||||
| Read after the patch | 200 | 200 | yes |
|
||||
| Create Granite Spur (id 6) | 201 | 201 | yes |
|
||||
| Patch a seeded site | 200 | 200 | yes |
|
||||
| Delete Tide Gauge Annex | 204 | 204 | yes |
|
||||
| Read the deleted site | 404 | 404 | yes |
|
||||
| Create Ledger Point (id 7, not reusing 5) | 201 | 201 | yes |
|
||||
| Delete a seeded site with devices | 409 | 204 | no |
|
||||
| Final list | 200 | 200 | no (follows from the delete) |
|
||||
|
||||
11 of 13 steps were equal. The difference is NetBox's referential protection:
|
||||
it refuses to delete a site that devices reference, while the environment's
|
||||
delete removes the entity alone. The capture only deleted a scratch site with
|
||||
no dependents, so the gate could not see the rule; it is listed under what is
|
||||
not modelled.
|
||||
|
||||
## Not covered
|
||||
|
||||
- A run of the hosted `real-agentenv` lane in CI (the AgentEnv runs here used
|
||||
the published framework release on a VM).
|
||||
- Document writes on a real capture: the four modelled ones are unserved.
|
||||
- Collections tied to a create's own collection on a real capture, and UUID
|
||||
or token identities through an AgentEnv run (both are covered by tests).
|
||||
- Agent token usage and cost.
|
||||
|
||||
## After merging the base branches and main
|
||||
|
||||
The branch then merged the serving branch at 5a61853a, the coverage branch at
|
||||
b4b7ebc7 (their review fixes) and main at edbd291a, then main at 17f581ff
|
||||
(the analysis-view tool description string, which main changes often). The gates above ran again
|
||||
and passed: `npm run jscpd` 3.44% over 3579 files; domain and services 7442
|
||||
passed with 5 pre-existing skips; the focused boundary set 72 passed. The
|
||||
write verdicts over the 50 captures were unchanged plan by plan, and the
|
||||
in-process smoke again applied all 33 served write tools. The AgentEnv runs
|
||||
and the live comparison above ran on cbb7b613, before these merges.
|
||||
After the second main merge: `npm run check:fast`, `npm run format:check`,
|
||||
`npm run docs:check` and `npm run mcp-catalog:check` passed; the contracts,
|
||||
analysis-view and environment tests 857 passed with 3 pre-existing skips; the
|
||||
analysis-view and environment MCP and CLI boundaries 53 passed.
|
||||
|
||||
## After the review round
|
||||
|
||||
The branch then merged the serving branch at c00fde41, the coverage branch at
|
||||
f6c756aa and main at 1629266e, and took the review fixes (82556cd5, 138ea4b1,
|
||||
6d21b62e). The coverage branch quotes field names in response summaries, so the
|
||||
replay card pinned by the serve MCP boundary is now 12,818 bytes; removing
|
||||
those quotes gives back the previous 12,618-byte digest exactly. The gates ran
|
||||
again and passed: `npm run check:fast`, `npm run format:check`, `npx knip`,
|
||||
`npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery`,
|
||||
`npm run verify:module-boundaries` and `npm run mcp-catalog:check`;
|
||||
`npm run jscpd` 3.47% over 3601 files; domain and services 7463 passed with 5
|
||||
pre-existing skips; the focused boundary set 75 passed; conformance 279
|
||||
passed. The gate is stricter, but the write verdicts over the 50 captures were
|
||||
unchanged plan by plan, and the in-process smoke again applied all 33 served
|
||||
write tools. The AgentEnv runs and the live comparison above ran on cbb7b613,
|
||||
before these merges and fixes.
|
||||
|
||||
A second review round then made the gate and serving agree in five more places
|
||||
(reads matched to checked lines as serving matches calls, increments and slugs
|
||||
held for every call, insert rules still learned after an item read, the
|
||||
document body check in the gate, and decoded parent path values). Each new test
|
||||
fails on the previous head and passes here. The same gates passed again
|
||||
(domain and services 7468 passed with 5 pre-existing skips, the focused
|
||||
boundary set 75, conformance 279, `jscpd` 3.47%), the write verdicts over the 50
|
||||
captures were again unchanged plan by plan, and the smoke applied all 33 served
|
||||
write tools.
|
||||
|
||||
A third round kept a checked line's path spelling (replay prefers a verbatim
|
||||
one, so two spellings of one parent stay separate lines) and compared a later
|
||||
slug rule with the values earlier calls recorded rather than only their member
|
||||
names; both new tests fail on the previous head. A fourth round chose a read's
|
||||
line as replay does, so a read spelled plainly is checked against the encoded
|
||||
line that answers it, and its list against that line's collection; its test
|
||||
fails on the previous head.
|
||||
|
||||
A review of the fidelity branch built on this one found four more places: a
|
||||
document write at another path is now unmodelled in the gate, a create picks
|
||||
the collection the call spells verbatim, a missing entity is answered with a
|
||||
recorded 2xx `null` the gate accepted as an absence, and `data/add` refuses a
|
||||
document whose `data/get` form would exceed the intake limit. Each new test
|
||||
fails on the previous head. The gates passed again (domain and services 7475
|
||||
with 5 pre-existing skips, boundaries 75, conformance, `jscpd` 3.46%), the
|
||||
verdicts over the 50 captures were unchanged plan by plan, and the smoke
|
||||
applied all 33 served write tools.
|
||||
|
||||
A sixth round, after merging the serving and coverage branches and main
|
||||
335c042d, settled five more findings: a read whose usable body no longer holds
|
||||
the entity or list fails its check, an entity lacking a field its view always
|
||||
carried confirms nothing, a write a later write replaced is not confirmed by
|
||||
reads of the replacement, sequences never reuse a value the state started
|
||||
with, and a collection keeps its spelling after it empties. Each new test fails
|
||||
on the previous head. Over the 50 captures one capture's verdicts change:
|
||||
Linkding's bundle create and patch, confirmed only by a list read after the
|
||||
same bundle was deleted, are now unconfirmed, so 31 write tools are served (10
|
||||
creates, 11 updates, 10 deletes) and the in-process smoke applies all 31. The
|
||||
gates passed again (domain and services, boundaries, conformance and `jscpd`).
|
||||
|
||||
A seventh round settled two more findings. An update keeps its confirmation
|
||||
when a later write overwrites only part of what it changed: a derived change,
|
||||
such as an incremented version, counts as part of the update's effect, and a
|
||||
later increment adds to the field rather than replacing it. A reset or load
|
||||
forgets the collection spellings of the state it replaced. Both new tests fail
|
||||
on the previous head. The verdicts over the 50 captures are unchanged plan by
|
||||
plan, the in-process smoke applies all 31 served write tools, and the gates
|
||||
passed again.
|
||||
|
||||
An eighth round settled three more. A later write erases the fields of an
|
||||
earlier effect it overwrites, so two writes that together overwrite all of an
|
||||
update's changes leave it nothing to be confirmed by. A recorded outcome erases
|
||||
what earlier writes changed on its entity. A load keeps the Evidence's own
|
||||
collection spellings, so a create in a collection the loaded state does not
|
||||
hold keeps its recorded spelling. Each new test fails on the previous head. The
|
||||
verdicts over the 50 captures are unchanged plan by plan, the smoke applies all
|
||||
31 served write tools, and the gates passed again.
|
||||
|
||||
A ninth round made a create prefer a collection the state holds over an equally
|
||||
fitting spelling known only from the Evidence, so a create after a load lands in
|
||||
the loaded collection and reads back. Its test fails on the previous head; the
|
||||
verdicts and the smoke are unchanged and the gates passed again.
|
||||
|
||||
A tenth round settled six findings Greptile raised on the list-lines branch
|
||||
against this branch's code:
|
||||
|
||||
- a read with no query on a whole list is checked on the paging line serving
|
||||
answers it from, and a line whose query repeats a name is never checked;
|
||||
- a read made while a write to its targets was still running checks nothing;
|
||||
- a later call that stores a member otherwise than the plan refuses the plan;
|
||||
- a create keeps the spelling of a collection the capture listed empty;
|
||||
- a wide array no longer overflows the stack while checking path-dependent
|
||||
values;
|
||||
- the guide counts served write tools among the catalog's.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures
|
||||
are unchanged plan by plan, the smoke applies all 31 served write tools, and
|
||||
the gates passed again.
|
||||
|
||||
An eleventh round settled six more findings, five raised on this branch and one on the fidelity branch:
|
||||
|
||||
- a passing read confirms only the writes it shows something of;
|
||||
- an item read names its entity as serving does, percent-decoded spellings included;
|
||||
- a read a write's effect was learned from still contradicts the other writes;
|
||||
- a line repeating a query name with one value stays checkable;
|
||||
- document writes join the effect tracking, so a replacement overwrites an earlier merge;
|
||||
- a call naming a page that the collection has outgrown answers from its recording (`page-exceeded`), and the gate does not check such reads.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A twelfth round settled four findings Greptile raised on the list-lines branch against this branch's code:
|
||||
|
||||
- a create that answers with its id alone is observed only through a read of that id;
|
||||
- a later call that is the first to send the path's id in its body adds the identity member to the plan;
|
||||
- the copy search walks a request body with an index, so a wide body no longer takes quadratic time (500,000 zeros: 63 s
|
||||
before, 74 ms after);
|
||||
- the sent values of fixed members a plan keeps are counted against the view's allowance.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A thirteenth round settled seven findings Greptile raised on this branch, the fidelity branch and the list-lines branch:
|
||||
|
||||
- a passing read that shows nothing of a write leaves it for a later check, instead of clearing it;
|
||||
- a read begun while a write to an unknown entity still ran neither checks nor shows the entity again;
|
||||
- a later create whose sequence fields do not follow the learned sequence refuses the plan;
|
||||
- a document replacement below the root overwrites only what it replaced;
|
||||
- a built document plan writes later calls at its own object, even one sharing no member with the document;
|
||||
- an update or delete finds its entity as an item read does, percent-decoded spellings included;
|
||||
- a `_start`–`_end` range bounds a page as `limit` does, in serving and in the gate.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A fourteenth round settled two findings Greptile raised on the list-queries branch against this branch's code:
|
||||
|
||||
- the type's identity field never becomes a constant: an update whose id member the application ignored no longer
|
||||
writes the exemplar's id into the entity another call names;
|
||||
- a page size of zero bounds nothing, so a whole line recorded with `limit=0` answers from the state.
|
||||
|
||||
Both tests fail on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke applies
|
||||
all 31 served write tools, and the gates passed again.
|
||||
|
||||
A fifteenth round settled nine findings Greptile raised on this branch, the fidelity branch and the list-lines branch,
|
||||
five of them on the previous rounds' own changes:
|
||||
|
||||
- an entity field whose name looks like a pointer is matched by name; only a document's pointers cover the members
|
||||
under them;
|
||||
- a failed read contradicts each exercised write no unmodelled write came after, rather than skipping all of them
|
||||
when the oldest pending write came before one;
|
||||
- an item read, update or delete names its entity as serving does, decoding the id once, so `book%2531` no longer
|
||||
reaches `book1`;
|
||||
- a plan whose calls use a member both as the identity and as a fixed member, or fix it at different values, is
|
||||
refused;
|
||||
- a size of zero bounds nothing only on a line whose recording listed items under it;
|
||||
- an update whose observation drops a field the entity held is refused;
|
||||
- a later create whose clock takes another format, or whose unique value is not a UUID, is refused;
|
||||
- a derived state `data/get` would write past 32 MiB is not served.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A sixteenth round settled four findings Greptile raised on the fifteenth round's changes, here and on the fidelity,
|
||||
list-lines and list-queries branches:
|
||||
|
||||
- a size of zero is no bound only by itself: a range or another size on the same line still bounds the page;
|
||||
- an update is refused for a dropped field only when a later read shows it dropped, since a write's own response may
|
||||
leave fields out;
|
||||
- interference is weighed against a tool's latest call on the read's entity or collection, not its latest call
|
||||
anywhere.
|
||||
|
||||
Each new test fails on the previous head. The verdicts over the 50 captures are unchanged plan by plan, the smoke
|
||||
applies all 31 served write tools, and the gates passed again.
|
||||
|
||||
A seventeenth round settled two more findings Greptile raised on the list-queries branch against this branch's
|
||||
interference rule: an unmodelled write counts against a tool when it followed the tool's call on any target the read
|
||||
covers, so a later call on another entity does not hide it; and a failed item read contradicts the writes no
|
||||
unmodelled write followed, checking only the others on their members. Both tests fail on the previous head. The
|
||||
verdicts over the 50 captures are unchanged plan by plan, the smoke applies all 31 served write tools, and the gates
|
||||
passed again. A follow-up weighs interference only on targets where the tool's effect survives, so a call
|
||||
whose effect a later write overwrote no longer excuses a later mismatch; its test fails on the previous head.
|
||||
|
||||
## Collection-history accounting check
|
||||
|
||||
On 2026-10-22, Node 24.18.0, I compared the source before the write-history
|
||||
change (`cfb42ddde`) with the bounded-history implementation. Each operation
|
||||
inserted an entity at a new collection path with a 128-character repeated
|
||||
suffix and immediately deleted it. Across five fresh Vitest worker processes,
|
||||
the median for 1,000 completed cycles was 14.57 ms wall / 23,867 us CPU before
|
||||
and 9.98 ms / 16,914 us after. The process RSS samples were 381.1–387.9 MiB
|
||||
before and 380.9–387.6 MiB after. This fixture benchmark measures one matched
|
||||
completed trace, not a representative user task or process-family peak memory.
|
||||
The values include Vitest and V8 and are noisy; they do not establish a memory
|
||||
bound. The 1 MiB policy is a deterministic accounting allowance for
|
||||
additional identifier history, set to 1/32 of the separate 32 MiB state-intake
|
||||
ceiling; it is not derived from RSS samples and does not bound total store or
|
||||
process memory.
|
||||
+5
-26
@@ -65,18 +65,6 @@ API dependency, and independent fixture compiler oracles keep their existing
|
||||
TypeScript versions.
|
||||
|
||||
`npm run build:cached` compiles the CLI/MCP runtime and its packaged skill.
|
||||
The build also records `dist/mcp-catalog.json` (compact inputs) and
|
||||
`dist/mcp-catalog-full.json` from the canonical SDK registrations and binds
|
||||
each to the runtime with its own SHA-256 digest. These are
|
||||
runtime artifacts, separate from the test catalog below. Default target-free
|
||||
MCP startup serves its schemas without constructing analysis validators;
|
||||
the first tool or guided-workflow request materializes the canonical runtime
|
||||
on that same SDK server. Explicit compact/full input selections preserve
|
||||
their advertised schemas throughout materialization. Initial-target,
|
||||
HTTP and output-schema-off startup keep the eager path; source runs without
|
||||
recorded artifacts also use eager composition. Compare memory both after discovery and after a completed
|
||||
request; report diagnostic GC separately from normal resident memory.
|
||||
|
||||
Turbo's strict task environment preserves caller-selected `GOMAXPROCS`,
|
||||
`GOMEMLIMIT`, and `UV_THREADPOOL_SIZE` for resource-constrained builds and
|
||||
checks. These execution controls do not change artifact content, so changing
|
||||
@@ -708,16 +696,9 @@ artifacts, dependency installs, package archives, and runtime verification remai
|
||||
host-local. Keep platform-dependent outputs out of this shared cache.
|
||||
|
||||
The Windows native verifier's private-runtime ACL observer reads SID rules once
|
||||
for both trustee validation and counting, using .NET APIs and a fixed JSON
|
||||
projection of booleans and a count. Module autoloading is disabled; the observer
|
||||
does not need PowerShell utility cmdlets or account-name resolution. A separate
|
||||
PowerShell regression checks the shared projection with utility cmdlet traps
|
||||
and strict property access on a language-defined ACL fixture that has no `Access`
|
||||
convenience property. Fixture setup also avoids utility cmdlets. It covers
|
||||
foreign trustees, different owners, an unprotected DACL, and empty/single rules;
|
||||
these substitutes do not establish real Windows ACL behavior. The observer's
|
||||
`private-runtime-acl` stderr markers report completed phases and elapsed time;
|
||||
inspect these on a timeout
|
||||
for both trustee validation and counting. It deliberately rejects the convenience
|
||||
`Access` property, which resolves account names. Its `private-runtime-acl` stderr
|
||||
markers report completed phases and elapsed time; inspect these on a timeout
|
||||
before attributing it to startup, ACL reads, or identity lookup. The original
|
||||
15-second deadline and all ownership, DACL, and readonly assertions remain in
|
||||
force. A passing retry alone does not explain a previous timeout.
|
||||
@@ -1250,9 +1231,7 @@ host behavior or native process-supervision controls.
|
||||
real `.nib` with Xcode `ibtool`, wraps it in a temporary app bundle, and runs
|
||||
the compiled CLI and stdio MCP server with production providers. It compares
|
||||
their decoded results and checks the view hierarchy, outlet, action, evidence coverage, and truncation
|
||||
status. It also recompiles the XIB with toggled control booleans and requires
|
||||
both decodes to match `ibtool --export` for each source. Storyboard compilation
|
||||
additionally requires an installed iOS platform.
|
||||
status. Storyboard compilation additionally requires an installed iOS platform.
|
||||
|
||||
Keep the provider-specific acceptance path independent from optional
|
||||
cross-compilers. Cross-format failures belong to the cross-format lane and must
|
||||
@@ -1827,7 +1806,7 @@ the built CLI, packs the checkout with `npm pack`, exports the bundle with
|
||||
registry package, and runs `agent-env run` on the bundle. It checks that the
|
||||
export reports the planned file set with the inputs copied byte-exact and a
|
||||
bundle served from the derived state, which serves fewer tools than the
|
||||
manifest because write tools the capture did not confirm are not served, that the runner exits `0` with
|
||||
manifest because write tools are not served, that the runner exits `0` with
|
||||
`tasks/smoke.json` reported as passed and the verifier's description naming
|
||||
that served tool count, and that no container exists
|
||||
after the run that did not exist before; the container inventory runs even when
|
||||
|
||||
@@ -116,5 +116,3 @@ Then rerun this installer.",implemented,verified by shell branch inspection
|
||||
90,install.sh:version mismatch,Installed rea reports a different version,installed version ACTUAL does not match EXPECTED.,Low risk; gives exact integrity mismatch,Keep current copy,unchanged,verified by exact installer test
|
||||
90.5,@modelcontextprotocol/server:tool input validation,Caller sends arguments that violate a declared tool schema,"Canonical invalid_request with the SDK diagnostic plus recovery.kind=inspect_input_contract and a bounded instruction to read the advertised schema or call tools/list",Keeps field/type correction while telling the agent how to recover; diagnostics over 4 KiB stay on the existing retained-error path,Normalize ordinary validation failures and retain oversized diagnostics,verified by real in-memory and stdio MCP client/server calls with wrong argument types
|
||||
90.6,incur:unknown CLI command,Caller enters an unknown REA command,"'COMMAND' is not a command for 'rea'. Suggested command: rea --help",Low risk; identifies input and exact discovery action,Keep framework-generated copy,unchanged,verified by real packaged dispatcher invocation
|
||||
91,src/application/binaryDiagnostics/ByteEntropyService.ts,Invalid file or window,"Expected a regular file.; Selected file could not be found.; Selected byte window extends beyond the end of the file.; File size exceeds exact numeric offset representation.",Invalid selection cannot produce complete entropy Evidence,Choose a readable regular file and a window within its byte size,implemented,entropy service and MCP boundary tests
|
||||
92,src/domain/analysisView/byteEntropyView.ts,Non-summary entropy view,Byte entropy Evidence supports the summary view; retrieve parent Evidence for the complete block series.,Unsupported view selection,Use the summary view or retrieve complete parent Evidence,implemented,typed input rejection
|
||||
|
@@ -595,12 +595,12 @@ Every entity cites its `record_ordinals`.
|
||||
|
||||
`views` lists one plan per read tool in tool order:
|
||||
|
||||
| Shape | Plan |
|
||||
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity object at `pointer` (the root, one member, or `/0` or `/<member>/0`), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares |
|
||||
| `list` | The array at `pointer`, its `fields`, `membership` (`all`: every entity of the collection in state order, when each recording listed exactly that, had no query or only paging parameters (`limit`, `offset`, `page`, `per_page`, `perpage`, `page_size`, `pagesize`, `size`, `start`, `_start`, `_end`), no item-total member (`total`, `count`, `totalCount`, `item_count`, `totalcount`, `itemcount`, but not `total_pages` or `per_page`; a number or a string of digits) differing from the number listed, and no `has_more`, `next` or next-page token member set; `recorded`: the entities each recording listed) and `counters` that equal the number listed |
|
||||
| `document` | The latest initial value of a list tool whose responses hold no identified list, when its initial reads share one request line; held in `documents` with that `path`, whether or not the capture writes it, unless its value changed across any recorded write between those reads (`fields-differ`) |
|
||||
| `recorded` | A `reason` the tool replays its recordings instead |
|
||||
| Shape | Plan |
|
||||
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `item` | The entity object at `pointer` (the root, one member, or `/0` or `/<member>/0`), the ordered `fields` the view shows and the `required_fields` every recording had, and `envelope_ordinal`: the recording whose response, with the entity replaced, every item response shares |
|
||||
| `list` | The array at `pointer`, its `fields`, `membership` (`all`: every entity of the collection in state order, when each recording listed exactly that, had no query, no item-total member (`total`, `count`, `totalCount`, `item_count`, `totalcount`, `itemcount`, but not `total_pages` or `per_page`; a number or a string of digits) differing from the number listed, and no `has_more`, `next` or next-page token member set; `recorded`: the entities each recording listed) and `counters` that equal the number listed |
|
||||
| `document` | The latest initial value of a list tool whose responses hold no identified list, when its initial reads share one request line; held in `documents` with that `path`, whether or not the capture writes it, unless its value changed across any recorded write between those reads (`fields-differ`) |
|
||||
| `recorded` | A `reason` the tool replays its recordings instead |
|
||||
|
||||
An item or list plan is kept only when it reproduces the latest initial
|
||||
recording of each request line from the state, comparing JSON values without
|
||||
@@ -618,65 +618,20 @@ when its content type is
|
||||
JSON-like, it is UTF-8 text of at most 256 KiB, every number survives native
|
||||
conversion exactly, and no member is named `__proto__`.
|
||||
|
||||
`writes` lists one plan per tool whose request may change state, in tool
|
||||
order. Its `effect` is `create` for a `POST` to a type's
|
||||
`collection_template`, `update` for a `PUT`, `PATCH` or `POST` to that template
|
||||
plus one trailing parameter, `delete` for a `DELETE` to it, `document` for a
|
||||
`PUT`, `PATCH` or `POST` to a document's path, and otherwise `none` with reason
|
||||
`no-state-target`. A plan is built from the tool's first successful call: its
|
||||
request body must be a usable JSON object (as response bodies are usable), and
|
||||
the written entity must show in that call's response or in a later read before
|
||||
the next write. Each field of a created entity gets the first rule that
|
||||
explains it: generated `identity` (a sequence, UUID or token, by the recorded
|
||||
value's shape), `echo` of a body member, `copy` of a body string of 8 or more
|
||||
characters, a `path` value, a `slug` of a body member, a `clock` value within 10
|
||||
minutes of the request in its recorded format, a `sequence` one past the
|
||||
largest value of its collection or type, a `unique` UUID, or a `template` around
|
||||
a generated value; other fields keep the observed value. An update's members
|
||||
echo when stored as sent; a member stored differently is accepted only with its
|
||||
recorded value (`body.fixed`) and stores the value the call that showed it
|
||||
recorded (the `value` of its `constant` rule), and a field it changed without
|
||||
sending must be a clock, an increment or a slug, or the plan is
|
||||
`unexplained-change`; an increment or slug must hold for every call of the
|
||||
tool. Every member the first call sent is required, except by `PATCH` and
|
||||
`POST` updates.
|
||||
|
||||
The view then replays the timeline on a copy of the initial state. Modelled
|
||||
writes apply their effect; only a `2xx` write is modelled or an example, and
|
||||
any other write (a redirect, an unanswered or unmodelled one) leaves the entity
|
||||
below whose item path it wrote unknown until a read shows it again (a whole
|
||||
collection when the path names no held entity), and taints what it wrote at or
|
||||
above. A read the served environment would answer from the state is compared
|
||||
with its recording as the read plans are, as the whole response it would serve:
|
||||
the entity or list rendered into the envelope of the item view's or the line's
|
||||
base recording, with the members the state still holds. A 400, 404 or 410 (or a
|
||||
`null` body) agrees with an entity the copy does not hold. A match confirms
|
||||
the writes that changed its entity or collection since it was last read; a
|
||||
mismatch contradicts them, or, when an unmodelled write came after them, is
|
||||
checked only on the members they sent; a mismatch no write explains marks the related writes since as
|
||||
`side-effect`. A plan is `served` when a read confirmed it and none
|
||||
contradicted it; otherwise its `reason` is the first of its build reason
|
||||
(`no-call`, `body-unusable`, `no-observation`, `unexplained-change`),
|
||||
`contradicted` (with `contradicted_by`), `side-effect`, `unconfirmed` and `incomplete-entities` (an update or delete that answers with the entity, when no initial entity holds every field its recorded answer shows, counting for an update the fields its rules write, so it could answer only for entities the environment created).
|
||||
`checked_record_ordinals` lists the confirming reads, `insert` places created
|
||||
entities (sorted by a field the collection's recorded order shows, else last),
|
||||
and `write_clock` is the end of the last timed record, to the next second.
|
||||
|
||||
Values in the state are recorded data, so the view refuses the same parents as
|
||||
the manifest, including those inspected with `sensitive_values`, and a
|
||||
`capture_browser_scenario` parent must have selected response bodies. Types and
|
||||
identity are inferences, so the Evidence is always inferred. `unknowns` states
|
||||
that the state establishes no authorization model, no way the application
|
||||
computes responses or derived values, and write effects only as modelled from
|
||||
recorded calls. The state
|
||||
computes responses or derived values, and no effects of writes. The state
|
||||
accumulator charges the manifest's admission and grouping, every record line,
|
||||
each body as temporary bytes before it is decoded and parsed, and every value
|
||||
it keeps, within the same 96 MiB budget. Documents hold whole recorded response
|
||||
values whose recorded text and rendered JSON are each at most 256 KiB, so a
|
||||
capture with many read-only paths yields a correspondingly larger state result.
|
||||
The view performs no network activity and serves nothing; `rea environment-serve`
|
||||
derives it from the capture at startup, serves read tools and the served write
|
||||
plans from it and loads replacement states through the AgentEnv data plane, as
|
||||
capture with many read-only paths yields a correspondingly larger state result. The view performs no network activity
|
||||
and serves nothing; `rea environment-serve` derives it from the capture at
|
||||
startup, serves read tools from it and loads replacement states through the
|
||||
AgentEnv data plane, as
|
||||
[serve from the state](agent-environments.md#serve-from-the-state) describes.
|
||||
|
||||
## Resource and lifecycle boundaries
|
||||
|
||||
@@ -6,9 +6,8 @@ import { join, relative } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
const sha = (value) => createHash("sha256").update(value).digest("hex");
|
||||
|
||||
/** Bind a trial to its source checkout and compiled runtime, including deferred assets. */
|
||||
export const captureRuntimeBinding = async (runtime, kind = "cli") => {
|
||||
assert.ok(kind === "cli" || kind === "mcp", "Unknown runtime entry");
|
||||
/** Bind a trial to its source checkout and the compiled CLI actually launched. */
|
||||
export const captureRuntimeBinding = async (runtime) => {
|
||||
const root = await realpath(runtime);
|
||||
const git = (args) =>
|
||||
execFileSync("git", args, { cwd: root, maxBuffer: 16 * 1024 * 1024 });
|
||||
@@ -51,16 +50,10 @@ export const captureRuntimeBinding = async (runtime, kind = "cli") => {
|
||||
files.push(relative(root, path));
|
||||
}
|
||||
};
|
||||
// MCP materializes additional workers and reads build-owned JSON after
|
||||
// discovery. Bind every compiled JS/JSON asset, rather than only its entry.
|
||||
await visit(join(root, kind === "mcp" ? "dist" : "dist/cli-runtime"));
|
||||
await visit(join(root, "dist/cli-runtime"));
|
||||
assert.ok(
|
||||
files.includes(
|
||||
kind === "mcp"
|
||||
? join("dist", "mcp", "main.js")
|
||||
: join("dist", "cli-runtime", "cli.js"),
|
||||
),
|
||||
"Missing compiled runtime entry",
|
||||
files.includes(join("dist", "cli-runtime", "cli.js")),
|
||||
"Missing compiled CLI entry",
|
||||
);
|
||||
const hash = createHash("sha256");
|
||||
let bytes = 0;
|
||||
@@ -70,23 +63,15 @@ export const captureRuntimeBinding = async (runtime, kind = "cli") => {
|
||||
hash.update(JSON.stringify([name, value.length, sha(value)]) + "\n");
|
||||
}
|
||||
return {
|
||||
binding_version: kind === "mcp" ? 2 : 1,
|
||||
binding_version: 1,
|
||||
root,
|
||||
node: process.version,
|
||||
git_revision: gitRevision,
|
||||
source_diff_sha256: sha(sourceDiff),
|
||||
untracked_source_sha256: sha(JSON.stringify(untrackedEntries)),
|
||||
...(kind === "mcp"
|
||||
? {
|
||||
compiled_runtime_sha256: hash.digest("hex"),
|
||||
compiled_runtime_files: files.length,
|
||||
compiled_runtime_bytes: bytes,
|
||||
}
|
||||
: {
|
||||
compiled_cli_sha256: hash.digest("hex"),
|
||||
compiled_cli_files: files.length,
|
||||
compiled_cli_bytes: bytes,
|
||||
}),
|
||||
compiled_cli_sha256: hash.digest("hex"),
|
||||
compiled_cli_files: files.length,
|
||||
compiled_cli_bytes: bytes,
|
||||
};
|
||||
};
|
||||
|
||||
@@ -97,10 +82,6 @@ export const assertRuntimeBinding = (actual, expected, mode) => {
|
||||
|
||||
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
await captureRuntimeBinding(process.argv[2], process.argv[3]),
|
||||
null,
|
||||
2,
|
||||
),
|
||||
JSON.stringify(await captureRuntimeBinding(process.argv[2]), null, 2),
|
||||
);
|
||||
}
|
||||
@@ -226,8 +226,3 @@ keeps default valid calls and typed refusals while checking full producer
|
||||
constraints separately. Five isolated suites passed 14 tests across call paths,
|
||||
ADB start, browser scenarios, source selection and managed analysis. The first
|
||||
browser attempt failed and remains recorded; no performance cohort changed.
|
||||
|
||||
The subsequent [startup audit](mcp-startup.md) extends this harness with
|
||||
source/bundle bindings and a post-materialization directory check. Its new
|
||||
measurements are a separate cohort; reproduce this report at its recorded
|
||||
revision when comparing the original timing and model-session receipts.
|
||||
@@ -7,11 +7,6 @@ import { parseArgs } from "node:util";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import { Ajv2020 } from "ajv/dist/2020.js";
|
||||
import { sampleRetainedHeap } from "./mcp-retained-heap.mjs";
|
||||
import {
|
||||
captureRuntimeBinding,
|
||||
assertRuntimeBinding,
|
||||
} from "./managed-cli-runtime.mjs";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
@@ -20,135 +15,54 @@ const { values } = parseArgs({
|
||||
oracle: { type: "string" },
|
||||
output: { type: "string" },
|
||||
profile: { type: "string", default: "default" },
|
||||
phase: { type: "string", default: "analysis" },
|
||||
cache: { type: "string", default: "cold" },
|
||||
"cache-root": { type: "string" },
|
||||
bindings: { type: "string" },
|
||||
role: { type: "string" },
|
||||
"diagnostic-heap": { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
assert.ok(["analysis", "discovery"].includes(values.phase));
|
||||
assert.ok(["cold", "warm"].includes(values.cache));
|
||||
assert.ok(!values["diagnostic-heap"] || values.phase === "analysis");
|
||||
for (const key of values.phase === "analysis"
|
||||
? ["runtime", "artifact", "oracle", "output"]
|
||||
: ["runtime", "output"])
|
||||
for (const key of ["runtime", "artifact", "oracle", "output"])
|
||||
assert.ok(
|
||||
values[key] && resolve(values[key]) === values[key],
|
||||
`${key} must be absolute`,
|
||||
);
|
||||
assert.ok(["default", "full", "compact"].includes(values.profile));
|
||||
if (values.cache === "warm")
|
||||
assert.ok(
|
||||
values["cache-root"] &&
|
||||
resolve(values["cache-root"]) === values["cache-root"],
|
||||
);
|
||||
await mkdir(values.output, { recursive: false });
|
||||
const hash = (value) => createHash("sha256").update(value).digest("hex");
|
||||
let oracle;
|
||||
let artifactSha;
|
||||
if (values.phase === "analysis") {
|
||||
oracle = JSON.parse(await readFile(values.oracle, "utf8"));
|
||||
artifactSha = hash(await readFile(values.artifact));
|
||||
assert.equal(artifactSha, oracle.artifact_sha256);
|
||||
}
|
||||
let expectedBinding;
|
||||
let bindingBefore;
|
||||
let auditBeforeMs;
|
||||
let measurementInputsSha256;
|
||||
const producerSources = {};
|
||||
if (values.bindings !== undefined) {
|
||||
assert.ok(["baseline", "candidate"].includes(values.role));
|
||||
const inputBytes = await readFile(values.bindings);
|
||||
measurementInputsSha256 = hash(inputBytes);
|
||||
const inputs = JSON.parse(inputBytes.toString("utf8"));
|
||||
for (const name of [
|
||||
"mcp-input-catalog.mjs",
|
||||
"managed-cli-runtime.mjs",
|
||||
"mcp-retained-heap.mjs",
|
||||
]) {
|
||||
producerSources[name] = hash(
|
||||
await readFile(new URL(name, import.meta.url)),
|
||||
);
|
||||
assert.equal(producerSources[name], inputs.harness_sources_sha256[name]);
|
||||
}
|
||||
expectedBinding = inputs.runtime_bindings[values.role];
|
||||
const start = performance.now();
|
||||
bindingBefore = await captureRuntimeBinding(values.runtime, "mcp");
|
||||
auditBeforeMs = performance.now() - start;
|
||||
assertRuntimeBinding(bindingBefore, expectedBinding, values.role);
|
||||
}
|
||||
const oracle = JSON.parse(await readFile(values.oracle, "utf8"));
|
||||
const artifactSha = hash(await readFile(values.artifact));
|
||||
assert.equal(artifactSha, oracle.artifact_sha256);
|
||||
const environment = {
|
||||
PATH: process.env.PATH,
|
||||
HOME: values.output,
|
||||
REA_LOG_LEVEL: "silent",
|
||||
UV_THREADPOOL_SIZE: "1",
|
||||
NODE_OPTIONS: "--max-old-space-size=2048",
|
||||
...(values.cache === "cold"
|
||||
? { NODE_DISABLE_COMPILE_CACHE: "1" }
|
||||
: { NODE_COMPILE_CACHE: values["cache-root"] }),
|
||||
NODE_DISABLE_COMPILE_CACHE: "1",
|
||||
...(values.profile === "default"
|
||||
? {}
|
||||
: { REA_MCP_INPUT_SCHEMA_PROFILE: values.profile }),
|
||||
};
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [
|
||||
...(values["diagnostic-heap"] ? ["--inspect=127.0.0.1:0"] : []),
|
||||
join(values.runtime, "scripts/rea.mjs"),
|
||||
"mcp",
|
||||
],
|
||||
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
|
||||
cwd: values.output,
|
||||
env: environment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
const protocolRequests = {};
|
||||
const send = transport.send.bind(transport);
|
||||
transport.send = (...args) => {
|
||||
const method = args[0].method;
|
||||
if (typeof method === "string")
|
||||
protocolRequests[method] = (protocolRequests[method] ?? 0) + 1;
|
||||
return send(...args);
|
||||
};
|
||||
const client = new Client({ name: "rea-catalog-task", version: "1" });
|
||||
const report = {
|
||||
profile: values.profile,
|
||||
phase: values.phase,
|
||||
cache: values.cache,
|
||||
cache_root: values["cache-root"] ?? null,
|
||||
role: values.role,
|
||||
measurement_inputs_sha256: measurementInputsSha256,
|
||||
producer_sources_sha256: producerSources,
|
||||
artifact_sha256: artifactSha,
|
||||
calls: [],
|
||||
checks: [],
|
||||
status: "incomplete",
|
||||
cleanup_verified: false,
|
||||
protocol_requests: protocolRequests,
|
||||
};
|
||||
const transcript = [];
|
||||
let stderr = "";
|
||||
let inspectorEndpoint;
|
||||
transport.stderr?.on("data", (value) => {
|
||||
stderr += value;
|
||||
if (values["diagnostic-heap"] && inspectorEndpoint === undefined)
|
||||
inspectorEndpoint = stderr.match(
|
||||
/Debugger listening on (ws:\/\/127\.0\.0\.1:\d+\/[^\s]+)/u,
|
||||
)?.[1];
|
||||
});
|
||||
const began = performance.now();
|
||||
try {
|
||||
await client.connect(transport);
|
||||
report.protocol_era = client.getProtocolEra();
|
||||
await writeFile(
|
||||
join(values.output, "server.json"),
|
||||
JSON.stringify({
|
||||
info: client.getServerVersion(),
|
||||
capabilities: client.getServerCapabilities(),
|
||||
instructions: client.getInstructions(),
|
||||
}),
|
||||
);
|
||||
const catalog = await client.listTools();
|
||||
assert.equal(catalog.nextCursor, undefined);
|
||||
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
|
||||
@@ -159,154 +73,97 @@ try {
|
||||
ajv.validateSchema(tool.inputSchema),
|
||||
`Invalid schema: ${tool.name}`,
|
||||
);
|
||||
{
|
||||
const encoded = JSON.stringify(catalog);
|
||||
await writeFile(join(values.output, "catalog.json"), encoded);
|
||||
report.catalog_sha256 = hash(encoded);
|
||||
}
|
||||
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
|
||||
report.tool_names = [...definitions.keys()];
|
||||
if (values.phase === "discovery") {
|
||||
const prompts = await client.listPrompts();
|
||||
assert.equal(prompts.nextCursor, undefined);
|
||||
await writeFile(
|
||||
join(values.output, "prompts.json"),
|
||||
JSON.stringify(prompts),
|
||||
const call = async (name, args) => {
|
||||
const definition = definitions.get(name);
|
||||
assert.ok(definition, `Missing named tool ${name}`);
|
||||
const start = performance.now();
|
||||
const reply = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ toolDefinition: definition },
|
||||
);
|
||||
report.prompts_sha256 = hash(JSON.stringify(prompts));
|
||||
report.answer = {
|
||||
tools: report.tool_names,
|
||||
prompts: prompts.prompts.map((prompt) => prompt.name),
|
||||
};
|
||||
report.status = "passed";
|
||||
} else {
|
||||
const call = async (name, args) => {
|
||||
const definition = definitions.get(name);
|
||||
assert.ok(definition, `Missing named tool ${name}`);
|
||||
const start = performance.now();
|
||||
const reply = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ toolDefinition: definition },
|
||||
);
|
||||
report.calls.push({
|
||||
name,
|
||||
elapsed_ms: performance.now() - start,
|
||||
is_error: reply.isError === true,
|
||||
});
|
||||
transcript.push({ name, arguments: args, reply });
|
||||
return reply;
|
||||
};
|
||||
const input = {
|
||||
path: values.artifact,
|
||||
report.calls.push({
|
||||
name,
|
||||
elapsed_ms: performance.now() - start,
|
||||
is_error: reply.isError === true,
|
||||
});
|
||||
transcript.push({ name, arguments: args, reply });
|
||||
return reply;
|
||||
};
|
||||
const input = {
|
||||
path: values.artifact,
|
||||
view: {
|
||||
collection: "packages",
|
||||
names: ["serde"],
|
||||
fields: ["name", "version", "kind"],
|
||||
},
|
||||
};
|
||||
const first = await call("inspect_rust_binary", input);
|
||||
assert.notEqual(first.isError, true);
|
||||
const selected = first.structuredContent;
|
||||
assert.ok(selected?.normalized_result?.page);
|
||||
assert.deepEqual(selected.normalized_result.page.rows, oracle.serde_rows);
|
||||
assert.equal(selected.subject.digest.sha256, artifactSha);
|
||||
const parentId = selected.normalized_result.parent_evidence_id;
|
||||
for (let index = 0; index < 2; index++) {
|
||||
const reply = await call("inspect_analysis_view", {
|
||||
source: { kind: "retained-evidence", evidence_id: parentId },
|
||||
view: {
|
||||
kind: "rust",
|
||||
collection: "packages",
|
||||
names: ["serde"],
|
||||
fields: ["name", "version", "kind"],
|
||||
},
|
||||
};
|
||||
const first = await call("inspect_rust_binary", input);
|
||||
assert.notEqual(first.isError, true);
|
||||
const selected = first.structuredContent;
|
||||
assert.ok(selected?.normalized_result?.page);
|
||||
assert.deepEqual(selected.normalized_result.page.rows, oracle.serde_rows);
|
||||
assert.equal(selected.subject.digest.sha256, artifactSha);
|
||||
const parentId = selected.normalized_result.parent_evidence_id;
|
||||
for (let index = 0; index < 2; index++) {
|
||||
const reply = await call("inspect_analysis_view", {
|
||||
source: { kind: "retained-evidence", evidence_id: parentId },
|
||||
view: {
|
||||
kind: "rust",
|
||||
collection: "packages",
|
||||
names: ["serde"],
|
||||
fields: ["name", "version", "kind"],
|
||||
},
|
||||
});
|
||||
assert.notEqual(reply.isError, true);
|
||||
assert.deepEqual(
|
||||
reply.structuredContent.normalized_result.page.rows,
|
||||
oracle.serde_rows,
|
||||
);
|
||||
assert.equal(
|
||||
reply.structuredContent.normalized_result.parent_evidence_id,
|
||||
parentId,
|
||||
);
|
||||
}
|
||||
const malformed = await call("inspect_rust_binary", {
|
||||
path: values.artifact,
|
||||
view: { collection: "packages", offset: -1 },
|
||||
});
|
||||
assert.equal(malformed.isError, true);
|
||||
report.checks.push(
|
||||
"Independent package rows and digest; repeated retained identity; canonical negative-offset refusal",
|
||||
);
|
||||
const exported = await call("export_evidence_bundle", {
|
||||
path: join(values.output, "evidence.json"),
|
||||
});
|
||||
assert.notEqual(exported.isError, true);
|
||||
const bundle = JSON.parse(
|
||||
await readFile(join(values.output, "evidence.json"), "utf8"),
|
||||
);
|
||||
const parent = bundle.records.find(
|
||||
(record) => record.evidence_id === parentId,
|
||||
);
|
||||
assert.ok(parent, "Complete original Evidence is retained");
|
||||
assert.equal(parent.subject.digest.sha256, artifactSha);
|
||||
assert.notEqual(reply.isError, true);
|
||||
assert.deepEqual(
|
||||
parent.normalized_result.auditable_metadata.packages,
|
||||
oracle.packages,
|
||||
reply.structuredContent.normalized_result.page.rows,
|
||||
oracle.serde_rows,
|
||||
);
|
||||
assert.deepEqual(
|
||||
parent.normalized_result.auditable_metadata.dependency_edges,
|
||||
oracle.edges,
|
||||
assert.equal(
|
||||
reply.structuredContent.normalized_result.parent_evidence_id,
|
||||
parentId,
|
||||
);
|
||||
report.parent_evidence_id = parentId;
|
||||
report.parent_semantic_sha256 = hash(JSON.stringify(parent));
|
||||
report.answer = oracle.serde_rows;
|
||||
assert.deepEqual(
|
||||
await client.listTools(),
|
||||
catalog,
|
||||
"Advertisement changed after canonical materialization",
|
||||
);
|
||||
report.checks.push(
|
||||
"Catalog remains identical after canonical materialization",
|
||||
);
|
||||
report.status = "passed";
|
||||
}
|
||||
if (values["diagnostic-heap"]) {
|
||||
report.status = "incomplete";
|
||||
assert.ok(inspectorEndpoint, "Missing explicitly launched inspector");
|
||||
report.diagnostic_heap = await sampleRetainedHeap(
|
||||
inspectorEndpoint,
|
||||
transport.pid,
|
||||
);
|
||||
report.status = "passed";
|
||||
}
|
||||
const malformed = await call("inspect_rust_binary", {
|
||||
path: values.artifact,
|
||||
view: { collection: "packages", offset: -1 },
|
||||
});
|
||||
assert.equal(malformed.isError, true);
|
||||
report.checks.push(
|
||||
"Independent package rows and digest; repeated retained identity; canonical negative-offset refusal",
|
||||
);
|
||||
const exported = await call("export_evidence_bundle", {
|
||||
path: join(values.output, "evidence.json"),
|
||||
});
|
||||
assert.notEqual(exported.isError, true);
|
||||
const bundle = JSON.parse(
|
||||
await readFile(join(values.output, "evidence.json"), "utf8"),
|
||||
);
|
||||
const parent = bundle.records.find(
|
||||
(record) => record.evidence_id === parentId,
|
||||
);
|
||||
assert.ok(parent, "Complete original Evidence is retained");
|
||||
assert.equal(parent.subject.digest.sha256, artifactSha);
|
||||
assert.deepEqual(
|
||||
parent.normalized_result.auditable_metadata.packages,
|
||||
oracle.packages,
|
||||
);
|
||||
assert.deepEqual(
|
||||
parent.normalized_result.auditable_metadata.dependency_edges,
|
||||
oracle.edges,
|
||||
);
|
||||
report.parent_evidence_id = parentId;
|
||||
report.parent_semantic_sha256 = hash(JSON.stringify(parent));
|
||||
report.answer = oracle.serde_rows;
|
||||
report.status = "passed";
|
||||
} finally {
|
||||
await client.close();
|
||||
await transport.close();
|
||||
report.cleanup_verified = transport.pid === null;
|
||||
assert.ok(report.cleanup_verified);
|
||||
report.elapsed_ms = performance.now() - began;
|
||||
if (expectedBinding !== undefined) {
|
||||
const start = performance.now();
|
||||
const after = await captureRuntimeBinding(values.runtime, "mcp");
|
||||
report.runtime_audit_ms = {
|
||||
before: auditBeforeMs,
|
||||
after: performance.now() - start,
|
||||
};
|
||||
assertRuntimeBinding(after, expectedBinding, values.role);
|
||||
assert.deepEqual(after, bindingBefore);
|
||||
assert.equal(
|
||||
hash(await readFile(values.bindings)),
|
||||
measurementInputsSha256,
|
||||
);
|
||||
for (const [name, digest] of Object.entries(producerSources))
|
||||
assert.equal(
|
||||
hash(await readFile(new URL(name, import.meta.url))),
|
||||
digest,
|
||||
);
|
||||
report.runtime_binding = after;
|
||||
report.runtime_binding_verified = true;
|
||||
}
|
||||
await writeFile(
|
||||
join(values.output, "report.json"),
|
||||
JSON.stringify(report, null, 2) + "\n",
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,143 +0,0 @@
|
||||
# Integer validator allocation after MCP materialization
|
||||
|
||||
Issue #6236; PR #4360. Linux x64, 2026-10-22.
|
||||
|
||||
<!-- BEGIN:allocation-report:provenance -->
|
||||
|
||||
Baseline `main@4a4bbab3c337701febeb6b532ace49041c4876e0`; candidate `461f342c71a72ef2be60912391df2393aa4e2ea3`. This is a pinned comparison.
|
||||
|
||||
In separate inspector/forced-GC diagnostics, retained REA V8 heap decreases by about 9.6 MiB. The three-pair ranges do not overlap in either profile. Inspector perturbation is included. Normal task timing, CPU and peak memory are reported separately; these observations do not establish a general latency or peak-memory reduction.
|
||||
|
||||
<!-- END:allocation-report:provenance -->
|
||||
|
||||
Directory discovery uses recorded advertisements after #4424; this study separates it from completed tasks that materialize canonical handlers. Context is unchanged.
|
||||
|
||||
## Change and compatibility
|
||||
|
||||
Replace 1,247 argument-free `z.number().int()` constructions with `z.int()` across 175 existing source files. Keep current shared schema owners, refinements, bounds, defaults and metadata. AST-token comparison against the baseline, after normalizing this construction and formatting, is equal for every touched source file. No new tools, provider effects, caches or response contracts.
|
||||
|
||||
The locked Zod 4.4.3 chained form retains a `ZodNumber` with a separate `ZodNumberFormat` check; direct construction uses the format itself. Incur 0.6.1 coerces argv strings only for `ZodNumber`. Existing CLI declarations and shared-input exceptions therefore retain the chained form. The actual compiled CLI registers 160 leaf commands; 104 numeric argument/option fields have identical results/errors in 974 installed-Incur parser cases across both runtimes, each field with an accepted probe. Fields are isolated from unrelated required options and no handlers are invoked. This is consumer-boundary coverage, not a real-engine claim.
|
||||
|
||||
Seven dependency-boundary variants match parsed values and complete error issues for 126 cases, including fractions, nonfinite values, strings/null/undefined and unsafe integers; 14 JSON Schema projections match in draft-07/draft-2020-12. Actual SDK advertisements match byte-for-byte in both profiles before/after materialization and across roles.
|
||||
|
||||
## Tasks, artifacts and bounds
|
||||
|
||||
Reuse the shared SDK task driver and independent ELF/section/zlib/JSON oracles from [the catalog study](mcp-input-catalog.md): official Alpine v3.21 x86_64 ripgrep 14.1.1-r0 and Zellij 0.40.0-r0. All 42/422 packages and 88/1,063 edges match the independent oracles. Select serde name/version/kind, repeat exact retained-parent reads twice, verify a canonical negative-offset refusal, export/check complete Evidence and confirm the unchanged directory. `kind: null` stays unknown. Targets are not executed; dependency presence does not establish runtime use. Acquisition URLs, archive/executable/oracle hashes and complete answers are in [the receipts](mcp-integer-allocation.json). Previously acquired public bytes are rehashed, not redownloaded.
|
||||
|
||||
Normal cohort: three alternating baseline/candidate pairs for each task × cold/warm server compile cache × default compact/full inputs: 72 trials, plus four excluded full-task warm priming runs. Separate diagnostic cohort: three alternating pairs per profile, cold Zellij tasks, 12 trials. All pass, with identical complete parent IDs/hashes and answers.
|
||||
|
||||
Node 24.18.0/npm 11.16.0; fresh client/server and HOME per trial, driver compile cache disabled. Each trial is serial, pinned to one CPU, 4 GiB hard/3 GiB early stop, no swap, at most 192 tasks. Before/after checks bind clean source, all compiled JS/JSON assets (1,990 baseline / 1,990 candidate files), scripts and manifests; producer and resource-observer hashes are also bound. No OOM, resource stop or leftover payload processes. Whole-family measurements include SDK validation, extra calls, source/compiled audits and oracle checking; kernel peak includes file cache. Build/acquisition/oracle generation and priming are excluded. Warm uses a populated role/profile-specific Node cache; hit rate and OS page cache are unmeasured, and audits warm compiled-file pages.
|
||||
|
||||
## Normal task costs
|
||||
|
||||
Medians, baseline → candidate; all ranges are in the JSON. No inspector or diagnostic GC runs in this cohort.
|
||||
|
||||
<!-- BEGIN:allocation-report:normal -->
|
||||
|
||||
| Task | Cache | Input profile | Wall s | Family CPU s | Family peak MiB |
|
||||
| --------- | ----- | ------------- | --------------: | --------------: | --------------: |
|
||||
| directory | cold | default | 2.739 → 2.239 | 2.553 → 2.075 | 247.4 → 248.4 |
|
||||
| directory | cold | full | 2.940 → 2.437 | 2.682 → 2.217 | 348.0 → 346.8 |
|
||||
| directory | warm | default | 2.639 → 2.135 | 2.461 → 1.980 | 251.8 → 246.2 |
|
||||
| directory | warm | full | 2.840 → 2.338 | 2.666 → 2.144 | 348.6 → 347.2 |
|
||||
| ripgrep | cold | default | 7.571 → 7.169 | 7.372 → 6.918 | 798.8 → 791.1 |
|
||||
| zellij | cold | default | 7.876 → 7.373 | 7.623 → 7.134 | 812.2 → 810.5 |
|
||||
| ripgrep | cold | full | 7.876 → 7.475 | 7.710 → 7.295 | 803.7 → 801.2 |
|
||||
| zellij | cold | full | 8.079 → 7.573 | 7.852 → 7.378 | 815.7 → 819.1 |
|
||||
| ripgrep | warm | default | 7.575 → 6.977 | 7.352 → 6.810 | 815.0 → 791.6 |
|
||||
| zellij | warm | default | 7.774 → 7.178 | 7.581 → 6.981 | 820.9 → 822.0 |
|
||||
| ripgrep | warm | full | 7.976 → 7.875 | 7.746 → 7.679 | 812.7 → 789.3 |
|
||||
| zellij | warm | full | 14.995 → 13.978 | 14.742 → 13.718 | 818.5 → 811.5 |
|
||||
|
||||
<!-- END:allocation-report:normal -->
|
||||
|
||||
### SDK span and audit separation
|
||||
|
||||
<!-- BEGIN:allocation-report:sdk -->
|
||||
|
||||
Cold/default directory audit wall medians are 1.273 s baseline / 0.782 s candidate; SDK spans are 0.944 → 0.948 s. Whole-family costs include those experimental audits. Their cause and per-span CPU are unmeasured.
|
||||
|
||||
SDK span from client connect through task checks and client/server close. Excludes pre/post runtime binding and preflight artifact/oracle loading. Diagnostic spans include inspector and forced GC. CPU is measured for the whole family only; per-span CPU is unmeasured.
|
||||
|
||||
| Task | Cache | Inputs | SDK span s, baseline → candidate |
|
||||
| --------- | ----- | ------- | -------------------------------: |
|
||||
| directory | cold | default | 0.944 → 0.948 |
|
||||
| directory | cold | full | 1.067 → 1.102 |
|
||||
| directory | warm | default | 0.888 → 0.878 |
|
||||
| directory | warm | full | 1.038 → 1.042 |
|
||||
| ripgrep | cold | default | 5.778 → 5.752 |
|
||||
| zellij | cold | default | 5.998 → 5.991 |
|
||||
| ripgrep | cold | full | 6.077 → 6.119 |
|
||||
| zellij | cold | full | 6.228 → 6.209 |
|
||||
| ripgrep | warm | default | 5.743 → 5.656 |
|
||||
| zellij | warm | default | 5.965 → 5.788 |
|
||||
| ripgrep | warm | full | 6.100 → 6.492 |
|
||||
| zellij | warm | full | 11.776 → 11.529 |
|
||||
|
||||
<!-- END:allocation-report:sdk -->
|
||||
|
||||
Full-profile warm Zellij is slower than its cold condition in both roles in this cohort. Cache hit rate and the cause of this observation are unmeasured, and cold/warm groups run sequentially. These timings support neither a general cache speedup nor a causal regression attribution.
|
||||
|
||||
## Separate retained-heap diagnostics
|
||||
|
||||
The same completed SDK task launches only its owned REA process with `--inspect=127.0.0.1:0`. Four CDP calls verify the owned transport PID, read heap, force GC and read heap again. The endpoint comes from that child’s stderr; the PID is a cross-check of the owned endpoint, not sole ownership proof. The inspector socket closes before MCP shutdown. These V8 values include inspector perturbation and exclude the client; diagnostic wall/CPU/family peaks include inspector and forced-GC overhead and are not normal-task timing results.
|
||||
|
||||
<!-- BEGIN:allocation-report:heap -->
|
||||
|
||||
| Inputs | REA V8 retained MiB | Baseline range | Candidate range | Diagnostic wall s | CPU s | Family peak MiB |
|
||||
| ------- | ------------------: | --------------: | --------------: | ----------------: | --------------: | --------------: |
|
||||
| default | 189.835 → 180.242 | 189.823–189.855 | 180.208–180.366 | 15.490 → 14.291 | 15.246 → 14.123 | 814.1 → 786.3 |
|
||||
| full | 191.260 → 181.693 | 191.165–191.304 | 181.588–181.848 | 8.979 → 8.377 | 8.769 → 8.142 | 810.3 → 824.9 |
|
||||
|
||||
<!-- END:allocation-report:heap -->
|
||||
|
||||
A mismatched PID is rejected; removing only that guard makes the negative oracle fail. Both connections in that check use the same owned REA endpoint, with no foreign-process attachment. Aggregation rejects diagnostic GC in a normal trial. Historical #4424 receipt reaggregation preserves previous metrics and adds the separately reported SDK-span fields; its aggregator source hash changes.
|
||||
|
||||
## Context accounting
|
||||
|
||||
<!-- BEGIN:allocation-report:context -->
|
||||
|
||||
tiktoken==0.14.0/cl100k_base; estimates, not model usage. Actual model usage is unmeasured.
|
||||
|
||||
| Profile | Input directory once | Directory task | ripgrep task | Zellij task |
|
||||
| ------- | -------------------: | -------------: | -----------: | ----------: |
|
||||
| default | 49,670 | 50,840 | 54,029 | 53,949 |
|
||||
| full | 205,744 | 206,914 | 210,103 | 210,023 |
|
||||
|
||||
Raw SDK directory costs per list: 756,165/912,436 tokens and 2,935,148/3,587,792 bytes for default/full.
|
||||
|
||||
Question, explicit recipe, server instructions, one complete name/description/input-schema directory, every tool argument and text reply, plus prompt directory for discovery. This projection assumes the client retains a single unchanged directory. Raw catalog result costs include every observed tools/list result separately, with output schemas and without JSON-RPC envelopes. SDK initialization/framing, reasoning and actual model usage are unmeasured. The analysis task includes retained-read, rejection, export/oracle and post-materialization directory checks.
|
||||
|
||||
<!-- END:allocation-report:context -->
|
||||
|
||||
## Validation and reproduction
|
||||
|
||||
- Both runtime builds and all six static checks plus architecture guards pass. Before the final main synchronization, 5,972 source/contract/public CLI/MCP tests pass across 323 files; four skip for unavailable/guarded coverage. After synchronization, both builds/static checks repeat successfully, 77 affected producer and CLI/MCP tests pass, and all 974 registered numeric-parameter parser cases match again. Includes real SQLite CLI/MCP record/column selection and JavaScript heap/deadline preservation. A first direct schema-validity invocation lacked generated catalog metadata; the supported focused entry generates it and all 15 cases pass. A later affected-test invocation selected two nonexistent paths; the corrected three-file producer/consumer run is the 77-case result above. Neither setup failure was counted as a passing test, and no production or timeout change was needed.
|
||||
- All 84 measured tasks and four warm priming tasks pass independent oracles, canonical refusal, parent/directory identity and owned cleanup checks. This does not verify external Hopper/Ghidra/IDA, Android devices, all providers or foreign OS behavior. The earlier macOS numbers in #4360 describe a different revision/platform and are not combined with these results.
|
||||
|
||||
Build the pinned baseline/candidate in separate clean worktrees with the locked toolchain. Use the candidate’s producer scripts for both roles. `measurement_inputs` in the JSON records the complete matrix, questions/recipes, runtime bindings and producer hashes; extract it unchanged as the bindings file. Historical receipts require their matching producer revision. One normal task:
|
||||
|
||||
```bash
|
||||
NODE_DISABLE_COMPILE_CACHE=1 node experiments/analysis-efficiency/mcp-input-catalog.mjs \
|
||||
--runtime "$CANDIDATE_RUNTIME" --bindings "$MEASUREMENT_INPUTS" --role candidate \
|
||||
--artifact "$RIPGREP_BINARY" --oracle "$RIPGREP_ORACLE" \
|
||||
--output "$NEW_CASE_DIRECTORY" --profile default --phase analysis --cache cold
|
||||
```
|
||||
|
||||
Run each task through `resource-run.py --observe-payload-exit --` to obtain whole-family counters. For warm runs replace `--cache cold` with `--cache warm --cache-root <role/profile cache>` after one complete priming task. For the separate Zellij diagnostic tasks add `--diagnostic-heap` and use cold cache. Keep their case names/logs aligned with `measurement_inputs`; aggregate both cohorts with:
|
||||
|
||||
```bash
|
||||
python experiments/analysis-efficiency/mcp-startup-costs.py "$RECEIPT_ROOT" \
|
||||
--output mcp-integer-allocation.json
|
||||
```
|
||||
|
||||
The aggregator requires the recorded tiktoken version, exact catalog/parent/runtime/producer/guard identities, clean source, successful payload cleanup and matching matrix coverage. Default driver behavior continues without an inspector; the heap flag is confined to the experiment.
|
||||
|
||||
Regenerate or verify every numerical section directly from the committed JSON:
|
||||
|
||||
```bash
|
||||
node experiments/analysis-efficiency/mcp-integer-report.mjs
|
||||
node experiments/analysis-efficiency/mcp-integer-report.mjs --check
|
||||
```
|
||||
|
||||
The documentation conformance gate checks current receipts, rejects stale normal/heap tables, and refuses missing producer markers without writing.
|
||||
@@ -1,212 +0,0 @@
|
||||
// One producer for the numerical sections of the committed allocation report.
|
||||
import assert from "node:assert/strict";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { parseArgs } from "node:util";
|
||||
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
data: { type: "string" },
|
||||
markdown: { type: "string" },
|
||||
check: { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
const dataPath =
|
||||
values.data ?? new URL("mcp-integer-allocation.json", import.meta.url);
|
||||
const markdownPath =
|
||||
values.markdown ?? new URL("mcp-integer-allocation.md", import.meta.url);
|
||||
const data = JSON.parse(await readFile(dataPath, "utf8"));
|
||||
assert.equal(data.status, "passed");
|
||||
const original = await readFile(markdownPath, "utf8");
|
||||
const fixed = (value, digits = 3) => {
|
||||
assert.ok(Number.isFinite(value));
|
||||
return value.toFixed(digits);
|
||||
};
|
||||
const mib = (value, digits = 1) => fixed(value / 2 ** 20, digits);
|
||||
const median = (values) => {
|
||||
assert.ok(values.length > 0);
|
||||
const sorted = [...values].sort((a, b) => a - b);
|
||||
const middle = Math.floor(sorted.length / 2);
|
||||
return sorted.length % 2
|
||||
? sorted[middle]
|
||||
: (sorted[middle - 1] + sorted[middle]) / 2;
|
||||
};
|
||||
const table = (headers, rows, right = []) =>
|
||||
[
|
||||
`| ${headers.join(" | ")} |`,
|
||||
`| ${headers.map((_, index) => (right.includes(index) ? "---:" : "---")).join(" | ")} |`,
|
||||
...rows.map((row) => `| ${row.join(" | ")} |`),
|
||||
].join("\n");
|
||||
const pair = (item, key, format = fixed) =>
|
||||
`${format(item.baseline[key].median)} → ${format(item.candidate[key].median)}`;
|
||||
const heapMetric = "retained_v8_heap_bytes_after_gc";
|
||||
const heaps = data.heap_diagnostics.summary;
|
||||
const changes = heaps.map(
|
||||
(item) =>
|
||||
(item.baseline[heapMetric].median - item.candidate[heapMetric].median) /
|
||||
2 ** 20,
|
||||
);
|
||||
const changesText = [...new Set(changes.map((value) => fixed(value, 1)))].join(
|
||||
"–",
|
||||
);
|
||||
const separated = heaps.every(
|
||||
(item) => item.baseline[heapMetric].min > item.candidate[heapMetric].max,
|
||||
);
|
||||
const provenance = `Baseline \`main@${data.measurement_inputs.baseline_revision}\`; candidate \`${data.measurement_inputs.candidate_revision}\`. This is a pinned comparison.\n\nIn separate inspector/forced-GC diagnostics, retained REA V8 heap decreases by about ${changesText} MiB. ${separated ? "The three-pair ranges do not overlap in either profile." : "Ranges overlap; inspect the observations below."} Inspector perturbation is included. Normal task timing, CPU and peak memory are reported separately; these observations do not establish a general latency or peak-memory reduction.`;
|
||||
const normal = table(
|
||||
[
|
||||
"Task",
|
||||
"Cache",
|
||||
"Input profile",
|
||||
"Wall s",
|
||||
"Family CPU s",
|
||||
"Family peak MiB",
|
||||
],
|
||||
data.summary.map((item) => [
|
||||
item.artifact ?? "directory",
|
||||
item.cache,
|
||||
item.profile,
|
||||
pair(item, "wall_seconds"),
|
||||
pair(item, "cpu_seconds"),
|
||||
pair(item, "kernel_family_peak_bytes_including_page_cache", mib),
|
||||
]),
|
||||
[3, 4, 5],
|
||||
);
|
||||
const discovery = data.summary.find(
|
||||
(item) =>
|
||||
item.phase === "discovery" &&
|
||||
item.cache === "cold" &&
|
||||
item.profile === "default",
|
||||
);
|
||||
assert.ok(discovery);
|
||||
const audit = (role) =>
|
||||
median(
|
||||
data.records
|
||||
.filter(
|
||||
(row) =>
|
||||
row.phase === "discovery" &&
|
||||
row.cache === "cold" &&
|
||||
row.profile === "default" &&
|
||||
row.role === role,
|
||||
)
|
||||
.map(
|
||||
(row) =>
|
||||
(row.runtime_audit_ms.before + row.runtime_audit_ms.after) / 1000,
|
||||
),
|
||||
);
|
||||
const sdk =
|
||||
`Cold/default directory audit wall medians are ${fixed(audit("baseline"))} s baseline / ${fixed(audit("candidate"))} s candidate; SDK spans are ${pair(discovery, "sdk_task_elapsed_seconds")} s. Whole-family costs include those experimental audits. Their cause and per-span CPU are unmeasured.\n\n${data.sdk_task_elapsed_scope}\n\n` +
|
||||
table(
|
||||
["Task", "Cache", "Inputs", "SDK span s, baseline → candidate"],
|
||||
data.summary.map((item) => [
|
||||
item.artifact ?? "directory",
|
||||
item.cache,
|
||||
item.profile,
|
||||
pair(item, "sdk_task_elapsed_seconds"),
|
||||
]),
|
||||
[3],
|
||||
);
|
||||
const heap = table(
|
||||
[
|
||||
"Inputs",
|
||||
"REA V8 retained MiB",
|
||||
"Baseline range",
|
||||
"Candidate range",
|
||||
"Diagnostic wall s",
|
||||
"CPU s",
|
||||
"Family peak MiB",
|
||||
],
|
||||
heaps.map((item) => [
|
||||
item.profile,
|
||||
pair(item, heapMetric, (value) => mib(value, 3)),
|
||||
...["baseline", "candidate"].map(
|
||||
(role) =>
|
||||
`${mib(item[role][heapMetric].min, 3)}–${mib(item[role][heapMetric].max, 3)}`,
|
||||
),
|
||||
pair(item, "wall_seconds"),
|
||||
pair(item, "cpu_seconds"),
|
||||
pair(item, "kernel_family_peak_bytes_including_page_cache", mib),
|
||||
]),
|
||||
[1, 2, 3, 4, 5, 6],
|
||||
);
|
||||
const integer = (value) => new Intl.NumberFormat("en-US").format(value);
|
||||
const tokens = (profile, phase, artifact) => {
|
||||
const values = data.records
|
||||
.filter(
|
||||
(row) =>
|
||||
row.profile === profile &&
|
||||
row.phase === phase &&
|
||||
row.artifact === artifact,
|
||||
)
|
||||
.map((row) => row.tokens_estimated.whole_task);
|
||||
assert.ok(values.length > 0);
|
||||
return [...new Set([Math.min(...values), Math.max(...values)])]
|
||||
.map(integer)
|
||||
.join("–");
|
||||
};
|
||||
const catalogs = data.catalogs;
|
||||
const context =
|
||||
`${data.tokenizer}. Actual model usage is unmeasured.\n\n` +
|
||||
table(
|
||||
[
|
||||
"Profile",
|
||||
"Input directory once",
|
||||
"Directory task",
|
||||
"ripgrep task",
|
||||
"Zellij task",
|
||||
],
|
||||
["default", "full"].map((profile) => [
|
||||
profile,
|
||||
integer(catalogs[profile].input_catalog_tokens),
|
||||
tokens(profile, "discovery", null),
|
||||
tokens(profile, "analysis", "ripgrep"),
|
||||
tokens(profile, "analysis", "zellij"),
|
||||
]),
|
||||
[1, 2, 3, 4],
|
||||
) +
|
||||
`\n\nRaw SDK directory costs per list: ${integer(catalogs.default.raw_catalog_tokens)}/${integer(catalogs.full.raw_catalog_tokens)} tokens and ${integer(catalogs.default.raw_catalog_bytes)}/${integer(catalogs.full.raw_catalog_bytes)} bytes for default/full.\n\n${data.context_scope}`;
|
||||
const sections = { provenance, normal, sdk, heap, context };
|
||||
// oxfmt may pad table cells and separator dashes; neither changes report facts.
|
||||
const normalized = (value) =>
|
||||
value
|
||||
.trim()
|
||||
.replace(/\s+/gu, " ")
|
||||
.replace(/\s*\|\s*/gu, "|")
|
||||
.replace(
|
||||
/(?<=\|)(:)?-{3,}(:)?(?=\|)/gu,
|
||||
(_, left, right) => `${left ?? ""}---${right ?? ""}`,
|
||||
);
|
||||
let rendered = original;
|
||||
const changed = [];
|
||||
for (const [name, content] of Object.entries(sections)) {
|
||||
const begin = `<!-- BEGIN:allocation-report:${name} -->`;
|
||||
const end = `<!-- END:allocation-report:${name} -->`;
|
||||
assert.equal(
|
||||
rendered.split(begin).length,
|
||||
2,
|
||||
`Expected one opening report marker: ${name}`,
|
||||
);
|
||||
assert.equal(
|
||||
rendered.split(end).length,
|
||||
2,
|
||||
`Expected one closing report marker: ${name}`,
|
||||
);
|
||||
const start = rendered.indexOf(begin) + begin.length;
|
||||
const finish = rendered.indexOf(end);
|
||||
assert.ok(finish > start, `Invalid report marker order: ${name}`);
|
||||
if (normalized(rendered.slice(start, finish)) !== normalized(content)) {
|
||||
changed.push(name);
|
||||
rendered =
|
||||
rendered.slice(0, start) + `\n\n${content}\n\n` + rendered.slice(finish);
|
||||
}
|
||||
}
|
||||
if (values.check) {
|
||||
assert.equal(
|
||||
changed.length,
|
||||
0,
|
||||
`Generated allocation report differs: ${changed.join(", ")}`,
|
||||
);
|
||||
console.log("Allocation report matches committed receipt data");
|
||||
} else if (changed.length) {
|
||||
await writeFile(markdownPath, rendered);
|
||||
console.log(`Updated allocation report sections: ${changed.join(", ")}`);
|
||||
}
|
||||
@@ -1,107 +0,0 @@
|
||||
// Diagnostic only: inspect the explicitly launched REA process after its task.
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
export const sampleRetainedHeap = async (endpoint, expectedPid) => {
|
||||
const url = new URL(endpoint);
|
||||
assert.equal(url.protocol, "ws:");
|
||||
assert.equal(url.hostname, "127.0.0.1");
|
||||
assert.ok(Number.isSafeInteger(expectedPid) && expectedPid > 0);
|
||||
const socket = new WebSocket(url);
|
||||
let sequence = 0;
|
||||
const pending = new Map();
|
||||
const closed = new Promise((resolve) => {
|
||||
socket.addEventListener("close", resolve, { once: true });
|
||||
});
|
||||
const rejectPending = (cause) => {
|
||||
for (const request of pending.values()) {
|
||||
clearTimeout(request.timer);
|
||||
request.reject(cause);
|
||||
}
|
||||
pending.clear();
|
||||
};
|
||||
socket.addEventListener("error", () =>
|
||||
rejectPending(new Error("Owned REA inspector connection failed")),
|
||||
);
|
||||
socket.addEventListener("close", () =>
|
||||
rejectPending(new Error("Owned REA inspector connection closed")),
|
||||
);
|
||||
socket.addEventListener("message", (event) => {
|
||||
try {
|
||||
const reply = JSON.parse(event.data);
|
||||
const request = pending.get(reply.id);
|
||||
if (request === undefined) return;
|
||||
pending.delete(reply.id);
|
||||
clearTimeout(request.timer);
|
||||
if (reply.error !== undefined)
|
||||
request.reject(new Error(JSON.stringify(reply.error)));
|
||||
else request.resolve(reply.result);
|
||||
} catch (cause) {
|
||||
rejectPending(cause);
|
||||
}
|
||||
});
|
||||
const withinDeadline = async (promise) => {
|
||||
let timer;
|
||||
try {
|
||||
return await Promise.race([
|
||||
promise,
|
||||
new Promise((_, reject) => {
|
||||
timer = setTimeout(
|
||||
() => reject(new Error("Owned REA inspector deadline exceeded")),
|
||||
10_000,
|
||||
);
|
||||
}),
|
||||
]);
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
};
|
||||
const call = (method, params = {}) =>
|
||||
new Promise((resolve, reject) => {
|
||||
const id = ++sequence;
|
||||
const timer = setTimeout(() => {
|
||||
pending.delete(id);
|
||||
reject(new Error(`Owned REA inspector timed out: ${method}`));
|
||||
}, 10_000);
|
||||
pending.set(id, { resolve, reject, timer });
|
||||
try {
|
||||
socket.send(JSON.stringify({ id, method, params }));
|
||||
} catch (cause) {
|
||||
pending.delete(id);
|
||||
clearTimeout(timer);
|
||||
reject(cause);
|
||||
}
|
||||
});
|
||||
try {
|
||||
await withinDeadline(
|
||||
new Promise((resolve, reject) => {
|
||||
socket.addEventListener("open", resolve, { once: true });
|
||||
socket.addEventListener("error", reject, { once: true });
|
||||
socket.addEventListener("close", reject, { once: true });
|
||||
}),
|
||||
);
|
||||
const identity = await call("Runtime.evaluate", {
|
||||
expression: "process.pid",
|
||||
returnByValue: true,
|
||||
});
|
||||
assert.equal(identity.result.value, expectedPid);
|
||||
const before = await call("Runtime.getHeapUsage");
|
||||
await call("HeapProfiler.collectGarbage");
|
||||
const after = await call("Runtime.getHeapUsage");
|
||||
for (const sample of [before, after]) {
|
||||
assert.ok(Number.isSafeInteger(sample.usedSize) && sample.usedSize >= 0);
|
||||
assert.ok(Number.isSafeInteger(sample.totalSize));
|
||||
assert.ok(sample.totalSize >= sample.usedSize);
|
||||
}
|
||||
return {
|
||||
scope:
|
||||
"Owned REA V8 heap after the complete SDK task; inspector and forced GC are diagnostic overhead",
|
||||
pid_verified: true,
|
||||
before_gc: before,
|
||||
after_gc: after,
|
||||
};
|
||||
} finally {
|
||||
rejectPending(new Error("Owned REA inspector diagnostic finished"));
|
||||
socket.close();
|
||||
await withinDeadline(closed);
|
||||
}
|
||||
};
|
||||
@@ -1,151 +0,0 @@
|
||||
"""Verify paired SDK startup/analysis receipts and account for the whole task."""
|
||||
import argparse
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import math
|
||||
import statistics
|
||||
from pathlib import Path
|
||||
|
||||
import tiktoken
|
||||
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('root', type=Path)
|
||||
parser.add_argument('--output', type=Path, required=True)
|
||||
args = parser.parse_args()
|
||||
root = args.root
|
||||
input_bytes = (root / 'measurement-inputs.json').read_bytes()
|
||||
inputs = json.loads(input_bytes)
|
||||
expected = inputs['runtime_bindings']
|
||||
sha = lambda value: hashlib.sha256(value).hexdigest()
|
||||
compact = lambda value: json.dumps(value, ensure_ascii=False, separators=(',', ':'), allow_nan=False)
|
||||
encoding = tiktoken.get_encoding('cl100k_base')
|
||||
count = lambda value: len(encoding.encode(value, disallowed_special=()))
|
||||
for role in ['baseline', 'candidate']:
|
||||
assert expected[role]['git_revision'] == inputs[f'{role}_revision']
|
||||
assert expected[role]['source_diff_sha256'] == sha(b'')
|
||||
assert expected[role]['untracked_source_sha256'] == sha(b'[]')
|
||||
assert expected['baseline']['root'] != expected['candidate']['root']
|
||||
assert expected['baseline']['compiled_runtime_sha256'] != expected['candidate']['compiled_runtime_sha256']
|
||||
spec = importlib.util.spec_from_file_location('catalog_costs', Path(__file__).with_name('mcp-input-catalog-costs.py'))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
catalogs = {}
|
||||
identities = {}
|
||||
rows = []
|
||||
heap_rows = []
|
||||
for case in inputs['cases'] + inputs.get('diagnostic_cases', []):
|
||||
directory = root / case['case']
|
||||
report = json.loads((directory / 'report.json').read_text())
|
||||
assert report['status'] == 'passed' and report['cleanup_verified']
|
||||
assert report['runtime_binding_verified']
|
||||
assert report['measurement_inputs_sha256'] == sha(input_bytes)
|
||||
for name, producer_digest in report['producer_sources_sha256'].items():
|
||||
assert producer_digest == inputs['harness_sources_sha256'][name]
|
||||
assert report['runtime_binding'] == expected[case['role']]
|
||||
assert type(report['elapsed_ms']) in [int, float] and math.isfinite(report['elapsed_ms']) and report['elapsed_ms'] >= 0
|
||||
for key in ['role', 'cache', 'profile', 'phase']:
|
||||
assert report[key] == case[key]
|
||||
raw = (directory / 'catalog.json').read_bytes()
|
||||
catalog_digest = sha(raw)
|
||||
assert catalog_digest == report['catalog_sha256']
|
||||
profile = case['profile']
|
||||
if profile not in catalogs:
|
||||
catalog = json.loads(raw)
|
||||
definitions = [{key: tool[key] for key in ['name', 'description', 'inputSchema']} for tool in catalog['tools']]
|
||||
catalogs[profile] = {'sha256': catalog_digest, 'tool_count': len(catalog['tools']),
|
||||
'input_catalog_tokens': count(compact(definitions)),
|
||||
'raw_catalog_tokens': count(raw.decode()), 'raw_catalog_bytes': len(raw)}
|
||||
assert catalog_digest == catalogs[profile]['sha256'], 'Directory differs across roles or phases'
|
||||
transcript = json.loads((directory / 'transcript.json').read_text())
|
||||
server = json.loads((directory / 'server.json').read_text())
|
||||
tokens = {'question_and_recipe': count(inputs['questions'][case['phase']] + '\n' + inputs['recipes'][case['phase']]),
|
||||
'server_instructions': count(server.get('instructions') or ''),
|
||||
'input_catalog_once': catalogs[profile]['input_catalog_tokens'],
|
||||
'arguments': sum(count(compact({'name': call['name'], 'arguments': call['arguments']})) for call in transcript),
|
||||
'text_replies': sum(count(block['text']) for call in transcript for block in call['reply']['content'] if block['type'] == 'text')}
|
||||
requests = report['protocol_requests']
|
||||
assert requests['initialize'] == 1
|
||||
if case['phase'] == 'analysis':
|
||||
assert len(transcript) == 5 and requests['tools/call'] == 5 and requests['tools/list'] == 2
|
||||
assert report['artifact_sha256'] == inputs['artifacts'][case['artifact']]['sha256']
|
||||
identity = (report['parent_evidence_id'], report['parent_semantic_sha256'], sha(compact(report['answer']).encode()))
|
||||
assert identities.setdefault(case['artifact'], identity) == identity
|
||||
assert sum(call['is_error'] for call in report['calls']) == 1
|
||||
else:
|
||||
assert not transcript and requests.get('tools/call', 0) == 0
|
||||
assert requests['tools/list'] == 1 and requests['prompts/list'] == 1
|
||||
prompts = (directory / 'prompts.json').read_bytes()
|
||||
assert sha(prompts) == report['prompts_sha256']
|
||||
assert identities.setdefault('prompts', report['prompts_sha256']) == report['prompts_sha256']
|
||||
tokens['prompt_directory'] = count(prompts.decode())
|
||||
tokens['whole_task'] = sum(tokens.values())
|
||||
resource = module.resource(directory)
|
||||
for name, guard_digest in resource['guard_sources_sha256'].items():
|
||||
assert guard_digest == inputs['harness_sources_sha256'][name]
|
||||
del resource['guard_sources_sha256']
|
||||
row = {**case, 'runtime_binding_sha256': sha(compact(report['runtime_binding']).encode()), 'runtime_audit_ms': report['runtime_audit_ms'],
|
||||
'sdk_task_elapsed_seconds': report['elapsed_ms'] / 1000,
|
||||
'catalog_sha256': catalog_digest, 'parent_evidence_id': report.get('parent_evidence_id'),
|
||||
'parent_semantic_sha256': report.get('parent_semantic_sha256'),
|
||||
'answer_sha256': sha(compact(report['answer']).encode()), 'tokens_estimated': tokens,
|
||||
'raw_catalog_result_tokens_all_lists': catalogs[profile]['raw_catalog_tokens'] * requests['tools/list'],
|
||||
'raw_catalog_result_bytes_all_lists': catalogs[profile]['raw_catalog_bytes'] * requests['tools/list'],
|
||||
'actual_model_usage': None, 'protocol_era': report['protocol_era'], 'protocol_requests': requests, 'calls': report['calls'],
|
||||
**resource,
|
||||
'receipt_sha256': {name: sha((directory / name).read_bytes()) for name in ['report.json', 'catalog.json', 'server.json', 'transcript.json']}}
|
||||
assert row['catalog_sha256'] == row['receipt_sha256']['catalog.json'] == catalogs[profile]['sha256']
|
||||
heap = report.get('diagnostic_heap')
|
||||
if case.get('diagnostic_heap'):
|
||||
assert case['phase'] == 'analysis' and case['cache'] == 'cold' and case['artifact'] == 'zellij'
|
||||
assert heap is not None and heap['pid_verified'] is True
|
||||
for sample in [heap['before_gc'], heap['after_gc']]:
|
||||
assert type(sample['usedSize']) is int and sample['usedSize'] >= 0
|
||||
assert type(sample['totalSize']) is int and sample['totalSize'] >= sample['usedSize']
|
||||
row['heap_observation'] = heap
|
||||
row['retained_v8_heap_bytes_after_gc'] = heap['after_gc']['usedSize']
|
||||
heap_rows.append(row)
|
||||
else:
|
||||
assert heap is None, 'Diagnostic GC contaminated a normal task trial'
|
||||
rows.append(row)
|
||||
assert len(rows) == 72
|
||||
summary = []
|
||||
for phase in ['discovery', 'analysis']:
|
||||
for cache in ['cold', 'warm']:
|
||||
for profile in ['default', 'full']:
|
||||
for artifact in ([None] if phase == 'discovery' else ['ripgrep', 'zellij']):
|
||||
item = {'phase': phase, 'cache': cache, 'profile': profile, 'artifact': artifact}
|
||||
for role in ['baseline', 'candidate']:
|
||||
selected = [row for row in rows if all(row[key] == item[key] for key in ['phase', 'cache', 'profile', 'artifact']) and row['role'] == role]
|
||||
assert len(selected) == 3
|
||||
item[role] = {metric: {'median': statistics.median(row[metric] for row in selected),
|
||||
'min': min(row[metric] for row in selected), 'max': max(row[metric] for row in selected)}
|
||||
for metric in ['wall_seconds', 'cpu_seconds', 'kernel_family_peak_bytes_including_page_cache', 'sdk_task_elapsed_seconds']}
|
||||
item[role]['tokens_estimated'] = [row['tokens_estimated']['whole_task'] for row in selected]
|
||||
summary.append(item)
|
||||
result = {'status': 'passed', 'tokenizer': f'tiktoken=={tiktoken.__version__}/cl100k_base; estimates, not model usage',
|
||||
'aggregation_sources_sha256': {path.name: sha(path.read_bytes()) for path in [Path(__file__), Path(spec.origin)]},
|
||||
'context_scope': 'Question, explicit recipe, server instructions, one complete name/description/input-schema directory, every tool argument and text reply, plus prompt directory for discovery. This projection assumes the client retains a single unchanged directory. Raw catalog result costs include every observed tools/list result separately, with output schemas and without JSON-RPC envelopes. SDK initialization/framing, reasoning and actual model usage are unmeasured. The analysis task includes retained-read, rejection, export/oracle and post-materialization directory checks.',
|
||||
'cache_scope': 'Cold disables server Node compile cache. Warm enables a prepopulated role/profile-specific cache after one complete analysis priming run. Fresh server/client processes and HOME each trial. Driver compile cache disabled in both cohorts. Cache hit rate and OS page cache are unmeasured; source/runtime audits warm compiled-file pages before startup.',
|
||||
'resource_scope': 'Whole SDK driver, resource observer and owned REA process family. Includes before/after source-and-all-compiled-JS/JSON binding, SDK schema checks, extra validation calls, bundle export and independent oracle checking. Kernel cgroup peak includes file cache. Build/download/oracle preparation and cache priming excluded. Retained V8 heap and forced-GC diagnostics were not separately measured.',
|
||||
'measurement_inputs': inputs, 'catalogs': catalogs, 'summary': summary, 'records': rows,
|
||||
'sdk_task_elapsed_scope': 'SDK span from client connect through task checks and client/server close. Excludes pre/post runtime binding and preflight artifact/oracle loading. Diagnostic spans include inspector and forced GC. CPU is measured for the whole family only; per-span CPU is unmeasured.',
|
||||
'warmup_receipts': {name: sha((root / name / 'report.json').read_bytes()) for name in [f'warmup-{profile}-{role}' for profile in ['default', 'full'] for role in ['baseline', 'candidate']]}}
|
||||
if heap_rows:
|
||||
assert len(heap_rows) == 12
|
||||
heap_summary = []
|
||||
for profile in ['default', 'full']:
|
||||
item = {'profile': profile}
|
||||
for role in ['baseline', 'candidate']:
|
||||
selected = [row for row in heap_rows if row['profile'] == profile and row['role'] == role]
|
||||
assert len(selected) == 3
|
||||
item[role] = {metric: {'median': statistics.median(row[metric] for row in selected),
|
||||
'min': min(row[metric] for row in selected), 'max': max(row[metric] for row in selected)}
|
||||
for metric in ['retained_v8_heap_bytes_after_gc', 'wall_seconds', 'cpu_seconds', 'sdk_task_elapsed_seconds',
|
||||
'kernel_family_peak_bytes_including_page_cache']}
|
||||
heap_summary.append(item)
|
||||
result['heap_diagnostics'] = {
|
||||
'scope': 'Separate three-pair Zellij tasks per profile, with an explicitly launched loopback inspector. Four CDP calls verify the owned REA PID, read heap, force GC and read heap again after the completed SDK task. These whole-family costs include inspector and diagnostic GC overhead; they are excluded from the normal timing cohort. Heap values describe REA V8 only, not the client or the whole process family.',
|
||||
'summary': heap_summary, 'records': heap_rows}
|
||||
args.output.write_text(json.dumps(result, indent=2) + '\n')
|
||||
print(json.dumps(summary, indent=2))
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,61 +0,0 @@
|
||||
# MCP discovery and complete Rust tasks
|
||||
|
||||
Target-free stdio discovery previously constructed the canonical analysis runtime before the client could inspect its directory. The candidate records SDK advertisements at build time for default/explicit compact and explicit full inputs. Its first tool, prompt or completion request materializes canonical validators and handlers on the same SDK server. HTTP, output-schema-off, initial-target and unrecorded source runs retain eager composition.
|
||||
|
||||
Pinned baseline: [`main@c465ff5a8`](https://github.com/morluto/rea/commit/c465ff5a814b8e1a386fc547cbcd94df5c35f086). Measured candidate: [`04900abbc`](https://github.com/leeguooooo/rea/commit/04900abbcd3c3438fce4af932ff28400fe744a7d). Both use the same main base and locked dependencies. Before/after source and all compiled JS/JSON bindings identify the exact runtime used in every trial. The input manifest and producer scripts are also hashed and checked.
|
||||
|
||||
## Real tasks and independent facts
|
||||
|
||||
Official Alpine v3.21 packages supply ripgrep 14.1.1-r0 (3,416,168-byte executable) and Zellij 0.40.0-r0 (17,151,864 bytes). Their download URLs, archive/executable digests and aports revisions are retained in [the measurement data](mcp-startup.json). The independent Python ELF/section/zlib/JSON oracle checks 42 packages and 88 dependency edges for ripgrep, and 422 packages and 1,063 edges for Zellij. Serde versions are 1.0.210 and 1.0.137 respectively; the selected package kind remains explicit unknown (`null`). Verification covers embedded static build metadata; execution and runtime reachability remain unverified.
|
||||
|
||||
Each analysis task initializes the actual SDK client, lists and validates the input directory, selects serde name/version/kind, reads the exact retained parent twice, verifies canonical rejection of a negative offset, exports and independently checks the complete package/edge inventory, confirms unchanged advertisements after materialization, then closes. All five tool calls and both directory reads are included in the measured task. Discovery tasks initialize, list tools/prompts, validate the input directory and close without an analysis call.
|
||||
|
||||
All 72 paired trials and four priming runs passed. Each artifact has identical answers, parent Evidence IDs and complete semantic hashes across roles, profiles and cache modes. Both profiles retain byte-identical directories across baseline/candidate and before/after canonical materialization. The directory contains 213 tools. Complete Evidence is retained and exported through the existing shared workflow.
|
||||
|
||||
## Measured cost
|
||||
|
||||
Three alternating pairs per condition; table values are medians. Min/max ranges and individual receipts are in [mcp-startup.json](mcp-startup.json). Each cell is baseline → candidate. Kernel process-family peak includes file cache, SDK driver, observer and REA children.
|
||||
|
||||
| Task | Cache | Inputs | Wall seconds | Family CPU seconds | Family peak MiB |
|
||||
| --------- | ----- | ----------------- | ------------: | -----------------: | --------------: |
|
||||
| Directory | cold | compact (default) | 5.661 → 2.236 | 5.403 → 2.050 | 559.3 → 248.6 |
|
||||
| Directory | cold | full | 5.256 → 2.337 | 5.040 → 2.175 | 650.9 → 349.1 |
|
||||
| Directory | warm | compact (default) | 5.056 → 2.236 | 4.800 → 2.015 | 572.8 → 251.4 |
|
||||
| Directory | warm | full | 4.957 → 2.338 | 4.754 → 2.139 | 672.7 → 349.6 |
|
||||
| ripgrep | cold | compact (default) | 8.177 → 8.078 | 7.945 → 7.899 | 849.7 → 792.5 |
|
||||
| zellij | cold | compact (default) | 8.276 → 7.276 | 8.041 → 7.105 | 846.2 → 819.0 |
|
||||
| ripgrep | cold | full | 8.180 → 7.483 | 8.000 → 7.312 | 846.5 → 794.4 |
|
||||
| zellij | cold | full | 8.484 → 7.776 | 8.284 → 7.514 | 850.4 → 813.1 |
|
||||
| ripgrep | warm | compact (default) | 7.976 → 6.879 | 7.739 → 6.678 | 826.8 → 802.6 |
|
||||
| zellij | warm | compact (default) | 8.285 → 7.472 | 8.019 → 7.254 | 882.1 → 811.4 |
|
||||
| ripgrep | warm | full | 8.074 → 7.476 | 7.890 → 7.265 | 852.1 → 803.6 |
|
||||
| zellij | warm | full | 8.382 → 7.479 | 8.151 → 7.293 | 854.3 → 830.0 |
|
||||
|
||||
Discovery improves clearly in this cohort. Completed-task changes depend on the workload: default cold ripgrep is approximately unchanged. Initialization moves into the first actual analysis request; its latency must be assessed with the full workflow.
|
||||
|
||||
| Default ripgrep first RPC | Baseline ms | Candidate ms |
|
||||
| ------------------------- | ----------: | -----------: |
|
||||
| cold | 231.5 | 2532.1 |
|
||||
| warm | 227.1 | 2118.2 |
|
||||
|
||||
Within each profile, advertised context remains identical. Input-directory estimates are 49,670 tokens for default compact and 205,744 for full. Whole-task estimates include the supplied question/recipe, server instructions, directory once, every tool argument and text reply, and the prompt directory in discovery tasks. This projection assumes the client retains one unchanged directory; raw catalog result costs count every observed list result separately, including output schemas. These use tiktoken 0.14.0/cl100k_base; actual model usage, reasoning and SDK initialization/framing are unmeasured.
|
||||
|
||||
Linux x64 / AMD EPYC 7B12 / kernel 6.1.0-54-cloud-amd64 / Node 24.18.0, npm 11.16.0. Timed SDK tasks negotiated the legacy protocol era; auto/modern correctness is covered separately by compiled boundary tests. Trials run serially under a 1-CPU quota pinned to CPU 15, 4 GiB hard memory / 3 GiB early stop, no swap and 192 tasks. No resource stop, OOM or remaining payload process was observed. Cold disables server Node compile cache; warm uses a role/profile-specific cache populated by one complete task. Fresh processes and HOME are used each trial. Cache hit rate and OS page cache are unmeasured. Runtime audits warm compiled-file pages and are included in whole-task costs. Builds, downloads, oracle generation and cache priming are preparation outside those costs. Retained V8 heap and forced-GC measurements were not separately collected.
|
||||
|
||||
## Ownership and validation
|
||||
|
||||
`OwnedMcpServers` seals publication before taking its close snapshot, keeps failed exact close capabilities for retry and serves both eager and deferred paths. Deferred startup still joins servers published before an asynchronous serving handle settles. Post-snapshot and post-close factory calls are refused before allocation. Both production-boundary regressions fail with only the shared guard removed and pass restored.
|
||||
|
||||
The moved transport/delivery implementation preserves current complete-error retention, escaped-preview fitting, response/string-limit recovery, target-content provenance and native Evidence continuations. Canonical parsers remain authoritative after lazy binding. Both protocol eras, input/output profile choices, invalid input/output, completion cancellation, partial startup and cleanup retry are exercised.
|
||||
|
||||
Aggregation keeps catalog, producer-script and guard-script digests in distinct variables. Before publishing a trial it verifies that the catalog digest matches both its catalog-file receipt and the shared profile identity. Deliberately selecting the guard digest for that field is rejected. Regeneration preserves all measured timing, CPU, memory, context and parent-Evidence values.
|
||||
|
||||
Executed local gates: 125 focused tests across 14 files; a repeated 22-case schema/profile subset after the pinned main merge; all six static-check tasks plus architecture guards; builds for both pinned runtimes; six documentation checks and five README consistency tests. HTTP coverage includes the real compiled CLI session/replacement/Host tests. Runtime-binding admission rejects a mismatched compiled graph before MCP launch; aggregation rejects a tampered trial binding. The raw receipts remain in `/tmp/rea-perf-mcp-startup-20261022`, identified by hashes in the data. Real initial-target providers, foreign OS runtime behavior and other providers remain outside this measurement.
|
||||
|
||||
## Reproduction
|
||||
|
||||
Use the pinned Node/npm toolchain and separate clean baseline/candidate checkouts. Build each with `npm ci` and `npm run build:cached` (the measured workspace shared one locked dependency installation). Acquire and verify the official packages listed in the data, extract the named executables and generate independent oracles with `mcp-input-catalog-oracle.py`.
|
||||
|
||||
Capture each checkout with `node experiments/analysis-efficiency/managed-cli-runtime.mjs CHECKOUT mcp`. Construct `measurement-inputs.json` using the recorded schema, updating absolute checkout/artifact/oracle paths and their independently verified bindings. Keep it immutable through a cohort; producer source digests cover `mcp-input-catalog.mjs`, `managed-cli-runtime.mjs`, `resource-run.py` and `resource_payload.py`.
|
||||
|
||||
Run `mcp-input-catalog.mjs` through `resource-run.py --observe-payload-exit` with `--runtime`, `--output`, `--bindings`, `--role baseline|candidate`, `--profile default|full`, `--phase discovery|analysis` and `--cache cold|warm`. Analysis also takes `--artifact` and `--oracle`; warm requires a role/profile-specific `--cache-root`. Disable the driver compile cache for both cohorts. Prime each warm role/profile once with the complete ripgrep task. Follow the 72-case order in the data, reversing role order in every second pair. Aggregate with `python mcp-startup-costs.py LAB --output mcp-startup.json` using tiktoken 0.14.0. The aggregator verifies every input/runtime/guard binding and completed receipt before producing summaries.
|
||||
+2
-2
@@ -305,8 +305,8 @@
|
||||
"verify:solidity:metadata": "npm run build:cached && node scripts/verify-solidity-metadata.mjs",
|
||||
"verify:recorded:crash": "npm run build:cached && node scripts/verify-recorded-crash.mjs",
|
||||
"mcp-catalog:check:built": "node scripts/generate-mcp-tool-catalog.mjs --check",
|
||||
"compile": "node scripts/clean-build-output.mjs && node scripts/run-typescript.mjs -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/record-mcp-catalog.mjs && node scripts/bundle-runtime.mjs",
|
||||
"compile:termux": "node scripts/clean-build-output.mjs && node --max-old-space-size=3072 node_modules/typescript-termux/lib/tsc.js -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/record-mcp-catalog.mjs && node scripts/bundle-runtime.mjs",
|
||||
"compile": "node scripts/clean-build-output.mjs && node scripts/run-typescript.mjs -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/bundle-runtime.mjs",
|
||||
"compile:termux": "node scripts/clean-build-output.mjs && node --max-old-space-size=3072 node_modules/typescript-termux/lib/tsc.js -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/bundle-runtime.mjs",
|
||||
"artifacts:mcp-catalog": "node scripts/generate-mcp-tool-catalog.mjs",
|
||||
"artifacts:skills": "node scripts/generate-skill-metadata.mjs",
|
||||
"artifacts:product-catalog": "node scripts/generate-product-catalog.mjs",
|
||||
|
||||
@@ -118,7 +118,6 @@ export const SCOPES = {
|
||||
"tests/boundary/process/releaseArtifactCanary.test.ts",
|
||||
"src/windows/**",
|
||||
"scripts/lib/windows-owned-command.mjs",
|
||||
"scripts/lib/windows-private-runtime-acl.ps1",
|
||||
"scripts/run-exclusive-command.mjs",
|
||||
"tests/process-global/runnerCancellation.test.ts",
|
||||
"tests/process-global/windowsOwnedCommand.test.ts",
|
||||
@@ -171,7 +170,6 @@ export const SCOPES = {
|
||||
"tests/boundary/process/packageNpmInvocation.test.ts",
|
||||
"tests/boundary/process/packedBridgeVerification.test.ts",
|
||||
"tests/boundary/process/windowsOutputCapture.test.ts",
|
||||
"tests/boundary/process/windowsPrivateRuntimeAclObserver.test.ts",
|
||||
"tests/acceptance/webModuleCli.test.ts",
|
||||
"tests/acceptance/webSourceLocationCli.test.ts",
|
||||
"tests/boundary/cli/referenceSourceImportStatus.test.ts",
|
||||
|
||||
@@ -238,9 +238,7 @@ export const providerCatalog = (sources) => {
|
||||
identity: sources.pwntoolsProvider.PWNTOOLS_PROVIDER_IDENTITY,
|
||||
contracts:
|
||||
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
|
||||
({ name }) =>
|
||||
name === "inspect_binary_layout" ||
|
||||
name === "inspect_recorded_crash",
|
||||
({ name }) => name !== "inspect_pe_resources",
|
||||
),
|
||||
},
|
||||
{
|
||||
@@ -250,20 +248,6 @@ export const providerCatalog = (sources) => {
|
||||
({ name }) => name === "inspect_pe_resources",
|
||||
),
|
||||
},
|
||||
{
|
||||
identity: sources.artifactProviders.ELF_NOTES_PROVIDER,
|
||||
contracts:
|
||||
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
|
||||
({ name }) => name === "inspect_elf_notes",
|
||||
),
|
||||
},
|
||||
{
|
||||
identity: sources.artifactProviders.BYTE_ENTROPY_PROVIDER,
|
||||
contracts:
|
||||
sources.binaryDiagnosticsContracts.BINARY_DIAGNOSTICS_TOOL_CONTRACTS.filter(
|
||||
({ name }) => name === "inspect_byte_entropy",
|
||||
),
|
||||
},
|
||||
{
|
||||
identity: sources.captureProvider.TSHARK_PROVIDER_IDENTITY,
|
||||
contracts: sources.captureContracts.CAPTURE_TOOL_CONTRACTS,
|
||||
|
||||
@@ -39,7 +39,6 @@ export const parseDevelopmentTestRequest = (mode, arguments_) => {
|
||||
// runtime or typecheck prerequisites. Add a consumer here when it reads one.
|
||||
const artifactConsumers = {
|
||||
"artifacts:mcp-catalog": [
|
||||
"tests/boundary/mcp/deferredRuntime.test.ts",
|
||||
"tests/boundary/mcp/contractPresentation.test.ts",
|
||||
"tests/boundary/mcp/toolSchemaValidity.test.ts",
|
||||
],
|
||||
|
||||
@@ -1,33 +0,0 @@
|
||||
param($Acl, [string]$UserSid, [string]$Path, [Diagnostics.Stopwatch]$Clock)
|
||||
|
||||
# Keep this independent observer on .NET SID APIs. PowerShell convenience
|
||||
# properties and utility cmdlets can introduce account lookup/module loading.
|
||||
$PSModuleAutoLoadingPreference = 'None'
|
||||
function Trace-AclPhase([string]$phase) {
|
||||
[Console]::Error.WriteLine('private-runtime-acl: ' + $phase + ' ' + $Clock.ElapsedMilliseconds + 'ms')
|
||||
}
|
||||
$owner = $Acl.GetOwner([Security.Principal.SecurityIdentifier])
|
||||
Trace-AclPhase 'owner-sid-read'
|
||||
$rules = @($Acl.GetAccessRules($true, $true, [Security.Principal.SecurityIdentifier]))
|
||||
Trace-AclPhase 'rule-sids-read'
|
||||
$trusteesOnlyUserAndSystem = $true
|
||||
foreach ($rule in $rules) {
|
||||
$sid = $rule.IdentityReference.Value
|
||||
if ($sid -ne $UserSid -and $sid -ne 'S-1-5-18') {
|
||||
$trusteesOnlyUserAndSystem = $false
|
||||
}
|
||||
}
|
||||
[IO.File]::SetAttributes($Path, [IO.FileAttributes]([int][IO.File]::GetAttributes($Path) -bor 1))
|
||||
Trace-AclPhase 'readonly-set'
|
||||
|
||||
# The wire values are exclusively booleans and a collection count, so no
|
||||
# arbitrary strings need JSON escaping or a general-purpose serializer.
|
||||
$values = @(
|
||||
([bool]$Acl.AreAccessRulesProtected).ToString().ToLowerInvariant(),
|
||||
($owner.Value -eq $UserSid).ToString().ToLowerInvariant(),
|
||||
$trusteesOnlyUserAndSystem.ToString().ToLowerInvariant(),
|
||||
$rules.Count.ToString([Globalization.CultureInfo]::InvariantCulture),
|
||||
(([int][IO.File]::GetAttributes($Path) -band 1) -ne 0).ToString().ToLowerInvariant()
|
||||
)
|
||||
[Console]::WriteLine(('{{"protected":{0},"ownerMatchesUser":{1},"trusteesOnlyUserAndSystem":{2},"count":{3},"rootReadonly":{4}}}' -f $values))
|
||||
Trace-AclPhase 'complete'
|
||||
@@ -1,119 +0,0 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdtemp, rm, writeFile } from "node:fs/promises";
|
||||
import { tmpdir } from "node:os";
|
||||
import { dirname, join, resolve } from "node:path";
|
||||
import { fileURLToPath, pathToFileURL } from "node:url";
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
|
||||
// Runtime discovery is generated from the same canonical registrations, after
|
||||
// default identity has been recorded and before bundling. Test catalogs stay separate.
|
||||
const compiled = resolve(dirname(fileURLToPath(import.meta.url)), "../dist");
|
||||
const read = (name) => import(pathToFileURL(join(compiled, name)).href);
|
||||
const [
|
||||
{ createServer },
|
||||
{ createBinarySession },
|
||||
{ parseConfig },
|
||||
{ silentLogger },
|
||||
{ PROMPT_CONTRACTS },
|
||||
] = await Promise.all([
|
||||
read("server/createServer.js"),
|
||||
read("composition/binary.js"),
|
||||
read("config/parseConfig.js"),
|
||||
read("logger.js"),
|
||||
read("contracts/promptContracts.js"),
|
||||
]);
|
||||
const recordCatalog = async (profile) => {
|
||||
// Catalog generation is independent of the host account and user configuration.
|
||||
const home = await mkdtemp(join(tmpdir(), "rea-mcp-catalog-"));
|
||||
const environment = {
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: profile,
|
||||
HOME: home,
|
||||
USERPROFILE: home,
|
||||
XDG_CONFIG_HOME: join(home, "config"),
|
||||
XDG_CACHE_HOME: join(home, "cache"),
|
||||
XDG_STATE_HOME: join(home, "state"),
|
||||
};
|
||||
const cleanup = [];
|
||||
let producerFailure;
|
||||
try {
|
||||
const config = parseConfig(environment);
|
||||
if (!config.ok) throw config.error;
|
||||
const session = createBinarySession(
|
||||
config.value,
|
||||
silentLogger,
|
||||
environment,
|
||||
);
|
||||
cleanup.push(async () => {
|
||||
const closed = await session.close({ retainProviderDocuments: true });
|
||||
if (!closed.ok) throw closed.error;
|
||||
});
|
||||
const server = createServer({ kind: "session", session }, { environment });
|
||||
cleanup.push(() => server.close());
|
||||
const client = new Client({ name: "rea-catalog-recorder", version: "1" });
|
||||
cleanup.push(() => client.close());
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
const tools = (await client.listTools()).tools;
|
||||
const prompts = (await client.listPrompts()).prompts.map((prompt) => {
|
||||
const contract = PROMPT_CONTRACTS.find(
|
||||
(candidate) => candidate.name === prompt.name,
|
||||
);
|
||||
if (contract === undefined)
|
||||
throw new Error(`Missing prompt ${prompt.name}`);
|
||||
return { ...prompt, parameters: contract.arguments };
|
||||
});
|
||||
const catalog = {
|
||||
version: 1,
|
||||
info: client.getServerVersion(),
|
||||
capabilities: client.getServerCapabilities(),
|
||||
instructions: client.getInstructions(),
|
||||
identity: server.catalogIdentity,
|
||||
tools,
|
||||
prompts,
|
||||
};
|
||||
const encoded = JSON.stringify(catalog);
|
||||
const digest = createHash("sha256").update(encoded).digest("hex");
|
||||
await writeFile(
|
||||
join(
|
||||
compiled,
|
||||
profile === "full" ? "mcp-catalog-full.json" : "mcp-catalog.json",
|
||||
),
|
||||
encoded,
|
||||
);
|
||||
return digest;
|
||||
} catch (cause) {
|
||||
producerFailure = { cause };
|
||||
throw cause;
|
||||
} finally {
|
||||
const results = await Promise.allSettled(
|
||||
cleanup.reverse().map((close) => Promise.resolve().then(close)),
|
||||
);
|
||||
const failures = results.flatMap((result) =>
|
||||
result.status === "rejected" ? [result.reason] : [],
|
||||
);
|
||||
if (failures.length === 0) {
|
||||
try {
|
||||
await rm(home, { recursive: true, force: true });
|
||||
} catch (cause) {
|
||||
failures.push(cause);
|
||||
}
|
||||
}
|
||||
if (failures.length > 0)
|
||||
throw new AggregateError(
|
||||
producerFailure === undefined
|
||||
? failures
|
||||
: [producerFailure.cause, ...failures],
|
||||
`Catalog cleanup failed; temporary home retained at ${home}`,
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
const digests = {};
|
||||
for (const profile of ["compact", "full"])
|
||||
digests[profile] = await recordCatalog(profile);
|
||||
await writeFile(
|
||||
join(compiled, "recordedMcpCatalog.js"),
|
||||
`/** Recorded from the compiled runtime; do not edit. */\nexport const RECORDED_MCP_CATALOG_SHA256 = ${JSON.stringify(digests)};\n`,
|
||||
);
|
||||
@@ -3,26 +3,6 @@ import { dirname, join, relative } from "node:path";
|
||||
|
||||
import { exec, execNpm, pathExists } from "./lib/verify-package-core.mjs";
|
||||
|
||||
/** Install the separately locked Skills CLI so verification resolves no registry metadata. */
|
||||
const lockedSkillsCli = async (root, workspace) => {
|
||||
const tool = join(workspace, "skills-cli");
|
||||
await mkdir(tool, { recursive: true });
|
||||
for (const file of ["package.json", "package-lock.json"])
|
||||
await cp(join(root, "scripts/verify-tools/skills", file), join(tool, file));
|
||||
await execNpm(
|
||||
["ci", "--prefer-offline", "--ignore-scripts", "--no-audit", "--no-fund"],
|
||||
{ cwd: tool, timeout: 120_000 },
|
||||
);
|
||||
const installed = join(tool, "node_modules/skills");
|
||||
const manifest = JSON.parse(
|
||||
await readFile(join(installed, "package.json"), "utf8"),
|
||||
);
|
||||
return {
|
||||
version: manifest.version,
|
||||
entry: join(installed, manifest.bin.skills),
|
||||
};
|
||||
};
|
||||
|
||||
/** Exercise Skills' default root discovery without ignored build outputs. */
|
||||
export async function verifyRepositorySkill({ root, workspace }) {
|
||||
const snapshot = join(workspace, "repository");
|
||||
@@ -53,14 +33,15 @@ export async function verifyRepositorySkill({ root, workspace }) {
|
||||
throw new Error("repository skill check must run without generated skills");
|
||||
await mkdir(consumer, { recursive: true });
|
||||
await mkdir(home, { recursive: true });
|
||||
const { version: skillsVersion, entry } = await lockedSkillsCli(
|
||||
root,
|
||||
workspace,
|
||||
);
|
||||
await exec(
|
||||
process.execPath,
|
||||
const skillsVersion = "1.7.2";
|
||||
await execNpm(
|
||||
[
|
||||
entry,
|
||||
"exec",
|
||||
"--yes",
|
||||
"--package",
|
||||
`skills@${skillsVersion}`,
|
||||
"--",
|
||||
"skills",
|
||||
"add",
|
||||
snapshot,
|
||||
"--skill",
|
||||
|
||||
-112
@@ -1,112 +0,0 @@
|
||||
{
|
||||
"name": "rea-verify-skills-cli",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "rea-verify-skills-cli",
|
||||
"dependencies": {
|
||||
"skills": "1.7.2"
|
||||
}
|
||||
},
|
||||
"node_modules/@isaacs/fs-minipass": {
|
||||
"version": "4.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz",
|
||||
"integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"minipass": "^7.0.4"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/chownr": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/chownr/-/chownr-3.0.0.tgz",
|
||||
"integrity": "sha512-+IxzY9BZOQd/XuYPRmrvEVjF/nqj5kgT4kEq7VofrDoM1MxoRjEWkrCC3EtLi59TVawxTAn+orJwFQcrqEN1+g==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/minipass": {
|
||||
"version": "7.1.3",
|
||||
"resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz",
|
||||
"integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": ">=16 || 14 >=14.17"
|
||||
}
|
||||
},
|
||||
"node_modules/minizlib": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/minizlib/-/minizlib-3.1.0.tgz",
|
||||
"integrity": "sha512-KZxYo1BUkWD2TVFLr0MQoM8vUUigWD3LlD83a/75BqC+4qE0Hb1Vo5v1FgcfaNXvfXzr+5EhQ6ing/CaBijTlw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"minipass": "^7.1.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 18"
|
||||
}
|
||||
},
|
||||
"node_modules/skills": {
|
||||
"version": "1.7.2",
|
||||
"resolved": "https://registry.npmjs.org/skills/-/skills-1.7.2.tgz",
|
||||
"integrity": "sha512-j+GzyF/oNahsjjjup6y3lJpFkC1lQjIkf4kAMXICA1hboBqi0fAGXAGoE4x9FRZiP1br/YE9+L3/Tqq6PIAe0A==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"tar": "^7.5.20",
|
||||
"yaml": "^2.8.3"
|
||||
},
|
||||
"bin": {
|
||||
"add-skill": "bin/cli.mjs",
|
||||
"skills": "bin/cli.mjs"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=22.20.0"
|
||||
}
|
||||
},
|
||||
"node_modules/tar": {
|
||||
"version": "7.5.22",
|
||||
"resolved": "https://registry.npmjs.org/tar/-/tar-7.5.22.tgz",
|
||||
"integrity": "sha512-MFO/QzvtAOmJbkhOaCTvbGcFN9L9b+JunIsDwaKljSOdcLMea3NJ1k9Usz/rjdfSXTq4dfzfeS7W4p4YOAAHeA==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"dependencies": {
|
||||
"@isaacs/fs-minipass": "^4.0.0",
|
||||
"chownr": "^3.0.0",
|
||||
"minipass": "^7.1.2",
|
||||
"minizlib": "^3.1.0",
|
||||
"yallist": "^5.0.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/yallist": {
|
||||
"version": "5.0.0",
|
||||
"resolved": "https://registry.npmjs.org/yallist/-/yallist-5.0.0.tgz",
|
||||
"integrity": "sha512-YgvUTfwqyc7UXVMrB+SImsVYSmTS8X/tSrtdNZMImM+n7+QTriRXyXim0mBrTXNeqzVF0KWGgHPeiyViFFrNDw==",
|
||||
"license": "BlueOak-1.0.0",
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/yaml": {
|
||||
"version": "2.9.1",
|
||||
"resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.1.tgz",
|
||||
"integrity": "sha512-3NxN8+78OdzbT7C/WjGsyfPAtJaN3FNDsWxv7Y7mcDsT/oOmgW8BpyQQFFBnvZE3j9Y2Sdz1ULFLezL7Eb2yFw==",
|
||||
"license": "ISC",
|
||||
"bin": {
|
||||
"yaml": "bin.mjs"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">= 14.6"
|
||||
},
|
||||
"funding": {
|
||||
"url": "https://github.com/sponsors/eemeli"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
"name": "rea-verify-skills-cli",
|
||||
"private": true,
|
||||
"description": "Lockfile-pinned Skills CLI for the repository skill check in verify:package.",
|
||||
"dependencies": {
|
||||
"skills": "1.7.2"
|
||||
}
|
||||
}
|
||||
@@ -48,10 +48,6 @@ const { PrivateRuntimeRootUnavailableError } = await import(
|
||||
join(packageRoot, "dist/process/PrivateRuntimeRootUnavailableError.js"),
|
||||
)
|
||||
);
|
||||
const privateRuntimeAclObserver = await readFile(
|
||||
new URL("./lib/windows-private-runtime-acl.ps1", import.meta.url),
|
||||
"utf8",
|
||||
);
|
||||
const workspace = await mkdtemp(join(tmpdir(), "rea-native-conformance-"));
|
||||
// A pwsh caller can supply module paths incompatible with Windows PowerShell.
|
||||
// Each independent observer must discover its own module path.
|
||||
@@ -375,17 +371,35 @@ process.stdout.write(JSON.stringify({executable:basename(process.execPath),abiVe
|
||||
"-Command",
|
||||
String.raw`
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$PSModuleAutoLoadingPreference = 'None'
|
||||
$clock = [Diagnostics.Stopwatch]::StartNew()
|
||||
function Trace-AclPhase([string]$phase) {
|
||||
[Console]::Error.WriteLine('private-runtime-acl: ' + $phase + ' ' + $clock.ElapsedMilliseconds + 'ms')
|
||||
}
|
||||
Trace-AclPhase 'started'
|
||||
$p = $env:REA_TEST_PRIVATE_RUNTIME_PATH
|
||||
$a = ([IO.DirectoryInfo]::new($p)).GetAccessControl()
|
||||
$a = Get-Acl -LiteralPath $p
|
||||
Trace-AclPhase 'acl-read'
|
||||
$u = [Security.Principal.WindowsIdentity]::GetCurrent().User
|
||||
& { ${privateRuntimeAclObserver} } -Acl $a -UserSid $u.Value -Path $p -Clock $clock
|
||||
$owner = $a.GetOwner([Security.Principal.SecurityIdentifier])
|
||||
Trace-AclPhase 'owner-sid-read'
|
||||
# Guard the independent observer against accidental account-name resolution.
|
||||
# Access is a PowerShell convenience property that requests NTAccount rules.
|
||||
Update-TypeData -TypeName ($a.GetType().FullName) -MemberType ScriptProperty -MemberName Access -Value { throw 'ACL observation must use SID rules' } -Force
|
||||
if ($a.PSObject.Properties['Access'].MemberType -ne [Management.Automation.PSMemberTypes]::ScriptProperty) {
|
||||
throw 'ACL observation Access guard was not installed'
|
||||
}
|
||||
$rules = @($a.GetAccessRules($true, $true, [Security.Principal.SecurityIdentifier]))
|
||||
Trace-AclPhase 'rule-sids-read'
|
||||
[IO.File]::SetAttributes($p, [IO.FileAttributes]([int][IO.File]::GetAttributes($p) -bor 1))
|
||||
Trace-AclPhase 'readonly-set'
|
||||
@{
|
||||
protected = $a.AreAccessRulesProtected
|
||||
ownerMatchesUser = ($owner.Value -eq $u.Value)
|
||||
trusteesOnlyUserAndSystem = (@($rules | Where-Object { $_.IdentityReference.Value -ne $u.Value -and $_.IdentityReference.Value -ne 'S-1-5-18' }).Count -eq 0)
|
||||
count = $rules.Count
|
||||
rootReadonly = (([int][IO.File]::GetAttributes($p) -band 1) -ne 0)
|
||||
} | ConvertTo-Json -Compress
|
||||
Trace-AclPhase 'complete'
|
||||
`,
|
||||
],
|
||||
{
|
||||
|
||||
@@ -10,7 +10,6 @@ import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
|
||||
import {
|
||||
artifactCli,
|
||||
artifactCliEvidence,
|
||||
artifactMcpResult,
|
||||
withArtifactMcp,
|
||||
@@ -26,56 +25,6 @@ const nibPath = join(resourcesPath, "Main.nib");
|
||||
const sourceRoot = fileURLToPath(
|
||||
new URL("../../../tests/conformance/interface-builder/", import.meta.url),
|
||||
);
|
||||
const mainSource = join(sourceRoot, "Main.xib");
|
||||
// ibtool property names mapped to the keys AppKit archives for NSControl.
|
||||
const buttonBooleanKeys = {
|
||||
continuous: "NSControlContinuous",
|
||||
refusesFirstResponder: "NSControlRefusesFirstResponder",
|
||||
};
|
||||
const plistDocument = (body) =>
|
||||
`<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">${body}</plist>\n`;
|
||||
const compileNib = (source) =>
|
||||
exec("xcrun", [
|
||||
"ibtool",
|
||||
"--errors",
|
||||
"--warnings",
|
||||
"--compile",
|
||||
nibPath,
|
||||
source,
|
||||
]);
|
||||
/** Apple's source-level property values: the oracle for compiled booleans. */
|
||||
const exportedButtonBooleans = async (source) => {
|
||||
const exported = join(fixtureRoot, "export.plist");
|
||||
const { stdout } = await exec("xcrun", [
|
||||
"ibtool",
|
||||
"--export",
|
||||
join(fixtureRoot, "export-request.plist"),
|
||||
source,
|
||||
]);
|
||||
await writeFile(exported, stdout);
|
||||
const { stdout: json } = await exec("plutil", [
|
||||
"-convert",
|
||||
"json",
|
||||
"-o",
|
||||
"-",
|
||||
exported,
|
||||
]);
|
||||
return JSON.parse(json)["com.apple.ibtool.document.export"].button;
|
||||
};
|
||||
const decodedButtonBooleans = (result) => {
|
||||
const button = result.graph.nodes.find(
|
||||
(node) => node.kind === "control" && node.name === "NSButton",
|
||||
);
|
||||
assert.ok(button, "compiled button control was not decoded");
|
||||
return Object.fromEntries(
|
||||
Object.entries(buttonBooleanKeys).map(([property, key]) => [
|
||||
property,
|
||||
button.attributes.nibValues?.[key],
|
||||
]),
|
||||
);
|
||||
};
|
||||
|
||||
let report;
|
||||
try {
|
||||
@@ -87,7 +36,14 @@ try {
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict><key>CFBundleExecutable</key><string>Fixture</string><key>CFBundleIdentifier</key><string>dev.rea.interface-builder-fixture</string></dict></plist>\n`,
|
||||
);
|
||||
await compileNib(mainSource);
|
||||
await exec("xcrun", [
|
||||
"ibtool",
|
||||
"--errors",
|
||||
"--warnings",
|
||||
"--compile",
|
||||
nibPath,
|
||||
join(sourceRoot, "Main.xib"),
|
||||
]);
|
||||
await exec("cc", [join(sourceRoot, "fixture.c"), "-o", executablePath]);
|
||||
|
||||
const evidence = await artifactCliEvidence(
|
||||
@@ -154,53 +110,6 @@ try {
|
||||
);
|
||||
assert.equal(decoded.graph.truncated, false);
|
||||
|
||||
// Compiled booleans must keep their source polarity in both directions.
|
||||
const properties = Object.keys(buttonBooleanKeys);
|
||||
await writeFile(
|
||||
join(fixtureRoot, "export-request.plist"),
|
||||
plistDocument(
|
||||
`<dict><key>NSControl</key><array>${properties.map((property) => `<string>${property}</string>`).join("")}</array></dict>`,
|
||||
),
|
||||
);
|
||||
await writeFile(
|
||||
join(fixtureRoot, "toggle.plist"),
|
||||
plistDocument(
|
||||
`<dict><key>button</key><dict>${properties.map((property) => `<key>${property}</key><true/>`).join("")}</dict></dict>`,
|
||||
),
|
||||
);
|
||||
const toggledSource = join(fixtureRoot, "Toggled.xib");
|
||||
await exec("xcrun", [
|
||||
"ibtool",
|
||||
"--import",
|
||||
join(fixtureRoot, "toggle.plist"),
|
||||
"--write",
|
||||
toggledSource,
|
||||
mainSource,
|
||||
]);
|
||||
const mainBooleans = await exportedButtonBooleans(mainSource);
|
||||
const toggledBooleans = await exportedButtonBooleans(toggledSource);
|
||||
assert.deepEqual(mainBooleans, {
|
||||
continuous: false,
|
||||
refusesFirstResponder: false,
|
||||
});
|
||||
assert.deepEqual(toggledBooleans, {
|
||||
continuous: true,
|
||||
refusesFirstResponder: true,
|
||||
});
|
||||
assert.deepEqual(
|
||||
decodedButtonBooleans(decoded),
|
||||
mainBooleans,
|
||||
"compiled false booleans differ from the source export",
|
||||
);
|
||||
await compileNib(toggledSource);
|
||||
assert.deepEqual(
|
||||
decodedButtonBooleans(
|
||||
await artifactCli("decode-interface-builder", appPath),
|
||||
),
|
||||
toggledBooleans,
|
||||
"compiled true booleans differ from the source export",
|
||||
);
|
||||
|
||||
report = {
|
||||
ok: true,
|
||||
fixture: "compiled-appkit-nib",
|
||||
@@ -213,7 +122,6 @@ try {
|
||||
connections: decoded.documents[0].connection_count,
|
||||
recovered_control: button.name,
|
||||
recovered_action: action.name,
|
||||
button_booleans: { main: mainBooleans, toggled: toggledBooleans },
|
||||
hierarchy_complete: decoded.documents[0].hierarchy_complete,
|
||||
truncated: decoded.graph.truncated,
|
||||
};
|
||||
|
||||
@@ -4,14 +4,14 @@ import { jsonValueSchema } from "../domain/jsonValue.js";
|
||||
import type { AndroidOperation } from "../domain/android/androidAnalysis.js";
|
||||
import { dexDescriptorSchema } from "../domain/android/androidAnalysis.js";
|
||||
|
||||
const count = z.int().nonnegative();
|
||||
const count = z.number().int().nonnegative();
|
||||
/** Typed allocation failures authored before upstream erases the JVM exception. */
|
||||
const jadxMemoryFailureSchema = z.strictObject({
|
||||
kind: z.literal("jvm-memory-failure"),
|
||||
memory_area: z.enum(["heap", "unknown"]),
|
||||
exception_type: z.literal("java.lang.OutOfMemoryError"),
|
||||
message: z.string().optional(),
|
||||
max_heap_bytes: z.int().positive().safe(),
|
||||
max_heap_bytes: z.number().int().positive().safe(),
|
||||
});
|
||||
/** Selector failures authored by REA's bridge, separate from arbitrary engine errors. */
|
||||
const jadxInputFailureSchema = z.strictObject({
|
||||
@@ -40,8 +40,8 @@ export const parseJadxFailure = (
|
||||
/** Values observed inside the exact JVM producing the analysis. */
|
||||
export const jadxRuntimeSchema = z.object({
|
||||
engine_reported_version: z.string().min(1),
|
||||
max_heap_bytes: z.int().positive().safe(),
|
||||
available_processors: z.int().positive(),
|
||||
max_heap_bytes: z.number().int().positive().safe(),
|
||||
available_processors: z.number().int().positive(),
|
||||
java_version: z.string().min(1),
|
||||
metadata_scope: z.literal("parsed_members"),
|
||||
});
|
||||
@@ -81,11 +81,11 @@ export const jadxClassSchema = z.object({
|
||||
signature: z.string(),
|
||||
dex_descriptor: dexDescriptorSchema,
|
||||
is_constructor: z.boolean(),
|
||||
def_pos: z.int(),
|
||||
def_pos: z.number().int(),
|
||||
}),
|
||||
),
|
||||
fields: z.array(
|
||||
z.object({ name: z.string(), type: z.string(), def_pos: z.int() }),
|
||||
z.object({ name: z.string(), type: z.string(), def_pos: z.number().int() }),
|
||||
),
|
||||
inner_classes: z.array(z.string()),
|
||||
});
|
||||
@@ -115,7 +115,7 @@ export const jadxXrefsSchema = z.object({
|
||||
full_name: z.string(),
|
||||
containing_class: z.string(),
|
||||
top_class: z.string(),
|
||||
def_pos: z.int(),
|
||||
def_pos: z.number().int(),
|
||||
direct_invokes_exhaustive: z.boolean().optional(),
|
||||
direct_invokes: z
|
||||
.array(
|
||||
|
||||
@@ -350,7 +350,6 @@ export const compactToolAvailability = (
|
||||
const availabilityFor = (context: AvailabilityContext): Availability => {
|
||||
// These built-in workflows accept a caller-selected endpoint; they do not load the optional passive provider.
|
||||
if (
|
||||
context.name === "inspect_byte_entropy" ||
|
||||
context.name === "observe_web_execution" ||
|
||||
context.name === "inspect_web_event_listeners"
|
||||
)
|
||||
@@ -500,8 +499,6 @@ const workflowAvailabilityFor = ({
|
||||
};
|
||||
if (name === "inspect_pe_resources")
|
||||
return { reason: "available", remediation: null };
|
||||
if (name === "inspect_elf_notes")
|
||||
return { reason: "available", remediation: null };
|
||||
if (name === "inspect_evm_interface") {
|
||||
if (policy.evmInterfaceEnabled !== true)
|
||||
return {
|
||||
|
||||
@@ -198,13 +198,12 @@ export class EnhancedTools {
|
||||
): EnhancedResult {
|
||||
switch (call.name) {
|
||||
case "inspect_native_dispatch_metadata":
|
||||
return inspectNativeDispatch(this.analysis, call.input.max_records, {
|
||||
return this.#inspectNativeDispatchMetadata(
|
||||
call.input.max_records,
|
||||
signal,
|
||||
objcClassImportAddresses: call.input.objc_class_import_addresses,
|
||||
objcClassImportAddressesComplete:
|
||||
call.input.objc_class_import_addresses_complete,
|
||||
maxInputBytes: call.input.max_input_bytes,
|
||||
});
|
||||
call.input.objc_class_import_addresses,
|
||||
call.input.objc_class_import_addresses_complete,
|
||||
);
|
||||
case "get_objc_classes":
|
||||
return this.#objcClasses(call.input.pattern, signal);
|
||||
case "get_objc_protocols":
|
||||
@@ -421,6 +420,21 @@ export class EnhancedTools {
|
||||
);
|
||||
}
|
||||
|
||||
#inspectNativeDispatchMetadata(
|
||||
maxRecords: number,
|
||||
signal?: AbortSignal,
|
||||
objcClassImportAddresses?: readonly string[],
|
||||
objcClassImportAddressesComplete?: boolean,
|
||||
): EnhancedResult {
|
||||
return inspectNativeDispatch(
|
||||
this.analysis,
|
||||
maxRecords,
|
||||
signal,
|
||||
objcClassImportAddresses,
|
||||
objcClassImportAddressesComplete,
|
||||
);
|
||||
}
|
||||
|
||||
async #objcProtocols(signal?: AbortSignal): EnhancedResult {
|
||||
const names = await this.#allAddressed("list_names", signal);
|
||||
return names.ok ? ok(discoverObjcProtocols(names.value)) : names;
|
||||
|
||||
@@ -98,60 +98,26 @@ export const deliverAnalysisEvidence = async (
|
||||
)
|
||||
: ok(evidence);
|
||||
if (!delivered.ok) return delivered;
|
||||
const published = await publishEvidenceDelivery({
|
||||
bundle,
|
||||
complete: evidence,
|
||||
delivered: delivered.value,
|
||||
outputPath,
|
||||
operation: evidence.operation,
|
||||
signal,
|
||||
});
|
||||
return published.ok ? ok(published.value.delivered) : published;
|
||||
};
|
||||
|
||||
/** Selected Evidence delivered after its complete bundle was committed. */
|
||||
export interface PublishedEvidenceDelivery {
|
||||
readonly delivered: Evidence;
|
||||
readonly evidenceBundle: PublishedEvidenceFile;
|
||||
}
|
||||
|
||||
/**
|
||||
* Publish a complete bundle, then deliver Evidence already projected from it.
|
||||
* Cancellation observed before commit leaves no output and retains the
|
||||
* complete Evidence. A committed publication wins a later cancellation race;
|
||||
* its receipt stays in the typed cancellation as diagnostics.evidence_bundle,
|
||||
* and a cleanup failure after commit keeps its own published receipt.
|
||||
*/
|
||||
export const publishEvidenceDelivery = async (input: {
|
||||
readonly bundle: EvidenceBundle;
|
||||
readonly complete: Evidence;
|
||||
readonly delivered: Evidence;
|
||||
readonly outputPath: string;
|
||||
readonly operation: string;
|
||||
readonly signal?: AbortSignal | undefined;
|
||||
}): Promise<Result<PublishedEvidenceDelivery, AnalysisError>> => {
|
||||
const { signal, operation } = input;
|
||||
const written = await persistEvidenceBundleCommand(
|
||||
input.bundle,
|
||||
input.outputPath,
|
||||
bundle,
|
||||
outputPath,
|
||||
false,
|
||||
signal,
|
||||
operation,
|
||||
evidence.operation,
|
||||
);
|
||||
// Deliver interrupts queued during final publication cleanup before the
|
||||
// command's signal owner accepts completion and removes its listeners.
|
||||
await dispatchQueuedInterrupts();
|
||||
if (!written.ok)
|
||||
return err(written.error.retainPartialObservation(input.complete));
|
||||
if (!written.ok) return err(written.error.retainPartialObservation(evidence));
|
||||
if (signal?.aborted)
|
||||
return err(
|
||||
new AnalysisCancelledError(operation, {
|
||||
new AnalysisCancelledError(evidence.operation, {
|
||||
cause: signal.reason,
|
||||
partialObservation: input.delivered,
|
||||
partialObservation: delivered.value,
|
||||
diagnostics: { evidence_bundle: { ...written.value } },
|
||||
}),
|
||||
);
|
||||
return ok({ delivered: input.delivered, evidenceBundle: written.value });
|
||||
return delivered;
|
||||
};
|
||||
|
||||
/** Resolve the caller's exact retained parent from an authenticated portable bundle. */
|
||||
|
||||
@@ -48,20 +48,6 @@ export const PE_RESOURCES_PROVIDER = Object.freeze({
|
||||
version: "1",
|
||||
} as const);
|
||||
|
||||
/** Provider identity for portable, execution-free ELF note-table decoding. */
|
||||
export const ELF_NOTES_PROVIDER = Object.freeze({
|
||||
id: "rea-elf-notes",
|
||||
name: "REA ELF note parser",
|
||||
version: "1",
|
||||
} as const);
|
||||
|
||||
/** Provider identity for execution-free Shannon byte-entropy profiling. */
|
||||
export const BYTE_ENTROPY_PROVIDER = Object.freeze({
|
||||
id: "byte-entropy",
|
||||
name: "REA byte entropy",
|
||||
version: "1",
|
||||
} as const);
|
||||
|
||||
/** Provider identity for deterministic managed-code comparison workflows. */
|
||||
export const MANAGED_WORKFLOW_PROVIDER = {
|
||||
id: "rea-dotnet-workflows",
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { uniqueSorted, uniqueSortedBy } from "../domain/canonicalOrdering.js";
|
||||
import { digestCanonicalValue } from "../domain/canonicalDigest.js";
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
import canonicalize from "canonicalize";
|
||||
|
||||
import type { Evidence } from "../domain/evidence.js";
|
||||
import {
|
||||
@@ -406,12 +407,12 @@ const addCandidate = (
|
||||
}
|
||||
candidates.set(candidate.obligation_id, {
|
||||
...current,
|
||||
authority_references: uniqueSortedBy(
|
||||
authority_references: uniqueBy(
|
||||
[...current.authority_references, ...candidate.authority_references],
|
||||
(reference) =>
|
||||
`${reference.evidence_id}:${reference.state}:${reference.location}`,
|
||||
),
|
||||
observed_cases: uniqueSortedBy(
|
||||
observed_cases: uniqueBy(
|
||||
[...current.observed_cases, ...candidate.observed_cases],
|
||||
(item) => `${item.kind}:${item.evidence_id}:${item.location}`,
|
||||
),
|
||||
@@ -420,14 +421,14 @@ const addCandidate = (
|
||||
...candidate.authority_references,
|
||||
]),
|
||||
// Each observation keeps its own unresolved authority and next steps.
|
||||
unavailable_authority: uniqueSorted([
|
||||
...current.unavailable_authority,
|
||||
...candidate.unavailable_authority,
|
||||
]),
|
||||
required_next_evidence: uniqueSorted([
|
||||
...current.required_next_evidence,
|
||||
...candidate.required_next_evidence,
|
||||
]),
|
||||
unavailable_authority: uniqueBy(
|
||||
[...current.unavailable_authority, ...candidate.unavailable_authority],
|
||||
(item) => item,
|
||||
),
|
||||
required_next_evidence: uniqueBy(
|
||||
[...current.required_next_evidence, ...candidate.required_next_evidence],
|
||||
(item) => item,
|
||||
),
|
||||
// A generated ID digests its derivation key, so two generated candidates
|
||||
// with one ID are the same claim seen by several records. Only another
|
||||
// reviewed definition can introduce a competing definition.
|
||||
@@ -447,5 +448,20 @@ const sourceState = (
|
||||
return "unknown";
|
||||
};
|
||||
|
||||
const uniqueBy = <Value>(
|
||||
values: readonly Value[],
|
||||
key: (value: Value) => string,
|
||||
): Value[] =>
|
||||
[...new Map(values.map((value) => [key(value), value])).values()].sort(
|
||||
(left, right) => compareUnicodeCodePoints(key(left), key(right)),
|
||||
);
|
||||
|
||||
const obligationId = (key: string): string =>
|
||||
`obl_${digestCanonicalValue({ schema: "ReconstructionObligation", key }, "Reconstruction obligation key")}`;
|
||||
`obl_${digest({ schema: "ReconstructionObligation", key })}`;
|
||||
|
||||
const digest = (value: unknown): string => {
|
||||
const encoded = canonicalize(value);
|
||||
if (encoded === undefined)
|
||||
throw new TypeError("Reconstruction obligation key is not canonical JSON");
|
||||
return createHash("sha256").update(encoded).digest("hex");
|
||||
};
|
||||
@@ -9,7 +9,6 @@ import {
|
||||
RegularFileCleanupFailure,
|
||||
retryRegularFileCleanup,
|
||||
} from "./RegularFileRead.js";
|
||||
import { referenceSourceResourceFailure } from "../reference/ReferenceSourceReaderErrors.js";
|
||||
import type { ReferenceSourceReaderError } from "../reference/ReferenceSourceReaderTypes.js";
|
||||
|
||||
export type ReferenceSourceVcsInfo =
|
||||
@@ -60,7 +59,13 @@ export const readReferenceSourceVcs = async (
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof ArtifactReaderFailure)) throw cause;
|
||||
return err(referenceSourceResourceFailure(cause));
|
||||
return err({
|
||||
tag: "reference-source-reader",
|
||||
code: cause.reason === "cancelled" ? "cancelled" : "io",
|
||||
message: cause.message,
|
||||
...(cause.cleanup === undefined ? {} : { cleanup: cause.cleanup }),
|
||||
cause,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
|
||||
@@ -1,373 +0,0 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { open, stat, rename, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect, it, vi } from "vitest";
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
import { byteEntropySchema } from "../../domain/native/byteEntropy.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import {
|
||||
ByteEntropyService,
|
||||
MAX_BYTE_ENTROPY_BLOCKS,
|
||||
} from "./ByteEntropyService.js";
|
||||
|
||||
it("keeps whole-file identity and entropy while selecting unaligned blocks and a partial tail", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-");
|
||||
const path = join(root, "bytes");
|
||||
const bytes = Buffer.from(Array.from({ length: 1024 }, (_, i) => i % 256));
|
||||
await writeFile(path, bytes);
|
||||
const result = await new ByteEntropyService().inspect({
|
||||
path,
|
||||
offset: 3,
|
||||
length: 513,
|
||||
block_size: 256,
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
const report = byteEntropySchema.parse(result.value.normalized_result);
|
||||
expect(report.blocks).toEqual([
|
||||
{ offset: 3, length: 256, entropy: 8 },
|
||||
{ offset: 259, length: 256, entropy: 8 },
|
||||
{ offset: 515, length: 1, entropy: 0 },
|
||||
]);
|
||||
expect(report.file_entropy).toBe(8);
|
||||
expect(report.artifact).toEqual({
|
||||
path,
|
||||
bytes: 1024,
|
||||
sha256: createHash("sha256").update(bytes).digest("hex"),
|
||||
});
|
||||
expect(result.value.subject?.digest.sha256).toBe(report.artifact.sha256);
|
||||
expect(result.value.confidence).toBe("observed");
|
||||
});
|
||||
|
||||
it("rejects missing files, non-files and windows beyond EOF, but accepts empty windows", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-input-");
|
||||
const path = join(root, "empty");
|
||||
await writeFile(path, "");
|
||||
const service = new ByteEntropyService();
|
||||
for (const input of [
|
||||
{ path: join(root, "missing") },
|
||||
{ path: root },
|
||||
{ path, offset: 1 },
|
||||
{ path, length: 1 },
|
||||
{ path: "relative" },
|
||||
]) {
|
||||
const result = await service.inspect(input);
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisInputError" },
|
||||
});
|
||||
}
|
||||
const result = await service.inspect({ path });
|
||||
if (!result.ok) throw result.error;
|
||||
expect(result.value.normalized_result).toMatchObject({
|
||||
blocks: [],
|
||||
runs: [],
|
||||
file_entropy: 0,
|
||||
window: { offset: 0, length: 0 },
|
||||
});
|
||||
expect(
|
||||
byteEntropySchema.parse(result.value.normalized_result).limitations,
|
||||
).toContainEqual(expect.stringContaining("reports a size of 0"));
|
||||
});
|
||||
|
||||
it.skipIf(process.platform !== "linux")(
|
||||
"marks a generated /proc file that reports size 0 as not shown empty",
|
||||
async () => {
|
||||
const result = await new ByteEntropyService().inspect({
|
||||
path: "/proc/version",
|
||||
});
|
||||
if (!result.ok) throw result.error;
|
||||
const report = byteEntropySchema.parse(result.value.normalized_result);
|
||||
expect(report.artifact.bytes).toBe(0);
|
||||
expect(report.limitations).toContainEqual(
|
||||
expect.stringContaining("Linux /proc entries"),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("adds no zero-size limitation to a file with content", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-nonempty-");
|
||||
const path = join(root, "one");
|
||||
await writeFile(path, "x");
|
||||
const result = await new ByteEntropyService().inspect({ path });
|
||||
if (!result.ok) throw result.error;
|
||||
expect(
|
||||
byteEntropySchema
|
||||
.parse(result.value.normalized_result)
|
||||
.limitations.join(" "),
|
||||
).not.toContain("reports a size of 0");
|
||||
});
|
||||
|
||||
it("honors cancellation before and during streaming", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-cancel-");
|
||||
const path = join(root, "bytes");
|
||||
await writeFile(path, Buffer.alloc(1024 * 1024));
|
||||
const service = new ByteEntropyService();
|
||||
expect(
|
||||
await service.inspect({ path }, { signal: AbortSignal.abort() }),
|
||||
).toMatchObject({ ok: false, error: { _tag: "AnalysisCancelledError" } });
|
||||
const controller = new AbortController();
|
||||
let reads = 0;
|
||||
const streaming = new ByteEntropyService({
|
||||
stat,
|
||||
open: async (...args) => {
|
||||
const file = await open(...args);
|
||||
const read = file.read.bind(file);
|
||||
vi.spyOn(file, "read").mockImplementation(async (...readArgs) => {
|
||||
const result = await read(...readArgs);
|
||||
reads += 1;
|
||||
controller.abort();
|
||||
return result;
|
||||
});
|
||||
return file;
|
||||
},
|
||||
});
|
||||
const pending = streaming.inspect(
|
||||
{ path, block_size: 256 },
|
||||
{ signal: controller.signal },
|
||||
);
|
||||
expect(await pending).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisCancelledError" },
|
||||
});
|
||||
expect(reads).toBe(1);
|
||||
});
|
||||
|
||||
it("maps cancellation during open to the cancelled algebra with its original cause", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-open-cancel-");
|
||||
const path = join(root, "bytes");
|
||||
await writeFile(path, Buffer.alloc(512));
|
||||
const controller = new AbortController();
|
||||
const reason = new Error("cancelled while opening");
|
||||
const service = new ByteEntropyService({
|
||||
stat,
|
||||
open: async (...args) => {
|
||||
const file = await open(...args);
|
||||
controller.abort(reason);
|
||||
return file;
|
||||
},
|
||||
});
|
||||
|
||||
const result = await service.inspect(
|
||||
{ path, block_size: 256 },
|
||||
{ signal: controller.signal },
|
||||
);
|
||||
|
||||
if (result.ok) throw new Error("Expected cancellation during open");
|
||||
expect(result.error).toMatchObject({ _tag: "AnalysisCancelledError" });
|
||||
expect(result.error.cause).toBe(reason);
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "cancelled",
|
||||
});
|
||||
await service.close();
|
||||
});
|
||||
|
||||
it("keeps finalizer cancellation primary when close also fails, then resolves its cleanup owner", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-finalizer-cancel-");
|
||||
const path = join(root, "bytes");
|
||||
await writeFile(path, Buffer.alloc(512));
|
||||
const controller = new AbortController();
|
||||
const reason = new Error("cancelled during close");
|
||||
const closeFailure = new Error("injected close failure");
|
||||
let failClose = true;
|
||||
const service = new ByteEntropyService({
|
||||
stat,
|
||||
open: async (...args) => {
|
||||
const file = await open(...args);
|
||||
const close = file.close.bind(file);
|
||||
vi.spyOn(file, "close").mockImplementation(async () => {
|
||||
if (failClose) {
|
||||
controller.abort(reason);
|
||||
throw closeFailure;
|
||||
}
|
||||
await close();
|
||||
});
|
||||
return file;
|
||||
},
|
||||
});
|
||||
|
||||
const result = await service.inspect(
|
||||
{ path, block_size: 256 },
|
||||
{ signal: controller.signal },
|
||||
);
|
||||
|
||||
if (result.ok) throw new Error("Expected cancellation during finalization");
|
||||
expect(result.error).toMatchObject({
|
||||
_tag: "AnalysisCancelledError",
|
||||
cleanupIncomplete: true,
|
||||
secondaryFailures: [
|
||||
{ _tag: "AnalysisOutputError", cleanupIncomplete: true },
|
||||
],
|
||||
});
|
||||
expect(result.error.cause).toBe(reason);
|
||||
expect(result.error.secondaryFailures[0]?.cause).toBeDefined();
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "cleanup_incomplete",
|
||||
retryable: false,
|
||||
details: {
|
||||
cleanup: "incomplete",
|
||||
execution_failure: "cancelled",
|
||||
},
|
||||
});
|
||||
|
||||
failClose = false;
|
||||
await service.close();
|
||||
expect(result.error.cleanupIncomplete).toBe(false);
|
||||
});
|
||||
|
||||
it("reports a changed artifact when its parent directory moves during streaming", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-moved-");
|
||||
const path = join(root, "bytes");
|
||||
await writeFile(path, Buffer.alloc(512));
|
||||
let moved = false;
|
||||
const service = new ByteEntropyService({
|
||||
stat,
|
||||
open: async (...args) => {
|
||||
const file = await open(...args);
|
||||
const read = file.read.bind(file);
|
||||
vi.spyOn(file, "read").mockImplementation(async (...readArgs) => {
|
||||
const result = await read(...readArgs);
|
||||
if (!moved) {
|
||||
moved = true;
|
||||
await rename(root, `${root}-moved`);
|
||||
}
|
||||
return result;
|
||||
});
|
||||
return file;
|
||||
},
|
||||
});
|
||||
try {
|
||||
expect(await service.inspect({ path, block_size: 256 })).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisArtifactChangedError" },
|
||||
});
|
||||
} finally {
|
||||
if (moved) await rename(`${root}-moved`, root);
|
||||
}
|
||||
});
|
||||
|
||||
it("refuses a block series above the output-row limit before scanning", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-block-limit-");
|
||||
const path = join(root, "bytes");
|
||||
await writeFile(path, Buffer.alloc(MAX_BYTE_ENTROPY_BLOCKS * 256 + 1));
|
||||
let reads = 0;
|
||||
const service = new ByteEntropyService({
|
||||
stat,
|
||||
open: async (...args) => {
|
||||
const file = await open(...args);
|
||||
vi.spyOn(file, "read").mockImplementation(async () => {
|
||||
reads += 1;
|
||||
return { bytesRead: 0, buffer: Buffer.alloc(0) };
|
||||
});
|
||||
return file;
|
||||
},
|
||||
});
|
||||
|
||||
const result = await service.inspect({ path, block_size: 256 });
|
||||
|
||||
expect(result).toMatchObject({
|
||||
ok: false,
|
||||
error: {
|
||||
_tag: "AnalysisResourceConstraintError",
|
||||
resource: "capacity",
|
||||
reportedLimits: {
|
||||
max_blocks: MAX_BYTE_ENTROPY_BLOCKS,
|
||||
requested_blocks: MAX_BYTE_ENTROPY_BLOCKS + 1,
|
||||
},
|
||||
},
|
||||
});
|
||||
if (result.ok) throw new Error("Expected a capacity refusal");
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "resource_constraint",
|
||||
retryable: false,
|
||||
remediation: {
|
||||
action: expect.stringMatching(/shorter window|increase block_size/i),
|
||||
},
|
||||
});
|
||||
expect(reads).toBe(0);
|
||||
await service.close();
|
||||
});
|
||||
|
||||
it("retains a failed descriptor close and retries it before opening another file", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-close-retry-");
|
||||
const path = join(root, "bytes");
|
||||
await writeFile(path, Buffer.alloc(512));
|
||||
let failClose = true;
|
||||
let opens = 0;
|
||||
const service = new ByteEntropyService({
|
||||
stat,
|
||||
open: async (...args) => {
|
||||
opens += 1;
|
||||
const file = await open(...args);
|
||||
const close = file.close.bind(file);
|
||||
vi.spyOn(file, "close").mockImplementation(async () => {
|
||||
if (failClose) throw new Error("injected close failure");
|
||||
await close();
|
||||
});
|
||||
return file;
|
||||
},
|
||||
});
|
||||
|
||||
const first = await service.inspect({ path, block_size: 256 });
|
||||
if (first.ok) throw new Error("Expected the first close to fail");
|
||||
expect(first.error).toMatchObject({
|
||||
_tag: "AnalysisOutputError",
|
||||
cleanupIncomplete: true,
|
||||
});
|
||||
expect(opens).toBe(1);
|
||||
|
||||
const retryDenied = await service.inspect({ path, block_size: 256 });
|
||||
expect(retryDenied).toMatchObject({
|
||||
ok: false,
|
||||
error: { _tag: "AnalysisOutputError", cleanupIncomplete: true },
|
||||
});
|
||||
expect(opens).toBe(1);
|
||||
|
||||
failClose = false;
|
||||
const recovered = await service.inspect({ path, block_size: 256 });
|
||||
expect(recovered.ok).toBe(true);
|
||||
expect(first.error.cleanupIncomplete).toBe(false);
|
||||
expect(opens).toBe(2);
|
||||
await service.close();
|
||||
});
|
||||
|
||||
it("preserves a primary read failure through failed close and resolves it on retry", async () => {
|
||||
const root = await createTestTempDirectory("rea-entropy-primary-close-");
|
||||
const path = join(root, "bytes");
|
||||
await writeFile(path, Buffer.alloc(512));
|
||||
const readFailure = new Error("injected read failure");
|
||||
let failClose = true;
|
||||
let failRead = true;
|
||||
const service = new ByteEntropyService({
|
||||
stat,
|
||||
open: async (...args) => {
|
||||
const file = await open(...args);
|
||||
const read = file.read.bind(file);
|
||||
const close = file.close.bind(file);
|
||||
vi.spyOn(file, "read").mockImplementation(async (...readArgs) => {
|
||||
if (failRead) throw readFailure;
|
||||
return await read(...readArgs);
|
||||
});
|
||||
vi.spyOn(file, "close").mockImplementation(async () => {
|
||||
if (failClose) throw new Error("injected close failure");
|
||||
await close();
|
||||
});
|
||||
return file;
|
||||
},
|
||||
});
|
||||
|
||||
const first = await service.inspect({ path, block_size: 256 });
|
||||
if (first.ok) throw new Error("Expected the read and close to fail");
|
||||
expect(first.error).toMatchObject({
|
||||
cleanupIncomplete: true,
|
||||
cleanupPrimary: { _tag: "AnalysisOutputError" },
|
||||
});
|
||||
expect(first.error.cleanupPrimary?.cause).toBe(readFailure);
|
||||
expect(first.error.cause).toBeDefined();
|
||||
|
||||
failClose = false;
|
||||
failRead = false;
|
||||
const recovered = await service.inspect({ path, block_size: 256 });
|
||||
expect(recovered.ok).toBe(true);
|
||||
expect(first.error.cleanupIncomplete).toBe(false);
|
||||
await service.close();
|
||||
});
|
||||
@@ -1,355 +0,0 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { constants } from "node:fs";
|
||||
import { open, stat } from "node:fs/promises";
|
||||
import { isAbsolute } from "node:path";
|
||||
import type { ExecutionOptions } from "../AnalysisProvider.js";
|
||||
import { BYTE_ENTROPY_PROVIDER } from "../InvestigationProviders.js";
|
||||
import { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import {
|
||||
AnalysisOutputError,
|
||||
AnalysisAccessDeniedError,
|
||||
AnalysisArtifactChangedError,
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import { createEvidence, type Evidence } from "../../domain/evidence.js";
|
||||
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
|
||||
import {
|
||||
computeBlockEntropy,
|
||||
entropyFromCounts,
|
||||
inspectByteEntropyInputSchema,
|
||||
mergeEntropyRuns,
|
||||
type ByteEntropy,
|
||||
} from "../../domain/native/byteEntropy.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import { ArtifactResourceScope } from "../../artifacts/ArtifactResourceScope.js";
|
||||
import {
|
||||
withArtifactFileHandle,
|
||||
bindArtifactFileCleanup,
|
||||
ArtifactFileCleanupFailure,
|
||||
artifactFileCancellationCause,
|
||||
} from "../../artifacts/ArtifactFileOperation.js";
|
||||
import { analysisErrorWithCleanupFailure } from "../../domain/analysisErrorCleanup.js";
|
||||
import { ArtifactReaderFailure } from "../../artifacts/ArtifactReader.js";
|
||||
|
||||
/** Bound retained result rows before constructing the Evidence representation. */
|
||||
export const MAX_BYTE_ENTROPY_BLOCKS = 32_768;
|
||||
|
||||
/** Generated files such as Linux /proc entries report size 0 but still hold bytes. */
|
||||
const ZERO_SIZE_LIMITATION =
|
||||
"The file reports a size of 0, so no bytes were read. Generated files such as Linux /proc entries can report 0 while holding content; this result does not show that the file is empty.";
|
||||
|
||||
const OPERATION = "inspect_byte_entropy";
|
||||
const invalid = (field: string, message: string) =>
|
||||
new AnalysisInputError(OPERATION, undefined, [
|
||||
{ path: [field], reason: "invalid_value", message },
|
||||
]);
|
||||
|
||||
/** Offline byte observations shared by CLI and MCP, with one file-wide read. */
|
||||
export class ByteEntropyService {
|
||||
readonly #resources: ArtifactResourceScope;
|
||||
|
||||
constructor(
|
||||
private readonly files: Pick<
|
||||
typeof import("node:fs/promises"),
|
||||
"open" | "stat"
|
||||
> = { open, stat },
|
||||
resources: ArtifactResourceScope = new ArtifactResourceScope(),
|
||||
) {
|
||||
this.#resources = resources;
|
||||
}
|
||||
|
||||
async inspect(
|
||||
rawInput: unknown,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<Evidence, AnalysisError>> {
|
||||
try {
|
||||
return await this.#resources.run(() => this.#inspect(rawInput, options));
|
||||
} catch (cause: unknown) {
|
||||
const failure = new AnalysisOutputError(
|
||||
OPERATION,
|
||||
"A retained byte-entropy file descriptor could not be released before inspection.",
|
||||
{
|
||||
cause,
|
||||
...(cause instanceof ArtifactReaderFailure &&
|
||||
cause.cleanup !== undefined
|
||||
? { cleanup: cause.cleanup }
|
||||
: {}),
|
||||
},
|
||||
);
|
||||
bindArtifactFileCleanup(cause, failure);
|
||||
return err(failure);
|
||||
}
|
||||
}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.#resources.close();
|
||||
}
|
||||
|
||||
async #inspect(
|
||||
rawInput: unknown,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<Evidence, AnalysisError>> {
|
||||
if (options?.signal?.aborted)
|
||||
return err(new AnalysisCancelledError(OPERATION));
|
||||
const parsed = inspectByteEntropyInputSchema.safeParse(rawInput);
|
||||
if (!parsed.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(OPERATION, parsed.error.issues, rawInput, {
|
||||
cause: parsed.error,
|
||||
}),
|
||||
);
|
||||
const input = parsed.data;
|
||||
if (!isAbsolute(input.path))
|
||||
return err(
|
||||
invalid("path", "Expected an absolute filesystem path on this host."),
|
||||
);
|
||||
let admitted = false;
|
||||
try {
|
||||
const before = await this.files.stat(input.path, { bigint: true });
|
||||
if (!before.isFile()) throw invalid("path", "Expected a regular file.");
|
||||
admitted = true;
|
||||
const file = await this.files.open(
|
||||
input.path,
|
||||
constants.O_RDONLY | constants.O_NONBLOCK,
|
||||
);
|
||||
const report = await withArtifactFileHandle(
|
||||
input.path,
|
||||
file,
|
||||
this.#resources,
|
||||
options?.signal,
|
||||
async (file) => {
|
||||
const initial = await file.stat({ bigint: true });
|
||||
const changed = () =>
|
||||
new AnalysisArtifactChangedError(
|
||||
OPERATION,
|
||||
input.path,
|
||||
"File identity, size or timestamps changed during entropy inspection.",
|
||||
);
|
||||
const same = (value: typeof initial) =>
|
||||
value.isFile() &&
|
||||
value.dev === initial.dev &&
|
||||
value.ino === initial.ino &&
|
||||
value.size === initial.size &&
|
||||
value.mtimeNs === initial.mtimeNs &&
|
||||
value.ctimeNs === initial.ctimeNs;
|
||||
if (!same(before)) throw changed();
|
||||
const size = Number(initial.size);
|
||||
if (!Number.isSafeInteger(size))
|
||||
throw invalid(
|
||||
"path",
|
||||
"File size exceeds exact numeric offset representation.",
|
||||
);
|
||||
const length = input.length ?? size - input.offset;
|
||||
if (input.offset > size || length > size - input.offset)
|
||||
throw invalid(
|
||||
"length",
|
||||
"Selected byte window extends beyond the end of the file.",
|
||||
);
|
||||
const selectedBlocks = Math.ceil(length / input.block_size);
|
||||
if (selectedBlocks > MAX_BYTE_ENTROPY_BLOCKS)
|
||||
throw new AnalysisResourceConstraintError(
|
||||
OPERATION,
|
||||
"capacity",
|
||||
`Selected window would produce ${String(selectedBlocks)} blocks; the limit is ${String(MAX_BYTE_ENTROPY_BLOCKS)}. Choose a shorter window or a larger block size.`,
|
||||
{
|
||||
max_blocks: MAX_BYTE_ENTROPY_BLOCKS,
|
||||
requested_blocks: selectedBlocks,
|
||||
block_size: input.block_size,
|
||||
window_length: length,
|
||||
},
|
||||
{
|
||||
retryable: false,
|
||||
remediationAction:
|
||||
"Reduce the selected window length or increase block_size so the selected window contains at most 32768 blocks.",
|
||||
},
|
||||
);
|
||||
const end = input.offset + length;
|
||||
const blocks: ByteEntropy["blocks"] = [];
|
||||
const buffer = Buffer.alloc(input.block_size);
|
||||
const counts = new Float64Array(256);
|
||||
const hash = createHash("sha256");
|
||||
let position = 0;
|
||||
while (position < size) {
|
||||
options?.signal?.throwIfAborted();
|
||||
const selected = position >= input.offset && position < end;
|
||||
const boundary =
|
||||
position < input.offset ? input.offset : selected ? end : size;
|
||||
const wanted = Math.min(input.block_size, boundary - position);
|
||||
let filled = 0;
|
||||
while (filled < wanted) {
|
||||
options?.signal?.throwIfAborted();
|
||||
const { bytesRead } = await file.read(
|
||||
buffer,
|
||||
filled,
|
||||
wanted - filled,
|
||||
position + filled,
|
||||
);
|
||||
if (bytesRead === 0) throw changed();
|
||||
filled += bytesRead;
|
||||
}
|
||||
const bytes = buffer.subarray(0, filled);
|
||||
hash.update(bytes);
|
||||
for (const byte of bytes) counts[byte] = (counts[byte] ?? 0) + 1;
|
||||
if (selected)
|
||||
blocks.push({
|
||||
offset: position,
|
||||
length: filled,
|
||||
entropy: computeBlockEntropy(bytes),
|
||||
});
|
||||
position += filled;
|
||||
}
|
||||
if (
|
||||
!same(await file.stat({ bigint: true })) ||
|
||||
!same(await this.files.stat(input.path, { bigint: true }))
|
||||
)
|
||||
throw changed();
|
||||
options?.signal?.throwIfAborted();
|
||||
const thresholds = { high: 7.2, low: 1.0 };
|
||||
return {
|
||||
artifact: {
|
||||
path: input.path,
|
||||
bytes: size,
|
||||
sha256: hash.digest("hex"),
|
||||
},
|
||||
block_size: input.block_size,
|
||||
window: { offset: input.offset, length },
|
||||
file_entropy: entropyFromCounts(counts, size),
|
||||
thresholds,
|
||||
blocks,
|
||||
runs: mergeEntropyRuns(blocks, thresholds),
|
||||
limitations: [
|
||||
"Entropy does not prove encryption; compressed and encrypted bytes cannot be distinguished by this measure.",
|
||||
"Entropy is rounded to four decimal places; thresholds classify those reported values. Small blocks can underestimate entropy.",
|
||||
"The file-wide hash and entropy cover the entire file; blocks and runs cover only the selected window. Empty input has entropy 0 by convention.",
|
||||
...(size === 0 ? [ZERO_SIZE_LIMITATION] : []),
|
||||
],
|
||||
};
|
||||
},
|
||||
);
|
||||
if (options?.signal?.aborted)
|
||||
return err(new AnalysisCancelledError(OPERATION));
|
||||
return ok(
|
||||
createEvidence(
|
||||
{ path: input.path, format: "file", sha256: report.artifact.sha256 },
|
||||
BYTE_ENTROPY_PROVIDER,
|
||||
{
|
||||
operation: OPERATION,
|
||||
parameters: {
|
||||
path: input.path,
|
||||
block_size: input.block_size,
|
||||
offset: input.offset,
|
||||
...(input.length === undefined ? {} : { length: input.length }),
|
||||
},
|
||||
result: report,
|
||||
rawResult: null,
|
||||
confidence: "observed",
|
||||
limitations: report.limitations,
|
||||
locations: [{ kind: "artifact-path", path: input.path }],
|
||||
},
|
||||
),
|
||||
);
|
||||
} catch (cause) {
|
||||
if (cause instanceof ArtifactFileCleanupFailure) {
|
||||
const cleanup = new AnalysisOutputError(
|
||||
OPERATION,
|
||||
"Could not confirm closing the selected file.",
|
||||
{ cause, cleanup: cause.cleanup },
|
||||
);
|
||||
bindArtifactFileCleanup(cause, cleanup);
|
||||
const outcome = cause.fileCleanupFailure.outcome;
|
||||
if (
|
||||
outcome.kind === "failed" &&
|
||||
outcome.cause instanceof ArtifactReaderFailure &&
|
||||
outcome.cause.reason === "cancelled"
|
||||
) {
|
||||
const cancelled = new AnalysisCancelledError(OPERATION, {
|
||||
cause: outcome.cause.cancellation?.reason,
|
||||
cleanup: cause.cleanup,
|
||||
}).retainSecondaryFailure(cleanup);
|
||||
bindArtifactFileCleanup(cause, cancelled);
|
||||
return err(cancelled);
|
||||
}
|
||||
if (options?.signal?.aborted && outcome.kind === "completed") {
|
||||
const cancelled = new AnalysisCancelledError(OPERATION, {
|
||||
cause: options.signal.reason,
|
||||
cleanup: cause.cleanup,
|
||||
}).retainSecondaryFailure(cleanup);
|
||||
bindArtifactFileCleanup(cause, cancelled);
|
||||
return err(cancelled);
|
||||
}
|
||||
if (outcome.kind === "failed") {
|
||||
const primary =
|
||||
outcome.cause instanceof AnalysisError
|
||||
? outcome.cause
|
||||
: outcome.cause instanceof ArtifactReaderFailure
|
||||
? new AnalysisOutputError(OPERATION, outcome.cause.message, {
|
||||
cause: outcome.cause,
|
||||
})
|
||||
: new AnalysisOutputError(
|
||||
OPERATION,
|
||||
"Could not read complete byte entropy evidence.",
|
||||
{ cause: outcome.cause },
|
||||
);
|
||||
return err(
|
||||
analysisErrorWithCleanupFailure(
|
||||
primary,
|
||||
cleanup,
|
||||
OPERATION,
|
||||
).bindCleanupDependencies([primary, cleanup]),
|
||||
);
|
||||
}
|
||||
return err(cleanup);
|
||||
}
|
||||
if (cause instanceof ArtifactReaderFailure) {
|
||||
if (cause.reason === "cancelled") {
|
||||
const cancelled = new AnalysisCancelledError(OPERATION, {
|
||||
cause: artifactFileCancellationCause(cause),
|
||||
});
|
||||
bindArtifactFileCleanup(cause, cancelled);
|
||||
return err(cancelled);
|
||||
}
|
||||
const retry = new AnalysisOutputError(
|
||||
OPERATION,
|
||||
"A previous byte-entropy file descriptor is still owned; cleanup must succeed before another file can be inspected.",
|
||||
{
|
||||
cause,
|
||||
...(cause.cleanup === undefined ? {} : { cleanup: cause.cleanup }),
|
||||
},
|
||||
);
|
||||
bindArtifactFileCleanup(cause, retry);
|
||||
return err(retry);
|
||||
}
|
||||
if (options?.signal?.aborted)
|
||||
return err(new AnalysisCancelledError(OPERATION, { cause }));
|
||||
if (cause instanceof AnalysisError) return err(cause);
|
||||
if (cause instanceof Error && "code" in cause) {
|
||||
if (cause.code === "EACCES" || cause.code === "EPERM")
|
||||
return err(
|
||||
new AnalysisAccessDeniedError(OPERATION, input.path, cause.code, {
|
||||
cause,
|
||||
}),
|
||||
);
|
||||
if (cause.code === "ENOENT" || cause.code === "ENOTDIR")
|
||||
return err(
|
||||
admitted
|
||||
? new AnalysisArtifactChangedError(
|
||||
OPERATION,
|
||||
input.path,
|
||||
"Selected file path changed during entropy inspection.",
|
||||
{ cause },
|
||||
)
|
||||
: invalid("path", "Selected file could not be found."),
|
||||
);
|
||||
}
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
OPERATION,
|
||||
"Could not read complete byte entropy evidence.",
|
||||
{ cause },
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
import type {
|
||||
ExecutionOptions,
|
||||
ProviderIdentity,
|
||||
} from "../AnalysisProvider.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import type {
|
||||
InspectElfNotesInput,
|
||||
ElfNotes,
|
||||
} from "../../domain/native/elfNotes.js";
|
||||
import type { Result } from "../../domain/result.js";
|
||||
|
||||
/** Stable read and ELF note-table decoding, independent of a disassembler session. */
|
||||
export interface ElfNotesPort {
|
||||
readonly identity: ProviderIdentity;
|
||||
close?(): Promise<void>;
|
||||
inspect(
|
||||
input: InspectElfNotesInput,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<ElfNotes, AnalysisError>>;
|
||||
}
|
||||
@@ -1,79 +0,0 @@
|
||||
import { expect, it } from "vitest";
|
||||
import { ElfNotesService } from "./ElfNotesService.js";
|
||||
import { ELF_NOTES_PROVIDER } from "../InvestigationProviders.js";
|
||||
import type { ElfNotes } from "../../domain/native/elfNotes.js";
|
||||
import { ok } from "../../domain/result.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
|
||||
const identity = ELF_NOTES_PROVIDER;
|
||||
const report: ElfNotes = {
|
||||
artifact: {
|
||||
path: "/artifacts/app",
|
||||
sha256: "a".repeat(64),
|
||||
bytes: 512,
|
||||
},
|
||||
format: "elf64",
|
||||
byte_order: "little",
|
||||
machine: 62,
|
||||
type: 3,
|
||||
type_name: "shared",
|
||||
tables: [],
|
||||
entries: [],
|
||||
coverage: { status: "complete", examined_tables: 0, examined_entries: 0 },
|
||||
limitations: ["No PT_NOTE program header or SHT_NOTE section was declared."],
|
||||
};
|
||||
|
||||
it("rejects a relative path and extra arguments before reading", async () => {
|
||||
let calls = 0;
|
||||
const service = new ElfNotesService({
|
||||
identity,
|
||||
inspect: () => {
|
||||
calls += 1;
|
||||
return Promise.resolve(ok(report));
|
||||
},
|
||||
});
|
||||
const relative = await service.inspect({ path: "app" });
|
||||
const extra = await service.inspect({
|
||||
path: report.artifact.path,
|
||||
extra: true,
|
||||
});
|
||||
expect(relative.ok).toBe(false);
|
||||
expect(extra.ok).toBe(false);
|
||||
if (!relative.ok) expect(relative.error).toBeInstanceOf(AnalysisInputError);
|
||||
if (!extra.ok) expect(extra.error).toBeInstanceOf(AnalysisInputError);
|
||||
expect(calls).toBe(0);
|
||||
});
|
||||
|
||||
it("records the selected ELF and an absent note table", async () => {
|
||||
const result = await new ElfNotesService({
|
||||
identity,
|
||||
inspect: () => Promise.resolve(ok(report)),
|
||||
}).inspect({ path: report.artifact.path });
|
||||
expect(result.ok).toBe(true);
|
||||
if (!result.ok) throw result.error;
|
||||
expect(result.value).toMatchObject({
|
||||
confidence: "observed",
|
||||
subject: { format: "elf", architecture: "x86_64" },
|
||||
normalized_result: report,
|
||||
});
|
||||
});
|
||||
|
||||
it("discards an observation if the caller cancels while the provider completes", async () => {
|
||||
const controller = new AbortController();
|
||||
const service = new ElfNotesService({
|
||||
identity,
|
||||
inspect: () => {
|
||||
controller.abort();
|
||||
return Promise.resolve(ok(report));
|
||||
},
|
||||
});
|
||||
const result = await service.inspect(
|
||||
{ path: report.artifact.path },
|
||||
{ signal: controller.signal },
|
||||
);
|
||||
expect(result.ok).toBe(false);
|
||||
if (!result.ok) expect(result.error).toBeInstanceOf(AnalysisCancelledError);
|
||||
});
|
||||
@@ -1,127 +0,0 @@
|
||||
import { isAbsolute } from "node:path";
|
||||
import type { ExecutionOptions } from "../AnalysisProvider.js";
|
||||
import type { ElfNotesPort } from "./ElfNotesPort.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import {
|
||||
AnalysisCancelledError,
|
||||
AnalysisInputError,
|
||||
AnalysisOutputError,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import { createEvidence, type Evidence } from "../../domain/evidence.js";
|
||||
import {
|
||||
inspectElfNotesInputSchema,
|
||||
elfNotesSchema,
|
||||
type ElfNotes,
|
||||
} from "../../domain/native/elfNotes.js";
|
||||
import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
|
||||
import { BINARY_ARCHITECTURES } from "../../domain/binaryTargetTypes.js";
|
||||
import { elfArchitecture } from "../../domain/elfFixedHeader.js";
|
||||
import { err, type Result } from "../../domain/result.js";
|
||||
import { completeProviderEvidence } from "../ProviderOperationExecution.js";
|
||||
|
||||
const OPERATION = "inspect_elf_notes";
|
||||
|
||||
/** Shared caller validation, output parsing and Evidence provenance for CLI/MCP. */
|
||||
export class ElfNotesService {
|
||||
constructor(readonly provider: ElfNotesPort) {}
|
||||
|
||||
close(): Promise<void> {
|
||||
return this.provider.close?.() ?? Promise.resolve();
|
||||
}
|
||||
|
||||
async inspect(
|
||||
rawInput: unknown,
|
||||
options?: ExecutionOptions,
|
||||
): Promise<Result<Evidence, AnalysisError>> {
|
||||
if (options?.signal?.aborted)
|
||||
return err(new AnalysisCancelledError(OPERATION));
|
||||
const input = inspectElfNotesInputSchema.safeParse(rawInput);
|
||||
if (!input.success)
|
||||
return err(
|
||||
analysisInputErrorFromIssues(OPERATION, input.error.issues, rawInput, {
|
||||
cause: input.error,
|
||||
}),
|
||||
);
|
||||
if (!isAbsolute(input.data.path))
|
||||
return err(
|
||||
new AnalysisInputError(OPERATION, undefined, [
|
||||
{
|
||||
path: ["path"],
|
||||
reason: "invalid_format",
|
||||
message: "Expected an absolute filesystem path on this host.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
const result = await this.provider.inspect(input.data, options);
|
||||
if (!result.ok) return result;
|
||||
const report = elfNotesSchema.safeParse(result.value);
|
||||
if (
|
||||
!report.success ||
|
||||
report.data.artifact.path !== input.data.path ||
|
||||
!validReport(
|
||||
report.data,
|
||||
input.data.max_entries,
|
||||
input.data.max_file_bytes,
|
||||
)
|
||||
)
|
||||
return err(
|
||||
new AnalysisOutputError(
|
||||
OPERATION,
|
||||
"ELF note provider returned malformed output or changed artifact identity.",
|
||||
),
|
||||
);
|
||||
const value = report.data;
|
||||
const architecture = BINARY_ARCHITECTURES.find(
|
||||
(supported) => supported === elfArchitecture(value.machine),
|
||||
);
|
||||
const locations = [
|
||||
{ kind: "artifact-path" as const, path: value.artifact.path },
|
||||
...value.tables.map((table) => ({
|
||||
kind: "file-offset-range" as const,
|
||||
start: table.location.offset,
|
||||
end: table.location.offset + table.location.bytes,
|
||||
})),
|
||||
];
|
||||
return completeProviderEvidence(
|
||||
createEvidence(
|
||||
{
|
||||
path: value.artifact.path,
|
||||
format: "elf",
|
||||
sha256: value.artifact.sha256,
|
||||
...(architecture === undefined ? {} : { architecture }),
|
||||
},
|
||||
this.provider.identity,
|
||||
{
|
||||
operation: OPERATION,
|
||||
parameters: input.data,
|
||||
result: value,
|
||||
confidence: "observed",
|
||||
limitations: value.limitations,
|
||||
locations,
|
||||
},
|
||||
),
|
||||
options?.signal,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const validReport = (
|
||||
report: ElfNotes,
|
||||
maxEntries: number,
|
||||
maxBytes: number,
|
||||
): boolean => {
|
||||
const inside = ({ offset, bytes }: { offset: number; bytes: number }) =>
|
||||
offset <= report.artifact.bytes - bytes;
|
||||
if (
|
||||
report.artifact.bytes > maxBytes ||
|
||||
report.coverage.examined_entries > maxEntries ||
|
||||
report.entries.length > maxEntries
|
||||
)
|
||||
return false;
|
||||
return (
|
||||
report.tables.every((table) => inside(table.location)) &&
|
||||
report.entries.every(
|
||||
(entry) => inside(entry.location) && inside(entry.descriptor.location),
|
||||
)
|
||||
);
|
||||
};
|
||||
@@ -22,7 +22,7 @@ import {
|
||||
} from "./environmentBundle.js";
|
||||
import type { EnvironmentSource } from "./environmentSource.js";
|
||||
import { prepareServedState } from "./environmentState.js";
|
||||
import { environmentToolAnswer } from "./environmentToolAnswer.js";
|
||||
import { environmentToolAnswer } from "./environmentStateAnswer.js";
|
||||
import { EnvironmentStateStore } from "./environmentStateStore.js";
|
||||
|
||||
const fixture = environmentServeFixture();
|
||||
@@ -218,9 +218,7 @@ describe("planEnvironmentBundle", () => {
|
||||
expect(readme).not.toContain("agent-env run grafana-example");
|
||||
expect(readme).toContain("## What the environment serves");
|
||||
expect(readme).not.toContain("What replay means");
|
||||
expect(readme).toContain(
|
||||
"write tools are served only where the capture's later reads confirmed\ntheir effect",
|
||||
);
|
||||
expect(readme).toContain("write tools are not served yet");
|
||||
expect(readme).toContain(
|
||||
"`rea environment-serve --replay-only` serves pure replay",
|
||||
);
|
||||
|
||||
@@ -7,7 +7,7 @@ import { servedCatalog, type PreparedState } from "./environmentState.js";
|
||||
import {
|
||||
environmentToolAnswer,
|
||||
type EnvironmentToolAnswer,
|
||||
} from "./environmentToolAnswer.js";
|
||||
} from "./environmentStateAnswer.js";
|
||||
import { EnvironmentStateStore } from "./environmentStateStore.js";
|
||||
import type { EnvironmentToolDefinition } from "./environmentTools.js";
|
||||
|
||||
@@ -52,7 +52,7 @@ export interface EnvironmentServedCheck {
|
||||
export interface EnvironmentBundlePlan {
|
||||
readonly name: string;
|
||||
readonly packageSource: EnvironmentPackageSource;
|
||||
/** Tools the bundle serves: every manifest tool for replay; from a state, the read tools and the served write tools. */
|
||||
/** Tools the bundle serves: every manifest tool for replay, the read tools from a state. */
|
||||
readonly toolCount: number;
|
||||
readonly check: EnvironmentServedCheck | null;
|
||||
/** Whether the bundle serves from the state, as `environment-serve` reports it. */
|
||||
@@ -282,8 +282,7 @@ server derives the state the capture observed, the \`environment-state\` view
|
||||
of the capture with the manifest's template rules, and serves from it: read
|
||||
tools answer from the state where it backs them and from their recordings
|
||||
otherwise, \`/agentenv\` serves \`data/reset\`, \`data/add\` and \`data/get\` on
|
||||
it, and write tools are served only where the capture's later reads confirmed
|
||||
their effect. When the state cannot be derived, or
|
||||
it, and write tools are not served yet. When the state cannot be derived, or
|
||||
its catalog exceeds the discovery budget, the server logs why and replays
|
||||
instead: each MCP tool call selects the retained record whose path and query
|
||||
values best match the arguments and returns that record's recorded status,
|
||||
@@ -333,7 +332,7 @@ export const planEnvironmentBundle = (
|
||||
// The catalog and answers the bundle's server will have, so the verifier and README predict them.
|
||||
const catalog = servedCatalog(
|
||||
source,
|
||||
state,
|
||||
state !== undefined,
|
||||
name,
|
||||
ENVIRONMENT_EXPORT_OPERATION,
|
||||
);
|
||||
|
||||
@@ -5,51 +5,20 @@ import {
|
||||
analysisViewCaptureFixture,
|
||||
environmentManifestCaptureFixture,
|
||||
} from "../../../tests/fixtures/analysisView.js";
|
||||
import { scenarioCaptureEvidence } from "../../../tests/fixtures/browserScenarioCapture.js";
|
||||
import {
|
||||
environmentServeFixture,
|
||||
environmentServeSource,
|
||||
} from "../../../tests/fixtures/environmentServe.js";
|
||||
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { createEvidence, type Evidence } from "../../domain/evidence.js";
|
||||
import { inspectAnalysisView } from "../analysisView/AnalysisViewService.js";
|
||||
import { loadEnvironmentSource } from "./environmentSource.js";
|
||||
|
||||
const refusal = (manifest: unknown, evidence: unknown, operation?: string) => {
|
||||
const loaded = loadEnvironmentSource(manifest, evidence, operation);
|
||||
const refusal = (manifest: unknown, evidence: unknown) => {
|
||||
const loaded = loadEnvironmentSource(manifest, evidence);
|
||||
if (loaded.ok) throw new Error("Expected the pair to be refused");
|
||||
return projectAnalysisError(loaded.error);
|
||||
};
|
||||
|
||||
/** Authenticated Evidence whose payload is rewritten; createEvidence recomputes the digest. */
|
||||
const resealed = (
|
||||
evidence: Evidence,
|
||||
edit: (result: Record<string, unknown>) => void,
|
||||
operation: string = evidence.operation,
|
||||
): Evidence => {
|
||||
const result = structuredClone(evidence.normalized_result) as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
edit(result);
|
||||
return createEvidence(undefined, evidence.provider, {
|
||||
operation,
|
||||
parameters: structuredClone(evidence.parameters),
|
||||
result: result as never,
|
||||
confidence: evidence.confidence,
|
||||
});
|
||||
};
|
||||
|
||||
/** Re-cite a manifest's parent so pairing passes and the capture parse alone decides. */
|
||||
const citing = (manifest: Evidence, capture: Evidence): Evidence =>
|
||||
resealed(manifest, (result) => {
|
||||
result.parent_evidence_id = capture.evidence_id;
|
||||
});
|
||||
|
||||
const record = (value: unknown, key: string): Record<string, unknown> =>
|
||||
(value as Record<string, Record<string, unknown>>)[key] ?? {};
|
||||
|
||||
describe("loadEnvironmentSource", () => {
|
||||
it("authenticates the pair and indexes every cited record as a replayable exchange", () => {
|
||||
const fixture = environmentServeFixture();
|
||||
@@ -196,111 +165,6 @@ describe("loadEnvironmentSource", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("loadEnvironmentSource payload attribution", () => {
|
||||
it("reports capture payload failures beneath normalized_result instead of the supported operation", () => {
|
||||
const fixture = environmentServeFixture();
|
||||
for (const operation of [
|
||||
"inspect_web_network_capture",
|
||||
"capture_browser_scenario",
|
||||
]) {
|
||||
const capture = createEvidence(undefined, fixture.capture.provider, {
|
||||
operation,
|
||||
parameters: {},
|
||||
result: {},
|
||||
confidence: "observed",
|
||||
});
|
||||
for (const command of ["environment-serve", "environment-export"]) {
|
||||
const loaded = loadEnvironmentSource(
|
||||
citing(fixture.manifest, capture),
|
||||
capture,
|
||||
command,
|
||||
);
|
||||
if (loaded.ok || !(loaded.error instanceof AnalysisInputError))
|
||||
throw new Error("Expected an input refusal");
|
||||
expect(loaded.error.issues.length).toBeGreaterThan(0);
|
||||
for (const issue of loaded.error.issues)
|
||||
expect(issue.path.slice(0, 2)).toEqual([
|
||||
"evidence",
|
||||
"normalized_result",
|
||||
]);
|
||||
expect(loaded.error.cause).toBeInstanceOf(Error);
|
||||
expect(projectAnalysisError(loaded.error)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
category: "invalid_input",
|
||||
details: { operation: command },
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it("reports the full path of a nested capture payload field", () => {
|
||||
const fixture = environmentServeFixture();
|
||||
const har = resealed(fixture.capture, (result) => {
|
||||
record(result, "artifact").sha256 = 42;
|
||||
});
|
||||
expect(refusal(citing(fixture.manifest, har), har).details?.issues).toEqual(
|
||||
[
|
||||
expect.objectContaining({
|
||||
path: ["evidence", "normalized_result", "artifact", "sha256"],
|
||||
reason: "invalid_type",
|
||||
}),
|
||||
],
|
||||
);
|
||||
const scenario = resealed(scenarioCaptureEvidence(), (result) => {
|
||||
record(result, "browser").product = 7;
|
||||
});
|
||||
expect(
|
||||
refusal(citing(fixture.manifest, scenario), scenario).details?.issues,
|
||||
).toEqual([
|
||||
expect.objectContaining({
|
||||
path: ["evidence", "normalized_result", "browser", "product"],
|
||||
reason: "invalid_type",
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it("still blames the operation of Evidence no capture schema owns", () => {
|
||||
const fixture = environmentServeFixture();
|
||||
const other = resealed(fixture.capture, () => {}, "inspect_other_capture");
|
||||
expect(
|
||||
refusal(citing(fixture.manifest, other), other).details?.issues,
|
||||
).toEqual([
|
||||
expect.objectContaining({
|
||||
path: ["evidence", "operation"],
|
||||
expected: "inspect_web_network_capture or capture_browser_scenario",
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it("separates the manifest operation, view kind and view payload failures", () => {
|
||||
const fixture = environmentServeFixture();
|
||||
expect(refusal(fixture.capture, fixture.capture).details?.issues).toEqual([
|
||||
expect.objectContaining({
|
||||
path: ["manifest", "operation"],
|
||||
expected: "inspect_analysis_view",
|
||||
}),
|
||||
]);
|
||||
const malformed = resealed(fixture.manifest, (result) => {
|
||||
result.name = 7;
|
||||
});
|
||||
expect(refusal(malformed, fixture.capture).details?.issues).toEqual([
|
||||
expect.objectContaining({
|
||||
path: ["manifest", "normalized_result", "name"],
|
||||
reason: "invalid_type",
|
||||
}),
|
||||
]);
|
||||
const unknownKind = resealed(fixture.manifest, (result) => {
|
||||
result.kind = "not-a-view";
|
||||
});
|
||||
expect(refusal(unknownKind, fixture.capture).details?.issues).toEqual([
|
||||
expect.objectContaining({
|
||||
path: ["manifest", "normalized_result", "kind"],
|
||||
expected: "environment-manifest",
|
||||
}),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("loadEnvironmentSource recorded request lines", () => {
|
||||
it("keeps the delimiter of an explicit empty query in the recorded exchange", () => {
|
||||
const capture = environmentManifestCaptureFixture();
|
||||
|
||||
@@ -143,34 +143,18 @@ export const parseEvidenceFile = (
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof z.ZodError)) throw cause;
|
||||
return err(
|
||||
schemaInputError(
|
||||
inputError(
|
||||
operation,
|
||||
{ prefix: [role], cause, value },
|
||||
projectInputIssues(cause.issues, value).map((issue) => ({
|
||||
...issue,
|
||||
path: [role, ...issue.path],
|
||||
})),
|
||||
`The ${role} file is not authenticated REA Evidence JSON.`,
|
||||
),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
/** A schema refusal of `value`, found at `prefix`; issue paths extend the prefix and the schema failure is retained. */
|
||||
const schemaInputError = (
|
||||
operation: string,
|
||||
failure: {
|
||||
readonly prefix: readonly (string | number)[];
|
||||
readonly cause: z.ZodError;
|
||||
readonly value: unknown;
|
||||
},
|
||||
userMessage: string,
|
||||
): AnalysisInputError =>
|
||||
new AnalysisInputError(
|
||||
operation,
|
||||
{ userMessage, cause: failure.cause },
|
||||
projectInputIssues(failure.cause.issues, failure.value).map((issue) => ({
|
||||
...issue,
|
||||
path: [...failure.prefix, ...issue.path],
|
||||
})),
|
||||
);
|
||||
|
||||
const recordedExchange = (
|
||||
record: WebNetworkCaptureRecord,
|
||||
): RecordedExchange | undefined => {
|
||||
@@ -201,27 +185,14 @@ export const parseViewEvidence = <Kind extends AnalysisViewResult["kind"]>(
|
||||
> => {
|
||||
const evidence = parseEvidenceFile(operation, role, value);
|
||||
if (!evidence.ok) return evidence;
|
||||
const message = `The ${role} file is not ${kind} view Evidence.`;
|
||||
if (evidence.value.operation !== "inspect_analysis_view")
|
||||
return err(
|
||||
inputError(
|
||||
operation,
|
||||
[
|
||||
{
|
||||
path: [role, "operation"],
|
||||
reason: "invalid_value",
|
||||
expected: "inspect_analysis_view",
|
||||
message: `The ${role} file must be inspect_analysis_view Evidence of kind ${kind}.`,
|
||||
},
|
||||
],
|
||||
message,
|
||||
),
|
||||
);
|
||||
const normalized = immutableJsonSnapshotMember(
|
||||
evidence.value,
|
||||
"normalized_result",
|
||||
const view = analysisViewResultSchema.safeParse(
|
||||
evidence.value.normalized_result,
|
||||
);
|
||||
if (immutableJsonSnapshotMember(normalized, "kind") !== kind)
|
||||
if (
|
||||
evidence.value.operation !== "inspect_analysis_view" ||
|
||||
!view.success ||
|
||||
view.data.kind !== kind
|
||||
)
|
||||
return err(
|
||||
inputError(
|
||||
operation,
|
||||
@@ -233,21 +204,7 @@ export const parseViewEvidence = <Kind extends AnalysisViewResult["kind"]>(
|
||||
message: `The ${role} file must be inspect_analysis_view Evidence of kind ${kind}.`,
|
||||
},
|
||||
],
|
||||
message,
|
||||
),
|
||||
);
|
||||
// The discriminator selected the requested view, so any failure is in that view's payload.
|
||||
const view = analysisViewResultSchema.safeParse(normalized);
|
||||
if (!view.success)
|
||||
return err(
|
||||
schemaInputError(
|
||||
operation,
|
||||
{
|
||||
prefix: [role, "normalized_result"],
|
||||
cause: view.error,
|
||||
value: normalized,
|
||||
},
|
||||
message,
|
||||
`The ${role} file is not ${kind} view Evidence.`,
|
||||
),
|
||||
);
|
||||
return ok({
|
||||
@@ -256,132 +213,89 @@ export const parseViewEvidence = <Kind extends AnalysisViewResult["kind"]>(
|
||||
});
|
||||
};
|
||||
|
||||
/** What replay needs from a parent capture once its payload has been validated. */
|
||||
interface CaptureFacts {
|
||||
readonly records: readonly WebNetworkCaptureRecord[];
|
||||
readonly captureKind: EnvironmentCaptureKind;
|
||||
readonly captureSha256: string | null;
|
||||
}
|
||||
|
||||
/** Issue paths of a capture payload refusal start here. */
|
||||
const CAPTURE_PAYLOAD_PATH = ["evidence", "normalized_result"] as const;
|
||||
|
||||
/** Validate a capture_browser_scenario payload; its failures name the payload fields. */
|
||||
const parseScenarioCapture = (
|
||||
operation: string,
|
||||
normalized: unknown,
|
||||
parameters: unknown,
|
||||
): Result<CaptureFacts, AnalysisError> => {
|
||||
const scenario = browserScenarioCaptureSchema.safeParse(normalized);
|
||||
if (!scenario.success)
|
||||
return err(
|
||||
schemaInputError(
|
||||
operation,
|
||||
{
|
||||
prefix: CAPTURE_PAYLOAD_PATH,
|
||||
cause: scenario.error,
|
||||
value: normalized,
|
||||
},
|
||||
"The evidence file is not a complete browser scenario capture.",
|
||||
),
|
||||
);
|
||||
const definition = z
|
||||
.object({ scenario_sha256: z.string() })
|
||||
.safeParse(parameters);
|
||||
return ok({
|
||||
records: scenarioCaptureRecords(scenario.data, "replay").records,
|
||||
captureKind: "browser-scenario",
|
||||
captureSha256: definition.success ? definition.data.scenario_sha256 : null,
|
||||
});
|
||||
};
|
||||
|
||||
/** Validate an inspect_web_network_capture payload as HAR; its failures name the payload fields. */
|
||||
const parseHarCapture = (
|
||||
operation: string,
|
||||
normalized: unknown,
|
||||
): Result<CaptureFacts, AnalysisError> => {
|
||||
const records = immutableJsonSnapshotMember(normalized, "records");
|
||||
if (Array.isArray(records))
|
||||
for (let index = 0; index < records.length; index += 1) {
|
||||
const record = immutableJsonSnapshotMember(records, String(index));
|
||||
immutableJsonSnapshotMember(record, "reported");
|
||||
}
|
||||
// An owned snapshot is validated once and borrowed, so deriving the state reuses this validation.
|
||||
const capture = safeParseOwnedWebNetworkCapture(normalized);
|
||||
if (!capture.success)
|
||||
return err(
|
||||
schemaInputError(
|
||||
operation,
|
||||
{
|
||||
prefix: CAPTURE_PAYLOAD_PATH,
|
||||
cause: capture.error,
|
||||
value: normalized,
|
||||
},
|
||||
"The evidence file is not a complete web network capture.",
|
||||
),
|
||||
);
|
||||
// The manifest view projects HAR captures only, and replay reads HAR's response fields.
|
||||
if (capture.data.format !== "har")
|
||||
return err(
|
||||
inputError(
|
||||
operation,
|
||||
[
|
||||
{
|
||||
path: [...CAPTURE_PAYLOAD_PATH, "format"],
|
||||
reason: "invalid_value",
|
||||
expected: "har",
|
||||
message: `Agent environments replay HAR captures; this capture's format is ${capture.data.format}.`,
|
||||
},
|
||||
],
|
||||
"The evidence file is not a HAR capture.",
|
||||
),
|
||||
);
|
||||
return ok({
|
||||
records: capture.data.records,
|
||||
captureKind: "web-network-capture",
|
||||
captureSha256: capture.data.artifact.sha256,
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* Authenticate the evidence file and require complete capture Evidence of
|
||||
* either kind. Only an operation no capture schema owns blames the operation;
|
||||
* a recognized operation's payload failures name the payload fields.
|
||||
*/
|
||||
/** Authenticate the evidence file and require complete capture Evidence of either kind. */
|
||||
const parseCaptureEvidence = (
|
||||
operation: string,
|
||||
value: unknown,
|
||||
): Result<CaptureFacts & { readonly evidenceId: string }, AnalysisError> => {
|
||||
): Result<
|
||||
{
|
||||
readonly records: readonly WebNetworkCaptureRecord[];
|
||||
readonly evidenceId: string;
|
||||
readonly captureKind: EnvironmentCaptureKind;
|
||||
readonly captureSha256: string | null;
|
||||
},
|
||||
AnalysisError
|
||||
> => {
|
||||
const evidence = parseEvidenceFile(operation, "evidence", value);
|
||||
if (!evidence.ok) return evidence;
|
||||
const { evidence_id: evidenceId } = evidence.value;
|
||||
const normalized = immutableJsonSnapshotMember(
|
||||
evidence.value,
|
||||
"normalized_result",
|
||||
);
|
||||
const capture =
|
||||
evidence.value.operation === "capture_browser_scenario"
|
||||
? parseScenarioCapture(operation, normalized, evidence.value.parameters)
|
||||
: evidence.value.operation === "inspect_web_network_capture"
|
||||
? parseHarCapture(operation, normalized)
|
||||
: err(
|
||||
inputError(
|
||||
operation,
|
||||
[
|
||||
{
|
||||
path: ["evidence", "operation"],
|
||||
reason: "invalid_value",
|
||||
expected:
|
||||
"inspect_web_network_capture or capture_browser_scenario",
|
||||
message:
|
||||
"The evidence file must be inspect_web_network_capture or capture_browser_scenario Evidence.",
|
||||
},
|
||||
],
|
||||
"The evidence file is not web network capture or browser scenario Evidence.",
|
||||
),
|
||||
);
|
||||
return capture.ok
|
||||
? ok({ ...capture.value, evidenceId: evidence.value.evidence_id })
|
||||
: capture;
|
||||
if (evidence.value.operation === "capture_browser_scenario") {
|
||||
const scenario = browserScenarioCaptureSchema.safeParse(normalized);
|
||||
const definition = z
|
||||
.object({ scenario_sha256: z.string() })
|
||||
.safeParse(evidence.value.parameters);
|
||||
if (scenario.success)
|
||||
return ok({
|
||||
records: scenarioCaptureRecords(scenario.data, "replay").records,
|
||||
evidenceId,
|
||||
captureKind: "browser-scenario",
|
||||
captureSha256: definition.success
|
||||
? definition.data.scenario_sha256
|
||||
: null,
|
||||
});
|
||||
} else if (evidence.value.operation === "inspect_web_network_capture") {
|
||||
const records = immutableJsonSnapshotMember(normalized, "records");
|
||||
if (Array.isArray(records))
|
||||
for (let index = 0; index < records.length; index += 1) {
|
||||
const record = immutableJsonSnapshotMember(records, String(index));
|
||||
immutableJsonSnapshotMember(record, "reported");
|
||||
}
|
||||
// An owned snapshot is validated once and borrowed, so deriving the state reuses this validation.
|
||||
const capture = safeParseOwnedWebNetworkCapture(normalized);
|
||||
if (capture.success) {
|
||||
// The manifest view projects HAR captures only, and replay reads HAR's response fields.
|
||||
if (capture.data.format !== "har")
|
||||
return err(
|
||||
inputError(
|
||||
operation,
|
||||
[
|
||||
{
|
||||
path: ["evidence", "normalized_result", "format"],
|
||||
reason: "invalid_value",
|
||||
expected: "har",
|
||||
message: `Agent environments replay HAR captures; this capture's format is ${capture.data.format}.`,
|
||||
},
|
||||
],
|
||||
"The evidence file is not a HAR capture.",
|
||||
),
|
||||
);
|
||||
return ok({
|
||||
records: capture.data.records,
|
||||
evidenceId,
|
||||
captureKind: "web-network-capture",
|
||||
captureSha256: capture.data.artifact.sha256,
|
||||
});
|
||||
}
|
||||
}
|
||||
return err(
|
||||
inputError(
|
||||
operation,
|
||||
[
|
||||
{
|
||||
path: ["evidence", "operation"],
|
||||
reason: "invalid_value",
|
||||
expected: "inspect_web_network_capture or capture_browser_scenario",
|
||||
message:
|
||||
"The evidence file must be complete inspect_web_network_capture or capture_browser_scenario Evidence.",
|
||||
},
|
||||
],
|
||||
"The evidence file is not web network capture or browser scenario Evidence.",
|
||||
),
|
||||
);
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -24,16 +24,12 @@ import {
|
||||
} from "../analysisView/AnalysisViewService.js";
|
||||
import { loadEnvironmentSource } from "./environmentSource.js";
|
||||
import {
|
||||
admitServedState,
|
||||
deriveEnvironmentState,
|
||||
loadEnvironmentState,
|
||||
prepareServedState,
|
||||
} from "./environmentState.js";
|
||||
import { STATE_INTAKE_LIMIT_BYTES } from "./environmentStateDocument.js";
|
||||
import {
|
||||
EnvironmentStateStore,
|
||||
ENVIRONMENT_COLLECTION_HISTORY_LIMIT_BYTES,
|
||||
} from "./environmentStateStore.js";
|
||||
import { EnvironmentStateStore } from "./environmentStateStore.js";
|
||||
|
||||
const fixture = environmentStateServeFixture();
|
||||
const { source, state } = environmentStateServeInputs(fixture);
|
||||
@@ -154,89 +150,6 @@ describe("loadEnvironmentState", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("served collection history", () => {
|
||||
it("bounds distinct paths added after reset while retaining baseline paths", () => {
|
||||
const original = state.entities[0];
|
||||
if (original === undefined) throw new Error("Expected a seeded entity");
|
||||
const run = () => {
|
||||
const store = new EnvironmentStateStore(state);
|
||||
for (let index = 0; index < 2000; index += 1) {
|
||||
const collection = `/api/history/${String(index)}-${"x".repeat(900)}`;
|
||||
const id = `history-${String(index)}`;
|
||||
const before = JSON.stringify(store.snapshot());
|
||||
const result = store.insert({ ...original, collection, id }, 0);
|
||||
if (!result.ok) {
|
||||
expect(JSON.stringify(store.snapshot())).toBe(before);
|
||||
expect(result.error).toMatchObject({
|
||||
resource: "capacity",
|
||||
retryable: false,
|
||||
remediationAction: expect.stringContaining("fewer distinct"),
|
||||
});
|
||||
expect(result.error.reason).toContain("accounting budget");
|
||||
return result.error.reportedLimits;
|
||||
}
|
||||
const inserted = store.entity(original.type, collection, id);
|
||||
if (inserted === undefined) throw new Error("Expected inserted entity");
|
||||
store.remove(inserted);
|
||||
}
|
||||
throw new Error("Expected a history budget refusal");
|
||||
};
|
||||
const refused = run();
|
||||
if (refused === null)
|
||||
throw new Error("Expected reported history budget details");
|
||||
expect(refused).toEqual(run());
|
||||
expect(refused).toMatchObject({
|
||||
max_collection_history_bytes: ENVIRONMENT_COLLECTION_HISTORY_LIMIT_BYTES,
|
||||
});
|
||||
expect(refused.collection_history_bytes).toBeGreaterThan(
|
||||
ENVIRONMENT_COLLECTION_HISTORY_LIMIT_BYTES,
|
||||
);
|
||||
});
|
||||
|
||||
it("does not charge a deleted loaded collection again at the history limit", () => {
|
||||
const original = state.entities[0];
|
||||
if (original === undefined) throw new Error("Expected a seeded entity");
|
||||
const loadedState = {
|
||||
...state,
|
||||
entities: Array.from({ length: 1200 }, (_, index) => ({
|
||||
...original,
|
||||
collection: `/api/loaded/${String(index)}-${"y".repeat(80)}`,
|
||||
id: `loaded-${String(index)}`,
|
||||
})),
|
||||
};
|
||||
const store = new EnvironmentStateStore(loadedState);
|
||||
for (const entity of store.ofType(original.type)) store.remove(entity);
|
||||
expect(store.knownCollectionsOf(original.type)).toHaveLength(1200);
|
||||
|
||||
const baseline = loadedState.entities[0];
|
||||
if (baseline === undefined) throw new Error("Expected a loaded entity");
|
||||
let refused = false;
|
||||
for (let index = 0; index < 2000; index += 1) {
|
||||
const collection = `/api/generated/${String(index)}-${"z".repeat(900)}`;
|
||||
const entity = {
|
||||
...original,
|
||||
collection,
|
||||
id: `generated-${String(index)}`,
|
||||
};
|
||||
const inserted = store.insert(entity, 0);
|
||||
if (!inserted.ok) {
|
||||
refused = true;
|
||||
break;
|
||||
}
|
||||
const held = store.entity(entity.type, collection, entity.id);
|
||||
if (held === undefined) throw new Error("Expected generated entity");
|
||||
store.remove(held);
|
||||
}
|
||||
expect(refused).toBe(true);
|
||||
|
||||
const restored = store.insert(baseline, 0);
|
||||
expect(restored.ok).toBe(true);
|
||||
expect(
|
||||
store.entity(baseline.type, baseline.collection, baseline.id),
|
||||
).toEqual(baseline);
|
||||
});
|
||||
});
|
||||
|
||||
describe("deriveEnvironmentState", () => {
|
||||
it("derives the state inspect-analysis-view projects from the capture with the manifest's rules, under the same Evidence id", () => {
|
||||
const derived = deriveEnvironmentState(source, fixture.capture);
|
||||
@@ -359,28 +272,6 @@ describe("prepareServedState", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("admitServedState", () => {
|
||||
it("refuses a derived state data/get would write past what data/add loads back", () => {
|
||||
expect(admitServedState(state, "environment_serve").ok).toBe(true);
|
||||
// A euro sign is one UTF-16 unit but three bytes as data/get writes it.
|
||||
const notes = "\u20ac".repeat(12_000_000);
|
||||
const [first] = state.entities;
|
||||
const large = {
|
||||
...state,
|
||||
entities: [
|
||||
...state.entities,
|
||||
{ ...first!, id: "large", fields: { ...first!.fields, notes } },
|
||||
],
|
||||
};
|
||||
const admitted = admitServedState(large, "environment_serve");
|
||||
if (admitted.ok) throw new Error("Expected the state to be refused");
|
||||
expect(projectAnalysisError(admitted.error)).toMatchObject({
|
||||
code: "resource_constraint",
|
||||
message: expect.stringContaining("larger than data/add loads"),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("EnvironmentStateStore", () => {
|
||||
it("reports its state as rea-state/v1, accepts that report back and resets to the initial state", () => {
|
||||
const store = new EnvironmentStateStore(state);
|
||||
|
||||
@@ -6,11 +6,10 @@ import {
|
||||
type StateType,
|
||||
type StateView,
|
||||
} from "../../domain/analysisView/environmentStateRules.js";
|
||||
import type { StateWritePlan } from "../../domain/analysisView/environmentStateWriteRules.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import {
|
||||
import type {
|
||||
AnalysisInputIssue,
|
||||
AnalysisResourceConstraintError,
|
||||
type AnalysisInputIssue,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
@@ -23,9 +22,6 @@ import {
|
||||
type EnvironmentManifestTool,
|
||||
type EnvironmentSource,
|
||||
} from "./environmentSource.js";
|
||||
import { STATE_INTAKE_LIMIT_BYTES } from "./environmentStateDocument.js";
|
||||
import { initialStateBytes } from "./environmentStateStore.js";
|
||||
import { describeWrite } from "./environmentStateWriteDescription.js";
|
||||
import {
|
||||
environmentToolDefinitions,
|
||||
type EnvironmentDataPlane,
|
||||
@@ -45,22 +41,11 @@ export interface EnvironmentState {
|
||||
readonly views: ReadonlyMap<string, StateView>;
|
||||
readonly entities: readonly StateEntity[];
|
||||
readonly documents: readonly StateDocument[];
|
||||
/** Write plans keyed by tool name. */
|
||||
readonly writes: ReadonlyMap<string, StateWritePlan>;
|
||||
/** The instant the served clock starts from, or null when the capture has no times. */
|
||||
readonly writeClock: string | null;
|
||||
}
|
||||
|
||||
/** The write plans a catalog serves from: none before a state is derived. */
|
||||
type ServedWrites = Pick<EnvironmentState, "writes">;
|
||||
|
||||
/** Serving from the state serves read tools and the write tools whose effect the capture confirmed. */
|
||||
const servedFromState = (
|
||||
tool: EnvironmentManifestTool,
|
||||
state: ServedWrites,
|
||||
): boolean =>
|
||||
!changesState(tool.method, tool.path_template) ||
|
||||
state.writes.get(tool.name)?.served === true;
|
||||
/** Serving from the state is read-only: tools whose requests may change state are not served. */
|
||||
const servedFromState = (tool: EnvironmentManifestTool): boolean =>
|
||||
!changesState(tool.method, tool.path_template);
|
||||
|
||||
/** The tools an environment serves and the data plane its card declares. */
|
||||
export interface ServedCatalog {
|
||||
@@ -68,36 +53,27 @@ export interface ServedCatalog {
|
||||
readonly dataPlane: EnvironmentDataPlane;
|
||||
}
|
||||
|
||||
/** The catalog an environment serves: every manifest tool for replay; from a state, the read tools, the confirmed write tools and the data plane. */
|
||||
/** The catalog an environment serves: every manifest tool for replay, the read tools and the data plane from a state. */
|
||||
export const servedCatalog = (
|
||||
source: EnvironmentSource,
|
||||
state: ServedWrites | undefined,
|
||||
fromState: boolean,
|
||||
name: string,
|
||||
operation: string = ENVIRONMENT_SERVE_OPERATION,
|
||||
): Result<ServedCatalog, AnalysisResourceConstraintError> => {
|
||||
const dataPlane: EnvironmentDataPlane =
|
||||
state === undefined ? "none" : "state";
|
||||
const tools = environmentToolDefinitions(
|
||||
state === undefined
|
||||
? source
|
||||
: {
|
||||
...source,
|
||||
manifest: {
|
||||
...source.manifest,
|
||||
tools: source.manifest.tools.filter((tool) =>
|
||||
servedFromState(tool, state),
|
||||
),
|
||||
},
|
||||
const dataPlane: EnvironmentDataPlane = fromState ? "state" : "none";
|
||||
const catalogSource: EnvironmentSource = fromState
|
||||
? {
|
||||
...source,
|
||||
manifest: {
|
||||
...source.manifest,
|
||||
tools: source.manifest.tools.filter(servedFromState),
|
||||
},
|
||||
name,
|
||||
{
|
||||
operation,
|
||||
dataPlane,
|
||||
...(state === undefined
|
||||
? {}
|
||||
: { describe: (tool) => describeWrite(state.writes.get(tool.name)) }),
|
||||
},
|
||||
);
|
||||
}
|
||||
: source;
|
||||
const tools = environmentToolDefinitions(catalogSource, name, {
|
||||
operation,
|
||||
dataPlane,
|
||||
});
|
||||
return tools.ok ? ok({ tools: tools.value, dataPlane }) : tools;
|
||||
};
|
||||
|
||||
@@ -186,8 +162,6 @@ export const loadEnvironmentState = (
|
||||
views: new Map(view.views.map((entry) => [entry.tool, entry])),
|
||||
entities: view.entities,
|
||||
documents: view.documents,
|
||||
writes: new Map(view.writes.map((plan) => [plan.tool, plan])),
|
||||
writeClock: view.write_clock,
|
||||
});
|
||||
};
|
||||
|
||||
@@ -240,29 +214,6 @@ export interface PreparedState extends ServedState {
|
||||
>;
|
||||
}
|
||||
|
||||
/** The derived state, unless data/get would write it past what data/add loads back, so its own report could not be restored. */
|
||||
export const admitServedState = (
|
||||
state: EnvironmentState,
|
||||
operation: string,
|
||||
): Result<EnvironmentState, AnalysisResourceConstraintError> => {
|
||||
const bytes = initialStateBytes(state);
|
||||
return bytes > STATE_INTAKE_LIMIT_BYTES
|
||||
? err(
|
||||
new AnalysisResourceConstraintError(
|
||||
operation,
|
||||
"capacity",
|
||||
"The derived state is larger than data/add loads, so it cannot be served",
|
||||
{ max_state_bytes: STATE_INTAKE_LIMIT_BYTES, state_bytes: bytes },
|
||||
{
|
||||
retryable: false,
|
||||
remediationAction:
|
||||
"Capture fewer or smaller responses, or serve the environment as replay only.",
|
||||
},
|
||||
),
|
||||
)
|
||||
: ok(state);
|
||||
};
|
||||
|
||||
const unavailable = (error: AnalysisError): PreparedState => {
|
||||
const { code, message } = projectAnalysisError(error);
|
||||
return {
|
||||
@@ -281,20 +232,19 @@ export const prepareServedState = (
|
||||
} = {},
|
||||
): PreparedState => {
|
||||
const operation = options.operation ?? ENVIRONMENT_SERVE_OPERATION;
|
||||
const name = options.name ?? source.manifest.name;
|
||||
// The read catalog depends only on the manifest and name, so checking it first spares deriving a state that cannot be served.
|
||||
const reads = servedCatalog(source, { writes: new Map() }, name, operation);
|
||||
if (!reads.ok) return unavailable(reads.error);
|
||||
// The state catalog depends only on the manifest and name, so checking it first spares deriving a state that cannot be served.
|
||||
const catalog = servedCatalog(
|
||||
source,
|
||||
true,
|
||||
options.name ?? source.manifest.name,
|
||||
operation,
|
||||
);
|
||||
if (!catalog.ok) return unavailable(catalog.error);
|
||||
const derived = deriveEnvironmentState(source, captureValue, operation);
|
||||
if (!derived.ok) return unavailable(derived.error);
|
||||
const admitted = admitServedState(derived.value, operation);
|
||||
if (!admitted.ok) return unavailable(admitted.error);
|
||||
// Served write tools join the catalog, so the whole catalog is admitted again.
|
||||
const catalog = servedCatalog(source, derived.value, name, operation);
|
||||
return catalog.ok
|
||||
return derived.ok
|
||||
? {
|
||||
state: derived.value,
|
||||
report: { served: true, evidence_id: derived.value.evidenceId },
|
||||
}
|
||||
: unavailable(catalog.error);
|
||||
: unavailable(derived.error);
|
||||
};
|
||||
@@ -267,27 +267,6 @@ describe("list answers", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("answers an unfiltered call from a paging-only line of the parent its path arguments name", () => {
|
||||
const books = servingFrom([
|
||||
["GET", "/api/orgs/1/books?limit=50", [{ id: 1 }, { id: 2 }]],
|
||||
["GET", "/api/orgs/2/books?limit=50", [{ id: 3 }]],
|
||||
["GET", "/api/orgs/3/books?limit=50", [{ id: 4 }]],
|
||||
]);
|
||||
expect(answer(books, "get_orgs_books_by_id", { id: "2" })).toMatchObject({
|
||||
body: [{ id: 3 }],
|
||||
replay: { record_ordinal: 1, exact_match: false },
|
||||
state: {
|
||||
source: "state",
|
||||
collection: "/api/orgs/2/books",
|
||||
membership: "all",
|
||||
listed: 1,
|
||||
},
|
||||
});
|
||||
expect(answer(books, "get_orgs_books_by_id", { id: "9" })).toMatchObject({
|
||||
state: { source: "recording", reason: "unrecorded-request" },
|
||||
});
|
||||
});
|
||||
|
||||
it("refuses a list whose member lacks a field every recording carried, never answering a partial body", () => {
|
||||
const serving = servingOf();
|
||||
load(serving, (entities) =>
|
||||
|
||||
@@ -2,7 +2,6 @@ import { stateBodyJson } from "../../domain/analysisView/EnvironmentStateAccumul
|
||||
import {
|
||||
collectionPathOf,
|
||||
listItems,
|
||||
missingRequiredFields,
|
||||
renderStateEntity,
|
||||
renderStateList,
|
||||
replaceAt,
|
||||
@@ -15,11 +14,6 @@ import {
|
||||
type StateListView,
|
||||
type StateView,
|
||||
} from "../../domain/analysisView/environmentStateRules.js";
|
||||
import {
|
||||
pageBound,
|
||||
pagesOnly,
|
||||
} from "../../domain/analysisView/environmentStatePaging.js";
|
||||
import { isAbsence } from "../../domain/analysisView/environmentStateWriteChecks.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import {
|
||||
AnalysisInputError,
|
||||
@@ -51,11 +45,7 @@ import { environmentToolArguments } from "./environmentTools.js";
|
||||
export type StateProvenance =
|
||||
| {
|
||||
readonly source: "recording";
|
||||
readonly reason:
|
||||
| RecordedViewReason
|
||||
| "no-view"
|
||||
| "unrecorded-request"
|
||||
| "page-exceeded";
|
||||
readonly reason: RecordedViewReason | "no-view" | "unrecorded-request";
|
||||
}
|
||||
| {
|
||||
readonly source: "state";
|
||||
@@ -74,16 +64,8 @@ export type StateProvenance =
|
||||
}
|
||||
| {
|
||||
readonly source: "state";
|
||||
readonly view: "document" | "document-write";
|
||||
readonly view: "document";
|
||||
readonly path: string;
|
||||
}
|
||||
| {
|
||||
readonly source: "state";
|
||||
/** A write applied to the state, or the recorded answer for an entity it does not hold. */
|
||||
readonly view: "create" | "update" | "delete" | "missing";
|
||||
readonly type: string;
|
||||
readonly collection: string;
|
||||
readonly id: string;
|
||||
};
|
||||
|
||||
/** A replay answer that also says whether the state or a recording produced it. */
|
||||
@@ -97,7 +79,7 @@ export interface StateServing {
|
||||
readonly store: EnvironmentStateStore;
|
||||
}
|
||||
|
||||
export const refused = (
|
||||
const refused = (
|
||||
tool: EnvironmentManifestTool,
|
||||
userMessage: string,
|
||||
): AnalysisError =>
|
||||
@@ -112,9 +94,12 @@ const shownEntity = (
|
||||
entity: StateEntity,
|
||||
volatile: ReadonlySet<string>,
|
||||
): Result<JsonObject, AnalysisError> => {
|
||||
const missing = missingRequiredFields(entity, view, volatile);
|
||||
const shown = renderStateEntity(entity, view, volatile);
|
||||
const missing = view.required_fields.filter(
|
||||
(name) => !Object.hasOwn(shown, name),
|
||||
);
|
||||
return missing.length === 0
|
||||
? ok(renderStateEntity(entity, view, volatile))
|
||||
? ok(shown)
|
||||
: err(
|
||||
refused(
|
||||
tool,
|
||||
@@ -137,12 +122,11 @@ const fromRecording = (
|
||||
};
|
||||
|
||||
/** A recorded body the state renders into, usable exactly when extraction could use it. */
|
||||
export const recordedJson = (
|
||||
exchange: RecordedExchange,
|
||||
): JsonValue | undefined => stateBodyJson(exchange.contentType, exchange);
|
||||
const recordedJson = (exchange: RecordedExchange): JsonValue | undefined =>
|
||||
stateBodyJson(exchange.contentType, exchange);
|
||||
|
||||
/** The rendered body on the base record's status, content type and provenance; never cut. */
|
||||
export const stateAnswer = (
|
||||
const stateAnswer = (
|
||||
serving: StateServing,
|
||||
base: { readonly exchange: RecordedExchange; readonly exact: boolean },
|
||||
body: JsonValue,
|
||||
@@ -173,22 +157,22 @@ export const stateAnswer = (
|
||||
};
|
||||
|
||||
/** One collection segment an item call names: a template literal, or a supplied path value. */
|
||||
export interface CallSegment {
|
||||
interface CallSegment {
|
||||
readonly text: string;
|
||||
readonly supplied: boolean;
|
||||
}
|
||||
|
||||
/** An item call's collection segments and identity value, kept as supplied rather than joined and re-split. */
|
||||
export interface ItemCall {
|
||||
interface ItemCall {
|
||||
readonly segments: readonly CallSegment[];
|
||||
readonly id: string;
|
||||
}
|
||||
|
||||
/** A call's path segments: template literals and supplied values, or undefined when a path argument is missing. */
|
||||
export const callSegments = (
|
||||
/** What an item call names, or undefined when a path argument is missing. */
|
||||
const itemCall = (
|
||||
tool: EnvironmentManifestTool,
|
||||
args: ReplayArguments,
|
||||
): CallSegment[] | undefined => {
|
||||
): ItemCall | undefined => {
|
||||
const names = environmentToolArguments(tool).path;
|
||||
const params = new Map(tool.params.map((param) => [param.index, param]));
|
||||
const segments: CallSegment[] = [];
|
||||
@@ -205,23 +189,12 @@ export const callSegments = (
|
||||
if (typeof value !== "string") return undefined;
|
||||
segments.push({ text: value, supplied: true });
|
||||
}
|
||||
return segments;
|
||||
};
|
||||
|
||||
/** What an item call names, or undefined when a path argument is missing. */
|
||||
export const itemCall = (
|
||||
tool: EnvironmentManifestTool,
|
||||
args: ReplayArguments,
|
||||
): ItemCall | undefined => {
|
||||
const segments = callSegments(tool, args);
|
||||
const last = segments?.pop();
|
||||
return segments !== undefined && last?.supplied === true
|
||||
? { segments, id: last.text }
|
||||
: undefined;
|
||||
const last = segments.pop();
|
||||
return last?.supplied === true ? { segments, id: last.text } : undefined;
|
||||
};
|
||||
|
||||
/** How many supplied segments a held collection matches verbatim, when it matches the call under replay's path rule. */
|
||||
export const collectionMatch = (
|
||||
const collectionMatch = (
|
||||
collection: string,
|
||||
segments: readonly CallSegment[],
|
||||
): number | undefined => {
|
||||
@@ -238,7 +211,7 @@ export const collectionMatch = (
|
||||
};
|
||||
|
||||
/** The entity an item call names, preferring verbatim collection spellings as replay does; the id is tried percent-decoded, then as written. */
|
||||
export const namedEntity = (
|
||||
const namedEntity = (
|
||||
store: EnvironmentStateStore,
|
||||
type: string,
|
||||
call: ItemCall,
|
||||
@@ -259,60 +232,7 @@ export const namedEntity = (
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/** The answer for an entity the state does not hold: the tool's latest recorded 400, 404 or 410, else a tool error. */
|
||||
export const missingAnswer = (
|
||||
serving: StateServing,
|
||||
tool: EnvironmentManifestTool,
|
||||
missing: {
|
||||
readonly type: string;
|
||||
readonly collection: string;
|
||||
readonly id: string;
|
||||
},
|
||||
): Result<StateAnswer, AnalysisError> => {
|
||||
// A read also answers with a 2xx JSON null, the absence the write gate accepts; a write never does, as a 2xx would
|
||||
// report success for an entity that is not there.
|
||||
const nullAbsence = tool.method === "GET";
|
||||
const latestAbsence = (ordinals: readonly number[]) =>
|
||||
ordinals
|
||||
.map((ordinal) => serving.source.exchanges.get(ordinal))
|
||||
.filter(
|
||||
(exchange): exchange is RecordedExchange =>
|
||||
exchange !== undefined &&
|
||||
(isAbsence(exchange.status) ||
|
||||
(nullAbsence &&
|
||||
exchange.status !== null &&
|
||||
exchange.status >= 200 &&
|
||||
exchange.status < 300 &&
|
||||
recordedJson(exchange) === null)),
|
||||
)
|
||||
.at(-1);
|
||||
// The tool's own recorded absence first, else that of an item view of the type.
|
||||
const itemTools = new Set(
|
||||
[...serving.store.state.views.values()].flatMap((view) =>
|
||||
view.shape === "item" && view.type === missing.type ? [view.tool] : [],
|
||||
),
|
||||
);
|
||||
const recorded =
|
||||
latestAbsence(tool.record_ordinals) ??
|
||||
latestAbsence(
|
||||
serving.source.manifest.tools
|
||||
.filter((candidate) => itemTools.has(candidate.name))
|
||||
.flatMap((candidate) => candidate.record_ordinals),
|
||||
);
|
||||
if (recorded === undefined)
|
||||
return err(
|
||||
refused(
|
||||
tool,
|
||||
`The state holds no ${missing.type} entity ${missing.id} in ${missing.collection}.`,
|
||||
),
|
||||
);
|
||||
return ok({
|
||||
...replayAnswer(serving.source, recorded, false),
|
||||
state: { source: "state", view: "missing", ...missing },
|
||||
});
|
||||
};
|
||||
|
||||
/** An item call renders the named entity into the recorded envelope; it never invents an entity; one the state does not hold answers the tool's recorded absence. */
|
||||
/** An item call renders the named entity into the recorded envelope; it never invents an entity. */
|
||||
const answerItem = (
|
||||
serving: StateServing,
|
||||
tool: EnvironmentManifestTool,
|
||||
@@ -325,11 +245,12 @@ const answerItem = (
|
||||
return err(refused(tool, `Tool ${tool.name} needs every path argument.`));
|
||||
const entity = namedEntity(store, view.type, call);
|
||||
if (entity === undefined)
|
||||
return missingAnswer(serving, tool, {
|
||||
type: view.type,
|
||||
collection: `/${call.segments.map(({ text }) => text).join("/")}`,
|
||||
id: call.id,
|
||||
});
|
||||
return err(
|
||||
refused(
|
||||
tool,
|
||||
`The state holds no ${view.type} entity ${call.id} in /${call.segments.map(({ text }) => text).join("/")}.`,
|
||||
),
|
||||
);
|
||||
const shown = shownEntity(
|
||||
tool,
|
||||
view,
|
||||
@@ -376,46 +297,18 @@ const checkedLine = (
|
||||
return selectReplayExchange(tool, lines, args)?.exchange;
|
||||
};
|
||||
|
||||
/** For a list of a whole collection, a call that supplies no query argument answers from a checked line of the collection its path arguments name that only pages it, chosen as replay chooses; a requested page answers from its recording. */
|
||||
const unfilteredLine = (
|
||||
source: EnvironmentSource,
|
||||
tool: EnvironmentManifestTool,
|
||||
checked: readonly number[],
|
||||
args: ReplayArguments,
|
||||
): RecordedExchange | undefined => {
|
||||
const query = environmentToolArguments(tool).query;
|
||||
const segments = callSegments(tool, args);
|
||||
if (
|
||||
segments === undefined ||
|
||||
[...query.values()].some((argument) => args[argument] !== undefined)
|
||||
)
|
||||
return undefined;
|
||||
const lines = new Map<number, RecordedExchange>();
|
||||
for (const ordinal of checked) {
|
||||
const exchange = source.exchanges.get(ordinal);
|
||||
if (
|
||||
exchange !== undefined &&
|
||||
pagesOnly(exchange.query ?? undefined) &&
|
||||
collectionMatch(collectionPathOf(exchange, false), segments) !== undefined
|
||||
)
|
||||
lines.set(ordinal, exchange);
|
||||
}
|
||||
return selectReplayExchange(tool, lines, args)?.exchange;
|
||||
};
|
||||
|
||||
type StateDocumentView = Extract<StateView, { shape: "document" }>;
|
||||
|
||||
/** One tool call: the tool, the arguments it was called with, and whether they name the answering line exactly. */
|
||||
/** One tool call: the tool and the arguments it was called with. */
|
||||
interface ToolCall {
|
||||
readonly tool: EnvironmentManifestTool;
|
||||
readonly args: ReplayArguments;
|
||||
readonly exact: boolean;
|
||||
}
|
||||
|
||||
/** A list call renders the members of the checked line it names from the state. */
|
||||
const answerList = (
|
||||
serving: StateServing,
|
||||
{ tool, args, exact }: ToolCall,
|
||||
{ tool, args }: ToolCall,
|
||||
view: StateListView,
|
||||
base: RecordedExchange,
|
||||
): Result<StateAnswer, AnalysisError> => {
|
||||
@@ -427,16 +320,6 @@ const answerList = (
|
||||
return fromRecording(source, tool, args, "body-unusable");
|
||||
const collection = collectionPathOf(base, false);
|
||||
const type = store.type(view.type);
|
||||
// A requested page the collection has outgrown is answered as recorded, never with more items than it asked for.
|
||||
const size = exact
|
||||
? pageBound(base.query ?? undefined, recorded.length)
|
||||
: undefined;
|
||||
if (
|
||||
view.membership === "all" &&
|
||||
size !== undefined &&
|
||||
store.inCollection(view.type, collection).length > size
|
||||
)
|
||||
return fromRecording(source, tool, args, "page-exceeded");
|
||||
const members: readonly StateEntity[] =
|
||||
view.membership === "all"
|
||||
? store.inCollection(view.type, collection)
|
||||
@@ -457,7 +340,7 @@ const answerList = (
|
||||
}
|
||||
return stateAnswer(
|
||||
serving,
|
||||
{ exchange: base, exact },
|
||||
{ exchange: base, exact: true },
|
||||
renderStateList(view, envelope, items),
|
||||
{
|
||||
source: "state",
|
||||
@@ -493,27 +376,16 @@ const answerCheckedLine = (
|
||||
view: StateListView | StateDocumentView,
|
||||
args: ReplayArguments,
|
||||
): Result<StateAnswer, AnalysisError> => {
|
||||
const named = checkedLine(
|
||||
const base = checkedLine(
|
||||
serving.source,
|
||||
tool,
|
||||
view.checked_record_ordinals,
|
||||
args,
|
||||
);
|
||||
const base =
|
||||
named ??
|
||||
(view.shape === "list" && view.membership === "all"
|
||||
? unfilteredLine(serving.source, tool, view.checked_record_ordinals, args)
|
||||
: undefined);
|
||||
if (base === undefined)
|
||||
return fromRecording(serving.source, tool, args, "unrecorded-request");
|
||||
// An unfiltered call answers on a line it did not name, so its replay match is not exact.
|
||||
return view.shape === "list"
|
||||
? answerList(
|
||||
serving,
|
||||
{ tool, args, exact: named !== undefined },
|
||||
view,
|
||||
base,
|
||||
)
|
||||
? answerList(serving, { tool, args }, view, base)
|
||||
: answerDocument(serving, tool, base);
|
||||
};
|
||||
|
||||
@@ -536,3 +408,18 @@ export const answerFromState = (
|
||||
return answerCheckedLine(serving, tool, view, args);
|
||||
}
|
||||
};
|
||||
|
||||
/** How a served environment answers one tool call. */
|
||||
export type EnvironmentToolAnswer = (
|
||||
tool: EnvironmentManifestTool,
|
||||
args: ReplayArguments,
|
||||
) => Result<ReplayResult, AnalysisError>;
|
||||
|
||||
/** The answer a served environment gives: replay without a state store, the live state with one. */
|
||||
export const environmentToolAnswer = (
|
||||
source: EnvironmentSource,
|
||||
store: EnvironmentStateStore | undefined,
|
||||
): EnvironmentToolAnswer =>
|
||||
store === undefined
|
||||
? (tool, args) => replayEnvironmentTool(source, tool, args)
|
||||
: (tool, args) => answerFromState({ source, store }, tool, args);
|
||||
@@ -1,50 +0,0 @@
|
||||
import { collectionPathOf } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import {
|
||||
callSegments,
|
||||
collectionMatch,
|
||||
type StateServing,
|
||||
} from "./environmentStateAnswer.js";
|
||||
import type { ReplayArguments } from "./environmentReplay.js";
|
||||
import type { EnvironmentManifestTool } from "./environmentSource.js";
|
||||
|
||||
/** The collections a create may go to: those the store knows, held ones first, then those the type's checked list lines read, so an empty one keeps its spelling. */
|
||||
const creatableCollectionsOf = (
|
||||
serving: StateServing,
|
||||
type: string,
|
||||
): readonly string[] => {
|
||||
const listed: string[] = [];
|
||||
for (const view of serving.store.state.views.values())
|
||||
if (view.shape === "list" && view.type === type)
|
||||
for (const ordinal of view.checked_record_ordinals) {
|
||||
const exchange = serving.source.exchanges.get(ordinal);
|
||||
if (exchange !== undefined)
|
||||
listed.push(collectionPathOf(exchange, false));
|
||||
}
|
||||
return [...new Set([...serving.store.knownCollectionsOf(type), ...listed])];
|
||||
};
|
||||
|
||||
/** The collection a create writes to: a known collection the call's path matches, else the path as supplied. */
|
||||
export const createdCollection = (
|
||||
serving: StateServing,
|
||||
tool: EnvironmentManifestTool,
|
||||
args: ReplayArguments,
|
||||
type: string,
|
||||
): string | undefined => {
|
||||
const segments = callSegments(tool, args);
|
||||
if (segments === undefined || segments.some(({ text }) => text.length === 0))
|
||||
return undefined;
|
||||
// The spelling the call supplies verbatim wins, as item calls choose their entity, then the first known: held ones lead.
|
||||
let held: string | undefined;
|
||||
let verbatim = -1;
|
||||
for (const collection of creatableCollectionsOf(serving, type)) {
|
||||
const matched = collectionMatch(collection, segments);
|
||||
if (matched !== undefined && matched > verbatim) {
|
||||
held = collection;
|
||||
verbatim = matched;
|
||||
}
|
||||
}
|
||||
return (
|
||||
held ??
|
||||
`/${segments.map(({ text, supplied }) => (supplied ? encodeURIComponent(text) : text)).join("/")}`
|
||||
);
|
||||
};
|
||||
@@ -7,7 +7,6 @@ import {
|
||||
same,
|
||||
scalarText,
|
||||
segmentsOf,
|
||||
type JsonObject,
|
||||
type StateDocument,
|
||||
type StateEntity,
|
||||
type StateType,
|
||||
@@ -17,11 +16,9 @@ import {
|
||||
type AnalysisInputIssue,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import { canonicalJson } from "../../domain/comparisonSemantics.js";
|
||||
import { parseExactJson } from "../../domain/exactJsonNumber.js";
|
||||
import { projectInputIssues } from "../../domain/inputIssueProjection.js";
|
||||
import {
|
||||
jsonValueSchema,
|
||||
jsonValueValidationIssue,
|
||||
ownedJsonValueSchema,
|
||||
type JsonValue,
|
||||
} from "../../domain/jsonValue.js";
|
||||
@@ -198,18 +195,6 @@ const prototypeMemberPath = (root: unknown): Path | undefined => {
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/** Whether `data/add` accepts a value as a `fields` member or a document's `value`: bounded nesting, no `__proto__` member, and numbers exact as `data/get` writes them. */
|
||||
export const isLoadableValue = (value: JsonValue): boolean =>
|
||||
jsonValueValidationIssue(value) === undefined &&
|
||||
prototypeMemberPath(value) === undefined &&
|
||||
parseExactJson(JSON.stringify(value)) !== undefined;
|
||||
|
||||
/** The first member of an object `data/add` would refuse as `fields`, if any: a `__proto__` member, or one whose value does not load. */
|
||||
export const unloadableMember = (object: JsonObject): string | undefined =>
|
||||
Object.keys(object).find(
|
||||
(member) => member === "__proto__" || !isLoadableValue(object[member]!),
|
||||
);
|
||||
|
||||
/** The first schema problem, located without naming the caller's data. */
|
||||
const schemaProblem = (
|
||||
issues: readonly z.core.$ZodIssue[],
|
||||
|
||||
@@ -2,12 +2,7 @@ import type {
|
||||
StateEntity,
|
||||
StateType,
|
||||
} from "../../domain/analysisView/environmentStateRules.js";
|
||||
import { nextSequence } from "../../domain/analysisView/environmentStateWriteValues.js";
|
||||
import {
|
||||
AnalysisInputError,
|
||||
AnalysisResourceConstraintError,
|
||||
} from "../../domain/analysisErrorCore.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
|
||||
import { jsonParts } from "../../domain/jsonSerialization.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import { compositeKey } from "../../domain/unicodeCodePointOrder.js";
|
||||
@@ -20,38 +15,11 @@ import {
|
||||
type StateContent,
|
||||
} from "./environmentStateDocument.js";
|
||||
|
||||
/** Maximum logical budget for collection spellings added by served creates since reset/load. */
|
||||
export const ENVIRONMENT_COLLECTION_HISTORY_LIMIT_BYTES = 1024 * 1024;
|
||||
|
||||
const collectionHistoryCost = (
|
||||
state: EnvironmentState,
|
||||
type: string,
|
||||
collection: string,
|
||||
): number => {
|
||||
// Account key bytes without building temporary key strings or a Set.
|
||||
let sequenceBytes = 0;
|
||||
for (const plan of state.writes.values()) {
|
||||
if (!plan.served || plan.target !== type) continue;
|
||||
for (const rule of plan.rules) {
|
||||
if (rule.rule === "sequence" && rule.scope === "collection")
|
||||
sequenceBytes +=
|
||||
64 +
|
||||
Buffer.byteLength(plan.tool, "utf8") +
|
||||
1 +
|
||||
Buffer.byteLength(rule.field, "utf8") +
|
||||
1 +
|
||||
Buffer.byteLength(collection, "utf8");
|
||||
else if (rule.rule === "identity" && rule.generator === "sequence")
|
||||
sequenceBytes +=
|
||||
64 +
|
||||
Buffer.byteLength(type, "utf8") +
|
||||
1 +
|
||||
Buffer.byteLength(rule.field, "utf8") +
|
||||
1 +
|
||||
Buffer.byteLength(collection, "utf8");
|
||||
}
|
||||
}
|
||||
return 64 + Buffer.byteLength(collection, "utf8") + sequenceBytes;
|
||||
/** The UTF-8 bytes a value's JSON takes, counted part by part so it is never one string. */
|
||||
const jsonBytes = (value: unknown): number => {
|
||||
let bytes = 0;
|
||||
for (const part of jsonParts(value)) bytes += Buffer.byteLength(part, "utf8");
|
||||
return bytes;
|
||||
};
|
||||
|
||||
/** What `data/get` reports: the current state as a `rea-state/v1` document. */
|
||||
@@ -65,45 +33,10 @@ export interface StateSnapshot {
|
||||
};
|
||||
}
|
||||
|
||||
/** The UTF-8 bytes a value's JSON takes, counted part by part so it is never one string. */
|
||||
const jsonBytes = (value: unknown): number => {
|
||||
let bytes = 0;
|
||||
for (const part of jsonParts(value)) bytes += Buffer.byteLength(part, "utf8");
|
||||
return bytes;
|
||||
};
|
||||
|
||||
/** The bytes JSON writes for these array items and the commas between them. */
|
||||
const itemsBytes = (items: readonly unknown[]): number =>
|
||||
items.reduce<number>((sum, item) => sum + jsonBytes(item), 0) +
|
||||
Math.max(0, items.length - 1);
|
||||
|
||||
/** What `data/get` reports for some content of a state. */
|
||||
const snapshotOf = (
|
||||
state: EnvironmentState,
|
||||
entities: readonly StateEntity[],
|
||||
documents: readonly HeldDocument[],
|
||||
): StateSnapshot => ({
|
||||
rea_state: {
|
||||
version: 1,
|
||||
state_evidence_id: state.evidenceId,
|
||||
types: state.types,
|
||||
entities,
|
||||
documents,
|
||||
},
|
||||
});
|
||||
|
||||
/** What `data/get` writes for a state as the Evidence holds it, in UTF-8 bytes, summed item by item. */
|
||||
export const initialStateBytes = (state: EnvironmentState): number =>
|
||||
jsonBytes(snapshotOf(state, [], [])) +
|
||||
itemsBytes(state.entities) +
|
||||
itemsBytes(state.documents);
|
||||
|
||||
/**
|
||||
* The live state one served environment answers from, shared by every MCP
|
||||
* session and the data plane. Mutations are synchronous and swap the whole
|
||||
* state, so every answer reads one consistent state. A mutation that would take
|
||||
* what `data/get` reports past the size `data/add` loads is refused, so that
|
||||
* report always loads back.
|
||||
* state, so every answer reads one consistent state.
|
||||
*/
|
||||
export class EnvironmentStateStore {
|
||||
#entities: ReadonlyMap<string, StateEntity> = new Map();
|
||||
@@ -112,44 +45,20 @@ export class EnvironmentStateStore {
|
||||
#documents: ReadonlyMap<string, HeldDocument> = new Map();
|
||||
#content: StateContent = { entities: [], documents: [] };
|
||||
readonly #types: ReadonlyMap<string, StateType>;
|
||||
readonly #clockStart: number;
|
||||
/** What `data/get` writes for the current state, in UTF-8 bytes; kept by each mutation rather than reserialized. */
|
||||
#bytes = 0;
|
||||
/** The bytes of `data/get` with no entities or documents. */
|
||||
readonly #frameBytes: number;
|
||||
readonly #initialBytes: number;
|
||||
/** Served writes since the state was last reset or loaded: the clock's seconds and the generators' counter. */
|
||||
#writes = 0;
|
||||
/** The largest value each sequence has generated since the last reset or load, so a deleted entity's value is not reused. */
|
||||
readonly #sequences = new Map<string, number>();
|
||||
/** The entities the state held when it was last reset or loaded, whose values no sequence reuses either. */
|
||||
#started: readonly StateEntity[] = [];
|
||||
/** Baseline and post-start spellings kept known after their entities are deleted. */
|
||||
readonly #knownCollections = new Map<string, Set<string>>();
|
||||
/** Logical UTF-8 key bytes plus a 64-byte allowance per collection and potential sequence-key record. */
|
||||
#collectionHistoryBytes = 0;
|
||||
/** The collections of the Evidence's own state, which stay known after any load. */
|
||||
#evidenceCollections: ReadonlyMap<string, readonly string[]> = new Map();
|
||||
|
||||
constructor(readonly state: EnvironmentState) {
|
||||
this.#types = new Map(
|
||||
state.types.map((type) => [type.collection_template, type]),
|
||||
);
|
||||
this.#clockStart =
|
||||
state.writeClock === null ? 0 : Date.parse(state.writeClock);
|
||||
this.#frameBytes = jsonBytes(snapshotOf(state, [], []));
|
||||
this.#initialBytes = initialStateBytes(state);
|
||||
this.reset();
|
||||
}
|
||||
|
||||
/** Restore the initial state the Evidence holds; the clock and counter start again. */
|
||||
/** Restore the initial state the Evidence holds. */
|
||||
reset(): void {
|
||||
this.#install(this.state, this.#initialBytes);
|
||||
this.#evidenceCollections = this.#typeCollections;
|
||||
this.#restart();
|
||||
this.#install(this.state);
|
||||
}
|
||||
|
||||
/** Validate a caller-owned document and copy its JSON values before retaining them; the clock and counter start again. */
|
||||
/** Validate a caller-owned document and copy its JSON values before retaining them. */
|
||||
replace(document: unknown): Result<void, AnalysisInputError> {
|
||||
return this.#replace(document, false);
|
||||
}
|
||||
@@ -166,8 +75,7 @@ export class EnvironmentStateStore {
|
||||
const content = parseStateDocument(this.state, document, owned);
|
||||
if (!content.ok) return content;
|
||||
// data/get writes members a document may omit, so the state it reports must still be one data/add loads.
|
||||
const bytes = this.#contentBytes(content.value);
|
||||
if (bytes > STATE_INTAKE_LIMIT_BYTES)
|
||||
if (jsonBytes(this.#snapshotOf(content.value)) > STATE_INTAKE_LIMIT_BYTES)
|
||||
return err(
|
||||
new AnalysisInputError(
|
||||
ENVIRONMENT_SERVE_OPERATION,
|
||||
@@ -184,200 +92,10 @@ export class EnvironmentStateStore {
|
||||
],
|
||||
),
|
||||
);
|
||||
this.#install(content.value, bytes);
|
||||
this.#restart();
|
||||
this.#install(content.value);
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
/** The write a served call would be: its number since the last reset or load, and its clock instant, one second past the previous applied write. */
|
||||
pendingWrite(): { readonly count: number; readonly clockMs: number } {
|
||||
const count = this.#writes + 1;
|
||||
return { count, clockMs: this.#clockStart + count * 1000 };
|
||||
}
|
||||
|
||||
/** Count an applied write and keep the sequence values it took, so a refused call takes none. */
|
||||
commitWrite(sequences: ReadonlyMap<string, number>): void {
|
||||
this.#writes += 1;
|
||||
for (const [key, value] of sequences) {
|
||||
const taken = this.#sequences.get(key);
|
||||
// No earlier value leaves this one in place, including when it is negative.
|
||||
this.#sequences.set(
|
||||
key,
|
||||
taken === undefined ? value : Math.max(taken, value),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/** The next value of a sequence: the next held integer, else one past the later of that and the last value taken. */
|
||||
sequenceAfter(key: string, held: number | undefined): number {
|
||||
return nextSequence(held, this.#sequences.get(key));
|
||||
}
|
||||
|
||||
#restart(): void {
|
||||
this.#writes = 0;
|
||||
this.#sequences.clear();
|
||||
this.#started = this.#content.entities;
|
||||
this.#knownCollections.clear();
|
||||
this.#collectionHistoryBytes = 0;
|
||||
for (const [type, collections] of this.#typeCollections)
|
||||
for (const collection of collections)
|
||||
this.#rememberCollection(type, collection);
|
||||
for (const [type, collections] of this.#evidenceCollections)
|
||||
for (const collection of collections)
|
||||
this.#rememberCollection(type, collection);
|
||||
}
|
||||
|
||||
#rememberCollection(type: string, collection: string): void {
|
||||
const known = this.#knownCollections.get(type) ?? new Set<string>();
|
||||
known.add(collection);
|
||||
this.#knownCollections.set(type, known);
|
||||
}
|
||||
|
||||
/** The entities of a type the state held when it was last reset or loaded. */
|
||||
startedWith(type: string): readonly StateEntity[] {
|
||||
return this.#started.filter((entity) => entity.type === type);
|
||||
}
|
||||
|
||||
/** Every held entity of a type, in state order. */
|
||||
ofType(type: string): readonly StateEntity[] {
|
||||
return this.#content.entities.filter((entity) => entity.type === type);
|
||||
}
|
||||
|
||||
/** Add an entity at `index` of its collection's order, unless the state would outgrow what `data/add` loads. */
|
||||
insert(
|
||||
entity: StateEntity,
|
||||
index: number,
|
||||
): Result<void, AnalysisResourceConstraintError> {
|
||||
const entities = this.#content.entities;
|
||||
const bytes = this.#grown(
|
||||
jsonBytes(entity) + (entities.length > 0 ? 1 : 0),
|
||||
);
|
||||
if (!bytes.ok) return bytes;
|
||||
const known = this.#knownCollections.get(entity.type);
|
||||
const newCollectionBytes =
|
||||
known?.has(entity.collection) === true
|
||||
? 0
|
||||
: collectionHistoryCost(this.state, entity.type, entity.collection);
|
||||
if (newCollectionBytes > 0) {
|
||||
const historyBytes = this.#collectionHistoryBytes + newCollectionBytes;
|
||||
if (historyBytes > ENVIRONMENT_COLLECTION_HISTORY_LIMIT_BYTES)
|
||||
return err(
|
||||
new AnalysisResourceConstraintError(
|
||||
ENVIRONMENT_SERVE_OPERATION,
|
||||
"capacity",
|
||||
"Served collection history would exceed its deterministic accounting budget",
|
||||
{
|
||||
max_collection_history_bytes:
|
||||
ENVIRONMENT_COLLECTION_HISTORY_LIMIT_BYTES,
|
||||
collection_history_bytes: historyBytes,
|
||||
},
|
||||
{
|
||||
retryable: false,
|
||||
remediationAction:
|
||||
"Reset the served state or load a smaller state to clear generated collection history, then use fewer distinct collection paths.",
|
||||
},
|
||||
),
|
||||
);
|
||||
}
|
||||
const members = this.inCollection(entity.type, entity.collection);
|
||||
const anchor =
|
||||
index < members.length
|
||||
? entities.indexOf(members[index]!)
|
||||
: members.length > 0
|
||||
? entities.indexOf(members.at(-1)!) + 1
|
||||
: entities.length;
|
||||
this.#install(
|
||||
{
|
||||
...this.#content,
|
||||
entities: [
|
||||
...entities.slice(0, anchor),
|
||||
entity,
|
||||
...entities.slice(anchor),
|
||||
],
|
||||
},
|
||||
bytes.value,
|
||||
);
|
||||
if (newCollectionBytes > 0) {
|
||||
this.#rememberCollection(entity.type, entity.collection);
|
||||
this.#collectionHistoryBytes += newCollectionBytes;
|
||||
}
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
/** Replace a held entity's values in place, unless the state would outgrow what `data/add` loads. */
|
||||
update(entity: StateEntity): Result<void, AnalysisResourceConstraintError> {
|
||||
const key = compositeKey([entity.type, entity.collection, entity.id]);
|
||||
const held = this.#entities.get(key);
|
||||
if (held === undefined) return ok(undefined);
|
||||
const bytes = this.#grown(jsonBytes(entity) - jsonBytes(held));
|
||||
if (!bytes.ok) return bytes;
|
||||
this.#install(
|
||||
{
|
||||
...this.#content,
|
||||
entities: this.#content.entities.map((candidate) =>
|
||||
candidate === held ? entity : candidate,
|
||||
),
|
||||
},
|
||||
bytes.value,
|
||||
);
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
remove(entity: StateEntity): void {
|
||||
const held = this.#entities.get(
|
||||
compositeKey([entity.type, entity.collection, entity.id]),
|
||||
);
|
||||
if (held === undefined) return;
|
||||
const entities = this.#content.entities.filter(
|
||||
(candidate) => candidate !== held,
|
||||
);
|
||||
this.#install(
|
||||
{ ...this.#content, entities },
|
||||
this.#bytes - jsonBytes(held) - (entities.length > 0 ? 1 : 0),
|
||||
);
|
||||
}
|
||||
|
||||
/** Set a held document's value, unless the state would outgrow what `data/add` loads; a value that is not the Evidence's cites no record. */
|
||||
setDocument(
|
||||
tool: string,
|
||||
value: JsonValue,
|
||||
): Result<void, AnalysisResourceConstraintError> {
|
||||
const held = this.#documents.get(tool);
|
||||
if (held === undefined) return ok(undefined);
|
||||
const written = { tool: held.tool, path: held.path, value };
|
||||
const bytes = this.#grown(jsonBytes(written) - jsonBytes(held));
|
||||
if (!bytes.ok) return bytes;
|
||||
this.#install(
|
||||
{
|
||||
...this.#content,
|
||||
documents: this.#content.documents.map((document) =>
|
||||
document === held ? written : document,
|
||||
),
|
||||
},
|
||||
bytes.value,
|
||||
);
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
/** The state's bytes after a change of `delta`, or a refusal when a change that grows it would pass the intake limit. */
|
||||
#grown(delta: number): Result<number, AnalysisResourceConstraintError> {
|
||||
const bytes = this.#bytes + delta;
|
||||
if (delta <= 0 || bytes <= STATE_INTAKE_LIMIT_BYTES) return ok(bytes);
|
||||
return err(
|
||||
new AnalysisResourceConstraintError(
|
||||
ENVIRONMENT_SERVE_OPERATION,
|
||||
"capacity",
|
||||
"The write would take the served state past the size data/add loads",
|
||||
{ max_state_bytes: STATE_INTAKE_LIMIT_BYTES, state_bytes: bytes },
|
||||
{
|
||||
retryable: false,
|
||||
remediationAction:
|
||||
"Delete entities, or restore a smaller state with data/reset or data/add.",
|
||||
},
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
type(collectionTemplate: string): StateType | undefined {
|
||||
return this.#types.get(collectionTemplate);
|
||||
}
|
||||
@@ -400,43 +118,28 @@ export class EnvironmentStateStore {
|
||||
return this.#typeCollections.get(type) ?? [];
|
||||
}
|
||||
|
||||
/** Every collection spelling in the active, baseline, Evidence, or generated history, with active entities first. */
|
||||
knownCollectionsOf(type: string): readonly string[] {
|
||||
const known = new Set([
|
||||
...this.collectionsOf(type),
|
||||
...(this.#knownCollections.get(type) ?? []),
|
||||
]);
|
||||
return [...known];
|
||||
}
|
||||
|
||||
document(tool: string): HeldDocument | undefined {
|
||||
return this.#documents.get(tool);
|
||||
}
|
||||
|
||||
snapshot(): StateSnapshot {
|
||||
return snapshotOf(
|
||||
this.state,
|
||||
this.#content.entities,
|
||||
this.#content.documents,
|
||||
);
|
||||
return this.#snapshotOf(this.#content);
|
||||
}
|
||||
|
||||
/** What `data/get` writes for the current state, in UTF-8 bytes. */
|
||||
byteLength(): number {
|
||||
return this.#bytes;
|
||||
#snapshotOf(content: StateContent): StateSnapshot {
|
||||
return {
|
||||
rea_state: {
|
||||
version: 1,
|
||||
state_evidence_id: this.state.evidenceId,
|
||||
types: this.state.types,
|
||||
entities: content.entities,
|
||||
documents: content.documents,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** What `data/get` writes for some content, summed item by item so the whole state is never one string. */
|
||||
#contentBytes(content: StateContent): number {
|
||||
return (
|
||||
this.#frameBytes +
|
||||
itemsBytes(content.entities) +
|
||||
itemsBytes(content.documents)
|
||||
);
|
||||
}
|
||||
|
||||
/** Build current-state indexes without copying baseline collections into generated history. */
|
||||
#install(content: StateContent, bytes: number): void {
|
||||
/** Build every index before assigning any, so a reader never sees half a state. */
|
||||
#install(content: StateContent): void {
|
||||
const entities = new Map<string, StateEntity>();
|
||||
const collections = new Map<string, StateEntity[]>();
|
||||
const typeCollections = new Map<string, string[]>();
|
||||
@@ -464,6 +167,5 @@ export class EnvironmentStateStore {
|
||||
content.documents.map((document) => [document.tool, document]),
|
||||
);
|
||||
this.#content = content;
|
||||
this.#bytes = bytes;
|
||||
}
|
||||
}
|
||||
@@ -1,941 +0,0 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
servedFrom,
|
||||
type Served,
|
||||
} from "../../../tests/fixtures/environmentStateServing.js";
|
||||
import {
|
||||
BOOK_INIT_ROWS,
|
||||
BOOK_NOT_FOUND,
|
||||
BOOK_ROWS,
|
||||
DELETE_NOT_FOUND,
|
||||
FIRST_BOOK,
|
||||
SECOND_BOOK,
|
||||
THIRD_BOOK,
|
||||
book,
|
||||
bookClock,
|
||||
renamedBook,
|
||||
type Book,
|
||||
} from "../../../tests/fixtures/environmentStateWrites.js";
|
||||
import type { TimedCaptureRow } from "../../../tests/fixtures/analysisView.js";
|
||||
import { seededUuid } from "../../domain/analysisView/environmentStateWriteValues.js";
|
||||
import type { ReplayArguments } from "./environmentReplay.js";
|
||||
import { servedCatalog } from "./environmentState.js";
|
||||
import {
|
||||
environmentToolArguments,
|
||||
environmentToolDefinitions,
|
||||
} from "./environmentTools.js";
|
||||
|
||||
/** The book the first served create of "New Book" makes: the next id, one second past the served clock's start. */
|
||||
const NEW_BOOK = book(3, "New Book", bookClock(13));
|
||||
const CREATE_NEW_BOOK = { body: { name: "New Book" } };
|
||||
|
||||
const entityOf = (books: Served, id: string) =>
|
||||
books.store.snapshot().rea_state.entities.find((entity) => entity.id === id);
|
||||
|
||||
/** Load what data/get reports back through data/add: the state must come back byte for byte. */
|
||||
const expectReloadsUnchanged = (books: Served): void => {
|
||||
const before = books.snapshot();
|
||||
expect(books.store.replace(JSON.parse(before)).ok).toBe(true);
|
||||
expect(books.snapshot()).toBe(before);
|
||||
};
|
||||
|
||||
describe("served creates", () => {
|
||||
it("refuses an exhausted generated-value collision search without changing state", () => {
|
||||
const existing = {
|
||||
id: "123e4567-e89b-42d3-a456-426614174000",
|
||||
ref: "223e4567-e89b-42d3-a456-426614174000",
|
||||
name: "Existing",
|
||||
};
|
||||
const created = {
|
||||
id: "323e4567-e89b-42d3-a456-426614174000",
|
||||
ref: "423e4567-e89b-42d3-a456-426614174000",
|
||||
name: "Created",
|
||||
};
|
||||
const books = servedFrom([
|
||||
["GET", "/api/items", [existing]],
|
||||
["GET", `/api/items/${existing.id}`, existing],
|
||||
["POST", "/api/items", created, 201, { name: "Created" }],
|
||||
["GET", `/api/items/${created.id}`, created],
|
||||
["GET", "/api/items", [existing, created]],
|
||||
]);
|
||||
const plan = books.store.state.writes.get("post_items");
|
||||
if (plan?.served !== true)
|
||||
throw new Error("Expected post_items to be served");
|
||||
const rule = plan.rules.find((candidate) => candidate.rule === "unique");
|
||||
expect(rule).toMatchObject({ field: "ref", rule: "unique" });
|
||||
if (rule?.rule !== "unique") throw new Error("Expected a unique ref rule");
|
||||
const candidates = Array.from({ length: 65 }, (_, attempt) =>
|
||||
seededUuid(
|
||||
`${books.state.evidenceId}:${plan.tool}:${rule.field}:1:${String(attempt)}`,
|
||||
created.ref,
|
||||
),
|
||||
);
|
||||
const snapshot = books.store.snapshot();
|
||||
const collisions = candidates.map((ref, index) => ({
|
||||
type: "/api/items",
|
||||
collection: "/api/items",
|
||||
id: `collision-${String(index)}`,
|
||||
fields: { id: `collision-${String(index)}`, ref, name: "Collision" },
|
||||
view_fields: [],
|
||||
record_ordinals: [],
|
||||
}));
|
||||
const loaded = {
|
||||
...snapshot,
|
||||
rea_state: {
|
||||
...snapshot.rea_state,
|
||||
entities: [...snapshot.rea_state.entities, ...collisions],
|
||||
},
|
||||
};
|
||||
const replaced = books.store.replace(loaded);
|
||||
if (!replaced.ok) throw new Error(JSON.stringify(replaced.error));
|
||||
const before = books.snapshot();
|
||||
const pending = books.store.pendingWrite();
|
||||
|
||||
const refusal = books.refusal(plan.tool, { body: { name: "Created" } });
|
||||
|
||||
expect(JSON.stringify(refusal)).toContain(
|
||||
"Could not generate a unique value for ref after 64 attempts",
|
||||
);
|
||||
expect(books.snapshot()).toBe(before);
|
||||
expect(books.store.pendingWrite()).toEqual(pending);
|
||||
});
|
||||
it("creates a book with the next id, the served clock and a derived slug, which later reads show", () => {
|
||||
const books = servedFrom();
|
||||
const created = books.answer("post_books", CREATE_NEW_BOOK);
|
||||
expect(created).toMatchObject({
|
||||
status: 201,
|
||||
content_type: "application/json",
|
||||
state: {
|
||||
source: "state",
|
||||
view: "create",
|
||||
type: "/api/books",
|
||||
collection: "/api/books",
|
||||
id: "3",
|
||||
},
|
||||
});
|
||||
expect(created.body).toEqual(NEW_BOOK);
|
||||
expect(books.answer("get_books_by_id", { id: "3" })).toMatchObject({
|
||||
status: 200,
|
||||
body: NEW_BOOK,
|
||||
state: { source: "state", view: "item", id: "3" },
|
||||
});
|
||||
expect(books.answer("get_books")).toMatchObject({
|
||||
body: [FIRST_BOOK, SECOND_BOOK, NEW_BOOK],
|
||||
state: { source: "state", view: "list", membership: "all", listed: 3 },
|
||||
});
|
||||
});
|
||||
|
||||
it("reports a created book in data/get without record citations and loads it back unchanged", () => {
|
||||
const books = servedFrom();
|
||||
books.answer("post_books", CREATE_NEW_BOOK);
|
||||
expect(entityOf(books, "3")).toEqual({
|
||||
type: "/api/books",
|
||||
collection: "/api/books",
|
||||
id: "3",
|
||||
fields: NEW_BOOK,
|
||||
view_fields: [],
|
||||
record_ordinals: [],
|
||||
});
|
||||
expectReloadsUnchanged(books);
|
||||
expect(books.answer("get_books_by_id", { id: "3" }).body).toEqual(NEW_BOOK);
|
||||
});
|
||||
});
|
||||
|
||||
describe("served updates and deletes", () => {
|
||||
it("keeps an entity's identity when the application ignored the id an update sent", () => {
|
||||
const renamed = renamedBook(FIRST_BOOK, "Renamed Book", 3);
|
||||
const books = servedFrom([
|
||||
...BOOK_INIT_ROWS,
|
||||
[
|
||||
"PATCH",
|
||||
"/api/books/1",
|
||||
renamed,
|
||||
200,
|
||||
{ id: 999, name: "Renamed Book" },
|
||||
],
|
||||
["GET", "/api/books/1", renamed],
|
||||
]);
|
||||
const answer = books.answer("patch_books_by_id", {
|
||||
id: "2",
|
||||
body: { id: 999, name: "Second Edition" },
|
||||
});
|
||||
expect(answer.body).toMatchObject({ id: 2, name: "Second Edition" });
|
||||
expect(books.answer("get_books_by_id", { id: "2" }).body).toMatchObject({
|
||||
id: 2,
|
||||
});
|
||||
});
|
||||
|
||||
const SECOND_EDITION = renamedBook(SECOND_BOOK, "Second Edition", 13);
|
||||
const RENAME_SECOND = { id: "2", body: { name: "Second Edition" } };
|
||||
|
||||
it("changes only the sent members, the slug that follows them and the clock", () => {
|
||||
const books = servedFrom();
|
||||
expect(books.answer("patch_books_by_id", RENAME_SECOND)).toMatchObject({
|
||||
status: 200,
|
||||
body: SECOND_EDITION,
|
||||
state: { source: "state", view: "update", id: "2" },
|
||||
});
|
||||
expect(books.answer("get_books_by_id", { id: "2" }).body).toEqual(
|
||||
SECOND_EDITION,
|
||||
);
|
||||
expect(books.answer("get_books").body).toEqual([
|
||||
FIRST_BOOK,
|
||||
SECOND_EDITION,
|
||||
]);
|
||||
});
|
||||
|
||||
it("cites no records for an updated book in data/get, so it loads back unchanged", () => {
|
||||
const books = servedFrom();
|
||||
books.answer("patch_books_by_id", RENAME_SECOND);
|
||||
expect(entityOf(books, "2")).toMatchObject({
|
||||
fields: SECOND_EDITION,
|
||||
record_ordinals: [],
|
||||
});
|
||||
expectReloadsUnchanged(books);
|
||||
});
|
||||
|
||||
it("removes a deleted book; a later read answers the recorded 404 as a missing entity", () => {
|
||||
const books = servedFrom();
|
||||
expect(books.answer("delete_books_by_id", { id: "2" })).toMatchObject({
|
||||
status: 204,
|
||||
body: null,
|
||||
state: { source: "state", view: "delete", id: "2" },
|
||||
});
|
||||
expect(books.answer("get_books_by_id", { id: "2" })).toMatchObject({
|
||||
status: 404,
|
||||
body: BOOK_NOT_FOUND,
|
||||
state: {
|
||||
source: "state",
|
||||
view: "missing",
|
||||
type: "/api/books",
|
||||
collection: "/api/books",
|
||||
id: "2",
|
||||
},
|
||||
});
|
||||
expect(books.answer("get_books").body).toEqual([FIRST_BOOK]);
|
||||
});
|
||||
|
||||
it("answers an update or delete of a missing book with a recorded 404, else refuses it", () => {
|
||||
const books = servedFrom();
|
||||
const before = books.snapshot();
|
||||
// The update recorded no absence, so it answers the item view's; the delete answers its own.
|
||||
expect(
|
||||
books.answer("patch_books_by_id", { id: "99", body: { name: "Ghost" } }),
|
||||
).toMatchObject({
|
||||
status: 404,
|
||||
body: BOOK_NOT_FOUND,
|
||||
state: { view: "missing", id: "99" },
|
||||
});
|
||||
expect(books.answer("delete_books_by_id", { id: "99" })).toMatchObject({
|
||||
status: 404,
|
||||
body: DELETE_NOT_FOUND,
|
||||
state: { view: "missing", id: "99" },
|
||||
});
|
||||
expect(books.snapshot()).toBe(before);
|
||||
const unrecorded = servedFrom(
|
||||
BOOK_ROWS.filter(([, , , status]) => status !== 404),
|
||||
);
|
||||
for (const [name, args] of [
|
||||
["patch_books_by_id", { id: "99", body: { name: "Ghost" } }],
|
||||
["delete_books_by_id", { id: "99" }],
|
||||
] as const)
|
||||
expect(unrecorded.refusal(name, args)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
message: expect.stringContaining(
|
||||
"holds no /api/books entity 99 in /api/books",
|
||||
),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("refusals", () => {
|
||||
it("refuses a body the capture does not support and leaves the state unchanged", () => {
|
||||
const books = servedFrom();
|
||||
const before = books.snapshot();
|
||||
const refused: [string, ReplayArguments, string][] = [
|
||||
[
|
||||
"post_books",
|
||||
{ body: { name: "Extra", color: "red" } },
|
||||
"Member color is not one the capture showed stored",
|
||||
],
|
||||
["post_books", { body: {} }, 'lacks required members "name"'],
|
||||
[
|
||||
"post_books",
|
||||
{ body: { name: 42 } },
|
||||
"Member name must be a JSON string",
|
||||
],
|
||||
["post_books", { body: ["New Book"] }, "takes a JSON object body"],
|
||||
[
|
||||
"patch_books_by_id",
|
||||
{ id: "1", body: "New Book" },
|
||||
"takes a JSON object body",
|
||||
],
|
||||
["patch_books_by_id", { id: "1", body: {} }, "sends none of the members"],
|
||||
[
|
||||
"patch_books_by_id",
|
||||
{ id: "1", body: { title: "New" } },
|
||||
"Member title is not one",
|
||||
],
|
||||
];
|
||||
for (const [name, args, message] of refused) {
|
||||
expect(books.refusal(name, args)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
message: expect.stringContaining(message),
|
||||
});
|
||||
expect(books.snapshot()).toBe(before);
|
||||
}
|
||||
});
|
||||
|
||||
it("accepts a member the application stored differently only with the value the capture sent", () => {
|
||||
const bound = (entry: Book, format: string): Book => ({ ...entry, format });
|
||||
const books = servedFrom([
|
||||
[
|
||||
"GET",
|
||||
"/api/books",
|
||||
[bound(FIRST_BOOK, "PAPERBACK"), bound(SECOND_BOOK, "PAPERBACK")],
|
||||
],
|
||||
["GET", "/api/books/1", bound(FIRST_BOOK, "PAPERBACK")],
|
||||
["GET", "/api/books/2", bound(SECOND_BOOK, "PAPERBACK")],
|
||||
[
|
||||
"POST",
|
||||
"/api/books",
|
||||
bound(THIRD_BOOK, "HARDCOVER"),
|
||||
201,
|
||||
{ name: "Third Book", format: "hardcover" },
|
||||
],
|
||||
["GET", "/api/books/3", bound(THIRD_BOOK, "HARDCOVER")],
|
||||
]);
|
||||
const before = books.snapshot();
|
||||
expect(
|
||||
books.refusal("post_books", {
|
||||
body: { name: "Soft", format: "softcover" },
|
||||
}),
|
||||
).toMatchObject({
|
||||
message: expect.stringContaining(
|
||||
"Member format is accepted only with the value the capture sent",
|
||||
),
|
||||
});
|
||||
expect(books.snapshot()).toBe(before);
|
||||
expect(
|
||||
books.answer("post_books", {
|
||||
body: { name: "Hard", format: "hardcover" },
|
||||
}).body,
|
||||
).toMatchObject({ id: 3, name: "Hard", format: "HARDCOVER" });
|
||||
});
|
||||
|
||||
it("holds a member a later update first stored to the JSON type it stored", () => {
|
||||
const shelved = { ...SECOND_BOOK, tags: ["a"] };
|
||||
const renamed = renamedBook(FIRST_BOOK, "Renamed Book", 3);
|
||||
const retagged = { ...shelved, tags: ["x"], updated_at: bookClock(5) };
|
||||
// The rename the plan was built from shows no tags, which the first book never had.
|
||||
const books = servedFrom([
|
||||
["GET", "/api/books", [FIRST_BOOK, shelved]],
|
||||
["GET", "/api/books/1", FIRST_BOOK],
|
||||
["GET", "/api/books/2", shelved],
|
||||
["PATCH", "/api/books/1", renamed, 200, { name: "Renamed Book" }],
|
||||
["GET", "/api/books/1", renamed],
|
||||
["PATCH", "/api/books/2", retagged, 200, { tags: ["x"] }],
|
||||
["GET", "/api/books/2", retagged],
|
||||
]);
|
||||
expect(books.state.writes.get("patch_books_by_id")).toMatchObject({
|
||||
served: true,
|
||||
body: { kinds: [{ member: "tags", kind: "array" }] },
|
||||
});
|
||||
const before = books.snapshot();
|
||||
expect(
|
||||
books.refusal("patch_books_by_id", { id: "1", body: { tags: "x" } }),
|
||||
).toMatchObject({
|
||||
message: expect.stringContaining("Member tags must be a JSON array"),
|
||||
});
|
||||
expect(books.snapshot()).toBe(before);
|
||||
books.answer("patch_books_by_id", { id: "1", body: { tags: ["y"] } });
|
||||
expect(entityOf(books, "1")?.fields).toMatchObject({ tags: ["y"] });
|
||||
});
|
||||
|
||||
it("generates the same values after a refused call as without it", () => {
|
||||
const refusedFirst = servedFrom();
|
||||
refusedFirst.refusal("post_books", { body: {} });
|
||||
refusedFirst.refusal("patch_books_by_id", {
|
||||
id: "1",
|
||||
body: { title: "New" },
|
||||
});
|
||||
expect(refusedFirst.answer("post_books", CREATE_NEW_BOOK).body).toEqual(
|
||||
servedFrom().answer("post_books", CREATE_NEW_BOOK).body,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
/** Items whose `rank` is a collection sequence, with a recorded create at `recorded`. */
|
||||
const rankedItems = (initial: readonly [number, number], recorded: number) => {
|
||||
const item = (id: number, rank: number) => ({
|
||||
id,
|
||||
name: `n${String(id)}`,
|
||||
rank,
|
||||
});
|
||||
const [first, second] = initial;
|
||||
return servedFrom([
|
||||
["GET", "/api/items", [item(1, first), item(2, second)]],
|
||||
["GET", "/api/items/1", item(1, first)],
|
||||
["GET", "/api/items/2", item(2, second)],
|
||||
["POST", "/api/items", item(3, recorded), 201, { name: "n3" }],
|
||||
["GET", "/api/items/3", item(3, recorded)],
|
||||
["GET", "/api/items", [item(1, first), item(2, second), item(3, recorded)]],
|
||||
]);
|
||||
};
|
||||
|
||||
describe("generated values", () => {
|
||||
it("answers the same calls identically after a reset, never reusing a deleted id", () => {
|
||||
const books = servedFrom();
|
||||
const session = () => {
|
||||
const answers = [
|
||||
books.answer("post_books", CREATE_NEW_BOOK),
|
||||
books.answer("patch_books_by_id", {
|
||||
id: "3",
|
||||
body: { name: "Newer Book" },
|
||||
}),
|
||||
books.answer("delete_books_by_id", { id: "3" }),
|
||||
books.answer("post_books", { body: { name: "Another Book" } }),
|
||||
books.answer("get_books"),
|
||||
];
|
||||
return { answers, text: JSON.stringify([answers, books.snapshot()]) };
|
||||
};
|
||||
const first = session();
|
||||
books.store.reset();
|
||||
expect(session().text).toBe(first.text);
|
||||
expect(first.answers[3]?.body).toMatchObject({
|
||||
id: 4,
|
||||
name: "Another Book",
|
||||
});
|
||||
});
|
||||
|
||||
it("serves the next rank when a collection sequence runs through negative values", () => {
|
||||
const acrossZero = rankedItems([-2, -1], 0);
|
||||
expect(
|
||||
acrossZero.answer("post_items", { body: { name: "n4" } }).body,
|
||||
).toMatchObject({ id: 3, rank: 0 });
|
||||
expect(
|
||||
acrossZero.answer("post_items", { body: { name: "n5" } }).body,
|
||||
).toMatchObject({ id: 4, rank: 1 });
|
||||
// The value after a stored negative rank stays on that sequence.
|
||||
const negative = rankedItems([-4, -3], -2);
|
||||
expect(
|
||||
negative.answer("post_items", { body: { name: "n4" } }).body,
|
||||
).toMatchObject({ id: 3, rank: -2 });
|
||||
expect(
|
||||
negative.answer("post_items", { body: { name: "n5" } }).body,
|
||||
).toMatchObject({ id: 4, rank: -1 });
|
||||
});
|
||||
|
||||
it("restarts the clock and the sequences when data/add loads a state", () => {
|
||||
const books = servedFrom();
|
||||
const initial: unknown = JSON.parse(books.snapshot());
|
||||
const created = books.answer("post_books", CREATE_NEW_BOOK);
|
||||
books.answer("post_books", { body: { name: "Another Book" } });
|
||||
expect(books.store.replace(initial).ok).toBe(true);
|
||||
// Had the sequence and clock run on, this create would take id 5 one second later.
|
||||
expect(books.answer("post_books", CREATE_NEW_BOOK).body).toEqual(
|
||||
created.body,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("served catalog", () => {
|
||||
it("adds only the served write tools, saying what each does to the state", () => {
|
||||
const { source, state } = servedFrom();
|
||||
const catalog = servedCatalog(source, state, "books");
|
||||
if (!catalog.ok) throw catalog.error;
|
||||
const described = new Map(
|
||||
catalog.value.tools.map((tool) => [tool.name, tool]),
|
||||
);
|
||||
// The export write targets no state type, so it is not served.
|
||||
expect([...described.keys()].sort()).toEqual([
|
||||
"delete_books_by_id",
|
||||
"get_books",
|
||||
"get_books_by_id",
|
||||
"patch_books_by_id",
|
||||
"post_books",
|
||||
]);
|
||||
expect(described.get("post_books")?.description).toContain(
|
||||
'creates an entity of /api/books in the served environment state (body members "name"; required: "name"), modelled on the recorded calls',
|
||||
);
|
||||
expect(described.get("patch_books_by_id")?.description).toContain(
|
||||
'changes the entity the path names in the served environment state (body members "name"; send at least one)',
|
||||
);
|
||||
expect(described.get("delete_books_by_id")?.description).toContain(
|
||||
"removes the entity the path names from the served environment state",
|
||||
);
|
||||
expect(described.get("post_books")?.annotations.readOnlyHint).toBe(false);
|
||||
expect(described.get("get_books")?.description).toContain(
|
||||
"answers from the served environment state where the state backs this tool",
|
||||
);
|
||||
});
|
||||
|
||||
it("quotes the member names a write's description lists and samples a long one", () => {
|
||||
const long = "n".repeat(90_000);
|
||||
const item = (id: number, name: string) => ({
|
||||
id,
|
||||
name,
|
||||
"x, y": name,
|
||||
[long]: name,
|
||||
});
|
||||
const { source, state } = servedFrom([
|
||||
["GET", "/api/books", [item(1, "A")]],
|
||||
["GET", "/api/books/1", item(1, "A")],
|
||||
[
|
||||
"POST",
|
||||
"/api/books",
|
||||
item(2, "B"),
|
||||
201,
|
||||
{ name: "B", "x, y": "B", [long]: "B" },
|
||||
],
|
||||
["GET", "/api/books/2", item(2, "B")],
|
||||
["GET", "/api/books", [item(1, "A"), item(2, "B")]],
|
||||
]);
|
||||
const catalog = servedCatalog(source, state, "books");
|
||||
if (!catalog.ok) throw catalog.error;
|
||||
const created = catalog.value.tools.find(
|
||||
(tool) => tool.name === "post_books",
|
||||
);
|
||||
expect(created?.description).toContain(
|
||||
'body members "name", "x, y", … (2/3 shown; full list in state Evidence)',
|
||||
);
|
||||
});
|
||||
|
||||
it("serves every manifest tool unchanged in replay mode", () => {
|
||||
const { source } = servedFrom();
|
||||
const replay = servedCatalog(source, undefined, "books");
|
||||
const definitions = environmentToolDefinitions(source, "books");
|
||||
if (!replay.ok || !definitions.ok) throw new Error("Expected a catalog");
|
||||
expect(replay.value).toEqual({
|
||||
tools: definitions.value,
|
||||
dataPlane: "none",
|
||||
});
|
||||
expect(replay.value.tools.map(({ name }) => name)).toContain(
|
||||
"post_exports",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("paged list lines", () => {
|
||||
it("answers an argument-less list from the state on a line that only pages, and another page from its recording", () => {
|
||||
const paged = servedFrom([
|
||||
["GET", "/api/books?limit=50", [FIRST_BOOK, SECOND_BOOK]],
|
||||
...BOOK_INIT_ROWS.slice(1),
|
||||
["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
|
||||
["GET", "/api/books/3", THIRD_BOOK],
|
||||
["GET", "/api/books?limit=50", [FIRST_BOOK, SECOND_BOOK, THIRD_BOOK]],
|
||||
]);
|
||||
const created = paged.answer("post_books", CREATE_NEW_BOOK).body;
|
||||
expect(paged.answer("get_books")).toMatchObject({
|
||||
body: [FIRST_BOOK, SECOND_BOOK, created],
|
||||
replay: { record_ordinal: 0, exact_match: false },
|
||||
state: { source: "state", view: "list", listed: 3 },
|
||||
});
|
||||
expect(paged.answer("get_books", { limit: "10" })).toMatchObject({
|
||||
body: [FIRST_BOOK, SECOND_BOOK],
|
||||
state: { source: "recording", reason: "unrecorded-request" },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("path values", () => {
|
||||
const orgBook = (id: number, org: string, name: string): Book => ({
|
||||
id,
|
||||
org,
|
||||
name,
|
||||
});
|
||||
const ALPHA = orgBook(1, "my org1", "Alpha");
|
||||
const BETA = orgBook(2, "acme2", "Beta");
|
||||
const GAMMA = orgBook(3, "my org1", "Gamma");
|
||||
const DELTA = orgBook(4, "acme2", "Delta");
|
||||
/** Books under two organisations, one whose name the paths percent-encode, and a create under each. */
|
||||
const ROWS: readonly TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/my%20org1/books", [ALPHA]],
|
||||
["GET", "/api/orgs/acme2/books", [BETA]],
|
||||
["GET", "/api/orgs/my%20org1/books/1", ALPHA],
|
||||
["GET", "/api/orgs/acme2/books/2", BETA],
|
||||
["POST", "/api/orgs/my%20org1/books", GAMMA, 201, { name: "Gamma" }],
|
||||
["GET", "/api/orgs/my%20org1/books", [ALPHA, GAMMA]],
|
||||
["POST", "/api/orgs/acme2/books", DELTA, 201, { name: "Delta" }],
|
||||
["GET", "/api/orgs/acme2/books", [BETA, DELTA]],
|
||||
];
|
||||
|
||||
it("stores a parent path value decoded, as the gate learned it, whichever spelling the call uses", () => {
|
||||
for (const spelling of ["my%20org1", "my org1"]) {
|
||||
const books = servedFrom(ROWS);
|
||||
const tool = books.source.manifest.tools.find(
|
||||
(candidate) => candidate.method === "POST",
|
||||
)!;
|
||||
expect(books.state.writes.get(tool.name)).toMatchObject({
|
||||
served: true,
|
||||
rules: expect.arrayContaining([
|
||||
expect.objectContaining({ field: "org", rule: "path" }),
|
||||
]),
|
||||
});
|
||||
const [org] = environmentToolArguments(tool).path.values();
|
||||
books.answer(tool.name, { [org!]: spelling, body: { name: "Epsilon" } });
|
||||
const created = books.store
|
||||
.snapshot()
|
||||
.rea_state.entities.find(
|
||||
(entity) => entity.fields["name"] === "Epsilon",
|
||||
);
|
||||
expect(created).toMatchObject({
|
||||
collection: "/api/orgs/my%20org1/books",
|
||||
fields: { org: "my org1" },
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("collections a create chooses", () => {
|
||||
const orgBook = (id: number, name: string): Book => ({ id, name });
|
||||
/** One organisation listed under two spellings and another organisation, each with a create. */
|
||||
const ROWS: readonly TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/%61cme1/books", [orgBook(1, "Alpha")]],
|
||||
["GET", "/api/orgs/acme1/books", [orgBook(2, "Beta")]],
|
||||
["GET", "/api/orgs/beta2/books", [orgBook(3, "Gamma")]],
|
||||
[
|
||||
"POST",
|
||||
"/api/orgs/acme1/books",
|
||||
orgBook(4, "Delta"),
|
||||
201,
|
||||
{ name: "Delta" },
|
||||
],
|
||||
["GET", "/api/orgs/acme1/books", [orgBook(2, "Beta"), orgBook(4, "Delta")]],
|
||||
[
|
||||
"POST",
|
||||
"/api/orgs/beta2/books",
|
||||
orgBook(5, "Epsilon"),
|
||||
201,
|
||||
{ name: "Epsilon" },
|
||||
],
|
||||
[
|
||||
"GET",
|
||||
"/api/orgs/beta2/books",
|
||||
[orgBook(3, "Gamma"), orgBook(5, "Epsilon")],
|
||||
],
|
||||
];
|
||||
|
||||
it("creates in the collection the call spells verbatim, as item calls choose their entity", () => {
|
||||
const books = servedFrom(ROWS);
|
||||
const tool = books.source.manifest.tools.find(
|
||||
(candidate) => candidate.method === "POST",
|
||||
)!;
|
||||
expect(books.state.writes.get(tool.name)).toMatchObject({ served: true });
|
||||
const [org] = environmentToolArguments(tool).path.values();
|
||||
books.answer(tool.name, { [org!]: "acme1", body: { name: "Zeta" } });
|
||||
expect(
|
||||
books.store
|
||||
.snapshot()
|
||||
.rea_state.entities.find((entity) => entity.fields["name"] === "Zeta")
|
||||
?.collection,
|
||||
).toBe("/api/orgs/acme1/books");
|
||||
});
|
||||
|
||||
it("refuses an empty parent segment before storing an unreadable collection", () => {
|
||||
const books = servedFrom(ROWS);
|
||||
const tool = books.source.manifest.tools.find(
|
||||
(candidate) => candidate.method === "POST",
|
||||
)!;
|
||||
const [org] = environmentToolArguments(tool).path.values();
|
||||
const before = books.snapshot();
|
||||
expect(
|
||||
books.refusal(tool.name, { [org!]: "", body: { name: "Invalid" } }),
|
||||
).toMatchObject({
|
||||
code: "invalid_request",
|
||||
message: expect.stringContaining("path argument to be non-empty"),
|
||||
});
|
||||
expect(books.snapshot()).toBe(before);
|
||||
expect(
|
||||
books.store.knownCollectionsOf("/api/orgs/{org}/books"),
|
||||
).not.toContain("/api/orgs//books");
|
||||
});
|
||||
});
|
||||
|
||||
describe("missing entities after a delete", () => {
|
||||
it("answers with a recorded 2xx null, which the gate also took as an absence", () => {
|
||||
const books = servedFrom([
|
||||
...BOOK_INIT_ROWS,
|
||||
["DELETE", "/api/books/2", "", 204],
|
||||
["GET", "/api/books/2", null],
|
||||
]);
|
||||
expect(books.state.writes.get("delete_books_by_id")).toMatchObject({
|
||||
served: true,
|
||||
});
|
||||
books.answer("delete_books_by_id", { id: "1" });
|
||||
expect(books.answer("get_books_by_id", { id: "1" })).toMatchObject({
|
||||
status: 200,
|
||||
body: null,
|
||||
state: { view: "missing", id: "1" },
|
||||
});
|
||||
});
|
||||
|
||||
it("never answers a write of a missing entity with success, even when the write recorded a 2xx null", () => {
|
||||
const books = servedFrom([
|
||||
...BOOK_INIT_ROWS,
|
||||
["DELETE", "/api/books/2", null, 200],
|
||||
["GET", "/api/books/2", BOOK_NOT_FOUND, 404],
|
||||
]);
|
||||
expect(books.state.writes.get("delete_books_by_id")).toMatchObject({
|
||||
served: true,
|
||||
});
|
||||
expect(books.answer("delete_books_by_id", { id: "9" })).toMatchObject({
|
||||
status: 404,
|
||||
body: BOOK_NOT_FOUND,
|
||||
state: { view: "missing", id: "9" },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("values the state started with", () => {
|
||||
it("never reuses a seeded id for a create, even after its entity is deleted first", () => {
|
||||
const books = servedFrom();
|
||||
books.answer("delete_books_by_id", { id: "2" });
|
||||
books.answer("post_books", CREATE_NEW_BOOK);
|
||||
expect(entityOf(books, "2")).toBeUndefined();
|
||||
expect(entityOf(books, "3")?.fields).toMatchObject({ name: "New Book" });
|
||||
});
|
||||
|
||||
const orgBook = (id: number, org: string, name: string): Book => ({
|
||||
id,
|
||||
org,
|
||||
name,
|
||||
});
|
||||
/** Books under two organisations, one spelled `first` in its paths, with creates and a delete the lists confirm. */
|
||||
const twoOrgs = (first: string, org: string) => {
|
||||
const alpha = orgBook(1, org, "Alpha");
|
||||
const beta = orgBook(2, "acme2", "Beta");
|
||||
const gamma = orgBook(3, org, "Gamma");
|
||||
const delta = orgBook(4, "acme2", "Delta");
|
||||
const books = servedFrom([
|
||||
["GET", `/api/orgs/${first}/books`, [alpha]],
|
||||
["GET", "/api/orgs/acme2/books", [beta]],
|
||||
["GET", `/api/orgs/${first}/books/1`, alpha],
|
||||
["GET", "/api/orgs/acme2/books/2", beta],
|
||||
["POST", `/api/orgs/${first}/books`, gamma, 201, { name: "Gamma" }],
|
||||
["GET", `/api/orgs/${first}/books`, [alpha, gamma]],
|
||||
["POST", "/api/orgs/acme2/books", delta, 201, { name: "Delta" }],
|
||||
["GET", "/api/orgs/acme2/books", [beta, delta]],
|
||||
["DELETE", `/api/orgs/${first}/books/3`, "", 204],
|
||||
["GET", `/api/orgs/${first}/books`, [alpha]],
|
||||
]);
|
||||
const toolOf = (method: string) =>
|
||||
books.source.manifest.tools.find(
|
||||
(candidate) => candidate.method === method,
|
||||
)!;
|
||||
const create = (value: string, name: string): string | undefined => {
|
||||
const tool = toolOf("POST");
|
||||
const [parent] = environmentToolArguments(tool).path.values();
|
||||
books.answer(tool.name, { [parent!]: value, body: { name } });
|
||||
return books.store
|
||||
.snapshot()
|
||||
.rea_state.entities.find((entity) => entity.fields["name"] === name)
|
||||
?.collection;
|
||||
};
|
||||
return { books, toolOf, create };
|
||||
};
|
||||
|
||||
it("creates in a collection's recorded spelling after the collection empties", () => {
|
||||
const { books, toolOf, create } = twoOrgs("my%20org1", "my org1");
|
||||
const remove = toolOf("DELETE");
|
||||
const [parent, id] = environmentToolArguments(remove).path.values();
|
||||
for (const held of ["1", "3"])
|
||||
books.answer(remove.name, { [parent!]: "my%20org1", [id!]: held });
|
||||
expect(books.store.collectionsOf("/api/orgs/{org}/books")).not.toContain(
|
||||
"/api/orgs/my%20org1/books",
|
||||
);
|
||||
expect(create("my%20org1", "Epsilon")).toBe("/api/orgs/my%20org1/books");
|
||||
});
|
||||
|
||||
it("creates in the Evidence's spelling of a collection a loaded state does not hold", () => {
|
||||
const { books, create } = twoOrgs("my%20org1", "my org1");
|
||||
const { rea_state: held } = books.store.snapshot();
|
||||
expect(
|
||||
books.store.replace({
|
||||
rea_state: {
|
||||
version: held.version,
|
||||
entities: held.entities.filter(
|
||||
(entity) => !entity.collection.includes("%20"),
|
||||
),
|
||||
documents: held.documents,
|
||||
},
|
||||
}).ok,
|
||||
).toBe(true);
|
||||
expect(create("my%20org1", "Epsilon")).toBe("/api/orgs/my%20org1/books");
|
||||
});
|
||||
|
||||
it("creates in the loaded state's collection when the Evidence spells the same one differently", () => {
|
||||
const { books, create } = twoOrgs("%61cme1", "acme1");
|
||||
const { rea_state: held } = books.store.snapshot();
|
||||
expect(
|
||||
books.store.replace({
|
||||
rea_state: {
|
||||
version: held.version,
|
||||
entities: held.entities.map((entity) =>
|
||||
entity.collection.includes("%61cme1")
|
||||
? {
|
||||
...entity,
|
||||
collection: entity.collection.replace("%61cme1", "a%63me1"),
|
||||
record_ordinals: [],
|
||||
}
|
||||
: entity,
|
||||
),
|
||||
documents: held.documents,
|
||||
},
|
||||
}).ok,
|
||||
).toBe(true);
|
||||
// Both spellings decode to the argument; the one holding books wins over the Evidence's emptied one.
|
||||
expect(create("acme1", "Epsilon")).toBe("/api/orgs/a%63me1/books");
|
||||
});
|
||||
|
||||
it("creates in the spelling of a collection the capture listed empty", () => {
|
||||
const beta = orgBook(2, "acme2", "Beta");
|
||||
const gamma = orgBook(3, "my org1", "Gamma");
|
||||
const delta = orgBook(4, "acme2", "Delta");
|
||||
// The state starts before the first create, when the encoded organisation's list was empty.
|
||||
const books = servedFrom([
|
||||
["GET", "/api/orgs/my%20org1/books", []],
|
||||
["GET", "/api/orgs/acme2/books", [beta]],
|
||||
["GET", "/api/orgs/acme2/books/2", beta],
|
||||
["POST", "/api/orgs/my%20org1/books", gamma, 201, { name: "Gamma" }],
|
||||
["GET", "/api/orgs/my%20org1/books", [gamma]],
|
||||
["POST", "/api/orgs/acme2/books", delta, 201, { name: "Delta" }],
|
||||
["GET", "/api/orgs/acme2/books", [beta, delta]],
|
||||
]);
|
||||
const tool = books.source.manifest.tools.find(
|
||||
(candidate) => candidate.method === "POST",
|
||||
)!;
|
||||
const [parent] = environmentToolArguments(tool).path.values();
|
||||
books.answer(tool.name, {
|
||||
[parent!]: "my%20org1",
|
||||
body: { name: "Epsilon" },
|
||||
});
|
||||
expect(
|
||||
books.store
|
||||
.snapshot()
|
||||
.rea_state.entities.find(
|
||||
(entity) => entity.fields["name"] === "Epsilon",
|
||||
)?.collection,
|
||||
).toBe("/api/orgs/my%20org1/books");
|
||||
});
|
||||
|
||||
it("forgets the spellings of a loaded state once reset", () => {
|
||||
const { books, create } = twoOrgs("%61cme1", "acme1");
|
||||
const { rea_state: held } = books.store.snapshot();
|
||||
// A loaded state spells the first organisation plainly; the Evidence's state spells it encoded.
|
||||
expect(
|
||||
books.store.replace({
|
||||
rea_state: {
|
||||
version: held.version,
|
||||
// Moved to another spelling, the entity cites none of the Evidence's records.
|
||||
entities: held.entities.map((entity) =>
|
||||
entity.collection.includes("%61cme1")
|
||||
? {
|
||||
...entity,
|
||||
collection: entity.collection.replace("%61cme1", "acme1"),
|
||||
record_ordinals: [],
|
||||
}
|
||||
: entity,
|
||||
),
|
||||
documents: held.documents,
|
||||
},
|
||||
}).ok,
|
||||
).toBe(true);
|
||||
books.store.reset();
|
||||
expect(create("acme1", "Epsilon")).toBe("/api/orgs/%61cme1/books");
|
||||
});
|
||||
});
|
||||
|
||||
describe("paged lines", () => {
|
||||
it("answers a requested page the collection has outgrown from its recording, and a call with no query from the state", () => {
|
||||
const books = servedFrom([
|
||||
["GET", "/api/books?limit=2", [FIRST_BOOK, SECOND_BOOK]],
|
||||
["GET", "/api/books/1", FIRST_BOOK],
|
||||
["GET", "/api/books/2", SECOND_BOOK],
|
||||
["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
|
||||
// The application answers the same page with two books; serving would not answer it from the state.
|
||||
["GET", "/api/books?limit=2", [FIRST_BOOK, SECOND_BOOK]],
|
||||
["GET", "/api/books", [FIRST_BOOK, SECOND_BOOK, THIRD_BOOK]],
|
||||
]);
|
||||
const created = books.answer("post_books", CREATE_NEW_BOOK).body as Book;
|
||||
expect(books.answer("get_books", { limit: "2" })).toMatchObject({
|
||||
body: [FIRST_BOOK, SECOND_BOOK],
|
||||
state: { source: "recording", reason: "page-exceeded" },
|
||||
});
|
||||
expect(books.answer("get_books")).toMatchObject({
|
||||
body: [FIRST_BOOK, SECOND_BOOK, created],
|
||||
state: { source: "state", membership: "all", listed: 3 },
|
||||
});
|
||||
});
|
||||
|
||||
it("answers a whole list a zero limit names from the state, as zero lists every match", () => {
|
||||
const books = servedFrom([
|
||||
["GET", "/api/books?limit=0", [FIRST_BOOK, SECOND_BOOK]],
|
||||
["GET", "/api/books/1", FIRST_BOOK],
|
||||
["GET", "/api/books/2", SECOND_BOOK],
|
||||
["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
|
||||
["GET", "/api/books/3", THIRD_BOOK],
|
||||
]);
|
||||
const created = books.answer("post_books", CREATE_NEW_BOOK).body as Book;
|
||||
expect(books.answer("get_books", { limit: "0" })).toMatchObject({
|
||||
body: [FIRST_BOOK, SECOND_BOOK, created],
|
||||
state: { source: "state", membership: "all", listed: 3 },
|
||||
});
|
||||
});
|
||||
|
||||
it("answers a zero limit a whole line recorded empty from that recording once a create fills its collection", () => {
|
||||
const alpha = { id: 1, name: "Alpha", org: "1" };
|
||||
const beta = { id: 2, name: "Beta", org: "2" };
|
||||
const books = servedFrom([
|
||||
["GET", "/api/orgs/1/books", [alpha]],
|
||||
["GET", "/api/orgs/1/books/1", alpha],
|
||||
["GET", "/api/orgs/2/books?limit=0", []],
|
||||
["POST", "/api/orgs/2/books", beta, 201, { name: "Beta" }],
|
||||
["GET", "/api/orgs/2/books/2", beta],
|
||||
]);
|
||||
books.answer("post_orgs_books_by_id", { id: "2", body: { name: "Gamma" } });
|
||||
// Only a recording that listed items under it shows that zero lists every entity.
|
||||
expect(
|
||||
books.answer("get_orgs_books_by_id", { id: "2", limit: "0" }),
|
||||
).toMatchObject({
|
||||
body: [],
|
||||
state: { source: "recording", reason: "page-exceeded" },
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps a range's bound beside a zero size its recording shows unbounded", () => {
|
||||
const books = servedFrom([
|
||||
["GET", "/api/books?limit=0&_start=0&_end=2", [FIRST_BOOK, SECOND_BOOK]],
|
||||
["GET", "/api/books/1", FIRST_BOOK],
|
||||
["GET", "/api/books/2", SECOND_BOOK],
|
||||
["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
|
||||
["GET", "/api/books/3", THIRD_BOOK],
|
||||
]);
|
||||
books.answer("post_books", CREATE_NEW_BOOK);
|
||||
expect(
|
||||
books.answer("get_books", { limit: "0", _start: "0", _end: "2" }),
|
||||
).toMatchObject({
|
||||
body: [FIRST_BOOK, SECOND_BOOK],
|
||||
state: { source: "recording", reason: "page-exceeded" },
|
||||
});
|
||||
});
|
||||
|
||||
it("answers a range the collection has outgrown from its recording", () => {
|
||||
const books = servedFrom([
|
||||
["GET", "/api/books?_start=0&_end=2", [FIRST_BOOK, SECOND_BOOK]],
|
||||
["GET", "/api/books/1", FIRST_BOOK],
|
||||
["GET", "/api/books/2", SECOND_BOOK],
|
||||
["POST", "/api/books", THIRD_BOOK, 201, { name: "Third Book" }],
|
||||
["GET", "/api/books?_start=0&_end=2", [FIRST_BOOK, SECOND_BOOK]],
|
||||
["GET", "/api/books/3", THIRD_BOOK],
|
||||
]);
|
||||
books.answer("post_books", CREATE_NEW_BOOK);
|
||||
expect(books.answer("get_books", { _start: "0", _end: "2" })).toMatchObject(
|
||||
{
|
||||
body: [FIRST_BOOK, SECOND_BOOK],
|
||||
state: { source: "recording", reason: "page-exceeded" },
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -1,535 +0,0 @@
|
||||
import {
|
||||
createdFields,
|
||||
documentBodyProblem,
|
||||
updatedEntity,
|
||||
updatedFields,
|
||||
valueAt,
|
||||
writtenDocument,
|
||||
writtenResponse,
|
||||
type GenerateValue,
|
||||
} from "../../domain/analysisView/environmentStateWriteApply.js";
|
||||
import {
|
||||
insertIndex,
|
||||
nextInteger,
|
||||
} from "../../domain/analysisView/environmentStateWriteExplain.js";
|
||||
import {
|
||||
pathValuesOf,
|
||||
type GeneratedRule,
|
||||
type StateWritePlan,
|
||||
type WriteCall,
|
||||
} from "../../domain/analysisView/environmentStateWriteRules.js";
|
||||
import {
|
||||
formatClock,
|
||||
isUuid,
|
||||
seededToken,
|
||||
seededUuid,
|
||||
spelledLike,
|
||||
} from "../../domain/analysisView/environmentStateWriteValues.js";
|
||||
import {
|
||||
isObject,
|
||||
scalarText,
|
||||
type JsonObject,
|
||||
type StateEntity,
|
||||
} from "../../domain/analysisView/environmentStateRules.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import {
|
||||
replayAnswer,
|
||||
selectReplayExchange,
|
||||
type ReplayArguments,
|
||||
} from "./environmentReplay.js";
|
||||
import type { EnvironmentManifestTool } from "./environmentSource.js";
|
||||
import {
|
||||
callSegments,
|
||||
collectionMatch,
|
||||
itemCall,
|
||||
missingAnswer,
|
||||
namedEntity,
|
||||
recordedJson,
|
||||
refused,
|
||||
stateAnswer,
|
||||
type StateAnswer,
|
||||
type StateProvenance,
|
||||
type StateServing,
|
||||
} from "./environmentStateAnswer.js";
|
||||
import { createdCollection } from "./environmentStateCollections.js";
|
||||
import {
|
||||
isLoadableValue,
|
||||
unloadableMember,
|
||||
type HeldDocument,
|
||||
} from "./environmentStateDocument.js";
|
||||
import { environmentToolArguments } from "./environmentTools.js";
|
||||
|
||||
/** Attempts at a generated UUID or token before a collision with held values is refused. */
|
||||
const GENERATION_ATTEMPTS = 64;
|
||||
|
||||
/** One served write call: the tool, its plan and its arguments. */
|
||||
interface WriteRequest {
|
||||
readonly tool: EnvironmentManifestTool;
|
||||
readonly plan: StateWritePlan;
|
||||
readonly args: ReplayArguments;
|
||||
}
|
||||
|
||||
/** The path values a call supplies, keyed by template segment as write plans name them. */
|
||||
const pathValues = (
|
||||
tool: EnvironmentManifestTool,
|
||||
args: ReplayArguments,
|
||||
): Map<string, string> => {
|
||||
const names = environmentToolArguments(tool).path;
|
||||
const template = tool.path_template
|
||||
.slice(tool.path_template.indexOf("/") + 1)
|
||||
.split("/");
|
||||
const values = new Map<string, string>();
|
||||
for (const param of tool.params) {
|
||||
const value = args[names.get(param.name) ?? param.name];
|
||||
const segment = template[param.index] ?? "";
|
||||
if (typeof value === "string") values.set(segment.slice(1, -1), value);
|
||||
}
|
||||
return values;
|
||||
};
|
||||
|
||||
/** A refusal for a value `data/add` would refuse, so `data/get` always reports a state that loads back. */
|
||||
const unloadable = (
|
||||
tool: EnvironmentManifestTool,
|
||||
holder: string,
|
||||
): AnalysisError =>
|
||||
refused(
|
||||
tool,
|
||||
`${holder} would hold a value that is not exactly representable JSON, which data/add could not load back.`,
|
||||
);
|
||||
|
||||
/** The call's body: an object, nothing, or a refusal for any other value or one holding a value the state cannot hold. */
|
||||
const callBody = ({
|
||||
tool,
|
||||
args,
|
||||
}: WriteRequest): Result<JsonObject | undefined, AnalysisError> => {
|
||||
const name = environmentToolArguments(tool).body;
|
||||
const value = name === undefined ? undefined : args[name];
|
||||
if (value === undefined) return ok(undefined);
|
||||
if (!isObject(value as JsonValue))
|
||||
return err(refused(tool, `Tool ${tool.name} takes a JSON object body.`));
|
||||
const member = unloadableMember(value as JsonObject);
|
||||
return member === undefined
|
||||
? ok(value as JsonObject)
|
||||
: err(unloadable(tool, `Member ${member}`));
|
||||
};
|
||||
|
||||
/** The entity a recorded observation shows: the values a create keeps and an update's constants. */
|
||||
const observedEntity = (
|
||||
serving: StateServing,
|
||||
plan: StateWritePlan,
|
||||
): JsonObject | undefined => {
|
||||
if (plan.observation === null) return undefined;
|
||||
const exchange = serving.source.exchanges.get(plan.observation.ordinal);
|
||||
const body = exchange === undefined ? undefined : recordedJson(exchange);
|
||||
const object =
|
||||
body === undefined ? undefined : valueAt(body, plan.observation.pointer);
|
||||
return isObject(object) ? object : undefined;
|
||||
};
|
||||
|
||||
/** One past the largest integer these entities hold in the field, or undefined when none holds one. */
|
||||
const nextHeld = (entities: readonly StateEntity[], field: string) =>
|
||||
nextInteger(
|
||||
entities.map((entity) => entity.fields),
|
||||
field,
|
||||
);
|
||||
|
||||
/**
|
||||
* Values one served write generates: its clock instant, sequences one past the largest held
|
||||
* value, and UUIDs and tokens seeded by the state Evidence, tool, field and write count,
|
||||
* skipping values the type already holds.
|
||||
*/
|
||||
const generator = (
|
||||
serving: StateServing,
|
||||
request: WriteRequest,
|
||||
target: {
|
||||
readonly name: string;
|
||||
readonly collection: readonly StateEntity[];
|
||||
readonly observed: JsonObject;
|
||||
},
|
||||
): {
|
||||
readonly generate: GenerateValue;
|
||||
readonly taken: Map<string, number>;
|
||||
} => {
|
||||
const { store } = serving;
|
||||
const write = store.pendingWrite();
|
||||
const taken = new Map<string, number>();
|
||||
const sequence = (key: string, held: number | undefined): number => {
|
||||
const value = store.sequenceAfter(key, held);
|
||||
taken.set(key, value);
|
||||
return value;
|
||||
};
|
||||
const ofType = store.ofType(request.plan.target ?? "");
|
||||
// A sequence never reuses a value the state started with, even after its entity was deleted.
|
||||
const started = store.startedWith(request.plan.target ?? "");
|
||||
const startedIn = started.filter((held) => held.collection === target.name);
|
||||
const unused = (
|
||||
field: string,
|
||||
make: (attempt: number) => string,
|
||||
): Result<string, string> => {
|
||||
const held = new Set(
|
||||
ofType.map((entity) => scalarText(entity.fields[field])),
|
||||
);
|
||||
for (let attempt = 0; attempt < GENERATION_ATTEMPTS; attempt += 1) {
|
||||
const value = make(attempt);
|
||||
if (!held.has(value)) return ok(value);
|
||||
}
|
||||
return err(
|
||||
`Could not generate a unique value for ${field} after ${String(GENERATION_ATTEMPTS)} attempts without colliding with the held state.`,
|
||||
);
|
||||
};
|
||||
const generate = (rule: GeneratedRule): Result<JsonValue, string> => {
|
||||
const exemplar = target.observed[rule.field] ?? "";
|
||||
const seed = (attempt: number) =>
|
||||
`${store.state.evidenceId}:${request.tool.name}:${rule.field}:${String(write.count)}:${String(attempt)}`;
|
||||
switch (rule.rule) {
|
||||
case "clock":
|
||||
return ok(formatClock(write.clockMs, rule.format));
|
||||
case "sequence":
|
||||
return ok(
|
||||
spelledLike(
|
||||
exemplar,
|
||||
sequence(
|
||||
`${request.tool.name} ${rule.field} ${rule.scope === "collection" ? target.name : ""}`,
|
||||
nextHeld(
|
||||
rule.scope === "collection"
|
||||
? [...target.collection, ...startedIn]
|
||||
: [...ofType, ...started],
|
||||
rule.field,
|
||||
),
|
||||
),
|
||||
),
|
||||
);
|
||||
case "unique":
|
||||
return unused(rule.field, (attempt) =>
|
||||
seededUuid(seed(attempt), String(exemplar)),
|
||||
);
|
||||
case "identity":
|
||||
if (rule.generator === "sequence")
|
||||
return ok(
|
||||
spelledLike(
|
||||
exemplar,
|
||||
sequence(
|
||||
`${request.plan.target ?? ""} ${rule.field} ${target.name}`,
|
||||
nextHeld([...target.collection, ...startedIn], rule.field),
|
||||
),
|
||||
),
|
||||
);
|
||||
return unused(rule.field, (attempt) =>
|
||||
rule.generator === "uuid" || isUuid(exemplar)
|
||||
? seededUuid(seed(attempt), String(exemplar))
|
||||
: seededToken(seed(attempt), String(exemplar)),
|
||||
);
|
||||
}
|
||||
};
|
||||
return { generate, taken };
|
||||
};
|
||||
|
||||
/** A write's answer on its recorded response's status and content type: a rendered body, none, or the recorded one. */
|
||||
const writeAnswer = (
|
||||
serving: StateServing,
|
||||
request: WriteRequest,
|
||||
rendered: { readonly fields: JsonObject | undefined; readonly id: string },
|
||||
state: StateProvenance,
|
||||
): Result<StateAnswer, AnalysisError> => {
|
||||
const { plan, tool, args } = request;
|
||||
const exchange =
|
||||
plan.response === null
|
||||
? undefined
|
||||
: serving.source.exchanges.get(plan.response.ordinal);
|
||||
if (plan.response === null || exchange === undefined)
|
||||
return err(refused(tool, `Tool ${tool.name} has no recorded response.`));
|
||||
const exact =
|
||||
selectReplayExchange(tool, new Map([[exchange.ordinal, exchange]]), args)
|
||||
?.exact === true;
|
||||
if (plan.response.shape === "recorded")
|
||||
return ok({ ...replayAnswer(serving.source, exchange, exact), state });
|
||||
const body = writtenResponse(plan.response, recordedJson(exchange), rendered);
|
||||
if (!body.ok) return err(refused(tool, body.error));
|
||||
return stateAnswer(serving, { exchange, exact }, body.value ?? null, state);
|
||||
};
|
||||
|
||||
const answerCreate = (
|
||||
serving: StateServing,
|
||||
request: WriteRequest,
|
||||
call: WriteCall,
|
||||
): Result<StateAnswer, AnalysisError> => {
|
||||
const { tool, plan } = request;
|
||||
const type = plan.target ?? "";
|
||||
const idField = serving.store.type(type)?.id_field ?? "";
|
||||
const collection = createdCollection(
|
||||
serving,
|
||||
request.tool,
|
||||
request.args,
|
||||
plan.target ?? "",
|
||||
);
|
||||
const observed = observedEntity(serving, plan);
|
||||
if (collection === undefined || observed === undefined)
|
||||
return err(
|
||||
refused(
|
||||
tool,
|
||||
`Tool ${tool.name} needs every path argument to be non-empty.`,
|
||||
),
|
||||
);
|
||||
if (plan.collection !== null && plan.collection !== collection)
|
||||
return err(
|
||||
refused(
|
||||
tool,
|
||||
`The recorded create's values are tied to ${plan.collection}; the environment models creates only there.`,
|
||||
),
|
||||
);
|
||||
const members = serving.store.inCollection(type, collection);
|
||||
const values = generator(serving, request, {
|
||||
name: collection,
|
||||
collection: members,
|
||||
observed,
|
||||
});
|
||||
// The gate learned path rules from the recorded path, decoded; the collection is this call's.
|
||||
const created = {
|
||||
...call,
|
||||
pathValues: pathValuesOf(tool.path_template, collection) ?? call.pathValues,
|
||||
};
|
||||
const fields = createdFields(plan, observed, created, values.generate);
|
||||
if (!fields.ok) return err(refused(tool, fields.error));
|
||||
const field = unloadableMember(fields.value);
|
||||
if (field !== undefined) return err(unloadable(tool, `Field ${field}`));
|
||||
const id = scalarText(fields.value[idField]);
|
||||
if (
|
||||
id === undefined ||
|
||||
serving.store.entity(type, collection, id) !== undefined
|
||||
)
|
||||
return err(
|
||||
refused(
|
||||
tool,
|
||||
`The state already holds a ${type} entity ${id ?? ""} in ${collection}.`,
|
||||
),
|
||||
);
|
||||
const state = {
|
||||
source: "state",
|
||||
view: "create",
|
||||
type,
|
||||
collection,
|
||||
id,
|
||||
} as const;
|
||||
const answer = writeAnswer(
|
||||
serving,
|
||||
request,
|
||||
{ fields: fields.value, id },
|
||||
state,
|
||||
);
|
||||
if (!answer.ok) return answer;
|
||||
const inserted = serving.store.insert(
|
||||
{
|
||||
type,
|
||||
collection,
|
||||
id,
|
||||
fields: fields.value,
|
||||
view_fields: [],
|
||||
record_ordinals: [],
|
||||
},
|
||||
insertIndex(
|
||||
plan.insert ?? { rule: "last" },
|
||||
members.map((entity) => entity.fields),
|
||||
fields.value,
|
||||
),
|
||||
);
|
||||
if (!inserted.ok) return inserted;
|
||||
serving.store.commitWrite(values.taken);
|
||||
return answer;
|
||||
};
|
||||
|
||||
/** The held entity an update or delete names, or the answer for one the state does not hold. */
|
||||
const targetEntity = (
|
||||
serving: StateServing,
|
||||
request: WriteRequest,
|
||||
): Result<StateEntity, Result<StateAnswer, AnalysisError>> => {
|
||||
const { tool, plan, args } = request;
|
||||
const call = itemCall(tool, args);
|
||||
if (call === undefined)
|
||||
return err(
|
||||
err(refused(tool, `Tool ${tool.name} needs every path argument.`)),
|
||||
);
|
||||
const entity = namedEntity(serving.store, plan.target ?? "", call);
|
||||
return entity === undefined
|
||||
? err(
|
||||
missingAnswer(serving, tool, {
|
||||
type: plan.target ?? "",
|
||||
collection: `/${call.segments.map(({ text }) => text).join("/")}`,
|
||||
id: call.id,
|
||||
}),
|
||||
)
|
||||
: ok(entity);
|
||||
};
|
||||
|
||||
const answerUpdate = (
|
||||
serving: StateServing,
|
||||
request: WriteRequest,
|
||||
call: WriteCall,
|
||||
): Result<StateAnswer, AnalysisError> => {
|
||||
const { tool, plan } = request;
|
||||
const target = targetEntity(serving, request);
|
||||
if (!target.ok) return target.error;
|
||||
const entity = target.value;
|
||||
const observed = observedEntity(serving, plan) ?? {};
|
||||
const values = generator(serving, request, {
|
||||
name: entity.collection,
|
||||
collection: serving.store.inCollection(entity.type, entity.collection),
|
||||
observed,
|
||||
});
|
||||
const written = updatedFields(
|
||||
plan,
|
||||
{ before: entity.fields, observed },
|
||||
{ ...call, id: entity.id },
|
||||
values.generate,
|
||||
);
|
||||
if (!written.ok) return err(refused(tool, written.error));
|
||||
// An increment can pass the exact integers.
|
||||
const field = unloadableMember(written.value);
|
||||
if (field !== undefined) return err(unloadable(tool, `Field ${field}`));
|
||||
const answer = writeAnswer(
|
||||
serving,
|
||||
request,
|
||||
{ fields: written.value, id: entity.id },
|
||||
{
|
||||
source: "state",
|
||||
view: "update",
|
||||
type: entity.type,
|
||||
collection: entity.collection,
|
||||
id: entity.id,
|
||||
},
|
||||
);
|
||||
if (!answer.ok) return answer;
|
||||
const updated = serving.store.update(updatedEntity(entity, written.value));
|
||||
if (!updated.ok) return updated;
|
||||
serving.store.commitWrite(values.taken);
|
||||
return answer;
|
||||
};
|
||||
|
||||
const answerDelete = (
|
||||
serving: StateServing,
|
||||
request: WriteRequest,
|
||||
): Result<StateAnswer, AnalysisError> => {
|
||||
const target = targetEntity(serving, request);
|
||||
if (!target.ok) return target.error;
|
||||
const entity = target.value;
|
||||
const answer = writeAnswer(
|
||||
serving,
|
||||
request,
|
||||
{ fields: entity.fields, id: entity.id },
|
||||
{
|
||||
source: "state",
|
||||
view: "delete",
|
||||
type: entity.type,
|
||||
collection: entity.collection,
|
||||
id: entity.id,
|
||||
},
|
||||
);
|
||||
if (answer.ok) {
|
||||
serving.store.remove(entity);
|
||||
serving.store.commitWrite(new Map());
|
||||
}
|
||||
return answer;
|
||||
};
|
||||
|
||||
/** The held document a call changes: only at the path the state holds it, with members the capture sent as the JSON types the document holds. */
|
||||
const documentTarget = (
|
||||
serving: StateServing,
|
||||
{ tool, plan, args }: WriteRequest,
|
||||
call: WriteCall,
|
||||
): Result<HeldDocument, AnalysisError> => {
|
||||
const held = serving.store.document(plan.target ?? "");
|
||||
const segments = callSegments(tool, args);
|
||||
if (held === undefined || plan.body === null)
|
||||
return err(
|
||||
refused(tool, `The state holds no document for ${plan.target ?? ""}.`),
|
||||
);
|
||||
if (segments === undefined)
|
||||
return err(refused(tool, `Tool ${tool.name} needs every path argument.`));
|
||||
// Document reads answer only on their recorded line, so a write elsewhere has no document to change.
|
||||
if (collectionMatch(held.path, segments) === undefined)
|
||||
return err(
|
||||
refused(
|
||||
tool,
|
||||
`The state holds this document only at ${held.path}; the environment models writes to it only there.`,
|
||||
),
|
||||
);
|
||||
const initial = serving.store.state.documents.find(
|
||||
(document) => document.tool === held.tool,
|
||||
);
|
||||
const problem = documentBodyProblem(plan, call, {
|
||||
initial: initial?.value,
|
||||
current: held.value,
|
||||
});
|
||||
return problem === undefined ? ok(held) : err(refused(tool, problem));
|
||||
};
|
||||
|
||||
const answerDocument = (
|
||||
serving: StateServing,
|
||||
request: WriteRequest,
|
||||
call: WriteCall,
|
||||
): Result<StateAnswer, AnalysisError> => {
|
||||
const { tool, plan } = request;
|
||||
const target = documentTarget(serving, request, call);
|
||||
if (!target.ok) return target;
|
||||
const held = target.value;
|
||||
const written = writtenDocument(plan, held.value, call);
|
||||
if (!written.ok) return err(refused(tool, written.error));
|
||||
if (!isLoadableValue(written.value))
|
||||
return err(unloadable(tool, "The document"));
|
||||
const state = {
|
||||
source: "state",
|
||||
view: "document-write",
|
||||
path: held.path,
|
||||
} as const;
|
||||
const answer =
|
||||
plan.response?.shape === "entity"
|
||||
? (() => {
|
||||
const exchange = serving.source.exchanges.get(plan.response.ordinal);
|
||||
return exchange === undefined
|
||||
? err(refused(tool, `Tool ${tool.name} has no recorded response.`))
|
||||
: stateAnswer(
|
||||
serving,
|
||||
{ exchange, exact: true },
|
||||
written.value,
|
||||
state,
|
||||
);
|
||||
})()
|
||||
: writeAnswer(serving, request, { fields: undefined, id: "" }, state);
|
||||
if (!answer.ok) return answer;
|
||||
const set = serving.store.setDocument(held.tool, written.value);
|
||||
if (!set.ok) return set;
|
||||
serving.store.commitWrite(new Map());
|
||||
return answer;
|
||||
};
|
||||
|
||||
/** Apply one served write tool call to the live state and answer it; a refused call leaves the state unchanged. */
|
||||
export const answerWrite = (
|
||||
serving: StateServing,
|
||||
request: WriteRequest,
|
||||
): Result<StateAnswer, AnalysisError> => {
|
||||
const body = callBody(request);
|
||||
if (!body.ok) return body;
|
||||
const call: WriteCall = {
|
||||
path: request.tool.path_template,
|
||||
pathValues: pathValues(request.tool, request.args),
|
||||
body: body.value,
|
||||
};
|
||||
switch (request.plan.effect) {
|
||||
case "create":
|
||||
return answerCreate(serving, request, call);
|
||||
case "update":
|
||||
return answerUpdate(serving, request, call);
|
||||
case "delete":
|
||||
return answerDelete(serving, request);
|
||||
case "document":
|
||||
return answerDocument(serving, request, call);
|
||||
case "none":
|
||||
return err(
|
||||
refused(
|
||||
request.tool,
|
||||
`Tool ${request.tool.name} is not served from the state.`,
|
||||
),
|
||||
);
|
||||
}
|
||||
};
|
||||
@@ -1,47 +0,0 @@
|
||||
import type {
|
||||
StateWritePlan,
|
||||
WriteBody,
|
||||
} from "../../domain/analysisView/environmentStateWriteRules.js";
|
||||
import { describeList, quotedName } from "./environmentTools.js";
|
||||
|
||||
/** Captured member names, quoted and sampled as response fields are; the plan in the state Evidence holds every one. */
|
||||
const members = (names: readonly string[]): string =>
|
||||
describeList(names, names.length, quotedName, "state Evidence");
|
||||
|
||||
/** The body members a served write accepts, as a description clause. */
|
||||
const bodyClause = (
|
||||
body: WriteBody,
|
||||
effect: StateWritePlan["effect"],
|
||||
): string => {
|
||||
const parts: string[] = [];
|
||||
if (body.accepted.length > 0)
|
||||
parts.push(`body members ${members(body.accepted)}`);
|
||||
if (body.fixed.length > 0)
|
||||
parts.push(
|
||||
`${members(body.fixed.map(({ member }) => member))} only with the value the capture sent`,
|
||||
);
|
||||
if (body.required.length > 0)
|
||||
parts.push(`required: ${members(body.required)}`);
|
||||
else if (effect === "update") parts.push("send at least one");
|
||||
return parts.length === 0 ? "" : ` (${parts.join("; ")})`;
|
||||
};
|
||||
|
||||
/** What a served write tool does to the state, for its description; undefined for a tool the state does not serve. */
|
||||
export const describeWrite = (
|
||||
plan: StateWritePlan | undefined,
|
||||
): string | undefined => {
|
||||
if (plan?.served !== true) return undefined;
|
||||
const body = plan.body === null ? "" : bodyClause(plan.body, plan.effect);
|
||||
switch (plan.effect) {
|
||||
case "create":
|
||||
return `creates an entity of ${plan.target ?? ""} in the served environment state${body}, modelled on the recorded calls`;
|
||||
case "update":
|
||||
return `changes the entity the path names in the served environment state${body}, modelled on the recorded calls`;
|
||||
case "delete":
|
||||
return "removes the entity the path names from the served environment state, modelled on the recorded calls";
|
||||
case "document":
|
||||
return `changes a document of the served environment state${body}, modelled on the recorded calls`;
|
||||
case "none":
|
||||
return undefined;
|
||||
}
|
||||
};
|
||||
@@ -1,269 +0,0 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import type { TimedCaptureRow } from "../../../tests/fixtures/analysisView.js";
|
||||
import {
|
||||
servedFrom,
|
||||
type Served,
|
||||
} from "../../../tests/fixtures/environmentStateServing.js";
|
||||
import {
|
||||
FIRST_BOOK,
|
||||
SECOND_BOOK,
|
||||
renamedBook,
|
||||
type Book,
|
||||
} from "../../../tests/fixtures/environmentStateWrites.js";
|
||||
import { MAX_JSON_DEPTH, type JsonValue } from "../../domain/jsonValue.js";
|
||||
import type { ReplayArguments } from "./environmentReplay.js";
|
||||
import { STATE_INTAKE_LIMIT_BYTES } from "./environmentStateDocument.js";
|
||||
|
||||
const DARK = { theme: "dark", meta: { n: 1 } };
|
||||
const LIGHT = { theme: "light", meta: { n: 2 } };
|
||||
|
||||
/** One organisation's settings document, read, patched with both members and read again. */
|
||||
const SETTINGS_ROWS: readonly TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/1/settings", DARK],
|
||||
["PATCH", "/api/orgs/1/settings", LIGHT, 200, LIGHT],
|
||||
["GET", "/api/orgs/1/settings", LIGHT],
|
||||
];
|
||||
|
||||
const READ_SETTINGS = "get_orgs_settings_by_id";
|
||||
const PATCH_SETTINGS = "patch_orgs_settings_by_id";
|
||||
|
||||
/** Every refused call is a tool error naming its problem, and leaves the state as it was. */
|
||||
const expectRefusedUnchanged = (
|
||||
served: Served,
|
||||
refused: readonly (readonly [string, ReplayArguments, string])[],
|
||||
): void => {
|
||||
const before = served.snapshot();
|
||||
for (const [name, args, message] of refused) {
|
||||
expect(served.refusal(name, args)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
message: expect.stringContaining(message),
|
||||
});
|
||||
expect(served.snapshot()).toBe(before);
|
||||
}
|
||||
};
|
||||
|
||||
/** What data/get reports loads back through data/add byte for byte. */
|
||||
const expectReloadsUnchanged = (served: Served): void => {
|
||||
const before = served.snapshot();
|
||||
expect(served.store.replace(JSON.parse(before)).ok).toBe(true);
|
||||
expect(served.snapshot()).toBe(before);
|
||||
};
|
||||
|
||||
const nested = (depth: number): JsonValue =>
|
||||
Array.from({ length: depth }).reduce<JsonValue>((inner) => [inner], 1);
|
||||
|
||||
describe("served document writes", () => {
|
||||
it("accepts only the members the capture sent, as the JSON types the document holds", () => {
|
||||
const settings = servedFrom(SETTINGS_ROWS);
|
||||
expect(settings.state.writes.get(PATCH_SETTINGS)).toMatchObject({
|
||||
served: true,
|
||||
effect: "document",
|
||||
});
|
||||
expectRefusedUnchanged(settings, [
|
||||
[
|
||||
PATCH_SETTINGS,
|
||||
{ id: "1", body: { theme: 5, extra: true } },
|
||||
"Member theme must be a JSON string",
|
||||
],
|
||||
[
|
||||
PATCH_SETTINGS,
|
||||
{ id: "1", body: { extra: true } },
|
||||
"Member extra is not one the capture showed stored",
|
||||
],
|
||||
]);
|
||||
const blue = { theme: "blue", meta: DARK.meta };
|
||||
expect(
|
||||
settings.answer(PATCH_SETTINGS, { id: "1", body: { theme: "blue" } }),
|
||||
).toMatchObject({
|
||||
body: blue,
|
||||
state: { view: "document-write", path: "/api/orgs/1/settings" },
|
||||
});
|
||||
expect(settings.answer(READ_SETTINGS, { id: "1" }).body).toEqual(blue);
|
||||
});
|
||||
|
||||
it("leaves a document write unserved when its recording changed a member's JSON type, which serving would refuse", () => {
|
||||
const retyped = { theme: 5, meta: LIGHT.meta };
|
||||
const settings = servedFrom([
|
||||
["GET", "/api/orgs/1/settings", DARK],
|
||||
["PATCH", "/api/orgs/1/settings", retyped, 200, retyped],
|
||||
["GET", "/api/orgs/1/settings", retyped],
|
||||
]);
|
||||
expect(settings.state.writes.get(PATCH_SETTINGS)).toMatchObject({
|
||||
served: false,
|
||||
reason: "unexplained-change",
|
||||
});
|
||||
});
|
||||
|
||||
it("does not confirm a document write at another path, which serving refuses", () => {
|
||||
const settings = servedFrom([
|
||||
["GET", "/api/orgs/1/settings", DARK],
|
||||
["PATCH", "/api/orgs/2/settings", LIGHT, 200, LIGHT],
|
||||
["GET", "/api/orgs/1/settings", LIGHT],
|
||||
]);
|
||||
expect(settings.state.writes.get(PATCH_SETTINGS)).toMatchObject({
|
||||
served: false,
|
||||
});
|
||||
});
|
||||
|
||||
it("changes the document only at the path the state holds it", () => {
|
||||
const settings = servedFrom(SETTINGS_ROWS);
|
||||
expectRefusedUnchanged(settings, [
|
||||
[
|
||||
PATCH_SETTINGS,
|
||||
{ id: "2", body: { theme: "blue" } },
|
||||
"holds this document only at /api/orgs/1/settings",
|
||||
],
|
||||
]);
|
||||
expect(settings.answer(READ_SETTINGS, { id: "1" }).body).toEqual(DARK);
|
||||
});
|
||||
});
|
||||
|
||||
describe("values the state can load back", () => {
|
||||
it("refuses a body value data/add would refuse, whatever member carries it", () => {
|
||||
const settings = servedFrom(SETTINGS_ROWS);
|
||||
const unexact =
|
||||
"Member meta would hold a value that is not exactly representable JSON";
|
||||
expectRefusedUnchanged(settings, [
|
||||
[PATCH_SETTINGS, { id: "1", body: { meta: { n: 1e20 } } }, unexact],
|
||||
[
|
||||
PATCH_SETTINGS,
|
||||
{
|
||||
id: "1",
|
||||
body: JSON.parse('{"meta":{"__proto__":{"n":1}}}') as JsonValue,
|
||||
},
|
||||
unexact,
|
||||
],
|
||||
[
|
||||
PATCH_SETTINGS,
|
||||
{ id: "1", body: { meta: { n: nested(MAX_JSON_DEPTH) } } },
|
||||
unexact,
|
||||
],
|
||||
]);
|
||||
settings.answer(PATCH_SETTINGS, { id: "1", body: { meta: { n: 3 } } });
|
||||
expectReloadsUnchanged(settings);
|
||||
});
|
||||
|
||||
it("refuses an increment past the exact integers, so data/get still loads back", () => {
|
||||
const versioned = (entry: Book, version: number): Book => ({
|
||||
...entry,
|
||||
version,
|
||||
});
|
||||
const last = Number.MAX_SAFE_INTEGER;
|
||||
const edition = versioned(renamedBook(SECOND_BOOK, "Second Edition", 3), 2);
|
||||
const books = servedFrom([
|
||||
[
|
||||
"GET",
|
||||
"/api/books",
|
||||
[versioned(FIRST_BOOK, last - 1), versioned(SECOND_BOOK, 1)],
|
||||
],
|
||||
["GET", "/api/books/1", versioned(FIRST_BOOK, last - 1)],
|
||||
["GET", "/api/books/2", versioned(SECOND_BOOK, 1)],
|
||||
["PATCH", "/api/books/2", edition, 200, { name: "Second Edition" }],
|
||||
["GET", "/api/books/2", edition],
|
||||
]);
|
||||
const rename = (name: string) => ({ id: "1", body: { name } });
|
||||
expect(
|
||||
books.answer("patch_books_by_id", rename("Renamed")).body,
|
||||
).toMatchObject({ version: last });
|
||||
expectRefusedUnchanged(books, [
|
||||
[
|
||||
"patch_books_by_id",
|
||||
rename("Renamed Again"),
|
||||
"Field version would hold a value that is not exactly representable JSON",
|
||||
],
|
||||
]);
|
||||
expectReloadsUnchanged(books);
|
||||
});
|
||||
});
|
||||
|
||||
describe("the state size limit", () => {
|
||||
it("keeps the size data/get writes as each write changes the state", () => {
|
||||
const books = servedFrom();
|
||||
const settings = servedFrom(SETTINGS_ROWS);
|
||||
const tracked = (served: Served) =>
|
||||
expect(served.store.byteLength()).toBe(
|
||||
Buffer.byteLength(served.snapshot(), "utf8"),
|
||||
);
|
||||
tracked(books);
|
||||
books.answer("post_books", { body: { name: "Ünïcode Bøok" } });
|
||||
tracked(books);
|
||||
books.answer("patch_books_by_id", { id: "3", body: { name: "Short" } });
|
||||
tracked(books);
|
||||
books.answer("delete_books_by_id", { id: "1" });
|
||||
books.answer("delete_books_by_id", { id: "2" });
|
||||
books.answer("delete_books_by_id", { id: "3" });
|
||||
tracked(books);
|
||||
books.answer("post_books", { body: { name: "Only Book" } });
|
||||
tracked(books);
|
||||
settings.answer(PATCH_SETTINGS, { id: "1", body: { theme: "dark blue" } });
|
||||
tracked(settings);
|
||||
expect(settings.store.replace(JSON.parse(settings.snapshot())).ok).toBe(
|
||||
true,
|
||||
);
|
||||
tracked(settings);
|
||||
});
|
||||
|
||||
it("refuses a create that would take data/get past what data/add loads", () => {
|
||||
const books = servedFrom();
|
||||
// Each answer renders the book, so each stays under the reply body limit.
|
||||
const create = { body: { name: "a".repeat(100_000) } };
|
||||
let refused: ReturnType<Served["call"]> | undefined;
|
||||
for (let made = 0; made < 200 && refused === undefined; made += 1) {
|
||||
const result = books.call("post_books", create);
|
||||
if (!result.ok) refused = result;
|
||||
}
|
||||
const before = books.snapshot();
|
||||
expect(Buffer.byteLength(before, "utf8")).toBeLessThanOrEqual(
|
||||
STATE_INTAKE_LIMIT_BYTES,
|
||||
);
|
||||
expect(refused?.ok === false && refused.error).toMatchObject({
|
||||
resource: "capacity",
|
||||
reportedLimits: { max_state_bytes: STATE_INTAKE_LIMIT_BYTES },
|
||||
});
|
||||
expect(books.refusal("post_books", create)).toMatchObject({
|
||||
code: "resource_constraint",
|
||||
});
|
||||
expect(books.snapshot()).toBe(before);
|
||||
expectReloadsUnchanged(books);
|
||||
// A smaller state has room again.
|
||||
books.answer("delete_books_by_id", { id: "3" });
|
||||
expect(books.answer("post_books", create).status).toBe(201);
|
||||
});
|
||||
|
||||
it("refuses a document that data/get would write past what data/add loads", () => {
|
||||
const books = servedFrom();
|
||||
const { rea_state: state } = books.store.snapshot();
|
||||
// Without the members data/get adds back, the document fits only until they return.
|
||||
const document = {
|
||||
rea_state: {
|
||||
version: state.version,
|
||||
entities: state.entities.map(({ type, collection, id, fields }) => ({
|
||||
type,
|
||||
collection,
|
||||
id,
|
||||
fields: { ...fields },
|
||||
})),
|
||||
documents: state.documents,
|
||||
},
|
||||
};
|
||||
const first = document.rea_state.entities[0]!.fields;
|
||||
first["name"] = `${String(first["name"])}${"a".repeat(
|
||||
STATE_INTAKE_LIMIT_BYTES -
|
||||
Buffer.byteLength(JSON.stringify(document), "utf8"),
|
||||
)}`;
|
||||
expect(Buffer.byteLength(JSON.stringify(document), "utf8")).toBe(
|
||||
STATE_INTAKE_LIMIT_BYTES,
|
||||
);
|
||||
const before = books.snapshot();
|
||||
const replaced = books.store.replace(document);
|
||||
expect(replaced.ok).toBe(false);
|
||||
expect(!replaced.ok && replaced.error.issues).toEqual([
|
||||
expect.objectContaining({
|
||||
reason: "out_of_range",
|
||||
maximum: STATE_INTAKE_LIMIT_BYTES,
|
||||
}),
|
||||
]);
|
||||
expect(books.snapshot()).toBe(before);
|
||||
});
|
||||
});
|
||||
@@ -1,34 +0,0 @@
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import type { Result } from "../../domain/result.js";
|
||||
import {
|
||||
replayEnvironmentTool,
|
||||
type ReplayArguments,
|
||||
type ReplayResult,
|
||||
} from "./environmentReplay.js";
|
||||
import type {
|
||||
EnvironmentManifestTool,
|
||||
EnvironmentSource,
|
||||
} from "./environmentSource.js";
|
||||
import { answerFromState } from "./environmentStateAnswer.js";
|
||||
import type { EnvironmentStateStore } from "./environmentStateStore.js";
|
||||
import { answerWrite } from "./environmentStateWriteAnswer.js";
|
||||
|
||||
/** How a served environment answers one tool call. */
|
||||
export type EnvironmentToolAnswer = (
|
||||
tool: EnvironmentManifestTool,
|
||||
args: ReplayArguments,
|
||||
) => Result<ReplayResult, AnalysisError>;
|
||||
|
||||
/** The answer a served environment gives: replay without a state store; with one, a served write applies to the live state and a read answers from it. */
|
||||
export const environmentToolAnswer = (
|
||||
source: EnvironmentSource,
|
||||
store: EnvironmentStateStore | undefined,
|
||||
): EnvironmentToolAnswer =>
|
||||
store === undefined
|
||||
? (tool, args) => replayEnvironmentTool(source, tool, args)
|
||||
: (tool, args) => {
|
||||
const plan = store.state.writes.get(tool.name);
|
||||
return plan?.served === true
|
||||
? answerWrite({ source, store }, { tool, plan, args })
|
||||
: answerFromState({ source, store }, tool, args);
|
||||
};
|
||||
@@ -125,7 +125,7 @@ const describeText = (
|
||||
};
|
||||
|
||||
/** Sample whole values before joining; keep full lists in the Evidence that owns them. */
|
||||
export const describeList = <Value>(
|
||||
const describeList = <Value>(
|
||||
values: Iterable<Value>,
|
||||
count: number,
|
||||
display: (value: Value) => string = String,
|
||||
@@ -156,7 +156,7 @@ export const describeList = <Value>(
|
||||
* A captured name as a JSON string, so a separator, quote or newline in it cannot read as more of the
|
||||
* description. A name too long for a sample is left as it is: the sample refuses it either way.
|
||||
*/
|
||||
export const quotedName = (name: string): string =>
|
||||
const quotedName = (name: string): string =>
|
||||
name.length > DESCRIPTION_SAMPLE_BYTES ? name : JSON.stringify(name);
|
||||
|
||||
/** Field names of the objects in first-seen order; the recorded bodies stay in the capture Evidence. */
|
||||
@@ -225,15 +225,6 @@ const describeResponses = (
|
||||
return `${returns} ${describeJson(found.value)}; ${recorded}`;
|
||||
};
|
||||
|
||||
/** One tool's description: method, path, response summary, what it does, the records it cites and the Evidence. */
|
||||
const toolDescription = (
|
||||
source: EnvironmentSource,
|
||||
tool: EnvironmentManifestTool,
|
||||
describe: EnvironmentCatalogOptions["describe"],
|
||||
answers: string,
|
||||
): string =>
|
||||
`${REPLAY_CATALOG_INSTRUCTIONS} ${describeText(tool.method, "method")} ${describeText(tool.path_template, "path")}; ${describeResponses(tool, source.exchanges)}; ${describe?.(tool) ?? answers} [${describeList(tool.record_ordinals, tool.record_ordinals.length)}]; evidence ${source.evidenceId}`;
|
||||
|
||||
/** Captured argument names longer than this are advertised under a short alias; the manifest keeps the name. */
|
||||
export const MAX_ARGUMENT_NAME_CHARS = 64;
|
||||
|
||||
@@ -359,8 +350,6 @@ export type EnvironmentDataPlane = "none" | "state";
|
||||
export interface EnvironmentCatalogOptions {
|
||||
readonly operation?: string;
|
||||
readonly dataPlane?: EnvironmentDataPlane;
|
||||
/** What a served tool does, in place of how it answers; undefined keeps the answering sentence. */
|
||||
readonly describe?: (tool: EnvironmentManifestTool) => string | undefined;
|
||||
}
|
||||
|
||||
/** Admit complete discovery before binding; six bytes per UTF-16 unit covers JSON escaping and UTF-8 expansion. */
|
||||
@@ -370,7 +359,6 @@ export const environmentToolDefinitions = (
|
||||
{
|
||||
operation = ENVIRONMENT_SERVE_OPERATION,
|
||||
dataPlane = "none",
|
||||
describe,
|
||||
}: EnvironmentCatalogOptions = {},
|
||||
): Result<
|
||||
readonly EnvironmentToolDefinition[],
|
||||
@@ -426,7 +414,7 @@ export const environmentToolDefinitions = (
|
||||
// A lower bound rejects enormous argument lists before constructing their alias map.
|
||||
if (!reserve(count * EMPTY_PROPERTY_BYTES)) return refused();
|
||||
// The response summary is bounded by its samples and charged with the whole description.
|
||||
const description = toolDescription(source, tool, describe, answers);
|
||||
const description = `${REPLAY_CATALOG_INSTRUCTIONS} ${describeText(tool.method, "method")} ${describeText(tool.path_template, "path")}; ${describeResponses(tool, source.exchanges)}; ${answers} [${describeList(tool.record_ordinals, tool.record_ordinals.length)}]; evidence ${source.evidenceId}`;
|
||||
if (!text(description)) return refused();
|
||||
const args = environmentToolArguments(tool);
|
||||
const properties = new Map<string, EnvironmentToolProperty>();
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import { metadataReportSubjectFormat } from "../../domain/apple/machoTargetSelection.js";
|
||||
import type { ExecutionOptions } from "../AnalysisProvider.js";
|
||||
import type { GoBinaryPort } from "./GoBinaryPort.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
@@ -74,7 +73,7 @@ export class GoBinaryService {
|
||||
{
|
||||
path: value.artifact.path,
|
||||
sha256: value.artifact.sha256,
|
||||
format: metadataReportSubjectFormat(value),
|
||||
format: value.format === "macho" ? "mach-o" : value.format,
|
||||
...(architecture === undefined ? {} : { architecture }),
|
||||
},
|
||||
this.provider.identity,
|
||||
|
||||
@@ -1,190 +0,0 @@
|
||||
import { chmod, mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { expect, it, vi } from "vitest";
|
||||
|
||||
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { electronActiveObservationInputSchema } from "../../domain/javascript/electronActiveObservation.js";
|
||||
import { err } from "../../domain/result.js";
|
||||
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
|
||||
import type { ElectronActiveObservationPort } from "./ElectronActiveObservationPort.js";
|
||||
import { captureElectronScenario } from "./ElectronActiveObservationService.js";
|
||||
|
||||
const SKIP_PERMISSIONS =
|
||||
process.platform === "win32" || process.getuid?.() === 0;
|
||||
|
||||
/** A provider that records admitted input and never launches anything. */
|
||||
const recordingProvider = () => {
|
||||
const capture = vi.fn<ElectronActiveObservationPort["capture"]>(() =>
|
||||
Promise.resolve(
|
||||
err(
|
||||
new AnalysisCapabilityUnavailableError(
|
||||
"test",
|
||||
"capture_electron_scenario",
|
||||
"recording provider",
|
||||
),
|
||||
),
|
||||
),
|
||||
);
|
||||
const provider: ElectronActiveObservationPort = {
|
||||
identity: () => ({ id: "test", name: "test", version: "1" }),
|
||||
capture,
|
||||
};
|
||||
return { provider, capture };
|
||||
};
|
||||
|
||||
const fixture = async () => {
|
||||
const root = await createTestTempDirectory("rea-electron-admission-");
|
||||
const denied = join(root, "denied");
|
||||
const readable = join(root, "readable");
|
||||
await mkdir(join(denied, "root"), { recursive: true });
|
||||
await mkdir(readable);
|
||||
const executable = join(denied, "electron");
|
||||
await writeFile(executable, "#!/bin/sh\n", { mode: 0o755 });
|
||||
await writeFile(join(denied, "main.js"), "module.exports = {};\n");
|
||||
await writeFile(join(readable, "main.js"), "module.exports = {};\n");
|
||||
return { root, denied, readable, executable };
|
||||
};
|
||||
|
||||
/** Remove search permission from `directory` for the duration of `run`. */
|
||||
const withoutSearchPermission = async <T>(
|
||||
directory: string,
|
||||
run: () => Promise<T>,
|
||||
): Promise<T> => {
|
||||
await chmod(directory, 0o000);
|
||||
try {
|
||||
return await run();
|
||||
} finally {
|
||||
await chmod(directory, 0o700);
|
||||
}
|
||||
};
|
||||
|
||||
it
|
||||
.skipIf(SKIP_PERMISSIONS)
|
||||
.each(["application_path", "application_root", "executable_path"] as const)(
|
||||
"reports a search-denied %s as host access denial before any provider work",
|
||||
async (field) => {
|
||||
const { denied, readable, executable } = await fixture();
|
||||
const selected = {
|
||||
application_path: join(denied, "main.js"),
|
||||
application_root: join(denied, "root"),
|
||||
executable_path: executable,
|
||||
}[field];
|
||||
const { provider, capture } = recordingProvider();
|
||||
const result = await withoutSearchPermission(denied, () =>
|
||||
captureElectronScenario(
|
||||
provider,
|
||||
electronActiveObservationInputSchema.parse({
|
||||
executable_path: process.execPath,
|
||||
application_path: join(readable, "main.js"),
|
||||
actions: [],
|
||||
[field]: selected,
|
||||
}),
|
||||
),
|
||||
);
|
||||
expect(result.ok).toBe(false);
|
||||
if (result.ok) return;
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "access_denied",
|
||||
category: "unavailable",
|
||||
details: {
|
||||
operation: "capture_electron_scenario",
|
||||
path: selected,
|
||||
system_code: "EACCES",
|
||||
boundary: "filesystem-read",
|
||||
},
|
||||
});
|
||||
expect(capture).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
it.skipIf(SKIP_PERMISSIONS)(
|
||||
"reports invalid selections before a host denial on another path",
|
||||
async () => {
|
||||
const { root, denied } = await fixture();
|
||||
const missing = join(root, "missing.js");
|
||||
const { provider, capture } = recordingProvider();
|
||||
const result = await withoutSearchPermission(denied, () =>
|
||||
captureElectronScenario(
|
||||
provider,
|
||||
electronActiveObservationInputSchema.parse({
|
||||
executable_path: process.execPath,
|
||||
application_path: missing,
|
||||
application_root: join(denied, "root"),
|
||||
actions: [],
|
||||
}),
|
||||
),
|
||||
);
|
||||
expect(result.ok).toBe(false);
|
||||
if (result.ok) return;
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
issues: [
|
||||
{
|
||||
path: ["application_path"],
|
||||
reason: "invalid_value",
|
||||
message: expect.stringContaining("ENOENT"),
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(capture).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
it.skipIf(SKIP_PERMISSIONS).each([
|
||||
{ label: "0644", mode: 0o644 },
|
||||
{ label: "0000", mode: 0o000 },
|
||||
])(
|
||||
"keeps a reachable executable without execute permission (mode $label) as invalid input",
|
||||
async ({ mode }) => {
|
||||
const { root, readable } = await fixture();
|
||||
const executable = join(root, "not-executable");
|
||||
await writeFile(executable, "not executable");
|
||||
await chmod(executable, mode);
|
||||
const { provider, capture } = recordingProvider();
|
||||
const result = await captureElectronScenario(
|
||||
provider,
|
||||
electronActiveObservationInputSchema.parse({
|
||||
executable_path: executable,
|
||||
application_path: join(readable, "main.js"),
|
||||
actions: [],
|
||||
}),
|
||||
);
|
||||
expect(result.ok).toBe(false);
|
||||
if (result.ok) return;
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
issues: [
|
||||
{
|
||||
path: ["executable_path"],
|
||||
reason: "invalid_value",
|
||||
message: expect.stringContaining("is not an executable file"),
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(capture).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
it("admits readable selections and hands canonical paths to the provider", async () => {
|
||||
const { readable } = await fixture();
|
||||
const { provider, capture } = recordingProvider();
|
||||
const application = join(readable, "main.js");
|
||||
await captureElectronScenario(
|
||||
provider,
|
||||
electronActiveObservationInputSchema.parse({
|
||||
executable_path: process.execPath,
|
||||
application_path: application,
|
||||
actions: [],
|
||||
}),
|
||||
);
|
||||
expect(capture).toHaveBeenCalledOnce();
|
||||
expect(capture.mock.calls[0]?.[0]).toMatchObject({
|
||||
application_path: application,
|
||||
application_root: readable,
|
||||
});
|
||||
});
|
||||
@@ -1,5 +1,5 @@
|
||||
import { dirname } from "node:path";
|
||||
import { constants, type Stats } from "node:fs";
|
||||
import { constants } from "node:fs";
|
||||
import { access, realpath, stat } from "node:fs/promises";
|
||||
|
||||
import type { ExecutionOptions } from "../AnalysisProvider.js";
|
||||
@@ -8,7 +8,6 @@ import type {
|
||||
ElectronActiveObservationPort,
|
||||
} from "./ElectronActiveObservationPort.js";
|
||||
import {
|
||||
AnalysisAccessDeniedError,
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisInputError,
|
||||
type AnalysisInputIssue,
|
||||
@@ -22,107 +21,53 @@ import { createElectronActiveEvidence } from "./ElectronActiveEvidence.js";
|
||||
const OPERATION = "capture_electron_scenario" as const;
|
||||
const PROVIDER_ID = "rea-playwright-electron-active";
|
||||
|
||||
type ElectronPathField =
|
||||
| "executable_path"
|
||||
| "application_path"
|
||||
| "application_root";
|
||||
|
||||
/**
|
||||
* Why a selected path was not admitted. Host permission denial is not
|
||||
* malformed caller input: the path may be correct, but REA cannot read it.
|
||||
*/
|
||||
type PathAdmissionFailure =
|
||||
| {
|
||||
readonly kind: "invalid";
|
||||
readonly issue: AnalysisInputIssue;
|
||||
readonly cause: unknown;
|
||||
}
|
||||
| { readonly kind: "denied"; readonly error: AnalysisAccessDeniedError };
|
||||
|
||||
const hostAccessDenial = (
|
||||
selected: string,
|
||||
cause: unknown,
|
||||
): PathAdmissionFailure | undefined => {
|
||||
const code: unknown =
|
||||
cause instanceof Error ? Reflect.get(cause, "code") : undefined;
|
||||
return code === "EACCES" || code === "EPERM"
|
||||
? {
|
||||
kind: "denied",
|
||||
error: new AnalysisAccessDeniedError(OPERATION, selected, code, {
|
||||
cause,
|
||||
}),
|
||||
}
|
||||
: undefined;
|
||||
};
|
||||
|
||||
const resolveInputPath = async (
|
||||
field: ElectronPathField,
|
||||
field: "executable_path" | "application_path" | "application_root",
|
||||
selected: string,
|
||||
): Promise<Result<string, PathAdmissionFailure>> => {
|
||||
): Promise<
|
||||
Result<
|
||||
string,
|
||||
{ readonly issue: AnalysisInputIssue; readonly cause: unknown }
|
||||
>
|
||||
> => {
|
||||
try {
|
||||
return ok(await realpath(selected));
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
hostAccessDenial(selected, cause) ?? {
|
||||
kind: "invalid",
|
||||
issue: {
|
||||
path: [field],
|
||||
reason: "invalid_value",
|
||||
message: `Cannot resolve ${field} ${selected}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
},
|
||||
cause,
|
||||
return err({
|
||||
issue: {
|
||||
path: [field],
|
||||
reason: "invalid_value",
|
||||
message: `Cannot resolve ${field} ${selected}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
},
|
||||
);
|
||||
cause,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Resolve the executable and require a launchable regular file before any
|
||||
* spawn. Denied resolution or metadata reads are host access failures; a
|
||||
* readable file without execute permission is an invalid selection.
|
||||
*/
|
||||
/** Resolve the executable and require a launchable regular file before any spawn. */
|
||||
const resolveExecutablePath = async (
|
||||
selected: string,
|
||||
): ReturnType<typeof resolveInputPath> => {
|
||||
const resolved = await resolveInputPath("executable_path", selected);
|
||||
if (!resolved.ok) return resolved;
|
||||
let metadata: Stats;
|
||||
try {
|
||||
metadata = await stat(resolved.value);
|
||||
} catch (cause: unknown) {
|
||||
return err(
|
||||
hostAccessDenial(selected, cause) ?? notExecutable(selected, cause),
|
||||
);
|
||||
}
|
||||
if (!metadata.isFile())
|
||||
return err(notExecutable(selected, new Error("not a regular file")));
|
||||
if (process.platform !== "win32")
|
||||
try {
|
||||
if (!(await stat(resolved.value)).isFile())
|
||||
throw new Error("not a regular file");
|
||||
if (process.platform !== "win32")
|
||||
await access(resolved.value, constants.X_OK);
|
||||
} catch (cause: unknown) {
|
||||
return err(notExecutable(selected, cause));
|
||||
}
|
||||
return resolved;
|
||||
return resolved;
|
||||
} catch (cause: unknown) {
|
||||
return err({
|
||||
issue: {
|
||||
path: ["executable_path"],
|
||||
reason: "invalid_value",
|
||||
message: `executable_path ${selected} is not an executable file: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
},
|
||||
cause,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
const notExecutable = (
|
||||
selected: string,
|
||||
cause: unknown,
|
||||
): PathAdmissionFailure => ({
|
||||
kind: "invalid",
|
||||
issue: {
|
||||
path: ["executable_path"],
|
||||
reason: "invalid_value",
|
||||
message: `executable_path ${selected} is not an executable file: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
},
|
||||
cause,
|
||||
});
|
||||
|
||||
/**
|
||||
* Admit every selected path before any provider work. Invalid selections are
|
||||
* reported together as input issues; a host access denial is reported only
|
||||
* when no selection is invalid, because correcting the request comes first.
|
||||
*/
|
||||
const canonicalizeInput = async (
|
||||
input: ElectronActiveObservationInput,
|
||||
): Promise<Result<CanonicalElectronActiveObservationInput, AnalysisError>> => {
|
||||
@@ -137,23 +82,16 @@ const canonicalizeInput = async (
|
||||
const failures = [executable, application, root].flatMap((result) =>
|
||||
result.ok ? [] : [result.error],
|
||||
);
|
||||
const issues = failures.flatMap((failure) =>
|
||||
failure.kind === "invalid" ? [failure.issue] : [],
|
||||
);
|
||||
const denial = failures.find((failure) => failure.kind === "denied");
|
||||
if (issues.length === 0 && denial !== undefined) return err(denial.error);
|
||||
return err(
|
||||
new AnalysisInputError(
|
||||
OPERATION,
|
||||
{
|
||||
cause: new AggregateError(
|
||||
failures.map((failure) =>
|
||||
failure.kind === "invalid" ? failure.cause : failure.error,
|
||||
),
|
||||
failures.map(({ cause }) => cause),
|
||||
"Electron paths could not be resolved",
|
||||
),
|
||||
},
|
||||
issues,
|
||||
failures.map(({ issue }) => issue),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import { javascriptSourceRangeKey } from "../../domain/javascript/javascriptSourceCoordinates.js";
|
||||
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
||||
import type { ElectronCallPath } from "../../domain/javascript/electronStaticAnalysisTypes.js";
|
||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||
@@ -11,6 +10,7 @@ import {
|
||||
import {
|
||||
addElectronInferenceEdge,
|
||||
electronFindingSourceNode,
|
||||
electronRangeKey,
|
||||
} from "./ElectronBoundaryGraphContext.js";
|
||||
import {
|
||||
artifactLocalIdentity,
|
||||
@@ -191,15 +191,13 @@ const matches = (
|
||||
path.steps.some(
|
||||
(step) =>
|
||||
step.kind === sourceStepKind &&
|
||||
javascriptSourceRangeKey(step.location) ===
|
||||
javascriptSourceRangeKey(finding.location),
|
||||
electronRangeKey(step.location) === electronRangeKey(finding.location),
|
||||
) &&
|
||||
(invocation === undefined ||
|
||||
path.steps.some(
|
||||
(step) =>
|
||||
(step.kind === "call" || step.kind === "exposure") &&
|
||||
javascriptSourceRangeKey(step.location) ===
|
||||
javascriptSourceRangeKey(invocation),
|
||||
electronRangeKey(step.location) === electronRangeKey(invocation),
|
||||
))
|
||||
);
|
||||
};
|
||||
@@ -108,3 +108,23 @@ export const addElectronInferenceEdge = (
|
||||
}),
|
||||
});
|
||||
};
|
||||
|
||||
/** Deterministic source-range key for artifact-local graph identities. */
|
||||
export const electronRangeKey = (range: JavaScriptSourceRange): string =>
|
||||
`${String(range.start.line)}:${String(range.start.column)}-${String(range.end.line)}:${String(range.end.column)}`;
|
||||
|
||||
/** Test exact source-range containment without assuming runtime reachability. */
|
||||
export const electronRangeContains = (
|
||||
outer: JavaScriptSourceRange,
|
||||
inner: JavaScriptSourceRange,
|
||||
): boolean =>
|
||||
comparePoint(outer.start, inner.start) <= 0 &&
|
||||
comparePoint(outer.end, inner.end) >= 0;
|
||||
|
||||
const comparePoint = (
|
||||
left: JavaScriptSourceRange["start"],
|
||||
right: JavaScriptSourceRange["start"],
|
||||
): number =>
|
||||
left.line === right.line
|
||||
? left.column - right.column
|
||||
: left.line - right.line;
|
||||
@@ -1,7 +1,3 @@
|
||||
import {
|
||||
javascriptSourceRangeKey,
|
||||
javascriptSourceRangeContains,
|
||||
} from "../../domain/javascript/javascriptSourceCoordinates.js";
|
||||
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
||||
import type { ElectronSenderValidationFinding } from "../../domain/javascript/electronStaticAnalysisTypes.js";
|
||||
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
|
||||
@@ -25,6 +21,8 @@ import {
|
||||
addElectronInferenceEdge,
|
||||
electronFindingSourceNode,
|
||||
electronObservationIdentity,
|
||||
electronRangeContains,
|
||||
electronRangeKey,
|
||||
} from "./ElectronBoundaryGraphContext.js";
|
||||
|
||||
interface IpcGraphState {
|
||||
@@ -184,7 +182,7 @@ const addHandlerNode = (input: IpcHandlerInput): ApplicationNode => {
|
||||
identity: artifactLocalIdentity(
|
||||
file.sha256,
|
||||
"electron-ipc-handler",
|
||||
`${finding.side}:${finding.operation}:${finding.channel ?? finding.channel_expression}:${javascriptSourceRangeKey(handlerRange)}`,
|
||||
`${finding.side}:${finding.operation}:${finding.channel ?? finding.channel_expression}:${electronRangeKey(handlerRange)}`,
|
||||
),
|
||||
observations: [
|
||||
{
|
||||
@@ -340,7 +338,7 @@ const containingHandler = (
|
||||
(record) =>
|
||||
record.file.path === file.path &&
|
||||
record.finding.handler_location !== null &&
|
||||
javascriptSourceRangeContains(
|
||||
electronRangeContains(
|
||||
record.finding.handler_location,
|
||||
validation.location,
|
||||
),
|
||||
|
||||
Loaded 100 of 529 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user