Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
 morluto
|
7c4d871803 |
No files matched your search
@@ -68,10 +68,9 @@ closing-issue reporters. Grouping preserves all those credits in one line.
|
||||
When Highlights, detailed entries or migrations reference the same work,
|
||||
put its thanks in one of those entries, not all of them. Include issue links.
|
||||
Direct-commit bullets carry the commit link and known human credit.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor,
|
||||
Codex, and Copilot co-author trailers resolve to `@claude`, `@cursoragent`,
|
||||
`@codex`, and `@Copilot` with GitHub type `User`; the credit policy excludes
|
||||
them while retaining their metadata.
|
||||
Exclude GitHub bots and known agent accounts. REA's observed Anthropic, Cursor
|
||||
and Codex co-author trailers resolve to `@claude`, `@cursoragent` and `@codex`
|
||||
with GitHub type `User`; the credit policy excludes them while retaining their metadata.
|
||||
At the REA maintainer's request, omit `@morluto` self-thanks while preserving
|
||||
authorship in the inventory. Other human contributors remain eligible.
|
||||
Use explicit GitHub links for cross-repository references to avoid ambiguous
|
||||
|
||||
@@ -374,16 +374,12 @@ native target whose analysis tool operates on the active target. Do not call an
|
||||
unadvertised tool; inspect `binary_session` with `{}` and its
|
||||
`result.tool_availability` for availability reasons and remediation. Expand each
|
||||
group's `names` into entries, inheriting the inventory-level client requirements
|
||||
unless the group overrides them. The grouped name union is the complete tool
|
||||
inventory; use `tools/list` for canonical tool schemas. The default summary keeps
|
||||
each provider candidate's availability and target-support status, code and
|
||||
reason; `omitted_fields` lists what it leaves out. Request `{"detail": "complete"}`
|
||||
only when you need provider diagnostics, candidate capability sets, or tool
|
||||
surface, effect and annotation flags; in those closed flag vocabularies, listed
|
||||
flags are true and omitted known flags are false. Provider candidate facts
|
||||
describe possible providers, while top-level session capabilities describe the
|
||||
current route and `analysis_provider_binding` names the selected provider. Do
|
||||
not treat those scopes as interchangeable.
|
||||
unless the group overrides them. In its closed effect and annotation vocabularies,
|
||||
listed flags are true and omitted known flags are false. The grouped name union
|
||||
is the complete tool inventory; use `tools/list` for canonical tool schemas.
|
||||
Provider candidate facts describe possible providers, while top-level session
|
||||
capabilities describe the current route and `analysis_provider_binding` names the
|
||||
selected provider. Do not treat those scopes as interchangeable.
|
||||
|
||||
For a Ghidra startup timeout on a large native target, follow
|
||||
[deadline recovery](references/native-and-artifacts.md#ghidra-startup-deadline-recovery)
|
||||
|
||||
@@ -22,7 +22,7 @@ does not establish which server an approval wildcard covers. See
|
||||
| Aligned registration, but no connected tools | Restart/reconnect the client. Doctor checks registration; it cannot establish the current agent's connection. Inspect the client's launch error if reconnection fails. |
|
||||
| MCP connection dropped or REA could not start | If the connection drops, retry that call once only when it is safe to repeat or known not to have started. If it may already have changed the target, or the same failure returns, tell the user REA's connection failed and ask them to restart the MCP client. If REA could not start its connection, retry once, then tell the user and ask them to restart the MCP client. Do not retry a startup failure that returns the same unchanged result. Run `rea doctor` when REA could not start or tools stay missing after restart. Doctor cannot restore a live connection by itself. |
|
||||
| `MCP tool call requires approval, but approval policy is never` | Codex rejected the call before REA executed it. Do not install an engine or repeat setup to solve this client-policy decision. See the approval section below. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. Request `{"detail": "complete"}` only for provider diagnostics. |
|
||||
| REA returns `provider_unavailable` or an unavailable tool entry | Read that operation's provider diagnostics or `binary_session.result.tool_availability`. Expand each group's tool names and inherit the shared client requirements unless overridden. Use the operation's availability, reason, and remediation; a complete tool catalog does not imply every operation is available. |
|
||||
| `rea` is missing in a child shell, or Node is incompatible | Inspect the actual shell PATH and launcher/runtime. A prefix installation may be absent from a login shell's PATH; this does not establish a broken MCP registration. |
|
||||
|
||||
If the client cannot launch its shell/sandbox helper or read the installed skill,
|
||||
|
||||
@@ -316,28 +316,24 @@ run, use `observe_native_calls` with explicit breakpoints and a bounded
|
||||
`duration_ms`/`max_events`. It launches a new process under LLDB, so the target
|
||||
runs and may change files or use the network. Hardened-runtime targets need the
|
||||
`get-task-allow` entitlement.
|
||||
Function breakpoints use exact LLDB runtime names: pass an `inspect_macho` export
|
||||
or import `runtime_name` (`main`), not its symbol-table `name` (`_main`). REA does
|
||||
not translate breakpoint names. Set `module` to the image
|
||||
Function breakpoints use exact LLDB runtime names: a Darwin C `_main` static symbol
|
||||
may be `main` at runtime. REA does not translate names. Set `module` to the image
|
||||
file name when selecting that image only; otherwise loaded system images also
|
||||
match. Inspect retained locations and module paths before attributing a call.
|
||||
|
||||
Use `binary_session` with no arguments to check the open target, selected
|
||||
provider, and alignment. Its default summary `result.tool_availability` carries
|
||||
the complete tool inventory grouped by current status. Expand each group's
|
||||
`names` and inherit the inventory-level client requirements unless a group
|
||||
overrides them; availability, reasons, remediation, and modes remain inline.
|
||||
Provider candidates keep availability and target-support status, code, and
|
||||
reason. `omitted_fields` names what the summary leaves out; `{"detail":
|
||||
"complete"}` adds provider diagnostics, candidate capability sets, and tool
|
||||
surface, `effects`, and `annotations`. Those arrays name true flags from fixed
|
||||
vocabularies, so an omitted known flag is false. `tools/list` remains the
|
||||
canonical source for full tool schemas when target or provider state changes.
|
||||
provider, and alignment. Its default `result.tool_availability` carries a
|
||||
complete inventory grouped by shared facts. Expand each group's `names` and
|
||||
inherit the inventory-level client requirements unless a group overrides them;
|
||||
availability, reasons, remediation, modes, and flags remain inline. The
|
||||
`effects` and `annotations` arrays name true flags from fixed vocabularies, so an
|
||||
omitted known flag is false. `tools/list` remains the canonical source for full
|
||||
tool schemas when target or provider state changes.
|
||||
|
||||
Provider capability groups carry provider identity and ordered limitations.
|
||||
Prepend each set's `shared_limitations` to a group's `limitations` to recover the
|
||||
operation's full ordered list. Candidate capabilities, returned with complete
|
||||
detail, describe possible providers. Top-level session capabilities describe the current route, and
|
||||
operation's full ordered list. Candidate capabilities describe possible
|
||||
providers. Top-level session capabilities describe the current route, and
|
||||
`analysis_provider_binding` identifies the selected provider.
|
||||
|
||||
## Ghidra startup deadline recovery
|
||||
|
||||
@@ -86,8 +86,6 @@ Page-declared WebMCP tools are untrusted inventory and are never invoked by REA.
|
||||
browser and profile. Check `browser.cleanup` for the outcome.
|
||||
Request only the DOM, accessibility, or network details needed for the question.
|
||||
The default final capture contains the URL; step outcomes are reported separately.
|
||||
For multi-step flows with trees or events, select `detail: "summary"` and page
|
||||
exact steps, transactions, text slices or events with `inspect_analysis_view` kind `browser-scenario`.
|
||||
Add a wait action for the expected page state before capturing it. Report that
|
||||
REA performed the declared actions.
|
||||
|
||||
|
||||
@@ -55,12 +55,6 @@ jobs:
|
||||
- uses: ./.github/actions/portable-build-cache
|
||||
with:
|
||||
build_cache_artifact: ${{ inputs.build_cache_artifact }}
|
||||
- name: Install ELF32 cross toolchains for source-owned fixtures
|
||||
run: |
|
||||
sudo apt-get update
|
||||
sudo apt-get install --yes --no-install-recommends \
|
||||
gcc-i686-linux-gnu gcc-arm-linux-gnueabi gcc-mips-linux-gnu gcc-mipsel-linux-gnu \
|
||||
libc6-dev-i386-cross libc6-dev-armel-cross libc6-dev-mips-cross libc6-dev-mipsel-cross
|
||||
- name: Acquire exact upstream libraries in an isolated runner environment
|
||||
shell: bash
|
||||
run: |
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
".": "6.8.0"
|
||||
".": "6.7.0"
|
||||
}
|
||||
-102
@@ -1,107 +1,5 @@
|
||||
# Changelog
|
||||
|
||||
## [6.8.0](https://github.com/morluto/rea/compare/rea-agents-6.7.0...rea-agents-6.8.0) (2026-10-25)
|
||||
|
||||
### Highlights
|
||||
|
||||
- Build local agent environments from network captures, browser scenarios, or a prompt, then serve derived reads and evidence-gated writes and compare them with fresh application captures.
|
||||
- Inspect more shipped formats, including GPU code objects, BEAM, Python bytecode, Unity IL2CPP metadata, Godot PCK, PyInstaller, Node.js SEA, devicetree, and additional PE/ELF tables.
|
||||
- Keep large investigations focused with retained summaries, selected fact queries, configurable recovery limits, and row-oriented CLI exports. Complete Evidence remains available.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
- **Function analysis:** `analyze_function` now defaults to a retained summary. Set `detail: "complete"` for the previous inline dossier; the CLI retains its complete default and requires `--evidence_output` when selecting a summary. [#5660](https://github.com/morluto/rea/pull/5660). Thanks @CryptoJones, @gensokyo-dev, @RishTheFish21.
|
||||
- **Accessibility:** Accept JSON numbers and booleans in node `value`, alongside strings and null. Update consumers that previously assumed text-only values. [cc90771](https://github.com/morluto/rea/commit/cc90771af14f045538651615147d8bf86b576549), [#6759](https://github.com/morluto/rea/pull/6759), [#6467](https://github.com/morluto/rea/issues/6467). Thanks @coygeek, @gensokyo-dev, @mvanhorn.
|
||||
- **JSON Lines:** `--format jsonl` now emits object-collection rows followed by a `$rea: "jsonl/v1"`, `type: "metadata"` record; scalar selections and unsupported reports are refused. Parse control records separately, select a single supported collection when necessary, and use `--format json` for the previous whole-document output. See [JSON Lines framing](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/cli.md#json-lines-record-output). [#5629](https://github.com/morluto/rea/pull/5629), [#5487](https://github.com/morluto/rea/issues/5487). Thanks @gensokyo-dev, @jayykernel, @vijaykiran06.
|
||||
- **Interface Builder:** Keyed-archive graph identities use decimal table UIDs instead of authored `objectID`/`object-id`. Treat authored IDs as attributes and update persisted graph lookups; ibtool and native NIBArchive identities keep their source semantics. [6fbe67d](https://github.com/morluto/rea/commit/6fbe67d66e3d716db69929503a3a232a7ada0860).
|
||||
- **Frida:** `instrument-with-frida` JSON returns the canonical Evidence directly. Read `normalized_result.cleanup_error` instead of the former top-level wrapper field. [48f700e](https://github.com/morluto/rea/commit/48f700ee35e0988ed973e5486dc9db33f2a5d944).
|
||||
- **Reconstruction proof:** ID-array proof records no longer establish verified or ready ledgers. Supply implementation/revision bindings and individual claim outcomes; handle inapplicable, unavailable, and failed proof diagnostics. [f8b65e7](https://github.com/morluto/rea/commit/f8b65e78723b1e2169f94dd1226198ea3139707e).
|
||||
- **Mach-O imports:** Handle a null import `name` when identity is ambiguous; use `raw_name` and `name_status` to distinguish observed expressions from exact symbols. [5d0d243](https://github.com/morluto/rea/commit/5d0d243a45c9132e1aff75d454ab53d86a74cc31), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b).
|
||||
- **Node.js:** Use a runtime satisfying `^22.23.4 || ^24.22.1 || >=26.12.1`; Node 23 and 25 are unsupported. Upgrade Node before installing or updating REA; the installer does not upgrade it. [#6315](https://github.com/morluto/rea/pull/6315), [#6283](https://github.com/morluto/rea/issues/6283). Thanks @GamerDubz, @gensokyo-dev, @ShikamaruNaaraa.
|
||||
- **Browser storage:** Historical hashed storage remains readable, but comparisons stay unknown until both captures are recaptured in the new representation. [#6341](https://github.com/morluto/rea/pull/6341), [#6289](https://github.com/morluto/rea/issues/6289). Thanks @coygeek, @gongfpp.
|
||||
- **Environment serving:** `environment-serve` serves derived state by default, including write tools whose effects are confirmed by recorded evidence. Use `--replay-only` for stateless replay of manifest tools. Read stop-summary `state` instead of `state_evidence_id`. [#5351](https://github.com/morluto/rea/pull/5351), [#6158](https://github.com/morluto/rea/pull/6158). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **MCP schemas:** Input advertisements default to compact. Set `REA_MCP_INPUT_SCHEMA_PROFILE=full` for complete advertised guidance and nested constraints. [#5975](https://github.com/morluto/rea/pull/5975). Thanks @gensokyo-dev.
|
||||
- **Browser console arguments:** Accept null `argument_types` and `text_capture.status: "unavailable"` when the protocol argument inventory is omitted or malformed; preserve that uncertainty. [#6016](https://github.com/morluto/rea/pull/6016), [#5934](https://github.com/morluto/rea/issues/5934). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Reconstruction summaries:** `summary.by_evidence_authority` is keyed by Evidence authority; read the former source-state counts from required `summary.by_source_state`. Regenerate ledgers whose closure digests depended on the old projection. [#5875](https://github.com/morluto/rea/pull/5875), [#5838](https://github.com/morluto/rea/issues/5838). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Swift bindings:** Conformance `type_name` and `protocol_name` may be null; consume typed `type_binding`/`protocol_binding` evidence. Replace the former Objective-C-specific import-binding type/schema with `MachoImportBinding`. [d0067ba](https://github.com/morluto/rea/commit/d0067ba34a39ea47dd7e11bb2e6169bc0938cc3f).
|
||||
- **WebSocket frames:** Accept null opcodes and preserve unknown frame facts; numeric opcodes are restricted to the four-bit protocol range. [0bc2b42](https://github.com/morluto/rea/commit/0bc2b42001f684beffcf47b70e507443b1a5d320).
|
||||
- **Resource-only inspection:** Evidence subjects and executable digests may be null. Use `resource_root` and each resource's reported provenance instead of requiring an executable target. [6a6adf7](https://github.com/morluto/rea/commit/6a6adf703fa44100643947d20fbc670f86260b1b).
|
||||
- **Browser timestamps:** Console and session-timeline timestamps may be null when the protocol supplies no finite/comparable time. Keep unknown time explicit in sorting and comparisons. [690fc51](https://github.com/morluto/rea/commit/690fc518540bae982038a74f47ef7781201785aa), [2a0410f](https://github.com/morluto/rea/commit/2a0410f1717b3126e8a95063d77dea66a597010a).
|
||||
- **Reference sources:** Read failed root enumeration from `rootFailure` rather than a synthetic failed child at `"."`; preserve the top-level source-graph limitation. [6296fdf](https://github.com/morluto/rea/commit/6296fdf560f1ac4aa55985558edaa98300aacdbc).
|
||||
- **Filesystem capture:** Accept `entry_unavailable` and `identity_changed` enumeration omission reasons in addition to the previous reason literals. [c85b44e](https://github.com/morluto/rea/commit/c85b44e1b14851c54e65327a4807f79ad5cd5a61). Thanks @rudrakshagarwal4200-ai.
|
||||
- **APK locales:** Replace numeric legacy selectors such as `es-r419` with `es-419` or `b+es+419`. Two-letter regions remain supported. [29910d1](https://github.com/morluto/rea/commit/29910d16b98fe4b607a7aa15928789a73614b0b9). Thanks @mertkayacs.
|
||||
- **XML plist scalars:** Recognize `$plist_type: "unknown"` and read `element`, `literal`, and `reason`; unresolved XML scalars use this wrapper instead of the former real/null unknown marker or malformed scalar prefixes. Supported extended UTC years preserve their timestamp. [3537e58](https://github.com/morluto/rea/commit/3537e58c1ab29309f06eabf50bc52adba9817a0c).
|
||||
|
||||
### Changes
|
||||
|
||||
- **Binary layouts:** Inspect ELF32 i386, little-endian ARM, and little- or big-endian MIPS `EXEC`/`DYN`/`REL` layouts on Linux x64 using a caller-supplied pwntools environment. [#7052](https://github.com/morluto/rea/pull/7052), [#6934](https://github.com/morluto/rea/issues/6934). Thanks @roblennon, @salmanalmutairi1.
|
||||
- **Agent environments:** Project network captures and Evidence files into manifests, preserve template path pairs and positions, export AgentEnv bundles, and serve manifests as replay environments. [#4954](https://github.com/morluto/rea/pull/4954), [#5293](https://github.com/morluto/rea/pull/5293), [#5664](https://github.com/morluto/rea/pull/5664), [#4999](https://github.com/morluto/rea/pull/4999), [#4975](https://github.com/morluto/rea/pull/4975). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Capture to environment:** Build environments from browser scenarios or a prompt, extract the initial observed state, and compare the environment with application probe captures. [#5182](https://github.com/morluto/rea/pull/5182), [#6946](https://github.com/morluto/rea/pull/6946), [#5319](https://github.com/morluto/rea/pull/5319), [#5699](https://github.com/morluto/rea/pull/5699). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **State-backed tools:** Render read filters, search, sorting and paging from state; apply confirmed write effects with broader state targets and recording-based write admission. [#5352](https://github.com/morluto/rea/pull/5352), [293832b](https://github.com/morluto/rea/commit/293832bafc9a054914ec514fe906dd4936c7a372), [#5800](https://github.com/morluto/rea/pull/5800), [#5787](https://github.com/morluto/rea/pull/5787), [#5600](https://github.com/morluto/rea/pull/5600), [5375ff0](https://github.com/morluto/rea/commit/5375ff0f9aadb4fd0d46333815aa0431e3df1010), [#6169](https://github.com/morluto/rea/pull/6169), [#6385](https://github.com/morluto/rea/pull/6385), [#6897](https://github.com/morluto/rea/pull/6897). Thanks @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **State learning:** Model application-assigned values, preserve view-specific clock spellings, learn write clocks from scenario timing, derive fields, and learn creates across sorted arrays and referenced entities. [7ec9b6e](https://github.com/morluto/rea/commit/7ec9b6e56e1b3762285dca881db6d900a389ab9c), [#6136](https://github.com/morluto/rea/pull/6136), [957bcfa](https://github.com/morluto/rea/commit/957bcfa149c1a0593bb0aa2466cc56d1d4f62e7d), [#6138](https://github.com/morluto/rea/pull/6138), [#6967](https://github.com/morluto/rea/pull/6967), [#7025](https://github.com/morluto/rea/pull/7025), [#7034](https://github.com/morluto/rea/pull/7034). Thanks @earakely-scale.
|
||||
- **Recorded writes:** Serve creates whose compatible read-backs add omitted fields or express timestamps at a different precision, and whole-object updates that regenerate observed slugs or modification times. Conflicting observations still withhold writes. State views and bundles carrying the new read-back data need REA 6.8.0 or later to serve. [#7080](https://github.com/morluto/rea/pull/7080). Thanks @earakely-scale.
|
||||
- **Native investigations:** Page native inventory summaries, match functions across builds, export pseudocode to resumable directories, and select objdump disassembly by symbol. [#5378](https://github.com/morluto/rea/pull/5378), [#5390](https://github.com/morluto/rea/pull/5390), [#6170](https://github.com/morluto/rea/pull/6170), [#5171](https://github.com/morluto/rea/pull/5171), [#4249](https://github.com/morluto/rea/issues/4249), [#5343](https://github.com/morluto/rea/issues/5343), [#5159](https://github.com/morluto/rea/issues/5159). Thanks @coygeek, @CryptoJones, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Engine workflows:** Bound a single Ghidra decompilation without closing its session, analyze GameCube/Wii DOL executables, and inspect static CUDA/HIP code objects. [#5662](https://github.com/morluto/rea/pull/5662), [#6240](https://github.com/morluto/rea/pull/6240), [#5813](https://github.com/morluto/rea/pull/5813), [#5340](https://github.com/morluto/rea/issues/5340), [#5345](https://github.com/morluto/rea/issues/5345). Thanks @arielfikru, @coygeek, @gensokyo-dev, @RishTheFish21.
|
||||
- **PE/ELF:** Inspect export tables without an engine, PE imports/delay-loads, Authenticode certificates, TLS callbacks/load configuration, ELF notes, Linux kernel module metadata, and Zstandard section-name tables. [#6278](https://github.com/morluto/rea/pull/6278), [#6509](https://github.com/morluto/rea/pull/6509), [#4830](https://github.com/morluto/rea/pull/4830), [#7008](https://github.com/morluto/rea/pull/7008), [#4837](https://github.com/morluto/rea/pull/4837), [#7010](https://github.com/morluto/rea/pull/7010), [#6987](https://github.com/morluto/rea/pull/6987), [#4057](https://github.com/morluto/rea/issues/4057). Thanks @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia, @ShuaixinHuang.
|
||||
- **Apple/native metadata:** Inspect IPS crash reports, decode Swift nominal declarations, and select universal Mach-O slices in Go/Rust metadata readers. [#6511](https://github.com/morluto/rea/pull/6511), [#6257](https://github.com/morluto/rea/pull/6257), [#6219](https://github.com/morluto/rea/pull/6219), [#5341](https://github.com/morluto/rea/issues/5341), [#4102](https://github.com/morluto/rea/issues/4102). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @mvanhorn, @Rinkia.
|
||||
- **Packaged applications:** Triage desktop installation directories and inspect/extract Godot PCK, PyInstaller archives, and Node.js single-executable application blobs. [#5207](https://github.com/morluto/rea/pull/5207), [#3634](https://github.com/morluto/rea/pull/3634), [afa5da7](https://github.com/morluto/rea/commit/afa5da745e0b139e68029a96b776ae30e48ea2b2), [#2433](https://github.com/morluto/rea/pull/2433), [#7000](https://github.com/morluto/rea/pull/7000), [#4399](https://github.com/morluto/rea/issues/4399). Thanks @0xthearchitect, @coygeek, @gensokyo-dev, @mvanhorn.
|
||||
- **Unity and managed code:** Inventory IL2CPP images/types/methods and metadata versions 24/27; read member names and literals, link native methods and generic instances, and export Ghidra seeds. Project managed call/field-access edges and page managed member summaries. [#3807](https://github.com/morluto/rea/pull/3807), [#5326](https://github.com/morluto/rea/pull/5326), [52d1a72](https://github.com/morluto/rea/commit/52d1a72af047a73b787b8935b032786a34966036), [#5327](https://github.com/morluto/rea/pull/5327), [#5785](https://github.com/morluto/rea/pull/5785), [#6260](https://github.com/morluto/rea/pull/6260), [#5167](https://github.com/morluto/rea/pull/5167), [#3784](https://github.com/morluto/rea/issues/3784), [#5112](https://github.com/morluto/rea/issues/5112), [#4056](https://github.com/morluto/rea/issues/4056), [#5148](https://github.com/morluto/rea/issues/5148). Thanks @8rulerstar, @alexlanghart, @elricfd, @gensokyo-dev, @jaelliot, @kaoru0822-kitauji, @Rinkia.
|
||||
- **Android:** Inspect APK signing declarations and structured manifest components; report static-final DEX field values. [e93e3b1](https://github.com/morluto/rea/commit/e93e3b1e7d6e7e4f49c2de24b92452ec29e853f8), [#4158](https://github.com/morluto/rea/pull/4158), [#5025](https://github.com/morluto/rea/pull/5025), [#6981](https://github.com/morluto/rea/pull/6981), [#4537](https://github.com/morluto/rea/issues/4537), [#6397](https://github.com/morluto/rea/issues/6397). Thanks @Abelo9996, @BaaRai, @FerroLx, @gensokyo-dev, @sserrgo12.
|
||||
- **Bytecode and firmware:** Inspect and query CPython pyc and BEAM facts, inspect flattened devicetrees, and attribute MMIO peripheral regions in native disassembly. [#4826](https://github.com/morluto/rea/pull/4826), [#6928](https://github.com/morluto/rea/pull/6928), [#4833](https://github.com/morluto/rea/pull/4833), [#6943](https://github.com/morluto/rea/pull/6943), [#7009](https://github.com/morluto/rea/pull/7009), [#5348](https://github.com/morluto/rea/pull/5348), [#6922](https://github.com/morluto/rea/issues/6922), [#6938](https://github.com/morluto/rea/issues/6938). Thanks @gensokyo-dev, @liziye627-design, @mvanhorn.
|
||||
- **JavaScript:** Retain queryable static route inventories, link assignment values to writes, and accept source selections as a JSON array or file. [#6019](https://github.com/morluto/rea/pull/6019), [#6206](https://github.com/morluto/rea/pull/6206), [#5029](https://github.com/morluto/rea/pull/5029), [#4772](https://github.com/morluto/rea/issues/4772), [#6193](https://github.com/morluto/rea/issues/6193). Thanks @gensokyo-dev, @IvenKooLab, @jacobmalibiran, @JoshOsullivan-au, @kaoru0822-kitauji, @KJLavender.
|
||||
- **Data inspection:** Decode selected protobuf/gRPC bodies, inspect GGUF/safetensors declarations, and produce byte-entropy profiles. [#5031](https://github.com/morluto/rea/pull/5031), [#5603](https://github.com/morluto/rea/pull/5603), [#2331](https://github.com/morluto/rea/pull/2331), [#4986](https://github.com/morluto/rea/issues/4986), [#3323](https://github.com/morluto/rea/issues/3323). Thanks @AayushJB03, @gensokyo-dev, @kaoru0822-kitauji, @KJLavender, @michaelkillgta, @mvanhorn.
|
||||
- **CLI input/output:** Accept structured request documents on stdin and expose version/runtime output formats. Row-oriented JSONL exports use the migration above. [#7040](https://github.com/morluto/rea/pull/7040), [#5486](https://github.com/morluto/rea/issues/5486), [#6530](https://github.com/morluto/rea/pull/6530), [169512a](https://github.com/morluto/rea/commit/169512a35bb015ff209168fb8bc1c9b31a2c678b), [#5483](https://github.com/morluto/rea/issues/5483). Thanks @Abelo9996, @ahmed-husssain, @CassieFox23, @DeryFerd, @eco-null, @ezrabrilliant, @GamerDubz, @gensokyo-dev, @gongfpp, @hifzabuildsai, @jayykernel, @mhamzasakhi, @pablodiazjorge, @Plastic-time, @raidnouredden-ux, @SandstromPL, @ShuaixinHuang, @Sriram-PR, @XiaoBinGan.
|
||||
- **MCP and setup:** Add opt-in Streamable HTTP and FreeBuff setup registration. FreeBuff support first appears in 6.8.0; installed native FreeBuff loading and skill discovery remain unverified. [#4955](https://github.com/morluto/rea/pull/4955), [#2736](https://github.com/morluto/rea/pull/2736), [#3243](https://github.com/morluto/rea/issues/3243). Thanks @0xLLVEL, @codebuff-team, @earakely-scale, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Recovery controls:** Adjust per-call WASM/SQLite/firmware deadlines, MCP startup budgets, regex matching deadlines/worker heap, capture decoder heap, ASAR header admission, Apple dispatch input, and the bridge request ceiling. MCP runtime limit recovery reports actionable settings. [#5798](https://github.com/morluto/rea/pull/5798), [#5818](https://github.com/morluto/rea/pull/5818), [aa53c06](https://github.com/morluto/rea/commit/aa53c065387c58b4189d82c47f1f3445fc505e30), [#5863](https://github.com/morluto/rea/pull/5863), [#6304](https://github.com/morluto/rea/pull/6304), [#6267](https://github.com/morluto/rea/pull/6267), [#6301](https://github.com/morluto/rea/pull/6301), [#6299](https://github.com/morluto/rea/pull/6299), [#6209](https://github.com/morluto/rea/pull/6209), [#6233](https://github.com/morluto/rea/pull/6233), [#6376](https://github.com/morluto/rea/pull/6376), [#7049](https://github.com/morluto/rea/pull/7049), [#5797](https://github.com/morluto/rea/issues/5797), [#5812](https://github.com/morluto/rea/issues/5812), [#5851](https://github.com/morluto/rea/issues/5851), [#6176](https://github.com/morluto/rea/issues/6176), [#3917](https://github.com/morluto/rea/issues/3917). Thanks @gensokyo-dev, @jaychou813, @kaoru0822-kitauji.
|
||||
- **APK output:** Set `string_tables_only` or CLI `--string-tables-only` to omit duplicate primary strings and avoid their output-capacity charge. [7c458f0](https://github.com/morluto/rea/commit/7c458f04b9155707c8ba406f8b455b10016f1b99), [#4990](https://github.com/morluto/rea/pull/4990), [#4982](https://github.com/morluto/rea/issues/4982). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Retained analysis:** Reuse successful validation of authenticated immutable native/imported-JavaScript parents and captures. Query retained Go build identity, Rust dependencies, JavaScript source-to-bundle mappings, and native export facts without repeating full output. [8f7d12e](https://github.com/morluto/rea/commit/8f7d12e30d63f3d3b4cfe2a90336362841438f56), [#4962](https://github.com/morluto/rea/pull/4962), [#5349](https://github.com/morluto/rea/pull/5349), [#5524](https://github.com/morluto/rea/pull/5524), [#5815](https://github.com/morluto/rea/pull/5815), [#6128](https://github.com/morluto/rea/pull/6128), [#6907](https://github.com/morluto/rea/pull/6907), [#4810](https://github.com/morluto/rea/issues/4810), [#5499](https://github.com/morluto/rea/issues/5499), [#5802](https://github.com/morluto/rea/issues/5802), [#6112](https://github.com/morluto/rea/issues/6112), [#6900](https://github.com/morluto/rea/issues/6900). Thanks @gensokyo-dev, @kaoru0822-kitauji, @N0zoM1z0.
|
||||
- **Compact results:** Use retained Frida/GDB/objdump summaries, omit redundant Ghidra raw results, and defer the analysis runtime during MCP catalog discovery. [#5661](https://github.com/morluto/rea/pull/5661), [#5614](https://github.com/morluto/rea/pull/5614), [#5355](https://github.com/morluto/rea/pull/5355), [#4424](https://github.com/morluto/rea/pull/4424), [#5640](https://github.com/morluto/rea/issues/5640), [#5584](https://github.com/morluto/rea/issues/5584), [#5337](https://github.com/morluto/rea/issues/5337). Thanks @coygeek, @gensokyo-dev, @kaoru0822-kitauji, @leeguooooo.
|
||||
- **Analysis work:** Reuse Hopper literal-string records, JavaScript workers and captured prefix selections, Interface Builder validation and plist scalar schemas; bound stable reads to one snapshot buffer and avoid decoding omitted GGUF strings. [#5017](https://github.com/morluto/rea/pull/5017), [#5819](https://github.com/morluto/rea/pull/5819), [31de177](https://github.com/morluto/rea/commit/31de177404461d122761d8243abe4fd24564f4da), [#5885](https://github.com/morluto/rea/pull/5885), [85e38ac](https://github.com/morluto/rea/commit/85e38ac199dff15e8c43343d9170db7e0e46d284), [#6966](https://github.com/morluto/rea/pull/6966), [#5781](https://github.com/morluto/rea/pull/5781), [#5010](https://github.com/morluto/rea/issues/5010), [#5855](https://github.com/morluto/rea/issues/5855), [#6954](https://github.com/morluto/rea/issues/6954), [#5774](https://github.com/morluto/rea/issues/5774). Thanks @gensokyo-dev, @kaoru0822-kitauji.
|
||||
- **Native scheduling:** Batch modern IDA callee lookups, schedule decompilation reads on demand, index NativeAOT type closure, and retain call-path ancestry once per visited node. [0c6fed9](https://github.com/morluto/rea/commit/0c6fed90b74c75a9fd23d1ec1b510a7274f85887), [06fe7ee](https://github.com/morluto/rea/commit/06fe7eecbc8c387414b4c1aa0d88ad6314f54f0c), [ce61680](https://github.com/morluto/rea/commit/ce61680c04c95a9cdfad5335632a99542af26d37), [5e4ec0f](https://github.com/morluto/rea/commit/5e4ec0fa1b678aa3bf20704ec9110224ba8c1676).
|
||||
- **Selected processing:** Prepare CLI positional schemas only for the selected command, separate entropy reads from report resolution, prune typed-array reference escapes, index state-write cutoffs, decode only selected replay query values, and skip payload text during WAT framing. [315b234](https://github.com/morluto/rea/commit/315b23403f0819967878cee418c3f9cb4b1a69d9), [#6950](https://github.com/morluto/rea/pull/6950), [#5139](https://github.com/morluto/rea/pull/5139), [#5350](https://github.com/morluto/rea/pull/5350), [96ea32b](https://github.com/morluto/rea/commit/96ea32b7e4e28d3c9cd79a1de4b8e1ba3b4244e9), [70835d3](https://github.com/morluto/rea/commit/70835d3d5f030deadf606e1c79c70a60883e50c9), [#5766](https://github.com/morluto/rea/pull/5766), [#6947](https://github.com/morluto/rea/issues/6947), [#5760](https://github.com/morluto/rea/issues/5760). Thanks @galleonli, @gensokyo-dev, @kaoru0822-kitauji.
|
||||
|
||||
### Fixes
|
||||
|
||||
- **IPS report fields:** Preserve the recorded terminating process and PID, termination flags, parent PID, and build/platform metadata. [#7079](https://github.com/morluto/rea/pull/7079), [#7078](https://github.com/morluto/rea/issues/7078). Thanks @rodricksz4h5.
|
||||
- **Ghidra language selection:** Accept stock language IDs containing `LEBE` or slash-separated processor names, and report conflicting DOL language overrides as configuration errors naming `REA_GHIDRA_LANGUAGE_ID`. [#7070](https://github.com/morluto/rea/pull/7070), [#7072](https://github.com/morluto/rea/issues/7072), [#7081](https://github.com/morluto/rea/pull/7081), [#7082](https://github.com/morluto/rea/issues/7082). Thanks @Adolanium.
|
||||
- **Setup and Hopper:** Preserve the Linux embedding runtime, supported x86_64 host restriction, readiness-based provider choices, orderly demo-helper shutdown, and installed apt dependency versions. Keep cleanup and skill publication ownership through retries. [#5041](https://github.com/morluto/rea/pull/5041), [#3422](https://github.com/morluto/rea/pull/3422), [f5d1012](https://github.com/morluto/rea/commit/f5d1012fabfe99af82523c3078ce9d93eb3780c1), [#4966](https://github.com/morluto/rea/pull/4966), [#6974](https://github.com/morluto/rea/pull/6974), [211e353](https://github.com/morluto/rea/commit/211e3533b34f81ae0910ad4f53f2a3d3c8587d3e), [#6920](https://github.com/morluto/rea/pull/6920), [#5015](https://github.com/morluto/rea/issues/5015), [#3419](https://github.com/morluto/rea/issues/3419), [#4921](https://github.com/morluto/rea/issues/4921), [#6975](https://github.com/morluto/rea/issues/6975). Thanks @Abel-I-Akhigbe, @Adolanium, @DavidFB-creator, @DeryFerd, @gensokyo-dev, @kaoru0822-kitauji, @sx4im.
|
||||
- **Client setup:** Treat empty APPDATA as unset, distinguish MSIX Claude Desktop process views, preserve aliased Hermes server tables, reuse client dialect/alias bindings, and materialize the pinned package-runner cache before the first start. [#5126](https://github.com/morluto/rea/pull/5126), [#4903](https://github.com/morluto/rea/pull/4903), [#3785](https://github.com/morluto/rea/pull/3785), [b7eeded](https://github.com/morluto/rea/commit/b7eededf5af5061a834f93f57f2c888acf870344), [#7032](https://github.com/morluto/rea/pull/7032), [#5104](https://github.com/morluto/rea/issues/5104), [#4278](https://github.com/morluto/rea/issues/4278), [#3320](https://github.com/morluto/rea/issues/3320), [#7027](https://github.com/morluto/rea/issues/7027). Thanks @CmsChase, @coygeek, @myso-kr, @pablodiazjorge, @vaidehi-snk, @wxj006007.
|
||||
- **Target admission:** Preserve thin arm64_32 Mach-O support, distinguish JVM identity before FAT admission, validate FAT format/CPU identity, preserve original DOS header padding, and reject NULs in local objdump/rizin paths. [#5788](https://github.com/morluto/rea/pull/5788), [6843dbe](https://github.com/morluto/rea/commit/6843dbea9bce4709ee2e8bf46c38f0bbf640716b), [497037e](https://github.com/morluto/rea/commit/497037e6e3975cbb37cfd89ea0ad300faa2a4769), [#7013](https://github.com/morluto/rea/pull/7013), [#3600](https://github.com/morluto/rea/pull/3600), [#5713](https://github.com/morluto/rea/issues/5713), [#6510](https://github.com/morluto/rea/issues/6510). Thanks @coygeek, @dhruvac29, @kaoru0822-kitauji, @Lew1sWong, @qorvr.
|
||||
- **Mach-O and dylibs:** Preserve container framing, facts when dyld rejects a missing UUID, verified-signature snapshot identity, unsigned diagnostics, absolute paths inside the selected root, legacy platform rules, lazy-load payloads, and reasons for rejecting present dylibs. [cf3288d](https://github.com/morluto/rea/commit/cf3288d8cf7a09685acf89103b36c32359be56fa), [2b263a7](https://github.com/morluto/rea/commit/2b263a7fe9e9d4c185c7d6af6b4e67b21a73af30), [eb7acc7](https://github.com/morluto/rea/commit/eb7acc70bf06f119e8639e30a78081d5ba82b0bf), [1502cfb](https://github.com/morluto/rea/commit/1502cfbc8ca0d1d1d86eebf334018dfb418bbac0), [90062c0](https://github.com/morluto/rea/commit/90062c092228dea9d33fadc0d7e1b968105b8b2c), [#5650](https://github.com/morluto/rea/pull/5650), [3b67b36](https://github.com/morluto/rea/commit/3b67b362aa2cd79eac5af57d0965e81a7d0b8c2b), [#5551](https://github.com/morluto/rea/issues/5551). Thanks @coygeek.
|
||||
- **Native facts:** Admit direct dispatch-metadata providers; preserve record-local Objective-C omissions, snapshot profiles, class-import binding identity, stripped class labels, version-like names, and qualified Swift protocol identities. [50b6975](https://github.com/morluto/rea/commit/50b6975d861069e2c6cfa202780621449f863802), [9eec57b](https://github.com/morluto/rea/commit/9eec57b469087fe9bff3d29dcc02f47074159e91), [0ba4d6f](https://github.com/morluto/rea/commit/0ba4d6f43ad610dacc861a1548be1a5c4712cd61), [4d7de00](https://github.com/morluto/rea/commit/4d7de00d4cc9a95ec2b7a3123b04e01db2daafa0), [22c84ce](https://github.com/morluto/rea/commit/22c84ce64aa6c7601aa81af86d4117b5a4c2feac), [4852c83](https://github.com/morluto/rea/commit/4852c830af1e3b9df437235b7a8fa8970a6d1c59), [aa9043f](https://github.com/morluto/rea/commit/aa9043f6b61c453e4735795cfe6a559bf6b6cc16).
|
||||
- **Apple resources:** Preserve independent plist scalars, processing instructions, nonstandard table widths, unsupported NIB connector coverage, and strict IPS JSON surrogate validation; classify Interface Builder objects by their declared superclass chains. [fd728f2](https://github.com/morluto/rea/commit/fd728f271e7de7bcfa63890b573715720d5f948c), [#5806](https://github.com/morluto/rea/pull/5806), [4526862](https://github.com/morluto/rea/commit/4526862728af01f1135714079c0d8a8c754eb364), [14cd78c](https://github.com/morluto/rea/commit/14cd78c3224edcf79fca152915077112f547b9ed), [#6990](https://github.com/morluto/rea/pull/6990), [#5805](https://github.com/morluto/rea/issues/5805), [#5228](https://github.com/morluto/rea/pull/5228), [#5218](https://github.com/morluto/rea/issues/5218). Thanks @coygeek, @DeryFerd, @kaoru0822-kitauji, @Mangolycheematcha, @mhamzasakhi, @Shreyash021104.
|
||||
- **Archives and files:** Complete positional ZIP reads, preserve bounded ASAR signatures, report stable symlink loops as resolution failures, retain target applicability, and report unknown traversal coverage. Retain reported integrity contradictions for members present in only one compared artifact. [#4414](https://github.com/morluto/rea/pull/4414), [e8da7e1](https://github.com/morluto/rea/commit/e8da7e1edd075feec0e6beedec93d5645fed8125), [#5571](https://github.com/morluto/rea/pull/5571), [#5763](https://github.com/morluto/rea/pull/5763), [#5867](https://github.com/morluto/rea/pull/5867), [#5521](https://github.com/morluto/rea/issues/5521), [#5731](https://github.com/morluto/rea/issues/5731), [#5837](https://github.com/morluto/rea/issues/5837), [#7076](https://github.com/morluto/rea/pull/7076). Thanks @Afloat16, @coygeek, @jdotc1, @kaoru0822-kitauji, @WenyuChiou.
|
||||
- **Extraction and file ownership:** Restrict destination-case recovery to owned collisions, retain physical-read failures through late cancellation/cleanup, preserve cleanup capabilities for retry, and retain independent firmware extraction failures. [932fbb7](https://github.com/morluto/rea/commit/932fbb7d84a9a995b9aee3c1be7da05a8c4d7bbb), [9499667](https://github.com/morluto/rea/commit/949966712e68d8857a50a165ecddf04c2d7e6d3b), [56679dd](https://github.com/morluto/rea/commit/56679dd2ae31dda58e181f8bf99775014f7bb82b), [b3c912a](https://github.com/morluto/rea/commit/b3c912ab79ad147a37b6cfbe1552a1a5eecec629), [7298bd0](https://github.com/morluto/rea/commit/7298bd08d134860b84b4073f6703021916a5a4c5).
|
||||
- **Processes and cancellation:** Retain partial samples and shutdown gaps, unresolved descendants and caller abort reasons; bind cleanup errors to their owners, propagate comparison contradictions, and report no normal exit code for signalled targets. [ba866a2](https://github.com/morluto/rea/commit/ba866a26ecbe179a717e746cbc0d1605ae616ad5), [#5270](https://github.com/morluto/rea/pull/5270), [#6144](https://github.com/morluto/rea/pull/6144), [a721c42](https://github.com/morluto/rea/commit/a721c42c07dc674c32ab939627355b8a1bee97a5), [#6298](https://github.com/morluto/rea/pull/6298), [#6486](https://github.com/morluto/rea/pull/6486), [#5935](https://github.com/morluto/rea/issues/5935), [#5944](https://github.com/morluto/rea/issues/5944), [#6100](https://github.com/morluto/rea/issues/6100), [#6101](https://github.com/morluto/rea/issues/6101), [#6110](https://github.com/morluto/rea/issues/6110), [#6113](https://github.com/morluto/rea/issues/6113), [#6121](https://github.com/morluto/rea/issues/6121), [#6131](https://github.com/morluto/rea/issues/6131), [#6132](https://github.com/morluto/rea/issues/6132), [#6218](https://github.com/morluto/rea/issues/6218), [#6445](https://github.com/morluto/rea/issues/6445). Thanks @agorevski, @coygeek, @gongfpp, @kwakhyun, @mhamzasakhi.
|
||||
- **Provider cleanup:** Preserve Windows native handles until close is confirmed, native/Inspector/Electron cleanup ownership, Kuna probe identity and cleanup, IDA registration configuration, and Hopper preparation ownership. [#5772](https://github.com/morluto/rea/pull/5772), [87ab054](https://github.com/morluto/rea/commit/87ab054ea384b3fcbae2998b1f94685edcd5f624), [ff78165](https://github.com/morluto/rea/commit/ff78165978de9dbcf9fa16c2acf74bf85a69dc44), [04ae07a](https://github.com/morluto/rea/commit/04ae07ab8c9ae1a6697b6ad15489bbde515da8b5), [dd0015a](https://github.com/morluto/rea/commit/dd0015aaf156c5701ef9598bcc6c885e6b8da803), [b231c28](https://github.com/morluto/rea/commit/b231c28cf64c2f799ada99343f8f21e0e6acb15e), [#4505](https://github.com/morluto/rea/issues/4505), [#5616](https://github.com/morluto/rea/issues/5616).
|
||||
- **Provider diagnostics:** Give Ghidra restart recovery, stable Homebrew aliases, unsupported-host results for Linux-only integrations, correct IDA regex/readiness handling, and clear diagnostics when Windows refuses a verified native addon. [#5356](https://github.com/morluto/rea/pull/5356), [7c6142e](https://github.com/morluto/rea/commit/7c6142efdc19a42c148d0e8c96b69d18e63c7f87), [#5793](https://github.com/morluto/rea/pull/5793), [#4862](https://github.com/morluto/rea/pull/4862), [99ccfb6](https://github.com/morluto/rea/commit/99ccfb6e709ac4bc0afe176315280abde9ebcb9f), [#6832](https://github.com/morluto/rea/pull/6832), [#5730](https://github.com/morluto/rea/issues/5730), [#4860](https://github.com/morluto/rea/issues/4860), [#5723](https://github.com/morluto/rea/issues/5723). Thanks @coygeek, @dark13b, @gensokyo-dev, @jgt87, @jpmorris41488, @Lew1sWong.
|
||||
- **MCP results:** Preserve protocol envelopes on oversized errors, enforce and advertise absolute Evidence paths, retain typed output-filter errors and selected JSON paths, and give client/provider timeout recovery. Compact advertisements now normalize reported enum and union shapes for strict gateway translation. [562dab0](https://github.com/morluto/rea/commit/562dab00c5cbf76a9b2695bad93792eccdc9a49d), [3cf6206](https://github.com/morluto/rea/commit/3cf6206614e65b5ebb6284a5709090dfb8f9aaee), [0e83d92](https://github.com/morluto/rea/commit/0e83d92c583ae36afe00b2c23dd2a0f1d025cbca), [4714c7e](https://github.com/morluto/rea/commit/4714c7eaa3d1d8b6a5df0aff586e844b9d9ab06d), [1bfa60a](https://github.com/morluto/rea/commit/1bfa60ac20e1a344bcf3132c08d15b6860b05c61), [#5608](https://github.com/morluto/rea/pull/5608), [#3806](https://github.com/morluto/rea/pull/3806), [#5598](https://github.com/morluto/rea/issues/5598), [#3800](https://github.com/morluto/rea/issues/3800). Thanks @gensokyo-dev, @Hahaa909, @wang1xiang.
|
||||
- **Browser lifecycles:** Retain historical outcomes through cleanup, stop late navigation before session release, stop expired work, observe aborted fetch settlement, and mark later scenario actions skipped after failure or cancellation. [2e5171e](https://github.com/morluto/rea/commit/2e5171e391b83f8a467d0861fd452c1f78621fd8), [599fdac](https://github.com/morluto/rea/commit/599fdac029912f96c511f859660483405d2de26f), [b2fad36](https://github.com/morluto/rea/commit/b2fad362f2b37f81dc9a6f784f4521e4ca24d41e), [13a8640](https://github.com/morluto/rea/commit/13a864020414577006dce58d7fdaa877ddd0cb52), [#6023](https://github.com/morluto/rea/pull/6023), [#5943](https://github.com/morluto/rea/issues/5943). Thanks @coygeek, @kaoru0822-kitauji.
|
||||
- **Browser comparisons:** Preserve frame identity, default origin scope, unknown body/shape coverage, correspondence and uncertainty, and independent retained-observation budgets. [#4040](https://github.com/morluto/rea/pull/4040), [#4413](https://github.com/morluto/rea/pull/4413), [#4476](https://github.com/morluto/rea/pull/4476), [a0c22ff](https://github.com/morluto/rea/commit/a0c22ff0125b8af7612db54d3708413e2bfecb25), [a0fcc60](https://github.com/morluto/rea/commit/a0fcc60d89eb0cc09b08181eb9f961acfd07d8e9), [#3878](https://github.com/morluto/rea/issues/3878), [#3910](https://github.com/morluto/rea/issues/3910), [#3941](https://github.com/morluto/rea/issues/3941), [#4385](https://github.com/morluto/rea/issues/4385), [#4477](https://github.com/morluto/rea/issues/4477). Thanks @Adolanium, @coygeek, @eco-null, @gensokyo-dev, @kaoru0822-kitauji, @XY-code1.
|
||||
- **Electron observations:** Preserve listener callback ownership and count registrations by source callsite; block trash calls in the active hook, retain window handles through cancelled selection, and preserve cancellation during finalization. [dfe8378](https://github.com/morluto/rea/commit/dfe8378cf5a727e3d93d47a5986233175c6ef898), [5064007](https://github.com/morluto/rea/commit/506400748eb1d6c674f0cf51330073929c45f939), [816a9dc](https://github.com/morluto/rea/commit/816a9dc98886e6d824cd8ae88b68731b161d5dda), [1707d17](https://github.com/morluto/rea/commit/1707d171d750955bf6387e028e378183b360ffe3), [4e2a566](https://github.com/morluto/rea/commit/4e2a566d6c7e053ecbbee1b684311ed31ea163bf).
|
||||
- **JavaScript source maps:** Keep unmapped and missing mappings unknown, preserve partial indexed-map facts and section paths, propagate preparation capability errors, and retain observed worker failures through cancellation. [a4dbb36](https://github.com/morluto/rea/commit/a4dbb36d8ea83e9f94dd289ac1a2d220d2c72c9b), [2b2e362](https://github.com/morluto/rea/commit/2b2e362a00fc5ba2f93fa6c62e2d8780072d1b4b), [498a667](https://github.com/morluto/rea/commit/498a667234b0140c1cfa43c819b35454ab11d14e), [#6149](https://github.com/morluto/rea/pull/6149), [#6214](https://github.com/morluto/rea/pull/6214), [f01dcca](https://github.com/morluto/rea/commit/f01dccab77f712fb4ac936d0a6e92a70d12f0789), [#6082](https://github.com/morluto/rea/issues/6082), [#6061](https://github.com/morluto/rea/issues/6061). Thanks @coygeek, @diyasharma05, @kaoru0822-kitauji.
|
||||
- **JavaScript semantics:** Preserve overflowing timer uncertainty, unavailable-scope comparisons, directory-form URL/ESM path intent, and artifact confinement. Replaced globals lose built-in effects; erased imports no longer become runtime bindings. Treat a known truthy `fork()` `silent` option as piped stdio unless a string/array `stdio` override takes precedence; unresolved options stay dynamic. [#3456](https://github.com/morluto/rea/pull/3456), [8d198d6](https://github.com/morluto/rea/commit/8d198d6ec707402d75276f516e0090d71767b7e4), [#4646](https://github.com/morluto/rea/pull/4646), [#2700](https://github.com/morluto/rea/pull/2700), [#4565](https://github.com/morluto/rea/pull/4565), [#6994](https://github.com/morluto/rea/pull/6994), [#3400](https://github.com/morluto/rea/issues/3400), [#3882](https://github.com/morluto/rea/issues/3882), [#3367](https://github.com/morluto/rea/issues/3367), [#7083](https://github.com/morluto/rea/pull/7083), [#7084](https://github.com/morluto/rea/issues/7084). Thanks @Adolanium, @amyanger, @coygeek, @gensokyo-dev, @HawkForge, @rohitg00, @viniciusdsandrade.
|
||||
- **JavaScript routes and imports:** Treat path data as route candidates, preserve route flag and lexical RPC identities, package-main trailing-slash selection, catch aliases, and factory parameter ownership. [#6360](https://github.com/morluto/rea/pull/6360), [38ecc02](https://github.com/morluto/rea/commit/38ecc0248e1993d25a2f7dd350024484cce797bb), [#4219](https://github.com/morluto/rea/pull/4219), [21097b1](https://github.com/morluto/rea/commit/21097b192b1ab94647e3446b1a2a4dd00e924d5e), [f774dc4](https://github.com/morluto/rea/commit/f774dc420b3befe2b8e68971b4116da2d62fc800), [#6349](https://github.com/morluto/rea/issues/6349), [#4189](https://github.com/morluto/rea/issues/4189). Thanks @coygeek, @hemv-857, @kaoru0822-kitauji, @Nikhi00718.
|
||||
- **Managed/IL2CPP:** Preserve extension-only scope uncertainty and type discovery after member limits, nested names, ReadyToRun/declaration facts, selected type scope, valid shared-suffix strings, and truncated IL2CPP metadata prefixes. [#5674](https://github.com/morluto/rea/pull/5674), [#5312](https://github.com/morluto/rea/pull/5312), [87fa1b6](https://github.com/morluto/rea/commit/87fa1b613d2b6bda1722855a00f1390f27731b64), [bc2d494](https://github.com/morluto/rea/commit/bc2d494030efd1ccabaeaa0bd49226aa87c378a2), [1f0eb22](https://github.com/morluto/rea/commit/1f0eb228e50908ee2a2aaa72d329178ae1a2cdbd), [4c5fae5](https://github.com/morluto/rea/commit/4c5fae5179dd54c4a0d427cd2a115b9c887e3841), [b3aadaf](https://github.com/morluto/rea/commit/b3aadaf474e59b33e5ff465236008a67eccefb87), [#5467](https://github.com/morluto/rea/issues/5467), [#5146](https://github.com/morluto/rea/issues/5146). Thanks @coygeek, @eco-null, @elricfd, @umi008.
|
||||
- **Format boundaries:** Preserve independent facts across malformed inputs in model, GPU, WASM, Rust, Android, and SQLite inspection; report optional metadata warnings without discarding core results. [eb41f35](https://github.com/morluto/rea/commit/eb41f358cf83d5626f09de68332f9718f9aa6482), [#5789](https://github.com/morluto/rea/pull/5789), [00a9e95](https://github.com/morluto/rea/commit/00a9e95de91a4bd4d2a41f8aee704a53fa575750), [fc63fe8](https://github.com/morluto/rea/commit/fc63fe8682654086261540dcf3c59afc36c7d58f), [39a91d4](https://github.com/morluto/rea/commit/39a91d4db96ca13224c25936330ea84fe78970e5), [6c6e2ad](https://github.com/morluto/rea/commit/6c6e2ad62b20f810b54f5a567e66fc55241514b2), [929a0a7](https://github.com/morluto/rea/commit/929a0a76c870d9991e015dde58308fd9df0280b8), [d137a4c](https://github.com/morluto/rea/commit/d137a4cdde0049de967f1d5cd429ed9e1f7d0a10), [ef475fc](https://github.com/morluto/rea/commit/ef475fcf8b0601c61052b7091568ba574546417d), [#5749](https://github.com/morluto/rea/issues/5749). Thanks @coygeek, @gensokyo-dev.
|
||||
- **Capture decoding:** Preserve malformed protobuf trailer facts and wire/schema identity, HTTP/2 URI and UTF-8 failure facts, WebMCP property coverage, and offline capture availability without a browser provider. Report invalid record ordinals as selection errors with actual bounds where available, and trim surrounding SP/HTAB from gRPC-web trailer values while preserving internal whitespace. [9cf7a7c](https://github.com/morluto/rea/commit/9cf7a7cb0d3c72a1121fde280cb11bf911af8055), [b3c3b21](https://github.com/morluto/rea/commit/b3c3b21e43cab4e71b460d08829d8c6e4e9b9c21), [4cb1f1d](https://github.com/morluto/rea/commit/4cb1f1d820df4498ee464bcf72a5f47d40cf8dab), [51e5938](https://github.com/morluto/rea/commit/51e593877642ab42fb5bf25812f4a4110ee7ac26), [d23be5c](https://github.com/morluto/rea/commit/d23be5c0898c39c9f0c54e2e0b43ee746d92d316), [#6725](https://github.com/morluto/rea/pull/6725), [#3947](https://github.com/morluto/rea/issues/3947), [#7067](https://github.com/morluto/rea/pull/7067), [#7068](https://github.com/morluto/rea/issues/7068), [#7069](https://github.com/morluto/rea/pull/7069), [#7071](https://github.com/morluto/rea/issues/7071). Thanks @Adolanium, @coygeek, @Sriram-PR.
|
||||
- **Evidence and errors:** Authenticate ledger Evidence before committing; preserve parent-schema diagnostics, canonical correction details and owned cleanup outcomes; identify the rejected function-Evidence side/kind. Fixed allowances now name their actual limits. [0407bbf](https://github.com/morluto/rea/commit/0407bbfbe65e8fa475e8bb0c78c30170e2c2f55e), [#6361](https://github.com/morluto/rea/pull/6361), [1629266](https://github.com/morluto/rea/commit/1629266e3c08c90fed6bba5383528af7986a12b7), [#6969](https://github.com/morluto/rea/pull/6969), [#5860](https://github.com/morluto/rea/pull/5860), [#6034](https://github.com/morluto/rea/issues/6034), [#5932](https://github.com/morluto/rea/issues/5932), [#5847](https://github.com/morluto/rea/issues/5847). Thanks @AyoubElleuch, @coygeek, @dhruvac29, @kaoru0822-kitauji.
|
||||
- **ADB and firmware recovery:** Report an unparsed push checksum as unverified, normalize absent installer-package sentinels, describe extraction-relative Unblob links, and avoid recommending a budget increase at its fixed ceiling. [#6010](https://github.com/morluto/rea/pull/6010), [#4073](https://github.com/morluto/rea/pull/4073), [#4573](https://github.com/morluto/rea/pull/4573), [#6971](https://github.com/morluto/rea/pull/6971), [#5924](https://github.com/morluto/rea/issues/5924), [#4175](https://github.com/morluto/rea/issues/4175). Thanks @coygeek, @Ganesh2325, @kaoru0822-kitauji, @Rinkia, @XiaoBinGan, @yankikucuk.
|
||||
- **CLI formatting:** Share Markdown report formatting, exclude inherited row values, and preserve arrays of empty objects as JSON. Render Markdown table headers and cells literally while keeping intraword underscores readable. [b784912](https://github.com/morluto/rea/commit/b784912811fd10ac9dc425696dc0e9af632afa34), [#6984](https://github.com/morluto/rea/pull/6984), [#7058](https://github.com/morluto/rea/pull/7058), [#7057](https://github.com/morluto/rea/issues/7057), [#7073](https://github.com/morluto/rea/pull/7073). Thanks @Plastic-time, @ShuaixinHuang, @viniciusdsandrade.
|
||||
- **Format identity:** Preserve Python marshal scalar/decode limits, empty optional BEAM chunks, PE resource branch identities and certificate alignment, unsupported compressed ELF notes, and codec-framed GPU bundle extents. [0603223](https://github.com/morluto/rea/commit/06032237c6755e3dd5f9a8db732c89958b238cc7), [c46064c](https://github.com/morluto/rea/commit/c46064c3f6eaba121c5de7b17390bfdecdd132ef), [66671e8](https://github.com/morluto/rea/commit/66671e874888e98f7e2b27e68a7c6fb320bbb827), [03077d5](https://github.com/morluto/rea/commit/03077d54afc49613db4e38f70c506bc60f5223aa), [#6721](https://github.com/morluto/rea/pull/6721), [9fb8c11](https://github.com/morluto/rea/commit/9fb8c114a03ad60e57c11586967c6df845a3f97e), [f6a1e2c](https://github.com/morluto/rea/commit/f6a1e2c892dc23c7583c3d03af9edb01632ff5a5), [#6694](https://github.com/morluto/rea/issues/6694). Thanks @coygeek, @Deepak-Kambala.
|
||||
- **WASM and APK metadata:** Preserve ambiguous WASM identity and partial facts, glue URL query/fragment text across line terminators, APK signer identity, and raw issuer identity when comparison rules are unknown. [af8f0ba](https://github.com/morluto/rea/commit/af8f0badf64fd3546392b6aa4625777904b2f051), [#6802](https://github.com/morluto/rea/pull/6802), [f783f0f](https://github.com/morluto/rea/commit/f783f0f1e82472cdddd4de2156d974f729e332a6), [115a78f](https://github.com/morluto/rea/commit/115a78fe241276c34917b496621954c5f29bfc15), [#6639](https://github.com/morluto/rea/issues/6639). Thanks @coygeek, @gensokyo-dev, @gongfpp.
|
||||
- **Source identity:** Preserve Next route spelling, module grammar and loader request identity, directory package-main behavior, accessor effects, exact source targets, and resolver-discovery versus module-loading distinctions. [ddd7ac7](https://github.com/morluto/rea/commit/ddd7ac7b846b48191ead1f7dcd0e3bd735468fa8), [9974512](https://github.com/morluto/rea/commit/99745126009b15060d71c38809418858a07d5afa), [f2e5fd1](https://github.com/morluto/rea/commit/f2e5fd148a85fa737302dd194a33172c4561093e), [baf1678](https://github.com/morluto/rea/commit/baf16784af3592534687052431c145daef55cfd1), [6e324b2](https://github.com/morluto/rea/commit/6e324b224405f2befd953234f6dc210cf72ad50a), [75daf0c](https://github.com/morluto/rea/commit/75daf0c7df108f04ab41f34aec1a31d4a7ceb322), [f91b7f8](https://github.com/morluto/rea/commit/f91b7f8009c9af27a5e0148e3b07d20bfe3b7a2a).
|
||||
- **Graphs and selected facts:** Preserve JavaScript graph omissions/version facts, Electron native-loader facts, web JSON identity/coverage, module-path selection, feature-trace qualifications, selected native paths, endpoint context, and mixed Swift demangler outcomes. [30db499](https://github.com/morluto/rea/commit/30db499db310f073d9bf3d3acfd77985e1cb7e2e), [5a7bddb](https://github.com/morluto/rea/commit/5a7bddb3aeb0345854e76b79d529aed1c40e2054), [79805f2](https://github.com/morluto/rea/commit/79805f244a1ec0850f3109929168a9c8ebb1b908), [f719dc4](https://github.com/morluto/rea/commit/f719dc46c51a55b3926eec9659d06e7dfb95132b), [7784fdb](https://github.com/morluto/rea/commit/7784fdbad78f8b384d513f0e7dae8e983cd48ef8), [76500f1](https://github.com/morluto/rea/commit/76500f12b54e07b1ea700a403b2593890d63b42a), [676d29f](https://github.com/morluto/rea/commit/676d29f9eef794cea668f5f09f0ff510c848a513), [edf7b3d](https://github.com/morluto/rea/commit/edf7b3d32c3248f221e27f7ce5a3d34e8590285b), [93de7bc](https://github.com/morluto/rea/commit/93de7bc7e324a761162a31625b838ba0d4e3824d).
|
||||
- **Managed boundaries:** Honor native platform symbol lookup, consume resolved IL2CPP ELF declarations, and attribute malformed comparison inputs to the supplied argument. [64fa1db](https://github.com/morluto/rea/commit/64fa1db1309218283df19f3ee4dba61f5fbd2a67), [87745af](https://github.com/morluto/rea/commit/87745af52ab0ccd3a172c2534c516d41b0095035), [96ad94f](https://github.com/morluto/rea/commit/96ad94f69eda5d68c7335ff775b6927b1376fd2b).
|
||||
- **Provider and worker recovery:** Preserve output loss after successful command exit, physical refusal diagnostics, bounded worker control framing, JavaScript worker-phase/budget attribution, admitted ambiguity choices, and selected EVM limiter availability. Scope objdump debug-link limitations to DWARF requests and keep process-capture preparation nonexecuting. [e639edf](https://github.com/morluto/rea/commit/e639edf6204c05285d96840478926592daf95876), [171b33b](https://github.com/morluto/rea/commit/171b33b2c4defc6cbea49545bedf4272b72d1ea2), [34563fc](https://github.com/morluto/rea/commit/34563fcc9bd92af30668820dcac5635e27181094), [3f9357b](https://github.com/morluto/rea/commit/3f9357b5990d166a38440f56f439452266650093), [cec8f1e](https://github.com/morluto/rea/commit/cec8f1e8f305d6332272457a6ee419d3c5fb6359), [a74d859](https://github.com/morluto/rea/commit/a74d859a19122fb8f1282b3632d320997b1caae0), [c284168](https://github.com/morluto/rea/commit/c284168c4b1cd214ce7f504d43da28528d82d603), [4fb936f](https://github.com/morluto/rea/commit/4fb936fe86bb7a07f94f819ae8510301f2ef69c9).
|
||||
- **Setup and limits:** Retain unknown environment-budget facts, selected-skill/maintenance uncertainty, Hopper URL admission and bookmark verification, and listener failure recovery that matches the system cause. [a718037](https://github.com/morluto/rea/commit/a7180370332350bc0f998fbd643888221759570e), [d1fbebb](https://github.com/morluto/rea/commit/d1fbebb578538dc94fd5e376e6e1959218efa835), [60accad](https://github.com/morluto/rea/commit/60accad2de8f2862fe91901502fe26cc5390fb36), [fd7da77](https://github.com/morluto/rea/commit/fd7da77fc02dcfaf7d1bc9f2fb240d71cdd27c59), [0c440bf](https://github.com/morluto/rea/commit/0c440bfc07f3cb343e2e836d27cccaed539cfe9c).
|
||||
- **State and Evidence:** Retain export tarball ownership and SQLite primary outcomes through cleanup retry, count emitted SQLite schema names across encodings, scope display refresh to matching snapshot bindings, and report retained-result validation failures consistently. Reconstruction uses one effective completion scope and kind-based labels when names are blank. [ab58edc](https://github.com/morluto/rea/commit/ab58edcdf50b4bd687900ee2dc362ecc5c3dce15), [1006714](https://github.com/morluto/rea/commit/100671427105791a5c5913a918b63a394f6bc83e), [a47788a](https://github.com/morluto/rea/commit/a47788a6a4abb067273f954275a9d0a269adf1e7), [8aa40e9](https://github.com/morluto/rea/commit/8aa40e9a61155cff8b6f3c138c601bf805b20653), [b8c1b45](https://github.com/morluto/rea/commit/b8c1b45316882beeab4a9de49381d94e131fab0f), [a35b933](https://github.com/morluto/rea/commit/a35b9335a0124674ee5e3abe8c075f1fff6b4fa6), [#6137](https://github.com/morluto/rea/pull/6137), [#6126](https://github.com/morluto/rea/issues/6126). Thanks @carlogiovannineri-rgb, @mohithhhh.
|
||||
- **Capture coverage:** Name browser bundle budgets and deadline recovery, separate domain readiness from worker attribution, attribute rendered-frame omissions to their producing stage, share offline-capture host admission with discovery, preserve selected signal-file semantics, and retain short Flutter GNU build IDs. [#6036](https://github.com/morluto/rea/pull/6036), [ba78133](https://github.com/morluto/rea/commit/ba781330d6d28b22d15498e63c144fb40e086156), [5934387](https://github.com/morluto/rea/commit/5934387f4185129efe9dc8b7338d8bc3fa5b25bd), [ad629b2](https://github.com/morluto/rea/commit/ad629b2e31c3a9495b27179360b508fa021fe86a), [928ba76](https://github.com/morluto/rea/commit/928ba76882bfd216dde8ed79118789b2c3aad4f9), [678c4cf](https://github.com/morluto/rea/commit/678c4cfa68e2fee12f620075e12dc87d9b1195b0).
|
||||
- **Additional boundary facts:** Retain parser format/query occurrence identity and independent malformed-input facts, unresolved nested Swift identifiers, and completed signing-capture coverage; describe crash documents without assuming a provider. [5f702dd](https://github.com/morluto/rea/commit/5f702dd6ccd9e8e505e224854b91678439c3b41f), [99d2d7a](https://github.com/morluto/rea/commit/99d2d7ad22bcbd780919760008aba6a88e7d1077), [90e433d](https://github.com/morluto/rea/commit/90e433df06b3f83417176b8db1b0a905f9df1d3a), [#5548](https://github.com/morluto/rea/pull/5548), [ee12d51](https://github.com/morluto/rea/commit/ee12d517a25d34853d40e31fc039c7d1576c60e1), [#5538](https://github.com/morluto/rea/issues/5538). Thanks @coygeek, @ShuaixinHuang.
|
||||
- **Documentation:** Add the Romanian README. [#7026](https://github.com/morluto/rea/pull/7026). Thanks @An3treiu.
|
||||
|
||||
Full contribution record: [1740 commits and 497 merged PRs](https://github.com/morluto/rea/blob/rea-agents-6.8.0/docs/releases/6.8.0.contributions.json).
|
||||
|
||||
## [6.7.0](https://github.com/morluto/rea/compare/rea-agents-6.6.0...rea-agents-6.7.0) (2026-10-18)
|
||||
|
||||
### Highlights
|
||||
|
||||
@@ -15,11 +15,6 @@ import sys
|
||||
PROFILE = "pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2"
|
||||
PACKAGES = {"pwntools": "4.15.0", "pyelftools": "0.33", "unicorn": "2.1.2"}
|
||||
OUTPUT_BYTES = 64 * 1024 * 1024
|
||||
# (e_machine, ELF class bits, byte order) -> architecture pwntools must also report.
|
||||
PROFILES = {(62, 64, "little"): "amd64", (3, 32, "little"): "i386", (40, 32, "little"): "arm",
|
||||
(8, 32, "little"): "mips", (8, 32, "big"): "mips"}
|
||||
PROFILE_SCOPE = ("Layout profile requires ELF64 x86-64, ELF32 i386, ELF32 ARM little-endian "
|
||||
"or ELF32 MIPS little- or big-endian objects.")
|
||||
|
||||
|
||||
class LayoutFailure(Exception):
|
||||
@@ -121,16 +116,13 @@ def inspect_elf(path, cache):
|
||||
raise LayoutFailure("format", "Selected object does not contain an ELF header.")
|
||||
if len(content) < 16:
|
||||
raise LayoutFailure("format", "Selected ELF identification is truncated.")
|
||||
if content[4] not in (1, 2) or content[5] not in (1, 2):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF declares an unknown class or byte order.")
|
||||
bits, header_size = (32, 52) if content[4] == 1 else (64, 64)
|
||||
order = "little" if content[5] == 1 else "big"
|
||||
if len(content) < header_size:
|
||||
raise LayoutFailure("format", f"Selected ELF{bits} header is truncated.")
|
||||
profile = (int.from_bytes(content[18:20], order), bits, order)
|
||||
if profile not in PROFILES:
|
||||
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
|
||||
if int.from_bytes(content[16:18], order) not in (1, 2, 3):
|
||||
if content[4] != 2 or content[5] != 1:
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires ELF64 little-endian objects.")
|
||||
if len(content) < 64:
|
||||
raise LayoutFailure("format", "Selected ELF64 header is truncated.")
|
||||
if int.from_bytes(content[18:20], "little") != 62:
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF objects.")
|
||||
if int.from_bytes(content[16:18], "little") not in (1, 2, 3):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
|
||||
from pwnlib.context import context
|
||||
with context.local(cache_dir=str(cache), log_level="warning"):
|
||||
@@ -193,8 +185,8 @@ def inspect_elf(path, cache):
|
||||
if type is None or section.header.sh_type == type)
|
||||
try:
|
||||
image = LayoutELF(str(path), checksec=False)
|
||||
if (image.arch, image.bits, image.endian) != (PROFILES[profile], bits, order):
|
||||
raise LayoutFailure("unsupported-target", PROFILE_SCOPE)
|
||||
if image.arch != "amd64" or image.bits != 64 or image.endian != "little":
|
||||
raise LayoutFailure("unsupported-target", "Initial layout profile requires x86-64 ELF64 little-endian objects.")
|
||||
if image.header.e_type not in ("ET_EXEC", "ET_DYN", "ET_REL"):
|
||||
raise LayoutFailure("unsupported-target", "Selected ELF type is outside the EXEC/DYN/REL layout profile; recorded core analysis is separate.")
|
||||
length = len(content)
|
||||
@@ -216,7 +208,7 @@ def inspect_elf(path, cache):
|
||||
})
|
||||
if h.sh_type in ("SHT_SYMTAB", "SHT_DYNSYM"):
|
||||
if h.sh_entsize < image.structs.Elf_Sym.sizeof():
|
||||
raise LayoutFailure("format", f"ELF symbol entry is smaller than the decoded Elf{bits}_Sym structure.")
|
||||
raise LayoutFailure("format", "ELF symbol entry is smaller than the decoded Elf64_Sym structure.")
|
||||
if h.sh_link >= len(all_sections) or all_sections[h.sh_link].header.sh_type != "SHT_STRTAB":
|
||||
raise LayoutFailure("format", f"ELF symbol table {index} links to invalid string table {h.sh_link}.")
|
||||
strings = all_sections[h.sh_link]
|
||||
|
||||
+1
-4
@@ -31,10 +31,7 @@ stackshots, return no crash result.
|
||||
`process`, `exception`, `termination`, `faulting_thread`, `threads`, and
|
||||
`images` reproduce documented body fields when present, including each image's
|
||||
`name` and each frame's `symbolLocation` (as `symbol_location`, the byte offset
|
||||
from the start of `recorded_symbol`). `termination` keeps the terminating
|
||||
process as `by_proc` and `by_pid` and its `flags`; `process.parent_pid` keeps
|
||||
`parentPid` next to `parent`; and `metadata` keeps `build_version` and
|
||||
`platform`. Integers keep their
|
||||
from the start of `recorded_symbol`). Integers keep their
|
||||
decimal literals and a hexadecimal form, including arm64e registers above
|
||||
2^53. Frame roles follow the recorded frame index: row zero of the resolved
|
||||
faulting thread is `faulting_instruction`, row zero of other threads is
|
||||
|
||||
@@ -8,12 +8,9 @@ ordinary binary inspection use the same modular capability.
|
||||
|
||||
## Bring your own engine
|
||||
|
||||
Real verification covers **Linux x64** hosts and **EXEC/DYN/REL** objects in
|
||||
these profiles: ELF64 x86-64, ELF32 i386, ELF32 little-endian ARM and ELF32
|
||||
little- or big-endian MIPS. Other machines, classes and byte orders, including
|
||||
AArch64 and big-endian ARM, return `unsupported_target`. Provide an absolute
|
||||
Python executable whose environment already contains unchanged pwntools 4.15.0,
|
||||
pyelftools 0.33 and Unicorn 2.1.2:
|
||||
Initial real verification covers **Linux x64, ELF64 x86-64 little-endian
|
||||
EXEC/DYN/REL**. Provide an absolute Python executable whose environment already
|
||||
contains unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2:
|
||||
|
||||
```sh
|
||||
REA_PWNTOOLS_PYTHON=/absolute/isolated-env/bin/python \
|
||||
|
||||
@@ -621,58 +621,3 @@ rules and preserve the same missing fact.
|
||||
Network normalization also commits the removal of capture-local transaction IDs,
|
||||
attribution to the requesting step, and unordered transaction comparison. These
|
||||
built-in rule identities participate in the normalization digest.
|
||||
|
||||
## Summary delivery and retained views
|
||||
|
||||
`capture_browser_scenario` accepts `detail: "summary"` to retain the complete
|
||||
capture in the MCP session and return its summary. The default remains
|
||||
`complete`. Delivery detail does not change scenario parameters or identity.
|
||||
|
||||
The summary reports each step's status, error, URL and captured/missing/truncated
|
||||
section names. It omits `not_requested` sections and repeated `before_url` values.
|
||||
The accessibility tree comes from the first unsuccessful step, or the final
|
||||
step on success; when that step captured no tree, its nearest earlier captured
|
||||
tree is identified explicitly. Event and network counts cover retained records,
|
||||
with capture omissions reported separately.
|
||||
|
||||
For the one-shot CLI, save the complete Evidence before selecting a summary:
|
||||
|
||||
```bash
|
||||
rea capture-browser-scenario scenario.json --detail summary --evidence_output /absolute/path/scenario-evidence.json --json
|
||||
```
|
||||
|
||||
Use the returned `normalized_result.parent_evidence_id` with
|
||||
`inspect_analysis_view` source `{kind: "retained-evidence", evidence_id: "..."}`.
|
||||
For a CLI bundle, use an `evidence-file` source with its absolute path and the
|
||||
parent Evidence ID. Views read the recorded capture; they never replay actions.
|
||||
|
||||
- `{kind: "summary"}` returns the same overview.
|
||||
- `{kind: "browser-scenario", collection: "steps", step_indices: [2], artifacts: ["accessibility"], offset: 0, limit: 1}` returns selected recorded steps and artifacts.
|
||||
- `{kind: "browser-scenario", collection: "events", event_kinds: ["response"], transaction_ids: ["request-2"], offset: 0, limit: 20}` pages selected events with original item indices.
|
||||
|
||||
Omitted filters select all; empty arrays select none. Pagination counts matching
|
||||
retained rows. Exhausting a page does not establish capture completeness or
|
||||
current application state.
|
||||
|
||||
Transaction pages use `{kind: "browser-scenario", collection: "transactions", limit: 20}`.
|
||||
Their stable identity is the parent Evidence ID plus `transaction_id`; each row
|
||||
keeps original event indices. Metadata events are returned by default. Select
|
||||
`event_kinds: ["network-content"]` for recorded bodies and header values, or
|
||||
`event_kinds: []` for identity and indices alone. A `step_indices` filter selects
|
||||
a whole transaction when any retained event belongs to a selected step; this
|
||||
is receipt attribution, not proof that the action caused the request.
|
||||
|
||||
For a large retained tree, request `{kind: "browser-scenario", collection: "text",
|
||||
step_index: 10, artifact: "accessibility", offset: 0, limit: 2048}`. `dom` is also
|
||||
supported. Offsets and limits count UTF-16 code units. The full artifact digest
|
||||
and byte size remain attached to each page. Missing, truncated and unrequested
|
||||
text stay explicit; source truncation counts are preserved without treating an
|
||||
exhausted page as a complete capture.
|
||||
|
||||
Network summary counts include every retained transaction identity, including
|
||||
those whose request event was dropped. Missing request events, conflicting
|
||||
metadata and observed request failures have separate counts. `transactions` is
|
||||
`null` when network observation was unrequested or its legacy selection is
|
||||
unknown and there are no retained transactions; an observed empty inventory is
|
||||
`0`. Full capture omissions remain available and each page carries source
|
||||
completeness, dropped-event counts and omitted-transaction accounting.
|
||||
+5
-12
@@ -73,14 +73,9 @@ arbitrary nested graph array.
|
||||
A top-level array emits its elements in source order. An object must have one
|
||||
clear `rows`, `items`, `records`, or `occurrences` array, directly or under
|
||||
`page`. Evidence uses the same boundaries inside `normalized_result`.
|
||||
Any other selection is refused before a record is written, including an
|
||||
arbitrary report, a scalar, a nested array elsewhere, and a columnar object such
|
||||
as the `inspect-sqlite-database --table` `normalized_result.rows`
|
||||
(`{columns, values}`). The refusal is the caller error `invalid_request`
|
||||
(`invalid_input`) and names the selection and its observed shape; the analysis
|
||||
itself succeeded, so use `--format json` for the complete document. If several
|
||||
collections exist, select one with `--filter-output`, for example
|
||||
`normalized_result.rows` for an analysis view.
|
||||
An arbitrary report or a scalar selection is refused with an actionable error;
|
||||
use `--format json` for a complete document. If several collections exist,
|
||||
select one with `--filter-output`, for example `normalized_result.rows`.
|
||||
|
||||
Object collections finish with a control record:
|
||||
|
||||
@@ -929,10 +924,8 @@ redirected, which keeps large results smaller for agents and scripts; pipe
|
||||
through `jq .` for indented files. Output selection and formatting do not change
|
||||
operation status.
|
||||
|
||||
`--format md` escapes backslashes, pipes, and inline Markdown syntax in table
|
||||
headers and cells so source text is displayed literally, and renders cell line
|
||||
breaks as `<br>`. Underscores between letters or digits, as in `source_index`,
|
||||
cannot form emphasis and stay unescaped. These are presentation changes; retained
|
||||
`--format md` escapes backslashes and pipes in table headers and cells, and
|
||||
renders cell line breaks as `<br>`. These are presentation changes; retained
|
||||
Evidence and JSON output keep the original values. Nested objects and arrays
|
||||
use JSON inside their owning Markdown cells, retaining the field names and
|
||||
values instead of replacing them with an object label. Table widths are measured
|
||||
|
||||
@@ -522,9 +522,9 @@ with an installed native FreeBuff client/version; filesystem tests cover REA's
|
||||
editing, discovery, and ownership behavior only. Automatic applicability
|
||||
requires the FreeBuff profile directory even when the shared MCP file exists.
|
||||
|
||||
FreeBuff is main-only and is not in a published release yet; npm 6.7.0 is the
|
||||
latest. `rea setup --client freebuff` against `@latest` reports an unsupported
|
||||
client until a release containing it is published.
|
||||
FreeBuff setup support requires `rea-agents@6.8.0` or later. Older versions
|
||||
reject `rea setup --client freebuff` as an unsupported client. See the
|
||||
[installation guide](installation.md#supported-agents) for client availability.
|
||||
|
||||
## DeepSeek Harness
|
||||
|
||||
|
||||
@@ -15,7 +15,7 @@ snapshot parser and a separate optional annotation extension. Check the
|
||||
[release boundary](installation.md#released-package-and-main) when using npm.
|
||||
|
||||
See [import language and analysis seeds](installation.md#ghidra-import-language-and-analysis-seeds)
|
||||
to override the header-detected language or seed analysis. An override that conflicts with a fixed raw, DOS, PSP, MIPS, RISC-V or GameCube DOL import
|
||||
to override the header-detected language or seed analysis. An override that conflicts with a fixed raw, DOS, PSP, MIPS or RISC-V import
|
||||
is a configuration error naming the conflicting settings. It does not make an
|
||||
otherwise admitted target unsupported; REA preserves the requested settings.
|
||||
An inaccessible language
|
||||
|
||||
@@ -319,14 +319,14 @@ Register with the variable unset:
|
||||
<!-- x-release-please-start-version -->
|
||||
|
||||
```bash
|
||||
claude mcp add --scope local rea -- npx -y rea-agents@6.8.0 mcp
|
||||
claude mcp add --scope local rea -- npx -y rea-agents@6.7.0 mcp
|
||||
```
|
||||
|
||||
On native Windows, use the same project directory and wrap the package-runner
|
||||
shim so the stdio client can launch it:
|
||||
|
||||
```powershell
|
||||
claude mcp add --scope local rea -- cmd.exe /d /c npx -y rea-agents@6.8.0 mcp
|
||||
claude mcp add --scope local rea -- cmd.exe /d /c npx -y rea-agents@6.7.0 mcp
|
||||
```
|
||||
|
||||
<!-- x-release-please-end -->
|
||||
@@ -620,8 +620,8 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s
|
||||
|
||||
`droid`, `dsh`, `kimi`, `qoder`, `trae`, `workbuddy`, and `zcode` require
|
||||
`rea-agents@6.5.0` or later. The 6.4.0 registry check above does not cover them.
|
||||
`freebuff` requires `rea-agents@6.8.0` or later. Older releases reject
|
||||
`rea setup --client freebuff` as an unsupported client.
|
||||
`freebuff` is not in a release yet - `6.7.0` is the latest - so it also needs a
|
||||
release containing it.
|
||||
|
||||
See [client configuration and troubleshooting](client-configuration.md) for
|
||||
configuration paths, profiles, trust policies, and schema-limit workarounds.
|
||||
@@ -1471,7 +1471,7 @@ For a client that requires manual configuration, use:
|
||||
"mcpServers": {
|
||||
"rea": {
|
||||
"command": "npx",
|
||||
"args": ["-y", "rea-agents@6.8.0", "mcp"]
|
||||
"args": ["-y", "rea-agents@6.7.0", "mcp"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -426,17 +426,8 @@ and links whose callee was renamed by a minifier are not recognized.
|
||||
Bundler adapters own callback roles: Webpack supplies module, exports, then
|
||||
loader; esbuild CommonJS supplies exports, then module. Export extraction uses
|
||||
those binding identities for assignments and declarations, so renaming either
|
||||
parameter does not change the recovered names. The convention always supplies
|
||||
every role argument, so a role parameter's default never runs: a defaulted
|
||||
module or exports parameter keeps its role, and the default expression yields
|
||||
no exports, references or chunk keys. `{exports: alias}` in the module position
|
||||
binds `alias` to `module.exports`.
|
||||
Other role patterns, such as rest, array, nested or computed-key patterns or a
|
||||
non-identifier loader, receive no role and count as unknown findings, so the
|
||||
file's analysis is partial rather than an exact empty export list. An
|
||||
export-shaped write through a role binding whose value at that read cannot be
|
||||
ordered against a replacement is also an unknown finding. Global
|
||||
property-definition intrinsics require lexical and visible-write proof. Factory export declarations
|
||||
parameter does not change the recovered names. Global property-definition
|
||||
intrinsics require lexical and visible-write proof. Factory export declarations
|
||||
also require the loader's `d` member to have no prior or unordered visible
|
||||
replacement, including writes through lexical aliases. Unrelated loader members
|
||||
do not invalidate declarations. Ordering describes declarations within one factory
|
||||
|
||||
+8
-30
@@ -583,22 +583,11 @@ capabilities. The default result contains the complete inventory in groups that
|
||||
share status facts. `tool_availability.client_requirements` is the baseline for
|
||||
each group; a group's optional `client_requirements` replaces that baseline for
|
||||
the group's tools. Expand each group's `names` into one tool entry, carrying its
|
||||
availability, reason, remediation, default-mode status, modes, and client
|
||||
requirements. Compare the union of group names with `tools/list` when checking
|
||||
that a client received the full catalog. An operation absent from all groups is
|
||||
missing from the complete inventory; it is not an implicit unavailable entry.
|
||||
|
||||
`detail` selects the projection and the result echoes it. The default
|
||||
`summary` keeps open state, target identity, the binding, current-route
|
||||
`capabilities`, run, activity and health status, and each provider candidate's
|
||||
identity, selection, and availability and target-support `status`, `code`, and
|
||||
`reason`. It lists the fields it leaves out in `omitted_fields`: provider
|
||||
`diagnostics`, candidate `capabilities`, and each tool group's static `surface`,
|
||||
`effects`, and `annotations`. Omitted fields are not empty or false.
|
||||
`detail: "complete"` returns them inline, groups tools by those flags too, and
|
||||
has no `omitted_fields`. The output schema keys each result's candidate,
|
||||
tool-group and omission fields by its `detail`. CLI `capabilities` and
|
||||
`providers` always return complete provider facts.
|
||||
surface, availability, reason, remediation, default-mode status, modes, and
|
||||
client requirements. Compare the union of group names with `tools/list` when
|
||||
checking that a client received the full catalog. An operation absent from all
|
||||
groups is missing from the complete inventory; it is not an implicit
|
||||
unavailable entry.
|
||||
|
||||
`get_call_graph` and `trace_call_path` report `forward` and `backward` modes.
|
||||
Each requires `procedure_address` plus `procedure_callees` or `procedure_callers`,
|
||||
@@ -611,8 +600,8 @@ finish without reading relationships, even when neither traversal mode is
|
||||
available. Discovery does not block that direct invocation or replace its typed
|
||||
partial/error outcomes.
|
||||
|
||||
With complete detail, the `effects` and `annotations` arrays list the true
|
||||
members of their fixed vocabularies. A known member omitted from either array is false. This applies
|
||||
The `effects` and `annotations` arrays list the true members of their fixed
|
||||
vocabularies. A known member omitted from either array is false. This applies
|
||||
only to these closed flag sets; it does not make an omitted tool, provider, mode,
|
||||
or limitation false. Keep availability reasons, remediation, client-feature
|
||||
requirements, and mode facts inline with the grouped tool names. The optional
|
||||
@@ -643,8 +632,7 @@ identity, availability, reason, rejection code, effects, and ordered
|
||||
limitations.
|
||||
|
||||
`binary_session.analysis_provider_candidates` is authoritative for deep-engine
|
||||
discovery. Each candidate's capability sets, returned with complete detail,
|
||||
describe that possible provider;
|
||||
discovery. Each candidate's capability sets describe that possible provider;
|
||||
they do not select it. The top-level `capabilities` describe the current route,
|
||||
while `analysis_provider_binding` identifies the concrete selected route. These
|
||||
fields are not interchangeable. Target-free discovery is sorted by provider ID,
|
||||
@@ -1281,16 +1269,6 @@ This addresses the emitted shape reported through a Gemini-backed gateway in
|
||||
[#5201](https://github.com/morluto/rea/issues/5201). Gateway conversion and
|
||||
live provider acceptance have not been verified for this projection.
|
||||
|
||||
Gemini function declarations accept `items` only beside the single `array`
|
||||
type. Reduced array properties therefore advertise the element types their
|
||||
canonical item schemas accept, such as `{"type": "object"}` for a union of
|
||||
object actions, instead of an any-value type list with nested `items`. An
|
||||
element whose type is unknown, a tuple, or an array alongside other types uses
|
||||
the empty item schema, and a property that may be an array or another type
|
||||
lists typed `anyOf` branches. A boundary test rejects any compact
|
||||
advertisement that places `items` beside another type or uses an untyped item
|
||||
union.
|
||||
|
||||
The compact profile renders the same tool set, output schemas, annotations,
|
||||
and canonical server-side validation. It changes only the advertised input
|
||||
JSON Schema:
|
||||
|
||||
@@ -118,20 +118,8 @@ extract its matching Evidence record to a JSON file for the existing CLI
|
||||
`inspect-analysis-view` `evidence-file` source. Input bundles are unchanged.
|
||||
|
||||
Representation admission is bounded at 96 MiB before cloning selected observations
|
||||
and expanding matching indexes and repeated per-function source links. Each
|
||||
inventory row is charged before parsing for the member it becomes, including the
|
||||
report and Evidence copies that member is serialized into, so an oversized side
|
||||
is refused before its rows are parsed. Inline Evidence and bundle records are
|
||||
charged for the copies their authentication or loading creates. An exact
|
||||
session reference to retained Evidence is borrowed without a copy, so only
|
||||
what matching derives from it is charged; the inventory's own Evidence ID is
|
||||
charged once, not per function. Selected observations are charged in full.
|
||||
Measured with Ghidra-shaped rows, about 20-character names and no
|
||||
observations, about 22,000 rows per side fit through retained
|
||||
`list_procedures` references, and about 10,000 through inline Evidence or CLI
|
||||
`source_bundle` inputs. Work yields to
|
||||
cancellation. No worker/provider concurrency is introduced; this budget is not
|
||||
a bound on total process RSS.
|
||||
and expanding matching indexes and repeated per-function source links. Work yields to cancellation. No worker/provider
|
||||
concurrency is introduced; this budget is not a bound on total process RSS.
|
||||
Summary delivery happens after complete matching Evidence is constructed and
|
||||
retained. It reduces the initial reply, but does not avoid that construction or
|
||||
raise the allowance. There is no runtime override. Repeated provenance remains
|
||||
|
||||
@@ -14,10 +14,6 @@ device platform. A chained-import row ending in `+0x...` is ambiguous in
|
||||
dyld's text output: it may be an addend or part of a literal symbol name. REA
|
||||
keeps the full expression in `raw_name`, sets `name` and `addend` to null, and
|
||||
marks `name_status: "ambiguous"`; it does not synthesize a symbol identity.
|
||||
Each import and export `name` is the exact symbol-table spelling, such as
|
||||
`_main`; `runtime_name` is the derived name LLDB and `dlsym` use, such as `main`,
|
||||
and is null when `name` is null. See [native call observation](#native-call-observation) for
|
||||
breakpoint names.
|
||||
|
||||
## Start with a function
|
||||
|
||||
@@ -920,22 +916,15 @@ missing LLDB. `duration_ms` controls the later target observation window.
|
||||
rea observe-native-calls /Applications/Example.app '{"breakpoints":[{"kind":"objc-method","class_name":"NSURLSession","selector":"dataTaskWithRequest:completionHandler:"},{"kind":"function","name":"open","module":"libsystem_kernel.dylib"}],"duration_ms":5000,"backtrace_frames":4}' --json
|
||||
```
|
||||
|
||||
Function names select LLDB runtime symbols. On Darwin, the Mach-O symbol table
|
||||
spells language-level symbols with one leading underscore, which LLDB and `dlsym`
|
||||
omit: `_main` is `main`, C++ `__ZN3rea3addEii` is `_ZN3rea3addEii`, and Swift
|
||||
`_$s…` is `$s…`. Names without that underscore, such as `-[Class selector]`, are
|
||||
unchanged. `inspect_macho` keeps each export and import `name` exactly as the
|
||||
symbol table spells it and derives its `runtime_name`; pass `runtime_name` as the
|
||||
breakpoint `name`. `observe_native_calls` preserves the exact selected name; it
|
||||
does not remove underscores or translate static names, so `_main` selects a
|
||||
different runtime symbol. Use `{"kind":"function","name":"main","module":"Example"}`
|
||||
to limit that runtime symbol to the executable image named `Example`, then inspect
|
||||
the reported locations and image paths. Omitting `module` deliberately matches
|
||||
every loaded image, including system libraries. A match outside the executable is
|
||||
valid for that request and does not by itself make coverage partial. Event and
|
||||
location `symbol` values are LLDB display names, which are demangled for C++ and
|
||||
Swift; join them to `inspect_macho` through `breakpoint_index` and the request
|
||||
`name`, or through `file_address` and the export `address`.
|
||||
Function names select LLDB runtime symbols. On Darwin, a C function reported as
|
||||
`_main` by `inspect_macho` or `nm` can resolve as `main` in LLDB. REA preserves the
|
||||
exact selected name; it does not remove underscores or translate static names.
|
||||
Use `{"kind":"function","name":"main","module":"Example"}` to limit that
|
||||
runtime symbol to the executable image named `Example`, then inspect the reported
|
||||
locations and image paths. Omitting `module` deliberately matches every loaded
|
||||
image, including system libraries. A match outside the executable is valid for
|
||||
that request and does not by itself make coverage partial. This C naming example
|
||||
does not establish a translation rule for Swift, C++ or other mangled symbols.
|
||||
|
||||
Each event is an observation: thread, image, symbol, load and file address,
|
||||
the raw integer argument registers, optional caller frames and, for an
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -112,13 +112,6 @@ creation or retry mutations. An internal GraphQL error does not establish API
|
||||
quota exhaustion or prove that query size caused the failure. Publication
|
||||
continues through the pinned release-please action.
|
||||
|
||||
A nonzero manifest version must resolve to a GitHub release or tag before
|
||||
proposal history is collected. If publication has not created that boundary
|
||||
yet, preparation stops before opening or updating a PR; wait for publication
|
||||
before retrying. Missing manifest versions and `0.0.0` still allow an initial
|
||||
release. This prevents historical `Release-As` directives from being replayed
|
||||
as unreleased changes.
|
||||
|
||||
When upgrading release-please, review the upstream manifest implementation,
|
||||
refresh the versioned patch and run the release proposal and workflow boundary
|
||||
tests. Version mismatch or a patch that neither applies cleanly nor matches an
|
||||
|
||||
@@ -1,8 +0,0 @@
|
||||
# Implementation plan
|
||||
|
||||
1. **Read-backs.** A create's observation looks for the first read of its entity before the next write; an item read's
|
||||
fields and clock spellings are taken when it agrees otherwise, and the read observes; a list read adds fields only. The plan's observation records the
|
||||
read and its shape, and serving merges the same objects.
|
||||
2. **Sent members.** An update's sent member stored as a fresh clock after it sent back the held one is a `clock`; one
|
||||
stored as the slug of another member stored as sent is a `slug`.
|
||||
3. **Schema, tests and docs.**
|
||||
@@ -1,47 +0,0 @@
|
||||
# Writes whose answers show the entity otherwise than reads, and whole-object updates
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
Agents built environments from a prompt for five applications the prompt flow had never seen (Vikunja, Ghost, EspoCRM,
|
||||
Paperless-ngx and Miniflux). Two write-model gaps withheld most of their creates and edits:
|
||||
|
||||
- **A create's answer is another view of the entity than its reads.** Paperless answers a tag create without the
|
||||
`document_count` its item read shows. EspoCRM answers a contact create without nine fields its item read shows.
|
||||
Vikunja answers a label create with a partial `created_by` and clocks to the nanosecond, where reads show the full
|
||||
user and whole seconds. Ghost answers a tag create without the `count` its list shows. The state built the entity from
|
||||
the answer alone, so the read-back contradicted the create.
|
||||
- **A whole-object update sends back what the application derives.** Paperless's UI sends the tag's old `slug` with a
|
||||
new name, and the application slugs the name anew. Ghost's UI sends the `updated_at` it read, and the application
|
||||
stamps a new one. The update was withheld as `unexplained-change`, or its first call's value was pinned so that later
|
||||
calls conflicted.
|
||||
|
||||
## Behaviour
|
||||
|
||||
- **Read-backs.** When the first read of a created entity before the next write shows fields the create's answer lacks,
|
||||
or spells its clocks to another precision, the observed entity takes them, and the plan records the read
|
||||
(`observation.read`, with its shape).
|
||||
- An item read that agrees with the answer otherwise lends its fields and its clock spellings (whole seconds of a
|
||||
nanosecond clock, or microseconds of one), and no longer confirms the create. One that disagrees on another value,
|
||||
or omits an answer field its item view renders, still checks the create, which it contradicts.
|
||||
- A list read, which may show fewer keys of an object, only adds fields the answer lacks and still checks the create.
|
||||
- A read that shows nothing new leaves the observation as it was and still confirms the create.
|
||||
- Serving merges the same two recorded objects, so a served create answers in its answer's shape and stores the
|
||||
fields of both. Generated clocks in the create answer retain that answer's recorded precision independently of
|
||||
the stored read-back precision.
|
||||
- **Restamped clocks.** An update member sent back as the clock the entity held, and stored as a fresh clock near the
|
||||
call, is a `clock` rule, accepted with any value.
|
||||
- **Re-derived slugs.** An update member stored as the slug of another member the call stored as sent is a `slug` rule
|
||||
from that member, accepted with any value.
|
||||
- **Compatible saves.** A call that echoes an already-derived slug or holds a clock in its own recorded second does
|
||||
not conflict with a demonstrated derivation, in either call order. A slug derivation must reproduce earlier saves
|
||||
as well; an explicit custom slug that disagrees still withholds the update.
|
||||
|
||||
Plans and views that need none of this are unchanged.
|
||||
|
||||
## Acceptance
|
||||
|
||||
- The held-out recordings serve EspoCRM's contact create and Paperless's tag edit.
|
||||
- A create whose item read-back disagrees with its answer on another value is still contradicted.
|
||||
- No write tool of the 69 recorded captures goes from served to withheld.
|
||||
@@ -1,8 +0,0 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Take a create's read-back fields into its observation, and serve the same merge.
|
||||
- [x] Learn restamped clocks and re-derived slugs for sent update members.
|
||||
- [x] Add view and serving tests that fail without the change.
|
||||
- [x] Document the observation member and the rules.
|
||||
- [x] Measure the held-out and recorded captures.
|
||||
- [x] Run static, formatting, documentation and test gates.
|
||||
@@ -1,48 +0,0 @@
|
||||
# Verification
|
||||
|
||||
Executed on macOS with Node v24.22.1 at `2f807efb` (main `8e6a9578` and this branch).
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run check` (build, typecheck, format, lint, typed lint, module boundaries, architecture guards) and
|
||||
`npm run docs:check`: passed.
|
||||
- `npx vitest run --project domain --project services --project adapters`: 17,624 passed, 25 skipped, 1 failed ("stops
|
||||
Inspector target authorization after the current stat settles"), which fails the same way on main in this environment.
|
||||
- `npm run test:focused --` the analysis-view, environment build, compare, export and serve CLI boundaries, the
|
||||
analysis-view, environment bundle, serve and served-writes MCP boundaries, tool schema validity, output schema profile
|
||||
and MCP schema validation: 12 files, 106 passed.
|
||||
- The eight new view and serving tests fail on main and pass here.
|
||||
|
||||
## Recorded captures
|
||||
|
||||
The 69 recorded captures and the five held-out agent recordings, built as environment-state views on main and on this
|
||||
branch:
|
||||
|
||||
| Captures | Served on main | Served here | Served to withheld |
|
||||
| ----------------------------------------- | -------------- | ----------- | ------------------ |
|
||||
| 69 recorded | 132 | 132 | none |
|
||||
| 5 held-out (20 write tools with a target) | 8 | 10 | none |
|
||||
|
||||
- **Now served:** EspoCRM's contact create (its read-back adds nine fields the answer lacks) and Paperless's tag edit
|
||||
(the stale slug is slugged anew from the name).
|
||||
- **Past the read-back, withheld by other gaps:**
|
||||
- Paperless's tag create: its list's `all` member lists every id, which state rendering does not model.
|
||||
- Ghost's tag create and edit: the state does not list the created tags in Ghost's tag lists.
|
||||
- **Still contradicted:** Vikunja's label create, whose item read-back shows a fully loaded `created_by` where the answer
|
||||
showed a partial one.
|
||||
- Three plans of the recorded captures (Airbyte, BookStack, PhotoPrism) now record a read-back; their served state is
|
||||
unchanged.
|
||||
|
||||
## Served write tasks
|
||||
|
||||
The held-out round's write tasks, rebuilt from the same agent recordings with this branch, with
|
||||
`fireworks_ai/gpt-oss-120b` and `fireworks_ai/glm-5p3-flash`:
|
||||
|
||||
| Application | Main (gpt-oss, GLM) | This branch (gpt-oss, GLM) |
|
||||
| ---------------------------------------------- | ------------------- | -------------------------- |
|
||||
| EspoCRM contacts: create, update email, delete | 0/3, 0/3 | 1/3, 1/3 (create) |
|
||||
| Paperless tags: create, rename, delete | 1/3, 1/3 | 1/3, 2/3 (rename with GLM) |
|
||||
|
||||
- gpt-oss's rename did not apply. Its trajectory was not kept, so the cause is unconfirmed. A served `PUT` requires
|
||||
every member the recorded call sent (about 14 here), and GLM sent them all.
|
||||
- EspoCRM's served update accepts only the members its recording edited, so the email change is refused, as on main.
|
||||
+2
-6
@@ -2008,16 +2008,12 @@ coverage limits.
|
||||
|
||||
### Offline binary layout
|
||||
|
||||
`npm run verify:binary:layout` requires Linux x64, GCC/binutils, the
|
||||
`i686-linux-gnu`, `arm-linux-gnueabi`, `mips-linux-gnu` and `mipsel-linux-gnu`
|
||||
cross `gcc`/`readelf` toolchains, absolute
|
||||
`npm run verify:binary:layout` requires Linux x64, GCC/binutils, absolute
|
||||
`REA_PWNTOOLS_PYTHON` with pwntools 4.15.0/pyelftools 0.33/Unicorn 2.1.2 and
|
||||
absolute `REA_VERIFY_STRACE_COMMAND`. It compiles ephemeral source-owned ELF
|
||||
fixtures and checks public CLI/MCP, lossless addresses/names, file ranges,
|
||||
mitigation inferences, malformed/unsupported input, original file hashes and
|
||||
released process ownership. Cross-compiled ELF32 executables and relocatable
|
||||
objects check the reported class, machine and byte order, and every section
|
||||
name, address, offset and size against GNU `readelf`. Exec syscall tracing must identify only the declared
|
||||
released process ownership. Exec syscall tracing must identify only the declared
|
||||
Node/Python launchers; no target binary is executed. Core/debugger claims need
|
||||
separate verification lanes. Pass an installed package entrypoint as the script's
|
||||
first argument to verify packaging independently of the checkout.
|
||||
|
||||
@@ -746,16 +746,7 @@ integer of 10 or more digits that no entity of the type holds (where every one
|
||||
holds a distinct integer of the same length, as time-based ids do), a
|
||||
`nested-clock` object whose clocks the request stamped, a `template` around
|
||||
a generated value, a `reference` to an entity, or a `composed` string; other
|
||||
fields keep the observed value. When the first read of a created entity before
|
||||
the next write shows fields the create's answer lacks, or its clocks to another
|
||||
precision, the observed entity takes them (`observation.read`). An item read
|
||||
that agrees with the answer otherwise lends its fields and clock spellings and
|
||||
no longer confirms the create; one that disagrees or omits an answer field its
|
||||
item view renders still checks it. Clock agreement uses the coarser recorded
|
||||
precision, including microseconds. A list read
|
||||
only adds fields the answer lacks and still checks it. Serving answers a create
|
||||
in its answer's shape and clock precision and stores the fields of both with
|
||||
the item read-back's clock precision. A `reference` is an object holding fields of the
|
||||
fields keep the observed value. A `reference` is an object holding fields of the
|
||||
entity a sent member names, as the state holds it (a parent beside `parentId`):
|
||||
the member is accepted with any value, serving looks the entity up and refuses a
|
||||
call naming one the environment does not hold or holds in more than one
|
||||
@@ -787,14 +778,7 @@ with them. Other clocks the plan stamps take that spelling too unless their own
|
||||
value shows a fixed width. A body that wraps one
|
||||
object in a one-element array is that object, wrapped so on every call
|
||||
(`body.envelope`). An update's members
|
||||
echo when stored as sent. A clock sent back as the entity held it and stored as
|
||||
a fresh clock near the call is a `clock`, and a member stored as the slug of
|
||||
another member stored as sent is a `slug`, both accepted with any value, as a
|
||||
whole-object write that sends what it read shows. A save that echoes an
|
||||
already-derived slug or holds a clock in its own second remains compatible
|
||||
with that derivation in either call order; earlier saves must also reproduce
|
||||
a newly learned slug. Any other member stored
|
||||
differently is accepted only with its
|
||||
echo when stored as sent; a member stored differently is accepted only with its
|
||||
recorded value (`body.fixed`) and stores the value the call that showed it
|
||||
recorded (the `value` of its `constant` rule), and a field it changed without
|
||||
sending must be a clock, a nested clock, an increment, a slug, a fresh `unique`
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
# Browser scenario summary verification
|
||||
|
||||
The verifier runs the actual compiled CLI/MCP and caller-selected Chromium against
|
||||
TodoMVC's JavaScript ES5 distribution served on loopback. Ten actions add five
|
||||
todos. Every step captures URL, DOM, accessibility and a screenshot; the oracle
|
||||
checks all completed steps, the five authored labels, retained events and
|
||||
transaction identities. Summary runs also read step, text and transaction pages
|
||||
and authenticate an exported complete parent.
|
||||
|
||||
Download the public distribution at the pinned revision into a disposable folder:
|
||||
|
||||
```bash
|
||||
export REA_TODOMVC_APP="$(mktemp -d)"
|
||||
python3 - "$REA_TODOMVC_APP" <<'PY'
|
||||
import hashlib, json, sys, urllib.request
|
||||
from pathlib import Path
|
||||
root = Path(sys.argv[1])
|
||||
revision = "ff43b02e59dfa604386bb382034b2cd07c2bcd8a"
|
||||
files = ["index.html", "app.js", "base.css", "base.js", "controller.js",
|
||||
"helpers.js", "index.css", "model.js", "store.js", "template.js", "view.js"]
|
||||
manifest = []
|
||||
for name in files:
|
||||
url = f"https://raw.githubusercontent.com/tastejs/todomvc/{revision}/examples/javascript-es5/dist/{name}"
|
||||
with urllib.request.urlopen(url, timeout=30) as reply:
|
||||
data = reply.read()
|
||||
(root / name).write_bytes(data)
|
||||
manifest.append({"path": name, "url": url, "bytes": len(data),
|
||||
"sha256": hashlib.sha256(data).hexdigest()})
|
||||
(root / "provenance.json").write_text(json.dumps({"revision": revision, "files": manifest}, indent=2))
|
||||
PY
|
||||
```
|
||||
|
||||
Build each selected checkout with the repository's pinned Node/npm toolchain.
|
||||
Use new output directories; exports never overwrite an earlier run:
|
||||
|
||||
```bash
|
||||
node experiments/analysis-efficiency/browser-scenario-summary.mjs \
|
||||
--repository /absolute/path/to/baseline --mode complete \
|
||||
--application "$REA_TODOMVC_APP" --browser /absolute/path/to/chromium \
|
||||
--output /absolute/path/to/new-baseline-report
|
||||
node experiments/analysis-efficiency/browser-scenario-summary.mjs \
|
||||
--mode summary --application "$REA_TODOMVC_APP" \
|
||||
--browser /absolute/path/to/chromium \
|
||||
--output /absolute/path/to/new-summary-report
|
||||
```
|
||||
|
||||
`cold` is the first capture in a fresh MCP process; `warm` repeats the capture in
|
||||
that process with a new owned browser/profile. Neither label establishes cold OS
|
||||
file caches. Summary mode intentionally performs additional retained queries.
|
||||
Compare equivalent answers and include those calls when reporting task cost;
|
||||
smaller replies alone do not establish a faster task.
|
||||
|
||||
Add `--verify-boundaries` to a separate summary run for a real failed-step focus,
|
||||
CLI summary publication, and MCP text readback from the CLI's complete bundle.
|
||||
Those extra checks make its whole-run resource cost unsuitable for comparison
|
||||
against a complete-only run.
|
||||
|
||||
Reports retain the actual catalog, arguments, MCP text and complete Evidence.
|
||||
When estimating tokens, name the tokenizer and count discovery, arguments and
|
||||
all replies needed for the chosen task. Raw serialized catalog estimates are
|
||||
not actual model usage or a measurement of a client's rendered tool definitions.
|
||||
Measure CPU and peak process-family memory externally, state the accounting
|
||||
method, and include cleanup; the verifier's elapsed phase excludes final teardown.
|
||||
@@ -1,412 +0,0 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { createServer } from "node:http";
|
||||
import { readFile, writeFile, mkdir, readdir } from "node:fs/promises";
|
||||
import { resolve, join, extname } from "node:path";
|
||||
import { pathToFileURL, fileURLToPath } from "node:url";
|
||||
import { createRequire } from "node:module";
|
||||
import { performance } from "node:perf_hooks";
|
||||
import { execFile } from "node:child_process";
|
||||
import { promisify, parseArgs } from "node:util";
|
||||
// A developer verifier: the caller selects the pinned TodoMVC distribution,
|
||||
// browser executable, output directory and baseline/final repository checkout.
|
||||
const { values } = parseArgs({
|
||||
options: {
|
||||
repository: {
|
||||
type: "string",
|
||||
default: fileURLToPath(new URL("../../", import.meta.url)),
|
||||
},
|
||||
mode: { type: "string", default: "summary" },
|
||||
application: { type: "string" },
|
||||
browser: { type: "string" },
|
||||
output: { type: "string" },
|
||||
port: { type: "string", default: "0" },
|
||||
"verify-boundaries": { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
assert.ok(
|
||||
values.application && values.browser && values.output,
|
||||
"Pass --application <TodoMVC javascript-es5 dist> --browser <executable> --output <new directory>",
|
||||
);
|
||||
const repo = resolve(values.repository);
|
||||
const mode = values.mode;
|
||||
const output = resolve(values.output);
|
||||
const appRoot = resolve(values.application);
|
||||
const executablePath = resolve(values.browser);
|
||||
assert.ok(["complete", "summary"].includes(mode));
|
||||
assert.ok(
|
||||
!values["verify-boundaries"] || mode === "summary",
|
||||
"--verify-boundaries requires summary mode",
|
||||
);
|
||||
process.chdir(repo);
|
||||
const require = createRequire(join(repo, "package.json"));
|
||||
const { Client } = await import(
|
||||
pathToFileURL(require.resolve("@modelcontextprotocol/client")).href
|
||||
);
|
||||
const { StdioClientTransport } = await import(
|
||||
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio")).href
|
||||
);
|
||||
await mkdir(output);
|
||||
const files = await readdir(appRoot);
|
||||
const server = createServer(async (request, response) => {
|
||||
try {
|
||||
const name =
|
||||
decodeURIComponent(
|
||||
new URL(request.url, "http://127.0.0.1").pathname,
|
||||
).slice(1) || "index.html";
|
||||
assert.ok(files.includes(name));
|
||||
const data = await readFile(join(appRoot, name));
|
||||
response.setHeader(
|
||||
"Content-Type",
|
||||
{ ".html": "text/html", ".js": "text/javascript", ".css": "text/css" }[
|
||||
extname(name)
|
||||
] ?? "application/octet-stream",
|
||||
);
|
||||
response.end(data);
|
||||
} catch {
|
||||
response.writeHead(404);
|
||||
response.end();
|
||||
}
|
||||
});
|
||||
await new Promise((done, reject) => {
|
||||
server.once("error", reject);
|
||||
server.listen(Number(values.port), "127.0.0.1", done);
|
||||
});
|
||||
const address = server.address();
|
||||
assert.ok(address && typeof address === "object");
|
||||
const origin = `http://127.0.0.1:${address.port}`;
|
||||
const environment = {
|
||||
PATH: process.env.PATH,
|
||||
NODE_OPTIONS: "--max-old-space-size=3072",
|
||||
UV_THREADPOOL_SIZE: "2",
|
||||
REA_LOG_LEVEL: "silent",
|
||||
REA_MCP_INPUT_SCHEMA_PROFILE: "compact",
|
||||
};
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [join(repo, "scripts/rea.mjs"), "mcp"],
|
||||
cwd: repo,
|
||||
env: environment,
|
||||
stderr: "pipe",
|
||||
});
|
||||
let stderr = "";
|
||||
transport.stderr?.on("data", (chunk) => {
|
||||
if (stderr.length < 65536) stderr += chunk;
|
||||
});
|
||||
const client = new Client({
|
||||
name: "rea-browser-scenario-summary-verifier",
|
||||
version: "1",
|
||||
});
|
||||
const calls = [];
|
||||
const invoke = async (name, args, tools) => {
|
||||
const toolDefinition = tools.find((tool) => tool.name === name);
|
||||
assert.ok(
|
||||
toolDefinition?.outputSchema,
|
||||
`Missing named output schema: ${name}`,
|
||||
);
|
||||
const started = performance.now();
|
||||
const result = await client.callTool(
|
||||
{ name, arguments: args },
|
||||
{ toolDefinition, timeout: 90000 },
|
||||
);
|
||||
calls.push({
|
||||
name,
|
||||
arguments: args,
|
||||
content: result.content,
|
||||
elapsed_ms: performance.now() - started,
|
||||
});
|
||||
assert.notEqual(result.isError, true, JSON.stringify(result));
|
||||
return result.structuredContent;
|
||||
};
|
||||
const allStarted = performance.now();
|
||||
try {
|
||||
await client.connect(transport);
|
||||
const discoveryStarted = performance.now();
|
||||
const catalog = await client.listTools();
|
||||
const discoveryMs = performance.now() - discoveryStarted;
|
||||
await writeFile(join(output, "catalog.json"), JSON.stringify(catalog));
|
||||
const actions = Array.from({ length: 5 }, (_, index) => [
|
||||
{
|
||||
step_id: `fill_${index}`,
|
||||
action: "fill",
|
||||
locator: { kind: "css", selector: ".new-todo" },
|
||||
value: { source: "literal", value: `Review evidence ${index + 1}` },
|
||||
timeout_ms: 10000,
|
||||
},
|
||||
{
|
||||
step_id: `add_${index}`,
|
||||
action: "press",
|
||||
locator: { kind: "css", selector: ".new-todo" },
|
||||
key: "Enter",
|
||||
timeout_ms: 10000,
|
||||
},
|
||||
]).flat();
|
||||
const scenario = {
|
||||
browser: {
|
||||
mode: "launch",
|
||||
executable_path: executablePath,
|
||||
headless: true,
|
||||
},
|
||||
start_url: { url: origin, query: [] },
|
||||
actions,
|
||||
capture: {
|
||||
after_each_step: ["url", "dom", "accessibility", "screenshot"],
|
||||
at_end: ["url", "dom", "accessibility"],
|
||||
events: ["network"],
|
||||
},
|
||||
};
|
||||
const runs = [];
|
||||
for (const temperature of ["cold", "warm"]) {
|
||||
const callStart = calls.length;
|
||||
const result = await invoke(
|
||||
"capture_browser_scenario",
|
||||
{ ...scenario, ...(mode === "summary" ? { detail: "summary" } : {}) },
|
||||
catalog.tools,
|
||||
);
|
||||
const parentId =
|
||||
mode === "summary"
|
||||
? result.normalized_result.parent_evidence_id
|
||||
: result.evidence_id;
|
||||
const bundlePath = join(output, `${temperature}-evidence.json`);
|
||||
await invoke("export_evidence_bundle", { path: bundlePath }, catalog.tools);
|
||||
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
|
||||
const parent = bundle.records.find((row) => row.evidence_id === parentId);
|
||||
assert.ok(parent);
|
||||
const capture = parent.normalized_result;
|
||||
assert.equal(capture.steps.length, 11);
|
||||
assert.ok(capture.steps.every((step) => step.status === "completed"));
|
||||
assert.equal(capture.browser.cleanup, "terminated-owned-process");
|
||||
const final = capture.steps.at(-1).artifacts.accessibility;
|
||||
assert.equal(final.state, "captured");
|
||||
for (let index = 1; index <= 5; index++)
|
||||
assert.ok(final.value.text.includes(`Review evidence ${index}`));
|
||||
assert.ok(final.value.text.includes('strong: "5"'));
|
||||
assert.ok(final.value.text.includes("text: items left"));
|
||||
const transactionIds = new Set(
|
||||
capture.events.items
|
||||
.filter((event) => "transaction_id" in event)
|
||||
.map((event) => event.transaction_id),
|
||||
);
|
||||
const details = {
|
||||
temperature,
|
||||
parentId,
|
||||
steps: capture.steps.length,
|
||||
retained_events: capture.events.retained,
|
||||
transactions: transactionIds.size,
|
||||
};
|
||||
if (mode === "summary") {
|
||||
const summary = result.normalized_result.summary;
|
||||
assert.equal(summary.steps.length, 11);
|
||||
assert.equal(
|
||||
summary.final_accessibility.accessibility.value.text,
|
||||
final.value.text,
|
||||
);
|
||||
assert.equal(summary.events.retained, capture.events.retained);
|
||||
assert.equal(summary.limitation_count, parent.limitations.length);
|
||||
assert.equal(summary.network.transactions, transactionIds.size);
|
||||
const source = { kind: "retained-evidence", evidence_id: parentId };
|
||||
const pagingStarted = performance.now();
|
||||
for (const stepIndex of [1, 5, 10]) {
|
||||
const page = await invoke(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source,
|
||||
view: {
|
||||
kind: "browser-scenario",
|
||||
collection: "steps",
|
||||
step_indices: [stepIndex],
|
||||
artifacts: [],
|
||||
limit: 1,
|
||||
},
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.equal(
|
||||
page.normalized_result.items.rows[0].step_id,
|
||||
capture.steps[stepIndex].step_id,
|
||||
);
|
||||
}
|
||||
details.warm_page_total_ms = performance.now() - pagingStarted;
|
||||
const text = await invoke(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source,
|
||||
view: {
|
||||
kind: "browser-scenario",
|
||||
collection: "text",
|
||||
step_index: 10,
|
||||
artifact: "accessibility",
|
||||
offset: 0,
|
||||
limit: 32,
|
||||
},
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.equal(
|
||||
text.normalized_result.items.text,
|
||||
final.value.text.slice(0, 32),
|
||||
);
|
||||
const tx = await invoke(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source,
|
||||
view: {
|
||||
kind: "browser-scenario",
|
||||
collection: "transactions",
|
||||
limit: 2,
|
||||
},
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.deepEqual(
|
||||
tx.normalized_result.items.rows.map((row) => row.transaction_id),
|
||||
[...transactionIds].slice(0, 2),
|
||||
);
|
||||
details.returned_summary = summary;
|
||||
}
|
||||
details.calls = calls.slice(callStart);
|
||||
runs.push(details);
|
||||
}
|
||||
const boundaryChecks = [];
|
||||
if (values["verify-boundaries"]) {
|
||||
const failed = await invoke(
|
||||
"capture_browser_scenario",
|
||||
{
|
||||
...scenario,
|
||||
actions: [
|
||||
{
|
||||
step_id: "missing",
|
||||
action: "click",
|
||||
locator: {
|
||||
kind: "css",
|
||||
selector: "#rea-control-that-does-not-exist",
|
||||
},
|
||||
timeout_ms: 100,
|
||||
},
|
||||
],
|
||||
capture: {
|
||||
after_each_step: ["url", "accessibility"],
|
||||
at_end: ["url", "accessibility"],
|
||||
events: ["network"],
|
||||
},
|
||||
detail: "summary",
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.equal(failed.normalized_result.summary.steps[1].status, "failed");
|
||||
assert.equal(
|
||||
failed.normalized_result.summary.final_accessibility.focus_step_index,
|
||||
1,
|
||||
);
|
||||
assert.equal(
|
||||
failed.normalized_result.summary.completeness.equality_eligible,
|
||||
false,
|
||||
);
|
||||
boundaryChecks.push("real failed-step focus and incomplete coverage");
|
||||
const execute = promisify(execFile);
|
||||
const bundlePath = join(output, "cli-evidence.json");
|
||||
const { stdout } = await execute(
|
||||
process.execPath,
|
||||
[
|
||||
join(repo, "scripts/rea.mjs"),
|
||||
"capture-browser-scenario",
|
||||
JSON.stringify(scenario),
|
||||
"--detail",
|
||||
"summary",
|
||||
"--evidence_output",
|
||||
bundlePath,
|
||||
"--json",
|
||||
],
|
||||
{
|
||||
cwd: repo,
|
||||
env: environment,
|
||||
maxBuffer: 64 * 1024 * 1024,
|
||||
timeout: 90000,
|
||||
},
|
||||
);
|
||||
const cli = JSON.parse(stdout);
|
||||
const parentId = cli.summary.normalized_result.parent_evidence_id;
|
||||
const bundle = JSON.parse(await readFile(bundlePath, "utf8"));
|
||||
const parent = bundle.records.find((row) => row.evidence_id === parentId);
|
||||
assert.ok(parent);
|
||||
assert.equal(parent.normalized_result.steps.length, 11);
|
||||
assert.ok(
|
||||
parent.normalized_result.steps.every(
|
||||
(step) => step.status === "completed",
|
||||
),
|
||||
);
|
||||
assert.ok(!("detail" in parent.parameters));
|
||||
assert.equal(
|
||||
cli.summary.normalized_result.summary.final_accessibility.accessibility
|
||||
.value.text,
|
||||
parent.normalized_result.steps.at(-1).artifacts.accessibility.value.text,
|
||||
);
|
||||
const text = await invoke(
|
||||
"inspect_analysis_view",
|
||||
{
|
||||
source: {
|
||||
kind: "evidence-file",
|
||||
path: bundlePath,
|
||||
evidence_id: parentId,
|
||||
},
|
||||
view: {
|
||||
kind: "browser-scenario",
|
||||
collection: "text",
|
||||
step_index: 10,
|
||||
artifact: "accessibility",
|
||||
offset: 0,
|
||||
limit: 32,
|
||||
},
|
||||
},
|
||||
catalog.tools,
|
||||
);
|
||||
assert.equal(
|
||||
text.normalized_result.items.text,
|
||||
parent.normalized_result.steps
|
||||
.at(-1)
|
||||
.artifacts.accessibility.value.text.slice(0, 32),
|
||||
);
|
||||
boundaryChecks.push(
|
||||
"real CLI summary, complete bundle and MCP evidence-file readback",
|
||||
);
|
||||
}
|
||||
await writeFile(
|
||||
join(output, "result.json"),
|
||||
JSON.stringify(
|
||||
{
|
||||
repo,
|
||||
mode,
|
||||
application: appRoot,
|
||||
browser: executablePath,
|
||||
input_paths: {
|
||||
repository: values.repository,
|
||||
application: values.application,
|
||||
browser: values.browser,
|
||||
output: values.output,
|
||||
},
|
||||
origin,
|
||||
discovery_ms: discoveryMs,
|
||||
elapsed_ms: performance.now() - allStarted,
|
||||
runs,
|
||||
boundaryChecks,
|
||||
stderr,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
mode,
|
||||
discovery_ms: discoveryMs,
|
||||
runs: runs.map(
|
||||
({ calls: _calls, returned_summary: _summary, ...rest }) => rest,
|
||||
),
|
||||
boundaryChecks,
|
||||
elapsed_ms: performance.now() - allStarted,
|
||||
}),
|
||||
);
|
||||
} finally {
|
||||
await client.close();
|
||||
server.closeAllConnections();
|
||||
await new Promise((done) => server.close(done));
|
||||
}
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"version": "6.8.0",
|
||||
"version": "6.7.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "rea-agents",
|
||||
"version": "6.8.0",
|
||||
"version": "6.7.0",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@babel/parser": "8.0.4",
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "rea-agents",
|
||||
"mcpName": "io.github.morluto/rea",
|
||||
"version": "6.8.0",
|
||||
"version": "6.7.0",
|
||||
"description": "Reverse engineer anything from your terminal or agent with one CLI and MCP server.",
|
||||
"license": "MIT",
|
||||
"repository": {
|
||||
|
||||
@@ -105,22 +105,7 @@
|
||||
}
|
||||
--- a/build/src/manifest.js
|
||||
+++ b/build/src/manifest.js
|
||||
@@ -248,6 +248,14 @@
|
||||
releasesFound++;
|
||||
}
|
||||
}
|
||||
+ // A recorded version requires its release boundary, even while a
|
||||
+ // concurrent publisher has not created the matching release/tag yet.
|
||||
+ for (const path of Object.keys(strategiesByPath)) {
|
||||
+ const expectedVersion = this.releasedVersions[path];
|
||||
+ if (expectedVersion && expectedVersion.toString() !== '0.0.0' && !releasesByPath[path]) {
|
||||
+ throw new Error(`Missing expected release for path ${path} at version ${expectedVersion.toString()}: no matching release or tag found. Refusing to bootstrap an already released package.`);
|
||||
+ }
|
||||
+ }
|
||||
const needsBootstrap = releasesFound < expectedReleases;
|
||||
if (releasesFound < expectedReleases) {
|
||||
this.logger.warn(`Expected ${expectedReleases} releases, only found ${releasesFound}`);
|
||||
@@ -261,9 +269,20 @@
|
||||
@@ -261,9 +261,20 @@
|
||||
this.logger.info('Collecting commits since all latest releases');
|
||||
const commits = [];
|
||||
this.logger.debug(`commit search depth: ${this.commitSearchDepth}`);
|
||||
@@ -142,7 +127,7 @@
|
||||
batchSize: this.commitBatchSize,
|
||||
});
|
||||
const releaseShas = new Set(Object.values(releaseShasByPath));
|
||||
@@ -272,6 +291,7 @@
|
||||
@@ -272,6 +283,7 @@
|
||||
// sha => release pull request
|
||||
const releasePullRequestsBySha = {};
|
||||
let releaseCommitsFound = 0;
|
||||
@@ -150,7 +135,7 @@
|
||||
for await (const commit of commitGenerator) {
|
||||
if (releaseShas.has(commit.sha)) {
|
||||
if (commit.pullRequest) {
|
||||
@@ -284,14 +304,17 @@
|
||||
@@ -284,14 +296,17 @@
|
||||
}
|
||||
if (this.lastReleaseSha && this.lastReleaseSha === commit.sha) {
|
||||
this.logger.info(`Using configured lastReleaseSha ${this.lastReleaseSha} as last commit.`);
|
||||
@@ -168,7 +153,7 @@
|
||||
break;
|
||||
}
|
||||
commits.push({
|
||||
@@ -301,16 +324,18 @@
|
||||
@@ -301,16 +316,18 @@
|
||||
pullRequest: commit.pullRequest,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -45,19 +45,13 @@ const operationResultKey = (contract) => {
|
||||
return evidence ? "normalized_result" : "result";
|
||||
};
|
||||
|
||||
/**
|
||||
* Expand compact session metadata once at the verifier boundary. The caller
|
||||
* has validated the status against the binary_session contract, so a summary
|
||||
* carries summary candidates without capability sets; complete detail must
|
||||
* still expand every candidate's capabilities.
|
||||
*/
|
||||
/** Expand compact session metadata once at the verifier boundary. */
|
||||
export const expandMcpSessionStatus = (status) => ({
|
||||
...status,
|
||||
capabilities: expandProviderCapabilities(status.capabilities),
|
||||
analysis_provider_candidates:
|
||||
status.detail === "summary"
|
||||
? status.analysis_provider_candidates
|
||||
: expandProviderCandidates(status.analysis_provider_candidates),
|
||||
analysis_provider_candidates: expandProviderCandidates(
|
||||
status.analysis_provider_candidates,
|
||||
),
|
||||
tool_availability: expandToolAvailability(status.tool_availability),
|
||||
});
|
||||
|
||||
|
||||
@@ -104,7 +104,7 @@ export async function readReleaseHistory(run, base, target) {
|
||||
|
||||
// GitHub maps these coding-agent trailers to ordinary User actors.
|
||||
// Keep those source identities in the inventory while excluding automation credit.
|
||||
const AGENT_ACCOUNTS = new Set(["claude", "cursoragent", "codex", "copilot"]);
|
||||
const AGENT_ACCOUNTS = new Set(["claude", "cursoragent", "codex"]);
|
||||
// REA's maintainer requested external-contributor thanks without self-credit.
|
||||
const MAINTAINER_ACCOUNTS = new Set(["morluto"]);
|
||||
const human = (value) =>
|
||||
|
||||
@@ -35,27 +35,11 @@ if (
|
||||
!isAbsolute(strace ?? "")
|
||||
)
|
||||
throw new Error(
|
||||
"verify:binary:layout requires Linux x64, absolute REA_PWNTOOLS_PYTHON (pwntools 4.15.0/pyelftools 0.33/Unicorn 2.1.2) and REA_VERIFY_STRACE_COMMAND. gcc, ld, strip and the i686/ARM/MIPS/MIPSel linux-gnu cross gcc and readelf must be available. No target is executed.",
|
||||
"verify:binary:layout requires Linux x64, absolute REA_PWNTOOLS_PYTHON (pwntools 4.15.0/pyelftools 0.33/Unicorn 2.1.2) and REA_VERIFY_STRACE_COMMAND. gcc, ld and strip must be available. No target is executed.",
|
||||
);
|
||||
const run = createVerifierRun();
|
||||
const execute = promisify(execFile);
|
||||
const ELF32_TOOLCHAINS = [
|
||||
["i686-linux-gnu", "EM_386", "little"],
|
||||
["arm-linux-gnueabi", "EM_ARM", "little"],
|
||||
["mips-linux-gnu", "EM_MIPS", "big"],
|
||||
["mipsel-linux-gnu", "EM_MIPS", "little"],
|
||||
];
|
||||
for (const command of [
|
||||
"gcc",
|
||||
"ld",
|
||||
"strip",
|
||||
"/usr/bin/prlimit",
|
||||
strace,
|
||||
...ELF32_TOOLCHAINS.flatMap(([prefix]) => [
|
||||
`${prefix}-gcc`,
|
||||
`${prefix}-readelf`,
|
||||
]),
|
||||
]) {
|
||||
for (const command of ["gcc", "ld", "strip", "/usr/bin/prlimit", strace]) {
|
||||
try {
|
||||
await execute(command, [command === strace ? "-V" : "--version"], {
|
||||
timeout: 10_000,
|
||||
@@ -344,163 +328,6 @@ try {
|
||||
cases++;
|
||||
}
|
||||
}
|
||||
// ELF32 profiles: cross-compiled, never executed, checked against GNU readelf.
|
||||
for (const [
|
||||
prefix,
|
||||
machine,
|
||||
byteOrder,
|
||||
kind,
|
||||
flags,
|
||||
] of ELF32_TOOLCHAINS.flatMap((toolchain) => [
|
||||
[...toolchain, "executable", ["-fPIE", "-pie"]],
|
||||
[...toolchain, "relocatable", ["-c", "-fPIC"]],
|
||||
])) {
|
||||
const path = join(root.path, `${prefix}-${kind}`);
|
||||
await execute(`${prefix}-gcc`, ["-O1", ...flags, source, "-o", path], {
|
||||
timeout: 30_000,
|
||||
maxBuffer: 1024 * 1024,
|
||||
});
|
||||
const { stdout } = await execute(`${prefix}-readelf`, ["-SWsr", path], {
|
||||
timeout: 10_000,
|
||||
maxBuffer: 16 * 1024 * 1024,
|
||||
env: { ...process.env, LC_ALL: "C" },
|
||||
});
|
||||
const expectedSections = [
|
||||
...stdout.matchAll(
|
||||
/^\s*\[\s*(\d+)\]\s+(\S+)\s+\S+\s+([0-9a-f]+)\s+([0-9a-f]+)\s+([0-9a-f]+)\s+[0-9a-f]+\s+[A-Za-z]*\s+(\d+)\s+\d+\s+\d+\s*$/gm,
|
||||
),
|
||||
]
|
||||
.filter(([, index]) => index !== "0")
|
||||
.map(([, index, name, address, offset, size, link]) => ({
|
||||
index: Number(index),
|
||||
name,
|
||||
address: BigInt(`0x${address}`),
|
||||
offset: BigInt(`0x${offset}`),
|
||||
size: BigInt(`0x${size}`),
|
||||
link: Number(link),
|
||||
}));
|
||||
assert.ok(
|
||||
expectedSections.length > 0,
|
||||
`readelf listed no sections for ${path}`,
|
||||
);
|
||||
const expectedSymbols = [];
|
||||
for (const [, name, count, body] of stdout.matchAll(
|
||||
/^Symbol table '([^']+)' contains (\d+) entr(?:y|ies):\n((?:[^\n]+\n?)*)/gm,
|
||||
)) {
|
||||
const table = expectedSections.find((section) => section.name === name);
|
||||
assert.notEqual(table, undefined, `readelf symbol table absent: ${name}`);
|
||||
const rows = [...body.matchAll(/^\s*(\d+):\s+([0-9a-f]+)\s+(\d+)\s/gm)];
|
||||
assert.equal(rows.length, Number(count), `readelf symbol count: ${name}`);
|
||||
for (const [, index, value, size] of rows)
|
||||
expectedSymbols.push({
|
||||
table_index: table.index,
|
||||
entry_index: Number(index),
|
||||
value: BigInt(`0x${value}`),
|
||||
size: BigInt(size),
|
||||
});
|
||||
}
|
||||
assert.ok(
|
||||
expectedSymbols.length > 0,
|
||||
`readelf listed no symbols for ${path}`,
|
||||
);
|
||||
const expectedRelocations = [];
|
||||
for (const [, name, count, body] of stdout.matchAll(
|
||||
/^Relocation section '([^']+)' at offset 0x[0-9a-f]+ contains (\d+) entr(?:y|ies):\n((?:[^\n]+\n?)*)/gm,
|
||||
)) {
|
||||
const section = expectedSections.find((section) => section.name === name);
|
||||
assert.notEqual(
|
||||
section,
|
||||
undefined,
|
||||
`readelf relocation section absent: ${name}`,
|
||||
);
|
||||
const rows = [...body.matchAll(/^\s*([0-9a-f]+)\s+([0-9a-f]+)\s+\S+/gm)];
|
||||
assert.equal(
|
||||
rows.length,
|
||||
Number(count),
|
||||
`readelf relocation count: ${name}`,
|
||||
);
|
||||
for (const [entryIndex, [, offset, info]] of rows.entries()) {
|
||||
// ELF32 r_info uses the low byte for type and the upper 24 bits for
|
||||
// the symbol index. readelf has already decoded the file's byte order.
|
||||
const decodedInfo = BigInt(`0x${info}`);
|
||||
expectedRelocations.push({
|
||||
section_index: section.index,
|
||||
entry_index: entryIndex,
|
||||
reported_offset: BigInt(`0x${offset}`),
|
||||
type: Number(decodedInfo & 0xffn),
|
||||
symbol_table_index: section.link,
|
||||
symbol_index: Number(decodedInfo >> 8n),
|
||||
});
|
||||
}
|
||||
}
|
||||
assert.ok(
|
||||
expectedRelocations.length > 0,
|
||||
`readelf listed no relocations for ${path}`,
|
||||
);
|
||||
const bytes = await readFile(path);
|
||||
for (const mode of ["cli", "mcp"]) {
|
||||
const value = await inspect(mode, path);
|
||||
assert.deepEqual(value.architecture, {
|
||||
machine,
|
||||
bits: 32,
|
||||
byte_order: byteOrder,
|
||||
});
|
||||
assert.equal(
|
||||
value.image_type,
|
||||
kind === "executable" ? "ET_DYN" : "ET_REL",
|
||||
);
|
||||
for (const section of expectedSections) {
|
||||
const reported = value.sections[section.index];
|
||||
assert.equal(reported.name.display, section.name);
|
||||
assert.equal(BigInt(reported.address), section.address);
|
||||
assert.equal(BigInt(reported.offset), section.offset);
|
||||
assert.equal(BigInt(reported.size), section.size);
|
||||
}
|
||||
assert.equal(value.sections.length, expectedSections.length + 1);
|
||||
assert.deepEqual(
|
||||
value.symbols.map(({ table_index, entry_index, value, size }) => ({
|
||||
table_index,
|
||||
entry_index,
|
||||
value: BigInt(value),
|
||||
size: BigInt(size),
|
||||
})),
|
||||
expectedSymbols,
|
||||
`${mode} symbol values differ from readelf for ${path}`,
|
||||
);
|
||||
assert.deepEqual(
|
||||
value.relocations.map(
|
||||
({
|
||||
section_index,
|
||||
entry_index,
|
||||
reported_offset,
|
||||
type,
|
||||
symbol_table_index,
|
||||
symbol_index,
|
||||
}) => ({
|
||||
section_index,
|
||||
entry_index,
|
||||
reported_offset: BigInt(reported_offset),
|
||||
type,
|
||||
symbol_table_index,
|
||||
symbol_index,
|
||||
}),
|
||||
),
|
||||
expectedRelocations,
|
||||
`${mode} relocations differ from readelf for ${path}`,
|
||||
);
|
||||
if (kind === "executable") {
|
||||
assert.ok(value.segments.length > 0);
|
||||
assert.equal(value.linkage.interpreters.length, 1);
|
||||
assert.ok(
|
||||
value.linkage.needed_libraries.some(
|
||||
(library) => library.display === "libc.so.6",
|
||||
),
|
||||
);
|
||||
}
|
||||
assert.deepEqual(await readFile(path), bytes);
|
||||
cases++;
|
||||
}
|
||||
}
|
||||
const protectedReport = reports.get("protected");
|
||||
// Ordinary data markers are not a request to discover/decompress a kernel config.
|
||||
const markerBytes = Buffer.from(await readFile(join(root.path, "protected")));
|
||||
|
||||
+2
-2
@@ -8,13 +8,13 @@
|
||||
"url": "https://github.com/morluto/rea",
|
||||
"source": "github"
|
||||
},
|
||||
"version": "6.8.0",
|
||||
"version": "6.7.0",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "npm",
|
||||
"registryBaseUrl": "https://registry.npmjs.org",
|
||||
"identifier": "rea-agents",
|
||||
"version": "6.8.0",
|
||||
"version": "6.7.0",
|
||||
"runtimeHint": "npx",
|
||||
"transport": {
|
||||
"type": "stdio"
|
||||
|
||||
@@ -66,7 +66,6 @@ import {
|
||||
hasBinaryPlistSignature,
|
||||
MAX_XML_PLIST_PROBE_BYTES,
|
||||
} from "../domain/apple/appleArchiveFormat.js";
|
||||
import { hasDiskImageSignature } from "../domain/apple/diskImageFormat.js";
|
||||
import { hasGodotPckHeader } from "../domain/godotPckFormat.js";
|
||||
import {
|
||||
hasZipSignature,
|
||||
@@ -639,10 +638,11 @@ const detectArtifactFormat = async (
|
||||
const { handle, initial } = source;
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const lower = path.toLowerCase();
|
||||
// Fill the prefix so a short host read cannot hide a signature.
|
||||
const magic = Buffer.from(await readHandleRange(handle, 0, 4096, signal));
|
||||
const magic = Buffer.alloc(4096);
|
||||
const observed = await handle.read(magic, 0, magic.length, 0);
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
if (
|
||||
magic.length >= 8 &&
|
||||
observed.bytesRead >= 8 &&
|
||||
magic.subarray(0, 8).equals(Buffer.from([0, 0x61, 0x73, 0x6d, 1, 0, 0, 0]))
|
||||
)
|
||||
throw new BinaryTargetError(
|
||||
@@ -650,32 +650,39 @@ const detectArtifactFormat = async (
|
||||
"WebAssembly v1 artifacts are not supported by the native opening route",
|
||||
{ constraint: "wasm_requires_static_inspection" },
|
||||
);
|
||||
if (hasZipSignature(magic)) return zipRootPackageFormat(lower);
|
||||
if (hasGodotPckHeader(magic)) return "godot-pck";
|
||||
if (hasZipSignature(magic.subarray(0, observed.bytesRead)))
|
||||
return zipRootPackageFormat(lower);
|
||||
if (hasGodotPckHeader(magic.subarray(0, observed.bytesRead)))
|
||||
return "godot-pck";
|
||||
// Binary plists, including keyed archives, often use other extensions.
|
||||
if (hasBinaryPlistSignature(magic)) return "plist";
|
||||
if (hasBinaryPlistSignature(magic.subarray(0, observed.bytesRead)))
|
||||
return "plist";
|
||||
const named = namedArtifactFormat(lower);
|
||||
if (named !== undefined) return named;
|
||||
if (
|
||||
lower.endsWith(".pkg") &&
|
||||
magic.length >= 4 &&
|
||||
observed.bytesRead >= 4 &&
|
||||
magic.subarray(0, 4).equals(Buffer.from("xar!"))
|
||||
)
|
||||
return "pkg";
|
||||
if (
|
||||
lower.endsWith(".dmg") &&
|
||||
(await hasDiskImageSignature(initial.size, (position, length) =>
|
||||
readHandleRange(handle, position, length, signal),
|
||||
))
|
||||
)
|
||||
return "dmg";
|
||||
if (lower.endsWith(".dmg")) {
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const size = initial.size;
|
||||
if (size >= 512) {
|
||||
const trailer = Buffer.alloc(4);
|
||||
const read = await handle.read(trailer, 0, trailer.length, size - 512);
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
if (read.bytesRead === 4 && trailer.equals(Buffer.from("koly")))
|
||||
return "dmg";
|
||||
}
|
||||
}
|
||||
// XML property lists can use archive-specific suffixes such as .archive.
|
||||
// The named-format check above keeps .js, .map, and .asar classifications.
|
||||
const xmlProbe = createXmlPlistSignatureProbe();
|
||||
let xmlState = xmlProbe.write(magic);
|
||||
let xmlState = xmlProbe.write(magic.subarray(0, observed.bytesRead));
|
||||
if (xmlState === "matched") return "plist";
|
||||
const chunk = Buffer.alloc(XML_PLIST_PROBE_CHUNK_BYTES);
|
||||
let bytesRead = magic.length;
|
||||
let bytesRead = observed.bytesRead;
|
||||
while (xmlState === "pending" && bytesRead < MAX_XML_PLIST_PROBE_BYTES) {
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const read = await handle.read(
|
||||
@@ -740,7 +747,7 @@ const claimsLeadingExecutableHeader = async (
|
||||
size,
|
||||
);
|
||||
|
||||
/** Fill one bounded range, honoring cancellation; a short result means end of file. */
|
||||
/** Read one bounded range, honoring cancellation around the host read. */
|
||||
const readHandleRange = async (
|
||||
handle: FileHandle,
|
||||
position: number,
|
||||
@@ -749,20 +756,9 @@ const readHandleRange = async (
|
||||
): Promise<Uint8Array> => {
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
const bytes = Buffer.alloc(length);
|
||||
let filled = 0;
|
||||
while (filled < length) {
|
||||
if (filled > 0) throwIfTargetResolutionCancelled(signal);
|
||||
const observed = await handle.read(
|
||||
bytes,
|
||||
filled,
|
||||
length - filled,
|
||||
position + filled,
|
||||
);
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
if (observed.bytesRead === 0) break;
|
||||
filled += observed.bytesRead;
|
||||
}
|
||||
return bytes.subarray(0, filled);
|
||||
const observed = await handle.read(bytes, 0, length, position);
|
||||
throwIfTargetResolutionCancelled(signal);
|
||||
return bytes.subarray(0, observed.bytesRead);
|
||||
};
|
||||
|
||||
/** Read exactly `size` bytes by offset; a short read means the file changed or ended early. */
|
||||
|
||||
@@ -9,7 +9,6 @@ import { TOOL_CONTRACTS } from "../contracts/toolContracts.js";
|
||||
import {
|
||||
expandToolAvailability,
|
||||
toolAvailabilityInventory,
|
||||
toolAvailabilitySummaryInventory,
|
||||
type ToolAvailability,
|
||||
} from "../contracts/toolOutputSchemaPrimitives.js";
|
||||
|
||||
@@ -886,44 +885,3 @@ it("keeps independent native sessions discoverable without an unrelated binary t
|
||||
{ available: false, reason: "target_required" },
|
||||
);
|
||||
});
|
||||
|
||||
describe("summary tool availability", () => {
|
||||
it("summarizes every tool's current availability without static catalog flags", () => {
|
||||
const full = buildCapabilityInventory(status(), enabledPolicy);
|
||||
const withOverride = full.map((entry, index) =>
|
||||
index === 1
|
||||
? {
|
||||
...entry,
|
||||
available: false as const,
|
||||
reason: "client_capability_missing" as const,
|
||||
remediation: "Use an MCP client with roots support.",
|
||||
client_requirements: {
|
||||
required: ["roots" as const],
|
||||
optional: [],
|
||||
missing_required: ["roots" as const],
|
||||
missing_optional: [],
|
||||
},
|
||||
}
|
||||
: entry,
|
||||
);
|
||||
const complete = compactToolAvailability(withOverride);
|
||||
const summary = compactToolAvailability(withOverride, "summary");
|
||||
|
||||
expect(toolAvailabilitySummaryInventory.parse(summary)).toEqual(summary);
|
||||
expect(toolAvailabilityInventory.safeParse(summary).success).toBe(false);
|
||||
expect(summary.groups.length).toBeLessThan(complete.groups.length);
|
||||
expect(JSON.stringify(summary).length).toBeLessThan(
|
||||
JSON.stringify(complete).length / 2,
|
||||
);
|
||||
for (const group of summary.groups)
|
||||
for (const field of ["surface", "effects", "annotations"])
|
||||
expect(Object.hasOwn(group, field)).toBe(false);
|
||||
const byName = (left: { name: string }, right: { name: string }) =>
|
||||
left.name.localeCompare(right.name);
|
||||
expect(expandToolAvailability(summary).toSorted(byName)).toEqual(
|
||||
withOverride
|
||||
.map(({ surface: _s, effects: _e, annotations: _a, ...entry }) => entry)
|
||||
.toSorted(byName),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -6,7 +6,6 @@ import { NATIVE_DISPATCH_METADATA_SOURCES } from "./native/NativeDispatchMetadat
|
||||
import { NATIVE_UI_ACTION_OPERATIONS } from "./native/NativeUiActionTrace.js";
|
||||
import { NATIVE_CALL_ROUTE_SOURCES } from "./native/NativeCallRoutes.js";
|
||||
import { z } from "zod";
|
||||
import type { BinarySessionDetail } from "../contracts/sessionStatusContract.js";
|
||||
import { sqliteDatabaseCapability } from "./sqlite/SqliteDatabaseCapability.js";
|
||||
import type {
|
||||
OptionalProviderLoadFailures,
|
||||
@@ -17,13 +16,11 @@ import {
|
||||
expandProviderCapabilities,
|
||||
providerCapabilitySet,
|
||||
toolAvailabilityInventory,
|
||||
toolAvailabilitySummaryInventory,
|
||||
TOOL_ANNOTATION_NAMES,
|
||||
TOOL_EFFECT_NAMES,
|
||||
type ClientFeatureAvailability,
|
||||
type ProviderCapability,
|
||||
type ToolAvailabilityInventory,
|
||||
type ToolAvailabilitySummaryInventory,
|
||||
type ToolAvailability,
|
||||
type ToolAvailabilityMode,
|
||||
type ToolUnavailabilityReason,
|
||||
@@ -326,6 +323,10 @@ export const buildCapabilityInventory = (
|
||||
};
|
||||
|
||||
type ClientRequirements = ToolAvailability["client_requirements"];
|
||||
type MutableAvailabilityGroup = Omit<
|
||||
ToolAvailabilityInventory["groups"][number],
|
||||
"names"
|
||||
> & { names: string[] };
|
||||
|
||||
const EMPTY_CLIENT_REQUIREMENTS: ClientRequirements = {
|
||||
required: [],
|
||||
@@ -334,29 +335,23 @@ const EMPTY_CLIENT_REQUIREMENTS: ClientRequirements = {
|
||||
missing_optional: [],
|
||||
};
|
||||
|
||||
/**
|
||||
* Group every operation's current availability inline. Complete detail also
|
||||
* groups by the static surface, effect, and annotation flags; summary omits
|
||||
* those flags so tools that share a status collapse into one group.
|
||||
*/
|
||||
export function compactToolAvailability(
|
||||
/** Compact repeated metadata while retaining every operation's current availability inline. */
|
||||
export const compactToolAvailability = (
|
||||
availability: readonly ToolAvailability[],
|
||||
detail?: "complete",
|
||||
): ToolAvailabilityInventory;
|
||||
export function compactToolAvailability(
|
||||
availability: readonly ToolAvailability[],
|
||||
detail: BinarySessionDetail,
|
||||
): ToolAvailabilityInventory | ToolAvailabilitySummaryInventory;
|
||||
export function compactToolAvailability(
|
||||
availability: readonly ToolAvailability[],
|
||||
detail: BinarySessionDetail = "complete",
|
||||
): ToolAvailabilityInventory | ToolAvailabilitySummaryInventory {
|
||||
): ToolAvailabilityInventory => {
|
||||
const clientRequirements =
|
||||
availability[0]?.client_requirements ?? EMPTY_CLIENT_REQUIREMENTS;
|
||||
const groups = new Map<string, { names: string[] }>();
|
||||
const groups = new Map<string, MutableAvailabilityGroup>();
|
||||
|
||||
for (const entry of availability) {
|
||||
const statusFacts = {
|
||||
const effects = TOOL_EFFECT_NAMES.filter((name) => entry.effects[name]);
|
||||
const annotations = TOOL_ANNOTATION_NAMES.filter(
|
||||
(name) => entry.annotations[name],
|
||||
);
|
||||
const groupFacts = {
|
||||
surface: entry.surface,
|
||||
effects,
|
||||
annotations,
|
||||
available: entry.available,
|
||||
reason: entry.reason,
|
||||
remediation: entry.remediation,
|
||||
@@ -369,31 +364,17 @@ export function compactToolAvailability(
|
||||
? {}
|
||||
: { client_requirements: entry.client_requirements }),
|
||||
};
|
||||
const groupFacts =
|
||||
detail === "summary"
|
||||
? statusFacts
|
||||
: {
|
||||
surface: entry.surface,
|
||||
effects: TOOL_EFFECT_NAMES.filter((name) => entry.effects[name]),
|
||||
annotations: TOOL_ANNOTATION_NAMES.filter(
|
||||
(name) => entry.annotations[name],
|
||||
),
|
||||
...statusFacts,
|
||||
};
|
||||
const key = JSON.stringify(groupFacts);
|
||||
const existing = groups.get(key);
|
||||
if (existing) existing.names.push(entry.name);
|
||||
else groups.set(key, { ...groupFacts, names: [entry.name] });
|
||||
}
|
||||
|
||||
const inventory = {
|
||||
return toolAvailabilityInventory.parse({
|
||||
client_requirements: clientRequirements,
|
||||
groups: [...groups.values()],
|
||||
};
|
||||
return detail === "summary"
|
||||
? toolAvailabilitySummaryInventory.parse(inventory)
|
||||
: toolAvailabilityInventory.parse(inventory);
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
const availabilityFor = (context: AvailabilityContext): Availability => {
|
||||
// These built-in workflows accept a caller-selected endpoint; they do not load the optional passive provider.
|
||||
|
||||
@@ -378,31 +378,6 @@ it.each([
|
||||
if (!result.ok) expect(result.error._tag).toBe("AnalysisInputError");
|
||||
});
|
||||
|
||||
it("reports an ordinal past the capture's records as an out-of-range selection", async () => {
|
||||
const report = fixture();
|
||||
const result = await new WebNetworkCaptureService({
|
||||
inspect: () => Promise.resolve(ok(report)),
|
||||
}).inspect({
|
||||
capture_path: "/capture.har",
|
||||
format: "har",
|
||||
record_ordinals: [0, 5],
|
||||
});
|
||||
if (result.ok) throw new Error("expected an out-of-range ordinal");
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
issues: [
|
||||
{
|
||||
path: ["record_ordinals", 1],
|
||||
reason: "out_of_range",
|
||||
minimum: 0,
|
||||
maximum: report.total_records - 1,
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a port changing the artifact identity or producer ordinal sequence", async () => {
|
||||
const report = fixture();
|
||||
report.artifact.path = "/different.har";
|
||||
|
||||
@@ -155,25 +155,16 @@ export class WebNetworkCaptureService {
|
||||
const selected =
|
||||
input.record_ordinals ?? value.records.map(({ ordinal }) => ordinal);
|
||||
const records = [];
|
||||
for (const [index, ordinal] of selected.entries()) {
|
||||
for (const ordinal of selected) {
|
||||
const record = safeValue.records[ordinal];
|
||||
// A selection past the capture's records is a caller range fault.
|
||||
if (record === undefined)
|
||||
return err(
|
||||
new AnalysisInputError(OPERATION, undefined, [
|
||||
value.total_records === 0
|
||||
? {
|
||||
path: ["record_ordinals", index],
|
||||
reason: "invalid_value",
|
||||
message: "The selected capture contains no records.",
|
||||
}
|
||||
: {
|
||||
path: ["record_ordinals", index],
|
||||
reason: "out_of_range",
|
||||
minimum: 0,
|
||||
maximum: value.total_records - 1,
|
||||
message: `Producer ordinal ${ordinal} is outside the retained capture (${value.total_records} records).`,
|
||||
},
|
||||
{
|
||||
path: ["record_ordinals"],
|
||||
reason: "invalid_format",
|
||||
message: `Producer ordinal ${ordinal} is outside the retained capture (${value.total_records} records).`,
|
||||
},
|
||||
]),
|
||||
);
|
||||
records.push({
|
||||
|
||||
@@ -1,521 +0,0 @@
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
SCENARIO_ORIGIN,
|
||||
scenarioCaptureEvidence,
|
||||
scenarioCaptureFixture,
|
||||
} from "../../../tests/fixtures/browserScenarioCapture.js";
|
||||
import { inspectAnalysisView } from "./AnalysisViewService.js";
|
||||
import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
|
||||
import { sanitizeBrowserUrl } from "../../domain/browserObservation.js";
|
||||
import type { BrowserScenarioCapture } from "../../domain/browserScenarioCapture.js";
|
||||
import type { AnalysisViewRequest } from "../../domain/analysisView/analysisView.js";
|
||||
import { projectBrowserScenarioView } from "../../domain/analysisView/browserScenarioView.js";
|
||||
|
||||
const tree = (text: string) => ({
|
||||
state: "captured" as const,
|
||||
value: {
|
||||
sha256: createHash("sha256").update(text).digest("hex"),
|
||||
bytes: Buffer.byteLength(text),
|
||||
text,
|
||||
},
|
||||
});
|
||||
|
||||
/** Three steps: navigate to /quiz, capture its tree, then fail on a missing control. */
|
||||
const quizCapture = (
|
||||
thirdStep: "failed" | "completed" = "failed",
|
||||
): BrowserScenarioCapture => {
|
||||
const base = scenarioCaptureFixture();
|
||||
const [initial, browse] = base.steps;
|
||||
if (initial === undefined || browse === undefined)
|
||||
throw new Error("Fixture lacks steps");
|
||||
const quiz = sanitizeBrowserUrl(`${SCENARIO_ORIGIN}/quiz`);
|
||||
return {
|
||||
...base,
|
||||
scenario: { ...base.scenario, action_count: 2 },
|
||||
steps: [
|
||||
{
|
||||
...initial,
|
||||
artifacts: {
|
||||
...initial.artifacts,
|
||||
url: { state: "captured", value: initial.after_url },
|
||||
},
|
||||
},
|
||||
{
|
||||
...browse,
|
||||
after_url: quiz,
|
||||
artifacts: {
|
||||
...browse.artifacts,
|
||||
accessibility: tree('- heading "Question 1"'),
|
||||
screenshot: { state: "missing", reason: "page closed" },
|
||||
},
|
||||
},
|
||||
{
|
||||
...browse,
|
||||
step_index: 2,
|
||||
step_id: "answer",
|
||||
action: "click",
|
||||
before_url: quiz,
|
||||
after_url: quiz,
|
||||
...(thirdStep === "failed"
|
||||
? { status: "failed" as const, error: "locator not found" }
|
||||
: { status: "completed" as const, error: null }),
|
||||
},
|
||||
],
|
||||
};
|
||||
};
|
||||
|
||||
const view = async (
|
||||
capture: BrowserScenarioCapture,
|
||||
request: AnalysisViewRequest,
|
||||
) => {
|
||||
const projected = await inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: scenarioCaptureEvidence(capture) },
|
||||
view: request,
|
||||
});
|
||||
if (!projected.ok) throw projected.error;
|
||||
return projected.value.normalized_result as Record<string, unknown>;
|
||||
};
|
||||
|
||||
describe("browser scenario analysis views", () => {
|
||||
it("counts inherited Evidence limitations alongside capture-local caveats", async () => {
|
||||
const capture = scenarioCaptureFixture();
|
||||
expect(capture.limitations).toEqual([]);
|
||||
const summary = await view(capture, { kind: "summary" });
|
||||
expect(summary).toMatchObject({
|
||||
limitations: [
|
||||
"Artifact identity is unavailable for this observation.",
|
||||
"Browser host platform and architecture were not observed.",
|
||||
],
|
||||
summary: { limitation_count: 2 },
|
||||
});
|
||||
});
|
||||
|
||||
it("summarizes steps compactly and focuses the first failing step's nearest captured tree", async () => {
|
||||
const summary = await view(quizCapture(), { kind: "summary" });
|
||||
expect(summary).toMatchObject({
|
||||
kind: "summary",
|
||||
parent_operation: "capture_browser_scenario",
|
||||
artifact: null,
|
||||
coverage: { status: "complete-within-view", exhausted: true },
|
||||
});
|
||||
const { steps, final_accessibility, network, events } =
|
||||
summary.summary as Record<string, unknown> & {
|
||||
steps: Record<string, unknown>[];
|
||||
};
|
||||
// before_url repeats only across a discontinuity; not_requested sections are omitted.
|
||||
expect(steps[0]).toHaveProperty("before_url");
|
||||
expect(steps[1]).not.toHaveProperty("before_url");
|
||||
expect(steps[2]).not.toHaveProperty("before_url");
|
||||
expect(steps[0]?.sections).toEqual({
|
||||
captured: ["url"],
|
||||
missing: [],
|
||||
truncated: [],
|
||||
});
|
||||
expect(steps[1]?.sections).toEqual({
|
||||
captured: ["accessibility"],
|
||||
missing: ["screenshot"],
|
||||
truncated: [],
|
||||
});
|
||||
expect(steps[2]).toMatchObject({
|
||||
status: "failed",
|
||||
error: "locator not found",
|
||||
after_url: { url: `${SCENARIO_ORIGIN}/quiz` },
|
||||
});
|
||||
expect(final_accessibility).toEqual({
|
||||
focus_step_index: 2,
|
||||
focus: "first-unsuccessful-step",
|
||||
step_index: 1,
|
||||
accessibility: tree('- heading "Question 1"'),
|
||||
});
|
||||
expect(network).toEqual({
|
||||
selected: true,
|
||||
transactions: 8,
|
||||
without_request: 0,
|
||||
ambiguous_transactions: 0,
|
||||
failed_transactions: 1,
|
||||
by_origin: [{ origin: SCENARIO_ORIGIN, transactions: 8 }],
|
||||
by_resource_type: [
|
||||
{ resource_type: "document", transactions: 1 },
|
||||
{ resource_type: "fetch", transactions: 7 },
|
||||
],
|
||||
by_status_class: [
|
||||
{ status_class: "2xx", transactions: 7 },
|
||||
{ status_class: "failed", transactions: 1 },
|
||||
],
|
||||
});
|
||||
expect(events).toMatchObject({ retained: 24, dropped: 0 });
|
||||
});
|
||||
|
||||
it("reports the last step's own state when no step captured a tree", async () => {
|
||||
const capture = scenarioCaptureFixture();
|
||||
expect(await view(capture, { kind: "summary" })).toMatchObject({
|
||||
summary: {
|
||||
final_accessibility: {
|
||||
focus_step_index: 1,
|
||||
focus: "last-step",
|
||||
step_index: null,
|
||||
accessibility: { state: "not_requested" },
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser scenario analysis view pages", () => {
|
||||
it("pages exact steps with only the selected captured sections", async () => {
|
||||
const page = await view(quizCapture("completed"), {
|
||||
kind: "browser-scenario",
|
||||
collection: "steps",
|
||||
step_indices: [1, 2],
|
||||
artifacts: ["accessibility"],
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
});
|
||||
expect(page).toMatchObject({
|
||||
kind: "browser-scenario",
|
||||
source_completeness: { status: "complete" },
|
||||
source_events: { retained: 24, dropped: 0 },
|
||||
coverage: { status: "page", examined: 1, total: 2, next_offset: 1 },
|
||||
items: {
|
||||
collection: "steps",
|
||||
rows: [
|
||||
{
|
||||
step_index: 1,
|
||||
artifacts: { accessibility: tree('- heading "Question 1"') },
|
||||
omitted_artifacts: [{ section: "screenshot", state: "missing" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("filters events by step, kind and transaction, keeping original item indices", async () => {
|
||||
const capture = scenarioCaptureFixture();
|
||||
const responses = (await view(capture, {
|
||||
kind: "browser-scenario",
|
||||
collection: "events",
|
||||
event_kinds: ["response", "request-failed"],
|
||||
transaction_ids: ["request-2", "request-8"],
|
||||
offset: 0,
|
||||
limit: 64,
|
||||
})) as { items: { rows: { item_index: number; event: unknown }[] } };
|
||||
expect(
|
||||
responses.items.rows.map(({ item_index, event }) => ({
|
||||
item_index,
|
||||
event,
|
||||
})),
|
||||
).toEqual(
|
||||
capture.events.items.flatMap((event, item_index) =>
|
||||
(event.kind === "response" || event.kind === "request-failed") &&
|
||||
["request-2", "request-8"].includes(event.transaction_id)
|
||||
? [{ item_index, event }]
|
||||
: [],
|
||||
),
|
||||
);
|
||||
expect(responses.items.rows).toHaveLength(2);
|
||||
for (const empty of [
|
||||
{ step_indices: [] },
|
||||
{ event_kinds: [] },
|
||||
{ transaction_ids: [] },
|
||||
{ step_indices: [0] },
|
||||
])
|
||||
expect(
|
||||
await view(capture, {
|
||||
kind: "browser-scenario",
|
||||
collection: "events",
|
||||
offset: 0,
|
||||
limit: 64,
|
||||
...empty,
|
||||
}),
|
||||
).toMatchObject({
|
||||
items: { rows: [] },
|
||||
coverage: { status: "empty", total: 0, exhausted: true },
|
||||
});
|
||||
});
|
||||
|
||||
it("states dropped events and refuses views of other parents", async () => {
|
||||
const base = scenarioCaptureFixture();
|
||||
const dropped = {
|
||||
...base,
|
||||
events: {
|
||||
...base.events,
|
||||
dropped: 3,
|
||||
omissions: {
|
||||
...base.events.omissions,
|
||||
by_kind: [
|
||||
{
|
||||
kind: "console" as const,
|
||||
count: 3,
|
||||
reasons: ["max-events" as const],
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
completeness: {
|
||||
status: "truncated" as const,
|
||||
equality_eligible: false as const,
|
||||
missing_sections: [],
|
||||
truncated_sections: ["events" as const],
|
||||
},
|
||||
};
|
||||
const summary = await view(dropped, { kind: "summary" });
|
||||
expect(summary.unknowns).toContain(
|
||||
"3 events were dropped by capture budgets; counts and pages cover retained events only.",
|
||||
);
|
||||
expect(summary).toMatchObject({
|
||||
summary: {
|
||||
events: {
|
||||
dropped: 3,
|
||||
omissions: {
|
||||
by_kind: [{ kind: "console", count: 3 }],
|
||||
network_transactions: 0,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
const refused = await inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: scenarioCaptureEvidence() },
|
||||
view: { kind: "go", collection: "dependencies" },
|
||||
});
|
||||
if (refused.ok) throw new Error("Expected an unsupported view refusal");
|
||||
expect(projectAnalysisError(refused.error)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser scenario retained boundaries", () => {
|
||||
it("counts transactions whose request event was omitted and keeps source omissions on an exhausted page", async () => {
|
||||
const base = scenarioCaptureFixture();
|
||||
const omitted = (
|
||||
event: BrowserScenarioCapture["events"]["items"][number],
|
||||
) =>
|
||||
event.kind === "request" ||
|
||||
(event.kind === "network-content" && event.phase === "request");
|
||||
const removed = base.events.items.filter(omitted);
|
||||
const items = base.events.items.filter((event) => !omitted(event));
|
||||
const capture: BrowserScenarioCapture = {
|
||||
...base,
|
||||
events: {
|
||||
...base.events,
|
||||
items,
|
||||
retained: items.length,
|
||||
dropped: removed.length,
|
||||
omissions: {
|
||||
...base.events.omissions,
|
||||
by_kind: [
|
||||
{ kind: "request", count: 8, reasons: ["max-events"] },
|
||||
{ kind: "network-content", count: 1, reasons: ["max-events"] },
|
||||
],
|
||||
},
|
||||
},
|
||||
completeness: {
|
||||
status: "truncated",
|
||||
equality_eligible: false,
|
||||
missing_sections: [],
|
||||
truncated_sections: ["events"],
|
||||
},
|
||||
};
|
||||
expect(await view(capture, { kind: "summary" })).toMatchObject({
|
||||
summary: {
|
||||
network: {
|
||||
transactions: 8,
|
||||
without_request: 8,
|
||||
failed_transactions: 1,
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(
|
||||
await view(capture, {
|
||||
kind: "browser-scenario",
|
||||
collection: "transactions",
|
||||
transaction_ids: ["request-8"],
|
||||
event_kinds: [],
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
}),
|
||||
).toMatchObject({
|
||||
source_completeness: { status: "truncated" },
|
||||
source_events: { dropped: 9 },
|
||||
coverage: { total: 1, exhausted: true },
|
||||
items: {
|
||||
rows: [
|
||||
{
|
||||
transaction_id: "request-8",
|
||||
events: [],
|
||||
omitted_events: [{ kind: "request-failed" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps network bodies opt-in while retaining each transaction's event identities", async () => {
|
||||
const capture = scenarioCaptureFixture();
|
||||
const request = {
|
||||
kind: "browser-scenario" as const,
|
||||
collection: "transactions" as const,
|
||||
transaction_ids: ["request-2"],
|
||||
offset: 0,
|
||||
limit: 1,
|
||||
};
|
||||
// Parse defaults at the public boundary rather than supplying a manually normalized request.
|
||||
const page = await inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: scenarioCaptureEvidence(capture) },
|
||||
view: request,
|
||||
});
|
||||
if (!page.ok) throw page.error;
|
||||
const result = page.value.normalized_result;
|
||||
expect(result).toMatchObject({
|
||||
items: {
|
||||
collection: "transactions",
|
||||
rows: [{ transaction_id: "request-2" }],
|
||||
},
|
||||
});
|
||||
expect(JSON.stringify(result)).not.toContain('"content":');
|
||||
const selected = await view(capture, {
|
||||
...request,
|
||||
event_kinds: ["network-content"],
|
||||
});
|
||||
expect(JSON.stringify(selected)).toContain('"content":');
|
||||
expect(selected).toMatchObject({
|
||||
items: {
|
||||
rows: [
|
||||
{
|
||||
omitted_events: expect.arrayContaining([
|
||||
{ item_index: expect.any(Number), kind: "request" },
|
||||
]),
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("distinguishes unrequested and legacy-unknown network observation from an observed empty inventory", () => {
|
||||
const base = scenarioCaptureFixture([], {
|
||||
request_body: false,
|
||||
response_body: false,
|
||||
header_values: false,
|
||||
});
|
||||
const project = (capture: unknown) => {
|
||||
const result = projectBrowserScenarioView(
|
||||
{
|
||||
evidenceId: `ev:${"a".repeat(64)}`,
|
||||
operation: "capture_browser_scenario",
|
||||
normalizedResult: capture,
|
||||
limitations: [],
|
||||
},
|
||||
{ kind: "summary" },
|
||||
);
|
||||
if (!result.ok) throw result.error;
|
||||
return result.value;
|
||||
};
|
||||
expect(project(base)).toMatchObject({
|
||||
summary: { network: { selected: true, transactions: 0 } },
|
||||
});
|
||||
expect(
|
||||
project({ ...base, scenario: { ...base.scenario, event_families: [] } }),
|
||||
).toMatchObject({
|
||||
summary: { network: { selected: false, transactions: null } },
|
||||
});
|
||||
const { event_families: _families, ...legacyScenario } = base.scenario;
|
||||
expect(project({ ...base, scenario: legacyScenario })).toMatchObject({
|
||||
summary: { network: { selected: null, transactions: null } },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser scenario text and selection boundaries", () => {
|
||||
it("pages exact Unicode text with the full artifact digest and reports unavailable text explicitly", async () => {
|
||||
const capture = quizCapture("completed");
|
||||
const step = capture.steps[1]!;
|
||||
step.artifacts.accessibility = tree("A💡B\nC");
|
||||
const request = {
|
||||
kind: "browser-scenario" as const,
|
||||
collection: "text" as const,
|
||||
step_index: 1,
|
||||
artifact: "accessibility" as const,
|
||||
offset: 1,
|
||||
limit: 2,
|
||||
};
|
||||
expect(await view(capture, request)).toMatchObject({
|
||||
items: {
|
||||
state: "captured",
|
||||
text: "💡",
|
||||
sha256: tree("A💡B\nC").value.sha256,
|
||||
bytes: Buffer.byteLength("A💡B\nC"),
|
||||
},
|
||||
coverage: { examined: 2, total: 6, next_offset: 3 },
|
||||
});
|
||||
expect(await view(capture, { ...request, offset: 99 })).toMatchObject({
|
||||
items: { text: "" },
|
||||
coverage: { status: "empty", total: 6, exhausted: true },
|
||||
});
|
||||
expect(await view(capture, { ...request, step_index: 0 })).toMatchObject({
|
||||
items: { state: "not_requested", text: null, sha256: null, bytes: null },
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects irrelevant selection fields and nonexistent text steps with typed diagnostics", async () => {
|
||||
for (const request of [
|
||||
{ kind: "browser-scenario", collection: "events", artifacts: [] },
|
||||
{
|
||||
kind: "browser-scenario",
|
||||
collection: "text",
|
||||
step_index: 99,
|
||||
artifact: "dom",
|
||||
},
|
||||
]) {
|
||||
const result = await inspectAnalysisView({
|
||||
source: { kind: "inline", evidence: scenarioCaptureEvidence() },
|
||||
view: request,
|
||||
});
|
||||
if (result.ok)
|
||||
throw new Error("Expected the invalid view to be rejected");
|
||||
expect(projectAnalysisError(result.error)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps source truncation facts when a text page has no retained payload", async () => {
|
||||
const capture = quizCapture("completed");
|
||||
const completeness = {
|
||||
status: "truncated" as const,
|
||||
equality_eligible: false as const,
|
||||
missing_sections: [],
|
||||
truncated_sections: ["accessibility" as const],
|
||||
};
|
||||
capture.steps[1]!.artifacts.accessibility = {
|
||||
state: "truncated",
|
||||
observed: 1000,
|
||||
retained: 0,
|
||||
reason: "capture budget",
|
||||
};
|
||||
capture.steps[1]!.completeness = completeness;
|
||||
capture.completeness = completeness;
|
||||
expect(
|
||||
await view(capture, {
|
||||
kind: "browser-scenario",
|
||||
collection: "text",
|
||||
step_index: 1,
|
||||
artifact: "accessibility",
|
||||
offset: 0,
|
||||
limit: 20,
|
||||
}),
|
||||
).toMatchObject({
|
||||
source_completeness: { status: "truncated" },
|
||||
items: {
|
||||
state: "truncated",
|
||||
observed: 1000,
|
||||
retained: 0,
|
||||
reason: "capture budget",
|
||||
text: null,
|
||||
},
|
||||
coverage: { status: "empty", exhausted: true },
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -36,7 +36,6 @@ import { InvestigationRecords } from "../investigation/InvestigationRecords.js";
|
||||
import { recordEvidenceSources } from "../investigation/recordEvidenceSources.js";
|
||||
import { BinarySessionRecords } from "./BinarySessionRecords.js";
|
||||
import { binarySessionStatus } from "./BinarySessionStatus.js";
|
||||
import type { BinarySessionDetail } from "../../contracts/sessionStatusContract.js";
|
||||
import {
|
||||
resolveSessionOpen,
|
||||
resolveSessionTarget,
|
||||
@@ -1025,12 +1024,7 @@ export class BinarySession
|
||||
}
|
||||
|
||||
/** Describe the active binary session. */
|
||||
status(
|
||||
options: {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly detail?: BinarySessionDetail;
|
||||
} = {},
|
||||
): Promise<JsonValue> {
|
||||
status(options: { readonly signal?: AbortSignal } = {}): Promise<JsonValue> {
|
||||
const resourceRun = this.#activeBundleResources;
|
||||
const run =
|
||||
this.#active ??
|
||||
|
||||
@@ -8,7 +8,6 @@ import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import type { AnalysisSnapshot } from "../../domain/analysisSnapshot.js";
|
||||
import type { AnalysisProfileCommitment } from "../../domain/analysisProfile.js";
|
||||
import type { AnalysisProviderSelector } from "../../contracts/providerSelection.js";
|
||||
import type { BinarySessionDetail } from "../../contracts/sessionStatusContract.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import type { Result } from "../../domain/result.js";
|
||||
import type { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
|
||||
@@ -94,11 +93,7 @@ export interface BinarySessionPort
|
||||
overwrite: boolean,
|
||||
options?: Pick<ExecutionOptions, "progress">,
|
||||
): Promise<Result<SavedAnalysisSnapshot, AnalysisError>>;
|
||||
/** Default complete detail; summary omits provider diagnostics and candidate capabilities. */
|
||||
status(options?: {
|
||||
readonly signal?: AbortSignal;
|
||||
readonly detail?: BinarySessionDetail;
|
||||
}): Promise<JsonValue>;
|
||||
status(options?: { readonly signal?: AbortSignal }): Promise<JsonValue>;
|
||||
activeTarget(): BinaryTarget | undefined;
|
||||
/** Read a snapshot with failed descriptors retained for session cleanup retry. */
|
||||
readAnalysisSnapshot(
|
||||
|
||||
@@ -5,12 +5,9 @@ import type {
|
||||
CapabilityDescriptor,
|
||||
ProviderIdentity,
|
||||
} from "../AnalysisProvider.js";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
analysisProviderCandidate,
|
||||
analysisProviderCandidateSummary,
|
||||
expandProviderCapabilities,
|
||||
sessionProviderFacts,
|
||||
sessionProvider,
|
||||
type ProviderCapabilitySet,
|
||||
} from "../../contracts/toolOutputSchemaPrimitives.js";
|
||||
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
|
||||
@@ -26,13 +23,9 @@ import {
|
||||
nativeMacOSCapabilities,
|
||||
} from "../../native/NativeMacOSProviderMetadata.js";
|
||||
|
||||
const projectedStatusSchema = z.object({
|
||||
capabilities: sessionProviderFacts.shape.capabilities,
|
||||
analysis_provider_candidates: z.array(analysisProviderCandidate),
|
||||
});
|
||||
const summaryStatusSchema = z.object({
|
||||
capabilities: sessionProviderFacts.shape.capabilities,
|
||||
analysis_provider_candidates: z.array(analysisProviderCandidateSummary),
|
||||
const projectedStatusSchema = sessionProvider.pick({
|
||||
capabilities: true,
|
||||
analysis_provider_candidates: true,
|
||||
});
|
||||
|
||||
const providerA: ProviderIdentity = {
|
||||
@@ -133,25 +126,6 @@ const project = async (
|
||||
{ readonly reason: string }
|
||||
> = new Map(),
|
||||
target?: BinaryTarget,
|
||||
) =>
|
||||
projectedStatusSchema.parse(
|
||||
await projectStatus(routeCapabilities, candidates, {
|
||||
runtimeUnavailability,
|
||||
...(target === undefined ? {} : { target }),
|
||||
}),
|
||||
);
|
||||
|
||||
const projectStatus = async (
|
||||
routeCapabilities: readonly CapabilityDescriptor[],
|
||||
candidates: readonly AnalysisProviderCandidateStatus[],
|
||||
options: {
|
||||
readonly runtimeUnavailability?: ReadonlyMap<
|
||||
string,
|
||||
{ readonly reason: string }
|
||||
>;
|
||||
readonly target?: BinaryTarget;
|
||||
readonly detail?: "summary" | "complete";
|
||||
} = {},
|
||||
) => {
|
||||
const route: SessionProviderRoute = {
|
||||
identity: providerA,
|
||||
@@ -168,16 +142,17 @@ const projectStatus = async (
|
||||
const router: Pick<SessionProviderRouter, "candidateStatuses"> = {
|
||||
candidateStatuses: async () => candidates,
|
||||
};
|
||||
return binarySessionStatus({
|
||||
target: options.target,
|
||||
route,
|
||||
router,
|
||||
runtimeUnavailability: options.runtimeUnavailability ?? new Map(),
|
||||
runId: undefined,
|
||||
runtimeLineageSnapshots: [],
|
||||
requestActivitySnapshots: [],
|
||||
...(options.detail === undefined ? {} : { detail: options.detail }),
|
||||
});
|
||||
return projectedStatusSchema.parse(
|
||||
await binarySessionStatus({
|
||||
target,
|
||||
route,
|
||||
router,
|
||||
runtimeUnavailability,
|
||||
runId: undefined,
|
||||
runtimeLineageSnapshots: [],
|
||||
requestActivitySnapshots: [],
|
||||
}),
|
||||
);
|
||||
};
|
||||
|
||||
const expandCanonicalCapabilities = (sets: readonly ProviderCapabilitySet[]) =>
|
||||
@@ -469,87 +444,3 @@ it.each(["elf", "pe"] as const)(
|
||||
]);
|
||||
},
|
||||
);
|
||||
|
||||
describe("binary session summary projection", () => {
|
||||
it("keeps each candidate's status, code and reason in summary without diagnostics or capability sets", async () => {
|
||||
const unavailable: AnalysisProviderCandidateStatus = {
|
||||
...candidate(providerB, otherCapabilities),
|
||||
availability: {
|
||||
status: "unavailable",
|
||||
code: "not_configured",
|
||||
reason: "Set PROVIDER_B_HOME; see docs/provider-b.md.",
|
||||
diagnostics: { checks: [{ name: "configuration", ok: false }] },
|
||||
},
|
||||
targetSupport: {
|
||||
status: "unsupported",
|
||||
code: "architecture_unsupported",
|
||||
reason: "Provider B cannot select this slice.",
|
||||
diagnostics: { architecture: "arm64" },
|
||||
},
|
||||
selected: false,
|
||||
};
|
||||
const pending: AnalysisProviderCandidateStatus = {
|
||||
...candidate(providerA, selectedCapabilities),
|
||||
availability: {
|
||||
status: "unknown",
|
||||
code: null,
|
||||
reason: "Provider A startup is still pending.",
|
||||
diagnostics: { startup: "pending" },
|
||||
},
|
||||
};
|
||||
const candidates = [pending, unavailable];
|
||||
const before = structuredClone(candidates);
|
||||
const runtimeUnavailability = new Map([
|
||||
["procedure_info", { reason: "Runtime operation failure" }],
|
||||
]);
|
||||
const complete = projectedStatusSchema.parse(
|
||||
await projectStatus(selectedCapabilities, candidates, {
|
||||
runtimeUnavailability,
|
||||
}),
|
||||
);
|
||||
const raw = await projectStatus(selectedCapabilities, candidates, {
|
||||
runtimeUnavailability,
|
||||
detail: "summary",
|
||||
});
|
||||
const summary = summaryStatusSchema.parse(raw);
|
||||
|
||||
expect(candidates).toEqual(before);
|
||||
expect(summary.capabilities).toEqual(complete.capabilities);
|
||||
expect(summary.analysis_provider_candidates).toEqual([
|
||||
{
|
||||
provider: providerA,
|
||||
availability: {
|
||||
status: "unknown",
|
||||
code: null,
|
||||
reason: "Provider A startup is still pending.",
|
||||
},
|
||||
target_support: {
|
||||
status: "unknown",
|
||||
code: null,
|
||||
reason: "No target is open.",
|
||||
},
|
||||
selected: true,
|
||||
},
|
||||
{
|
||||
provider: providerB,
|
||||
availability: {
|
||||
status: "unavailable",
|
||||
code: "not_configured",
|
||||
reason: "Set PROVIDER_B_HOME; see docs/provider-b.md.",
|
||||
},
|
||||
target_support: {
|
||||
status: "unsupported",
|
||||
code: "architecture_unsupported",
|
||||
reason: "Provider B cannot select this slice.",
|
||||
},
|
||||
selected: false,
|
||||
},
|
||||
]);
|
||||
expect(JSON.stringify(raw)).not.toContain("diagnostics");
|
||||
expect(
|
||||
analysisProviderCandidate
|
||||
.array()
|
||||
.safeParse(summary.analysis_provider_candidates).success,
|
||||
).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -19,7 +19,6 @@ import type {
|
||||
SessionProviderRouter,
|
||||
} from "./SessionProviderRouter.js";
|
||||
import { compareUnicodeCodePoints } from "../../domain/unicodeCodePointOrder.js";
|
||||
import type { BinarySessionDetail } from "../../contracts/sessionStatusContract.js";
|
||||
|
||||
interface BinarySessionStatusInput {
|
||||
readonly target: BinaryTarget | undefined;
|
||||
@@ -36,7 +35,6 @@ interface BinarySessionStatusInput {
|
||||
readonly requestActivitySnapshots: readonly ProviderRequestActivitySnapshot[];
|
||||
readonly providerOperationHealth?: ProviderOperationHealth | null;
|
||||
readonly cleanupPending?: boolean;
|
||||
readonly detail?: BinarySessionDetail;
|
||||
readonly signal?: AbortSignal;
|
||||
}
|
||||
|
||||
@@ -53,7 +51,6 @@ export const binarySessionStatus = async ({
|
||||
requestActivitySnapshots,
|
||||
providerOperationHealth,
|
||||
cleanupPending = false,
|
||||
detail = "complete",
|
||||
signal,
|
||||
}: BinarySessionStatusInput): Promise<JsonValue> => {
|
||||
const routeCapabilities = (
|
||||
@@ -100,31 +97,20 @@ export const binarySessionStatus = async ({
|
||||
const analysisProviderCandidates =
|
||||
bundleResourceRoot !== undefined
|
||||
? []
|
||||
: (await router.candidateStatuses(route, signal)).map((candidate) =>
|
||||
detail === "summary"
|
||||
? {
|
||||
provider: providerSummary(candidate.provider),
|
||||
availability: providerStateSummary(candidate.availability),
|
||||
target_support: providerStateSummary(candidate.targetSupport),
|
||||
selected: candidate.selected,
|
||||
}
|
||||
: {
|
||||
provider: providerSummary(candidate.provider),
|
||||
availability: structuredClone(candidate.availability),
|
||||
target_support: structuredClone(candidate.targetSupport),
|
||||
selected: candidate.selected,
|
||||
capabilities: providerCapabilitySets(
|
||||
candidate.capabilities
|
||||
.slice()
|
||||
.filter(
|
||||
({ operation }) => operation !== "inventory_artifact",
|
||||
)
|
||||
.sort((left, right) =>
|
||||
compareUnicodeCodePoints(left.operation, right.operation),
|
||||
),
|
||||
),
|
||||
},
|
||||
);
|
||||
: (await router.candidateStatuses(route, signal)).map((candidate) => ({
|
||||
provider: providerSummary(candidate.provider),
|
||||
availability: structuredClone(candidate.availability),
|
||||
target_support: structuredClone(candidate.targetSupport),
|
||||
selected: candidate.selected,
|
||||
capabilities: providerCapabilitySets(
|
||||
candidate.capabilities
|
||||
.slice()
|
||||
.filter(({ operation }) => operation !== "inventory_artifact")
|
||||
.sort((left, right) =>
|
||||
compareUnicodeCodePoints(left.operation, right.operation),
|
||||
),
|
||||
),
|
||||
}));
|
||||
const common = {
|
||||
cleanup_pending: cleanupPending,
|
||||
capabilities,
|
||||
@@ -199,19 +185,6 @@ const requestActivityStatus = (
|
||||
};
|
||||
};
|
||||
|
||||
/** Keep a provider-owned status, code, and reason; diagnostics stay in complete detail. */
|
||||
const providerStateSummary = <
|
||||
State extends {
|
||||
readonly status: string;
|
||||
readonly code: string | null;
|
||||
readonly reason: string | null;
|
||||
},
|
||||
>({
|
||||
status,
|
||||
code,
|
||||
reason,
|
||||
}: State) => ({ status, code, reason });
|
||||
|
||||
const providerSummary = ({
|
||||
id,
|
||||
name,
|
||||
|
||||
@@ -22,8 +22,6 @@ const report: AppleIps = {
|
||||
bundle_id: null,
|
||||
incident_id: null,
|
||||
name: null,
|
||||
build_version: null,
|
||||
platform: null,
|
||||
unlisted_keys: [],
|
||||
},
|
||||
process: {
|
||||
@@ -31,7 +29,6 @@ const report: AppleIps = {
|
||||
path: null,
|
||||
pid: null,
|
||||
parent: null,
|
||||
parent_pid: null,
|
||||
cpu_type: null,
|
||||
translated: null,
|
||||
os_version: null,
|
||||
@@ -203,23 +200,19 @@ it.each(["threads", "images"] as const)(
|
||||
);
|
||||
|
||||
it.each([
|
||||
[131, false],
|
||||
[132, true],
|
||||
[43, false],
|
||||
[44, true],
|
||||
] as const)(
|
||||
"checks repeated integer text against %s UTF-16 bytes",
|
||||
async (maximum, accepted) => {
|
||||
const value = { decimal: "12345", hex: "0x3039" };
|
||||
const repeated: AppleIps = {
|
||||
...report,
|
||||
metadata: { ...report.metadata, platform: value },
|
||||
process: { ...report.process, pid: value, parent_pid: value },
|
||||
process: { ...report.process, pid: value },
|
||||
termination: {
|
||||
namespace: null,
|
||||
code: value,
|
||||
indicator: null,
|
||||
flags: value,
|
||||
by_pid: value,
|
||||
by_proc: null,
|
||||
reasons: null,
|
||||
unlisted_keys: [],
|
||||
},
|
||||
|
||||
@@ -56,28 +56,6 @@ it("records the decoded layout once in its Evidence", async () => {
|
||||
expect(evidence.raw_result).toBeNull();
|
||||
});
|
||||
|
||||
it.each([
|
||||
[{ machine: "EM_X86_64", bits: 64, byte_order: "little" }, "x86_64"],
|
||||
[{ machine: "EM_386", bits: 32, byte_order: "little" }, "x86"],
|
||||
[{ machine: "EM_ARM", bits: 32, byte_order: "little" }, "arm"],
|
||||
[{ machine: "EM_MIPS", bits: 32, byte_order: "big" }, "mips"],
|
||||
] as const)(
|
||||
"names the reported %j machine in its Evidence subject",
|
||||
async (architecture, expected) => {
|
||||
const layout = { ...binaryLayoutFixture(), architecture };
|
||||
const service = new BinaryLayoutService({
|
||||
inspectAndClose(input, options) {
|
||||
return this.inspect(input, options);
|
||||
},
|
||||
identity: BINARY_LAYOUT_TEST_PROVIDER,
|
||||
inspect: () => Promise.resolve(ok(layout)),
|
||||
});
|
||||
const result = await service.inspect({ path: layout.artifact.path });
|
||||
if (!result.ok) throw result.error;
|
||||
expect(parseEvidence(result.value).subject?.architecture).toBe(expected);
|
||||
},
|
||||
);
|
||||
|
||||
it("preserves a provider's actionable unsupported reason", async () => {
|
||||
const failure = new AnalysisCapabilityUnavailableError(
|
||||
"fixture-layout",
|
||||
|
||||
@@ -15,12 +15,6 @@ import { err, ok, type Result } from "../../domain/result.js";
|
||||
import { completeProviderEvidence } from "../ProviderOperationExecution.js";
|
||||
|
||||
const OPERATION = "inspect_binary_layout";
|
||||
const EVIDENCE_ARCHITECTURES = {
|
||||
EM_X86_64: "x86_64",
|
||||
EM_386: "x86",
|
||||
EM_ARM: "arm",
|
||||
EM_MIPS: "mips",
|
||||
} as const;
|
||||
|
||||
/** Shared CLI/MCP validation and Evidence composition for offline layout inspection. */
|
||||
export class BinaryLayoutService {
|
||||
@@ -93,7 +87,7 @@ export class BinaryLayoutService {
|
||||
{
|
||||
path: value.artifact.path,
|
||||
format: "elf",
|
||||
architecture: EVIDENCE_ARCHITECTURES[value.architecture.machine],
|
||||
architecture: "x86_64",
|
||||
sha256: value.artifact.sha256,
|
||||
},
|
||||
this.provider.identity,
|
||||
|
||||
@@ -76,15 +76,15 @@ describe("environment manifests from capture_browser_scenario parents", () => {
|
||||
kind: "inline",
|
||||
evidence: scenarioCaptureEvidence(scenarioCaptureFixture()),
|
||||
},
|
||||
view: { kind: "dart-aot", collection: "package_uris" },
|
||||
view: { kind: "summary" },
|
||||
});
|
||||
if (refused.ok) throw new Error("Expected the unsupported view refusal");
|
||||
expect(projectAnalysisError(refused.error)).toMatchObject({
|
||||
code: "invalid_request",
|
||||
code: "unsupported_target",
|
||||
message: expect.stringContaining(
|
||||
"supports only the environment-manifest, environment-state and environment-coverage views",
|
||||
),
|
||||
});
|
||||
expect(JSON.stringify(projectAnalysisError(refused.error))).toContain(
|
||||
"supports summary and browser-scenario views",
|
||||
);
|
||||
});
|
||||
|
||||
it("serves and exports a scenario-built manifest from the same capture", () => {
|
||||
|
||||
@@ -10,12 +10,6 @@ import {
|
||||
referencedCreateRows,
|
||||
shelvedRows,
|
||||
} from "../../../tests/fixtures/environmentStateDerivedValues.js";
|
||||
import {
|
||||
LISTED_LABEL_ROWS,
|
||||
READ_BACK_TAG_ROWS,
|
||||
RESTAMPED_ROWS,
|
||||
STALE_SLUG_ROWS,
|
||||
} from "../../../tests/fixtures/environmentStateWriteViews.js";
|
||||
import { servedFrom } from "../../../tests/fixtures/environmentStateServing.js";
|
||||
import {
|
||||
bookClock,
|
||||
@@ -186,72 +180,3 @@ describe("served objects that repeat a named entity, and sorted arrays", () => {
|
||||
).toEqual(["fog", "ice", "wind"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("served writes whose answers show the entity otherwise than reads do", () => {
|
||||
it("answers a create as the application answered it and holds the fields its read-back showed", () => {
|
||||
const tags = servedFrom(READ_BACK_TAG_ROWS);
|
||||
const created = objectOf(
|
||||
tags.answer("post_tags", { body: { name: "Saltmarsh Gate" } }).body,
|
||||
);
|
||||
expect(Object.keys(created).sort()).toEqual(
|
||||
["created", "id", "name", "slug"].sort(),
|
||||
);
|
||||
expect(created).toMatchObject({
|
||||
name: "Saltmarsh Gate",
|
||||
slug: "saltmarsh-gate",
|
||||
created: firstServedClock(READ_BACK_TAG_ROWS).replace("Z", ".000000000Z"),
|
||||
});
|
||||
expect(
|
||||
tags.answer("get_tags_by_id", { id: String(created["id"]) }).body,
|
||||
).toMatchObject({
|
||||
name: "Saltmarsh Gate",
|
||||
document_count: 0,
|
||||
created: firstServedClock(READ_BACK_TAG_ROWS),
|
||||
});
|
||||
});
|
||||
|
||||
it("lists a created label with the count its list read showed", () => {
|
||||
const labels = servedFrom(LISTED_LABEL_ROWS);
|
||||
labels.answer("post_labels", { body: { title: "Slipway" } });
|
||||
expect((labels.answer("get_labels").body as JsonValue[]).at(-1)).toEqual({
|
||||
id: 3,
|
||||
title: "Slipway",
|
||||
count: { posts: 0 },
|
||||
});
|
||||
});
|
||||
|
||||
it("slugs a renamed tag anew whatever slug the call sends back", () => {
|
||||
const tags = servedFrom(STALE_SLUG_ROWS);
|
||||
const renamed = objectOf(
|
||||
tags.answer("put_tags_by_id", {
|
||||
id: "2",
|
||||
body: {
|
||||
id: 2,
|
||||
slug: "tide-tables",
|
||||
name: "Neap Tide Tables",
|
||||
document_count: 0,
|
||||
created: "2026-10-01T10:00:00Z",
|
||||
},
|
||||
}).body,
|
||||
);
|
||||
expect(renamed).toMatchObject({
|
||||
name: "Neap Tide Tables",
|
||||
slug: "neap-tide-tables",
|
||||
});
|
||||
});
|
||||
|
||||
it("stamps a post's modification time anew rather than storing the one the call sends back", () => {
|
||||
const posts = servedFrom(RESTAMPED_ROWS);
|
||||
const edited = objectOf(
|
||||
posts.answer("put_posts_by_id", {
|
||||
id: "1",
|
||||
body: { id: 1, title: "Winter rota v4", updated_at: bookClock(4) },
|
||||
}).body,
|
||||
);
|
||||
expect(edited["title"]).toBe("Winter rota v4");
|
||||
expect(edited["updated_at"]).not.toBe(bookClock(4));
|
||||
expect(Date.parse(String(edited["updated_at"]))).toBeGreaterThan(
|
||||
Date.parse(bookClock(4)),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -29,7 +29,6 @@ import {
|
||||
type StateWritePlan,
|
||||
type WriteCall,
|
||||
} from "../../domain/analysisView/environmentStateWriteRules.js";
|
||||
import { withRead } from "../../domain/analysisView/environmentStateWriteObserve.js";
|
||||
import {
|
||||
formatClock,
|
||||
generatedIntegerDigits,
|
||||
@@ -140,30 +139,17 @@ const callBody = ({
|
||||
: err(unloadable(tool, `Member ${member}`));
|
||||
};
|
||||
|
||||
/** The object a recorded response shows at a pointer. */
|
||||
const recordedObject = (
|
||||
serving: StateServing,
|
||||
at: { readonly ordinal: number; readonly pointer: string },
|
||||
): JsonObject | undefined => {
|
||||
const exchange = serving.source.exchanges.get(at.ordinal);
|
||||
const body = exchange === undefined ? undefined : recordedJson(exchange);
|
||||
const object = body === undefined ? undefined : valueAt(body, at.pointer);
|
||||
return isObject(object) ? object : undefined;
|
||||
};
|
||||
|
||||
/** The entity a recorded observation shows, with a create's read-back over it: the values a create keeps and an update's constants. */
|
||||
/** The entity a recorded observation shows: the values a create keeps and an update's constants. */
|
||||
export const observedEntity = (
|
||||
serving: StateServing,
|
||||
plan: StateWritePlan,
|
||||
): JsonObject | undefined => {
|
||||
if (plan.observation === null) return undefined;
|
||||
const object = recordedObject(serving, plan.observation);
|
||||
if (object === undefined || plan.observation.read === undefined)
|
||||
return object;
|
||||
const read = recordedObject(serving, plan.observation.read);
|
||||
return read === undefined
|
||||
? undefined
|
||||
: withRead(object, read, plan.observation.read.shape);
|
||||
const exchange = serving.source.exchanges.get(plan.observation.ordinal);
|
||||
const body = exchange === undefined ? undefined : recordedJson(exchange);
|
||||
const object =
|
||||
body === undefined ? undefined : valueAt(body, plan.observation.pointer);
|
||||
return isObject(object) ? object : undefined;
|
||||
};
|
||||
|
||||
/** One past the largest integer these entities hold in the field, read as the exemplar spells it, or undefined when none holds one. */
|
||||
@@ -361,15 +347,7 @@ export const writeAnswer = (
|
||||
: writtenResponse(
|
||||
plan.response,
|
||||
recorded,
|
||||
{
|
||||
...rendered,
|
||||
clocks:
|
||||
plan.observation?.read?.shape === "item"
|
||||
? plan.rules.flatMap((rule) =>
|
||||
rule.rule === "clock" ? [rule.field] : [],
|
||||
)
|
||||
: [],
|
||||
},
|
||||
rendered,
|
||||
optionalFields(plan),
|
||||
);
|
||||
if (!body.ok) return err(refused(tool, body.error));
|
||||
|
||||
@@ -2,7 +2,7 @@ import { expect, it } from "vitest";
|
||||
import { createHash } from "node:crypto";
|
||||
import { matchNativeFunctions } from "./NativeFunctionMatchingService.js";
|
||||
import { nativeFunctionMatchingSchema } from "../../domain/native/functionMatching.js";
|
||||
import { createEvidence, immutableEvidence } from "../../domain/evidence.js";
|
||||
import { createEvidence } from "../../domain/evidence.js";
|
||||
import { createAnalysisProfile } from "../../domain/analysisProfile.js";
|
||||
import { functionDossierSchema } from "../../domain/hopperValues.js";
|
||||
import { procedureInfoSchema } from "../../domain/native/procedureInfo.js";
|
||||
@@ -791,134 +791,6 @@ it("cooperatively preserves the exact cancellation reason and rejects expansion
|
||||
expect(large.error._tag).toBe("AnalysisResourceConstraintError");
|
||||
});
|
||||
|
||||
const retainedInventories = (
|
||||
rows: (side: "1" | "2") => readonly Record<string, unknown>[],
|
||||
) => {
|
||||
const [left, right] = (["1", "2"] as const).map((side) =>
|
||||
immutableEvidence(
|
||||
createEvidence(nativeMatchingTarget(side), nativeMatchingProvider, {
|
||||
operation: "list_procedures",
|
||||
parameters: {},
|
||||
result: rows(side),
|
||||
}),
|
||||
),
|
||||
);
|
||||
const ledger = new Map([left!, right!].map((x) => [x.evidence_id, x]));
|
||||
const reference = (record: { evidence_id: string }) => ({
|
||||
kind: "retained-evidence" as const,
|
||||
evidence_id: record.evidence_id,
|
||||
});
|
||||
return {
|
||||
left: left!,
|
||||
right: right!,
|
||||
references: { left: reference(left!), right: reference(right!) },
|
||||
lookup: (id: string) => ledger.get(id),
|
||||
};
|
||||
};
|
||||
const ghidraRow = (side: "1" | "2", index: number) => ({
|
||||
address: `0x${(0x400000 + index * 16 + (side === "2" ? 8 : 0)).toString(16)}`,
|
||||
value: `subsystem_function_${index}`,
|
||||
symbol: {
|
||||
primary: true,
|
||||
dynamic: false,
|
||||
external: false,
|
||||
type: "Function",
|
||||
source: "imported",
|
||||
},
|
||||
procedure: { external: false, thunk: false, thunk_target: null },
|
||||
});
|
||||
|
||||
it("charges a borrowed retained inventory for its emitted members, not its resident payload", async () => {
|
||||
// One shared 40 KB string per row: ~40 KB resident, but a per-occurrence
|
||||
// walk would charge 1000 copies of it.
|
||||
const resident = "r".repeat(40_000);
|
||||
const { left, right, references, lookup } = retainedInventories((side) =>
|
||||
Array.from({ length: 1000 }, (_, i) => ({
|
||||
...ghidraRow(side, i),
|
||||
provider_note: resident,
|
||||
})),
|
||||
);
|
||||
const borrowed = await matchNativeFunctions(references, undefined, lookup);
|
||||
if (!borrowed.ok) throw borrowed.error;
|
||||
expect(
|
||||
nativeFunctionMatchingSchema.parse(
|
||||
borrowed.value.evidence.normalized_result,
|
||||
).summary.matched,
|
||||
).toBe(1000);
|
||||
// Inline Evidence is authenticated by cloning, so its whole payload is charged.
|
||||
const inline = await matchNativeFunctions({
|
||||
left: structuredClone(left),
|
||||
right: structuredClone(right),
|
||||
});
|
||||
expect(inline.ok).toBe(false);
|
||||
if (!inline.ok)
|
||||
expect(inline.error._tag).toBe("AnalysisResourceConstraintError");
|
||||
});
|
||||
|
||||
it("still charges a borrowed inventory's envelope, which matching copies into its result", async () => {
|
||||
// Limitations are prefixed into the report and Evidence; 25 distinct
|
||||
// 1 MiB entries are charged above the allowance although rows are few.
|
||||
const record = (side: "1" | "2", limitations: readonly string[]) =>
|
||||
immutableEvidence(
|
||||
createEvidence(nativeMatchingTarget(side), nativeMatchingProvider, {
|
||||
operation: "list_procedures",
|
||||
parameters: {},
|
||||
result: [ghidraRow(side, 0)],
|
||||
limitations: [...limitations],
|
||||
}),
|
||||
);
|
||||
const left = record(
|
||||
"1",
|
||||
Array.from({ length: 25 }, (_, i) => `${"l".repeat(1024 * 1024)}${i}`),
|
||||
),
|
||||
right = record("2", []);
|
||||
const ledger = new Map([left, right].map((x) => [x.evidence_id, x]));
|
||||
const refused = await matchNativeFunctions(
|
||||
{
|
||||
left: { kind: "retained-evidence", evidence_id: left.evidence_id },
|
||||
right: { kind: "retained-evidence", evidence_id: right.evidence_id },
|
||||
},
|
||||
undefined,
|
||||
(id) => ledger.get(id),
|
||||
);
|
||||
expect(refused.ok).toBe(false);
|
||||
if (!refused.ok)
|
||||
expect(refused.error._tag).toBe("AnalysisResourceConstraintError");
|
||||
});
|
||||
|
||||
it("matches 20,000 retained Ghidra-shaped rows per side and refuses an oversized side before parsing its rows", async () => {
|
||||
const large = retainedInventories((side) =>
|
||||
Array.from({ length: 20_000 }, (_, i) => ghidraRow(side, i)),
|
||||
);
|
||||
const matched = await matchNativeFunctions(
|
||||
large.references,
|
||||
undefined,
|
||||
large.lookup,
|
||||
);
|
||||
if (!matched.ok) throw matched.error;
|
||||
expect(
|
||||
nativeFunctionMatchingSchema.parse(matched.value.evidence.normalized_result)
|
||||
.summary,
|
||||
).toEqual({ matched: 20_000, only_left: 0, only_right: 0, unresolved: 0 });
|
||||
// A malformed final row would be an integrity failure if parsing began.
|
||||
const oversized = retainedInventories((side) => [
|
||||
...Array.from({ length: 50_000 }, (_, i) => ghidraRow(side, i)),
|
||||
{ address: "not an address", value: 1 },
|
||||
]);
|
||||
const refused = await matchNativeFunctions(
|
||||
oversized.references,
|
||||
undefined,
|
||||
oversized.lookup,
|
||||
);
|
||||
expect(refused.ok).toBe(false);
|
||||
if (!refused.ok)
|
||||
expect(refused.error).toMatchObject({
|
||||
_tag: "AnalysisResourceConstraintError",
|
||||
reportedLimits: { maximum_bytes: 96 * 1024 * 1024 },
|
||||
retryable: false,
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps duplicate candidate storage linear and emits every input occurrence exactly once", async () => {
|
||||
const entries = Array.from({ length: 1000 }, (_, i) => ({
|
||||
address: `0x${i.toString(16)}`,
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import { setImmediate } from "node:timers/promises";
|
||||
import { AnalysisResourceConstraintError } from "../../domain/analysisErrorCore.js";
|
||||
import { fixedNativeAllowance } from "../../domain/native/nativeDeadlines.js";
|
||||
import type { Evidence } from "../../domain/evidence.js";
|
||||
|
||||
const NATIVE_MATCHING_REPRESENTATION_BYTES = 96 * 1024 * 1024;
|
||||
|
||||
@@ -35,17 +34,6 @@ export class NativeMatchingAdmission {
|
||||
this.signal === undefined ? {} : { signal: this.signal },
|
||||
);
|
||||
};
|
||||
/**
|
||||
* Charge a borrowed record's envelope, which matching copies into its
|
||||
* limitations, digests and links. Its result payloads stay resident: the
|
||||
* caller charges what matching derives from them, or never parses them.
|
||||
*/
|
||||
async admitBorrowed(record: Evidence): Promise<void> {
|
||||
await this.checkpoint(128);
|
||||
for (const [key, value] of Object.entries(record))
|
||||
if (key !== "raw_result" && key !== "normalized_result")
|
||||
await this.admit(value);
|
||||
}
|
||||
async admit(root: unknown): Promise<void> {
|
||||
const pending: Iterator<unknown>[] = [[root][Symbol.iterator]()];
|
||||
while (pending.length) {
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { MatchNativeFunctionsInput } from "../../contracts/native/functionMatchingContracts.js";
|
||||
import { digestCanonicalValue } from "../../domain/canonicalDigest.js";
|
||||
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
|
||||
import { isImmutableEvidence, type Evidence } from "../../domain/evidence.js";
|
||||
import type { Evidence } from "../../domain/evidence.js";
|
||||
import { readEvidence } from "../../domain/evidenceValidation.js";
|
||||
import { retainedEvidenceReferenceSchema } from "../../domain/evidenceReference.js";
|
||||
import {
|
||||
@@ -34,30 +34,18 @@ export const resolveNativeMatchingSources = async (
|
||||
readonly sources: readonly Evidence[];
|
||||
}> => {
|
||||
const sources = new Map<string, Evidence>();
|
||||
/**
|
||||
* Charge only what matching copies. A retained immutable record is borrowed
|
||||
* without cloning. When matching never parses its result, or charges the
|
||||
* representation it derives from it (inventory rows), only its envelope is
|
||||
* charged; walking the result would charge resident memory as if matching
|
||||
* had allocated it.
|
||||
*/
|
||||
const resolve = async (
|
||||
input: unknown,
|
||||
path?: readonly (string | number)[],
|
||||
borrowable = false,
|
||||
): Promise<Evidence> => {
|
||||
const reference = retainedEvidenceReferenceSchema.safeParse(input);
|
||||
let raw = input,
|
||||
borrowed: Evidence | undefined;
|
||||
let raw = input;
|
||||
if (reference.success) {
|
||||
const found = resolveEvidenceInput(reference.data, lookup);
|
||||
if (!found.ok) throw found.error;
|
||||
raw = found.value;
|
||||
if (borrowable && isImmutableEvidence(found.value))
|
||||
borrowed = found.value;
|
||||
}
|
||||
if (borrowed !== undefined) await admission.admitBorrowed(borrowed);
|
||||
else await admission.admit(raw);
|
||||
await admission.admit(raw);
|
||||
const checked = readEvidence(raw);
|
||||
if (!checked.ok)
|
||||
throw path !== undefined && !reference.success
|
||||
@@ -78,9 +66,8 @@ export const resolveNativeMatchingSources = async (
|
||||
raw: unknown,
|
||||
path: readonly (string | number)[],
|
||||
operations: readonly string[],
|
||||
borrowable = false,
|
||||
): Promise<Evidence> => {
|
||||
const evidence = await resolve(raw, path, borrowable);
|
||||
const evidence = await resolve(raw, path);
|
||||
if (
|
||||
evidence.predicate_type !== "rea.analysis" ||
|
||||
!operations.includes(evidence.operation) ||
|
||||
@@ -95,14 +82,8 @@ export const resolveNativeMatchingSources = async (
|
||||
}
|
||||
return evidence;
|
||||
};
|
||||
// readFunctionInventory charges each inventory row before parsing it.
|
||||
const left = await select(input.left, ["left"], INVENTORY_OPERATIONS, true);
|
||||
const right = await select(
|
||||
input.right,
|
||||
["right"],
|
||||
INVENTORY_OPERATIONS,
|
||||
true,
|
||||
);
|
||||
const left = await select(input.left, ["left"], INVENTORY_OPERATIONS);
|
||||
const right = await select(input.right, ["right"], INVENTORY_OPERATIONS);
|
||||
const observations = async (
|
||||
inputs: readonly unknown[],
|
||||
field: "left_observations" | "right_observations",
|
||||
@@ -155,12 +136,7 @@ export const resolveNativeMatchingSources = async (
|
||||
for (const source of sources.values())
|
||||
for (const id of source.evidence_links)
|
||||
if (!sources.has(id))
|
||||
// Closure records are only returned for retention, never parsed.
|
||||
await resolve(
|
||||
{ kind: "retained-evidence", evidence_id: id },
|
||||
undefined,
|
||||
true,
|
||||
);
|
||||
await resolve({ kind: "retained-evidence", evidence_id: id });
|
||||
return {
|
||||
left,
|
||||
right,
|
||||
|
||||
@@ -31,8 +31,6 @@ import {
|
||||
type NativeDmgInfo,
|
||||
} from "./NativeDmgOwnership.js";
|
||||
const HDIUTIL_COMMAND_TIMEOUT_MS = 120_000;
|
||||
const DMG_CHECKSUM_ABSENT_LIMITATION =
|
||||
"hdiutil verify found no stored checksum in this disk image, so its integrity was not verified; the root SHA-256 identifies the inventoried bytes.";
|
||||
const VERIFY_SETTLE_DELAYS_MS = [250, 500] as const;
|
||||
// APFS ejects and busy reporting can lag a detach attempt.
|
||||
const DETACH_SETTLE_DELAYS_MS = [250, 500, 1000] as const;
|
||||
@@ -62,15 +60,12 @@ interface NativeDmgFailureClassification {
|
||||
* localized text; the cause stays unknown unless the target cannot be opened.
|
||||
*/
|
||||
readonly unrecognizedImage?: boolean;
|
||||
/** hdiutil found no stored checksum, so verification neither passed nor failed. */
|
||||
readonly checksumAbsent?: boolean;
|
||||
}
|
||||
|
||||
class NativeDmgCommandFailure extends ArtifactReaderFailure {
|
||||
readonly processNotStarted: boolean;
|
||||
readonly verifyDiagnostic: string | undefined;
|
||||
readonly unrecognizedImage: boolean;
|
||||
readonly checksumAbsent: boolean;
|
||||
readonly exitCode: number | null;
|
||||
|
||||
constructor(
|
||||
@@ -99,7 +94,6 @@ class NativeDmgCommandFailure extends ArtifactReaderFailure {
|
||||
this.processNotStarted = options?.processNotStarted ?? false;
|
||||
this.verifyDiagnostic = failure.verifyDiagnostic;
|
||||
this.unrecognizedImage = failure.unrecognizedImage ?? false;
|
||||
this.checksumAbsent = failure.checksumAbsent ?? false;
|
||||
this.exitCode = options?.exitCode ?? null;
|
||||
}
|
||||
}
|
||||
@@ -168,7 +162,6 @@ export class NativeDmgArtifactReader implements ArtifactReader {
|
||||
#closePromise: Promise<void> | undefined;
|
||||
readonly #activeDirectoryOperations = new Set<Promise<unknown>>();
|
||||
#verificationProbeHandle: OwnedFileHandle | undefined;
|
||||
#checksumAbsent = false;
|
||||
|
||||
constructor(
|
||||
private readonly path: string,
|
||||
@@ -191,11 +184,6 @@ export class NativeDmgArtifactReader implements ArtifactReader {
|
||||
const directory = this.#directory;
|
||||
if (directory === undefined)
|
||||
throw new ArtifactReaderFailure("unavailable", "DMG is not attached");
|
||||
if (this.#checksumAbsent)
|
||||
options?.onCoverage?.({
|
||||
status: "complete",
|
||||
limitations: [DMG_CHECKSUM_ABSENT_LIMITATION],
|
||||
});
|
||||
const prefix = basename(this.path);
|
||||
const entries = directory
|
||||
.entries(signal, {
|
||||
@@ -510,11 +498,6 @@ export class NativeDmgArtifactReader implements ArtifactReader {
|
||||
} catch (cause: unknown) {
|
||||
if (!(cause instanceof NativeDmgCommandFailure)) throw cause;
|
||||
this.#provenance.push(command(arguments_, ["read"], cause.exitCode));
|
||||
// No stored checksum (e.g. raw UDRW): attach with integrity unverified.
|
||||
if (cause.checksumAbsent) {
|
||||
this.#checksumAbsent = true;
|
||||
return;
|
||||
}
|
||||
if (cause.unrecognizedImage) {
|
||||
const unreadable = await this.#isUnreadableTarget(signal, cause);
|
||||
this.#throwIfVerificationCancelled(signal);
|
||||
@@ -869,7 +852,6 @@ const verifyFailureDiagnostic = (
|
||||
|
||||
// hdiutil localizes the trailing failure reason but not its own status lines.
|
||||
const INVALID_CHECKSUM_LINE = /^hdiutil: verify: checksum of ".*" is INVALID$/u;
|
||||
const NO_CHECKSUM_LINE = /^hdiutil: verify: ".*" has no checksum\.$/u;
|
||||
const UNRECOGNIZED_IMAGE_LINE =
|
||||
/^hdiutil: verify: unable to recognize ".*" as a disk image\./u;
|
||||
|
||||
@@ -880,8 +862,6 @@ const untranslatedVerifyFailure = (
|
||||
const lines = outputLines(stdout, stderr);
|
||||
if (lines.some((line) => INVALID_CHECKSUM_LINE.test(line)))
|
||||
return { reason: "integrity" };
|
||||
if (lines.some((line) => NO_CHECKSUM_LINE.test(line)))
|
||||
return { reason: "unavailable", checksumAbsent: true };
|
||||
// Malformed, unreadable and transiently busy targets all produce this line;
|
||||
// only the localized reason distinguishes them, so the cause stays unknown.
|
||||
if (lines.some((line) => UNRECOGNIZED_IMAGE_LINE.test(line)))
|
||||
|
||||
@@ -29,7 +29,6 @@ import {
|
||||
} from "./hashStableRootArtifact.js";
|
||||
import type { ArtifactInventoryOptions } from "./types.js";
|
||||
import { readMachoHeaderPrefixFacts } from "../../domain/apple/machoContainer.js";
|
||||
import { hasDiskImageSignature } from "../../domain/apple/diskImageFormat.js";
|
||||
import {
|
||||
observeMzWindowsHeader,
|
||||
type MzWindowsHeaderSignatureEvidence,
|
||||
@@ -153,27 +152,9 @@ const classifyRootFormat = async (
|
||||
evidence.prefix.subarray(0, 4).equals(Buffer.from("xar!"))
|
||||
)
|
||||
return "pkg";
|
||||
// Fill each range: a short result is end of file, not a partial host read.
|
||||
const read = async (position: number, length: number): Promise<Buffer> => {
|
||||
const bytes = Buffer.alloc(length);
|
||||
let filled = 0;
|
||||
while (filled < length) {
|
||||
abortIfNeeded(signal);
|
||||
const observed = await handle.read(
|
||||
bytes,
|
||||
filled,
|
||||
length - filled,
|
||||
position + filled,
|
||||
);
|
||||
abortIfNeeded(signal);
|
||||
if (observed.bytesRead === 0) break;
|
||||
filled += observed.bytesRead;
|
||||
}
|
||||
return bytes.subarray(0, filled);
|
||||
};
|
||||
if (
|
||||
lower.endsWith(".dmg") &&
|
||||
(await hasDiskImageSignature(archiveBytes, read))
|
||||
(await hasKolyTrailer(handle, archiveBytes, signal))
|
||||
)
|
||||
return "dmg";
|
||||
if (await locateGodotPck(handle, archiveBytes, signal, evidence.prefix))
|
||||
@@ -199,7 +180,13 @@ const classifyRootFormat = async (
|
||||
if (
|
||||
(format === "file" || format === "unknown") &&
|
||||
!claimsExecutableHeader(evidence.prefix, archiveBytes) &&
|
||||
(await locateTrailingZipArchive(archiveBytes, read)) !== undefined
|
||||
(await locateTrailingZipArchive(archiveBytes, async (position, length) => {
|
||||
abortIfNeeded(signal);
|
||||
const bytes = Buffer.alloc(length);
|
||||
const read = await handle.read(bytes, 0, length, position);
|
||||
abortIfNeeded(signal);
|
||||
return bytes.subarray(0, read.bytesRead);
|
||||
})) !== undefined
|
||||
)
|
||||
return zipRootPackageFormat(lower);
|
||||
if (["elf", "pe", "mach-o", "mach-o-universal"].includes(format)) {
|
||||
@@ -318,3 +305,26 @@ const readRootMzWindowsHeaderSignature = async (
|
||||
if (available === 0) return undefined;
|
||||
return { offset, bytes: window.subarray(0, available) };
|
||||
};
|
||||
|
||||
const hasKolyTrailer = async (
|
||||
handle: FileHandle,
|
||||
size: number,
|
||||
signal?: AbortSignal,
|
||||
): Promise<boolean> => {
|
||||
if (size < 512) return false;
|
||||
const trailer = Buffer.alloc(4);
|
||||
let bytes = 0;
|
||||
while (bytes < trailer.length) {
|
||||
abortIfNeeded(signal);
|
||||
const read = await handle.read(
|
||||
trailer,
|
||||
bytes,
|
||||
trailer.length - bytes,
|
||||
size - 512 + bytes,
|
||||
);
|
||||
abortIfNeeded(signal);
|
||||
if (read.bytesRead === 0) return false;
|
||||
bytes += read.bytesRead;
|
||||
}
|
||||
return trailer.equals(Buffer.from("koly"));
|
||||
};
|
||||
@@ -73,25 +73,6 @@ it("retains allowed whitespace, visible text, and valid UTF-8 obs-text", async (
|
||||
expect(result.examined_bytes).toBe(5 + Buffer.byteLength(text));
|
||||
});
|
||||
|
||||
it("excludes trailing whitespace from trailer field values", async () => {
|
||||
const text =
|
||||
"grpc-status: 0 \r\ngrpc-message: alpha\tbeta \t\r\nx-empty: \t\r\n";
|
||||
const result = await decodeProtobufBody(
|
||||
trailerFrame(text),
|
||||
"grpc-web",
|
||||
source,
|
||||
same,
|
||||
new ProtobufBudget(),
|
||||
);
|
||||
expect(result.status).toBe("complete");
|
||||
expect(result.frames[0]?.trailers).toEqual([
|
||||
{ name: "grpc-status", value: "0" },
|
||||
{ name: "grpc-message", value: "alpha\tbeta" },
|
||||
{ name: "x-empty", value: "" },
|
||||
]);
|
||||
expect(result.examined_bytes).toBe(5 + Buffer.byteLength(text));
|
||||
});
|
||||
|
||||
it.each(["grpc-web", "grpc-web-text"] as const)(
|
||||
"keeps valid trailer fields around an opaque non-UTF-8 value in %s framing",
|
||||
async (framing) => {
|
||||
|
||||
@@ -84,13 +84,9 @@ export const parseGrpcWebTrailers = (
|
||||
"gRPC-web trailer field value contains a forbidden control character",
|
||||
});
|
||||
}
|
||||
// RFC 9110 section 5.5: a field value excludes surrounding whitespace.
|
||||
let valueEnd = lineEnd;
|
||||
while (valueEnd > valueStart && [9, 32].includes(bytes[valueEnd - 1]!))
|
||||
valueEnd -= 1;
|
||||
let value: string;
|
||||
try {
|
||||
value = decoder.decode(bytes.subarray(valueStart, valueEnd));
|
||||
value = decoder.decode(bytes.subarray(valueStart, lineEnd));
|
||||
} catch (cause) {
|
||||
if (!(cause instanceof TypeError)) throw cause;
|
||||
decoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
|
||||
|
||||
@@ -43,27 +43,13 @@ const isObjectArray = (value: unknown): value is Record<string, unknown>[] =>
|
||||
) &&
|
||||
value.some((item) => Object.keys(item).length > 0);
|
||||
|
||||
const isWordCharacter = (character: string | undefined): boolean =>
|
||||
character !== undefined && /[\p{L}\p{N}]/u.test(character);
|
||||
|
||||
/** Underscore runs between word characters cannot open or close emphasis. */
|
||||
const escapeInlineSyntax = (
|
||||
match: string,
|
||||
offset: number,
|
||||
text: string,
|
||||
): string =>
|
||||
match.startsWith("_") &&
|
||||
isWordCharacter(text[offset - 1]) &&
|
||||
isWordCharacter(text[offset + match.length])
|
||||
? match
|
||||
: match.replace(/./g, "\\$&");
|
||||
|
||||
const escapeTableCell = (value: string): string =>
|
||||
value
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/[\\|`*[\]~]|_+/g, escapeInlineSyntax)
|
||||
.replace(/\\/g, "\\\\")
|
||||
.replace(/\|/g, "\\|")
|
||||
.replace(/\r\n|\r|\n/g, "<br>");
|
||||
|
||||
const renderTable = (
|
||||
|
||||
@@ -337,7 +337,6 @@ export const createCliResultOutput = (
|
||||
...options,
|
||||
source: value,
|
||||
selected: filtered,
|
||||
filterPaths,
|
||||
operationFailure,
|
||||
command: metadata.command,
|
||||
duration: metadata.duration,
|
||||
|
||||
@@ -21,7 +21,6 @@ export const finishAnalysisCommand = async <
|
||||
canonicalOperation: string,
|
||||
signal: AbortSignal,
|
||||
fallbackResource?: string,
|
||||
options: { readonly failureProjection?: "initial" | "final" } = {},
|
||||
): Promise<Value | ReturnType<typeof projectAnalysisError>> => {
|
||||
let result: Result<Value, AnalysisError>;
|
||||
let cleanup: AnalysisError | undefined;
|
||||
@@ -36,12 +35,6 @@ export const finishAnalysisCommand = async <
|
||||
: new ProviderAdapterError(providerId, canonicalOperation, { cause }),
|
||||
};
|
||||
}
|
||||
// Some one-shot commands promise the original failure snapshot even when
|
||||
// close retries and mutates the producer's live cleanup status.
|
||||
const initialFailure =
|
||||
!result.ok && options.failureProjection === "initial"
|
||||
? projectAnalysisError(result.error)
|
||||
: undefined;
|
||||
try {
|
||||
// Resource owners may resolve their retained errors during close. This
|
||||
// adapter cannot infer resolution of a producer's earlier cleanup failure.
|
||||
@@ -68,7 +61,6 @@ export const finishAnalysisCommand = async <
|
||||
}
|
||||
}
|
||||
await dispatchQueuedInterrupts();
|
||||
if (initialFailure !== undefined) return initialFailure;
|
||||
if (result.ok && signal.aborted)
|
||||
result = {
|
||||
ok: false,
|
||||
|
||||
+67
-215
@@ -1,8 +1,6 @@
|
||||
import type { Filter } from "incur";
|
||||
|
||||
import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
|
||||
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
|
||||
import { jsonParts } from "../domain/jsonSerialization.js";
|
||||
|
||||
/** Reserved control-record marker, not an advertised external JSON Schema. */
|
||||
@@ -29,21 +27,6 @@ interface JsonLinesMetadataRecord {
|
||||
|
||||
const COLLECTION_KEYS = ["rows", "items", "records", "occurrences"] as const;
|
||||
|
||||
const RECORD_BOUNDARIES =
|
||||
"JSON Lines needs a top-level array, or exactly one rows, items, records, or occurrences array directly in the result, under page, or under normalized_result.";
|
||||
|
||||
/** Objects whose collection fields JSON Lines admits, relative to the result. */
|
||||
const RECORD_BOUNDARY_PATHS = [
|
||||
[],
|
||||
["page"],
|
||||
["normalized_result"],
|
||||
["normalized_result", "page"],
|
||||
] as const;
|
||||
|
||||
/** Recovery for every JSON Lines selection refusal; the analysis result itself is valid. */
|
||||
export const JSON_LINES_SELECTION_REMEDIATION =
|
||||
"Use --format json for the complete document, or select one record collection with --filter-output.";
|
||||
|
||||
interface Collection {
|
||||
readonly records: readonly unknown[];
|
||||
readonly path: readonly string[];
|
||||
@@ -67,100 +50,42 @@ const atPath = (value: unknown, path: readonly string[]): unknown => {
|
||||
return value;
|
||||
};
|
||||
|
||||
interface BoundaryExamination {
|
||||
readonly found: readonly Collection[];
|
||||
/** Collection fields that exist but are not arrays. */
|
||||
readonly rejected: readonly Observation[];
|
||||
}
|
||||
const collections = (
|
||||
value: unknown,
|
||||
prefix: readonly string[],
|
||||
): Collection[] => {
|
||||
if (Array.isArray(value)) return [{ records: value, path: prefix }];
|
||||
if (!isRecord(value)) return [];
|
||||
const found: Collection[] = [];
|
||||
const collect = (
|
||||
object: Record<string, unknown>,
|
||||
path: readonly string[],
|
||||
) => {
|
||||
for (const key of COLLECTION_KEYS) {
|
||||
const records = object[key];
|
||||
if (Object.hasOwn(object, key) && Array.isArray(records))
|
||||
found.push({ records, path: [...path, key] });
|
||||
}
|
||||
};
|
||||
collect(value, prefix);
|
||||
if (Object.hasOwn(value, "page") && isRecord(value.page))
|
||||
collect(value.page, [...prefix, "page"]);
|
||||
return found;
|
||||
};
|
||||
|
||||
/** Examine exactly the documented record boundaries, once, for admission and refusal. */
|
||||
const examineBoundaries = (
|
||||
const documentCollections = (
|
||||
value: unknown,
|
||||
fullOutput: boolean,
|
||||
): BoundaryExamination => {
|
||||
): Collection[] => {
|
||||
const prefix = fullOutput ? ["data"] : [];
|
||||
const data = atPath(value, prefix);
|
||||
const found: Collection[] = [];
|
||||
const rejected: Observation[] = [];
|
||||
for (const boundary of RECORD_BOUNDARY_PATHS) {
|
||||
const object = atPath(data, boundary);
|
||||
const path = [...prefix, ...boundary];
|
||||
// A result or normalized_result may itself be the array; a page is a container.
|
||||
if (Array.isArray(object) && boundary.at(-1) !== "page")
|
||||
found.push({ records: object, path });
|
||||
if (!isRecord(object)) continue;
|
||||
for (const key of COLLECTION_KEYS) {
|
||||
if (!Object.hasOwn(object, key)) continue;
|
||||
const records = object[key];
|
||||
if (Array.isArray(records)) found.push({ records, path: [...path, key] });
|
||||
else rejected.push(observe([...path, key], records));
|
||||
}
|
||||
}
|
||||
return { found, rejected };
|
||||
};
|
||||
|
||||
type Shape =
|
||||
| "absent"
|
||||
| "null"
|
||||
| "array"
|
||||
| "object"
|
||||
| "string"
|
||||
| "number"
|
||||
| "boolean";
|
||||
|
||||
/** One examined value that JSON Lines refused, at its emitted-document path. */
|
||||
interface Observation {
|
||||
readonly path: readonly (string | number)[];
|
||||
readonly shape: Shape;
|
||||
}
|
||||
|
||||
const shapeOf = (value: unknown): Shape => {
|
||||
if (value === null) return "null";
|
||||
if (Array.isArray(value)) return "array";
|
||||
switch (typeof value) {
|
||||
case "object":
|
||||
return "object";
|
||||
case "string":
|
||||
return "string";
|
||||
case "boolean":
|
||||
return "boolean";
|
||||
case "number":
|
||||
case "bigint":
|
||||
return "number";
|
||||
default:
|
||||
// JSON omits undefined, functions and symbols.
|
||||
return "absent";
|
||||
}
|
||||
};
|
||||
|
||||
const observe = (
|
||||
path: readonly (string | number)[],
|
||||
value: unknown,
|
||||
): Observation => ({ path, shape: shapeOf(value) });
|
||||
|
||||
const describe = ({ path, shape }: Observation): string =>
|
||||
`${path.length === 0 ? "the result" : path.join(".")} is ${
|
||||
shape === "absent" || shape === "null"
|
||||
? shape
|
||||
: shape === "array" || shape === "object"
|
||||
? `an ${shape}`
|
||||
: `a ${shape}`
|
||||
}`;
|
||||
|
||||
/** Caller-visible key paths, before any slice, selected by --filter-output. */
|
||||
const selectionPaths = (
|
||||
paths: ReturnType<typeof Filter.parse> | undefined,
|
||||
): string[][] => {
|
||||
const selected = new Map<string, string[]>();
|
||||
for (const path of paths ?? []) {
|
||||
const keys: string[] = [];
|
||||
for (const segment of path) {
|
||||
if (!("key" in segment)) break;
|
||||
keys.push(segment.key);
|
||||
}
|
||||
selected.set(JSON.stringify(keys), keys);
|
||||
}
|
||||
return [...selected.values()];
|
||||
return [
|
||||
...collections(data, prefix),
|
||||
...collections(atPath(data, ["normalized_result"]), [
|
||||
...prefix,
|
||||
"normalized_result",
|
||||
]),
|
||||
];
|
||||
};
|
||||
|
||||
/** Shallow copies only ancestors of the emitted array; every other fact survives. */
|
||||
@@ -198,116 +123,40 @@ const withoutRecords = (
|
||||
return context;
|
||||
};
|
||||
|
||||
interface JsonLinesOptions {
|
||||
readonly command: string;
|
||||
readonly source?: unknown;
|
||||
readonly filter?: string;
|
||||
/** Parsed --filter-output paths, used only to name the refused selection. */
|
||||
readonly filterPaths?: ReturnType<typeof Filter.parse>;
|
||||
readonly fullOutput?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Choose the one clear collection, or refuse the caller's selection. The
|
||||
* structured `observed` facts are the contract; the message derives from them.
|
||||
*/
|
||||
const selectCollection = (
|
||||
selected: unknown,
|
||||
source: unknown,
|
||||
options: JsonLinesOptions,
|
||||
{ found, rejected }: BoundaryExamination,
|
||||
): Result<Collection, AnalysisInputError> => {
|
||||
const prefix = options.fullOutput === true ? ["data"] : [];
|
||||
const filtered = options.filter !== undefined;
|
||||
const selections = selectionPaths(options.filterPaths).filter(
|
||||
(path) => path.length > 0,
|
||||
);
|
||||
const subject =
|
||||
filtered && selections.length > 0
|
||||
? `The --filter-output selection ${selections.map((path) => path.join(".")).join(", ")}`
|
||||
: "The command result";
|
||||
const refuse = (
|
||||
statement: string,
|
||||
observed: readonly Observation[],
|
||||
rule: string,
|
||||
) => {
|
||||
const observation = `${subject} ${statement}: ${observed.map(describe).join("; ")}.`;
|
||||
return err(
|
||||
new AnalysisInputError(
|
||||
options.command,
|
||||
{
|
||||
userMessage: `${observation} ${rule}`,
|
||||
remediationAction: JSON_LINES_SELECTION_REMEDIATION,
|
||||
},
|
||||
[
|
||||
{
|
||||
path: [filtered ? "filter-output" : "format"],
|
||||
reason: "invalid_value",
|
||||
message: observation,
|
||||
observed: observed.map(({ path, shape }) => ({
|
||||
path: [...path],
|
||||
shape,
|
||||
})),
|
||||
},
|
||||
],
|
||||
),
|
||||
);
|
||||
};
|
||||
if (found.length > 1)
|
||||
return refuse(
|
||||
"is not one JSON Lines record collection",
|
||||
found.map(({ path }) => observe(path, [])),
|
||||
"Select one with --filter-output.",
|
||||
);
|
||||
const [collection] = found;
|
||||
if (collection === undefined) {
|
||||
const [selection] = selections;
|
||||
// A non-array collection field is the most specific fact. Otherwise one
|
||||
// named selection is described by its source value, since a filter that
|
||||
// matches nothing leaves no selected value to describe.
|
||||
const observed =
|
||||
rejected.length > 0
|
||||
? rejected
|
||||
: [
|
||||
selections.length === 1 && selection !== undefined
|
||||
? observe(
|
||||
[...prefix, ...selection],
|
||||
atPath(atPath(source, prefix), selection),
|
||||
)
|
||||
: observe(prefix, atPath(selected, prefix)),
|
||||
];
|
||||
return refuse(
|
||||
"is not a JSON Lines record collection",
|
||||
observed,
|
||||
RECORD_BOUNDARIES,
|
||||
);
|
||||
}
|
||||
for (const [index, record] of collection.records.entries()) {
|
||||
if (isRecord(record) && Object.hasOwn(record, "$rea"))
|
||||
return refuse(
|
||||
"uses the field JSON Lines reserves for control records",
|
||||
[observe([...collection.path, index, "$rea"], record.$rea)],
|
||||
"Nested $rea values are ordinary data.",
|
||||
);
|
||||
}
|
||||
return ok(collection);
|
||||
};
|
||||
|
||||
/**
|
||||
* Project a clear collection without interpreting or whitelisting its context.
|
||||
* A refusal is the caller's output selection, not a failed analysis: the same
|
||||
* result remains available through --format json.
|
||||
*/
|
||||
/** Project a clear collection without interpreting or whitelisting its context. */
|
||||
export const prepareJsonLines = (
|
||||
selected: unknown,
|
||||
options: JsonLinesOptions,
|
||||
): Result<JsonLinesPlan, AnalysisInputError> => {
|
||||
const fullOutput = options.fullOutput === true;
|
||||
const examination = examineBoundaries(selected, fullOutput);
|
||||
options: {
|
||||
readonly command: string;
|
||||
readonly source?: unknown;
|
||||
readonly filter?: string;
|
||||
readonly fullOutput?: boolean;
|
||||
},
|
||||
): Result<JsonLinesPlan, AnalysisOutputError> => {
|
||||
const found = documentCollections(selected, options.fullOutput === true);
|
||||
const reject = (reason: string) =>
|
||||
err(
|
||||
new AnalysisOutputError(options.command, reason, {
|
||||
remediationAction:
|
||||
"Use --format json for the complete document, or select one record collection with --filter-output.",
|
||||
}),
|
||||
);
|
||||
if (found.length !== 1)
|
||||
return reject(
|
||||
found.length === 0
|
||||
? "JSON Lines requires an array or one explicit rows, items, records, or occurrences collection."
|
||||
: "JSON Lines cannot choose between multiple record collections. Select one collection explicitly.",
|
||||
);
|
||||
const collection = found[0];
|
||||
if (collection === undefined)
|
||||
return reject("The record collection is unavailable.");
|
||||
for (const record of collection.records) {
|
||||
if (isRecord(record) && Object.hasOwn(record, "$rea"))
|
||||
return reject(
|
||||
"A data record uses the reserved $rea JSON Lines control marker.",
|
||||
);
|
||||
}
|
||||
const source = Object.hasOwn(options, "source") ? options.source : selected;
|
||||
const chosen = selectCollection(selected, source, options, examination);
|
||||
if (!chosen.ok) return chosen;
|
||||
const collection = chosen.value;
|
||||
const originalRecords = atPath(source, collection.path);
|
||||
const sourceMatches = Array.isArray(originalRecords);
|
||||
let context = withoutRecords(
|
||||
@@ -319,7 +168,10 @@ export const prepareJsonLines = (
|
||||
if (options.filter !== undefined && sourceMatches) {
|
||||
// Excluded peer rows are data, not context. A filter can disambiguate them
|
||||
// without restoring an unselected collection into the final record.
|
||||
for (const peer of examineBoundaries(source, fullOutput).found) {
|
||||
for (const peer of documentCollections(
|
||||
source,
|
||||
options.fullOutput === true,
|
||||
)) {
|
||||
if (peer.path.join(".") !== collection.path.join("."))
|
||||
context = withoutRecords(context, selected, peer.path, true);
|
||||
}
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import { constants } from "node:buffer";
|
||||
import type { Writable } from "node:stream";
|
||||
import type { Filter } from "incur";
|
||||
import { estimateTokenCount } from "tokenx";
|
||||
|
||||
import { AnalysisResourceConstraintError } from "../domain/analysisErrorCore.js";
|
||||
@@ -30,7 +29,6 @@ interface CliJsonLinesInput {
|
||||
readonly fullOutput: boolean;
|
||||
readonly tokenCount: boolean;
|
||||
readonly filterOutput: string | undefined;
|
||||
readonly filterPaths: ReturnType<typeof Filter.parse> | undefined;
|
||||
}
|
||||
|
||||
/** Render one already classified result, sharing a plan for output and counting. */
|
||||
@@ -59,9 +57,6 @@ export const writeCliJsonLines = async (
|
||||
...(options.filterOutput === undefined
|
||||
? {}
|
||||
: { filter: options.filterOutput }),
|
||||
...(options.filterPaths === undefined
|
||||
? {}
|
||||
: { filterPaths: options.filterPaths }),
|
||||
fullOutput: options.fullOutput,
|
||||
})
|
||||
: undefined;
|
||||
|
||||
@@ -2,17 +2,15 @@ import { Cli } from "incur";
|
||||
import { z } from "zod";
|
||||
|
||||
import { captureBrowserScenario } from "./application/BrowserScenarioCaptureService.js";
|
||||
import { finishAnalysisCommand } from "./cli/finishAnalysisCommand.js";
|
||||
import { withCommandCancellation } from "./cli/commandCancellation.js";
|
||||
import {
|
||||
analysisEvidenceDeliveryOptions,
|
||||
cliEvidenceDeliveryInputError,
|
||||
publishCliAnalysisEvidence,
|
||||
} from "./cli/analysisEvidenceDelivery.js";
|
||||
import { CLI_COMMANDS } from "./cliCommandNames.js";
|
||||
import { parseCliJsonInput, resolveCliJsonPaths } from "./cliJsonInput.js";
|
||||
import { logCliCommand } from "./cliLogging.js";
|
||||
import { projectAnalysisError } from "./domain/analysisErrorProjection.js";
|
||||
import {
|
||||
BrowserObservationError,
|
||||
browserCleanupObservation,
|
||||
} from "./domain/browserObservationError.js";
|
||||
import { analysisInputErrorFromIssues } from "./domain/inputIssueProjection.js";
|
||||
import { browserScenarioSchema } from "./domain/browserScenario.js";
|
||||
import type { BrowserScenarioCapturePort } from "./application/BrowserScenarioCapturePort.js";
|
||||
@@ -40,16 +38,9 @@ export const registerBrowserScenarioCommands = (
|
||||
.string()
|
||||
.describe("Inline browser scenario JSON or JSON file path"),
|
||||
}),
|
||||
options: z.object(analysisEvidenceDeliveryOptions),
|
||||
run: ({ args, options: delivery }) =>
|
||||
run: ({ args }) =>
|
||||
withCommandCancellation((signal) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.captureBrowserScenario, async () => {
|
||||
const deliveryError = cliEvidenceDeliveryInputError({
|
||||
detail: delivery.detail,
|
||||
evidenceOutput: delivery.evidence_output,
|
||||
operation: OPERATION,
|
||||
});
|
||||
if (deliveryError !== undefined) return cliError(deliveryError);
|
||||
const input = await parseCliJsonInput(
|
||||
args.inputJson,
|
||||
OPERATION,
|
||||
@@ -76,28 +67,28 @@ export const registerBrowserScenarioCommands = (
|
||||
createBrowserScenarioProvider(environment),
|
||||
)
|
||||
: options.createProvider(environment);
|
||||
const result = await finishAnalysisCommand(
|
||||
() => captureBrowserScenario(provider, scenario.data, { signal }),
|
||||
{
|
||||
close: async () => {
|
||||
await provider?.close?.();
|
||||
},
|
||||
},
|
||||
"rea-playwright-browser-scenario",
|
||||
OPERATION,
|
||||
signal,
|
||||
undefined,
|
||||
{ failureProjection: "initial" },
|
||||
);
|
||||
if (!("evidence_id" in result))
|
||||
return { error: "Browser scenario capture failed", ...result };
|
||||
return publishCliAnalysisEvidence(result, {
|
||||
providerId: "rea-playwright-browser-scenario",
|
||||
operation: OPERATION,
|
||||
signal,
|
||||
detail: delivery.detail,
|
||||
evidenceOutput: delivery.evidence_output,
|
||||
});
|
||||
let result:
|
||||
| Awaited<ReturnType<typeof captureBrowserScenario>>
|
||||
| undefined;
|
||||
try {
|
||||
result = await captureBrowserScenario(provider, scenario.data, {
|
||||
signal,
|
||||
});
|
||||
return result.ok ? result.value : cliError(result.error);
|
||||
} finally {
|
||||
try {
|
||||
await provider?.close?.();
|
||||
} catch (cause: unknown) {
|
||||
// Failed captures already retain their primary error and partial
|
||||
// observations. A cleanup retry must not replace that outcome.
|
||||
if (result?.ok)
|
||||
throw new BrowserObservationError(OPERATION, "cleanup_failed", {
|
||||
cause,
|
||||
cleanup: browserCleanupObservation(cause),
|
||||
partialObservation: result.value,
|
||||
});
|
||||
}
|
||||
}
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
@@ -13,12 +13,10 @@ export interface GhidraLanguageOverride {
|
||||
|
||||
// Ghidra language IDs are four colon-separated fields; variants may contain
|
||||
// spaces ("x86:LE:16:Real Mode") but never leading or trailing whitespace.
|
||||
// Stock .ldefs also use "/" in processors ("M16C/60") and LEBE for ARM BE8
|
||||
// (big-endian data, little-endian instructions).
|
||||
export const ghidraLanguageIdSchema = z
|
||||
.string()
|
||||
.regex(
|
||||
/^[A-Za-z0-9_.+-]+(?:\/[A-Za-z0-9_.+-]+)*:(?:LE|BE|LEBE):[1-9]\d*:[A-Za-z0-9_.+-](?:[A-Za-z0-9_.+ -]*[A-Za-z0-9_.+-])?$/u,
|
||||
/^[A-Za-z0-9_.+-]+:(?:LE|BE):[1-9]\d*:[A-Za-z0-9_.+-](?:[A-Za-z0-9_.+ -]*[A-Za-z0-9_.+-])?$/u,
|
||||
"must be a Ghidra language ID such as x86:LE:32:default",
|
||||
);
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
|
||||
...toolContractMetadata("inspect_analysis_view"),
|
||||
kind: "application",
|
||||
description:
|
||||
'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Native export tables use kind native-exports, collection exports/issues, exact names or original indices, selected fields and matching-row pagination; parent Evidence plus source_index identifies each row. Source coverage stays separate from page coverage. BEAM views use kind beam, collections atoms/exports/imports/lambdas/chunks, exact names/indices/fields and matching-row pages; import modules intersects function names. source_index identifies original parent-and-collection rows, preserving every arity and raw atom index. beam.counts preserves null missing tables versus decoded empty tables; runtime absence remains unknown. PYC views use kind python-pyc, collection code_objects, names/indices/fields and paging; source_index scopes every same-name alternative to the parent, nested_code keeps code-constant references, and pyc.coverage preserves source coverage. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent also supports summary and browser-scenario steps/events/transactions/text pages, preserving step_indices, event_kinds, transaction_ids and selected artifacts. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where viewLine truncated
|
||||
'Use javascript-routes collections to query route patterns, source-linked occurrences, source status, unknowns or summary counts from retained JavaScript application or web-bundle Evidence. Inspect selected views of supported completed Evidence without repeating analysis. Choose a view compatible with the parent operation described below. Command summary views preserve ordered GDB records or opaque objdump stdout/stderr once, source parameters, completion facts and parent identity. They do not establish current target state. Historical capture summaries return identity and counts without bodies or archive inventories. Capture views page retained records: view {kind: "capture", detail: "preview" | "complete", offset: 0, limit: 20}; preview selects recorded headline fields and binary descriptors, complete returns exact records with all sidecars. Offsets follow the selected parent record order, preserving original producer ordinals. Paging never reopens the source or reruns the decoder. Native export tables use kind native-exports, collection exports/issues, exact names or original indices, selected fields and matching-row pagination; parent Evidence plus source_index identifies each row. Source coverage stays separate from page coverage. BEAM views use kind beam, collections atoms/exports/imports/lambdas/chunks, exact names/indices/fields and matching-row pages; import modules intersects function names. source_index identifies original parent-and-collection rows, preserving every arity and raw atom index. beam.counts preserves null missing tables versus decoded empty tables; runtime absence remains unknown. PYC views use kind python-pyc, collection code_objects, names/indices/fields and paging; source_index scopes every same-name alternative to the parent, nested_code keeps code-constant references, and pyc.coverage preserves source coverage. Go metadata uses kind go, collection dependencies/settings, exact paths/keys or indices, fields and pagination. Row index refers to the original collection; empty selections return no rows, and dependency coverage remains unknown. Rust metadata uses kind rust, collection packages/edges, exact names/versions or original indices, selected fields, edge from_indices/to_indices and matching-row pagination. Embedded-record coverage is separate from unknown runtime dependencies. For inspect_pcap_capture parents, kind pcap pages endpoints, dns-queries, tls-sni, protocols, http-uris or unattributed-addresses. Select an exact query for dns-addresses, or a tshark/capinfos stream for UTF-16 diagnostics text. Page totals count retained facts; source coverage remains explicit. Qlog summaries keep profile/scope and counts; kind qlog pages traces, streams, loss-declarations, incomplete-records, contributions, ranges or final-sizes. Except traces, select trace_index; stream-specific facets also require exact stream_id and direction. Facts retain all-recorded-events-before-selection scope, and interval overlap or endpoint loss declarations do not prove retransmission or network loss. The api-surface view groups retained HAR requests by method, origin and format-rule path template; templated segments are inferences. It reads no headers or bodies and reports no query values. The environment-manifest view projects the same HAR requests on the primary origin (most JSON-like records) into a replayable tool surface: view {kind: "environment-manifest", template_rules: ["after-uid-literal", "opaque-token", "sibling-variance"], include_query_parameter_names: true}. Each tool has a deterministic name, a path template and parameters from the four format rules plus the selected opt-in rules, observed query parameter names, a value-free body shape for POST/PUT/PATCH, status and content-type sets and record ordinals; records are pointers into the parent, excluded groups carry a reason and three fixed unknowns are always stated. Templates, parameters and names are inferences; the view copies no header, cookie or body values and refuses parents inspected with sensitive values. A capture_browser_scenario parent that selected response bodies or header values projects the same way, one record per network transaction in request order; it relies on capture-time redaction of declared secrets instead of that refusal, and a transaction whose URL was redacted is left unrepresented. The environment-state view extracts, from the same parents, the initial state the recorded reads observed: view {kind: "environment-state", template_rules: [...]} with the rules of the manifest it accompanies. It reads response bodies recorded before the first related write and reports entity types with an identity field (proved by a path parameter or inferred from a field name), entities in collection order with per-view values where views differ, documents for non-parameterized reads that are not entities and have one initial request line (item-shaped paths stay recorded), and a render plan per read tool: item and list views rLine truncated
|
||||
inputSchema: inspectAnalysisViewInputSchema,
|
||||
outputSchema: evidenceResultOf(analysisViewResultSchema),
|
||||
examples: [
|
||||
|
||||
@@ -1,10 +1,6 @@
|
||||
import type { ToolContract } from "./toolContractTypes.js";
|
||||
import { toolContractMetadata } from "./toolEffects.js";
|
||||
import {
|
||||
analysisDetailOutputSchemaOf,
|
||||
analysisDetailSchema,
|
||||
} from "./analysisDetail.js";
|
||||
import { browserScenarioSummaryViewResultSchema } from "../domain/analysisView/analysisView.js";
|
||||
import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js";
|
||||
import { browserScenarioInputSchema } from "../domain/browserScenario.js";
|
||||
import { browserScenarioCaptureSchema } from "../domain/browserScenarioCapture.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
@@ -30,15 +26,10 @@ export const BROWSER_SCENARIO_TOOL_CONTRACTS = [
|
||||
name: "capture_browser_scenario",
|
||||
...toolContractMetadata("capture_browser_scenario"),
|
||||
description:
|
||||
"Run a controlled browser scenario when passive observation cannot exercise the application. Launch the selected executable or connect to a loopback CDP target, provide a start URL and actions, and select storage or artifacts when needed. A request action sends one HTTP request to the start origin through the page's own fetch, with header values from secrets, page cookies or local storage; write methods change application state. Environment, empty storage, a URL-only final capture, and measured event-retention budgets are defaults. capture.network selects request_body, response_body, and header_values independently; capture.limits lets callers raise or lower the event count, retained-allocation estimates, network transaction count, and network content budgets. Selected content is returned inline with transaction and event references, byte digests, and explicit unavailable states. Budget omissions identify affected event kinds and network transactions. Credentials and declared secrets are redacted. Agents can run their own local fixture servers to control application responses. Actions run until completion or cancellation unless timeout_ms is set. Default detail complete returns the capture inline. Detail summary retains complete Evidence and returns step outcomes, the final or first unsuccessful step’s accessibility tree, and event/network counts. Read exact steps with selected artifacts, UTF-16 text pages, transactions with opt-in content, or events filtered by step, kind and transaction, through inspect_analysis_view kind browser-scenario.",
|
||||
"Run a controlled browser scenario when passive observation cannot exercise the application. Launch the selected executable or connect to a loopback CDP target, provide a start URL and actions, and select storage or artifacts when needed. A request action sends one HTTP request to the start origin through the page's own fetch, with header values from secrets, page cookies or local storage; write methods change application state. Environment, empty storage, a URL-only final capture, and measured event-retention budgets are defaults. capture.network selects request_body, response_body, and header_values independently; capture.limits lets callers raise or lower the event count, retained-allocation estimates, network transaction count, and network content budgets. Selected content is returned inline with transaction and event references, byte digests, and explicit unavailable states. Budget omissions identify affected event kinds and network transactions. Credentials and declared secrets are redacted. Agents can run their own local fixture servers to control application responses. Actions run until completion or cancellation unless timeout_ms is set. Results are returned inline.",
|
||||
kind: "browser-provider",
|
||||
inputSchema: browserScenarioInputSchema.extend({
|
||||
detail: analysisDetailSchema,
|
||||
}),
|
||||
outputSchema: analysisDetailOutputSchemaOf(
|
||||
browserScenarioCaptureSchema,
|
||||
browserScenarioSummaryViewResultSchema,
|
||||
),
|
||||
inputSchema: browserScenarioInputSchema,
|
||||
outputSchema: evidenceResultOf(browserScenarioCaptureSchema),
|
||||
examples: [
|
||||
{ title: "Capture a deterministic browser scenario", input: example },
|
||||
],
|
||||
|
||||
@@ -410,27 +410,24 @@ describe("compact input schema budget and reduction", () => {
|
||||
expect(presentation).toBe("reduced");
|
||||
expect(schema.properties).toEqual({
|
||||
capture: { type: "object", description: "Complete capture." },
|
||||
observations: { type: "array", items: { type: "object" } },
|
||||
observations: {
|
||||
type: "array",
|
||||
items: {
|
||||
type: ["null", "boolean", "object", "array", "number", "string"],
|
||||
items: {},
|
||||
},
|
||||
},
|
||||
label: { type: ["string", "null"] },
|
||||
});
|
||||
const validate = new Ajv2020({ strict: false }).compile(schema);
|
||||
expect(validate({ capture: {}, observations: [], label: null })).toBe(true);
|
||||
const observation = {
|
||||
captured_url: "https://example.test/",
|
||||
captured_at: "2026-10-25T00:00:00Z",
|
||||
status_code: 200,
|
||||
};
|
||||
const canonicalObservations = new Ajv2020({ strict: false }).compile(
|
||||
original.$defs.observations,
|
||||
);
|
||||
expect(canonicalObservations([observation])).toBe(true);
|
||||
expect(canonicalObservations(["text"])).toBe(false);
|
||||
expect(
|
||||
validate({ capture: {}, observations: [observation], label: null }),
|
||||
validate({
|
||||
capture: {},
|
||||
observations: [[[null, true, 42, "nested"], { values: [[], [false]] }]],
|
||||
label: null,
|
||||
}),
|
||||
).toBe(true);
|
||||
expect(validate({ capture: {}, observations: ["text"], label: null })).toBe(
|
||||
false,
|
||||
);
|
||||
expect(validate({ capture: {}, observations: [], label: "capture" })).toBe(
|
||||
true,
|
||||
);
|
||||
@@ -700,174 +697,3 @@ it("includes restored root identity in the compact byte budget", () => {
|
||||
expect(serializedBytes(result.schema)).toBeLessThanOrEqual(512);
|
||||
expect(result.schema.properties).toEqual({ value: { type: "number" } });
|
||||
});
|
||||
|
||||
// An untyped primitive enum is gateway-incompatible, so these schemas take
|
||||
// the reduced projection independently of the budget.
|
||||
const reducedPresentation = (
|
||||
properties: Record<string, unknown>,
|
||||
$defs: Record<string, unknown> = {},
|
||||
) => {
|
||||
const canonical = {
|
||||
type: "object",
|
||||
$defs,
|
||||
properties: { mode: { enum: ["fast", "safe"] }, ...properties },
|
||||
};
|
||||
const result = compactAdvertisedInputSchema(canonical, 13 * 1024);
|
||||
expect(result.presentation).toBe("reduced");
|
||||
const { mode: _mode, ...advertised } = result.schema.properties as Record<
|
||||
string,
|
||||
unknown
|
||||
>;
|
||||
return {
|
||||
advertised,
|
||||
validateCanonical: new Ajv2020({ strict: false }).compile(canonical),
|
||||
validateAdvertised: new Ajv2020({ strict: false }).compile(result.schema),
|
||||
};
|
||||
};
|
||||
|
||||
const reducedArrayCases = [
|
||||
[
|
||||
"string elements",
|
||||
{ type: "array", items: { type: "string", minLength: 1 } },
|
||||
{ type: "array", items: { type: "string" } },
|
||||
[["a", "b"]],
|
||||
],
|
||||
[
|
||||
"object union elements",
|
||||
{
|
||||
type: "array",
|
||||
minItems: 1,
|
||||
items: {
|
||||
oneOf: [
|
||||
{
|
||||
type: "object",
|
||||
properties: { kind: { const: "click" } },
|
||||
required: ["kind"],
|
||||
},
|
||||
{
|
||||
type: "object",
|
||||
properties: { kind: { const: "type" } },
|
||||
required: ["kind"],
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
{ type: "array", items: { type: "object" } },
|
||||
[[{ kind: "click" }, { kind: "type" }]],
|
||||
],
|
||||
[
|
||||
"referenced elements",
|
||||
{ type: "array", items: { $ref: "#/$defs/ordinal" } },
|
||||
{ type: "array", items: { type: "integer" } },
|
||||
[[0, 7]],
|
||||
],
|
||||
[
|
||||
"nested array elements",
|
||||
{
|
||||
type: "array",
|
||||
items: { type: "array", items: { type: "number" } },
|
||||
},
|
||||
{ type: "array", items: { type: "array", items: { type: "number" } } },
|
||||
[[[1.5], []]],
|
||||
],
|
||||
[
|
||||
"mixed scalar and object elements",
|
||||
{
|
||||
type: "array",
|
||||
items: { anyOf: [{ type: "string" }, { type: "object" }] },
|
||||
},
|
||||
{ type: "array", items: { type: ["string", "object"] } },
|
||||
[["text", {}]],
|
||||
],
|
||||
[
|
||||
"elements that may be arrays or scalars",
|
||||
{
|
||||
type: "array",
|
||||
items: {
|
||||
anyOf: [
|
||||
{ type: "string" },
|
||||
{ type: "array", items: { type: "string" } },
|
||||
],
|
||||
},
|
||||
},
|
||||
{ type: "array", items: {} },
|
||||
[["text", ["nested"]]],
|
||||
],
|
||||
[
|
||||
"tuple elements",
|
||||
{
|
||||
type: "array",
|
||||
prefixItems: [{ type: "integer" }],
|
||||
items: { type: "string" },
|
||||
},
|
||||
{ type: "array", items: {} },
|
||||
[[1, "rest"]],
|
||||
],
|
||||
[
|
||||
"nullable arrays",
|
||||
{
|
||||
anyOf: [{ type: "array", items: { type: "string" } }, { type: "null" }],
|
||||
},
|
||||
{
|
||||
anyOf: [{ type: "array", items: { type: "string" } }, { type: "null" }],
|
||||
},
|
||||
[null, ["text"]],
|
||||
],
|
||||
] as const;
|
||||
|
||||
describe("reduced array item presentation", () => {
|
||||
it.each(reducedArrayCases)(
|
||||
"advertises %s without placing items beside a non-array type",
|
||||
(_label, property, expected, canonicalValues) => {
|
||||
const { advertised, validateCanonical, validateAdvertised } =
|
||||
reducedPresentation(
|
||||
{ values: property },
|
||||
{ ordinal: { type: "integer", minimum: 0 } },
|
||||
);
|
||||
expect(advertised).toEqual({ values: expected });
|
||||
for (const values of canonicalValues) {
|
||||
expect(validateCanonical({ values })).toBe(true);
|
||||
expect(validateAdvertised({ values })).toBe(true);
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
it("stops deriving element types at the nested array depth bound", () => {
|
||||
let property: Record<string, unknown> = { type: "string" };
|
||||
for (let level = 0; level < 5; level += 1)
|
||||
property = { type: "array", items: property };
|
||||
const { advertised, validateAdvertised } = reducedPresentation({
|
||||
values: property,
|
||||
});
|
||||
expect(advertised).toEqual({
|
||||
values: {
|
||||
type: "array",
|
||||
items: {
|
||||
type: "array",
|
||||
items: { type: "array", items: { type: "array", items: {} } },
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(validateAdvertised({ values: [[[[["deep"]]]]] })).toBe(true);
|
||||
});
|
||||
|
||||
it("keeps an inline forbidden value as the empty schema under not", () => {
|
||||
const canonical = {
|
||||
type: "object",
|
||||
properties: { view: { type: "string" } },
|
||||
if: { properties: { view: { const: "x" } }, required: ["view"] },
|
||||
then: { required: ["view"] },
|
||||
else: { properties: { view: { not: {} } } },
|
||||
};
|
||||
const { schema, presentation } = compactAdvertisedInputSchema(
|
||||
canonical,
|
||||
13 * 1024,
|
||||
);
|
||||
expect(presentation).toBe("inline");
|
||||
expect(schema).toHaveProperty(["else", "properties", "view", "not"], {});
|
||||
const canonicalValidate = new Ajv2020({ strict: false }).compile(canonical);
|
||||
const advertisedValidate = new Ajv2020({ strict: false }).compile(schema);
|
||||
for (const input of [{}, { view: "x" }, { view: "y" }])
|
||||
expect(advertisedValidate(input)).toBe(canonicalValidate(input));
|
||||
});
|
||||
});
|
||||
@@ -28,8 +28,6 @@ const jsonValueTypes = [
|
||||
];
|
||||
// An empty item schema accepts every JSON value, including arbitrarily nested
|
||||
// arrays, while satisfying clients that require `items` for array types.
|
||||
// Gemini rejects `items` beside a type list; the compact catalog test fails
|
||||
// if a tool would advertise this presentation.
|
||||
const jsonValueSchema = () => ({ type: [...jsonValueTypes], items: {} });
|
||||
|
||||
const enumValueTypes = (values: readonly unknown[]): readonly string[] => [
|
||||
@@ -242,18 +240,6 @@ const inlineWithinBudget = (
|
||||
if (!first) yield ",";
|
||||
first = false;
|
||||
yield JSON.stringify(key) + ":";
|
||||
// `not: {}` forbids a value; no client infers a type for a value it
|
||||
// must omit, and the any-value list would place `items` beside a
|
||||
// non-array type, which Gemini function declarations reject.
|
||||
if (
|
||||
position === "schema" &&
|
||||
key === "not" &&
|
||||
isSchemaObject(child) &&
|
||||
Object.keys(child).length === 0
|
||||
) {
|
||||
yield "{}";
|
||||
continue;
|
||||
}
|
||||
const childPosition =
|
||||
position === "map"
|
||||
? "schema"
|
||||
@@ -337,106 +323,6 @@ const detectedTypes = (
|
||||
return [...new Set(types.flatMap((type) => type ?? []))];
|
||||
};
|
||||
|
||||
interface ScopedSchema {
|
||||
readonly schema: Record<string, unknown>;
|
||||
readonly scope: JsonSchemaScope;
|
||||
}
|
||||
|
||||
/**
|
||||
* Collect the item schema of every array alternative. Tuples, boolean item
|
||||
* schemas and unresolved references are unknown because one item type would
|
||||
* not describe every accepted element.
|
||||
*/
|
||||
const arrayItemAlternatives = (
|
||||
schema: unknown,
|
||||
references: JsonSchemaReferences,
|
||||
inherited: JsonSchemaScope | undefined,
|
||||
active: ReadonlySet<object> = new Set(),
|
||||
): readonly ScopedSchema[] | undefined => {
|
||||
if (!isSchemaObject(schema)) return undefined;
|
||||
const scope = references.enter(schema, inherited);
|
||||
if (scope === undefined) return undefined;
|
||||
if (typeof schema.$ref === "string") {
|
||||
if (active.has(schema)) return undefined;
|
||||
const target = references.resolve(scope, schema.$ref);
|
||||
return target.kind === "target"
|
||||
? arrayItemAlternatives(
|
||||
target.value,
|
||||
references,
|
||||
scope,
|
||||
new Set([...active, schema]),
|
||||
)
|
||||
: undefined;
|
||||
}
|
||||
const types =
|
||||
typeof schema.type === "string"
|
||||
? [schema.type]
|
||||
: Array.isArray(schema.type)
|
||||
? schema.type
|
||||
: undefined;
|
||||
if (types !== undefined) {
|
||||
if (!types.includes("array")) return [];
|
||||
return isSchemaObject(schema.items) && schema.prefixItems === undefined
|
||||
? [{ schema: schema.items, scope }]
|
||||
: undefined;
|
||||
}
|
||||
const alternatives = Array.isArray(schema.anyOf)
|
||||
? schema.anyOf
|
||||
: Array.isArray(schema.oneOf)
|
||||
? schema.oneOf
|
||||
: [];
|
||||
if (alternatives.length === 0) return undefined;
|
||||
const nested = alternatives.map((branch) =>
|
||||
arrayItemAlternatives(branch, references, scope, active),
|
||||
);
|
||||
return nested.some((items) => items === undefined)
|
||||
? undefined
|
||||
: nested.flatMap((items) => items ?? []);
|
||||
};
|
||||
|
||||
/** Nested arrays beyond this level advertise an unconstrained item schema. */
|
||||
const REDUCED_ITEM_MAX_DEPTH = 3;
|
||||
|
||||
/**
|
||||
* Advertise the element types the canonical item schemas accept. Gemini
|
||||
* function declarations permit `items` only beside the single `array` type,
|
||||
* so an element that may be an array alongside other types, or whose type is
|
||||
* unknown, uses the empty schema: it accepts every element and still gives
|
||||
* clients that require `items` an item schema.
|
||||
*/
|
||||
const reducedItemSchema = (
|
||||
alternatives: readonly ScopedSchema[] | undefined,
|
||||
references: JsonSchemaReferences,
|
||||
depth = 1,
|
||||
): Record<string, unknown> => {
|
||||
if (
|
||||
alternatives === undefined ||
|
||||
alternatives.length === 0 ||
|
||||
depth > REDUCED_ITEM_MAX_DEPTH
|
||||
)
|
||||
return {};
|
||||
const types: string[] = [];
|
||||
for (const { schema, scope } of alternatives) {
|
||||
const detected = detectedTypes(schema, references, scope);
|
||||
if (detected === undefined || detected.length === 0) return {};
|
||||
types.push(...detected);
|
||||
}
|
||||
const unique = [...new Set(types)];
|
||||
if (!unique.includes("array"))
|
||||
return { type: unique.length === 1 ? unique[0] : unique };
|
||||
if (unique.length > 1) return {};
|
||||
const nested: ScopedSchema[] = [];
|
||||
for (const { schema, scope } of alternatives) {
|
||||
const items = arrayItemAlternatives(schema, references, scope);
|
||||
if (items === undefined) return { type: "array", items: {} };
|
||||
nested.push(...items);
|
||||
}
|
||||
return {
|
||||
type: "array",
|
||||
items: reducedItemSchema(nested, references, depth + 1),
|
||||
};
|
||||
};
|
||||
|
||||
/** Give homogeneous array-item unions the type required by strict gateways. */
|
||||
const typeArrayItemUnions = (
|
||||
schema: Record<string, unknown>,
|
||||
@@ -520,34 +406,6 @@ const schemaChildrenAndMaps = (
|
||||
return [];
|
||||
});
|
||||
|
||||
/**
|
||||
* Present a reduced property's reference-resolved types. Reduced inputs omit
|
||||
* nested validation, but clients still require an item schema for an array
|
||||
* declaration, and Gemini accepts `items` only beside the single `array` type,
|
||||
* so a property that may be an array or another type lists typed branches.
|
||||
*/
|
||||
const reducedPropertyTypes = (
|
||||
value: unknown,
|
||||
references: JsonSchemaReferences,
|
||||
): Record<string, unknown> => {
|
||||
const types = detectedTypes(value, references, references.rootScope);
|
||||
if (types === undefined || types.length === 0) return {};
|
||||
if (!types.includes("array"))
|
||||
return { type: types.length === 1 ? types[0] : types };
|
||||
const array = {
|
||||
type: "array",
|
||||
items: reducedItemSchema(
|
||||
arrayItemAlternatives(value, references, references.rootScope),
|
||||
references,
|
||||
),
|
||||
};
|
||||
return types.length === 1
|
||||
? array
|
||||
: {
|
||||
anyOf: types.map((type) => (type === "array" ? array : { type })),
|
||||
};
|
||||
};
|
||||
|
||||
const REDUCED_ANNOTATION =
|
||||
"Advertised in reduced form by the compact input schema profile; the server still validates complete canonical inputs.";
|
||||
|
||||
@@ -583,13 +441,21 @@ const reduceToShallowProperties = (
|
||||
const references = new JsonSchemaReferences(schema);
|
||||
const shallow = Object.fromEntries(
|
||||
Object.entries(properties).map(([name, value]) => {
|
||||
const types = detectedTypes(value, references, references.rootScope);
|
||||
const type = types?.length === 1 ? types[0] : types;
|
||||
const description =
|
||||
isSchemaObject(value) && typeof value.description === "string"
|
||||
? { description: value.description }
|
||||
: {};
|
||||
return [
|
||||
name,
|
||||
{ ...reducedPropertyTypes(value, references), ...description },
|
||||
{
|
||||
...(type === undefined ? {} : { type }),
|
||||
// Reduced inputs intentionally omit nested validation, but clients
|
||||
// still require an item schema to accept an array declaration.
|
||||
...(types?.includes("array") ? { items: jsonValueSchema() } : {}),
|
||||
...description,
|
||||
},
|
||||
];
|
||||
}),
|
||||
);
|
||||
|
||||
@@ -246,7 +246,6 @@ const nativeExportEvidence = () => {
|
||||
exports: coverage([
|
||||
{
|
||||
name: "open_native",
|
||||
runtime_name: "open_native",
|
||||
address: "0x1000",
|
||||
weak: false,
|
||||
reexport: false,
|
||||
|
||||
@@ -250,7 +250,7 @@ export const BINARY_DIAGNOSTICS_TOOL_CONTRACTS = [
|
||||
...toolContractMetadata("inspect_binary_layout"),
|
||||
kind: "native-provider",
|
||||
description:
|
||||
"Inspect an explicit local binary without launching it as a host process. Returns file-backed section/segment ranges, linked addresses, original symbol/REL/RELA table identities, complete encoded RELR tables and upstream-derived offsets with unknown per-word locations/addends, raw name bytes, dependency/interpreter names and static mitigation inferences inline with artifact SHA-256 Evidence. Profile: EXEC/DYN/REL ELF64 x86-64, ELF32 i386, ELF32 little-endian ARM and ELF32 little- or big-endian MIPS on Linux x64 via caller-supplied Python with unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2. Runtime addresses/library paths, overall relocation inventory completeness and complete derived GOT/PLT coverage remain unknown. Zero executable entry values mean absence; relocatable entries are not applicable. Upstream can emulate PLT instructions with Unicorn for derived static maps. Uses bounded complete output and owned temporary files/processes; malformed, unsupported or oversized objects return no partial success. Select detail summary to retain the complete layout Evidence in this session and receive only its summary view, then inspect sections, symbols, mitigations or linkage with inspect_analysis_view. pwntools_timeout_ms is each owned pwntools command's deadline (omitted means 30000 ms, range 1..2147483647, next call). cpuSeconds 30 and the output, diagnostic, and address-space byte allowances have no runtime override.",
|
||||
"Inspect an explicit local binary without launching it as a host process. Returns file-backed section/segment ranges, linked addresses, original symbol/REL/RELA table identities, complete encoded RELR tables and upstream-derived offsets with unknown per-word locations/addends, raw name bytes, dependency/interpreter names and static mitigation inferences inline with artifact SHA-256 Evidence. Initial profile: ELF64 x86-64 little-endian EXEC/DYN/REL on Linux x64 via caller-supplied Python with unchanged pwntools 4.15.0, pyelftools 0.33 and Unicorn 2.1.2. Runtime addresses/library paths, overall relocation inventory completeness and complete derived GOT/PLT coverage remain unknown. Zero executable entry values mean absence; relocatable entries are not applicable. Upstream can emulate PLT instructions with Unicorn for derived static maps. Uses bounded complete output and owned temporary files/processes; malformed, unsupported or oversized objects return no partial success. Select detail summary to retain the complete layout Evidence in this session and receive only its summary view, then inspect sections, symbols, mitigations or linkage with inspect_analysis_view. pwntools_timeout_ms is each owned pwntools command's deadline (omitted means 30000 ms, range 1..2147483647, next call). cpuSeconds 30 and the output, diagnostic, and address-space byte allowances have no runtime override.",
|
||||
inputSchema: inspectBinaryLayoutRequestSchema,
|
||||
outputSchema: analysisDetailOutputSchemaOf(
|
||||
binaryLayoutSchema,
|
||||
|
||||
@@ -1,31 +1,8 @@
|
||||
import { z } from "zod";
|
||||
import { absoluteLocalPathStringSchema } from "../domain/localPath.js";
|
||||
|
||||
/** Caller-selected binary_session projections. */
|
||||
export const BINARY_SESSION_DETAILS = ["summary", "complete"] as const;
|
||||
export type BinarySessionDetail = (typeof BINARY_SESSION_DETAILS)[number];
|
||||
|
||||
/**
|
||||
* Fields a summary leaves out, as dotted paths through arrays. Each stays
|
||||
* available from the same session with detail: "complete".
|
||||
*/
|
||||
export const BINARY_SESSION_SUMMARY_OMITTED_FIELDS = [
|
||||
"analysis_provider_candidates.availability.diagnostics",
|
||||
"analysis_provider_candidates.target_support.diagnostics",
|
||||
"analysis_provider_candidates.capabilities",
|
||||
"tool_availability.groups.surface",
|
||||
"tool_availability.groups.effects",
|
||||
"tool_availability.groups.annotations",
|
||||
] as const;
|
||||
|
||||
/** Optional expectations used to compare the live session with its caller. */
|
||||
export const binarySessionInputSchema = z.strictObject({
|
||||
detail: z
|
||||
.enum(BINARY_SESSION_DETAILS)
|
||||
.default("summary")
|
||||
.describe(
|
||||
"summary (default) keeps open state, the binding, current-route capabilities, each provider candidate's availability and target-support status, code and reason, and every tool's availability, reason and remediation; omitted_fields lists what it leaves out. complete adds provider diagnostics, candidate capabilities and static tool surface, effect and annotation flags.",
|
||||
),
|
||||
expected_package_version: z.string().min(1).optional(),
|
||||
expected_catalog_digest: z
|
||||
.string()
|
||||
|
||||
@@ -80,7 +80,7 @@ export const SESSION_TOOL_CONTRACTS = [
|
||||
),
|
||||
session(
|
||||
"binary_session",
|
||||
'Report the active binary target or resource-only app root, provider and capability availability when applicable, client-feature, analysis, and server-identity status without starting analysis. The default summary lists omitted_fields; detail: "complete" adds provider diagnostics, candidate capabilities and static tool flags.',
|
||||
"Report the active binary target or resource-only app root, provider and capability availability when applicable, client-feature, analysis, and server-identity status without starting analysis.",
|
||||
binarySessionInputSchema,
|
||||
),
|
||||
session(
|
||||
|
||||
@@ -14,15 +14,7 @@ import {
|
||||
import { nativeFunctionMatchingSchema } from "../domain/native/functionMatching.js";
|
||||
import { nativeByteReadSchema } from "../domain/native/nativeByteRead.js";
|
||||
import { analysisDetailOutputSchemaOf } from "./analysisDetail.js";
|
||||
import {
|
||||
functionDossierSchema,
|
||||
analysisBookmarkSchema,
|
||||
functionInstructionWindowSchema,
|
||||
referenceEdgeSchema,
|
||||
unresolvedCallSchema,
|
||||
procedureIdentitySchema,
|
||||
containingProcedureResolutionSchema,
|
||||
} from "../domain/hopperValues.js";
|
||||
import { functionDossierSchema } from "../domain/hopperValues.js";
|
||||
import { nativeInventoryRowsSchemas } from "../domain/native/nativeInventory.js";
|
||||
import { z } from "zod";
|
||||
import { byteSearchResultSchema } from "../domain/native/bytePattern.js";
|
||||
@@ -51,6 +43,14 @@ import { residualUnknownSchema } from "../domain/residualUnknown.js";
|
||||
import { evidenceBundleSchema } from "../domain/evidenceBundle.js";
|
||||
|
||||
import { processCaptureComparisonSchema } from "../domain/process/processComparison.js";
|
||||
import {
|
||||
analysisBookmarkSchema,
|
||||
functionInstructionWindowSchema,
|
||||
referenceEdgeSchema,
|
||||
unresolvedCallSchema,
|
||||
procedureIdentitySchema,
|
||||
containingProcedureResolutionSchema,
|
||||
} from "../domain/hopperValues.js";
|
||||
import { nativeApiInspectionResultSchema } from "../domain/native/nativeApiBoundary.js";
|
||||
import {
|
||||
demangleSwiftSchema,
|
||||
@@ -97,8 +97,7 @@ import {
|
||||
objcClassDiscoveryOutput,
|
||||
evidenceResultOf,
|
||||
segmentOutput,
|
||||
sessionProviderFacts,
|
||||
sessionStatusDetailShapes,
|
||||
sessionProvider,
|
||||
symbolDiscoveryOutput,
|
||||
targetFormatSchema,
|
||||
targetKindSchema,
|
||||
@@ -565,27 +564,18 @@ export const sessionOutputSchemas = {
|
||||
}),
|
||||
]),
|
||||
),
|
||||
// Every session kind crossed with every detail level, so `detail` fixes
|
||||
// which candidate, tool-group and omission fields the result carries.
|
||||
binary_session: lifecycleResultOf(
|
||||
z.union(
|
||||
sessionStatusDetailShapes.flatMap((detail) => [
|
||||
sessionProviderFacts.extend({ open: z.literal(false), ...detail }),
|
||||
sessionProviderFacts.extend({
|
||||
open: z.literal(true),
|
||||
session_kind: z.literal("bundle_resources"),
|
||||
resource_root: bundleResourceRootSchema,
|
||||
...detail,
|
||||
}),
|
||||
...openLifecycleTargetVariants.map((variant) =>
|
||||
sessionProviderFacts.extend({
|
||||
open: z.literal(true),
|
||||
...variant.shape,
|
||||
...detail,
|
||||
}),
|
||||
),
|
||||
]),
|
||||
),
|
||||
z.union([
|
||||
sessionProvider.extend({ open: z.literal(false) }),
|
||||
sessionProvider.extend({
|
||||
open: z.literal(true),
|
||||
session_kind: z.literal("bundle_resources"),
|
||||
resource_root: bundleResourceRootSchema,
|
||||
}),
|
||||
...openLifecycleTargetVariants.map((variant) =>
|
||||
sessionProvider.extend({ open: z.literal(true), ...variant.shape }),
|
||||
),
|
||||
]),
|
||||
),
|
||||
export_evidence_bundle: lifecycleResultOf(
|
||||
z.object({
|
||||
|
||||
@@ -21,7 +21,6 @@ import { analysisErrorProjectionSchema } from "./errorSchemas.js";
|
||||
import { relationshipCoverageSchema } from "../domain/relationshipCoverage.js";
|
||||
import { ZIP_PACKAGE_FORMATS } from "../domain/zipPackageFormat.js";
|
||||
import { objcClassImportIdentitySchema } from "../domain/native/objcSwiftMetadata.js";
|
||||
import { BINARY_SESSION_SUMMARY_OMITTED_FIELDS } from "./sessionStatusContract.js";
|
||||
|
||||
/** Specialize the single normalized result inside a complete Evidence record. */
|
||||
export const evidenceResultOf = <Schema extends z.ZodType>(schema: Schema) =>
|
||||
@@ -257,48 +256,41 @@ const providerRejectionCode: z.ZodType<ProviderRejectionCode> = z.enum(
|
||||
|
||||
const providerDiagnostics = z.record(z.string(), jsonValueSchema);
|
||||
|
||||
const providerAvailabilityStates = {
|
||||
unknown: z.object({
|
||||
const providerAvailability = z.discriminatedUnion("status", [
|
||||
z.object({
|
||||
status: z.literal("unknown"),
|
||||
code: z.null(),
|
||||
reason: z.string().min(1),
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
available: z.object({
|
||||
z.object({
|
||||
status: z.literal("available"),
|
||||
code: z.null(),
|
||||
reason: z.null(),
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
unavailable: z.object({
|
||||
z.object({
|
||||
status: z.literal("unavailable"),
|
||||
code: providerRejectionCode,
|
||||
reason: z.string().min(1),
|
||||
}),
|
||||
};
|
||||
|
||||
const providerAvailability = z.discriminatedUnion("status", [
|
||||
providerAvailabilityStates.unknown.extend({
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
providerAvailabilityStates.available.extend({
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
providerAvailabilityStates.unavailable.extend({
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
]);
|
||||
|
||||
const providerTargetSupportStates = {
|
||||
unknown: z.object({
|
||||
const providerTargetSupport = z.discriminatedUnion("status", [
|
||||
z.object({
|
||||
status: z.literal("unknown"),
|
||||
code: z.null(),
|
||||
reason: z.string().min(1),
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
supported: z.object({
|
||||
z.object({
|
||||
status: z.literal("supported"),
|
||||
code: z.null(),
|
||||
reason: z.null(),
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
unsupported: z.object({
|
||||
z.object({
|
||||
status: z.literal("unsupported"),
|
||||
code: z.enum([
|
||||
"target_kind_unsupported",
|
||||
@@ -308,46 +300,10 @@ const providerTargetSupportStates = {
|
||||
"managed_target_unsupported",
|
||||
]),
|
||||
reason: z.string().min(1),
|
||||
}),
|
||||
};
|
||||
|
||||
const providerTargetSupport = z.discriminatedUnion("status", [
|
||||
providerTargetSupportStates.unknown.extend({
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
providerTargetSupportStates.supported.extend({
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
providerTargetSupportStates.unsupported.extend({
|
||||
diagnostics: providerDiagnostics,
|
||||
}),
|
||||
]);
|
||||
|
||||
/** Complete candidate facts, including provider-owned diagnostics and capability sets. */
|
||||
export const analysisProviderCandidate = z.object({
|
||||
provider: providerIdentity,
|
||||
availability: providerAvailability,
|
||||
target_support: providerTargetSupport,
|
||||
selected: z.boolean(),
|
||||
capabilities: z.array(providerCapabilitySet),
|
||||
});
|
||||
|
||||
/** Summary candidate: the same status, code, and reason without diagnostics or capabilities. */
|
||||
export const analysisProviderCandidateSummary = z.strictObject({
|
||||
provider: providerIdentity,
|
||||
availability: z.discriminatedUnion("status", [
|
||||
providerAvailabilityStates.unknown.strict(),
|
||||
providerAvailabilityStates.available.strict(),
|
||||
providerAvailabilityStates.unavailable.strict(),
|
||||
]),
|
||||
target_support: z.discriminatedUnion("status", [
|
||||
providerTargetSupportStates.unknown.strict(),
|
||||
providerTargetSupportStates.supported.strict(),
|
||||
providerTargetSupportStates.unsupported.strict(),
|
||||
]),
|
||||
selected: z.boolean(),
|
||||
});
|
||||
|
||||
const clientFeatureNameSchema = z.enum([
|
||||
"elicitation_form",
|
||||
"elicitation_url",
|
||||
@@ -448,93 +404,45 @@ export const TOOL_ANNOTATION_NAMES = Object.keys(
|
||||
toolAvailabilityFacts.annotations.shape,
|
||||
) as (keyof z.output<typeof toolAvailabilityFacts.annotations>)[];
|
||||
|
||||
const toolAvailabilityStatusGroupFacts = {
|
||||
const toolAvailabilityGroupFacts = {
|
||||
names: z.array(z.string().min(1)).min(1),
|
||||
surface: toolAvailabilityFacts.surface,
|
||||
default_mode_available: toolAvailabilityFacts.default_mode_available,
|
||||
modes: toolAvailabilityFacts.modes,
|
||||
client_requirements: toolAvailabilityFacts.client_requirements.optional(),
|
||||
};
|
||||
|
||||
const toolAvailabilityGroupFacts = {
|
||||
names: toolAvailabilityStatusGroupFacts.names,
|
||||
surface: toolAvailabilityFacts.surface,
|
||||
default_mode_available:
|
||||
toolAvailabilityStatusGroupFacts.default_mode_available,
|
||||
modes: toolAvailabilityStatusGroupFacts.modes,
|
||||
client_requirements: toolAvailabilityStatusGroupFacts.client_requirements,
|
||||
effects: z.array(toolAvailabilityFacts.effects.keyof()),
|
||||
annotations: z.array(toolAvailabilityFacts.annotations.keyof()),
|
||||
};
|
||||
|
||||
const toolAvailabilityGroupOf = <Facts extends z.ZodRawShape>(facts: Facts) =>
|
||||
z.discriminatedUnion("available", [
|
||||
z.strictObject({
|
||||
...facts,
|
||||
available: z.literal(true),
|
||||
reason: z.literal("available"),
|
||||
remediation: z.null(),
|
||||
}),
|
||||
z.strictObject({
|
||||
...facts,
|
||||
available: z.literal(false),
|
||||
reason: toolUnavailabilityReason,
|
||||
remediation: z.string().nullable(),
|
||||
}),
|
||||
]);
|
||||
const toolAvailabilityGroup = z.discriminatedUnion("available", [
|
||||
z.strictObject({
|
||||
...toolAvailabilityGroupFacts,
|
||||
available: z.literal(true),
|
||||
reason: z.literal("available"),
|
||||
remediation: z.null(),
|
||||
}),
|
||||
z.strictObject({
|
||||
...toolAvailabilityGroupFacts,
|
||||
available: z.literal(false),
|
||||
reason: toolUnavailabilityReason,
|
||||
remediation: z.string().nullable(),
|
||||
}),
|
||||
]);
|
||||
|
||||
/** Complete inventory with explicit inherited client requirements and enabled flags. */
|
||||
export const toolAvailabilityInventory = z.strictObject({
|
||||
client_requirements: toolAvailabilityFacts.client_requirements,
|
||||
groups: z.array(toolAvailabilityGroupOf(toolAvailabilityGroupFacts)),
|
||||
groups: z.array(toolAvailabilityGroup),
|
||||
});
|
||||
export type ToolAvailabilityInventory = z.output<
|
||||
typeof toolAvailabilityInventory
|
||||
>;
|
||||
|
||||
/**
|
||||
* Summary inventory: every tool name with its current availability, reason,
|
||||
* remediation, modes, and client requirements, grouped by those facts alone.
|
||||
* Static surface, effect, and annotation flags are omitted, not false.
|
||||
*/
|
||||
export const toolAvailabilitySummaryInventory = z.strictObject({
|
||||
client_requirements: toolAvailabilityFacts.client_requirements,
|
||||
groups: z.array(toolAvailabilityGroupOf(toolAvailabilityStatusGroupFacts)),
|
||||
});
|
||||
export type ToolAvailabilitySummaryInventory = z.output<
|
||||
typeof toolAvailabilitySummaryInventory
|
||||
>;
|
||||
|
||||
/** One tool's current availability without its static catalog flags. */
|
||||
export type ToolAvailabilitySummary = Omit<
|
||||
ToolAvailability,
|
||||
"surface" | "effects" | "annotations"
|
||||
>;
|
||||
|
||||
const isCompleteToolAvailabilityInventory = (
|
||||
inventory: ToolAvailabilityInventory | ToolAvailabilitySummaryInventory,
|
||||
): inventory is ToolAvailabilityInventory =>
|
||||
inventory.groups.every((group) => "surface" in group);
|
||||
|
||||
/** Expand the wire representation once at the consuming boundary. */
|
||||
export function expandToolAvailability(
|
||||
export const expandToolAvailability = (
|
||||
inventory: ToolAvailabilityInventory,
|
||||
): readonly ToolAvailability[];
|
||||
export function expandToolAvailability(
|
||||
inventory: ToolAvailabilityInventory | ToolAvailabilitySummaryInventory,
|
||||
): readonly (ToolAvailability | ToolAvailabilitySummary)[];
|
||||
export function expandToolAvailability(
|
||||
inventory: ToolAvailabilityInventory | ToolAvailabilitySummaryInventory,
|
||||
): readonly (ToolAvailability | ToolAvailabilitySummary)[] {
|
||||
if (!isCompleteToolAvailabilityInventory(inventory))
|
||||
return inventory.groups.flatMap(({ names, ...group }) =>
|
||||
names.map((name) => ({
|
||||
...group,
|
||||
name,
|
||||
client_requirements:
|
||||
group.client_requirements ?? inventory.client_requirements,
|
||||
})),
|
||||
);
|
||||
return inventory.groups.flatMap((group) =>
|
||||
): readonly ToolAvailability[] =>
|
||||
inventory.groups.flatMap((group) =>
|
||||
group.names.map((name) =>
|
||||
toolAvailability.parse({
|
||||
...group,
|
||||
@@ -556,10 +464,8 @@ export function expandToolAvailability(
|
||||
}),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
/** Session status facts shared by every detail level and session kind. */
|
||||
export const sessionProviderFacts = z
|
||||
export const sessionProvider = z
|
||||
.object({
|
||||
cleanup_pending: z.boolean(),
|
||||
capabilities: z.array(providerCapabilitySet),
|
||||
@@ -577,6 +483,16 @@ export const sessionProviderFacts = z
|
||||
analysis_profile: analysisProfileSchema,
|
||||
})
|
||||
.nullable(),
|
||||
analysis_provider_candidates: z.array(
|
||||
z.object({
|
||||
provider: providerIdentity,
|
||||
availability: providerAvailability,
|
||||
target_support: providerTargetSupport,
|
||||
selected: z.boolean(),
|
||||
capabilities: z.array(providerCapabilitySet),
|
||||
}),
|
||||
),
|
||||
tool_availability: toolAvailabilityInventory,
|
||||
client_features: z.object({
|
||||
elicitation_form: z.boolean(),
|
||||
elicitation_url: z.boolean(),
|
||||
@@ -630,34 +546,6 @@ export const sessionProviderFacts = z
|
||||
})
|
||||
.passthrough();
|
||||
|
||||
/**
|
||||
* Fields whose shape `detail` selects. A summary must list what it omits and
|
||||
* carry only summary candidates and tool groups; complete detail must carry
|
||||
* diagnostics, candidate capabilities and static tool flags, and no omission list.
|
||||
*/
|
||||
export const sessionStatusDetailShapes = [
|
||||
{
|
||||
detail: z.literal("summary"),
|
||||
omitted_fields: z
|
||||
.array(z.enum(BINARY_SESSION_SUMMARY_OMITTED_FIELDS))
|
||||
.min(1),
|
||||
analysis_provider_candidates: z.array(analysisProviderCandidateSummary),
|
||||
tool_availability: toolAvailabilitySummaryInventory,
|
||||
},
|
||||
{
|
||||
detail: z.literal("complete"),
|
||||
omitted_fields: z.never().optional(),
|
||||
analysis_provider_candidates: z.array(analysisProviderCandidate),
|
||||
tool_availability: toolAvailabilityInventory,
|
||||
},
|
||||
] as const;
|
||||
|
||||
/** Session status for either detail level, keyed by `detail`. */
|
||||
export const sessionProvider = z.discriminatedUnion("detail", [
|
||||
sessionProviderFacts.extend(sessionStatusDetailShapes[0]),
|
||||
sessionProviderFacts.extend(sessionStatusDetailShapes[1]),
|
||||
]);
|
||||
|
||||
export const nullableText = z.string().nullable();
|
||||
export const addressList = z.array(z.string());
|
||||
export const addressedEntry = z.object({
|
||||
|
||||
@@ -108,20 +108,14 @@ export class AnalysisProtocolError extends AnalysisError {
|
||||
export class AnalysisInputError extends AnalysisError {
|
||||
readonly _tag = "AnalysisInputError";
|
||||
override readonly userMessage: string | undefined;
|
||||
/** Refusal-specific correction, when the generic input-contract advice cannot fix it. */
|
||||
readonly remediationAction: string | undefined;
|
||||
|
||||
constructor(
|
||||
readonly operation: string,
|
||||
options?: AnalysisErrorOptions & {
|
||||
readonly userMessage?: string;
|
||||
readonly remediationAction?: string;
|
||||
},
|
||||
options?: AnalysisErrorOptions & { readonly userMessage?: string },
|
||||
readonly issues: readonly AnalysisInputIssue[] = [],
|
||||
) {
|
||||
super(`Invalid analysis input for ${operation}`, options);
|
||||
this.userMessage = options?.userMessage;
|
||||
this.remediationAction = options?.remediationAction;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -138,8 +132,6 @@ export interface AnalysisInputIssue {
|
||||
/** Schema-authored correction guidance for cross-field or custom checks. */
|
||||
readonly message?: string;
|
||||
readonly expected?: JsonValue;
|
||||
/** Producer-examined facts about the refused value, such as its location and shape. */
|
||||
readonly observed?: JsonValue;
|
||||
readonly minimum?: number;
|
||||
readonly maximum?: number;
|
||||
}
|
||||
|
||||
@@ -161,10 +161,7 @@ export const analysisErrorRemediationAction = (
|
||||
if (error instanceof ConfigurationError && error.settings.length > 0)
|
||||
return `Correct ${[...new Set(error.settings.map(({ setting }) => setting))].join(", ")} in the environment that starts REA (your shell or the MCP client's registration), then rerun.`;
|
||||
if (error instanceof AnalysisInputError)
|
||||
return (
|
||||
error.remediationAction ??
|
||||
`Read the declared input schema or CLI usage for ${error.operation}; if using MCP and the catalog may be stale, call tools/list. Correct the listed fields, then retry. Do not repeat the unchanged call.`
|
||||
);
|
||||
return `Read the declared input schema or CLI usage for ${error.operation}; if using MCP and the catalog may be stale, call tools/list. Correct the listed fields, then retry. Do not repeat the unchanged call.`;
|
||||
if (
|
||||
error instanceof AnalysisOutputError &&
|
||||
error.remediationAction !== undefined
|
||||
|
||||
@@ -368,7 +368,6 @@ const requestErrorDetails = (
|
||||
reason: issue.reason,
|
||||
...(issue.message === undefined ? {} : { message: issue.message }),
|
||||
...(issue.expected === undefined ? {} : { expected: issue.expected }),
|
||||
...(issue.observed === undefined ? {} : { observed: issue.observed }),
|
||||
...(issue.minimum === undefined ? {} : { minimum: issue.minimum }),
|
||||
...(issue.maximum === undefined ? {} : { maximum: issue.maximum }),
|
||||
})),
|
||||
|
||||
@@ -122,13 +122,6 @@ import {
|
||||
|
||||
import { absoluteLocalPathStringSchema } from "../localPath.js";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
browserScenarioSummarySchema,
|
||||
browserScenarioViewContextSchema,
|
||||
browserScenarioViewItemsSchema,
|
||||
browserScenarioViewRequestSchema,
|
||||
} from "./browserScenarioViewSchemas.js";
|
||||
import { projectBrowserScenarioView } from "./browserScenarioView.js";
|
||||
import {
|
||||
desktopViewRequestSchema,
|
||||
desktopViewItemsSchema,
|
||||
@@ -400,7 +393,6 @@ export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
|
||||
environmentStateViewRequestSchema,
|
||||
environmentCoverageViewRequestSchema,
|
||||
captureViewRequestSchema,
|
||||
browserScenarioViewRequestSchema,
|
||||
protobufViewRequestSchema,
|
||||
qlogViewRequestSchema,
|
||||
pcapViewRequestSchema,
|
||||
@@ -737,14 +729,6 @@ export const webNetworkCaptureSummaryViewResultSchema = z.strictObject({
|
||||
summary: captureSummarySchema,
|
||||
});
|
||||
|
||||
/** Compact outcomes and the focus screen from a retained browser scenario. */
|
||||
export const browserScenarioSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultBase,
|
||||
artifact: z.null(),
|
||||
parent_operation: z.literal("capture_browser_scenario"),
|
||||
summary: browserScenarioSummarySchema,
|
||||
});
|
||||
|
||||
/** Exact package declarations with retained decoded manifest text. */
|
||||
export const androidPackageSummaryViewResultSchema = z.strictObject({
|
||||
...summaryViewResultBase,
|
||||
@@ -1038,7 +1022,6 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
managedArtifactSummaryViewResultSchema,
|
||||
managedMemberSummaryViewResultSchema,
|
||||
webNetworkCaptureSummaryViewResultSchema,
|
||||
browserScenarioSummaryViewResultSchema,
|
||||
androidPackageSummaryViewResultSchema,
|
||||
pcapCaptureSummaryViewResultSchema,
|
||||
dartAotSummaryViewResultSchema,
|
||||
@@ -1093,15 +1076,6 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
|
||||
...viewResultBase,
|
||||
parent_operation: z.literal("inspect_web_network_capture"),
|
||||
}),
|
||||
z.strictObject({
|
||||
kind: z.literal("browser-scenario"),
|
||||
view: browserScenarioViewRequestSchema,
|
||||
items: browserScenarioViewItemsSchema,
|
||||
...viewResultBase,
|
||||
...browserScenarioViewContextSchema.shape,
|
||||
artifact: z.null(),
|
||||
parent_operation: z.literal("capture_browser_scenario"),
|
||||
}),
|
||||
z.strictObject({
|
||||
kind: z.literal("qlog"),
|
||||
view: qlogViewRequestSchema,
|
||||
@@ -1309,7 +1283,9 @@ const unsupportedParent = (
|
||||
new AnalysisUnsupportedTargetError(
|
||||
INSPECT_ANALYSIS_VIEW_OPERATION,
|
||||
parentArtifactPath(parent),
|
||||
`Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed Evidence from a supported parent operation with a compatible view. For a summary receipt, use its normalized_result.parent_evidence_id rather than the receipt's own evidence_id.`,
|
||||
parent.operation === "capture_browser_scenario"
|
||||
? `Parent Evidence operation "capture_browser_scenario" supports only the environment-manifest, environment-state and environment-coverage views.`
|
||||
: `Parent Evidence operation ${JSON.stringify(parent.operation)} does not support analysis views. Use completed Evidence from a supported parent operation with a compatible view. For a summary receipt, use its normalized_result.parent_evidence_id rather than the receipt's own evidence_id.`,
|
||||
);
|
||||
|
||||
const parentArtifactPath = (parent: AnalysisViewParent): string => {
|
||||
@@ -1355,7 +1331,6 @@ const retainedViewProjectors = new Map<string, typeof projectWasmArtifactView>([
|
||||
["inspect_managed_artifact", projectManagedArtifactView],
|
||||
["inspect_managed_members", projectManagedMemberView],
|
||||
["inspect_android_package", projectAndroidPackageView],
|
||||
["capture_browser_scenario", projectBrowserScenarioView],
|
||||
["inspect_dart_aot", projectDartAotView],
|
||||
["inspect_il2cpp_metadata", projectIl2cppMetadataView],
|
||||
["inspect_apk_signature", projectApkSignatureView],
|
||||
|
||||
@@ -1,193 +0,0 @@
|
||||
import { z } from "zod";
|
||||
import type { RetainedBrowserScenarioCapture } from "../browserScenarioCapture.js";
|
||||
import { isImmutableJsonSnapshot } from "../immutableJson.js";
|
||||
import { compareAsciiStrings } from "../canonicalOrdering.js";
|
||||
|
||||
const count = z.int().nonnegative();
|
||||
const statusClass = z.enum([
|
||||
"1xx",
|
||||
"2xx",
|
||||
"3xx",
|
||||
"4xx",
|
||||
"5xx",
|
||||
"failed",
|
||||
"unknown",
|
||||
]);
|
||||
|
||||
/** Counts concern retained transaction identities, including those without a request event. */
|
||||
export const browserScenarioNetworkSummarySchema = z.strictObject({
|
||||
selected: z.boolean().nullable(),
|
||||
transactions: count.nullable(),
|
||||
without_request: count,
|
||||
ambiguous_transactions: count,
|
||||
failed_transactions: count,
|
||||
by_origin: z.array(
|
||||
z.strictObject({ origin: z.string().nullable(), transactions: count }),
|
||||
),
|
||||
by_resource_type: z.array(
|
||||
z.strictObject({
|
||||
resource_type: z.string().nullable(),
|
||||
transactions: count,
|
||||
}),
|
||||
),
|
||||
by_status_class: z.array(
|
||||
z.strictObject({ status_class: statusClass, transactions: count }),
|
||||
),
|
||||
});
|
||||
|
||||
interface TransactionGroup {
|
||||
readonly transaction_id: string;
|
||||
readonly source_item_indices: readonly number[];
|
||||
readonly origin: string | null;
|
||||
readonly resource_type: string | null;
|
||||
readonly status_class: z.infer<typeof statusClass>;
|
||||
readonly has_request: boolean;
|
||||
readonly failed: boolean;
|
||||
readonly ambiguous: boolean;
|
||||
}
|
||||
|
||||
interface TransactionFacts {
|
||||
indices: number[];
|
||||
origins: Set<string | null>;
|
||||
resourceTypes: Set<string>;
|
||||
statuses: Set<number>;
|
||||
requests: number;
|
||||
failed: boolean;
|
||||
}
|
||||
|
||||
/** Sealed historical captures own this metadata-only index until their retained lifetime ends. */
|
||||
const transactionsByCapture = new WeakMap<
|
||||
object,
|
||||
readonly TransactionGroup[]
|
||||
>();
|
||||
|
||||
/** Keep all observed alternatives; one missing event never erases its transaction. */
|
||||
export const browserScenarioTransactionGroups = (
|
||||
capture: RetainedBrowserScenarioCapture,
|
||||
): readonly TransactionGroup[] => {
|
||||
const reusable = isImmutableJsonSnapshot(capture);
|
||||
const known = reusable ? transactionsByCapture.get(capture) : undefined;
|
||||
if (known !== undefined) return known;
|
||||
const groups = new Map<string, TransactionFacts>();
|
||||
for (const [index, event] of capture.events.items.entries()) {
|
||||
if (!("transaction_id" in event)) continue;
|
||||
let group = groups.get(event.transaction_id);
|
||||
if (group === undefined) {
|
||||
group = {
|
||||
indices: [],
|
||||
origins: new Set(),
|
||||
resourceTypes: new Set(),
|
||||
statuses: new Set(),
|
||||
requests: 0,
|
||||
failed: false,
|
||||
};
|
||||
groups.set(event.transaction_id, group);
|
||||
}
|
||||
group.indices.push(index);
|
||||
if (event.kind === "network-content") continue;
|
||||
group.origins.add(event.url.origin);
|
||||
group.resourceTypes.add(event.resource_type);
|
||||
if (event.kind === "request") group.requests += 1;
|
||||
if (event.kind === "response") group.statuses.add(event.status);
|
||||
if (event.kind === "request-failed") group.failed = true;
|
||||
}
|
||||
const result = [...groups].map(
|
||||
([transaction_id, group]): TransactionGroup => {
|
||||
const status = group.statuses.values().next().value;
|
||||
return Object.freeze({
|
||||
transaction_id,
|
||||
source_item_indices: Object.freeze(group.indices),
|
||||
origin:
|
||||
group.origins.size === 1
|
||||
? (group.origins.values().next().value ?? null)
|
||||
: null,
|
||||
resource_type:
|
||||
group.resourceTypes.size === 1
|
||||
? (group.resourceTypes.values().next().value ?? null)
|
||||
: null,
|
||||
status_class:
|
||||
group.statuses.size === 1 && status !== undefined
|
||||
? (`${String(Math.floor(status / 100))}xx` as z.infer<
|
||||
typeof statusClass
|
||||
>)
|
||||
: group.statuses.size === 0 && group.failed
|
||||
? "failed"
|
||||
: "unknown",
|
||||
has_request: group.requests > 0,
|
||||
failed: group.failed,
|
||||
ambiguous:
|
||||
group.requests > 1 ||
|
||||
group.statuses.size > 1 ||
|
||||
group.origins.size > 1 ||
|
||||
group.resourceTypes.size > 1,
|
||||
});
|
||||
},
|
||||
);
|
||||
if (reusable) transactionsByCapture.set(capture, result);
|
||||
return result;
|
||||
};
|
||||
|
||||
const increment = <Key>(values: Map<Key, number>, key: Key): void => {
|
||||
values.set(key, (values.get(key) ?? 0) + 1);
|
||||
};
|
||||
const sorted = <Key extends string | null>(
|
||||
values: ReadonlyMap<Key, number>,
|
||||
): [Key, number][] =>
|
||||
[...values].sort(([left], [right]) =>
|
||||
left === right
|
||||
? 0
|
||||
: left === null
|
||||
? -1
|
||||
: right === null
|
||||
? 1
|
||||
: compareAsciiStrings(left, right),
|
||||
);
|
||||
|
||||
/** Null selection/counts preserve legacy unknowns and unrequested network observation. */
|
||||
export const summarizeBrowserScenarioNetwork = (
|
||||
capture: RetainedBrowserScenarioCapture,
|
||||
): z.infer<typeof browserScenarioNetworkSummarySchema> => {
|
||||
const groups = browserScenarioTransactionGroups(capture);
|
||||
const contentSelected = Object.values(capture.scenario.network_content).some(
|
||||
Boolean,
|
||||
);
|
||||
const selected =
|
||||
contentSelected ||
|
||||
capture.scenario.event_families?.includes("network") === true
|
||||
? true
|
||||
: capture.scenario.event_families === undefined
|
||||
? null
|
||||
: false;
|
||||
const origins = new Map<string | null, number>();
|
||||
const resources = new Map<string | null, number>();
|
||||
const statuses = new Map<z.infer<typeof statusClass>, number>();
|
||||
let withoutRequest = 0;
|
||||
let ambiguous = 0;
|
||||
let failed = 0;
|
||||
for (const group of groups) {
|
||||
if (!group.has_request) withoutRequest += 1;
|
||||
if (group.ambiguous) ambiguous += 1;
|
||||
if (group.failed) failed += 1;
|
||||
increment(origins, group.origin);
|
||||
increment(resources, group.resource_type);
|
||||
increment(statuses, group.status_class);
|
||||
}
|
||||
return {
|
||||
selected,
|
||||
transactions: selected === true || groups.length > 0 ? groups.length : null,
|
||||
without_request: withoutRequest,
|
||||
ambiguous_transactions: ambiguous,
|
||||
failed_transactions: failed,
|
||||
by_origin: sorted(origins).map(([origin, transactions]) => ({
|
||||
origin,
|
||||
transactions,
|
||||
})),
|
||||
by_resource_type: sorted(resources).map(
|
||||
([resource_type, transactions]) => ({ resource_type, transactions }),
|
||||
),
|
||||
by_status_class: sorted(statuses).map(([status_class, transactions]) => ({
|
||||
status_class,
|
||||
transactions,
|
||||
})),
|
||||
};
|
||||
};
|
||||
@@ -1,388 +0,0 @@
|
||||
import type { z } from "zod";
|
||||
import type { AnalysisError } from "../analysisErrorBase.js";
|
||||
import {
|
||||
retainedBrowserScenarioCaptureSchema,
|
||||
type RetainedBrowserScenarioCapture as BrowserScenarioCapture,
|
||||
} from "../browserScenarioCapture.js";
|
||||
import type { BrowserScenarioStep as Step } from "../browserScenarioCaptureValues.js";
|
||||
import { compareAsciiStrings } from "../canonicalOrdering.js";
|
||||
import { err, ok, type Result } from "../result.js";
|
||||
import { analysisViewInputError } from "./analysisViewErrors.js";
|
||||
import {
|
||||
completeWithinViewCoverage,
|
||||
pageViewCoverage,
|
||||
} from "./analysisViewCoverage.js";
|
||||
import { createAnalysisViewResultReader } from "./analysisViewParent.js";
|
||||
import type {
|
||||
AnalysisViewCoverage,
|
||||
AnalysisViewParent,
|
||||
AnalysisViewRequest,
|
||||
UnsignedAnalysisView,
|
||||
} from "./analysisView.js";
|
||||
import {
|
||||
ARTIFACT_SECTIONS,
|
||||
stepRowSchema,
|
||||
browserScenarioSummarySchema,
|
||||
stepSectionsSchema,
|
||||
summaryStepSchema,
|
||||
focusAccessibilitySchema,
|
||||
browserScenarioViewItemsSchema,
|
||||
type BrowserScenarioViewRequest,
|
||||
type ArtifactSection,
|
||||
} from "./browserScenarioViewSchemas.js";
|
||||
|
||||
import {
|
||||
browserScenarioTransactionGroups,
|
||||
summarizeBrowserScenarioNetwork,
|
||||
} from "./browserScenarioNetworkView.js";
|
||||
|
||||
const OPERATION = "capture_browser_scenario" as const;
|
||||
const readCapture = createAnalysisViewResultReader(
|
||||
retainedBrowserScenarioCaptureSchema,
|
||||
);
|
||||
|
||||
const unsupportedView = (): AnalysisError =>
|
||||
analysisViewInputError([
|
||||
{
|
||||
path: ["view"],
|
||||
reason: "invalid_value",
|
||||
message:
|
||||
'Parent Evidence operation "capture_browser_scenario" supports summary and browser-scenario views, plus compatible environment views.',
|
||||
},
|
||||
]);
|
||||
|
||||
const sameUrl = (
|
||||
left: Step["after_url"] | undefined,
|
||||
right: Step["before_url"],
|
||||
): boolean =>
|
||||
left !== undefined &&
|
||||
left.url === right.url &&
|
||||
left.redacted === right.redacted;
|
||||
|
||||
const stepSections = (step: Step): z.infer<typeof stepSectionsSchema> => {
|
||||
const sections: z.infer<typeof stepSectionsSchema> = {
|
||||
captured: [],
|
||||
missing: [],
|
||||
truncated: [],
|
||||
};
|
||||
for (const section of ARTIFACT_SECTIONS) {
|
||||
const { state } = step.artifacts[section];
|
||||
if (state !== "not_requested") sections[state].push(section);
|
||||
}
|
||||
return sections;
|
||||
};
|
||||
|
||||
const summaryStep = (
|
||||
step: Step,
|
||||
previous: Step | undefined,
|
||||
): z.infer<typeof summaryStepSchema> => ({
|
||||
step_index: step.step_index,
|
||||
step_id: step.step_id,
|
||||
action: step.action,
|
||||
status: step.status,
|
||||
error: step.error,
|
||||
elapsed_ms: step.elapsed_ms,
|
||||
...(sameUrl(previous?.after_url, step.before_url)
|
||||
? {}
|
||||
: { before_url: step.before_url }),
|
||||
after_url: step.after_url,
|
||||
events: {
|
||||
sequence_start: step.event_sequence_start,
|
||||
sequence_end: step.event_sequence_end,
|
||||
},
|
||||
sections: stepSections(step),
|
||||
completeness: step.completeness.status,
|
||||
});
|
||||
|
||||
/** The first unsuccessful step explains a failure; otherwise the final screen does. */
|
||||
const focusAccessibility = (
|
||||
steps: readonly Step[],
|
||||
): z.infer<typeof focusAccessibilitySchema> => {
|
||||
const unsuccessful = steps.findIndex(({ status }) => status !== "completed");
|
||||
const focusIndex = unsuccessful === -1 ? steps.length - 1 : unsuccessful;
|
||||
const focus = steps[focusIndex];
|
||||
if (focus === undefined) throw new Error("Capture has no steps");
|
||||
const kind =
|
||||
unsuccessful === -1 ? ("last-step" as const) : "first-unsuccessful-step";
|
||||
for (let index = focusIndex; index >= 0; index -= 1) {
|
||||
const accessibility = steps[index]?.artifacts.accessibility;
|
||||
if (accessibility?.state === "captured")
|
||||
return {
|
||||
focus_step_index: focusIndex,
|
||||
focus: kind,
|
||||
step_index: index,
|
||||
accessibility,
|
||||
};
|
||||
}
|
||||
return {
|
||||
focus_step_index: focusIndex,
|
||||
focus: kind,
|
||||
step_index: null,
|
||||
accessibility: focus.artifacts.accessibility,
|
||||
};
|
||||
};
|
||||
|
||||
/** Counts in canonical key order; a null origin sorts first. */
|
||||
const sortedCounts = <Key extends string | null>(
|
||||
counts: ReadonlyMap<Key, number>,
|
||||
): [Key, number][] =>
|
||||
[...counts].sort(([left], [right]) =>
|
||||
left === right
|
||||
? 0
|
||||
: left === null
|
||||
? -1
|
||||
: right === null
|
||||
? 1
|
||||
: compareAsciiStrings(left, right),
|
||||
);
|
||||
|
||||
const increment = <Key>(counts: Map<Key, number>, key: Key): void => {
|
||||
counts.set(key, (counts.get(key) ?? 0) + 1);
|
||||
};
|
||||
|
||||
const eventCounts = (
|
||||
capture: BrowserScenarioCapture,
|
||||
): z.infer<typeof browserScenarioSummarySchema>["events"] => {
|
||||
const kinds = new Map<
|
||||
BrowserScenarioCapture["events"]["items"][number]["kind"],
|
||||
number
|
||||
>();
|
||||
for (const event of capture.events.items) increment(kinds, event.kind);
|
||||
const { omissions } = capture.events;
|
||||
return {
|
||||
retained: capture.events.retained,
|
||||
dropped: capture.events.dropped,
|
||||
by_kind: sortedCounts(kinds).map(([kind, count]) => ({ kind, count })),
|
||||
omissions: {
|
||||
by_kind: omissions.by_kind,
|
||||
network_transactions: omissions.network_transactions.length,
|
||||
network_transaction_range: omissions.network_transaction_range,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const selected = <Value>(
|
||||
filter: readonly Value[] | undefined,
|
||||
): ((value: Value) => boolean) => {
|
||||
if (filter === undefined) return () => true;
|
||||
const values = new Set(filter);
|
||||
return (value) => values.has(value);
|
||||
};
|
||||
|
||||
const stepRow = (
|
||||
{ artifacts: recorded, ...step }: Step,
|
||||
selection: ReadonlySet<ArtifactSection>,
|
||||
): z.infer<typeof stepRowSchema> => {
|
||||
const artifacts: z.infer<typeof stepRowSchema>["artifacts"] = {};
|
||||
const omitted: z.infer<typeof stepRowSchema>["omitted_artifacts"] = [];
|
||||
for (const section of ARTIFACT_SECTIONS) {
|
||||
const artifact = recorded[section];
|
||||
if (selection.has(section))
|
||||
Object.assign(artifacts, { [section]: artifact });
|
||||
else if (artifact.state !== "not_requested")
|
||||
omitted.push({ section, state: artifact.state });
|
||||
}
|
||||
return { ...step, artifacts, omitted_artifacts: omitted };
|
||||
};
|
||||
|
||||
/** Scan matching rows once and materialize only the requested page. */
|
||||
const selectedPage = <Input, Row>(
|
||||
source: readonly Input[],
|
||||
view: { readonly offset: number; readonly limit: number },
|
||||
matches: (item: Input, index: number) => boolean,
|
||||
project: (item: Input, index: number) => Row,
|
||||
): { rows: Row[]; coverage: AnalysisViewCoverage } => {
|
||||
const rows: Row[] = [];
|
||||
let total = 0;
|
||||
for (const [index, item] of source.entries()) {
|
||||
if (!matches(item, index)) continue;
|
||||
if (total >= view.offset && rows.length < view.limit)
|
||||
rows.push(project(item, index));
|
||||
total += 1;
|
||||
}
|
||||
return { rows, coverage: pageViewCoverage(view.offset, rows.length, total) };
|
||||
};
|
||||
|
||||
/** Text offsets are UTF-16 positions in the exact digest-bound retained artifact. */
|
||||
const textPage = (
|
||||
capture: BrowserScenarioCapture,
|
||||
view: Extract<BrowserScenarioViewRequest, { collection: "text" }>,
|
||||
) => {
|
||||
const artifact = capture.steps[view.step_index]!.artifacts[view.artifact];
|
||||
const value = artifact.state === "captured" ? artifact.value : null;
|
||||
const text = value?.text.slice(view.offset, view.offset + view.limit) ?? null;
|
||||
return {
|
||||
items: {
|
||||
collection: "text" as const,
|
||||
step_index: view.step_index,
|
||||
artifact: view.artifact,
|
||||
state: artifact.state,
|
||||
reason: "reason" in artifact ? artifact.reason : null,
|
||||
sha256: value?.sha256 ?? null,
|
||||
bytes: value?.bytes ?? null,
|
||||
observed: artifact.state === "truncated" ? artifact.observed : null,
|
||||
retained: artifact.state === "truncated" ? artifact.retained : null,
|
||||
text,
|
||||
},
|
||||
coverage: pageViewCoverage(
|
||||
view.offset,
|
||||
text?.length ?? 0,
|
||||
value?.text.length ?? 0,
|
||||
),
|
||||
};
|
||||
};
|
||||
|
||||
/** One stable page of exact recorded facts; no whole-result filtering arrays. */
|
||||
const pageRecords = (
|
||||
capture: BrowserScenarioCapture,
|
||||
view: BrowserScenarioViewRequest,
|
||||
): {
|
||||
readonly items: z.infer<typeof browserScenarioViewItemsSchema>;
|
||||
readonly coverage: AnalysisViewCoverage;
|
||||
} => {
|
||||
if (view.collection === "text") return textPage(capture, view);
|
||||
const stepSelected = selected(view.step_indices);
|
||||
if (view.collection === "steps") {
|
||||
const artifacts = new Set(view.artifacts);
|
||||
const page = selectedPage(
|
||||
capture.steps,
|
||||
view,
|
||||
({ step_index }) => stepSelected(step_index),
|
||||
(step) => stepRow(step, artifacts),
|
||||
);
|
||||
return {
|
||||
items: { collection: "steps", rows: page.rows },
|
||||
coverage: page.coverage,
|
||||
};
|
||||
}
|
||||
const kindSelected = selected(view.event_kinds);
|
||||
const transactionSelected = selected(view.transaction_ids);
|
||||
if (view.collection === "transactions") {
|
||||
const page = selectedPage(
|
||||
browserScenarioTransactionGroups(capture),
|
||||
view,
|
||||
(group) =>
|
||||
transactionSelected(group.transaction_id) &&
|
||||
(view.step_indices === undefined ||
|
||||
group.source_item_indices.some((index) =>
|
||||
stepSelected(capture.events.items[index]!.step_index),
|
||||
)),
|
||||
(group) => {
|
||||
const events: {
|
||||
item_index: number;
|
||||
event: BrowserScenarioCapture["events"]["items"][number];
|
||||
}[] = [];
|
||||
const omitted: {
|
||||
item_index: number;
|
||||
kind: BrowserScenarioCapture["events"]["items"][number]["kind"];
|
||||
}[] = [];
|
||||
for (const item_index of group.source_item_indices) {
|
||||
const event = capture.events.items[item_index]!;
|
||||
if (kindSelected(event.kind)) events.push({ item_index, event });
|
||||
else omitted.push({ item_index, kind: event.kind });
|
||||
}
|
||||
return {
|
||||
transaction_id: group.transaction_id,
|
||||
source_item_indices: [...group.source_item_indices],
|
||||
events,
|
||||
omitted_events: omitted,
|
||||
};
|
||||
},
|
||||
);
|
||||
return {
|
||||
items: { collection: "transactions", rows: page.rows },
|
||||
coverage: page.coverage,
|
||||
};
|
||||
}
|
||||
const page = selectedPage(
|
||||
capture.events.items,
|
||||
view,
|
||||
(event) =>
|
||||
stepSelected(event.step_index) &&
|
||||
kindSelected(event.kind) &&
|
||||
(view.transaction_ids === undefined ||
|
||||
("transaction_id" in event &&
|
||||
transactionSelected(event.transaction_id))),
|
||||
(event, item_index) => ({ item_index, event }),
|
||||
);
|
||||
return {
|
||||
items: { collection: "events", rows: page.rows },
|
||||
coverage: page.coverage,
|
||||
};
|
||||
};
|
||||
|
||||
/** Project retained scenario Evidence without relaunching the browser or replaying actions. */
|
||||
export const projectBrowserScenarioView = (
|
||||
parent: AnalysisViewParent,
|
||||
view: AnalysisViewRequest,
|
||||
): Result<UnsignedAnalysisView, AnalysisError> => {
|
||||
if (view.kind !== "summary" && view.kind !== "browser-scenario")
|
||||
return err(unsupportedView());
|
||||
const parsed = readCapture(parent);
|
||||
if (!parsed.ok) return parsed;
|
||||
const capture = parsed.value;
|
||||
const shared = {
|
||||
parent_evidence_id: parent.evidenceId,
|
||||
parent_operation: OPERATION,
|
||||
parent_digest: parent.evidenceId.slice(3),
|
||||
artifact: null,
|
||||
limitations: [...parent.limitations],
|
||||
unknowns: [
|
||||
"Step states record one controlled replay; they do not establish the application's current state.",
|
||||
"Event step references record receipt attribution; they do not prove an action caused a transaction.",
|
||||
...(capture.events.dropped > 0
|
||||
? [
|
||||
`${String(capture.events.dropped)} events were dropped by capture budgets; counts and pages cover retained events only.`,
|
||||
]
|
||||
: []),
|
||||
],
|
||||
};
|
||||
if (view.kind === "summary")
|
||||
return ok({
|
||||
kind: "summary",
|
||||
view,
|
||||
...shared,
|
||||
summary: {
|
||||
browser: capture.browser,
|
||||
scenario: capture.scenario,
|
||||
duration_ms: capture.duration_ms,
|
||||
completeness: capture.completeness,
|
||||
steps: capture.steps.map((step, index) =>
|
||||
summaryStep(step, capture.steps[index - 1]),
|
||||
),
|
||||
final_accessibility: focusAccessibility(capture.steps),
|
||||
events: eventCounts(capture),
|
||||
network: summarizeBrowserScenarioNetwork(capture),
|
||||
limitation_count: shared.limitations.length,
|
||||
},
|
||||
coverage: completeWithinViewCoverage(1, 1),
|
||||
});
|
||||
if (
|
||||
view.collection === "text" &&
|
||||
capture.steps[view.step_index] === undefined
|
||||
)
|
||||
return err(
|
||||
analysisViewInputError([
|
||||
{
|
||||
path: ["view", "step_index"],
|
||||
reason: "invalid_value",
|
||||
message: "The selected step is not present in the retained capture.",
|
||||
},
|
||||
]),
|
||||
);
|
||||
return ok({
|
||||
kind: "browser-scenario",
|
||||
view,
|
||||
...shared,
|
||||
source_completeness: capture.completeness,
|
||||
source_events: {
|
||||
retained: capture.events.retained,
|
||||
dropped: capture.events.dropped,
|
||||
},
|
||||
source_network_omissions: {
|
||||
transactions: capture.events.omissions.network_transactions.length,
|
||||
transaction_range: capture.events.omissions.network_transaction_range,
|
||||
},
|
||||
...pageRecords(capture, view),
|
||||
});
|
||||
};
|
||||
@@ -1,250 +0,0 @@
|
||||
import { z } from "zod";
|
||||
import { digestSchema } from "../digests.js";
|
||||
import { browserScenarioNetworkSummarySchema } from "./browserScenarioNetworkView.js";
|
||||
import { sanitizedBrowserUrlSchema } from "../browserObservation.js";
|
||||
import { retainedBrowserScenarioCaptureSchema } from "../browserScenarioCapture.js";
|
||||
import {
|
||||
browserScenarioCompletenessSchema,
|
||||
browserScenarioEventKindSchema,
|
||||
browserScenarioEventSchema,
|
||||
browserStepArtifactsSchema,
|
||||
} from "../browserScenarioCaptureValues.js";
|
||||
|
||||
export const ARTIFACT_SECTIONS = browserStepArtifactsSchema.keyof().options;
|
||||
const artifactSectionSchema = browserStepArtifactsSchema.keyof();
|
||||
export type ArtifactSection = z.infer<typeof artifactSectionSchema>;
|
||||
|
||||
const countSchema = z.number().int().nonnegative();
|
||||
/** Section names by recorded state; not_requested sections are omitted. */
|
||||
export const stepSectionsSchema = z.strictObject({
|
||||
captured: z.array(artifactSectionSchema),
|
||||
missing: z.array(artifactSectionSchema),
|
||||
truncated: z.array(artifactSectionSchema),
|
||||
});
|
||||
|
||||
export const summaryStepSchema = z.strictObject({
|
||||
step_index: countSchema,
|
||||
step_id: z.string().min(1),
|
||||
action: z.string().min(1),
|
||||
status: z.enum(["completed", "failed", "cancelled"]),
|
||||
error: z.string().nullable(),
|
||||
elapsed_ms: countSchema,
|
||||
before_url: sanitizedBrowserUrlSchema
|
||||
.exactOptional()
|
||||
.describe(
|
||||
"Present only when it differs from the previous step's after_url (always for step 0).",
|
||||
),
|
||||
after_url: sanitizedBrowserUrlSchema,
|
||||
events: z.strictObject({
|
||||
sequence_start: z.number().int().min(1),
|
||||
sequence_end: countSchema,
|
||||
}),
|
||||
sections: stepSectionsSchema,
|
||||
completeness: z.enum(["complete", "incomplete", "truncated"]),
|
||||
});
|
||||
|
||||
export const focusAccessibilitySchema = z.strictObject({
|
||||
focus_step_index: countSchema,
|
||||
focus: z.enum(["first-unsuccessful-step", "last-step"]),
|
||||
/** The step whose tree is returned: the focus step, or the nearest earlier step that captured one. */
|
||||
step_index: countSchema.nullable(),
|
||||
accessibility: browserStepArtifactsSchema.shape.accessibility,
|
||||
});
|
||||
|
||||
/** Compact capture overview; complete records stay in the retained parent. */
|
||||
export const browserScenarioSummarySchema = z.strictObject({
|
||||
browser: retainedBrowserScenarioCaptureSchema.shape.browser,
|
||||
scenario: retainedBrowserScenarioCaptureSchema.shape.scenario,
|
||||
duration_ms: countSchema,
|
||||
completeness: browserScenarioCompletenessSchema,
|
||||
steps: z.array(summaryStepSchema),
|
||||
final_accessibility: focusAccessibilitySchema,
|
||||
events: z.strictObject({
|
||||
retained: countSchema,
|
||||
dropped: countSchema,
|
||||
by_kind: z.array(
|
||||
z.strictObject({
|
||||
kind: browserScenarioEventKindSchema,
|
||||
count: countSchema,
|
||||
}),
|
||||
),
|
||||
omissions: z.strictObject({
|
||||
by_kind:
|
||||
retainedBrowserScenarioCaptureSchema.shape.events.shape.omissions.shape
|
||||
.by_kind,
|
||||
network_transactions: countSchema,
|
||||
network_transaction_range:
|
||||
retainedBrowserScenarioCaptureSchema.shape.events.shape.omissions.shape
|
||||
.network_transaction_range,
|
||||
}),
|
||||
}),
|
||||
network: browserScenarioNetworkSummarySchema,
|
||||
limitation_count: countSchema.describe(
|
||||
"Number of parent Evidence limitations returned with this summary, including inherited provenance caveats.",
|
||||
),
|
||||
});
|
||||
|
||||
const pageShape = {
|
||||
kind: z.literal("browser-scenario"),
|
||||
offset: countSchema.default(0),
|
||||
limit: z.int().positive().default(64),
|
||||
};
|
||||
const stepSelection = {
|
||||
step_indices: z
|
||||
.array(countSchema)
|
||||
.optional()
|
||||
.describe(
|
||||
"Exact recorded step indices; omitted selects all, [] selects none.",
|
||||
),
|
||||
};
|
||||
const transactionSelection = {
|
||||
transaction_ids: z
|
||||
.array(z.string().min(1))
|
||||
.optional()
|
||||
.describe(
|
||||
"Exact recorded transaction IDs; omitted selects all, [] selects none.",
|
||||
),
|
||||
};
|
||||
|
||||
/** Collection-specific fields are validated rather than silently ignored. */
|
||||
export const browserScenarioViewRequestSchema = z.discriminatedUnion(
|
||||
"collection",
|
||||
[
|
||||
z.strictObject({
|
||||
...pageShape,
|
||||
...stepSelection,
|
||||
collection: z.literal("steps"),
|
||||
artifacts: z
|
||||
.array(artifactSectionSchema)
|
||||
.default([])
|
||||
.describe(
|
||||
"Return only selected artifact payloads; omitted_artifacts names the other recorded sections.",
|
||||
),
|
||||
}),
|
||||
z.strictObject({
|
||||
...pageShape,
|
||||
...stepSelection,
|
||||
...transactionSelection,
|
||||
collection: z.literal("events"),
|
||||
event_kinds: z
|
||||
.array(browserScenarioEventKindSchema)
|
||||
.optional()
|
||||
.describe("Omitted selects all event kinds; [] selects none."),
|
||||
}),
|
||||
z.strictObject({
|
||||
...pageShape,
|
||||
...stepSelection,
|
||||
...transactionSelection,
|
||||
collection: z.literal("transactions"),
|
||||
event_kinds: z
|
||||
.array(browserScenarioEventKindSchema)
|
||||
.default([
|
||||
"request",
|
||||
"response",
|
||||
"request-finished",
|
||||
"request-failed",
|
||||
"request-unfinished",
|
||||
])
|
||||
.describe(
|
||||
"Metadata events by default; select network-content explicitly for bodies and header values. [] returns identity and source indices only.",
|
||||
),
|
||||
}),
|
||||
z.strictObject({
|
||||
...pageShape,
|
||||
collection: z.literal("text"),
|
||||
step_index: countSchema,
|
||||
artifact: z.enum(["accessibility", "dom"]),
|
||||
limit: z
|
||||
.int()
|
||||
.positive()
|
||||
.default(2048)
|
||||
.describe("Caller-selected UTF-16 code units, matching offset."),
|
||||
}),
|
||||
],
|
||||
);
|
||||
export type BrowserScenarioViewRequest = z.output<
|
||||
typeof browserScenarioViewRequestSchema
|
||||
>;
|
||||
|
||||
/** One exact step whose unselected captured sections are named rather than returned. */
|
||||
export const stepRowSchema = z.strictObject({
|
||||
step_index: countSchema,
|
||||
step_id: z.string().min(1),
|
||||
action: z.string().min(1),
|
||||
status: z.enum(["completed", "failed", "cancelled"]),
|
||||
error: z.string().nullable(),
|
||||
elapsed_ms: countSchema,
|
||||
before_url: sanitizedBrowserUrlSchema,
|
||||
after_url: sanitizedBrowserUrlSchema,
|
||||
event_sequence_start: z.number().int().min(1),
|
||||
event_sequence_end: countSchema,
|
||||
completeness: browserScenarioCompletenessSchema,
|
||||
artifacts: browserStepArtifactsSchema.partial(),
|
||||
omitted_artifacts: z.array(
|
||||
z.strictObject({
|
||||
section: artifactSectionSchema,
|
||||
state: z.enum(["captured", "missing", "truncated"]),
|
||||
}),
|
||||
),
|
||||
});
|
||||
const eventRowSchema = z.strictObject({
|
||||
item_index: countSchema,
|
||||
event: browserScenarioEventSchema,
|
||||
});
|
||||
|
||||
export const browserScenarioViewItemsSchema = z.discriminatedUnion(
|
||||
"collection",
|
||||
[
|
||||
z.strictObject({
|
||||
collection: z.literal("steps"),
|
||||
rows: z.array(stepRowSchema),
|
||||
}),
|
||||
z.strictObject({
|
||||
collection: z.literal("events"),
|
||||
rows: z.array(eventRowSchema),
|
||||
}),
|
||||
z.strictObject({
|
||||
collection: z.literal("transactions"),
|
||||
rows: z.array(
|
||||
z.strictObject({
|
||||
transaction_id: z.string().min(1),
|
||||
source_item_indices: z.array(countSchema),
|
||||
events: z.array(eventRowSchema),
|
||||
omitted_events: z.array(
|
||||
z.strictObject({
|
||||
item_index: countSchema,
|
||||
kind: browserScenarioEventKindSchema,
|
||||
}),
|
||||
),
|
||||
}),
|
||||
),
|
||||
}),
|
||||
z.strictObject({
|
||||
collection: z.literal("text"),
|
||||
step_index: countSchema,
|
||||
artifact: z.enum(["accessibility", "dom"]),
|
||||
state: z.enum(["captured", "missing", "truncated", "not_requested"]),
|
||||
reason: z.string().nullable(),
|
||||
sha256: digestSchema.nullable(),
|
||||
bytes: countSchema.nullable(),
|
||||
observed: countSchema.nullable(),
|
||||
retained: countSchema.nullable(),
|
||||
text: z.string().nullable(),
|
||||
}),
|
||||
],
|
||||
);
|
||||
|
||||
/** Capture-wide facts every page repeats so exhaustion is not read as completeness. */
|
||||
export const browserScenarioViewContextSchema = z.strictObject({
|
||||
source_completeness: browserScenarioCompletenessSchema,
|
||||
source_events: z.strictObject({
|
||||
retained: countSchema,
|
||||
dropped: countSchema,
|
||||
}),
|
||||
source_network_omissions: z.strictObject({
|
||||
transactions: countSchema,
|
||||
transaction_range:
|
||||
retainedBrowserScenarioCaptureSchema.shape.events.shape.omissions.shape
|
||||
.network_transaction_range,
|
||||
}),
|
||||
});
|
||||
@@ -350,20 +350,7 @@ const writePlanSchema = z.strictObject({
|
||||
.nullable()
|
||||
.describe("The first successful recorded call the plan was built from."),
|
||||
observation: z
|
||||
.strictObject({
|
||||
ordinal: ordinalSchema,
|
||||
pointer: z.string(),
|
||||
read: z
|
||||
.strictObject({
|
||||
ordinal: ordinalSchema,
|
||||
pointer: z.string(),
|
||||
shape: z.enum(["item", "list"]),
|
||||
})
|
||||
.exactOptional()
|
||||
.describe(
|
||||
"A create's later read of the entity: an item read lends fields the response lacks and its clock spellings, a list read only adds fields the response lacks.",
|
||||
),
|
||||
})
|
||||
.strictObject({ ordinal: ordinalSchema, pointer: z.string() })
|
||||
.nullable()
|
||||
.describe(
|
||||
"The record whose response showed the written entity or document, and a JSON pointer to it.",
|
||||
|
||||
@@ -33,7 +33,6 @@ import {
|
||||
echoed,
|
||||
integerOf,
|
||||
readClock,
|
||||
respellClock,
|
||||
isUuid,
|
||||
replaceDelimited,
|
||||
slugOf,
|
||||
@@ -489,12 +488,7 @@ export const optionalFields = (
|
||||
export const writtenResponse = (
|
||||
response: WriteResponse,
|
||||
recorded: JsonValue | undefined,
|
||||
written: {
|
||||
readonly fields: JsonObject | undefined;
|
||||
readonly id: string;
|
||||
/** Generated clocks keep the response's spelling independently of the stored entity's. */
|
||||
readonly clocks?: readonly string[];
|
||||
},
|
||||
written: { readonly fields: JsonObject | undefined; readonly id: string },
|
||||
optional: readonly string[] = [],
|
||||
): Applied<JsonValue | undefined> => {
|
||||
if (response.shape === "empty") return ok(undefined);
|
||||
@@ -531,17 +525,7 @@ export const writtenResponse = (
|
||||
recorded,
|
||||
pointer,
|
||||
Object.fromEntries(
|
||||
fields.map((field) => {
|
||||
const value = written.fields![field]!;
|
||||
const clock =
|
||||
written.clocks?.includes(field) === true
|
||||
? readClock(shown[field])
|
||||
: undefined;
|
||||
return [
|
||||
field,
|
||||
clock === undefined ? value : respellClock(value, clock.format),
|
||||
];
|
||||
}),
|
||||
fields.map((field) => [field, written.fields![field]!]),
|
||||
),
|
||||
),
|
||||
);
|
||||
|
||||
@@ -104,35 +104,7 @@ export interface UpdateCall {
|
||||
readonly kept: readonly string[];
|
||||
}
|
||||
|
||||
/** Ambiguous saves may echo an already-derived slug or hold a clock in its own second. Keep the demonstrated derivation; call evidence still has to reproduce it. */
|
||||
export const reconciledUpdateRule = (
|
||||
held: FieldRule,
|
||||
later: FieldRule,
|
||||
kept: ReadonlySet<string>,
|
||||
): FieldRule | undefined => {
|
||||
const alike = alikeRules(held, later, kept);
|
||||
if (alike !== undefined) return alike;
|
||||
for (const [derived, ambiguous] of [
|
||||
[held, later],
|
||||
[later, held],
|
||||
] as const) {
|
||||
if (
|
||||
derived.rule === "slug" &&
|
||||
ambiguous.rule === "echo" &&
|
||||
ambiguous.sorted !== true
|
||||
)
|
||||
return derived;
|
||||
if (
|
||||
derived.rule === "clock" &&
|
||||
ambiguous.rule === "held-clock" &&
|
||||
same(derived.format, ambiguous.format)
|
||||
)
|
||||
return derived;
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/** Whether a later rule reproduces earlier calls, including echoes promoted to derivations. */
|
||||
/** Whether a rule a later call showed gives an earlier call a value other than the one it showed for a field it did not send. */
|
||||
export const contradictsEarlierCalls = (
|
||||
draft: {
|
||||
readonly plan: { readonly rules: readonly FieldRule[] };
|
||||
@@ -140,12 +112,11 @@ export const contradictsEarlierCalls = (
|
||||
},
|
||||
rules: readonly FieldRule[],
|
||||
): boolean => {
|
||||
const ruled = new Map(draft.plan.rules.map((rule) => [rule.field, rule]));
|
||||
const ruled = new Set(draft.plan.rules.map((rule) => rule.field));
|
||||
return rules.some((rule) => {
|
||||
if (
|
||||
!(unsentChange(rule) || regenerates(rule) || rule.rule === "clock") ||
|
||||
(ruled.has(rule.field) &&
|
||||
!(rule.rule === "slug" && ruled.get(rule.field)?.rule === "echo"))
|
||||
ruled.has(rule.field)
|
||||
)
|
||||
return false;
|
||||
// Compute a structured rule's digest once, rather than once per earlier call.
|
||||
@@ -177,12 +148,9 @@ export const contradictsEarlierCalls = (
|
||||
);
|
||||
}
|
||||
const from = rule.rule === "slug" ? sent.get(rule.from) : undefined;
|
||||
// A slug rule runs only when its source is sent; otherwise serving keeps the field.
|
||||
return (
|
||||
rule.rule === "slug" &&
|
||||
(typeof from === "string"
|
||||
? !sameObserved(observation, observedValue(slugOf(from)))
|
||||
: !kept.includes(rule.field))
|
||||
typeof from === "string" &&
|
||||
!sameObserved(observation, observedValue(slugOf(from)))
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -237,10 +205,7 @@ export const conflictsWithPlan = (
|
||||
(identity !== null && fixed.has(identity)) ||
|
||||
explained.rules.some((rule) => {
|
||||
const held = rules.get(rule.field);
|
||||
return (
|
||||
held !== undefined &&
|
||||
reconciledUpdateRule(held, rule, kept) === undefined
|
||||
);
|
||||
return held !== undefined && alikeRules(held, rule, kept) === undefined;
|
||||
})
|
||||
);
|
||||
};
|
||||
|
||||
@@ -9,7 +9,6 @@ import type { FieldRule, WriteCall } from "./environmentStateWriteRules.js";
|
||||
import {
|
||||
integerOf,
|
||||
readClock,
|
||||
slugOf,
|
||||
sortedStrings,
|
||||
} from "./environmentStateWriteValues.js";
|
||||
import { freshUuid, renewedRule } from "./environmentStateWriteGenerated.js";
|
||||
@@ -140,35 +139,7 @@ export const sameSecondClocks = (evidence: UpdateEvidence): string[] =>
|
||||
inCallSecond(evidence.after[field], evidence.startMs),
|
||||
);
|
||||
|
||||
/** Another sent member stored as sent whose slug the application stored in place of what this member sent. */
|
||||
const sentSlugSource = (
|
||||
evidence: UpdateEvidence,
|
||||
member: string,
|
||||
): string | undefined => {
|
||||
const stored = evidence.after[member];
|
||||
if (typeof stored !== "string") return undefined;
|
||||
return Object.entries(evidence.call.body).find(
|
||||
([other, value]) =>
|
||||
other !== member &&
|
||||
typeof value === "string" &&
|
||||
same(evidence.after[other], value) &&
|
||||
slugOf(value) === stored,
|
||||
)?.[0];
|
||||
};
|
||||
|
||||
/** Whether a call sent back the clock the entity held, as a whole-object write does, and the application stamped it anew near the call. */
|
||||
const restampedClock = (
|
||||
evidence: UpdateEvidence,
|
||||
member: string,
|
||||
sent: JsonValue,
|
||||
): boolean =>
|
||||
!CREATION_CLOCK.test(member) &&
|
||||
readClock(sent) !== undefined &&
|
||||
same(evidence.before[member], sent) &&
|
||||
readClock(evidence.after[member]) !== undefined &&
|
||||
isClockNear(evidence.after[member]!, evidence.startMs);
|
||||
|
||||
/** How an update stored a sent member: stamped anew as the one held clock it sent back other than a creation clock, echoed, stamped anew or slugged from another member in place of what it sent, renewed as a fresh UUID, or as a constant. */
|
||||
/** How an update stored a sent member: stamped anew as the one held clock it sent back other than a creation clock, echoed, renewed as a fresh UUID, or as a constant. */
|
||||
export const sentRule = (
|
||||
evidence: UpdateEvidence,
|
||||
member: string,
|
||||
@@ -185,14 +156,6 @@ export const sentRule = (
|
||||
return { field: member, rule: "echo" };
|
||||
if (same(evidence.after[member], sortedStrings(sent)))
|
||||
return { field: member, rule: "echo", sorted: true };
|
||||
if (restampedClock(evidence, member, sent))
|
||||
return {
|
||||
field: member,
|
||||
rule: "clock",
|
||||
format: readClock(evidence.after[member])!.format,
|
||||
};
|
||||
const from = sentSlugSource(evidence, member);
|
||||
if (from !== undefined) return { field: member, rule: "slug", from };
|
||||
return freshUuid(evidence, member)
|
||||
? { field: member, rule: "unique" }
|
||||
: { field: member, rule: "constant", value: evidence.after[member]! };
|
||||
|
||||
@@ -61,8 +61,6 @@ import {
|
||||
import {
|
||||
laterObservation,
|
||||
newEntityIn,
|
||||
respelledInstant,
|
||||
withRead,
|
||||
type Observed,
|
||||
} from "./environmentStateWriteObserve.js";
|
||||
import {
|
||||
@@ -99,57 +97,6 @@ const respondedId = (
|
||||
return members.length === 1 ? scalarText(members[0]) : undefined;
|
||||
};
|
||||
|
||||
/**
|
||||
* A create's answer with the first read of its entity before the next write over it, when that read shows fields the
|
||||
* answer lacks or spells its clocks to another precision: the state holds what reads show. An item read that agrees
|
||||
* with the answer otherwise lends it its fields and then observes rather than checks; one that disagrees still checks
|
||||
* the create. A list read, which may show fewer keys of an object, only adds fields the answer lacks and still checks.
|
||||
*/
|
||||
const withReadBack = (
|
||||
context: EffectContext,
|
||||
step: WriteStep,
|
||||
target: {
|
||||
readonly type: string;
|
||||
readonly collection: string;
|
||||
readonly idField: string;
|
||||
},
|
||||
own: Observed,
|
||||
): Observed => {
|
||||
const later = laterObservation(searchFrom(context, step), {
|
||||
type: target.type,
|
||||
collection: target.collection,
|
||||
idField: target.idField,
|
||||
id: own.id,
|
||||
held: () => false,
|
||||
isWrite: context.isWrite,
|
||||
});
|
||||
if (later === undefined) return own;
|
||||
const view = context.viewOf(later.ordinal);
|
||||
if (
|
||||
view?.shape === "item" &&
|
||||
(view.fields.some(
|
||||
(field) =>
|
||||
Object.hasOwn(own.object, field) && !Object.hasOwn(later.object, field),
|
||||
) ||
|
||||
Object.entries(later.object).some(
|
||||
([field, value]) =>
|
||||
Object.hasOwn(own.object, field) &&
|
||||
!same(own.object[field], value) &&
|
||||
!respelledInstant(own.object[field], value),
|
||||
))
|
||||
)
|
||||
return own;
|
||||
const shape = view?.shape === "item" ? "item" : "list";
|
||||
const object = withRead(own.object, later.object, shape);
|
||||
if (same(object, own.object)) return own;
|
||||
if (shape === "item") context.observed(later.ordinal);
|
||||
return {
|
||||
...own,
|
||||
object,
|
||||
read: { ordinal: later.ordinal, pointer: later.pointer, shape },
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* The written entity a create's response or a later read shows. A linked create's read must name an
|
||||
* id its response returned, as the read that linked it did, envelopes included.
|
||||
@@ -169,7 +116,7 @@ const createdObservation = (
|
||||
const own = context.io.withBody(step.record.ordinal, (body) =>
|
||||
newEntityIn(step.record.ordinal, body, target.idField, held),
|
||||
);
|
||||
if (own !== undefined) return withReadBack(context, step, target, own);
|
||||
if (own !== undefined) return own;
|
||||
const { id, ids } = context.io.withBody(step.record.ordinal, (body) => {
|
||||
const returned = target.linked ? respondedIds(body) : [];
|
||||
// A response that names the created id alone ties the later read to that id.
|
||||
@@ -373,9 +320,6 @@ const buildCreate = (
|
||||
observation: {
|
||||
ordinal: evidence.observed.ordinal,
|
||||
pointer: evidence.observed.pointer,
|
||||
...(evidence.observed.read === undefined
|
||||
? {}
|
||||
: { read: evidence.observed.read }),
|
||||
},
|
||||
response: context.io.withBody(step.record.ordinal, (body) =>
|
||||
responseShapeOf(
|
||||
|
||||
@@ -39,7 +39,6 @@ import {
|
||||
type UpdateCall,
|
||||
conflictsWithPlan,
|
||||
refinedBody,
|
||||
reconciledUpdateRule,
|
||||
} from "./environmentStateWriteCalls.js";
|
||||
import {
|
||||
laterObservation,
|
||||
@@ -59,6 +58,7 @@ import {
|
||||
observationNeeds,
|
||||
} from "./environmentStateWriteAddress.js";
|
||||
import {
|
||||
alikeRules,
|
||||
keptOrders,
|
||||
pathValuesOf,
|
||||
siblingTrimmedClocks,
|
||||
@@ -394,7 +394,7 @@ const refineUpdate = (
|
||||
const shown = later.get(rule.field);
|
||||
return shown === undefined
|
||||
? rule
|
||||
: (reconciledUpdateRule(rule, shown, draft.kept) ?? rule);
|
||||
: (alikeRules(rule, shown, draft.kept) ?? rule);
|
||||
}),
|
||||
...evidence.explained.rules.filter((rule) => !ruled.has(rule.field)),
|
||||
];
|
||||
@@ -703,11 +703,7 @@ const contradictsGeneratedClock = (
|
||||
if (
|
||||
rule.rule !== "clock" ||
|
||||
!Object.hasOwn(after, rule.field) ||
|
||||
explained.unstamped.includes(rule.field) ||
|
||||
explained.rules.some(
|
||||
(candidate) =>
|
||||
candidate.field === rule.field && candidate.rule === "held-clock",
|
||||
)
|
||||
explained.unstamped.includes(rule.field)
|
||||
)
|
||||
return false;
|
||||
const previous = readClock(before[rule.field]);
|
||||
|
||||
@@ -15,41 +15,6 @@ import {
|
||||
type StateView,
|
||||
} from "./environmentStateRules.js";
|
||||
import { queryValues, replayReason } from "./environmentStateVariants.js";
|
||||
import { readClock } from "./environmentStateWriteValues.js";
|
||||
|
||||
/** A read a create's observation takes fields from: an item read's respell the response's clocks and add fields, a list read's only add fields. */
|
||||
export interface ObservedRead {
|
||||
readonly ordinal: number;
|
||||
readonly pointer: string;
|
||||
readonly shape: "item" | "list";
|
||||
}
|
||||
|
||||
/** Whether two clocks agree at the coarser recorded fraction precision, retaining all digits through nanoseconds. */
|
||||
export const respelledInstant = (
|
||||
left: JsonValue | undefined,
|
||||
right: JsonValue | undefined,
|
||||
): boolean => {
|
||||
const [a, b] = [readClock(left), readClock(right)];
|
||||
if (a?.format.kind !== "iso" || b?.format.kind !== "iso") return false;
|
||||
const [fine, coarse] =
|
||||
a.format.fraction >= b.format.fraction ? [a, b] : [b, a];
|
||||
if (coarse.format.kind !== "iso") return false;
|
||||
const unit = 10 ** (9 - coarse.format.fraction);
|
||||
const fraction = (clock: typeof fine) =>
|
||||
(((clock.ms % 1000) + 1000) % 1000) * 1_000_000 + clock.subMillisecond;
|
||||
return (
|
||||
Math.floor(fine.ms / 1000) === Math.floor(coarse.ms / 1000) &&
|
||||
Math.floor(fraction(fine) / unit) * unit === fraction(coarse)
|
||||
);
|
||||
};
|
||||
|
||||
/** A create's response object with a read's fields: an item read's values win, a list read only adds fields the response lacks. */
|
||||
export const withRead = (
|
||||
response: JsonObject,
|
||||
read: JsonObject,
|
||||
shape: ObservedRead["shape"],
|
||||
): JsonObject =>
|
||||
shape === "item" ? { ...response, ...read } : { ...read, ...response };
|
||||
|
||||
/** Where an observed entity object was found: the record and a JSON pointer into its response. */
|
||||
export interface Observed {
|
||||
@@ -57,8 +22,6 @@ export interface Observed {
|
||||
readonly pointer: string;
|
||||
readonly object: JsonObject;
|
||||
readonly id: string;
|
||||
/** A create's later read whose fields the object adds to the response's, and whose clock spellings an item read lends it. */
|
||||
readonly read?: ObservedRead;
|
||||
}
|
||||
|
||||
/** Objects a response carries where an entity may be, in precedence tiers: the root, one member, then the element of a one-element array at the root or in one member. */
|
||||
|
||||
@@ -253,15 +253,10 @@ export interface StateWritePlan {
|
||||
readonly served: boolean;
|
||||
readonly reason: WriteReason | null;
|
||||
readonly exemplar_ordinal: number | null;
|
||||
/** The record whose response showed the written entity or document, and where; a create's later read whose fields the entity adds to the response's. */
|
||||
/** The record whose response showed the written entity or document, and where. */
|
||||
readonly observation: {
|
||||
readonly ordinal: number;
|
||||
readonly pointer: string;
|
||||
readonly read?: {
|
||||
readonly ordinal: number;
|
||||
readonly pointer: string;
|
||||
readonly shape: "item" | "list";
|
||||
};
|
||||
} | null;
|
||||
readonly response: WriteResponse | null;
|
||||
readonly body: WriteBody | null;
|
||||
|
||||
@@ -1,172 +0,0 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { JsonValue } from "../jsonValue.js";
|
||||
|
||||
import {
|
||||
timedCaptureEvidence,
|
||||
type TimedCaptureRow,
|
||||
} from "../../../tests/fixtures/analysisView.js";
|
||||
import {
|
||||
LISTED_LABEL_ROWS,
|
||||
READ_BACK_TAG_ROWS,
|
||||
RESTAMPED_ROWS,
|
||||
STALE_SLUG_ROWS,
|
||||
clockSaveRows,
|
||||
slugSaveRows,
|
||||
slugOnlySaveRows,
|
||||
} from "../../../tests/fixtures/environmentStateWriteViews.js";
|
||||
import { environmentStateViewOf } from "../../../tests/fixtures/environmentStateWrites.js";
|
||||
|
||||
const planOf = (rows: readonly TimedCaptureRow[], tool: string) => {
|
||||
const plan = environmentStateViewOf(timedCaptureEvidence(rows)).writes.find(
|
||||
(candidate) => candidate.tool === tool,
|
||||
);
|
||||
if (plan === undefined) throw new Error(`No plan for ${tool}`);
|
||||
return plan;
|
||||
};
|
||||
|
||||
describe("creates whose answer shows the entity otherwise than reads do", () => {
|
||||
it.each(["123456", "123457"])(
|
||||
"checks read-back precision beyond milliseconds (%s)",
|
||||
(fraction) => {
|
||||
const rows = READ_BACK_TAG_ROWS.map(
|
||||
([method, path, body = null, ...rest]): TimedCaptureRow => {
|
||||
const rewrite = (value: JsonValue): JsonValue => {
|
||||
if (Array.isArray(value)) return value.map(rewrite);
|
||||
if (value !== null && typeof value === "object")
|
||||
return Object.fromEntries(
|
||||
Object.entries(value).map(([key, child]) => [
|
||||
key,
|
||||
key === "created" && typeof child === "string"
|
||||
? child.replace(
|
||||
/(?:\.\d+)?Z$/u,
|
||||
method === "POST" ? ".123456000Z" : `.${fraction}Z`,
|
||||
)
|
||||
: child,
|
||||
]),
|
||||
);
|
||||
return value;
|
||||
};
|
||||
return [method, path, rewrite(body), ...rest];
|
||||
},
|
||||
);
|
||||
expect(planOf(rows, "post_tags").served).toBe(fraction === "123456");
|
||||
},
|
||||
);
|
||||
|
||||
it("keeps an item read's omission as contradictory evidence even when a later list confirms the added fields", () => {
|
||||
const rows = READ_BACK_TAG_ROWS.map(
|
||||
([method, path, body = null, ...rest]): TimedCaptureRow => [
|
||||
method,
|
||||
path,
|
||||
body !== null &&
|
||||
typeof body === "object" &&
|
||||
!Array.isArray(body) &&
|
||||
(method === "POST" || path === "/api/tags/1" || path === "/api/tags/2")
|
||||
? { ...body, detail: "Only in the answer" }
|
||||
: body,
|
||||
...rest,
|
||||
],
|
||||
);
|
||||
expect(planOf(rows, "post_tags")).toMatchObject({
|
||||
served: false,
|
||||
reason: "contradicted",
|
||||
});
|
||||
});
|
||||
it("serves a create whose item read adds a count and spells the clock otherwise, taking the read's fields", () => {
|
||||
expect(planOf(READ_BACK_TAG_ROWS, "post_tags")).toMatchObject({
|
||||
served: true,
|
||||
observation: {
|
||||
ordinal: 3,
|
||||
pointer: "",
|
||||
read: { ordinal: 4, pointer: "", shape: "item" },
|
||||
},
|
||||
rules: expect.arrayContaining([
|
||||
{
|
||||
field: "created",
|
||||
rule: "clock",
|
||||
format: expect.objectContaining({ fraction: 0 }),
|
||||
},
|
||||
]),
|
||||
});
|
||||
});
|
||||
|
||||
it("serves a create whose list read adds a count the answer lacks", () => {
|
||||
expect(planOf(LISTED_LABEL_ROWS, "post_labels")).toMatchObject({
|
||||
served: true,
|
||||
observation: { read: { ordinal: 2, pointer: "/2", shape: "list" } },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("whole-object updates that send back what the application derives", () => {
|
||||
it.each([
|
||||
[true, true],
|
||||
[false, true],
|
||||
[true, false],
|
||||
[false, false],
|
||||
] as const)(
|
||||
"checks slug-only saves too (derived first: %s, changed: %s)",
|
||||
(derivedFirst, changed) => {
|
||||
expect(
|
||||
planOf(slugOnlySaveRows(derivedFirst, changed), "patch_tags_by_id")
|
||||
.served,
|
||||
).toBe(!changed);
|
||||
},
|
||||
);
|
||||
it.each([true, false])(
|
||||
"reconciles slug saves in either order (derived first: %s)",
|
||||
(derivedFirst) => {
|
||||
expect(
|
||||
planOf(slugSaveRows(derivedFirst), "put_tags_by_id"),
|
||||
).toMatchObject({
|
||||
served: true,
|
||||
rules: expect.arrayContaining([
|
||||
{ field: "slug", rule: "slug", from: "name" },
|
||||
]),
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it.each([true, false])(
|
||||
"rejects a slug rule contradicted by an explicit custom slug (derived first: %s)",
|
||||
(derivedFirst) => {
|
||||
expect(
|
||||
planOf(slugSaveRows(derivedFirst, true), "put_tags_by_id").served,
|
||||
).toBe(false);
|
||||
},
|
||||
);
|
||||
|
||||
it.each([true, false])(
|
||||
"reconciles held and fresh clocks in either order (derived first: %s)",
|
||||
(derivedFirst) => {
|
||||
expect(
|
||||
planOf(clockSaveRows(derivedFirst), "put_posts_by_id"),
|
||||
).toMatchObject({
|
||||
served: true,
|
||||
rules: expect.arrayContaining([
|
||||
expect.objectContaining({ field: "updated_at", rule: "clock" }),
|
||||
]),
|
||||
});
|
||||
},
|
||||
);
|
||||
it("slugs a sent slug anew from the name the call stored", () => {
|
||||
expect(planOf(STALE_SLUG_ROWS, "put_tags_by_id")).toMatchObject({
|
||||
served: true,
|
||||
rules: expect.arrayContaining([
|
||||
{ field: "slug", rule: "slug", from: "name" },
|
||||
{ field: "name", rule: "echo" },
|
||||
]),
|
||||
body: { accepted: expect.arrayContaining(["slug", "name"]) },
|
||||
});
|
||||
});
|
||||
|
||||
it("stamps anew a modification time each call sends back as it read it", () => {
|
||||
expect(planOf(RESTAMPED_ROWS, "put_posts_by_id")).toMatchObject({
|
||||
served: true,
|
||||
rules: expect.arrayContaining([
|
||||
expect.objectContaining({ field: "updated_at", rule: "clock" }),
|
||||
]),
|
||||
body: { accepted: expect.arrayContaining(["updated_at"]) },
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -7,7 +7,6 @@ import {
|
||||
|
||||
const symbol = (name: string) => ({
|
||||
name,
|
||||
runtime_name: name.slice(1),
|
||||
address: null,
|
||||
weak: null,
|
||||
reexport: null,
|
||||
@@ -112,7 +111,6 @@ it("preserves ambiguous import facts without adding import fields to exports", (
|
||||
const imported = {
|
||||
...symbol("unused"),
|
||||
name: null,
|
||||
runtime_name: null,
|
||||
raw_name: "_value+0x4",
|
||||
name_status: "ambiguous",
|
||||
addend: null,
|
||||
|
||||
@@ -1,86 +0,0 @@
|
||||
import type { PositionalRead } from "../zipPackageFormat.js";
|
||||
|
||||
const UDIF_TRAILER_BYTES = 512;
|
||||
const UDIF_TRAILER_SIGNATURE = Buffer.from("koly");
|
||||
const GPT_HEADER_SIGNATURE = Buffer.from("EFI PART");
|
||||
const GPT_REVISION_1_0 = 0x0001_0000;
|
||||
const HFS_PLUS_HEADER_END = 1028;
|
||||
/** Leading bytes covering every raw layout signature below. */
|
||||
const RAW_SIGNATURE_PREFIX_BYTES = HFS_PLUS_HEADER_END;
|
||||
|
||||
/**
|
||||
* Recognize Apple disk images by structure, not by name or mountability.
|
||||
*
|
||||
* UDIF images carry a `koly` trailer. Raw images (UDRW, UDTO and other
|
||||
* `CRawDiskImage` formats) are plain device bytes, so they are recognized by
|
||||
* the partition map or the HFS+ volume header that `hdiutil` writes. This
|
||||
* establishes neither integrity nor that the image will mount. `read` must
|
||||
* fill each range; a short result means end of file.
|
||||
*/
|
||||
export const hasDiskImageSignature = async (
|
||||
size: number,
|
||||
read: PositionalRead,
|
||||
): Promise<boolean> => {
|
||||
if (!Number.isSafeInteger(size) || size <= 0) return false;
|
||||
if (size >= UDIF_TRAILER_BYTES) {
|
||||
const trailer = await read(
|
||||
size - UDIF_TRAILER_BYTES,
|
||||
UDIF_TRAILER_SIGNATURE.length,
|
||||
);
|
||||
if (Buffer.from(trailer).equals(UDIF_TRAILER_SIGNATURE)) return true;
|
||||
}
|
||||
// Read the exact window here so every caller observes the same bytes.
|
||||
return hasRawDiskImageSignature(
|
||||
Buffer.from(await read(0, Math.min(size, RAW_SIGNATURE_PREFIX_BYTES))),
|
||||
);
|
||||
};
|
||||
|
||||
const hasRawDiskImageSignature = (prefix: Buffer): boolean =>
|
||||
hasGptHeader(prefix) ||
|
||||
hasApplePartitionMap(prefix) ||
|
||||
hasMbrPartitionTable(prefix) ||
|
||||
hasHfsPlusVolumeHeader(prefix);
|
||||
|
||||
/** GUID partition table header at LBA 1 of a 512-byte-sector device. */
|
||||
const hasGptHeader = (prefix: Buffer): boolean => {
|
||||
if (prefix.length < 528) return false;
|
||||
if (!prefix.subarray(512, 520).equals(GPT_HEADER_SIGNATURE)) return false;
|
||||
const headerSize = prefix.readUInt32LE(524);
|
||||
return (
|
||||
prefix.readUInt32LE(520) === GPT_REVISION_1_0 &&
|
||||
headerSize >= 92 &&
|
||||
headerSize <= 512
|
||||
);
|
||||
};
|
||||
|
||||
/** Driver descriptor block (`ER`) followed by a partition map entry (`PM`). */
|
||||
const hasApplePartitionMap = (prefix: Buffer): boolean =>
|
||||
prefix.length >= 514 &&
|
||||
prefix[0] === 0x45 &&
|
||||
prefix[1] === 0x52 &&
|
||||
prefix[512] === 0x50 &&
|
||||
prefix[513] === 0x4d;
|
||||
|
||||
/** Boot signature plus a well-formed table with at least one used entry. */
|
||||
const hasMbrPartitionTable = (prefix: Buffer): boolean => {
|
||||
if (prefix.length < 512 || prefix[510] !== 0x55 || prefix[511] !== 0xaa)
|
||||
return false;
|
||||
let used = false;
|
||||
for (let entry = 446; entry < 510; entry += 16) {
|
||||
const status = prefix[entry];
|
||||
if (status !== 0x00 && status !== 0x80) return false;
|
||||
used ||= prefix[entry + 4] !== 0 && prefix.readUInt32LE(entry + 12) !== 0;
|
||||
}
|
||||
return used;
|
||||
};
|
||||
|
||||
/** Unpartitioned HFS+ (`H+`, version 4) or HFSX (`HX`, version 5) volume. */
|
||||
const hasHfsPlusVolumeHeader = (prefix: Buffer): boolean => {
|
||||
if (prefix.length < HFS_PLUS_HEADER_END || prefix[1024] !== 0x48)
|
||||
return false;
|
||||
const version = prefix.readUInt16BE(1026);
|
||||
return (
|
||||
(prefix[1025] === 0x2b && version === 4) ||
|
||||
(prefix[1025] === 0x58 && version === 5)
|
||||
);
|
||||
};
|
||||
@@ -324,20 +324,17 @@ const buildArtifactChangeBase = (input: ClassifyPathInput) => ({
|
||||
evidence_links: [...input.evidenceLinks],
|
||||
});
|
||||
|
||||
const classifyPresence = (
|
||||
input: ClassifyPathInput,
|
||||
base: ReturnType<typeof buildArtifactChangeBase>,
|
||||
): ArtifactChange | null => {
|
||||
const classifyPresence = (input: ClassifyPathInput): ArtifactChange | null => {
|
||||
const covered = input.path === "." ? input.rootCovered : input.entriesCovered;
|
||||
if (input.leftOccurrence === undefined)
|
||||
return {
|
||||
...base,
|
||||
...buildArtifactChangeBase(input),
|
||||
classification: covered ? "added" : "unknown",
|
||||
dimensions: [covered ? "content" : "availability"],
|
||||
};
|
||||
if (input.rightOccurrence === undefined)
|
||||
return {
|
||||
...base,
|
||||
...buildArtifactChangeBase(input),
|
||||
classification: covered ? "removed" : "unknown",
|
||||
dimensions: [covered ? "content" : "availability"],
|
||||
};
|
||||
@@ -375,12 +372,11 @@ const classifyAvailability = (
|
||||
};
|
||||
|
||||
const classifyPath = (input: ClassifyPathInput): ArtifactChange | null => {
|
||||
const presence = classifyPresence(input);
|
||||
if (presence !== null) return presence;
|
||||
const base = buildArtifactChangeBase(input);
|
||||
// A declared/observed mismatch is known even without an opposite occurrence.
|
||||
const integrity = classifyIntegrity(input, base);
|
||||
if (integrity !== null) return integrity;
|
||||
const presence = classifyPresence(input, base);
|
||||
if (presence !== null) return presence;
|
||||
const declaration = classifyPyInstallerDeclaration(input, base);
|
||||
if (declaration.handled) return declaration.change;
|
||||
const availability = classifyAvailability(input, base);
|
||||
|
||||
@@ -164,9 +164,6 @@ const redactFailure = (
|
||||
...(issue.expected === undefined
|
||||
? {}
|
||||
: { expected: visit(issue.expected) }),
|
||||
...(issue.observed === undefined
|
||||
? {}
|
||||
: { observed: visit(issue.observed) }),
|
||||
})),
|
||||
);
|
||||
if (error instanceof AnalysisOutputError)
|
||||
|
||||
@@ -52,36 +52,6 @@ describe("child-process options detection", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it("reads fork's silent option when stdio is not a string or array", () => {
|
||||
const ir = analyzeJavaScriptSemantics(`
|
||||
import { fork, spawn } from "node:child_process";
|
||||
|
||||
fork("piped.js", { silent: true });
|
||||
fork("inherited.js", [], { silent: false });
|
||||
fork("nullish-stdio.js", { silent: 1, stdio: null });
|
||||
fork("explicit-stdio.js", { silent: true, stdio: "inherit" });
|
||||
fork("dynamic.js", { silent: process.env.QUIET });
|
||||
spawn("tool", { silent: true });
|
||||
fork("boolean-stdio.js", { stdio: false, silent: true });
|
||||
fork("object-stdio.js", { stdio: {}, silent: false });
|
||||
fork("unresolved-stdio.js", { stdio: streams, silent: true });
|
||||
spawn("tool", { stdio: false });
|
||||
`);
|
||||
|
||||
expect(ir.childProcessSpawns.map(({ stdioMode }) => stdioMode)).toEqual([
|
||||
"pipe",
|
||||
"default",
|
||||
"pipe",
|
||||
"inherit",
|
||||
"dynamic",
|
||||
"default",
|
||||
"pipe",
|
||||
"default",
|
||||
"dynamic",
|
||||
"dynamic",
|
||||
]);
|
||||
});
|
||||
|
||||
it("preserves behavior with explicit args arrays", () => {
|
||||
const ir = analyzeJavaScriptSemantics(`
|
||||
import { spawn, execFile, fork } from "node:child_process";
|
||||
|
||||
@@ -10,10 +10,6 @@ import { traverseJavaScriptAst } from "./javascriptSemanticTraversal.js";
|
||||
import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js";
|
||||
import { semanticStaticPropertyKey } from "./javascriptAstValues.js";
|
||||
import { stringValue } from "./javascriptStaticAnalysisHelpers.js";
|
||||
import {
|
||||
withoutUnusedFactoryDefaults,
|
||||
type JavaScriptBundlerFactoryRoles,
|
||||
} from "./javascriptBundlerFactoryRoles.js";
|
||||
import {
|
||||
resolveSemanticBindingState,
|
||||
type JavaScriptBindingIndex,
|
||||
@@ -22,8 +18,6 @@ import {
|
||||
/** Static CommonJS export names. */
|
||||
export interface StaticExports {
|
||||
readonly values: readonly string[];
|
||||
/** Export-shaped writes whose receiver role could not be resolved. */
|
||||
readonly unknown: number;
|
||||
}
|
||||
|
||||
/** Derive a rename-resistant syntax fingerprint without code execution. */
|
||||
@@ -38,14 +32,18 @@ export const fingerprintJavaScriptAst = (node: t.Node): string => {
|
||||
return createHash("sha256").update(JSON.stringify(tokens)).digest("hex");
|
||||
};
|
||||
|
||||
/** Calling-convention roles established by the bundler adapter. */
|
||||
export interface JavaScriptFactoryExportBindings {
|
||||
readonly exports: t.Identifier | undefined;
|
||||
readonly module: t.Identifier | undefined;
|
||||
readonly loader: t.Identifier | undefined;
|
||||
}
|
||||
|
||||
/** Collect statically declared CommonJS/bundler export names from one factory. */
|
||||
export const collectJavaScriptExports = (
|
||||
node: t.Node,
|
||||
factory?: {
|
||||
readonly roles: Pick<
|
||||
JavaScriptBundlerFactoryRoles,
|
||||
"module" | "exports" | "loader" | "unusedDefaults"
|
||||
>;
|
||||
readonly roles: JavaScriptFactoryExportBindings;
|
||||
readonly index: Pick<
|
||||
JavaScriptBindingIndex,
|
||||
"bindings" | "effectsForReferenceWithinActivationAt"
|
||||
@@ -56,61 +54,47 @@ export const collectJavaScriptExports = (
|
||||
const index = factory?.index;
|
||||
const bindings = index?.bindings;
|
||||
const exports = new Set<string>();
|
||||
let unknown = 0;
|
||||
const bindingId = (reference: t.Identifier): string | undefined =>
|
||||
bindings === undefined
|
||||
? undefined
|
||||
: resolveSemanticBindingState(bindings, reference, reference.name)
|
||||
?.bindingId;
|
||||
const bindingId = (
|
||||
reference: t.Node | null | undefined,
|
||||
): string | undefined =>
|
||||
bindings !== undefined && t.isIdentifier(reference)
|
||||
? resolveSemanticBindingState(bindings, reference, reference.name)
|
||||
?.bindingId
|
||||
: undefined;
|
||||
const identity = (
|
||||
parameters: readonly (t.Identifier | undefined)[],
|
||||
role: keyof JavaScriptFactoryExportBindings,
|
||||
fallback: string,
|
||||
) => {
|
||||
const ids = new Set(
|
||||
parameters.flatMap((parameter) => {
|
||||
const id = parameter === undefined ? undefined : bindingId(parameter);
|
||||
return id === undefined ? [] : [id];
|
||||
}),
|
||||
);
|
||||
return (reference: t.Node | null): ReceiverRole => {
|
||||
if (!t.isIdentifier(reference)) return undefined;
|
||||
if (roles === undefined)
|
||||
return reference.name === fallback ? "role" : undefined;
|
||||
if (bindings === undefined) return undefined;
|
||||
const id = bindingId(roles?.[role]);
|
||||
return (reference: t.Node | null): boolean => {
|
||||
if (!t.isIdentifier(reference)) return false;
|
||||
if (roles === undefined) return reference.name === fallback;
|
||||
if (id === undefined || bindings === undefined) return false;
|
||||
const binding = resolveSemanticBindingState(
|
||||
bindings,
|
||||
reference,
|
||||
reference.name,
|
||||
);
|
||||
if (binding === undefined || !ids.has(binding.bindingId))
|
||||
return undefined;
|
||||
if (binding?.bindingId !== id) return false;
|
||||
// The calling convention owns the parameter's entry value, not every
|
||||
// later value assigned to the same lexical binding.
|
||||
const read = semanticBindingAtRead(
|
||||
withoutUnusedFactoryDefaults(binding, roles),
|
||||
binding,
|
||||
reference,
|
||||
bindings.parentsByNode,
|
||||
);
|
||||
if (read.status !== "ordered") return "unresolved";
|
||||
return read.binding.initializers.length === 0 &&
|
||||
return (
|
||||
read.status === "ordered" &&
|
||||
read.binding.initializers.length === 0 &&
|
||||
read.binding.referenceInitializers.length === 0
|
||||
? "role"
|
||||
: undefined;
|
||||
);
|
||||
};
|
||||
};
|
||||
const isExportsAlias = identity(roles?.exports ?? [], "exports");
|
||||
const isModuleAlias = identity([roles?.module], "module");
|
||||
const isLoaderAlias = identity([roles?.loader], "__webpack_require__");
|
||||
const receivers: ExportReceivers = {
|
||||
isExportsAlias,
|
||||
isModuleAlias,
|
||||
unresolved: () => {
|
||||
unknown += 1;
|
||||
},
|
||||
};
|
||||
const isExportsAlias = identity("exports", "exports");
|
||||
const isModuleAlias = identity("module", "module");
|
||||
const isLoaderAlias = identity("loader", "__webpack_require__");
|
||||
const receivers = { isExportsAlias, isModuleAlias };
|
||||
traverseJavaScriptAst(node, {
|
||||
// A default the calling convention never runs declares no exports.
|
||||
skip: (current) => roles?.unusedDefaults.has(current) === true,
|
||||
enter: (current) => {
|
||||
if (t.isAssignmentExpression(current))
|
||||
collectAssignmentExports(
|
||||
@@ -127,40 +111,31 @@ export const collectJavaScriptExports = (
|
||||
"defineProperty",
|
||||
]);
|
||||
collectCallExports(current, exports, receivers, intrinsic, (root) => {
|
||||
const loader = isLoaderAlias(root);
|
||||
if (loader !== "role" || root === null) return loader;
|
||||
if (index === undefined) return "role";
|
||||
if (root === null || !isLoaderAlias(root)) return false;
|
||||
if (index === undefined) return true;
|
||||
const facts = index.effectsForReferenceWithinActivationAt(
|
||||
current,
|
||||
root,
|
||||
["d"],
|
||||
);
|
||||
return facts.memberEffectsIndexed &&
|
||||
return (
|
||||
facts.memberEffectsIndexed &&
|
||||
!facts.effects.some(
|
||||
(effect) =>
|
||||
effect.kind === "member-write" && effect.timing !== "after",
|
||||
)
|
||||
? "role"
|
||||
: undefined;
|
||||
);
|
||||
});
|
||||
}
|
||||
return undefined;
|
||||
},
|
||||
});
|
||||
return { values: [...exports].sort(compareUnicodeCodePoints), unknown };
|
||||
return { values: [...exports].sort(compareUnicodeCodePoints) };
|
||||
};
|
||||
|
||||
/**
|
||||
* `role` holds the parameter's entry value; `unresolved` reads its binding
|
||||
* where write order is unknown, so the write may or may not be an export.
|
||||
*/
|
||||
type ReceiverRole = "role" | "unresolved" | undefined;
|
||||
|
||||
interface ExportReceivers {
|
||||
readonly isExportsAlias: (root: t.Node | null) => ReceiverRole;
|
||||
readonly isModuleAlias: (root: t.Node | null) => ReceiverRole;
|
||||
/** Record an export-shaped write whose receiver could not be resolved. */
|
||||
readonly unresolved: () => void;
|
||||
readonly isExportsAlias: (root: t.Node | null) => boolean;
|
||||
readonly isModuleAlias: (root: t.Node | null) => boolean;
|
||||
}
|
||||
|
||||
const semanticTokens = (node: t.Node): string[] => {
|
||||
@@ -229,24 +204,19 @@ const collectAssignmentExports = (
|
||||
left: t.LVal | t.OptionalMemberExpression,
|
||||
right: t.Expression,
|
||||
output: Set<string>,
|
||||
{ isExportsAlias, isModuleAlias, unresolved }: ExportReceivers,
|
||||
{ isExportsAlias, isModuleAlias }: ExportReceivers,
|
||||
): void => {
|
||||
const path = memberPath(left);
|
||||
if (path === null) return;
|
||||
const root = memberRoot(left);
|
||||
// Rebinding the CommonJS alias does not replace module.exports.
|
||||
if (path.length === 1) return;
|
||||
const exportsAlias = isExportsAlias(root);
|
||||
const moduleAlias =
|
||||
exportsAlias === undefined && path[1] === "exports"
|
||||
? isModuleAlias(root)
|
||||
: undefined;
|
||||
if (exportsAlias === "unresolved" || moduleAlias === "unresolved") {
|
||||
unresolved();
|
||||
return;
|
||||
}
|
||||
const exportOffset =
|
||||
exportsAlias === "role" ? 1 : moduleAlias === "role" ? 2 : null;
|
||||
// Rebinding the CommonJS alias does not replace module.exports.
|
||||
if (path.length === 1 && exportsAlias) return;
|
||||
const exportOffset = exportsAlias
|
||||
? 1
|
||||
: isModuleAlias(root) && path[1] === "exports"
|
||||
? 2
|
||||
: null;
|
||||
if (exportOffset === null) return;
|
||||
if (path.length === exportOffset) {
|
||||
if (t.isObjectExpression(right)) collectObjectKeys(right, output);
|
||||
@@ -262,37 +232,27 @@ const collectAssignmentExports = (
|
||||
const collectCallExports = (
|
||||
call: t.CallExpression,
|
||||
output: Set<string>,
|
||||
{ isExportsAlias, isModuleAlias, unresolved }: ExportReceivers,
|
||||
{ isExportsAlias, isModuleAlias }: ExportReceivers,
|
||||
intrinsic: boolean,
|
||||
isLoaderAlias: (root: t.Node | null) => ReceiverRole,
|
||||
isLoaderAlias: (root: t.Node | null) => boolean,
|
||||
): void => {
|
||||
const callee = memberPath(call.callee);
|
||||
if (callee === null) return;
|
||||
const definePropertyCall = intrinsic;
|
||||
const declarationCall =
|
||||
!definePropertyCall && callee.length === 2 && callee[1] === "d"
|
||||
? isLoaderAlias(memberRoot(call.callee))
|
||||
: undefined;
|
||||
if (!definePropertyCall && declarationCall === undefined) return;
|
||||
callee.length === 2 &&
|
||||
callee[1] === "d" &&
|
||||
isLoaderAlias(memberRoot(call.callee));
|
||||
if (!definePropertyCall && !declarationCall) return;
|
||||
const targetNode = call.arguments[0];
|
||||
const target = memberPath(targetNode);
|
||||
const exportsAlias = isExportsAlias(targetNode ?? null);
|
||||
const moduleAlias =
|
||||
exportsAlias === undefined &&
|
||||
target !== null &&
|
||||
target.length === 2 &&
|
||||
target[1] === "exports"
|
||||
? isModuleAlias(memberRoot(targetNode))
|
||||
: undefined;
|
||||
if (exportsAlias === undefined && moduleAlias === undefined) return;
|
||||
if (
|
||||
exportsAlias === "unresolved" ||
|
||||
moduleAlias === "unresolved" ||
|
||||
declarationCall === "unresolved"
|
||||
) {
|
||||
unresolved();
|
||||
return;
|
||||
}
|
||||
const exportTarget =
|
||||
isExportsAlias(targetNode ?? null) ||
|
||||
(target !== null &&
|
||||
target.length === 2 &&
|
||||
isModuleAlias(memberRoot(targetNode)) &&
|
||||
target[1] === "exports");
|
||||
if (!exportTarget) return;
|
||||
if (definePropertyCall) {
|
||||
const name = stringValue(call.arguments[1]);
|
||||
if (name !== undefined) addExport(output, name);
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
import * as t from "@babel/types";
|
||||
|
||||
import { javascriptBootstrapAt } from "./javascriptBootstrap.js";
|
||||
import { javascriptBundlerFactoryRoles } from "./javascriptBundlerFactoryRoles.js";
|
||||
import { proveJavaScriptBootstrapIntegrity } from "./javascriptBootstrapIntegrity.js";
|
||||
import {
|
||||
semanticMemberChain,
|
||||
@@ -31,9 +30,6 @@ import {
|
||||
traverseJavaScriptValueAst,
|
||||
} from "./javascriptSemanticTraversal.js";
|
||||
|
||||
/** A standalone Webpack bootstrap uses the module, exports, loader convention. */
|
||||
const BOOTSTRAP_RUNTIME = "webpack-bootstrap";
|
||||
|
||||
/** Loader bindings and literal externals proven for one standalone bootstrap. */
|
||||
interface BootstrapExternals {
|
||||
/** The runtime loader and every factory parameter it is passed as. */
|
||||
@@ -371,18 +367,14 @@ const proveBootstrapExternals = (
|
||||
});
|
||||
for (const { factory } of factories) {
|
||||
// The bootstrap proves `table[key](module, module.exports, loader)`.
|
||||
const loaderParameter = javascriptBundlerFactoryRoles(
|
||||
factory,
|
||||
BOOTSTRAP_RUNTIME,
|
||||
).loader;
|
||||
const bindingId =
|
||||
loaderParameter === undefined
|
||||
? undefined
|
||||
: resolveSemanticBindingState(
|
||||
state,
|
||||
loaderParameter,
|
||||
loaderParameter.name,
|
||||
)?.bindingId;
|
||||
const loaderParameter = factory.params[2];
|
||||
const bindingId = t.isIdentifier(loaderParameter)
|
||||
? resolveSemanticBindingState(
|
||||
state,
|
||||
loaderParameter,
|
||||
loaderParameter.name,
|
||||
)?.bindingId
|
||||
: undefined;
|
||||
if (
|
||||
bindingId !== undefined &&
|
||||
singleDefinition(state, bindingId, "parameter")
|
||||
@@ -424,12 +416,14 @@ const externalSpecifier = (
|
||||
factory: t.FunctionExpression | t.ArrowFunctionExpression | t.ObjectMethod,
|
||||
state: JavaScriptSemanticAnalysisState,
|
||||
): string | undefined => {
|
||||
const moduleParameter = javascriptBundlerFactoryRoles(
|
||||
factory,
|
||||
BOOTSTRAP_RUNTIME,
|
||||
).module;
|
||||
const moduleParameter = factory.params[0];
|
||||
const names = factory.params.map((parameter) =>
|
||||
t.isIdentifier(parameter) ? parameter.name : null,
|
||||
);
|
||||
if (
|
||||
moduleParameter === undefined ||
|
||||
!t.isIdentifier(moduleParameter) ||
|
||||
// A duplicated name binds the later argument, not the module object.
|
||||
new Set(names).size !== names.length ||
|
||||
factory.generator ||
|
||||
factory.async ||
|
||||
(t.isObjectMethod(factory) && factory.kind !== "method")
|
||||
|
||||
Loaded 100 of 175 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user