Compare commits

..
Author SHA1 Message Date
morluto f2cffbdfdc fix(pcap): preserve executable read failures 2026-10-22 06:09:50 +08:00
morluto 5e046bb374 fix(pcap): read historical capture evidence 2026-10-22 06:05:55 +08:00
morluto 4924706e25 fix(pcap): report missing selected tools clearly 2026-10-22 05:52:17 +08:00
morluto 211d4b2219 perf(pcap): retain compact executable identity 2026-10-22 05:47:11 +08:00
morluto cb6dc4f3ef fix(pcap): share selected tool identity verification 2026-10-22 05:40:52 +08:00
Kaoru0822andClaude Opus 5.5 658c29b7fc chore: restore main's copies of files the merge hook reformatted
The pre-commit oxlint --fix touched two files this PR does not change; keep them identical to main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 03:22:28 +00:00
Kaoru0822andClaude Opus 5.5 a22adc7809 Merge origin/main into refactor/tshark-single-owner
Resolve the node:fs/promises import conflict in pcapProviderCrossPlatform.test.ts by importing open, symlink and writeFile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 03:04:35 +00:00
morluto d0067ba34a fix(native)!: preserve Swift pointer nullability and import bindings
Resolve relative pointers using field-owned ABI encodings. Keep null, raw, rebased, imported and unknown targets distinct, and retain imported conformance rows with exact Mach-O binding provenance. Decode legacy rebase membership with source-derived memory and work bounds, preserving validated prefix facts when streams are incomplete.

Validation: check:ci, docs:check and build passed; 158 source tests and 20 schema/acceptance tests passed. Ten source-built Apple dispatch variants and four issue-specific CLI/MCP workflows passed without executing targets. Paired cold/warm CLI/MCP cost observations retained locally; model tokens and maximum-sized input runtime were not measured.

Closes #5683
Closes #5685

BREAKING CHANGE: Swift conformance type_name and protocol_name can be null when names cannot be resolved; consumers must use typed type_binding and protocol_binding evidence. MachoImportBinding replaces the former Objective-C-specific binding type and schema.
2026-10-21 08:17:12 +08:00
morluto 9258df02cc test(apple): compare asset catalogs from matching root workflows
Compare explicit bundle-resource CLI and MCP results at the positive empty-inventory offset. Preserve a separate binary-bound MCP assertion with the exact fixture executable digest and complete metadata payload.

Validation: reproduced the stale root-versus-binary parity failure on current main, then passed the macOS public CLI/MCP acceptance case across all five NIB fixtures. Focused lint, formatting and diff checks passed. No target code was executed.
2026-10-21 07:37:36 +08:00
morluto 1502cfbc8c fix(apple): resolve absolute dylib paths inside the selected root
Preserve load-command paths as evidence while passing a separate root-relative lookup path to the bounded tree resolver. Apply the same containment rule to absolute install names and rpath bases, including caller-selected root symlink spellings.

Validation: 140 source tests, full check:ci, docs and runtime build passed. Compiler-built literal, rpath and relative fixtures passed CLI/stdio MCP parity, preserved both dependency edges and left all bytes unchanged. External system paths stayed outside-target; relative control was unchanged. Fixtures were not executed, and actual dyld execution was not qualified.

Closes #5637
2026-10-21 07:37:15 +08:00
morluto 84c89d1107 test(javascript): separate heap pressure from graph admission proof
Match memory failures to the exact source identity instead of graph ordering. Preserve CLI parse, file-evidence, partial-result, later-source and unchanged-input checks across valid memory-pressure outcomes. Use a controlled parent heap envelope with a real owned analyzer to prove module retention and graph rejection independently of host GC timing.

Validation: full check:ci passed; 17 focused tests passed, with the final 4 changed tests rerun successfully. Independent test-strength and isolation review passed. The earlier hosted failure exposed missing graph-only fields; its exact alternate memory stage was not captured.
2026-10-21 07:26:20 +08:00
morluto 0bd14ca16c fix(browser): preserve uncertain main-frame navigation identity
Own CDP frame applicability and identity at the parsed event boundary. Preserve malformed or incomplete main-frame facts, recognize replacement roots, and carry the same document identity through captures, screenshots, timelines, and execution windows.

Validation: full check:ci and docs:check; 763 browser and dispatch tests passed, 6 skipped. Independent source and fixture-lifecycle review passed. Local real-Chrome behavior was not exercised.

Closes #5665
2026-10-21 07:16:47 +08:00
morluto 9dee989a79 fix(native): count Objective-C coverage by class object
Count each admitted unique class or metaclass object at its parsing owner. Preserve emitted metadata, independent class-list scan bounds, partial failures, and truncation.

Closes #5706
2026-10-21 07:01:17 +08:00
morluto f789fd1fd8 fix(native): count Swift vtable coverage in slot units
Count examined and decoded vtable slots in the same population, preserving all returned slots and partial descriptor failures. Document the coverage unit and retain malformed-sibling and unresolved-slot checks.

Closes #5681
2026-10-21 06:56:18 +08:00
morluto ccc7da309d test(mcp): retain distinct Objective-C class observations
Require both fixture addresses and unknown identity coverage when no selected-image binding proof permits coalescing.
2026-10-21 06:45:34 +08:00
morluto 56679dd2ae fix(files): preserve physical read provenance through cleanup
Keep raw physical read diagnostics alongside explicit operation provenance on the same cleanup owner. Preserve provenance after successful close retry for opted-in readers, restore legacy error projections, and prevent borrowed cleanup-shaped failures from authorizing foreign resource retries.

Refs #5686
2026-10-21 06:45:34 +08:00
morluto 52c84f6f58 test(cli): bound ADB MCP validation to the advertised tool
Reuse the exact selected advertised descriptor for SDK output validation instead of compiling the entire cached catalog for every fresh parity-test client. Retain actual catalog discovery, schema validation, and all result assertions under the existing worker heap limit.
2026-10-21 06:23:20 +08:00
morluto 4852c830af fix(native): preserve Mach-O names equal to version
Parse only documented compound otool field labels so valid segment and section names are retained at the owning boundary. Verified real Apple clang fixtures through compiled CLI and stdio MCP without executing the targets.

Closes #5680
2026-10-21 06:19:43 +08:00
morluto 949966712e fix(files): preserve physical read failures across late cancellation
Tag physical read outcomes at the owning boundary and preserve opaque caller cancellation reasons through JSON pipelines and reference imports. Share named bounded-read options across consumers and retain failed cleanup ownership for retry.

Closes #5686
2026-10-21 06:14:03 +08:00
morluto 4d7de00d4c fix(native): coalesce Objective-C class imports by binding identity
Join provider class observations only when selected-image identity and exact import bindings establish equivalence. Preserve distinct observations and unsupported coverage.

Refs #5607
2026-10-21 06:12:21 +08:00
morluto b231c28cf6 fix(hopper): own Mach-O preparation through cleanup retry
Preserve exact selected-image identity and descriptor ownership across preparation, final cleanup, and retries. Keep physical failures and failed cleanup leases observable through provider and launcher boundaries.

Closes #5688
2026-10-21 06:12:09 +08:00
morluto 7dd6941393 refactor(test): share browser and apktool fixture lifecycles 2026-10-21 05:50:50 +08:00
morluto 1bb2fa23de test(apple): match plist provenance to the selected workflow 2026-10-21 05:50:50 +08:00
morluto ab58edcdf5 fix(environment): retain tarball descriptor ownership during export
Use the shared positional reader for both tarball passes so early output refusal cannot close the descriptor ahead of its owner.

Closes #5684
2026-10-21 05:33:35 +08:00
morluto 60956c8c56 fix(ci): run cache retention from its trusted workflow revision 2026-10-21 05:28:08 +08:00
morluto 5518363aa6 test(artifacts): compare matching resource-root workflows 2026-10-21 05:28:03 +08:00
morluto 4108d1363a test(cli): align Kuna readiness and cancellation expectations 2026-10-21 05:27:56 +08:00
morluto 0bc2b42001 fix(browser)!: preserve unknown WebSocket frame facts
Keep malformed or missing CDP opcodes and payload facts unknown, retain observed zero and empty payloads, and avoid false capture differences while preserving independently known facts.

Validation: 59 source tests; compiled public CLI and stdio MCP fake-CDP acceptance; check:fast; docs:check; changed-file formatting. Real Chromium malformed-event behavior was not tested.

BREAKING CHANGE: WebSocket event opcode is nullable and numeric values are restricted to the four-bit protocol range.

Fixes #5666
2026-10-21 05:00:37 +08:00
c38f96fa69 test(cli): keep IPC trace input under the Linux spawn limit (#5673)
* test(cli): keep IPC trace input under the Linux spawn limit

The listener-callsite CLI builds one trace argument from the full
application Evidence document. On 4 KiB-page Linux that string exceeds
MAX_ARG_STRLEN, so spawn throws E2BIG before the command starts. Keep the
document in a file, require the spawned argv to fit the single-argument
budget, and lock that budget through the test process registry.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(process): size the spawn ceiling from ARG_MAX and MAX_ARG_STRLEN

A payload at the single-string limit still fails execve with E2BIG when
the stack shrinks ARG_MAX below the rest of the image. Accept only the
bytes that fit both budgets, and skip that success spawn when the fixed
argv and environment already fill ARG_MAX.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* style(test): format files rejected by oxfmt in static checks

check:ci:fast failed format:check on the CLI fixture import and, after
merging main, on the admission test's new command-abort case.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 04:48:33 +08:00
Ulises Millán c4b6aa3061 fix(managed): treat extension-only module scopes as unknown identity (#5674)
Closes #5467
2026-10-21 04:35:45 +08:00
morluto dceefa22cb Merge remote-tracking branch 'origin/main' into codex/audit-continuation-20261020-refresh20 2026-10-21 04:22:27 +08:00
morluto 6a6adf703f fix(resources)!: inspect app resources without an executable
Add a read-only app resource session shared by CLI and MCP. Preserve resource-root provenance without inventing executable identity, reject binary-only snapshots and operations, and retain failed cleanup owners across switches and retries.

Keep filesystem admission failures typed and preserve cancellation ordering from observed cleanup facts. Document resource routing in the packaged investigation skill.

BREAKING CHANGE: resource-only observations may have a null Evidence subject and null executable target digests. Clients must use resource_root and reported per-resource provenance instead.

Refs #5589
2026-10-21 04:22:20 +08:00
de4579e993 fix(observation): keep incomplete cancellation observations (#5644)
* fix(observation): keep incomplete cancellation observations

Service projection treated a retained observation that failed the completed
schema as a malformed completed result. Cancellation became unreadable_output
and the partial was dropped. Incomplete observations now stay on the owner
error, including the exact abort reason.

* fix(observation): drop non-JSON incomplete observations

Keep a retained partial only when projectAnalysisError can serialize it.
Clear a non-JSON partial and record an AnalysisOutputError as a secondary
failure so cancellation and timeout stay the owner error.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(observation): validate incomplete partials without copying

jsonValueSchema.safeParse cloned a JSON-safe partial that projection
copies again. Use jsonValueValidationIssue so the gate only checks.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(ida): expect non-retryable unresolved procedure lookups

Main stops treating invalid_request as retryable. An unchanged missing
procedure stays invalid_request and is not marked retryable.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(observation): share incomplete-observation assertions

The new non-JSON cases repeated the same owner-error checks and pushed
duplication over the 3.5% limit. One expectation helper keeps those
checks without a second copy.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(observation): keep the assertion helper out of the package

The helper imports Vitest. Rename it to a fixture so the production
build excludes it and dist does not import a dev dependency.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(observation): clear non-JSON partials on cleanup primaries

Projection can return the cleanup primary after retry. Clearing only
the wrapper left that primary's partial in place, so projectAnalysisError
threw. The gate now clears every non-JSON partial in the chain.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(observation): freeze retained incomplete observations

A JSON partial stayed a live provider object, so a later mutation could
make projection throw. The gate now keeps one frozen snapshot and still
drops a non-JSON partial without copying it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* refactor(observation): share web runtime partial retention

Observe and inspect used the same promote-or-keep path. One helper
owns that sequence so the two operations cannot drift.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(observation): snapshot a shared partial once

A cleanup wrapper and its primary can hold the same partial object.
The gate now reuses the first frozen snapshot for that object.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(observation): parse a kept partial once

jsonValueSchema.safeParse validates and copies only a JSON partial.
The gate freezes that copy and no longer walks the same value first.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(observation): share repeated observation fixtures

The incomplete-observation tests repeated the same provider setup and
an empty Electron coverage tail, which pushed duplication over 3.5%.
One coverage fixture and shared assertions bring the tree back under
the threshold.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 04:17:46 +08:00
6b101ff23b fix(errors): preserve caller abort reasons at admission (#5671)
* fix(errors): preserve caller abort reasons at admission

Keep the opaque caller reason on reverse, pcap, and SQLite cancellation,
and leave a recognized physical digest or command failure in place when a
late abort is also observed.

Fixes #5670

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(reverse): keep a foreign digest ABORT_ERR as an adapter failure

Treat ABORT_ERR as cancellation on the command boundary, where that code
already meant an aborted provider process. A digest or stat rejection with
the same code stays a physical adapter failure unless the caller signal
is aborted. Cover aborted command EACCES, ETIMEDOUT, and ABORT_ERR with
and without captured output.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 04:16:44 +08:00
morluto 6389e50337 Merge remote-tracking branch 'origin/main' into codex/audit-continuation-20261020-refresh20 2026-10-21 03:57:03 +08:00
morluto d23be5c089 fix(web): preserve static WebMCP schema property coverage
Reuse the shared object-member owner for literal spreads, retain known property names under unresolved membership, and distinguish complete, partial, unknown, and absent projections. Keep historical coverage omission unchanged and preserve own keys across prototype setters.

Refs #5075
2026-10-21 03:56:35 +08:00
c0cbbf8d50 perf(ci): keep fast built shards under ten minutes (#5652)
* test(acceptance): split competing loader ownership across files

The loader matrix was one 578-second file, which set the floor for any
shard that received it. Three files keep every CLI and MCP case and let
workers overlap them.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* perf(ci): run eight fast built shards under a ten-minute budget

Charge each Vitest project by its resolved worker width so serial
process-boundary files spread as tail time. Fast feedback uses eight
built shards. The npm download cache stays; node_modules is too large
to ship as a shared artifact.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(ci): keep each scheduled file on one worker lane

Dividing a file by maxWorkers dropped its own wall-time floor and could
pack a parallel file onto a serial tail. Place each file whole on a lane,
overlap peers only inside a sequence group, and add later groups after.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* docs(testing): separate the shard budget from workflow time

The ten-minute figure is the built shard job. Classification, build, and
the portable-test merge are measured separately on run 39451548945.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* docs(testing): match fast shard count to lane scheduling

Main still described five shards and half-duration weights. Fast
qualification uses eight shards, and each file keeps its full duration
on one worker.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(ci): seed split loader durations from pre-split history

Timings published before the file split still name one 578-second path.
Until a new file appears in that artifact, share the recorded duration
by the pre-split case counts.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 03:55:40 +08:00
morluto a7d13893a6 Merge remote-tracking branch 'origin/main' into codex/audit-continuation-20261020-refresh20 2026-10-21 03:50:44 +08:00
morluto 1bfa60ac20 fix(cli): preserve caller-selected JSON paths
Keep POSIX symlink-parent traversal at the shared JSON input boundary and forward the selected browser executable without a second lexical normalization. Verify Electron, browser, and Evidence-file consumers through registered commands and the built CLI.

Refs #5103
2026-10-21 03:50:14 +08:00
9d8e780332 ci: keep npm download caches from Turbo snapshot growth (#5655)
* ci: keep npm download caches from Turbo snapshot growth

Per-commit static and portable Turbo snapshots had filled the 10 GB Actions
cache, which is what makes warm npm ci cheap. Trusted static runs now keep two
generations and one download-cache writer per host.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(ci): declare cache retention types for strict checks

Typecheck imports cache-retention.mjs and needs a sibling declaration.
The generation count stays module-private so knip does not report it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(ci): keep compatible caches and qualification read-only

Npm dedupe deletes a record only when another has the same key, version,
and ref. Cache deletion moves to a follow-up workflow so reusable
qualification can keep its read-only token.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(ci): retain caches after failed default-branch runs

Static and build save Turbo snapshots before later lanes finish. Retention
now follows a failed, timed-out, or cancelled trusted run, not only a green one.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(cli): pass application trace input as a file

The CLI already reads workflow JSON from a path. Putting the full
application Evidence document on argv exceeds the runner argument limit.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(ci): retain deep and Knip snapshots without deleting a repeated id

Fast and deep static checks publish rea-static-deep-v1 and knip-meta-v1
entries per commit. Retention prunes those families with the same two-generation
rule. A cache id listed twice is one record; inconsistent repeats abort
before any delete.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 03:48:14 +08:00
40428da1dd perf(test): reuse one stdio MCP server per JavaScript application file (#5658)
* perf(test): reuse one stdio MCP server per JavaScript application file

Repeated analyzeJavaScriptDirectory MCP cases started a full server for
every case. Share one file-scoped server and compile cache so later cases
skip that startup while CLI cases still launch a fresh process.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(test): close in-flight shared MCP startup with the file

File teardown waits for a stdio session that is still starting, skips a
server once close has begun, and still closes one registered during that
wait. The later-case reuse test can run without the earlier case.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(test): keep a failed MCP handshake when cleanup fails

The shared session reports the startup error and carries the cleanup failure
as its cause. A cleanup that failed stays owned and is retried on close
without running a cleanup that already succeeded.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(test): fail shared MCP cleanup when the rejection has no reason

A cleanup rejection of undefined still fails the drain and stays owned.
Close caches completion only after every retained cleanup succeeds, so a
later close can release it.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(test): keep one MCP server while failed cleanup is retained

A later call retries retained cleanup before opening another server. The
new server starts only after that cleanup succeeds.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 03:46:14 +08:00
morlutoandCursor Agent ef59174c1e fix(errors): name fixed allowances in resource-constraint recovery (#5633)
* fix(errors): name fixed allowances in resource-constraint recovery

Fixed parser, count, report, input, and capture refusals inherited
host-memory, wait-and-retry, or retained-Evidence advice. Producers now
attach allowance-specific recovery, and deterministic capacity caps are
not retryable. IL2CPP reader occupancy stays retryable capacity.

Refs #5556, #5405, #5456, #5459, #5460, #5504, #5505, #5506, #5537, #5628, #5630, #5631, #5632

* fix(il2cpp): keep a closing metadata reader non-retryable

close() stops admission before the active inspection finishes. A second
inspect in that window was projected as retryable capacity. Closing and
closed readers now report capability_unavailable. Occupied readers that
remain open stay retryable.

* fix(errors): align fixed-cap recovery with retained facts

IL2CPP report refusals keep observed source facts instead of denying Evidence. Command-output caps use the capture allowance. Ghidra seed recovery names the limit that was exceeded.

* fix(errors): call byte caps a byte allowance

WASM representation limits and the IL2CPP Ghidra seed-file cap are byte budgets. Their recovery no longer says the exceeded allowance is a fixed count.

* test(ida): keep unresolved procedure lookups non-retryable

Main now treats an unchanged invalid request as non-retryable. The IDA lookup tests follow that projection.

* fix(errors): qualify capture recovery for live output

A fixed capture allowance still rejects the same output. Live captures such as adb logcat can change between requests, so recovery no longer says every unchanged request fails again.

* fix(adb): keep a truncated live capture retryable

Device output can shrink between calls, so an adb capture that exceeds the allowance stays retryable. Deterministic capture and version-output limits stay non-retryable.

* fix(adb): keep truncated mutations non-retryable

Only live observations opt into retrying a capture that exceeded the allowance. Push, install, uninstall, and other mutations stay non-retryable.

* fix(errors): keep live capture recovery off deterministic limits

Artifact-derived capture allowances again say the unchanged request fails. Only volatile adb observations use the live-output sentence.

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-10-21 03:45:34 +08:00
53e0943223 fix(javascript): preserve path reasons, module coverage, and property provenance (#5651)
* fix(javascript): preserve selected path host resolution reasons

Classify ELOOP and ENAMETOOLONG while resolving the caller-selected input so the path and host code stay on the invalid-request issue.

Refs #5623

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(javascript): count summary coverage in module units

Use the module population for both examined and total counts. Non-module graph nodes remain in the application coverage section.

Refs #5464

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(javascript): connect direct property reads to return and argument flow

Record the unwrapped value expression and alias the object projection's read occurrence so a direct member read stays in backward provenance.

Refs #5585, #5648

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(javascript): count return wrappers against expression depth

Evaluate the original return expression so erased TypeScript and Flow wrappers stay inside the expression-depth budget. Keep the unwrapped node for identity and value ranges.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(javascript): describe ELOOP as unresolved symlink traversal

A host can return ELOOP for a cycle or for a chain past its traversal limit. Keep the selected path and code without asserting that a cycle exists.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(javascript): skip symlink traversal fixtures on Windows

Keep the overlong-name case portable. Windows does not produce ELOOP for these fixtures, matching the other path-failure tests.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(javascript): describe ENAMETOOLONG as a path or component limit

realpath reports that code for a long component and for a long full path. Keep the selected path and host code without claiming only a filename was too long.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* style(javascript): format return-depth and object-read changes

oxfmt rejected the return-depth test and the object-read index. No behavior change.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(ida): expect unchanged invalid requests to be nonretryable

The shared projector no longer marks invalid_request retryable. This lookup assertion was not updated with that contract.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 03:45:10 +08:00
b718ad166d fix(doctor): reject unusable Kuna caches and invalid native settings (#5647)
* fix(kuna): reject dangling cache roots in doctor

Treat a cache path as creatable only when no directory entry occupies it.
A dangling symbolic link, including one hidden by a trailing separator,
fails the cache check and clears provider availability while the installation
check and caller-selected registration settings remain.

Fixes #5554
Fixes #5105

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): admit the same settings native analysis rejects

Project parseConfig failures into scoped engine readiness. Invalid provider
preferences, startup timeouts, and other shared settings now fail the
selected provider and the full audit, while a JEB-only timeout stays outside
that admission.

Fixes #5454

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): report an invalid log level from doctor

The dispatcher rejected REA_LOG_LEVEL before every command, so doctor never
emitted its readiness check. Doctor now runs and names that setting; other
commands still stop before execution.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): require log-level failures in every scope

An invalid REA_LOG_LEVEL stops the executable and MCP server before a client
or skill can run, so those scopes now fail readiness too. Other native
admission failures stay limited to a selected provider.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(kuna): strip only the platform cache separator

On Darwin and Linux a trailing backslash is a filename character. Doctor now
removes only the selected platform separator before inspecting the cache path.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): fail every provider on an invalid Hopper timeout

REA_HOPPER_STARTUP_TIMEOUT_MS is rejected before any native engine starts.
A selected non-Hopper provider now includes the existing timeout check.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* style(doctor): format the Hopper timeout regression

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(doctor): fail client scopes on MCP startup errors

A registered client starts the MCP server, and parseConfig rejects every
native admission failure before that server starts. Client readiness now
includes those failures, not only REA_LOG_LEVEL.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 03:44:49 +08:00
morluto 690fc51854 fix(browser)!: preserve unknown console timestamps
Retain absent or invalid Runtime timestamps as null and mark console event coverage incomplete. Preserve observed finite timestamps, including zero, and document the epoch-millisecond clock.

BREAKING CHANGE: console event timestamps may now be null when the protocol does not provide a finite number.

Refs #5656
2026-10-21 03:43:20 +08:00
23196daa11 fix(apple): read LC_LAZY_LOAD_DYLIB_INFO dependency payloads (#5650)
* fix(apple): read LC_LAZY_LOAD_DYLIB_INFO dependency payloads

Parse the linkedit install name for lazy-load info commands, resolve those
edges without traversing them, and stop inspect-macho from treating an otool
dependency list as exhaustive when that payload is omitted.

Fixes #5551
Fixes #5649

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(apple): bound lazy-load payload reads and image offsets

Reuse identical LC_LAZY_LOAD_DYLIB_INFO ranges, cap their combined size,
and reject flag or chain-start offsets outside the slice VM range.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(apple): bound lazy-load string scans and contained reads

Reject payload strings that start in the 24-byte header, validate each
file-offset string once, and reuse payload bytes already read for a
containing range.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(apple): reject cached lazy-load strings outside a payload

Remember each cached string's terminating offset and require that NUL to
fall inside the payload being decoded.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(apple): bound lazy-load payload range lookups

Count containment checks across earlier payload reads and stop at the
existing 16 MiB work limit.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(apple): count every lazy-info string reference

Charge repeated and overlapping LC_LAZY_LOAD_DYLIB_INFO scans against the
16 MiB validation budget, and stop a scan once that budget is exhausted.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(apple): cap retained lazy-info dependency results

Stop after 1024 LC_LAZY_LOAD_DYLIB_INFO commands so a load-command region
full of identical commands cannot become a million traced edges.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(apple): reject a lazy-load path NUL in the final byte

dyld stops the load-path scan before the payload's last byte, so a
terminator in that byte leaves the image malformed.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(apple): keep may-be-missing lazy loads optional

A missing LC_LAZY_LOAD_DYLIB_INFO with the may-be-missing bit is weak, so the lazy finding says dyld continues on first use instead of failing.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* fix(apple): cap lazy-info symbol string references

A payload of distinct one-byte symbol strings stayed under the byte budgets while caching one entry per offset. A slice now stops at 65536 symbol references.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 03:33:07 +08:00
0add6a5aae ci: split fast and deep static checks (#5634)
* ci: split fast and deep static checks

Fast PRs run typecheck, lint, and format before provider admission.
Knip, duplication, architecture guards, and Python stay on the full tier.
Shared-boundary fallbacks keep the Linux x64 baseline, and Turbo jobs
export remote-cache credentials when they are configured.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* ci: keep pull requests from writing the remote cache

Trusted default-branch runs still receive TURBO_TOKEN. Pull requests get an
empty token and remote read-only mode. Knip saves a per-commit cache key so
the next trusted full run can refresh the restored snapshot.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* ci: forward the release cache token and upload deep static reports

Release qualification now passes TURBO_TOKEN into the called workflow.
Labeled pull requests still do not. The deep static job records its task
context and uploads duplication reports after a failed gate.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* style: format the turbo cache policy test

Co-authored-by: morluto <morluto@users.noreply.github.com>

* ci: keep fast checks exclusive and trusted cache writes explicit

Declare the static artifact name and context script so typecheck matches the
implementations. Lock check:ci:fast around dist, and publish the remote cache
only for release qualification or default-branch push, schedule, and dispatch.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* ci: record static task graphs without nesting the artifacts lock

Keep check:ci:fast locked, and point graph recording at the unlocked
check:ci:fast:tasks script so check:pr and check:test can record hashes
while they already hold that lock.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* ci: keep unauthenticated Turbo runs on the local cache

Open pull requests do not receive TURBO_TOKEN, so a remote read mode cannot
authenticate. Those runs now use the local cache. Trusted runs still publish.

Co-authored-by: morluto <morluto@users.noreply.github.com>

* test(ida): treat unchanged invalid requests as nonretryable

Main already stopped marking invalid_request retryable. The pull request
merge runs that projector, while the IDA lookup tests still expected
retryable true. Match the merged behavior.

Co-authored-by: morluto <morluto@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: morluto <morluto@users.noreply.github.com>
2026-10-21 03:29:31 +08:00
morluto 0dbc046ca6 Merge remote-tracking branch 'origin/main' into codex/audit-continuation-20261020-refresh20 2026-10-21 03:28:13 +08:00
morluto 2a0410f171 fix(browser)!: bind session timelines to the observation window
Sample the browser monotonic clock with observation already subscribed, filter historical lifecycle and network facts, and preserve scope and termination handling when event time is unknown.

Bound pre-arm retention using transport-owned wire bytes. Fail closed when the relevant-event buffer fills, and keep unrelated protocol payloads outside that budget.

BREAKING CHANGE: observe_web_session timeline timestamps can be null when the producer supplies no comparable timestamp. Consumers must handle unknown time explicitly.

Refs #5609
2026-10-21 03:27:31 +08:00
gensokyo-dev 4bfdadb7de perf(frida): add retained summaries for session status (#5661)
Add opt-in Frida session status summaries linked to complete retained Evidence. Share canonical session facts across contracts and preserve observed state, source identity and capture truncation.

Validated with native SDK/client workflows, default delivery compatibility, lifecycle and provenance tests. The completed large-history task uses 50.9% fewer estimated request tokens; no general CPU or latency improvement is claimed.

Fixes #5640
2026-10-21 03:21:30 +08:00
morluto 1629266e3c fix(errors): preserve canonical failures and correction details
Retain process input and unexpected failure diagnostics through the shared error projector. Preserve filesystem causes and firmware validation issues, and mark unchanged invalid requests nonretryable.

Use typed stat settlement and parse unknown command failures without losing null or undefined rejection causes.

Refs #5397, #5401, #5402, #5642
2026-10-21 02:03:17 +08:00
morluto b3aadaf474 fix(il2cpp): preserve recognized truncated metadata prefixes 2026-10-21 01:47:09 +08:00
morluto eb7acc70bf fix(native): verify signature snapshots and frame unsigned diagnostics
Bind and recheck the digest-verified snapshot used by codesign, retaining cleanup ownership when its identity changes. Classify unsigned only from the exact operand-framed diagnostic so echoed caller paths cannot turn unrelated failures into signature facts.

Closes #5636
2026-10-21 01:35:44 +08:00
morluto 998882045b Merge remote-tracking branch 'origin/main' into codex/audit-continuation-20261020-refresh20 2026-10-21 01:31:57 +08:00
morluto eb41f358cf fix(model): retain completed GGUF facts on interrupted reads
Keep parser-owned completed metadata and tensor declarations as typed partial Evidence on cancellation or physical read failure. Preserve unknown final identity and digests, exact caller inputs, primary failures, and retained cleanup owners; support inline and retained MCP error delivery.

Closes #5625
2026-10-21 01:31:17 +08:00
gensokyo-dev 17f581ff75 perf(reverse): add retained GDB and objdump summaries (#5614)
Add opt-in retained GDB and objdump summaries using the shared analysis-view layer. Preserve default complete Evidence, ordered records, provenance and completion facts.

Validate real Bash disassembly, whole-client context costs, CLI publication and cleanup; retain the original measured cohorts and explicit coverage limitations.

Closes #5584.
2026-10-21 01:20:52 +08:00
morluto edbd291a28 Merge remote-tracking branch 'origin/main' into codex/audit-continuation-20261020-refresh20 2026-10-21 01:13:13 +08:00
morluto 2b263a7fe9 fix(native): retain Mach-O facts when dyld rejects missing UUID
Admit only the corroborated missing-UUID validation diagnostic from header-stripped dyld stdout. Preserve readable load commands and nm exports with unknown dyld coverage, and keep unrelated process and tool failures typed.

Closes #5606
2026-10-21 01:12:51 +08:00
gensokyo-dev d98fc3859d test(evaluation): include native preparation in test deadlines (#5627)
Derive Darwin subprocess test budgets from the bounded ownership compiler budget while preserving short command deadlines. Always join the retained command scope after test failures.

Refs #5094.
2026-10-20 17:10:11 +00:00
555aaditya e74e4d8f0b Merge remote-tracking branch 'origin/main' into refactor/tshark-single-owner 2026-10-14 16:28:29 +05:30
555aaditya f4eeba3654 fix(pcap): execute the TShark the shared locator selects
resolvePcapTools still returned bare tool names for PATH selections, so
the operating system could run a tool from a relative PATH entry while
the packet tools skipped that entry and selected a different install.

On POSIX hosts, resolve PATH selections through locateWiresharkTool and
run that absolute executable, failing with the existing tshark_missing or
capinfos_missing code when no absolute PATH entry holds the tool. Both
selections are validated before PATH is consulted, so a relative
environment value still reports its own configuration error. Windows
keeps PATH/PATHEXT resolution in the process launcher.

Refs #3403
2026-10-14 16:28:29 +05:30
555aaditya bbecf6fb05 refactor(pcap): give TShark selection and version admission one owner
The pcap and packet-capture families wrapped the same caller-supplied
TShark with diverging rules. inspect_pcap_* accepted an absolute
REA_TSHARK_COMMAND or tshark from PATH, while inspect_packet_* and the
server availability policy required an absolute REA_TSHARK_COMMAND and
parsed the version banner with a separate regex.

PcapConfiguration now owns the rule: selectWiresharkTool selects an
absolute environment path or the bare name on PATH and refuses relative
values; locateWiresharkTool resolves a PATH selection to the first
absolute PATH entry holding an executable file; parseWiresharkVersion
admits Wireshark 4.x banners for both providers. TsharkCaptureProvider
and the session availability policy consume these facts instead of
reinterpreting REA_TSHARK_COMMAND, so all four tools accept and refuse
the same configurations. inspect_packet_* still resolves, hashes and
rechecks the located executable, and records selection_source in its
tool Evidence.

Host support and resource bounds are unchanged; unifying them is the
next milestone in #3403.

Refs #3403
2026-10-14 15:05:58 +05:30
457 changed files with 34128 additions and 3689 deletions

No files matched your search

@@ -94,6 +94,10 @@ explicit path or endpoint and do not need it.
- Archive/package member inventory (ZIP/APK/IPA/MSIX/AppX or DMG):
`open_binary` with the supplied local path; use `inspect_artifact` when its
graph and findings help answer the question.
- Apple app plist, Interface Builder, asset catalog, or keyed archive resources:
`open_bundle_resources`, then the selected resource tool. This does not
require `CFBundleExecutable` or a deep provider. Resource sessions have no
executable digest or binary snapshot; use `open_binary` for native code.
- Android APK code, classes, methods, or incoming references:
`inspect_android_package`, then focused Android tools when advertised.
Archive member inventory still uses the archive route above.
@@ -11,6 +11,22 @@ Select columns, batch IDs and page individual facts; aliases remain unresolved.
Unpaired observations do not prove additions/removals. Existing matched dossier
pairs include inputs for `compare_functions`; otherwise analyze just chosen pairs.
## Apple app resources
Use `open_bundle_resources` with the selected app path, then `inspect_plist`,
`decode_interface_builder`, `inspect_asset_catalog`, or `inspect_keyed_archive`.
The resource session reads the app without resolving `CFBundleExecutable` or
starting a deep provider. Preserve `resource_root` provenance, per-resource
hashes where reported, and null executable identity; a directory path is not
an executable digest. Export retained Evidence before closing or switching
roots. Binary analysis snapshots do not apply. Use `open_binary` to switch to
executable analysis, and `close_binary` to release either session kind.
The four matching CLI commands accept the app path directly. For an archive
inside the app, use `inspect-keyed-archive APP_PATH RELATIVE_ARCHIVE_PATH`.
Host requirements for plist and compiled asset/Interface Builder tooling still
apply; a resource-only session does not establish provider availability.
## Desktop installation directories
When an installation directory is supplied and its runtime is unknown, use
+13 -3
View File
@@ -1,5 +1,10 @@
name: Cache npm package downloads
description: Restore npm's download cache and save it only from trusted default-branch events.
description: Restore npm's download cache. Trusted default-branch jobs that opt in may save it.
inputs:
save:
description: Save the shared download cache when this event is allowed to write it.
required: false
default: "false"
outputs:
can-save:
description: Whether this run may update the shared npm download cache.
@@ -23,20 +28,25 @@ runs:
can_save=true
fi
cache_key="npm-download-${RUNNER_OS}-${runtime_arch}-${LOCKFILES_HASH}"
# Same platform only. A lockfile change can reuse tarballs from the previous key.
restore_key="npm-download-${RUNNER_OS}-${runtime_arch}-"
{
printf 'can-save=%s\n' "${can_save}"
printf 'path=%s\n' "${cache_path}"
printf 'key=%s\n' "${cache_key}"
printf 'restore-key=%s\n' "${restore_key}"
} >> "${GITHUB_OUTPUT}"
- name: Restore and save npm package downloads
if: steps.context.outputs.can-save == 'true'
if: steps.context.outputs.can-save == 'true' && inputs.save == 'true'
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ steps.context.outputs.path }}
key: ${{ steps.context.outputs.key }}
restore-keys: ${{ steps.context.outputs.restore-key }}
- name: Restore npm package downloads without saving
if: steps.context.outputs.can-save != 'true'
if: steps.context.outputs.can-save != 'true' || inputs.save != 'true'
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: ${{ steps.context.outputs.path }}
key: ${{ steps.context.outputs.key }}
restore-keys: ${{ steps.context.outputs.restore-key }}
+64
View File
@@ -0,0 +1,64 @@
name: Cache retention
on:
workflow_run:
workflows: ["CI"]
types: [completed]
# Qualification calls ci.yml with actions: read. A nested workflow cannot raise
# that token, so cache deletion stays in this workflow.
permissions:
contents: read
actions: write
concurrency:
group: cache-retention-${{ github.event.workflow_run.head_sha }}
cancel-in-progress: false
jobs:
retain:
name: Retain Turbo snapshots and npm download caches
# Restrict the triggering-source checkout to same-repository default-branch
# CI events. The helper runs from this trusted workflow revision; triggering
# source is used only for cache identity and is never executed. Static and
# build save snapshots before later lanes finish, so failed, timed-out, or
# cancelled runs still retain those snapshots.
if: >-
(github.event.workflow_run.conclusion == 'success' ||
github.event.workflow_run.conclusion == 'failure' ||
github.event.workflow_run.conclusion == 'timed_out' ||
github.event.workflow_run.conclusion == 'cancelled') &&
github.event.workflow_run.head_repository.full_name == github.repository &&
github.event.workflow_run.head_branch == github.event.repository.default_branch &&
(github.event.workflow_run.event == 'push' ||
github.event.workflow_run.event == 'schedule' ||
github.event.workflow_run.event == 'workflow_dispatch')
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
ref: ${{ github.workflow_sha }}
path: trusted-retention
persist-credentials: false
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
ref: ${{ github.event.workflow_run.head_sha }}
path: triggering-source
persist-credentials: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: trusted-retention/.nvmrc
- name: Retain current Turbo snapshots and npm download caches
env:
GH_TOKEN: ${{ github.token }}
CI_REPOSITORY: ${{ github.repository }}
CACHE_PREFIX: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-
CACHE_KEEP_KEY: rea-static-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('triggering-source/.nvmrc', 'triggering-source/.npmrc', 'triggering-source/package-lock.json') }}-${{ github.event.workflow_run.head_sha }}
CACHE_DEEP_PREFIX: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-
CACHE_DEEP_KEEP_KEY: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('triggering-source/.nvmrc', 'triggering-source/.npmrc', 'triggering-source/package-lock.json') }}-${{ github.event.workflow_run.head_sha }}
CACHE_KNIP_PREFIX: knip-meta-v1-${{ runner.os }}-
CACHE_KNIP_KEEP_KEY: knip-meta-v1-${{ runner.os }}-${{ hashFiles('triggering-source/knip.json', 'triggering-source/tsconfig.json', 'triggering-source/package.json', 'triggering-source/package-lock.json') }}-${{ github.event.workflow_run.head_sha }}
CACHE_PORTABLE_PREFIX: rea-portable-build-v2-${{ runner.os }}-${{ runner.arch }}-
DEDUPE_NPM_DOWNLOADS: "true"
run: node trusted-retention/scripts/ci/cache-retention.mjs
+131 -11
View File
@@ -15,6 +15,15 @@ on:
description: Exact qualification source; must match the caller revision.
required: true
type: string
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted release qualification passes it.
required: false
permissions:
contents: read
@@ -29,6 +38,15 @@ concurrency:
env:
HUSKY: "0"
# Set Actions secret TURBO_TOKEN and variable TURBO_TEAM. OIDC can mint
# TURBO_TOKEN via vercel/setup-turborepo-remote-cache-action; do not commit a token.
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Release qualification sets remote_cache_write and passes the
# token. Open pull requests get an empty token and the local cache only.
# Remote reads need a credential, and pull requests do not receive one.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
jobs:
changes:
@@ -128,6 +146,9 @@ jobs:
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
with:
# Linux x64 writer. Parallel Linux jobs restore this key.
save: "true"
- run: npm ci
- name: Restore static task cache
if: needs.changes.outputs.code == 'true'
@@ -151,14 +172,10 @@ jobs:
if: needs.changes.outputs.code == 'true'
env:
CI_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
CI_STATIC_CONTEXT_SCRIPT: check:ci:fast:tasks
run: npm run ci:static-context
- if: needs.changes.outputs.code == 'true'
run: npm run check:ci
- if: needs.changes.outputs.code == 'true'
run: npm run verify:test-discovery
- name: Check LLDB producer retention and launch contracts
if: needs.changes.outputs.code == 'true'
run: npm run test:python
run: npm run check:ci:fast
- if: needs.changes.outputs.code != 'true'
run: npm run format:check
- name: Save validated static task cache
@@ -180,6 +197,72 @@ jobs:
if-no-files-found: ignore
retention-days: 7
static-deep:
name: Deep static checks
needs: changes
# Full qualification only. Provider lanes wait on fast static, not this job.
if: needs.changes.outputs.tier == 'full' && needs.changes.outputs.code == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
- run: npm ci
- name: Restore deep static task cache
id: static-cache
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: .turbo/cache/
key: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-${{ github.sha }}
restore-keys: rea-static-deep-v1-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('.nvmrc', '.npmrc', 'package-lock.json') }}-
- name: Restore Knip analysis cache
id: knip-cache
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: node_modules/.cache/knip
# Each trusted commit publishes a new entry. Restore still matches the
# configuration prefix from the previous successful full run.
key: knip-meta-v1-${{ runner.os }}-${{ hashFiles('knip.json', 'tsconfig.json', 'package.json', 'package-lock.json') }}-${{ github.sha }}
restore-keys: knip-meta-v1-${{ runner.os }}-${{ hashFiles('knip.json', 'tsconfig.json', 'package.json', 'package-lock.json') }}-
- name: Record deep static task source and execution identity
continue-on-error: true
env:
CI_PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
CI_STATIC_CONTEXT_SCRIPT: check:ci:deep:tasks
run: npm run ci:static-context
- name: Run deep static checks
run: npm run check:ci:deep
- name: Save validated deep static task cache
if: >-
success() && steps.npm-cache.outputs.can-save == 'true' &&
steps.static-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: .turbo/cache/
key: ${{ steps.static-cache.outputs.cache-primary-key }}
- name: Save Knip analysis cache
if: >-
success() && steps.npm-cache.outputs.can-save == 'true' &&
steps.knip-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: node_modules/.cache/knip
key: ${{ steps.knip-cache.outputs.cache-primary-key }}
- name: Upload complete deep static diagnostics
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: static-deep-check-reports
path: .cache/static-checks/
include-hidden-files: true
if-no-files-found: ignore
retention-days: 7
test-source:
name: Source tests
needs: changes
@@ -219,9 +302,11 @@ jobs:
# every shard's diagnostics and coverage. Passing PRs run the full suite.
fail-fast: ${{ github.event_name == 'pull_request' && needs.changes.outputs.tier == 'fast' }}
matrix:
# Fast feedback spends the same six test jobs on one source group and
# five built groups; full qualification retains six complete shards.
shard: ${{ fromJSON(needs.changes.outputs.tier == 'fast' && '["1/5","2/5","3/5","4/5","5/5"]' || '["1/6","2/6","3/6","4/6","5/6","6/6"]') }}
# Fast feedback uses one source group and eight built groups so the
# slowest built shard job stays inside a ten-minute execution budget.
# Classification, build, and report merge sit outside that job.
# Full qualification retains six complete shards with coverage.
shard: ${{ fromJSON(needs.changes.outputs.tier == 'fast' && '["1/8","2/8","3/8","4/8","5/8","6/8","7/8","8/8"]' || '["1/6","2/6","3/6","4/6","5/6","6/6"]') }}
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
@@ -373,8 +458,9 @@ jobs:
retention-days: 7
sqlite-runtime:
# Preserve all selected coverage, but admit expensive host/provider work
# only after static validation. Portable tests still provide early feedback.
# Admit expensive host/provider work after fast static validation
# (typecheck, lint, format). Deep static does not gate these lanes.
# Portable tests still provide early feedback.
needs: [changes, static, build]
if: needs.changes.outputs.sqlite == 'true'
uses: ./.github/workflows/sqlite-runtime.yml
@@ -577,6 +663,9 @@ jobs:
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
with:
# ubuntu-latest is the Linux x64 writer above. Other hosts save their own key.
save: ${{ matrix.os != 'ubuntu-latest' }}
- name: Verify native runtime identity
env:
EXPECTED_CI_PLATFORM: ${{ matrix.platform }}
@@ -713,6 +802,8 @@ jobs:
- name: Cache npm package downloads
id: npm-cache
uses: ./.github/actions/npm-cache
with:
save: "true"
- run: npm ci
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
@@ -858,68 +949,96 @@ jobs:
skills/reverse-engineer-anything
if-no-files-found: error
# Provider lanes wait for fast static and the portable build, not deep static.
browser:
needs: [changes, static, build]
if: needs.changes.outputs.browser == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-browser.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
binary_layout:
needs: [changes, static, build]
if: needs.changes.outputs.binary_layout == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-binary-layout.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
evm:
needs: [changes, static, build]
if: needs.changes.outputs.evm == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-evm-interface.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
javascript_recovery:
needs: [changes, static, build]
if: needs.changes.outputs.javascript_recovery == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-javascript-recovery.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
recorded_crash:
needs: [changes, static, build]
if: needs.changes.outputs.recorded_crash == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-recorded-crash.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
web_network:
needs: [changes, static, build]
if: needs.changes.outputs.web_network == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-web-network-captures.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
web_runtime:
needs: [changes, static, build]
if: needs.changes.outputs.web_runtime == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-web-runtime.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
web_source_map:
needs: [changes, static, build]
if: needs.changes.outputs.web_source_map == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-web-source-map.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
agentenv:
needs: [changes, static, build]
if: needs.changes.outputs.agentenv == 'true'
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
uses: ./.github/workflows/real-agentenv.yml
with:
build_cache_artifact: rea-build-cache
remote_cache_write: ${{ inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')) }}
nativeaot:
needs: [changes, static]
@@ -944,6 +1063,7 @@ jobs:
[
changes,
static,
static-deep,
build,
reverse-tools-quality,
test-shard,
+5
View File
@@ -14,6 +14,11 @@ concurrency:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
jobs:
mutation:
+16
View File
@@ -9,12 +9,28 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
NODE_OPTIONS: --max-old-space-size=768
PIP_DISABLE_PIP_VERSION_CHECK: "1"
+16
View File
@@ -14,6 +14,15 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
@@ -21,6 +30,13 @@ permissions:
env:
NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
REA_VERIFY_LARGE_ELF_MCP: ${{ inputs.large_mcp && '1' || '0' }}
jobs:
+16
View File
@@ -8,10 +8,26 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
permissions:
contents: read
+8
View File
@@ -8,6 +8,14 @@ on:
permissions:
contents: read
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
jobs:
windows-native-artifact:
name: Build Windows bridge security boundary
+16
View File
@@ -9,6 +9,15 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
@@ -16,6 +25,13 @@ permissions:
env:
NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
jobs:
real-evm-interface:
@@ -6,6 +6,11 @@ on:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
permissions:
contents: read
+5 -12
View File
@@ -13,20 +13,8 @@ on:
- "src/composition/go.ts"
- "src/cli/goCommands.ts"
- "src/server/registerGoTools.ts"
- "src/server/createServer.ts"
- "src/cli.ts"
- "src/cliCommandNames.ts"
- "src/contracts/toolContracts.ts"
- "src/contracts/toolEffects.ts"
- "src/artifacts/readStableArtifact*.ts"
- "src/artifacts/ArtifactReader.ts"
- "src/process/PrivateRuntimeRoot.ts"
- "scripts/verify-go-binary.mjs"
- "scripts/lib/mcp-verifier-results.mjs"
- "scripts/rea.mjs"
- "package.json"
- "package-lock.json"
- ".nvmrc"
permissions:
contents: read
@@ -38,6 +26,11 @@ concurrency:
env:
NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
jobs:
real-go-binary:
+5
View File
@@ -5,6 +5,11 @@ on:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
permissions:
contents: read
+5
View File
@@ -5,6 +5,11 @@ on:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
permissions:
contents: read
@@ -8,6 +8,15 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
permissions:
@@ -15,6 +24,13 @@ permissions:
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
NODE_OPTIONS: "--max-old-space-size=768"
RAYON_NUM_THREADS: "1"
+7 -11
View File
@@ -20,21 +20,17 @@ on:
- "src/composition/captures.ts"
- "src/cli/captureCommands.ts"
- "src/server/registerCaptureTools.ts"
- "src/server/createServer.ts"
- "src/server/sessionAvailabilityPolicy.ts"
- "src/application/CapabilityInventory.ts"
- "src/contracts/toolContracts.ts"
- "src/contracts/toolEffects.ts"
- "src/cli.ts"
- "src/cliCommandNames.ts"
- "src/process/**"
- "src/artifacts/readStableArtifact.ts"
- "scripts/verify-packet-capture.mjs"
- ".github/workflows/real-packet-capture.yml"
- "package*.json"
- ".nvmrc"
permissions:
contents: read
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
concurrency:
group: real-packet-capture-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
+16
View File
@@ -9,6 +9,15 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
@@ -16,6 +25,13 @@ permissions:
env:
NODE_OPTIONS: --max-old-space-size=768
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
UV_CONCURRENT_BUILDS: "1"
UV_CONCURRENT_INSTALLS: "1"
UV_PYTHON_DOWNLOADS: never
+7 -11
View File
@@ -13,21 +13,17 @@ on:
- "src/composition/signals.ts"
- "src/cli/signalCommands.ts"
- "src/server/registerSignalTools.ts"
- "src/server/createServer.ts"
- "src/server/sessionAvailabilityPolicy.ts"
- "src/application/CapabilityInventory.ts"
- "src/contracts/toolContracts.ts"
- "src/contracts/toolEffects.ts"
- "src/cli.ts"
- "src/cliCommandNames.ts"
- "src/process/**"
- "src/artifacts/readStableArtifact.ts"
- "scripts/verify-signal-recording.mjs"
- ".github/workflows/real-signal-recording.yml"
- "package*.json"
- ".nvmrc"
permissions:
contents: read
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
concurrency:
group: real-signal-recording-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
+5
View File
@@ -12,6 +12,11 @@ permissions:
env:
HUSKY: "0"
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
jobs:
real-termux:
@@ -9,12 +9,28 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
NODE_OPTIONS: --max-old-space-size=768
jobs:
+16
View File
@@ -7,11 +7,27 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
jobs:
real-web-runtime:
runs-on: ubuntu-latest
+16
View File
@@ -7,11 +7,27 @@ on:
required: false
type: string
default: ""
remote_cache_write:
description: Trusted release qualification may publish the remote cache.
required: false
type: boolean
default: false
secrets:
TURBO_TOKEN:
description: Remote cache token. Only trusted callers pass it.
required: false
workflow_dispatch:
permissions:
contents: read
env:
HUSKY: "0"
# A called workflow sees the caller event, so workflow_call is not an event
# name here. Writes when remote_cache_write is set, or on the default branch
# for push, schedule, and manual dispatch. Open pull requests stay empty-token
# and use the local cache only.
TURBO_TOKEN: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (inputs.remote_cache_write == true || (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'))) && 'local:rw,remote:rw' || 'local:rw' }}"
jobs:
real-web-source-map:
runs-on: ubuntu-latest
+9
View File
@@ -21,6 +21,12 @@ on:
env:
HUSKY: "0"
# Remote cache stays local when either value is empty.
# Writes only on the default branch for push, schedule, manual dispatch,
# and workflow_call. Pull requests get an empty token and remote read-only mode.
TURBO_TOKEN: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && secrets.TURBO_TOKEN || '' }}"
TURBO_TEAM: ${{ vars.TURBO_TEAM }}
TURBO_CACHE: "${{ (github.ref == format('refs/heads/{0}', github.event.repository.default_branch) && (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call')) && 'local:rw,remote:rw' || 'local:rw,remote:r' }}"
concurrency:
group: release-${{ inputs.release_branch || github.event.pull_request.merge_commit_sha || github.ref_name }}
@@ -301,8 +307,11 @@ jobs:
contents: read
actions: read
uses: ./.github/workflows/ci.yml
secrets:
TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
with:
source_sha: ${{ needs.merged-release.outputs.sha || needs.release-please.outputs.sha }}
remote_cache_write: true
rehearsal:
name: Rehearse qualified release on a clean runner
+28 -7
View File
@@ -180,8 +180,21 @@ rea trace-dylib-resolution MyApp.app --root Contents/MacOS/MyApp --architecture
Load commands are parsed in TypeScript from Mach-O and FAT headers, including
the `dylib_use_command` encoding newer linkers emit for flags such as delayed
initialization. Nothing is executed and no Apple tool is required, so the tool
also works off macOS.
initialization, and `LC_LAZY_LOAD_DYLIB_INFO` (`0x3a`). That command stores its
install name in a linkedit payload rather than in the load command itself.
Its flag and chain-start image offsets must lie in the slice VM range, which
is dyld's one-page floor extended by every segment except `__LINKEDIT`,
measured from the `__TEXT` vmaddr. Payload strings must start after the 24-byte
header. The load-path NUL must occur before the payload's final byte, matching
dyld's scan. Distinct payload bytes and payload-range lookups stay within the 16 MiB
limit, and a file range already read is reused. String scanning and UTF-8
checks count every reference, including repeats and overlaps, and stop at that
same aggregate. A slice retains at most 1024 of these commands; further copies are
unsupported instead of becoming one traced edge each. Symbol references,
including repeats, stop at 65536 per slice. A larger table is unsupported
instead of caching one string per offset.
Nothing is executed and no Apple tool is required, so the tool also works off
macOS.
Resolution follows dyld:
@@ -215,15 +228,23 @@ root supplies. Every edge below it has `loader_conditional: true`, and a
request that reuses it by install name is `conditional` too. Findings on those
edges say that they apply only if the conditional image loads.
Absolute install names such as `/usr/lib/libSystem.B.dylib` are
`outside-target`. On macOS 11 and later most of them live in the dyld shared
cache rather than on disk, so REA does not check them against the host.
Absolute install names that name a path beneath the analyzed root are checked
within that root, with symlinks and `..` resolved by the same bounded tree
resolver. Absolute paths outside the root, such as
`/usr/lib/libSystem.B.dylib`, are `outside-target`; on macOS 11 and later most
system libraries live in the dyld shared cache rather than on disk, so REA does
not check them against the host. Mach-O path strings are not converted from
`file:` URLs or looked up through unrelated host paths.
`findings` are derived, not observed:
- `required-load-unresolved`, `weak-load-unresolved`, and `lazy-load-unresolved`.
`LC_LAZY_LOAD_DYLIB` dependencies are resolved but not traversed, because dyld
loads them on first use; an unresolved one is not a launch failure.
`LC_LAZY_LOAD_DYLIB` and `LC_LAZY_LOAD_DYLIB_INFO` dependencies are resolved
but not traversed, because dyld loads them on first use; an unresolved one is
not a launch failure. `LC_LAZY_LOAD_DYLIB_INFO` has no compatibility version,
so those fields are null. Its may-be-missing flag is reported as `weak`, and
an unresolved dependency with that flag is not a first-use failure: dyld
continues if the image is absent.
- `earlier-rpath-candidate-absent`: a compatible Mach-O placed at an earlier
modeled search path could take precedence, subject to unmodeled search inputs
and code-signing library validation. These findings do not establish which
+32 -2
View File
@@ -19,8 +19,24 @@ Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate
- `analyze_web_bundle` / `rea analyze-web-bundle` parses selected script artifacts without execution and derives chunk edges, route and endpoint candidates, vendor fingerprints, static WebMCP declarations, and optionally source-map/original-source evidence.
- `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata.
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
For `analyze_web_bundle`, each `webmcp_declarations` entry's
`schema_property_names` is a source-derived projection of statically known own
property keys, not schema validation. `schema_property_names_coverage` reports
how complete that projection is: `complete` means a literal properties object
was fully enumerated (including an empty literal); `partial` means a literal
object was available but some computed or spread membership was unresolved,
and the name array retains keys that were known; `unknown` means the schema or
properties owner could not be enumerated as an object literal, including
explicit non-object values; `not_present` means no effective schema or own
`properties` member was present after the `inputSchema ?? input_schema` alias
selection. `not_present` describes this projection only: it does not validate a
schema or rule out properties expressed through other schema constructs. Older
observations may omit the coverage field; treat omission as `unknown` rather
than inferring completeness from the array.
Passive DOM comparison checks complete node inventories and complete frame
parent/child structure independently. Frame IDs and sibling enumeration order
are capture-local and do not establish a change; duplicate IDs, missing parents,
@@ -288,6 +304,20 @@ Call `list_browser_targets` first because target IDs are browser-instance-specif
REA establishes a main-frame origin boundary before enabling Runtime, Debugger, or Network observation, rechecks it before document capture, and validates it again afterward. If the main frame navigates during final capture, REA discards the mixed result and returns `target_changed`.
The web-session timeline uses CDP monotonic seconds where the event producer
provides a comparable timestamp; absent or invalid timestamps are `null` rather
than a synthetic zero. Events buffered while REA samples the window boundary are
retained only when their timestamp places them after that boundary; older events
are excluded. A buffered event without a comparable timestamp may remain as
contextual evidence with `timestamp: null`, accompanied by a limitation stating
that it does not establish activity in the selected window. Main-frame scope
enforcement still applies to contextual navigation events. Network redirect
events retain their paired prior-hop response context when the timestamped
next-request event falls inside the window. The pre-arm timeline buffer is bounded
by original CDP message bytes. If clock sampling completes after that budget is
exhausted, the operation returns `payload_limit` instead of a partial window.
Cancellation and clock-request failures retain their own outcomes.
## Data minimization
The Link projection parses repeated CDP header field lines independently and
@@ -331,8 +361,8 @@ REA retains local observation data and removes only URL userinfo credentials, ex
WebSocket stream inventories and connection counts likewise require complete
WebSocket coverage. Redirect resource types use only shared known metadata:
an omitted type leaves equality unknown rather than proving a modification.
- Console observations with a stack source retain call type, argument types, timestamp, and source location. When console text capture is selected, already-delivered primitive values are retained verbatim; objects, getters, and remote properties are never expanded.
- WebSocket observations retain direction, opcode, and payload byte length. When shape capture is selected, text frames are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw frames are never retained.
- Console observations with a stack source retain call type, argument types, the CDP `Runtime.Timestamp` in milliseconds since Unix epoch, and source location. An absent or invalid timestamp is `null` and makes `console_events` incomplete; an observed zero remains zero. This epoch-millisecond clock is separate from the monotonic-second timestamps in `observe-web-session`. When console text capture is selected, already-delivered primitive values are retained verbatim; objects, getters, and remote properties are never expanded.
- WebSocket observations retain direction, opcode, and payload byte length. A missing or malformed CDP opcode is returned as `null`, with opcode-dependent payload byte length and shape also `null`, and marks `websocket_frames` incomplete. Missing or wrong-typed payload data is not treated as empty; its byte length is `null`, and a text payload shape is unknown. Numeric opcode `0` and an observed empty payload remain distinct from missing data. Opcode is nullable in the public inspection schema, so clients must handle unknown protocol values. When shape capture is selected, text messages are classified as text or complete value-free JSON shape; binary bytes, hashes, prefixes, and raw payloads are never retained.
- Storage observations always redact values. Key names, IndexedDB names, and cache names require `include_storage_keys`; stable content fingerprints require the additional `include_storage_fingerprints` selection. Cookie fingerprints cover cookies applicable to the current main-frame URL, including its path, rather than every cookie path on that origin. Other storage remains scoped to the origin. Cache fingerprints hash all decoded response bytes, including bodies above 64 KiB, subject to the [64 MiB per-message CDP limit](#request-boundary). Missing or undecodable cache bodies and partial IndexedDB remote objects make fingerprint coverage incomplete, so identical observations remain `unknown`.
- Unavailable or malformed storage inventories mark `storage_keys` unavailable, even without fingerprints. Comparisons suppress inferred key additions and removals while preserving modifications proven by complete fingerprints on both sides. Previously exported captures are not retroactively corrected; recapture them to obtain these coverage markers.
- Script metadata is included only when CDP supplies a URL on an allowed origin. Stable keys exclude transient CDP script IDs, and exact transient raw URLs are used only during script/resource reconciliation. URL-less scripts are excluded because their origin cannot be established. When CDP supplies an execution-context association, the accepted script retains its authorized frame ID for later attribution. Script metadata admission bounds retained identity and source-map strings to 8 MiB and 50,000 records. When a source-map URL cannot fit, the script identity remains and source-map coverage reports `resource_budget_exhausted`; rejected script observations are counted without retaining their IDs. Source content is omitted unless explicitly requested and is returned inline as a self-verifying artifact with its SHA-256 digest.
+13 -10
View File
@@ -91,17 +91,20 @@ same bundle. Export writes only the caller-selected output file.
## Facts and uncertainty
| Family | Evidence used |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| .NET | A PE optional header declares a nonempty CLI directory. |
| Unity Mono | A managed assembly below `*_Data/Managed` or `Contents/Resources/Data/Managed` has an exact UnityEngine assembly reference. |
| Unity IL2CPP | An IL2CPP metadata header coexists with a conventionally named native `GameAssembly.dll`, `GameAssembly.so`, `GameAssembly.dylib` or `libil2cpp.so` image. Build correspondence remains unknown. |
| Godot | A `GDPC` header at file start or a validated executable trailer offset. |
| GameMaker | A bounded `FORM` container starts with a `GEN8` chunk. |
| Unreal | `Content/Paks` path conventions or an IoStore TOC signature; arbitrary `.pak` filenames are insufficient. |
| Electron | A `resources/app.asar` or `resources/default_app.asar` path convention; archive validity is unverified. |
| Family | Evidence used |
| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| .NET | A PE optional header declares a nonempty CLI directory. |
| Unity Mono | A managed assembly below `*_Data/Managed` or `Contents/Resources/Data/Managed` has an exact UnityEngine assembly reference. |
| Unity IL2CPP | A recognized IL2CPP metadata magic (with the raw version when complete) coexists with a conventionally named native `GameAssembly.dll`, `GameAssembly.so`, `GameAssembly.dylib` or `libil2cpp.so` image. Build correspondence remains unknown. |
| Godot | A `GDPC` header at file start or a validated executable trailer offset. |
| GameMaker | A bounded `FORM` container starts with a `GEN8` chunk. |
| Unreal | `Content/Paks` path conventions or an IoStore TOC signature; arbitrary `.pak` filenames are insufficient. |
| Electron | A `resources/app.asar` or `resources/default_app.asar` path convention; archive validity is unverified. |
An observed metadata header with supported raw version 29, 31 or 39 offers an
When the four-byte metadata magic is present but version bytes are incomplete,
the fact retains `raw_version: null`; it does not offer an IL2CPP metadata
inspection probe. A complete magic/version prefix with supported raw version 29,
31 or 39 offers an
`inspect_il2cpp_metadata` follow-up at its admitted file path, including when no
native library was found. This probe does not establish full metadata validity
or correspondence to a native binary. See [IL2CPP metadata](il2cpp-metadata.md).
+16
View File
@@ -6,6 +6,22 @@ For remote devices, pass a host with an optional port (`host[:port]`; bracket IP
MCP exposes `list_frida_devices`, `list_frida_processes`, `start_frida_session`, `load_frida_script`, `resume_frida_session`, `unload_frida_script`, `frida_session_status`, and `close_frida_session`. A spawned process starts paused so scripts can load first. Closing unloads session scripts and detaches. If a spawn remains paused, REA resumes it only after script cleanup or session detach confirms the instrumentation is released; if both cleanup paths fail, the target stays paused and the cleanup error is retained. REA never kills the target. MCP sessions end with the server process. The one-shot CLI command `instrument-with-frida` accepts one inline script or script file, observes messages for a bounded duration, and returns Evidence with the source hash and captured messages.
For state and capture-coverage checks, pass `{"session_id":"SESSION_ID","detail":"summary"}` to `frida_session_status`. Each call reads the observed session state and retains the complete observation. The reply is an `inspect_analysis_view` Evidence summary with the session identity, state, scripts, `messages_retained`, and `messages_truncated`; message payloads remain in its `parent_evidence_id`. The count covers retained queued messages, not every target event. Omitted `detail` or `detail=complete` preserves the existing complete reply.
A summary's selected fields look like this (excerpt):
```json
{
"summary": {
"state": "running",
"messages_retained": 50,
"messages_truncated": false
}
}
```
Use `get_evidence_bundle` with `detail="summary"` and `filters={"evidence_id":"PARENT_ID"}` to discover parent metadata without message payloads. Filters apply only to summary delivery. Use `export_evidence_bundle` for the complete canonical bundle, then select the record whose `evidence_id` matches `normalized_result.parent_evidence_id` from the status summary. To use the existing CLI `inspect-analysis-view`, extract that record into a standalone Evidence JSON file and select it as an Evidence-file source with `view={"kind":"summary"}`. Reading retained Evidence does not refresh a live session or verify a process after exit or PID reuse.
Failed startup cleanup retains acquired Frida handles, including a paused spawn that has not attached, for later cleanup retries. Its remote connection stays leased until session detach and target resume complete. CLI commands await provider cleanup before returning; MCP retains these resources for cleanup when its server closes. Server shutdown stops admitting new Frida operations and waits for admitted startup, discovery, and queued work before releasing their resources. A failed cleanup remains reported and can be retried by closing the server again, rather than treating an unregistered session as fully released.
Frida scripts execute inside the selected target with that process's memory, filesystem, and network authority. Message capture is bounded to 16 MiB and reports truncation. Script source is not stored in Evidence; its selected path and SHA-256 digest are. For a file, `source_sha256` identifies the original selected bytes. Invalid UTF-8 continues to use replacement decoding; Evidence then records that limitation and `raw_result.source_representation` with the encoding, decoding mode, and `loaded_source_sha256` of the UTF-8 string passed to Frida. Valid UTF-8 files and inline scripts do not need this additional representation fact. Evidence retains non-authentication target selection such as device or endpoint, mode, program or PID, and spawn arguments. Authentication values remain excluded. Results report the observed session and messages, while unsupported platform metadata remains unknown. A request cancelled before a Frida operation starts has no provider effects. If cancellation arrives during a native Frida call, REA waits for that call to return, then closes any session it created and returns the observed session as a partial observation. A cancelled script load on an existing session releases only its new script before queued operations can run; the session and previously loaded scripts remain available. If cancellation is observed when the native call returns, partial Evidence retains the source identity and captured messages even if that call rejects, including messages emitted during awaited cleanup. The caller's cancellation reason remains primary and a native rejection is retained as a sanitized diagnostic. A failure already observed before cleanup stays primary if cancellation arrives during cleanup. The cancellation cutoff is REA's observation of native completion, rather than a claim about unobservable native event order. `script_id` is REA's load-attempt identity. When creation rejects without returning a handle, `script_cleanup: "no_handle"` means REA acquired no handle to unload; it does not prove that the engine performed no work. Once the provider commits a successful load, later request cancellation does not unload that script; use `unload_frida_script` to release it. If script cleanup fails, the script remains owned and a paused target cannot resume until the pending instrumentation is released. Queued script loads, resumes, and unloads check cancellation before dispatching native effects. Cancellation Evidence separately reports whether a target resume was observed, whether instrumentation cleanup completed, and that final target liveness remains unknown. REA does not kill the target. Remote discovery retains its observations and reports a separate cleanup error if the temporary connection cannot be removed.
+7
View File
@@ -340,6 +340,13 @@ A scoped report has `scope.mode: "explicit"`. Its `scope_checks` determine
`environment_healthy` summarizes the full audit. `--target` adds a target check,
but the report remains audit-wide unless a scope option is also supplied.
An invalid `REA_LOG_LEVEL` fails every scope, including one client or the
installed skill, because the executable rejects it before any command. A
selected client also fails for every other setting that MCP startup rejects,
including `REA_GHIDRA_STARTUP_TIMEOUT_MS` and `REA_HOPPER_STARTUP_TIMEOUT_MS`.
Those settings still fail a selected provider and the audit-wide result. A
skill-only report leaves them informational.
Follow the remediation for the failed check. If a native target has several
available providers, choose one with `--provider` on the CLI or `provider_id`
on `open_binary`. A `capability_unavailable` failure carries
+11
View File
@@ -528,6 +528,17 @@ concrete version, selection source, and complete analysis profile through
choices return typed selection details. A selected provider is never replaced
automatically after a runtime failure.
`open_bundle_resources` selects a separate read-only app resource session
without resolving `CFBundleExecutable`, including when it is absent or
unresolvable. `binary_session` reports `session_kind: "bundle_resources"` and the
caller and canonical resource-root paths; it does not report a binary target or
deep-provider binding. `inspect_plist`, `decode_interface_builder`,
`inspect_asset_catalog`, and `inspect_keyed_archive` retain resource-root
provenance and use a null Evidence subject when executable identity is unknown.
Binary-only operations and binary analysis snapshots do not apply in this
session. `open_binary` can switch back to executable analysis, and
`close_binary` retires either session kind.
Each provider client receives `binary_session.analysis_run.run_id` before any
provider startup, including clients whose startup fails. `process_lineage` is
`not_observed` until a dynamic provider starts, then becomes `snapshots` with
+31 -1
View File
@@ -425,10 +425,25 @@ complete observation.
vtable descriptors. Records include exact virtual addresses/file offsets and
artifact evidence. Vtable indexes are metadata word offsets; witness indexes
start after the conformance header. Names unavailable in metadata remain null.
The `swift_class_vtable_descriptors` coverage counters count examined vtable
slots and successfully decoded slots, rather than type-section entries.
The `objc_class_method_ivar_metadata` counters count admitted class and
metaclass objects and successfully decoded objects.
Pointers are decoded through the image's fixups:
- `LC_DYLD_CHAINED_FIXUPS` rebases and binds, including the authenticated
arm64e formats;
- legacy `LC_DYLD_INFO` bind opcodes.
- legacy `LC_DYLD_INFO` rebase and bind opcodes.
A Swift conformance whose type or protocol reference is an external bind
keeps its name as `null` and carries the raw dyld identity in
`type_binding` or `protocol_binding`; that record and its facet coverage are
partial. Consumers of this output must handle nullable conformance names
and use the typed binding fields instead of substituting a symbol as a
display name. Indirect relative-pointer records also retain their binding
identity. A decoded rebase to virtual address zero remains an address; only
an unfixed zero in a nullable ABI field is treated as null. Incomplete or
conflicting fixup streams keep affected pointers unresolved; coverage reports
parsing and resource limits while retaining validated prefix facts.
So an external superclass resolves from its `_OBJC_CLASS_$_` bind (for
example `NSObject`). The `pointer_fixups` coverage facet names the mechanism.
@@ -466,6 +481,21 @@ complete observation.
that operation is unavailable; runtime failures remain failures rather than
being replaced by symbol-only results.
`get_objc_classes` keeps names from the provider inventory. It coalesces rows
only when their exact selected-image pointer slots decode to the same class
import, including its binding stream, raw library ordinal, resolved library, weak status, and addend. A literal filter is
applied to the original names before that join; if the filter selects only an
alias, the output keeps that exact row. `identity_coverage` distinguishes a
complete selected-inventory scan from partial, unsupported, and unknown cases.
Recognized Mach-O formats outside the native metadata reader's support retain
the provider's class labels with unknown import identity; malformed headers and
read failures remain errors.
The join also requires matching whole-artifact digests and matching provider-
owned selected-image facts: container, byte range, full raw CPU type/subtype,
and FAT table digest when present. A missing or overridden loader selection
remains unknown; the result describes agreement on the selected artifact image,
not an atomic observation of a running process or whole-program class coverage.
## Native desktop observation
`observe_native_ui` captures one explicitly selected existing PID/window ID.
+9 -2
View File
@@ -10,10 +10,17 @@ Supply an existing TShark 4.x installation **on Linux**, with util-linux
`prlimit` available at `/usr/bin/prlimit`. Override its absolute path with
`REA_TSHARK_PRLIMIT_COMMAND` if required. Unsupported hosts, including macOS
and Windows, are refused before inspection because this integration requires
an enforced address-space limit:
an enforced address-space limit.
TShark selection follows the same rule as [`inspect_pcap_capture`](pcap-analysis.md):
an absolute `REA_TSHARK_COMMAND`, otherwise `tshark` from the first absolute
`PATH` entry that holds an executable file. A relative `REA_TSHARK_COMMAND` is
refused rather than looked up. Either way the selected executable is resolved,
hashed and rechecked after each run, and Evidence records its
`selection_source` (`environment` or `path`):
```sh
export REA_TSHARK_COMMAND=/absolute/path/to/tshark
export REA_TSHARK_COMMAND=/absolute/path/to/tshark # optional when tshark is on PATH
rea inspect-packet-capture /artifacts/device.pcapng \
--fields ip.addr --fields udp.port --fields data.data \
--display_filter 'udp' --format json
+10 -1
View File
@@ -17,7 +17,16 @@ rea inspect-pcap-capture /captures/session.pcapng --json
```
Without explicit tool commands, this family resolves them through PATH and
records that selection source. Client inspection reads no capture file. Capture
records that selection source. On Linux and macOS it runs the tool found in the
first absolute PATH entry holding an executable file, the same selection the
[packet capture tools](packet-captures.md) make; relative PATH entries are
skipped. Windows keeps PATH/PATHEXT resolution in the process launcher. Client
inspection reads no capture file; on hosts where PATH/PATHEXT does not expose
an absolute candidate, its executable digest and resolved path are reported as
unresolved. Otherwise the canonical executable path and SHA-256 are reported.
The pcap provider and packet tools share the selected-tool identity check, and
changes to a resolved executable during a run fail with artifact-changed
Evidence. Capture
inspection requires Linux with util-linux prlimit or Windows with owned Job
Objects. macOS capture inspection returns a capability gap because the new
native memory boundary is unverified; its version probes remain available.
+135 -39
View File
@@ -69,8 +69,14 @@ Source projects also contain large capacity regressions; `test:fast` promises
no compiled-runtime prerequisite, not a fixed time budget.
`npm run lint` prints errors and a warning summary; `npm run jscpd` prints
duplication statistics. Complete reports remain in `.cache/static-checks/` and
CI uploads them as `static-check-reports`, including after a failed static gate.
duplication statistics. Complete reports remain in `.cache/static-checks/`.
The fast gate uploads them as `static-check-reports`. Full qualification
uploads duplication diagnostics from the deep job as
`static-deep-check-reports`, including after that gate fails.
Task-graph recording runs `check:ci:fast:tasks` and `check:ci:deep:tasks`
without the artifacts lock, so it can record hashes while `check:pr` or
`check:test` already holds that lock. The CI jobs still run the locked
`check:ci:fast` and `check:ci:deep` commands.
Use `npm run lint:details` or `npm run jscpd:details` for all console diagnostics.
Both duplicate commands scan production and test code with the same 3% limit;
the summary commands preserve checker failures and restore reports from cached
@@ -140,6 +146,17 @@ the fixture cache. These subprocesses are outside aggregate Vitest coverage;
subprocess coverage attachment remains disabled. Cold-start import checks launch
independently without this fixture cache.
`analyzeJavaScriptDirectory` stdio MCP cases in one `cliTest` file share one
server and that compile cache. The first case pays startup; later cases reuse
the process. File teardown waits for startup still in flight, closes a server
registered during that wait, and does not open a server after close. A failed
handshake stays the caller error when its cleanup also fails, including a
rejection without a reason. That cleanup stays owned until a later close or
call releases it. Another server does not open while the cleanup remains, and
a cleanup that already succeeded is not run again. Pass
`mcpCall` when a case needs its own server, environment, or lifecycle. CLI
cases still start a fresh process.
The Apple artifact verifier step shares a job-temporary bytecode cache among
its fresh CLI/MCP children via `REA_ARTIFACT_NODE_COMPILE_CACHE`. The artifact
helper maps this setting to child `NODE_COMPILE_CACHE`; npm, Turbo, and nested
@@ -454,8 +471,8 @@ partial diagnostics and preflights do not establish complete mutation outcomes
or qualify the release revision for mutation coverage.
Fast CI starts the source-only projects after classification, independently of
build and static checks, and runs five built-dependent shards after the build.
The six test jobs still select every portable test. Full qualification keeps six
build and static checks, and runs eight built-dependent shards after the build.
Those nine test jobs still select every portable test. Full qualification keeps six
complete Vitest shards with coverage. Neither tier retries tests. Canonical
project ownership lives in `scripts/lib/test-projects.mjs`, shared by Vitest,
CI group execution, and development-test preparation. A test importing compiled
@@ -481,14 +498,16 @@ real-system lanes remain separate jobs so one kind of evidence cannot stand in
for another. Workflow syntax and expression validation runs inside the required
static-check job when workflow changes select it, avoiding a separate runner.
Native, provider, fixture, SQLite compatibility, and website lanes wait for
static checks to succeed before allocating their runners. This avoids spending
those runner slots on revisions that already fail static validation. Portable
test shards and documentation checks remain independent of static results, so
they can still provide diagnostic feedback. The required gate rejects skipped
selected lanes when static checks fail; passing revisions retain all selected
coverage. This ordering reduces peak demand on invalid revisions but can add
static-check waiting to otherwise passing runs; it is not a measured reduction
in end-to-end latency.
the fast static checks (typecheck, built lint, and format) to succeed before
allocating their runners. Knip, duplication, architecture guards, test
discovery, and the Python producer run in the separate full-tier deep static
job and do not gate provider admission. Portable test shards and documentation
checks remain independent of static results, so they can still provide
diagnostic feedback. The required gate rejects skipped selected lanes when
static checks fail; on the full tier it also requires the deep static job.
Passing revisions retain all selected coverage. This ordering reduces peak
demand on invalid revisions but can add static-check waiting to otherwise
passing runs; it is not a measured reduction in end-to-end latency.
Core jobs have execution limits of five minutes for classification, ten minutes
for static checks, builds, docs, and report merging, and thirty minutes per
@@ -514,11 +533,35 @@ JSON artifact members without executing artifact content. Reports and limits
are retained in `ci-health-diagnostics`; its ten-minute job limit covers the
existing four-minute timing collection plus the bounded diagnostic stages.
Shard-count reduction remains deferred: the available successful timing sample
covers an older inventory, and file phase weights are not job wall time. The
one-source/five-built split retains six test jobs; it makes no hosted speedup
claim. Current complete timings and completed candidate hosted runs are needed
before enabling a smaller adaptive matrix.
Fast built shards target under ten minutes of uncontended shard-job execution.
That figure is the `test-shard` job. The job waits for classification and the
build, and the portable-test merge runs after the slowest shard. On 2026-10-20,
run `39451548945` measured those surrounding jobs with runner queue excluded:
classification 15 seconds, build 49 seconds, and the portable-test merge 29
seconds. Shard 1/5 of that run spent 22 seconds before built Vitest and 3
seconds after it, including `npm ci` at 9 seconds. `node_modules` is about 674
MiB, so publishing it as a shared artifact would transfer more data than that
install saves. The npm download cache stays the setup cache.
File timings from that run put the built suite at 7944 seconds of module time.
The sequencer keeps each file's full duration on one worker. Extra workers
overlap other files in the same sequence group, and `process-boundary` runs
after that group. Scheduling the unsplit inventory onto five shards peaks at
820 seconds of tests. That run's slowest built shard job lasted 925 seconds
(shard 4/5), and shard 1/5 spent 874 seconds in built Vitest, so hosted job
time sits above the file schedule. Six shards of the same inventory peak at
684 seconds of tests. Eight shards, after the 578-second loader file is split
by case count (54 cases stay in that file and 46 move into each of three
files), peak at 513 seconds of tests. With the measured 25-second shard setup,
that shard job is about 538 seconds. Adding the measured classification, build,
and merge makes an uncontended portable critical path of about 631 seconds,
still before the gap between this file schedule and hosted Vitest. Runner queue
is separate and grows when more shard jobs compete for runners.
`process-boundary` stays serial because process-tree sampling shares the host.
Raising the worker cap from two to four on this four-core host cut a six-file
acceptance sample from 96–101 seconds to 71 seconds and did not remove the
single-file floor, so the cap stays two. Full qualification keeps six covered
shards; its longer instrumentation is outside this fast budget.
The separate `CI timings` workflow batches recent completed CI and Release runs
hourly or on manual dispatch. It uploads `ci-job-timings-batch` with raw API pages and
@@ -560,8 +603,9 @@ metadata, and available manifests into `.cache/ci-build-cache/`.
`portable-build-cache.json` records availability per task. Evicted or partially
missing entries are omitted as whole entries, and consumers rebuild cache misses,
including when every entry is unavailable. Malformed plans, non-file entries,
and I/O errors other than disappearance still fail publication. Older cross-run entries stay in the
original cache. The Windows lane relies on the Linux static job for typechecking. They retain the normal build and prepack
and I/O errors other than disappearance still fail publication. After a trusted
static run, superseded cross-run Turbo snapshots for that OS and architecture
are deleted down to the two newest generations. The Windows lane relies on the Linux static job for typechecking. They retain the normal build and prepack
commands, which restore matching outputs or rebuild on a cache miss. Native
artifacts, dependency installs, package archives, and runtime verification remain
host-local. Keep platform-dependent outputs out of this shared cache.
@@ -1110,7 +1154,7 @@ suite, without the source-test shards or native package
lanes. Classification compares the PR head with its merge base, so later
base-branch changes do not expand that scope. Static and portable-test aggregate jobs
remain present and fail if classification fails. Every code change retains the
complete six-shard Linux suite; individual test files are not selected by
complete portable Linux suite; individual test files are not selected by
imports or filenames.
Ordinary PRs also produce an **observational** test-selection plan. The candidate
@@ -1121,7 +1165,7 @@ tests. Shared inputs, unknown paths, missing owner tests, and removed tests fall
back to the complete inventory. Renames account for both paths. Main, scheduled,
manual, and `ci:full` runs always propose the full suite.
All six shards still execute the complete suite. The existing report merge
The source group and every built shard together still execute the complete suite. The existing report merge
compares the proposal with actual module outcomes and retains
`.cache/ci-test-selection/report.json` in `test-results`, including selected and
omitted paths, changed input paths, missed failures, unobserved files, scheduling weights, PR head and
@@ -1167,13 +1211,16 @@ scripts also select it. Unrelated code edits retain the complete portable suite
on the pinned Node version without starting four additional compatibility jobs.
Full qualification retains every SQLite version.
Shared process/contract/CLI boundaries, dependencies, other packaging inputs,
unknown paths, and changes to the planner select the full baseline.
unknown paths, and changes to the planner select the fast Linux x64 baseline:
portable code checks and the installed-package runtime lane, without provider
lanes or the multi-host package matrix. A shared file with exactly one owner
still selects that owner. Full qualification still selects every lane.
CI timing collection, timing history, and test-report helper edits retain static
checks and the complete portable suite without selecting unrelated host/provider
lanes. Classification, qualification, gate, release receipt, and unknown CI
scripts keep the broad fallback.
scripts keep that fast baseline.
Package script-only edits select their owning verifier when its entrypoint is
identifiable; unknown or shared script changes select everything. Android
identifiable; unknown or shared script changes select the fast baseline. Android
build-script edits retain the Linux suite and workflow validation; real Android
verification stays manual. Renames account for both paths.
@@ -1197,9 +1244,10 @@ or provider effects outside its owners.
Ordinary main pushes use the same owner scopes as PRs. Their diff starts at a
successful ancestor push run of this CI workflow, rather than the preceding
push: changes from failed or cancelled intermediate runs remain in scope.
Missing, untrusted, or non-ancestor history selects every lane. The lookup is
Missing, untrusted, or non-ancestor history selects the fast Linux x64
baseline: portable checks and the Linux runtime lane. The lookup is
bounded to the last 100 successful push runs; lookup failures broaden selection.
Every code change still selects all six portable shards, without coverage
Every code change still selects the source group and all eight fast built shards, without coverage
instrumentation in this fast tier. Ordinary PR matrices cancel remaining
shards after the first failure; passing PRs execute the complete suite. Main,
`ci:full`, scheduled, manual, and release qualification retain every shard's
@@ -1227,8 +1275,11 @@ immediately starts the separate `Full PR qualification` workflow against the
label event's merge revision; subsequent PR revisions also select the full tier
in normal CI while that label remains. Unrelated labels do not start validation
jobs or replace the main CI checks. Manual dispatch also requests full validation.
Fast-tier broad fallbacks retain all owner lanes and the three-host
package matrix; Intel macOS and coverage remain full-tier obligations. Each
Fast-tier broad fallbacks keep the portable suite and the Linux x64 runtime
baseline. They do not start every owner lane or the three-host package
matrix. Intel macOS and coverage remain full-tier obligations. Real provider
lanes stay path-gated, and a shared registration file listed by many lanes
does not start those lanes on a fast PR. Each
classification job records its selection and broad-fallback reasons in the run
summary. Representative Git-diff cases live in
`tests/boundary/filesystem/ciChangeScope.test.ts`.
@@ -1308,9 +1359,16 @@ report in each shard's snapshot. The bounded lookup verifies the artifact's
producer; newer schedules that reuse qualification without producing timings do
not hide the last full run's artifact.
CI assigns discovered files by historical duration, with a deterministic
fallback weight for new files, rather than dividing only by file count. Every
discovered file still belongs to exactly one shard; worker limits and project
ordering remain unchanged. The merge job rejects successful reports with missing,
fallback weight for new files, rather than dividing only by file count.
Each file keeps its full duration on one worker. Extra workers overlap other
files in the same sequence group, and `process-boundary` runs after that
group. History published before the competing-loader split still names only
`javascriptLiteralPercentPaths.test.ts`. Until that artifact records one of
the three new files, the sequencer assigns that recorded duration by the
pre-split case counts: 54/192 to the remaining file and 46/192 to each new
file. Every discovered file still belongs to exactly one shard; worker limits
and project ordering remain unchanged. Fast qualification keeps eight built
shards without coverage. The merge job rejects successful reports with missing,
unexpected, or duplicate portable files. Coverage instrumentation and aggregate
thresholds remain enforced in full qualification.
Missing or unusable history falls back to Vitest's default assignment. Only
@@ -1318,18 +1376,56 @@ timing data is reused; live provider results are never cached. Turbo outputs are
restored by task hash, with trusted default-branch runs as cache writers and PRs
as restore-only consumers. The static job also restores its Turbo snapshot;
each task revalidates its own input hash before reusing outputs or diagnostics.
The Node pin and npm install settings are global hash inputs. Knip includes its ignore rules and VitePress
The Node pin and npm install settings are global hash inputs. Knip runs only
in full qualification, with `--cache` and a GitHub cache of
`node_modules/.cache/knip`. The saved key includes the commit SHA. Restore uses
the configuration prefix, so the next trusted full run can publish a refreshed
snapshot. That cache is an AST cache keyed by file mtime and size, so a
fresh checkout often misses it; Turbo still hashes the Knip task inputs. Knip
includes its ignore rules and VitePress
entry, so changes to those inputs cannot reuse an older successful check.
Jobs that run Turbo export `TURBO_TEAM`. A called workflow sees the caller
event, so `workflow_call` is not an event name inside CI. `TURBO_TOKEN` is set
when release qualification passes `remote_cache_write`, or when the ref is the
default branch and the event is push, schedule, or workflow_dispatch, and only
when that secret is available to the run. Release qualification passes
`TURBO_TOKEN` and `remote_cache_write` into the called workflow. Labeled
pull-request qualification does not. Pull requests receive an empty token and
`TURBO_CACHE=local:rw`. Remote reads require `TURBO_TOKEN`, and this repository
does not pass a separate read-only credential into pull requests. An empty
token leaves the local Turbo cache in place.
The shared npm-cache action caches downloads, not `node_modules`, and every
consumer still runs `npm ci`. Keys cover the runner OS, actual Node architecture,
root lockfiles, and `.npmrc`. Only push, schedule, and manual events on the
repository's default branch may save shared npm caches; PRs restore only.
Static snapshots use the same write authority and are saved only after the
static job's validation succeeds. Main warms the new cache namespace after
merge; a cold cache or changed lockfile falls back to the normal install.
This prevents each PR from saving another branch-scoped copy of the same
download cache. It does not remove runner assignment waits.
root lockfiles, and `.npmrc`. An exact-key miss restores the newest cache for
that OS and architecture, so a lockfile change reuses unchanged tarballs while
`npm ci` still verifies the lockfile. Only push, schedule, and manual events on
the repository's default branch may save shared npm caches, and only from the
opted-in writer for that host: Static checks for Linux x64, the package matrix
for every other included host, and the Windows curated lane. Every other job
restores without saving, so parallel jobs do not publish duplicate copies.
Pull requests restore only. Static snapshots are saved only after the static
job's validation succeeds. Cache deletion is a
separate workflow, because reusable qualification calls this workflow and
cannot raise its read-only token. After a push, schedule, or manual CI run on
the default branch finishes, including when a later lane fails, times out, or
is cancelled, that workflow runs on `ubuntu-latest` and deletes older
`rea-static-v1`, `rea-static-deep-v1`, and `rea-portable-build-v2` entries for
that runner's OS and architecture, plus older `knip-meta-v1` entries for that
OS. It keeps two generations plus the key that run saved. A fast run that did
not publish a deep or Knip snapshot still leaves the two newest entries in
those families. It
deletes an npm download record only when another record has the same key, cache
version, and git ref, and it keeps the most recently accessed copy. A different
version or ref is a different restore target and stays. One cache id is one
record: a repeated id keeps the later access time when its key, version, ref,
size, and creation time agree, and an inconsistent repeat deletes nothing. An
incomplete or unexpected cache list deletes nothing.
Distinct lockfile generations of the download cache stay, so the restore
prefix still has a seed.
Main warms the new cache namespace after merge; a cold cache falls back to the
normal install. This prevents each PR from saving another branch-scoped copy of
the same download cache. It does not remove runner assignment waits.
Portable owner-based test selection remains observational. Its reports compare
proposed omissions with the full run; incomplete or skipped observations do not
@@ -0,0 +1,32 @@
"""Count all actual client JSON inputs and preserve bounded family resource receipts."""
from pathlib import Path
import argparse, json, hashlib
import tiktoken
parser=argparse.ArgumentParser(description=__doc__)
parser.add_argument('--root',type=Path,required=True)
parser.add_argument('--output',type=Path,required=True)
args=parser.parse_args();lab=args.root;enc=tiktoken.get_encoding('cl100k_base')
compact=lambda value:json.dumps(value,ensure_ascii=False,separators=(',',':'),allow_nan=False)
count=lambda value:len(enc.encode(compact(value),disallowed_special=()))
def resources(root):
scope=json.loads(Path(str(root)+'.log').read_text().splitlines()[-1]);o=scope['payload_resource_observation']
assert scope['abort'] is None and scope['payload_resource_observation_verified'] and o['payload_exit_code']==0 and o['payload_cleanup_verified']
for event in o['resource_events'].values():assert not any(event.values())
return {'wall_seconds':scope['elapsed_seconds'],'cpu_seconds':o['cpu_stat']['usage_usec']/1e6,'family_peak_mib':o['memory_peak_bytes']/1048576,'payload_cleanup_verified':True,'final_whole_scope_counters_verified':scope['resource_counters_verified']}
wire=[];sdk=[]
for root in sorted(lab.glob('final-wire-*')):
if not root.is_dir():continue
report=json.loads((root/'report.json').read_text());requests=json.loads((root/'model-requests.json').read_text());assert report['status'] in ['passed','expected-incomplete-task'] and not report['cleanup_failures'] and report['appserver_exit_code']==0 and report['owned_scope_drain']['remaining_processes']==[]
rows=[]
for index,captured in enumerate(requests):
body=captured['body'];namespace=[n for n in body.get('tools',[]) if n.get('type')=='namespace' and n.get('name')=='mcp__rea_audit'];compaction=not namespace
if compaction:assert 'CONTEXT CHECKPOINT COMPACTION' in compact(body['input'][-1])
else:assert len(namespace)==1 and len(namespace[0]['tools'])==210
rows.append({'index':index+1,'kind':'compaction' if compaction else 'normal','whole_json_tokens_proxy':count(body),'rea_namespace_tokens_proxy':0 if not namespace else count(namespace[0]),'rea_function_count':0 if not namespace else len(namespace[0]['tools'])})
wire.append({'case':root.name,'task_completed':report['task_completed'],'mode':report['mode'],'budget':report['observations']['tool_output_budget'],'requests':len(rows),'compactions':sum(r['kind']=='compaction' for r in rows),'whole_json_tokens_proxy':sum(r['whole_json_tokens_proxy'] for r in rows),'request_details':rows,**resources(root)})
for root in sorted(lab.glob('final-sdk-*')):
if not root.is_dir():continue
report=json.loads((root/'report.json').read_text());assert report['status']=='passed' and not report['cleanup_failures'] and not report['receipt_failures']
sdk.append({'case':root.name,'provider':report['provider'],'symbol':report['symbol'],'detail':report['detail'],'warm':report['warm'],'tool_calls':len(report['calls']),'exported_bundle_bytes':(root/'evidence.json').stat().st_size,'disassembly_calls':[r for r in report['calls'] if r['phase'].startswith('disassembly:')],**resources(root)})
receipt={'measurement_scope':'Actual public SDK and Codex0.163 app-server with real GDB/BFD. Local deterministic SSE fixture and observed-ID checkpoint; no AI inference, model usage/quality/acceptance or billing claim. Whole compact JSON serialization tokens are proxies and include all schemas/replays/compaction. Incomplete default baseline is excluded from completed-workflow gains. Fresh processes have unflushed OS caches; warm SDK cases repeat the selected disassembly in the same server. Native target never executed.','tokenizer':{'package':tiktoken.__version__,'encoding':enc.name},'runtime_pins':{variant:{k:v for k,v in json.loads((lab/(variant+'-manifest.json')).read_text()).items() if k!='files'} for variant in ['baseline','candidate']},'native':json.loads((lab/'native-prerequisite-manifest.json').read_text()),'wire':wire,'sdk':sdk}
args.output.write_text(json.dumps(receipt,indent=2)+'\n');print(json.dumps({'wire':[{k:v for k,v in row.items() if k!='request_details'} for row in wire],'sdk':[{k:v for k,v in row.items() if k!='disassembly_calls'} for row in sdk]},indent=2))
@@ -0,0 +1,22 @@
from pathlib import Path
import struct,hashlib,json,argparse
parser=argparse.ArgumentParser(description='Independent ELF64 little-endian native-command byte oracle');parser.add_argument('--input',type=Path,required=True);parser.add_argument('--output',type=Path,required=True);args=parser.parse_args();path=args.input;b=path.read_bytes();assert b[:6]==b'\x7fELF\x02\x01'
shoff=struct.unpack_from('<Q',b,40)[0];entsize,n,strings=struct.unpack_from('<HHH',b,58)
assert entsize==64 and shoff+n*entsize<=len(b)
sections=[struct.unpack_from('<IIQQQQIIQQ',b,shoff+i*entsize) for i in range(n)]
rows=[]
for si,s in enumerate(sections):
if s[1]!=11:continue
text=sections[s[6]];names=b[text[4]:text[4]+text[5]]
assert s[9]==24 and s[5]%s[9]==0
for ordinal,off in enumerate(range(s[4],s[4]+s[5],24)):
name,info,other,section,value,size=struct.unpack_from('<IBBHQQ',b,off)
end=names.find(b'\0',name);assert end>=name
if names[name:end] not in [b'execute_command',b'execute_command_internal'] or info&15!=2:continue
target=sections[section];file_offset=target[4]+value-target[3]
assert target[4]<=file_offset and file_offset+size<=target[4]+target[5]
rows.append({'symbol_table_section':si,'symbol_ordinal':ordinal,'name':names[name:end].decode('ascii'),'address':value,'bytes':size,'file_offset':file_offset,'first_16_bytes_hex':b[file_offset:file_offset+16].hex()})
assert rows
result={'path':str(path),'artifact_sha256':hashlib.sha256(b).hexdigest(),'artifact_bytes':len(b),'elf_class':64,'byte_order':'little','elf_machine':struct.unpack_from('<H',b,18)[0],'symbols':rows,'scope':'Independent ELF64 little-endian dynamic symbol/section mapping; no target execution or REA parser.'}
args.output.write_text(json.dumps(result,indent=2)+'\n')
print(json.dumps(result))
@@ -0,0 +1,212 @@
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { finished } from "node:stream/promises";
import { pathToFileURL } from "node:url";
import { parseArgs } from "node:util";
import { createRetainedQueryCaller } from "./retained-query-calls.mjs";
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
const { values } = parseArgs({
options: {
runtime: { type: "string" },
input: { type: "string" },
oracle: { type: "string" },
output: { type: "string" },
provider: { type: "string" },
symbol: { type: "string" },
detail: { type: "string", default: "complete" },
warm: { type: "boolean", default: false },
},
});
for (const key of ["runtime", "input", "oracle", "output"])
assert.equal(values[key], resolve(values[key]));
assert.ok(["gdb", "objdump"].includes(values.provider));
assert.ok(["complete", "summary"].includes(values.detail));
const oracle = JSON.parse(await readFile(values.oracle, "utf8"));
assert.ok(oracle.symbols.some((row) => row.name === values.symbol));
const inputBytes = await readFile(values.input);
assert.equal(
createHash("sha256").update(inputBytes).digest("hex"),
oracle.artifact_sha256,
);
const require = createRequire(join(values.runtime, "package.json"));
const { Client } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client"))
);
const { StdioClientTransport } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio"))
);
await mkdir(values.output, { mode: 0o700 });
const account = await mkdtemp(join(tmpdir(), "rea-command-views-"));
const client = new Client({ name: "command-observation-task", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],
env: {
HOME: account,
PATH: process.env.PATH,
NODE_OPTIONS: "--max-old-space-size=2048",
UV_THREADPOOL_SIZE: "1",
REA_LOG_LEVEL: "silent",
},
stderr: "pipe",
});
const serverLog = createWriteStream(join(values.output, "server.log"));
const serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
transport.stderr?.pipe(serverLog);
const transcript = [];
const report = {
question:
"Read architecture and the complete byte-annotated disassembly of one selected native function; export complete Evidence. No inferior execution.",
provider: values.provider,
symbol: values.symbol,
detail: values.detail,
warm: values.warm,
calls: [],
observations: [],
};
const call = createRetainedQueryCaller(client, (receipt, event) => {
report.calls.push(receipt);
transcript.push(event);
});
let sessionId;
try {
await client.connect(transport);
const catalog = await client.listTools();
await writeFile(join(values.output, "catalog.json"), JSON.stringify(catalog));
const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));
const originalCall = client.callTool.bind(client);
client.callTool = (input, options) =>
originalCall(input, {
...options,
timeout: 60000,
toolDefinition: definitions.get(input.name),
});
const observe = async (name, input, phase = name) => {
const response = await call(name, input, phase);
assert.ok(!response.isError, JSON.stringify(response));
report.observations.push({
name,
phase,
evidence: response.structuredContent,
});
return response.structuredContent;
};
const completeParents = [];
const disassemble = async (phase) => {
const selection = values.detail === "summary" ? { detail: "summary" } : {};
const result =
values.provider === "gdb"
? await observe(
"gdb_console",
{
session_id: sessionId,
command: `disassemble /r ${values.symbol}`,
...selection,
},
phase,
)
: await observe(
"inspect_with_objdump",
{
path: values.input,
operation: "disassemble",
symbol: values.symbol,
...selection,
},
phase,
);
completeParents.push(result);
};
if (values.provider === "gdb") {
const started = await observe("start_gdb_session", {});
sessionId = started.session_id;
for (const command of [
"set auto-load off",
"set debuginfod enabled off",
`file ${values.input}`,
"show architecture",
])
await observe("gdb_console", { session_id: sessionId, command });
} else
await observe("inspect_with_objdump", {
path: values.input,
operation: "file_headers",
});
await disassemble("disassembly:fresh");
if (values.warm) await disassemble("disassembly:repeat");
if (sessionId) {
await observe("close_gdb_session", { session_id: sessionId });
sessionId = undefined;
}
await observe("export_evidence_bundle", {
path: join(values.output, "evidence.json"),
});
const bundle = JSON.parse(
await readFile(join(values.output, "evidence.json"), "utf8"),
);
for (const delivered of completeParents) {
const parent =
values.detail === "complete"
? delivered
: bundle.records.find(
(record) =>
record.evidence_id ===
delivered.normalized_result.parent_evidence_id,
);
assert.ok(parent);
assert.equal(parent.normalized_result.completion_status, "complete");
assert.equal(parent.normalized_result.output_truncated, false);
if (values.provider === "gdb") assert.equal(parent.subject, null);
else assert.equal(parent.subject.digest.sha256, oracle.artifact_sha256);
if (values.detail === "summary") {
const selected = delivered.normalized_result.summary;
const { available_counts: counts, ...facts } = selected;
const { console, target, log, ...records } = parent.normalized_result;
assert.deepEqual(
values.provider === "gdb" ? facts : selected,
values.provider === "gdb" ? records : parent.normalized_result,
);
if (values.provider === "gdb")
assert.equal(counts.console_utf16_units, console.length);
assert.equal(delivered.raw_result, null);
}
}
report.status = "passed";
} catch (error) {
report.status = "failed";
report.failure = String(error?.stack ?? error);
process.exitCode = 1;
} finally {
if (sessionId)
try {
const closed = await call(
"close_gdb_session",
{ session_id: sessionId },
"cleanup:gdb",
);
assert.ok(!closed.isError);
} catch (error) {
report.gdb_cleanup_failure = String(error);
process.exitCode = 1;
}
if (
!(await finalizeRetainedQueryReceipt({
report,
client,
transport,
account,
serverLog,
serverLogFinished,
output: values.output,
transcript,
}))
)
process.exitCode = 1;
}
@@ -0,0 +1,28 @@
from pathlib import Path
import json,re,hashlib,argparse
parser=argparse.ArgumentParser(description='Match every native instruction byte against independently mapped ELF file ranges');parser.add_argument('--root',type=Path,required=True);args=parser.parse_args();lab=args.root;oracle=json.loads((lab/'elf-oracle.json').read_text());image=Path(oracle['path']).read_bytes();assert hashlib.sha256(image).hexdigest()==oracle['artifact_sha256'];rows=[]
for directory in sorted(lab.iterdir()):
bundle=directory/'evidence.json'
if not bundle.is_file():continue
for parent in json.loads(bundle.read_text())['records']:
if parent['operation'] not in ['gdb_console','inspect_with_objdump']:continue
params=parent['parameters'];gdb=parent['operation']=='gdb_console'
symbol=params.get('command','').removeprefix('disassemble /r ') if gdb else params.get('symbol')
if symbol not in {s['name'] for s in oracle['symbols']}:continue
spec=next(s for s in oracle['symbols'] if s['name']==symbol)
report=parent['normalized_result'];assert report['completion_status']=='complete' and not report['output_truncated']
if gdb:
assert parent['subject'] is None
console='\n'.join(line for line in report['records'] if line.startswith('~'));assert console==report['console']
text=''.join(json.loads(line[1:]) for line in report['records'] if line.startswith('~'))
pattern=r'^\s*(?:=>\s*)?0x([0-9a-f]+)[^:]*:\s*((?:[0-9a-f]{2}(?:\s+|$))+)'
else:
assert parent['subject']['digest']['sha256']==oracle['artifact_sha256'];text=report['stdout'];pattern=r'^\s*([0-9a-f]+):\s*((?:[0-9a-f]{2}(?:\s+|$))+)'
address=spec['address'];collected=bytearray();count=0
for line in text.splitlines():
m=re.match(pattern,line)
if not m:continue
actual=int(m[1],16);raw=bytes.fromhex(m[2]);assert actual==address+len(collected),(directory,actual,address+len(collected));collected.extend(raw);count+=1
expected=image[spec['file_offset']:spec['file_offset']+spec['bytes']];assert collected==expected,(directory,len(collected),len(expected));rows.append({'case':directory.name,'mode':'gdb' if gdb else 'objdump','symbol':symbol,'byte_bearing_lines':count,'bytes':len(collected),'exact_file_bytes':True,'parent_evidence_id':parent['evidence_id']})
assert rows
(lab/'native-final-verification.json').write_text(json.dumps({'status':'passed','artifact_sha256':oracle['artifact_sha256'],'records':rows},indent=2)+'\n');print(json.dumps(rows))
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,139 @@
# Retained command observations
Addresses [#5584](https://github.com/morluto/rea/issues/5584). GDB command Evidence
repeats ordered MI records in stream groups and a raw mirror; objdump repeats
stdout/stderr in its raw mirror. Explicit `detail: summary` retains the complete
parent and projects existing owner facts once. Default complete Evidence is
unchanged. This adds no tool, MI decoder, live-state cache or command authority.
GDB summaries preserve **every ordered record**, including asynchronous and
unrecognized records, correlated MI result, command, completion/truncation,
invalid-UTF-8 and process-exit facts. They omit the convenience console/target/log
groupings and report their original UTF-16 sizes. Objdump summaries preserve
opaque stdout/stderr, exit/signal, completion and pre/post digest-change facts.
Both carry source parameters/provider/profile, parent identity, limitations and
unknowns. Subjectless observations stay subjectless. Historical observations do
not establish current target state or global coverage. The existing
`inspect_analysis_view` summary operation works after debugger close.
The domain codec now owns the command fact shapes used by contracts and GDB's
capture type; protocol interpretation stays with its provider. MCP requires
retention before command execution. CLI uses the existing shared projection and
nonoverwriting bundle publisher after provider cleanup:
```sh
rea inspect-with-objdump /usr/bin/bash disassemble --symbol execute_command \
--detail summary --evidence_output /tmp/bash-command-evidence.json --json
```
## Real input and correctness
Linux x86-64, Node 24.18.0/npm 11.16.0; GNU GDB 13.1 (Debian 13.1-3),
GNU objdump 2.40, installed Bash ELF 1,265,648 bytes, SHA-256
`55b89ab22bee4792a210f493a53fb066accd5d30b69837c28d98be5ff863efcf`.
The [receipt](command-observation-views.json) pins native binaries, baseline main
`c2858d2aa0916f0dd2d073d848b8c2195d9bf6b3`, source patch and packaged runtime digests.
No target execution, attach, GUI or database mutation occurred.
An [independent ELF reader](command-observation-oracle.py) maps dynamic symbols
to exact file ranges: `execute_command`, 239 bytes at `0x4ba50`, and
`execute_command_internal`, 15,428 bytes at `0x47e00`. The
[verifier](command-observation-verify.py) checks every disassembled byte and its
contiguous address against those ranges. GDB reports 66/3,365 instructions;
objdump byte lines also include continuations, so their count is not an
instruction count. Ordered summary facts match each exported complete parent.
GDB Evidence has a null subject; the external oracle does not manufacture a
target binding. Actual CLI default objdump Evidence matches the baseline exactly,
including identity; symbol/pathless summaries preserve it in bundles. CLI checks
also cover rejected overwrite, missing destination before dispatch, and a real
GDB command with a caller-owned PTY that remains available after debugger exit.
## Whole tasks and resource costs
The [SDK driver](command-observation-queries.mjs) discovers all 210 tools, reads
architecture and symbol-scoped disassembly, closes GDB where applicable, exports
complete Evidence and finalizes owned resources. GDB uses 8 calls; objdump uses 3.
Warm cases repeat disassembly in the same process, adding one call. Fresh-process
cases have **unflushed OS caches**, not disk-cold inputs. Native commands are
rerun; results are not reused as current live state.
The [actual Codex 0.163 app-server driver](command-observation-wire.py) uses a
loopback deterministic SSE fixture, no AI inference or paid service. All 210
function schemas are present in every normal model-facing request. Matched tasks
use full input/output profiles; only disassembly selects summary. The
[counter](command-observation-costs.py) counts compact JSON serialization with
`tiktoken 0.12.0 / cl100k_base` across **every** request, including schemas,
replayed results and the actual client's local compaction request. The fixture
answers compaction with observed identifiers and the retained-body location.
These are serialization-token proxies, not model rendering, acceptance, answer
quality, actual usage or billing. Compaction behavior is qualified only for this
client/configuration; no inference quality is established.
| Completed GDB task | JSON token proxy before → after | Requests | Wall seconds | CPU seconds | Family peak MiB |
| ------------------------------ | ------------------------------: | -------: | ------------: | ------------: | --------------: |
| Small, 30,000 per-tool budget | 605,133 → 580,070 | 9 → 9 | 7.906 → 8.618 | 7.195 → 7.832 | 788.9 → 785.9 |
| Large, 350,000 per-tool budget | 1,269,405 → 1,091,290 | 10 → 9 | 9.551 → 9.127 | 8.832 → 8.370 | 805.2 → 770.8 |
Whole-task context proxies decrease **4.14% / 14.03%**, rather than the much
larger reply-byte reduction. The large baseline makes one compaction request;
the candidate makes none. The small task is slower. These single paired samples
do not establish a general speedup. With the client's default output budget,
the small baseline delivers truncated invalid inner JSON and does **not**
complete the question; summary completes it. That incomplete baseline is
excluded from the percentage comparison.
| SDK task | Reply bytes before → after | Whole wall seconds | CPU seconds | Family peak MiB |
| ------------- | -------------------------: | -----------------: | ------------: | --------------: |
| GDB small | 54,123 → 18,009 | 6.631 → 5.447 | 6.102 → 5.240 | 536.3 → 535.8 |
| GDB large | 2,763,983 → 700,453 | 6.965 → 6.660 | 6.729 → 6.451 | 595.0 → 545.8 |
| Objdump small | 22,467 → 15,990 | 7.599 → 7.699 | 7.110 → 7.264 | 527.8 → 532.5 |
| Objdump large | 964,889 → 487,219 | 7.737 → 7.499 | 7.334 → 6.935 | 537.0 → 536.6 |
Same-process repeat disassembly takes 1,100 → 1,016 ms for GDB and
1,070 → 939 ms for objdump in these observations. Whole warm-task CPU improves
7.613 → 7.162 s for GDB but increases 7.819 → 7.859 s for objdump. Timings include
existing ownership checks; no check was removed or cached.
Retaining a summary also retains an additional derived record: large GDB bundles
grow 1,310,073 → 1,640,022 bytes; objdump 474,288 → 712,656. This trades retained
storage for smaller delivery; long-session heap growth is unmeasured. Retained-data lifetime follows the existing investigation ledger and cleanup
owners. Output still scales
with the requested complete disassembly; summary is not a fixed-size preview.
Each experiment runs in one owned systemd scope, one CPU, no swap, 4 GiB hard /
3 GiB early memory thresholds and host-memory admission/stop thresholds. Recorded
payload exit and cleanup are verified, with zero OOM/resource events. Final
whole-scope counters disappear after teardown and remain explicitly unverified.
The receipt includes CPU time, wall time, peak family memory, warm cases and
bundle growth. Build/check scopes use separate bounded budgets. Host-memory
check stops and initial fixture failures are preserved locally and are not passes.
## Coverage and reproduction
320 focused tests cover the new SDK schemas/views plus existing GDB queue,
malformed output, cancellation, process exit, target identity and cleanup cases,
objdump symbol selection and command admission. Real provider evidence is from
the CLI/SDK/client runs above, not simulated-provider tests.
Run the oracle for the selected Bash image, then use the SDK driver with
`--runtime`, `--input`, `--oracle`, `--output`, `--provider gdb|objdump`,
`--symbol`, `--detail complete|summary` and optional `--warm`. The wire driver
also takes the fixed Codex/Node paths, `--mode`, `--symbol` and `--budget`; it always performs the
export (eight tool calls). Run each through `resource-run.py --observe-payload-exit`, preserving
its sibling `.log`; the verifier and counter accept `--root` for those results.
Default Rizin execution returned ENOENT with no configured alternative. Cutter
reported no live bridge; no configured selector or default bridge directory was
present. Real Rizin/Cutter performance and GUI workflows remain unverified.
Their command semantics were not changed. Existing objdump native symbol
selection was reused. No claim covers arbitrary GDB target execution or another
host client's output budget.
Review follow-up adds nine CLI publication regressions (16 CLI tests including the
existing cleanup cases), three deterministic delivery-oracle tests, and a focused
strict typecheck of the new CLI boundary and its transitive source. Real client
small/default and large/generous summary runs pass the strengthened oracle and
export complete parents; only the explicit complete/default baseline may finish
with an incomplete answer. The unused export switch was removed: export is
mandatory in this question. Product source and the original paired measurements
are unchanged; the receipt records these additional validation runs separately.
@@ -0,0 +1,301 @@
"""Actual Codex wire capture with a deterministic local SSE backend; no AI inference."""
import argparse
import http.server
import threading
import asyncio
import hashlib
import json
import os
from pathlib import Path
import re
import time
import traceback
import subprocess
from command_observation_delivery import verify_command_delivery
async def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--codex", type=Path, required=True)
parser.add_argument("--node", type=Path, required=True)
parser.add_argument("--input-profile", choices=["full", "compact"], default="full")
parser.add_argument("--output-profile", choices=["full", "off"], default="full")
parser.add_argument("--input", type=Path, required=True)
parser.add_argument("--oracle", type=Path, required=True)
parser.add_argument("--expected-tools", type=int, default=210)
parser.add_argument("--mode", choices=["complete", "summary"], required=True)
parser.add_argument("--question", choices=["disassembly"], default="disassembly")
parser.add_argument("--symbol", choices=["execute_command", "execute_command_internal"], required=True)
parser.add_argument("--budget", type=int, default=30000)
args = parser.parse_args()
oracle=json.loads(args.oracle.read_text())
target=str(args.input)
assert args.input.is_absolute() and args.oracle.is_absolute() and args.output.is_absolute()
assert hashlib.sha256(args.input.read_bytes()).hexdigest()==oracle["artifact_sha256"]
def delivered(pieces):
assert isinstance(pieces,str) and pieces.startswith("Wall time: "), pieces[:300]
_,sep,payload=pieces.partition("Output:\n");assert sep
try: value=json.loads(payload)
except json.JSONDecodeError: return {"_invalid_json_payload":payload}
return value["result"] if "result" in value and "normalized_result" not in value else value
def outputs_of(body):
return {item["call_id"]:delivered(item["output"]) for item in body.get("input",[]) if item.get("type")=="function_call_output"}
steps=8
args.output.mkdir(mode=0o700)
audit_home = args.output / "account"
codex_state = audit_home / ".codex"
mcp_home = audit_home / "mcp"
workspace = audit_home / "workspace"
for directory in [codex_state, mcp_home, workspace]:
directory.mkdir(parents=True)
captured = []
known_outputs = {}
state = {"normal_requests":0,"compactions":0}
class Capture(http.server.BaseHTTPRequestHandler):
def log_message(self, *_):
pass
def do_POST(self):
size = int(self.headers["Content-Length"])
assert 0 < size <= 16 * 1024 * 1024
assert self.headers.get("Content-Encoding", "identity") == "identity"
body = json.loads(self.rfile.read(size))
captured.append({"path": self.path, "body": body})
(args.output / "model-requests.json").write_text(json.dumps(captured) + "\n")
http_index = len(captured)
known_outputs.update(outputs_of(body))
names = [t.get("name", "") for ns in body.get("tools", []) if ns.get("type") == "namespace" and ns.get("name") == "mcp__rea_audit" for t in ns.get("tools", [])]
if not names:
last=body.get('input',[])[-1]
assert 'CONTEXT CHECKPOINT COMPACTION' in json.dumps(last)
state['compactions']+=1;assert state['compactions']<=1
evidence=known_outputs['call_local_6'];assert 'normalized_result' in evidence
captured_result=evidence['normalized_result'];command_report=captured_result.get('summary',captured_result)
checkpoint={'fixture':'deterministic protocol checkpoint; no AI inference or model-quality claim','task':'Full native Bash function disassembly has been delivered to this compaction request. Retain complete Evidence and finish owned cleanup.','path':target,'artifact_sha256_external_oracle':oracle['artifact_sha256'],'symbol':args.symbol,'session_id':known_outputs['call_local_1']['session_id'],'parent_evidence_id':captured_result.get('parent_evidence_id',evidence['evidence_id']),'subject':evidence['subject'],'completion_status':command_report['completion_status'],'output_truncated':command_report['output_truncated'],'native_console_utf16_units':command_report.get('available_counts',{}).get('console_utf16_units',len(command_report.get('console','').encode('utf-16-le'))//2),'full_body_location':'same-session retained Evidence; export next to '+str(args.output/'evidence.json'),'remaining':['close_gdb_session','export_evidence_bundle']}
text=json.dumps(checkpoint)
item={"type":"message","id":f"msg_checkpoint_{state['compactions']}","role":"assistant","status":"completed","content":[{"type":"output_text","text":text,"annotations":[]}]}
captured[-1]['kind']='deterministic-compaction'
else:
state['normal_requests']+=1;index=state['normal_requests'];assert index<=steps+1
captured[-1]['kind']='tool-planning-or-final'
if index<=steps:
if index==1:
operation="start_gdb_session";arguments={}
else:
session=known_outputs["call_local_1"]["session_id"]
if index<=6:
command=['set auto-load off','set debuginfod enabled off',f'file {target}','show architecture',f'disassemble /r {args.symbol}'][index-2]
operation="gdb_console";arguments={"session_id":session,"command":command}
if index==6 and args.mode=="summary":arguments["detail"]="summary"
elif index==7:
operation="close_gdb_session";arguments={"session_id":session}
else:
operation="export_evidence_bundle";arguments={"path":str(args.output/'evidence.json')}
selected=[name for name in names if name.endswith(operation)];assert len(selected)==1
item={"type":"function_call","id":f"fc_local_{index}","call_id":f"call_local_{index}","name":selected[0],"namespace":"mcp__rea_audit","arguments":json.dumps(arguments),"status":"completed"}
else:
item={"type":"message","id":"msg_local_done","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Scripted local GDB disassembly task complete.","annotations":[]}]}
response_id=f"resp_local_{http_index}"
base = {"id": response_id, "object": "response", "model": body.get("model"), "status": "completed", "output": [item]}
events = [
{"type": "response.created", "response": {**base, "status": "in_progress", "output": []}},
{"type": "response.output_item.added", "output_index": 0, "item": {**item, "status": "in_progress"}},
{"type": "response.output_item.done", "output_index": 0, "item": item},
{"type": "response.completed", "response": base},
]
wire = "".join("event: " + e["type"] + "\ndata: " + json.dumps(e) + "\n\n" for e in events).encode()
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.send_header("Content-Length", str(len(wire)))
self.end_headers()
self.wfile.write(wire)
endpoint = http.server.HTTPServer(("127.0.0.1", 0), Capture)
endpoint_thread = threading.Thread(target=endpoint.serve_forever, daemon=True)
endpoint_thread.start()
task_path = f"{args.node.parent}:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
mcp_env = {"HOME": str(mcp_home), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1",
"REA_LOG_LEVEL": "silent", "REA_MCP_INPUT_SCHEMA_PROFILE": args.input_profile,
"REA_MCP_OUTPUT_SCHEMA_PROFILE": args.output_profile}
config = 'model_provider = "rea_capture"\nmodel = "rea-wire-probe"\n[analytics]\nenabled = false\n[mcp_servers.rea_audit]\n'
config += f'command = {json.dumps(str(args.node))}\n'
config += f'args = {json.dumps([str(args.runtime / "scripts/rea.mjs"), "mcp"])}\n'
config += 'startup_timeout_sec = 60\ntool_timeout_sec = 60\n'
config += '[mcp_servers.rea_audit.env]\n'
config += ''.join(f'{key} = {json.dumps(value)}\n' for key, value in mcp_env.items())
for name in ['start_gdb_session','gdb_console','close_gdb_session','export_evidence_bundle']:
config += f'[mcp_servers.rea_audit.tools.{name}]\napproval_mode = "approve"\n'
if args.budget: config += f'output_token_limit = {args.budget}\n'
config += f'\n[model_providers.rea_capture]\nname = "Local scripted capture"\nbase_url = "http://127.0.0.1:{endpoint.server_port}/v1"\nwire_api = "responses"\nrequires_openai_auth = false\nsupports_websockets = false\nrequest_max_retries = 0\nstream_max_retries = 0\n'
(codex_state / "config.toml").write_text(config)
(args.output / "declared-config.toml").write_text(config)
environment = {"HOME": str(audit_home), "CODEX_HOME": str(codex_state), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1"}
report = {"question": "Return architecture and complete byte-annotated disassembly for one selected native Bash function and export complete Evidence. No target execution.",
"client": "Codex app-server 0.163.0", "model_turns_started": 1, "backend": "local deterministic SSE fixture; no AI model or paid service",
"actual_model_usage": None, "input_profile": args.input_profile,
"output_profile": args.output_profile, "calls": [], "observations": [],
"unexpected_server_requests": [], "cleanup_failures": []}
client_version = subprocess.check_output([str(args.codex), "--version"], env=environment, cwd=workspace, text=True, timeout=10).strip()
assert client_version == "codex-cli 0.163.0", client_version
transcript = []
pending = {}
completed_turn = asyncio.Event()
next_id = 0
stderr = (args.output / "server.log").open("wb")
process = await asyncio.create_subprocess_exec(str(args.codex), "app-server", "--stdio",
stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=stderr,
cwd=workspace, env=environment, limit=16 * 1024 * 1024)
async def send(message):
encoded = (json.dumps(message, separators=(",", ":")) + "\n").encode()
transcript.append({"direction": "client", "bytes": len(encoded), "message": message})
process.stdin.write(encoded)
await process.stdin.drain()
async def read_messages():
try:
while line := await process.stdout.readline():
value = json.loads(line)
transcript.append({"direction": "server", "bytes": len(line), "message": value})
if value.get("method") == "turn/completed":
report["turn"] = value["params"]["turn"]
completed_turn.set()
if "id" in value and "method" not in value:
future = pending.pop(value["id"], None)
if future and not future.done():
future.set_result(value)
elif "id" in value:
report["unexpected_server_requests"].append(value["method"])
await send({"id": value["id"], "error": {"code": -32601, "message": "Unexpected request in scripted metadata probe"}})
raise RuntimeError("app-server stdout closed")
except Exception as error:
for future in pending.values():
if not future.done():
future.set_exception(error)
reader = asyncio.create_task(read_messages())
async def request(method, params):
nonlocal next_id
assert method in ["initialize", "thread/start", "mcpServerStatus/list", "mcpServer/tool/call", "turn/start"]
next_id += 1
message = {"id": next_id, "method": method, "params": params}
future = asyncio.get_running_loop().create_future()
pending[next_id] = future
start = time.monotonic()
await send(message)
reply = await asyncio.wait_for(future, 90)
report["calls"].append({"method": method, "elapsed_ms": (time.monotonic() - start) * 1000,
"error": reply.get("error")})
if "error" in reply:
raise RuntimeError(json.dumps(reply["error"])[:4000])
return reply["result"]
try:
await request("initialize", {"clientInfo": {"name": "rea_host_probe", "title": "REA host compatibility probe", "version": "1"},
"capabilities": {"experimentalApi": True, "explicitGatewayOauth": True}})
await send({"method": "initialized"})
created = await request("thread/start", {"cwd": str(workspace), "ephemeral": True,
"approvalPolicy": "never", "sandbox": "read-only"})
thread_id = created["thread"]["id"]
assert not created["thread"].get("turns", [])
inventory = await request("mcpServerStatus/list", {"threadId": thread_id,
"serverName": "rea_audit", "detail": "toolsAndAuthOnly", "limit": 10})
assert inventory.get("nextCursor") is None
assert len(inventory["data"]) == 1
server = inventory["data"][0]
assert server["name"] == "rea_audit" and server.get("toolsError") is None
tools = list(server["tools"].values())
names = [tool["name"] for tool in tools]
assert len(names) == len(set(names)) == args.expected_tools
assert "gdb_console" in names
output_schemas = sum(tool.get("outputSchema") is not None for tool in tools)
assert output_schemas == (args.expected_tools if args.output_profile == "full" else 0)
raw = json.dumps(inventory, separators=(",", ":"), ensure_ascii=False).encode()
(args.output / "catalog.json").write_bytes(raw)
report["catalog"] = {"tools": len(tools), "output_schemas": output_schemas,
"bytes": len(raw), "sha256": hashlib.sha256(raw).hexdigest(),
"server_info": server.get("serverInfo"), "runtime_status": server.get("runtimeStatus")}
await request("turn/start", {"threadId":thread_id,"input":[{"type":"text","text":f"Read architecture and the complete byte-annotated disassembly of {args.symbol} in the selected Bash executable. Retain and export complete Evidence, then close the owned debugger. The local fixture scripts the required tools; no AI inference or target execution is used."}]})
await asyncio.wait_for(completed_turn.wait(), 90)
assert report["turn"]["status"] == "completed", report["turn"]
assert state["normal_requests"]==steps+1
assert len(captured)==steps+1+state["compactions"]
outputs=known_outputs
assert len(outputs)==steps
bundle=json.loads((args.output/'evidence.json').read_text())
evidence=next(e for e in bundle['records'] if e['operation']=='gdb_console' and e['parameters']['command']==f'disassemble /r {args.symbol}')
assert evidence['subject'] is None
assert evidence['normalized_result']['completion_status']=='complete' and not evidence['normalized_result']['output_truncated']
delivered_result=outputs['call_local_6']
delivery_complete=verify_command_delivery(delivered_result,evidence,args.mode,args.budget)
if delivery_complete and args.mode=='summary':
assert next(e for e in bundle['records'] if e['evidence_id']==delivered_result['evidence_id'])==delivered_result
report['task_completed']=delivery_complete
report['observations']={'parent_evidence_id':evidence['evidence_id'],'exported':True,'native_console_utf16_units':len(evidence['normalized_result']['console'].encode('utf-16-le'))//2,'client_delivered_full_evidence':delivery_complete,'selected_symbol':args.symbol,'tool_output_budget':args.budget or 'client default','subject':evidence['subject']}
report["captured_requests"]=len(captured)
report["compaction_requests"]=state["compactions"]
report["compaction_backend"]="Deterministic checkpoint of observed identifiers and retained-body location; no AI inference or quality claim."
report["mode"]=args.mode
report["request_tool_counts"] = [len(c["body"].get("tools", [])) for c in captured]
assert not report["unexpected_server_requests"]
assert hashlib.sha256(args.input.read_bytes()).hexdigest()==oracle["artifact_sha256"]
report["status"] = "passed" if report["task_completed"] else "expected-incomplete-task"
except Exception:
report["status"] = "failed"
report["failure"] = traceback.format_exc()[-6000:]
finally:
process.stdin.close()
try:
await asyncio.wait_for(process.wait(), 15)
except asyncio.TimeoutError:
report["cleanup_failures"].append("app-server did not exit after stdin close; terminated owned process")
process.terminate()
try:
await asyncio.wait_for(process.wait(), 5)
except asyncio.TimeoutError:
process.kill()
await process.wait()
await reader
# app-server can acknowledge exit before its MCP child finishes EOF
# cleanup. Wait on the allocated resource scope; never target a PID.
try:
groups = [line[3:] for line in Path("/proc/self/cgroup").read_text().splitlines()
if line.startswith("0::")]
assert len(groups) == 1
group = Path("/sys/fs/cgroup") / groups[0].lstrip("/")
assert re.fullmatch(r"rea-efficiency-[0-9a-f]{32}\.scope", group.name)
observer = os.getppid()
observer_args = Path(f"/proc/{observer}/cmdline").read_bytes().split(b"\0")
assert any(Path(os.fsdecode(value)).name == "resource_payload.py" for value in observer_args if value)
expected = {os.getpid(), observer}
until = time.monotonic() + 10
while True:
actors = {int(value) for value in (group / "cgroup.procs").read_text().splitlines()}
assert expected <= actors
remaining = sorted(actors - expected)
if not remaining or time.monotonic() >= until:
break
await asyncio.sleep(0.05)
report["owned_scope_drain"] = {"remaining_processes": remaining}
if remaining:
report["cleanup_failures"].append("owned MCP child did not settle after app-server exit")
except (OSError, ValueError, AssertionError) as error:
report["owned_scope_drain"] = {"status": "unavailable", "reason": type(error).__name__}
report["cleanup_failures"].append("allocated resource-scope cleanup could not be verified")
endpoint.shutdown()
endpoint.server_close()
endpoint_thread.join(5)
stderr.close()
report["appserver_exit_code"] = process.returncode
if process.returncode != 0:
report["cleanup_failures"].append(f"app-server exit code: {process.returncode}")
(args.output / "transcript.json").write_text(json.dumps(transcript) + "\n")
(args.output / "report.json").write_text(json.dumps(report, indent=2) + "\n")
print(json.dumps({key: report.get(key) for key in ["status", "failure", "catalog", "observations", "cleanup_failures", "appserver_exit_code"]}))
return 0 if report["status"] in ["passed","expected-incomplete-task"] and not report["cleanup_failures"] else 1
raise SystemExit(asyncio.run(main()))
@@ -0,0 +1,39 @@
"""Acceptance of the captured GDB answer, separate from native execution success."""
def verify_command_delivery(delivered, parent, mode, budget):
assert mode in ('complete', 'summary')
if '_invalid_json_payload' in delivered:
assert mode == 'complete' and budget == 0, (
'Incomplete delivery is allowed only for the '
'complete/default-budget baseline'
)
return False
if mode == 'complete':
assert delivered == parent
return True
selected = delivered['normalized_result']
assert selected['parent_evidence_id'] == parent['evidence_id']
assert selected['artifact'] is None
profile = parent['analysis_profile']
expected_profile = None if profile is None else profile['digest']
assert selected['source_analysis_profile_digest'] == expected_profile
assert selected['source_parameters'] == parent['parameters']
assert selected['source_provider'] == parent['provider']
assert delivered['raw_result'] is None and delivered['subject'] is None
expected = {
key: value for key, value in parent['normalized_result'].items()
if key not in ('console', 'target', 'log')
}
actual = {
key: value for key, value in selected['summary'].items()
if key != 'available_counts'
}
assert actual == expected
counts = {'records': len(expected['records'])}
for name in ('console', 'target', 'log'):
text = parent['normalized_result'][name]
counts[name + '_utf16_units'] = len(text.encode('utf-16-le')) // 2
assert selected['summary']['available_counts'] == counts
return True
@@ -0,0 +1,81 @@
"""Count complete actual-client requests, not only smaller tool-reply bytes."""
import argparse
import hashlib
import json
from pathlib import Path
import tiktoken
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--lab', type=Path, required=True)
parser.add_argument('--output', type=Path, required=True)
args = parser.parse_args()
enc = tiktoken.get_encoding('cl100k_base')
def tokens(value):
return len(enc.encode(json.dumps(value, ensure_ascii=False, separators=(',', ':'), allow_nan=False), disallowed_special=()))
def resources(root):
receipt = json.loads(Path(str(root) + '.log').read_text().splitlines()[-1])
payload = receipt['payload_resource_observation']
assert receipt['abort'] is None and receipt['payload_resource_observation_verified']
assert payload['payload_exit_code'] == 0 and payload['payload_cleanup_verified']
for events in payload['resource_events'].values():
assert not any(events.values())
return {'wall_seconds': receipt['elapsed_seconds'], 'cpu_seconds': payload['cpu_stat']['usage_usec'] / 1e6,
'family_peak_mib': payload['memory_peak_bytes'] / 1048576,
'payload_cleanup_verified': True, 'final_whole_scope_counters_verified': receipt['resource_counters_verified']}
wire, sdk = [], []
for root in sorted(args.lab.glob('wire-v2-*')):
if not root.is_dir():
continue
report = json.loads((root / 'report.json').read_text())
assert report['status'] in ['passed', 'expected-incomplete-task'] and not report['cleanup_failures']
assert report['appserver_exit_code'] == 0 and report['owned_scope_drain']['remaining_processes'] == []
requests = json.loads((root / 'model-requests.json').read_text())
details = []
for captured in requests:
body = captured['body']
namespace = [n for n in body.get('tools', []) if n.get('type') == 'namespace' and n.get('name') == 'mcp__rea_audit']
assert len(namespace) == 1 and len(namespace[0]['tools']) == 211, 'Unexpected compaction/catalog: audit before comparing'
details.append({'whole_json_tokens_proxy': tokens(body), 'rea_namespace_tokens_proxy': tokens(namespace[0]), 'rea_functions': 211})
wire.append({'case': root.name, 'task_completed': report['task_completed'], 'mode': report['mode'],
'budget': report['observations']['tool_output_budget'], 'requests': len(details), 'compactions': report['compaction_requests'],
'whole_json_tokens_proxy': sum(r['whole_json_tokens_proxy'] for r in details), 'request_details': details, **resources(root)})
for root in sorted(args.lab.glob('sdk-*')):
if not root.is_dir():
continue
report = json.loads((root / 'report.json').read_text())
assert report['status'] == 'passed' and not report['cleanup_failures'] and not report['receipt_failures']
transcript = json.loads((root / 'transcript.json').read_text())
catalog = json.loads((root / 'catalog.json').read_text())
assert len(catalog['tools']) == 211 and report['original_target_exited']
sdk.append({'case': root.name, 'detail': report['detail'], 'capture': report['capture'], 'tool_calls': len(report['calls']),
'exported_bundle_bytes': report['exported_bundle_bytes'], 'answer': report['answer'],
'status_calls': [c for c in report['calls'] if c['name'] == 'frida_session_status'],
'one_catalog_plus_all_sdk_calls_tokens_proxy': tokens(catalog) + sum(tokens({'name': t['name'], 'input': t['input'], 'reply': t['reply']}) for t in transcript), **resources(root)})
assert len(sdk) == len(wire) == 4, 'A partial cohort cannot qualify a comparison'
for capture in ['small', 'libc']:
pair = [r for r in sdk if r['capture'] == capture]
assert len(pair) == 2 and {r['detail'] for r in pair} == {'complete', 'summary'}
assert {k: v for k, v in pair[0]['answer'].items() if k != 'pid'} == {k: v for k, v in pair[1]['answer'].items() if k != 'pid'}
reports = [json.loads((args.lab / r['case'] / 'report.json').read_text()) for r in pair]
assert reports[0]['load']['source_sha256'] == reports[1]['load']['source_sha256']
completed = [r for r in wire if r['budget'] == 300000]
assert len(completed) == 2 and all(r['task_completed'] for r in completed)
answers = [json.loads((args.lab / r['case'] / 'report.json').read_text())['observations']['answers'] for r in completed]
comparable_answers = [[{k: v for k, v in a.items() if k != 'parent_evidence_id'} for a in run] for run in answers]
assert comparable_answers[0] == comparable_answers[1]
result = {
'scope': 'Real local Frida binding with controlled sleep target and caller-authored libc export inventory. Actual Codex0.163 app-server uses a deterministic local SSE fixture: no AI inference, model usage, quality or billing claim. Whole compact JSON token estimates include all 211 schemas and every request replay. Incomplete default baseline is excluded from completed-task gains. SDK exchange estimates are not model-input measurements. Fresh processes have unflushed OS caches; both status reads query live session observations. Timings include the eight-second target lifetime and cleanup; single pairs establish no general speedup. Payload-exit cgroup counters are verified; final whole-scope counters may be unavailable.',
'tokenizer': {'package': tiktoken.__version__, 'encoding': enc.name},
'runtime_pins': {v: {k: x for k, x in json.loads((args.lab / (v + '-manifest.json')).read_text()).items() if k != 'files'} for v in ['baseline', 'candidate']},
'native': json.loads((args.lab / 'native-prerequisite-manifest.json').read_text()),
'script_pins': {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in Path(__file__).parent.glob('frida*') if p.suffix in ['.py', '.mjs']},
'wire': wire, 'sdk': sdk,
}
args.output.write_text(json.dumps(result, indent=2) + '\n')
print(json.dumps({'wire': [{k: v for k, v in r.items() if k != 'request_details'} for r in wire], 'sdk': [{k: v for k, v in r.items() if k not in ['status_calls', 'answer']} for r in sdk]}, indent=2))
@@ -0,0 +1,267 @@
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { createWriteStream } from "node:fs";
import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
import { createRequire } from "node:module";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { pathToFileURL } from "node:url";
import { finished } from "node:stream/promises";
import { parseArgs } from "node:util";
import { createRetainedQueryCaller } from "./retained-query-calls.mjs";
import { finalizeRetainedQueryReceipt } from "./retained-query-finalization.mjs";
const { values } = parseArgs({
options: {
runtime: { type: "string" },
output: { type: "string" },
detail: { type: "string", default: "complete" },
capture: { type: "string", default: "libc" },
},
});
const { runtime, output, detail, capture } = values;
assert.ok(runtime && output);
assert.ok(["complete", "summary"].includes(detail));
assert.ok(["libc", "small"].includes(capture));
const require = createRequire(join(runtime, "package.json"));
const { Client } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client"))
);
const { StdioClientTransport } = await import(
pathToFileURL(require.resolve("@modelcontextprotocol/client/stdio"))
);
const { parseLinuxProcessStat } = await import(
pathToFileURL(join(runtime, "dist/process/LinuxProcessStat.js"))
);
await mkdir(output, { mode: 0o700 });
const account = await mkdtemp(join(tmpdir(), "rea-frida-real-"));
const serverLog = createWriteStream(join(output, "server.log"));
const serverLogFinished = finished(serverLog);
serverLogFinished.catch(() => {});
const client = new Client({ name: "rea-frida-audit", version: "1" });
const transport = new StdioClientTransport({
command: process.execPath,
args: [join(runtime, "scripts/rea.mjs"), "mcp"],
env: {
PATH: process.env.PATH,
HOME: account,
NODE_OPTIONS: "--max-old-space-size=2048",
UV_THREADPOOL_SIZE: "1",
REA_LOG_LEVEL: "silent",
},
stderr: "pipe",
});
transport.stderr?.pipe(serverLog, { end: false });
const report = {
question:
"Retain a complete caller-requested libc export inventory, then check only session identity/state and message capture coverage twice, preserving complete Evidence.",
detail,
capture,
calls: [],
status: "pending",
};
const transcript = [];
const caller = createRetainedQueryCaller(client, (receipt, event) => {
report.calls.push(receipt);
transcript.push(event);
});
const call = async (name, input) => {
const r = await caller(name, input, name);
assert.ok(!r.isError, JSON.stringify(r));
return r.structuredContent;
};
const delay = (ms) => new Promise((r) => setTimeout(r, ms));
let session;
let birth;
const stat = async () => {
try {
return parseLinuxProcessStat(
await readFile(`/proc/${session.pid}/stat`, "utf8"),
);
} catch (e) {
if (e.code === "ENOENT") return null;
throw e;
}
};
try {
await client.connect(transport);
const catalog = await client.listTools();
await writeFile(join(output, "catalog.json"), JSON.stringify(catalog));
report.catalog_tools = catalog.tools.length;
const defs = new Map(catalog.tools.map((t) => [t.name, t]));
const orig = client.callTool.bind(client);
client.callTool = (input, options) =>
orig(input, {
...options,
timeout: 30000,
toolDefinition: defs.get(input.name),
});
report.devices = await call("list_frida_devices", {});
session = await call("start_frida_session", {
mode: "spawn",
device_id: "local",
program: "/usr/bin/sleep",
argv: ["8"],
});
report.session = session;
birth = await stat();
assert.ok(birth);
report.target_birth = birth;
report.same_owned_cgroup =
(await readFile(`/proc/${session.pid}/cgroup`, "utf8")) ===
(await readFile("/proc/self/cgroup", "utf8"));
assert.ok(report.same_owned_cgroup);
const source = `send({kind:'ready',pid:Process.id});
setTimeout(() => {
if (${JSON.stringify(capture)} === 'small') { send({kind:'complete',total:0}); return; }
const module = Process.enumerateModules().find(m => /(^|\\/)libc\\.so\\.6$/.test(m.path));
if (!module) { send({kind:'missing-libc'}); return; }
const exports = module.enumerateExports().map(e => ({type:e.type,name:e.name,address:e.address.toString()}));
send({kind:'module',pid:Process.id,path:module.path,base:module.base.toString(),size:module.size,total:exports.length});
for (let offset = 0; offset < exports.length; offset += 64) send({kind:'exports',offset,items:exports.slice(offset,offset+64)});
send({kind:'complete',total:exports.length});
}, 300);`;
await writeFile(join(output, "selected-script.js"), source);
report.source_sha256 = createHash("sha256").update(source).digest("hex");
const loaded = await call("load_frida_script", {
session_id: session.session_id,
source_kind: "inline",
source,
});
assert.equal(loaded.normalized_result.source_sha256, report.source_sha256);
report.load = loaded.normalized_result;
await call("resume_frida_session", { session_id: session.session_id });
await delay(750);
const snapshots = [];
for (let n = 0; n < 2; n++)
snapshots.push(
await call("frida_session_status", {
session_id: session.session_id,
...(detail === "summary" ? { detail } : {}),
}),
);
await call("export_evidence_bundle", { path: join(output, "evidence.json") });
const bundle = JSON.parse(
await readFile(join(output, "evidence.json"), "utf8"),
);
for (const snapshot of snapshots) {
assert.equal(snapshot.raw_result, null);
const parent =
detail === "summary"
? bundle.records.find(
(e) =>
e.evidence_id === snapshot.normalized_result.parent_evidence_id,
)
: snapshot;
assert.ok(parent);
assert.equal(parent.authority, "external-service");
assert.equal(parent.environment, null);
assert.equal(parent.raw_result, null);
assert.equal(parent.operation, "frida_session_status");
assert.deepEqual(parent.parameters, { session_id: session.session_id });
const r = parent.normalized_result;
assert.equal(r.pid, session.pid);
assert.equal(r.state, "running");
assert.equal(r.messages_truncated, false);
const payloads = r.messages
.filter((m) => m.type === "send")
.map((m) => m.payload);
const ready = payloads.find((p) => p?.kind === "ready");
const complete = payloads.find((p) => p?.kind === "complete");
assert.equal(ready?.pid, session.pid);
assert.ok(complete);
let count = 0;
let header;
if (capture === "libc") {
header = payloads.find((p) => p?.kind === "module");
assert.ok(header);
assert.equal(header.pid, session.pid);
for (const chunk of payloads.filter((p) => p?.kind === "exports")) {
assert.equal(chunk.offset, count);
count += chunk.items.length;
}
assert.equal(count, header.total);
assert.ok(count > 1000);
const maps = await readFile(`/proc/${session.pid}/maps`, "utf8");
assert.ok(maps.includes(header.path));
}
assert.equal(count, complete.total);
if (detail === "summary") {
const view = snapshot.normalized_result;
assert.equal(view.parent_operation, parent.operation);
assert.equal(view.parent_digest, parent.evidence_id.slice(3));
assert.equal(view.artifact, null);
assert.deepEqual(view.source_parameters, parent.parameters);
assert.deepEqual(view.source_provider, parent.provider);
const { messages, ...metadata } = r;
assert.deepEqual(view.summary, {
...metadata,
messages_retained: messages.length,
});
assert.ok(view.unknowns.length > 0);
assert.ok(view.limitations.some((m) => m.includes("operating system")));
assert.deepEqual(
bundle.records.find((e) => e.evidence_id === snapshot.evidence_id),
snapshot,
);
}
report.answer = {
pid: r.pid,
state: r.state,
scripts: r.scripts.length,
messages: r.messages.length,
messages_truncated: r.messages_truncated,
exports: count,
module_path: header?.path ?? null,
};
}
report.native_oracle =
"Caller-authored script digest and contiguous chunk coverage; PID/birthtime/cgroup and libc path independently checked in procfs. Export addresses and native export completeness not independently qualified.";
report.exported_bundle_bytes = Buffer.byteLength(
await readFile(join(output, "evidence.json")),
);
await call("close_frida_session", { session_id: session.session_id });
session.closed = true;
const deadline = performance.now() + 12000;
while (true) {
const current = await stat();
if (
current === null ||
current.startTime !== birth.startTime ||
["zombie", "dead"].includes(current.lifecycle)
) {
report.original_target_exited = true;
break;
}
assert.ok(
performance.now() < deadline,
"Original target exit not observed",
);
await delay(50);
}
report.status = "passed";
} catch (error) {
report.status = "failed";
report.failure = String(error?.stack ?? error);
process.exitCode = 1;
} finally {
if (session && !session.closed)
try {
await call("close_frida_session", { session_id: session.session_id });
} catch (error) {
report.session_cleanup_failure = String(error);
process.exitCode = 1;
}
if (
!(await finalizeRetainedQueryReceipt({
report,
client,
transport,
account,
serverLog,
serverLogFinished,
output,
transcript,
}))
)
process.exitCode = 1;
}
@@ -0,0 +1,468 @@
{
"scope": "Real local Frida binding with controlled sleep target and caller-authored libc export inventory. Actual Codex0.163 app-server uses a deterministic local SSE fixture: no AI inference, model usage, quality or billing claim. Whole compact JSON token estimates include all 211 schemas and every request replay. Incomplete default baseline is excluded from completed-task gains. SDK exchange estimates are not model-input measurements. Fresh processes have unflushed OS caches; both status reads query live session observations. Timings include the eight-second target lifetime and cleanup; single pairs establish no general speedup. Payload-exit cgroup counters are verified; final whole-scope counters may be unavailable.",
"tokenizer": {
"package": "0.12.0",
"encoding": "cl100k_base"
},
"runtime_pins": {
"baseline": {
"source_base": "17f581ff75d2b6cf643a997275e3e296f6c95760",
"patch_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"archive_sha256": "cd16ca3e36633459bffc10bb90d1fe4ff5887fdd403729765b9a812969013807"
},
"candidate": {
"source_base": "32a4690059907f09fb29e284ad492d94c1722278",
"patch_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"archive_sha256": "4ac402f1de93f76f71b7ccdb8b8e36d8c8d0a87b1184812f06393bfc6fadccba"
}
},
"native": {
"host": "Linux-6.1.0-53-cloud-amd64-x86_64-with-glibc2.36",
"toolchain": {
"node": "24.18.0",
"npm": "11.16.0"
},
"frida_binding_package_version": "17.23.2",
"files": [
{
"path": "/usr/bin/sleep",
"bytes": 43888,
"sha256": "4add4bb89d8ca0e3b1bd861130ddd7ae0fd9617a8055de0a38c8d2ca1ac95723"
},
{
"path": "/usr/lib/x86_64-linux-gnu/libc.so.6",
"bytes": 1926232,
"sha256": "6b4a45352fd0c540a9c7c718f35ce8c8e46a4e482f9d3885a910c32d1a0e1421"
},
{
"path": "/home/yann/yann/rea-perf-next-audit-20261020/node_modules/frida/package.json",
"bytes": 2252,
"sha256": "7efb59d2050b7b6304e1617f463c4eb7b2f7cb7e3e80e677976825a76ee31e8d"
},
{
"path": "/home/yann/yann/rea-perf-next-audit-20261020/node_modules/frida/build/frida_binding.node",
"bytes": 112775824,
"sha256": "5b2db5ae57c058ad0765e627d9d9c9c98e4e90c09746c6f727b7a80769eed7a5"
}
],
"qualifications": {
"Inspector": "real REA MCP target, concurrent native metadata only oracle, 776 rows matched hash/length; not global completeness or execution proof",
"Frida": "local native binding and controlled sleep spawn; caller script digest and procfs PID/birthtime/cgroup/libc path verified; native export address/global completeness oracle unavailable",
"ADB": "default executable unavailable in prior public call; no real device qualified"
}
},
"script_pins": {
"frida_status_delivery.py": "ed19bcd003165c7b522c8cdd4799bcf987178e4b79fc8d5c2442012f3bf60b8d",
"frida-status-costs.py": "2f3196bcc3198a265f7f4842effc8dce31669723b42f07367f065afee0073906",
"frida-status-queries.mjs": "c5919ae8bba046909858e92a945b4472645bbacbfcf88dd75c89ba959a986980",
"frida-status-wire.py": "b4d20e64bf77b7815bd43b52569b3e4756983b254ab44c50ddae874d1964ef69"
},
"wire": [
{
"case": "wire-v2-baseline-0",
"task_completed": false,
"mode": "complete",
"budget": "client default",
"requests": 8,
"compactions": 0,
"whole_json_tokens_proxy": 540065,
"request_details": [
{
"whole_json_tokens_proxy": 62729,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 62917,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63599,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63739,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 68261,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 72785,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 72926,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 73109,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
}
],
"wall_seconds": 15.993,
"cpu_seconds": 8.211179,
"family_peak_mib": 830.6484375,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "wire-v2-baseline-300000",
"task_completed": true,
"mode": "complete",
"budget": 300000,
"requests": 8,
"compactions": 0,
"whole_json_tokens_proxy": 1049997,
"request_details": [
{
"whole_json_tokens_proxy": 62715,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 62903,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63577,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63714,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 141108,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 218504,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 218646,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 218830,
"rea_namespace_tokens_proxy": 53518,
"rea_functions": 211
}
],
"wall_seconds": 15.418,
"cpu_seconds": 8.026944,
"family_peak_mib": 824.28125,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "wire-v2-candidate-0",
"task_completed": true,
"mode": "summary",
"budget": "client default",
"requests": 8,
"compactions": 0,
"whole_json_tokens_proxy": 515539,
"request_details": [
{
"whole_json_tokens_proxy": 62791,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 62978,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63661,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63803,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 64749,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 65697,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 65837,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 66023,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
}
],
"wall_seconds": 15.889,
"cpu_seconds": 7.939387,
"family_peak_mib": 823.69921875,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "wire-v2-candidate-300000",
"task_completed": true,
"mode": "summary",
"budget": 300000,
"requests": 8,
"compactions": 0,
"whole_json_tokens_proxy": 515713,
"request_details": [
{
"whole_json_tokens_proxy": 62804,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 62990,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63677,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 63815,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 64772,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 65731,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 65870,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
},
{
"whole_json_tokens_proxy": 66054,
"rea_namespace_tokens_proxy": 53581,
"rea_functions": 211
}
],
"wall_seconds": 15.439,
"cpu_seconds": 8.371664,
"family_peak_mib": 813.6328125,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
}
],
"sdk": [
{
"case": "sdk-baseline-libc",
"detail": "complete",
"capture": "libc",
"tool_calls": 8,
"exported_bundle_bytes": 214214,
"answer": {
"pid": 517427,
"state": "running",
"scripts": 1,
"messages": 50,
"messages_truncated": false,
"exports": 2963,
"module_path": "/usr/lib/x86_64-linux-gnu/libc.so.6"
},
"status_calls": [
{
"name": "frida_session_status",
"input": {
"session_id": "ef8982d3-dbf2-43cb-8256-3a4f4bc83cc3"
},
"phase": "frida_session_status",
"elapsed_ms": 197.58100300000024,
"reply_bytes": 462831,
"is_error": false
},
{
"name": "frida_session_status",
"input": {
"session_id": "ef8982d3-dbf2-43cb-8256-3a4f4bc83cc3"
},
"phase": "frida_session_status",
"elapsed_ms": 108.50572199999988,
"reply_bytes": 462831,
"is_error": false
}
],
"one_catalog_plus_all_sdk_calls_tokens_proxy": 1186065,
"wall_seconds": 13.759,
"cpu_seconds": 6.097489,
"family_peak_mib": 578.5625,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "sdk-baseline-small",
"detail": "complete",
"capture": "small",
"tool_calls": 8,
"exported_bundle_bytes": 2335,
"answer": {
"pid": 516970,
"state": "running",
"scripts": 1,
"messages": 2,
"messages_truncated": false,
"exports": 0,
"module_path": null
},
"status_calls": [
{
"name": "frida_session_status",
"input": {
"session_id": "3f1f778f-ca18-48ba-a7db-b58afbf4710d"
},
"phase": "frida_session_status",
"elapsed_ms": 79.25232200000028,
"reply_bytes": 2643,
"is_error": false
},
{
"name": "frida_session_status",
"input": {
"session_id": "3f1f778f-ca18-48ba-a7db-b58afbf4710d"
},
"phase": "frida_session_status",
"elapsed_ms": 6.851230000000214,
"reply_bytes": 2643,
"is_error": false
}
],
"one_catalog_plus_all_sdk_calls_tokens_proxy": 882323,
"wall_seconds": 13.039,
"cpu_seconds": 4.953337,
"family_peak_mib": 577.140625,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "sdk-candidate-libc",
"detail": "summary",
"capture": "libc",
"tool_calls": 8,
"exported_bundle_bytes": 216931,
"answer": {
"pid": 517901,
"state": "running",
"scripts": 1,
"messages": 50,
"messages_truncated": false,
"exports": 2963,
"module_path": "/usr/lib/x86_64-linux-gnu/libc.so.6"
},
"status_calls": [
{
"name": "frida_session_status",
"input": {
"session_id": "9af7f188-05f8-4559-b570-98556196090e",
"detail": "summary"
},
"phase": "frida_session_status",
"elapsed_ms": 159.16798299999937,
"reply_bytes": 5597,
"is_error": false
},
{
"name": "frida_session_status",
"input": {
"session_id": "9af7f188-05f8-4559-b570-98556196090e",
"detail": "summary"
},
"phase": "frida_session_status",
"elapsed_ms": 61.96135099999992,
"reply_bytes": 5597,
"is_error": false
}
],
"one_catalog_plus_all_sdk_calls_tokens_proxy": 885626,
"wall_seconds": 12.843,
"cpu_seconds": 5.012703,
"family_peak_mib": 579.0,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
},
{
"case": "sdk-candidate-small",
"detail": "summary",
"capture": "small",
"tool_calls": 8,
"exported_bundle_bytes": 5051,
"answer": {
"pid": 517162,
"state": "running",
"scripts": 1,
"messages": 2,
"messages_truncated": false,
"exports": 0,
"module_path": null
},
"status_calls": [
{
"name": "frida_session_status",
"input": {
"session_id": "1bcda0f5-e3cd-45ee-b82c-5886d8312bc6",
"detail": "summary"
},
"phase": "frida_session_status",
"elapsed_ms": 107.78244100000029,
"reply_bytes": 5595,
"is_error": false
},
{
"name": "frida_session_status",
"input": {
"session_id": "1bcda0f5-e3cd-45ee-b82c-5886d8312bc6",
"detail": "summary"
},
"phase": "frida_session_status",
"elapsed_ms": 5.996030000000246,
"reply_bytes": 5595,
"is_error": false
}
],
"one_catalog_plus_all_sdk_calls_tokens_proxy": 885676,
"wall_seconds": 13.533,
"cpu_seconds": 5.495048,
"family_peak_mib": 576.48046875,
"payload_cleanup_verified": true,
"final_whole_scope_counters_verified": false
}
]
}
@@ -0,0 +1,163 @@
# Frida status delivery and runtime-observation audit
Issue: [#5640](https://github.com/morluto/rea/issues/5640). Paired runtime base:
`17f581ff75d2b6cf643a997275e3e296f6c95760`. Final candidate source: `32a4690059907f09fb29e284ad492d94c1722278` (clean tree).
Exact package, patch, native input,
script and resource pins are in [the measurement record](frida-status-views.json).
The analyst requests a complete libc export inventory as retained evidence,
then checks the session's identity/state, script identity, queued message count
and truncation twice. The caller-authored script enumerates actual native module
exports in a controlled `/usr/bin/sleep` spawn, in chunks of 64. Both workflows
retain all captured messages and export the complete Evidence bundle.
Start, status and one-shot contracts share one canonical session envelope.
Message JSON is validated at the Evidence owner; the summary validates the
envelope without reinterpreting opaque payloads.
The owning application service now accepts optional `detail=summary` for
`frida_session_status`. It reads the provider on every call, retains the original
complete Evidence, then uses the existing shared analysis-view projector.
The summary omits arbitrary message payloads and preserves all other status
fields, the retained message count, truncation, exact parent identity/digest,
provider and source parameters. Default and explicit complete delivery keep the
original Evidence identity and payload. Historical CLI/file projections use the
same projector. Source script decoding/representation Evidence and session
resource ownership are unchanged.
## Completed tasks
The native Frida Node binding package 17.23.2 captured 2,963 libc exports in 50 queued
messages. Independent checks establish the test-owned process birthtime, cgroup,
PID and libc mapping; the selected script digest and contiguous chunks are
checked against the complete retained parent. This does not independently
validate every native export address or establish global event coverage.
| Public SDK task | Complete | Summary |
| ------------------------------------------ | -------: | ------: |
| Large-history status reply, bytes per call | 462,831 | 5,597 |
| Small-history status reply, bytes per call | 2,643 | 5,595 |
| Large complete bundle, bytes | 214,214 | 216,931 |
| Small complete bundle, bytes | 2,335 | 5,051 |
| Large task elapsed, seconds | 13.759 | 12.843 |
| Large task CPU, seconds | 6.097 | 5.013 |
| Large family peak, MiB | 578.6 | 579.0 |
| Small task elapsed, seconds | 13.039 | 13.533 |
| Small task CPU, seconds | 4.953 | 5.495 |
| Small family peak, MiB | 577.1 | 576.5 |
Each SDK task discovers all 211 tools and executes eight public calls, including
two status reads and complete export. Small history contains two readiness/
completion messages. Fresh processes retain unflushed OS caches. The second
status read is a repeated live observation, not a cache hit. Every task includes
the controlled target's eight-second lifetime and verified natural exit.
Single pairs and this fixed wait establish no general latency/CPU advantage.
Large status calls took 159.2/62.0 ms versus 197.6/108.5 ms. The first small
summary status was slower (107.8 versus 79.3 ms).
## Actual client context
Codex app-server 0.163.0 performs the native Frida workflow through the actual
REA MCP package. A deterministic local Responses/SSE fixture selects the calls;
there is no AI inference, paid model, model-quality or billing claim. The capture
contains all 211 tool input schemas on each request and all history replays. `tiktoken`
0.12.0 `cl100k_base` counts compact JSON as a token estimate, not actual model
usage. Both completed paths use the same 300,000 tool-output budget.
| Completed client task | Complete | Summary |
| -------------------------------------- | --------: | ------: |
| Requests / compactions | 8 / 0 | 8 / 0 |
| Whole request JSON token estimate, sum | 1,049,997 | 515,713 |
| Elapsed seconds | 15.418 | 15.439 |
| CPU seconds | 8.027 | 8.372 |
| Family peak MiB | 824.3 | 813.6 |
The completed task uses **50.9% fewer estimated request tokens**, including
schemas and continuations. Client elapsed and CPU time increased in this pair. At the client's
default output budget, complete status JSON is truncated and the task is
incomplete; summary status is intact and completes. That failed delivery is
excluded from completed-task gains. Three acceptance tests exercise invalid
JSON, incorrect counts/state/source linkage, missing chunks and wrong sessions.
The complete catalog plus all SDK exchanges also shrinks for the large case
(1,186,065 to 885,626 estimated tokens), but grows for the small case (882,323 to
885,676). These SDK exchange estimates include advertised output schemas and
are **not** actual model-input measurements. Summary delivery is optional
because its provenance wrapper costs more than a small complete observation.
## Resource accounting and limits
Runtime cases use one CPU, a 4 GiB cgroup ceiling, a 3 GiB early stop, no swap,
192 tasks, and host-memory admission/early-stop guards. Payload exit, descendant
cleanup and zero resource-limit/OOM events are verified while the scope observer
remains alive. Final whole-scope counters disappear after teardown and remain
unknown. Runtime family peaks stay below 1 GiB. Build/typecheck use separately
guarded larger limits; they are not runtime performance measurements.
Full queued messages still undergo validation, Evidence creation and retention;
this change primarily reduces delivery and repeated context. The extra derived
record increases retained bundle size. Long-lived heap retention, many-script
sessions, remote devices, binary messages and saturated capture budgets are not
performance-qualified by these native cases. Boundary tests preserve malformed
input, truncation, cancellation and cleanup semantics; simulated providers do
not qualify real remote behavior.
## Other providers in this audit
Inspector was measured against a test-owned real REA MCP process while it
handled a verified GitHub CLI Go build-information request. A concurrent native
CDP metadata subscription matched all 776 retained scripts by hash and reported
length: 565 file locations and 211 built-ins. The reply was 608,719 bytes;
capture retained 996/996 events with no reported drop/truncation. The qualification
run took 20.956 seconds, 18.076 CPU seconds and 987.4 MiB peak family memory,
including the independent oracle. This is not a paired performance cohort,
global completeness proof, source-byte validation or evidence of script
execution. Only `Runtime.enable` and `Debugger.enable` were sent. A prior
post-capture oracle missed 25 historical rows and was rejected; it does not
establish an Inspector regression. Existing raw-null delivery, grouped location
validation and eight-worker projection were already present and were preserved.
ADB's configured default executable is unavailable on this host, and no real
device is qualified. This audit makes no ADB device-performance claim. Inspector
selection/caching and Frida queue/filter redesign remain unimplemented without
an equivalent completed-task measurement that justifies them.
The existing bundled Frida verifier also returned functional success, but its
1,500 ms fixture outlived the verifier's 500 ms observation. Its payload-exit
cleanup receipt was therefore unverified and is excluded from qualified
resource/performance results. That unchanged verifier has no process birthtime
stamp to safely wait on an arbitrary later PID. The paired harness instead owns
the paused spawn and records birthtime/cgroup before resume, then waits for the
original target's natural exit. No PID-only cleanup or target-kill policy was
introduced.
The shared-service import also exposed an existing module-mock load-order
assumption in the Evidence-file cancellation test. That one mocked test now
runs in the existing isolated `process-global` lane, with an assertion that the
mocked read actually ran. Other file/service tests keep the shared module graph.
The final exact ten-file gate passes 225 tests with file parallelism disabled
under one CPU. An earlier parallel gate hit one existing 30-second MCP lifecycle
timeout; its serial retry passed without increasing a timeout or changing the
production lifecycle contract.
## Reproduction
Use the pinned Node/npm toolchain and native Frida dependency. Freeze separate
baseline/candidate built packages and install the same locked dependencies.
Run each case independently with the resource observer:
```sh
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
node experiments/analysis-efficiency/frida-status-queries.mjs \
--runtime /absolute/built-package --output /absolute/new-case \
--capture libc --detail summary
```
Repeat with `--detail complete` and `--capture small`. For actual-client captures,
`frida-status-wire.py --help` names the runtime, pinned Codex and Node executables,
output directory and selected script file (`selected-script.js` from the SDK
case). Use both `--mode complete`/`summary` and `--budget 300000`/`0`.
`frida-status-costs.py` checks task/resource receipts and counts complete requests.
The first wire attempt used ambiguous suffix matching for load/unload names;
the fixture now selects exact operation names. Failed attempts remain outside
the performance cohort. No production behavior changed to accommodate them.
@@ -0,0 +1,299 @@
"""Actual Codex wire capture with a deterministic local SSE backend; no AI inference."""
import argparse
import http.server
import threading
import asyncio
import hashlib
import json
import os
from pathlib import Path
import re
import time
import traceback
import subprocess
from frida_status_delivery import verify_frida_delivery
async def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, required=True)
parser.add_argument("--runtime", type=Path, required=True)
parser.add_argument("--codex", type=Path, required=True)
parser.add_argument("--node", type=Path, required=True)
parser.add_argument("--input-profile", choices=["full", "compact"], default="full")
parser.add_argument("--output-profile", choices=["full", "off"], default="full")
parser.add_argument("--script", type=Path, required=True)
parser.add_argument("--expected-tools", type=int, default=211)
parser.add_argument("--mode", choices=["complete", "summary"], required=True)
parser.add_argument("--budget", type=int, default=30000)
args = parser.parse_args()
source=args.script.read_text()
assert args.runtime.is_absolute() and args.output.is_absolute() and args.script.is_absolute()
def delivered(pieces):
assert isinstance(pieces,str) and pieces.startswith("Wall time: "), pieces[:300]
_,sep,payload=pieces.partition("Output:\n");assert sep
try: value=json.loads(payload)
except json.JSONDecodeError: return {"_invalid_json_payload":payload}
return value["result"] if "result" in value and "normalized_result" not in value else value
def outputs_of(body):
return {item["call_id"]:delivered(item["output"]) for item in body.get("input",[]) if item.get("type")=="function_call_output"}
steps=7
args.output.mkdir(mode=0o700)
audit_home = args.output / "account"
codex_state = audit_home / ".codex"
mcp_home = audit_home / "mcp"
workspace = audit_home / "workspace"
for directory in [codex_state, mcp_home, workspace]:
directory.mkdir(parents=True)
captured = []
known_outputs = {}
state = {"normal_requests":0,"compactions":0}
class Capture(http.server.BaseHTTPRequestHandler):
def log_message(self, *_):
pass
def do_POST(self):
size = int(self.headers["Content-Length"])
assert 0 < size <= 16 * 1024 * 1024
assert self.headers.get("Content-Encoding", "identity") == "identity"
body = json.loads(self.rfile.read(size))
captured.append({"path": self.path, "body": body})
(args.output / "model-requests.json").write_text(json.dumps(captured) + "\n")
http_index = len(captured)
known_outputs.update(outputs_of(body))
names = [t.get("name", "") for ns in body.get("tools", []) if ns.get("type") == "namespace" and ns.get("name") == "mcp__rea_audit" for t in ns.get("tools", [])]
if not names:
last=body.get('input',[])[-1]
assert 'CONTEXT CHECKPOINT COMPACTION' in json.dumps(last)
state['compactions']+=1;assert state['compactions']<=1
checkpoint={'fixture':'deterministic observed-ID checkpoint, no AI inference','task':'Check Frida session state, queued message count and truncation twice; retain/export complete observations and close instrumentation.','session_id':known_outputs['call_local_1']['session_id'],'delivered_call_ids':list(known_outputs),'remaining':'Continue scripted tools; source message bodies remain in retained Evidence.'}
text=json.dumps(checkpoint)
item={"type":"message","id":f"msg_checkpoint_{state['compactions']}","role":"assistant","status":"completed","content":[{"type":"output_text","text":text,"annotations":[]}]}
captured[-1]['kind']='deterministic-compaction'
else:
state['normal_requests']+=1;index=state['normal_requests'];assert index<=steps+1
captured[-1]['kind']='tool-planning-or-final'
if index<=steps:
if index==1:
operation='start_frida_session';arguments={'mode':'spawn','device_id':'local','program':'/usr/bin/sleep','argv':['8']}
else:
session=known_outputs['call_local_1']['session_id']
if index==2:
operation='load_frida_script';arguments={'session_id':session,'source_kind':'inline','source':source}
elif index==3:
operation='resume_frida_session';arguments={'session_id':session}
elif index in [4,5]:
if index==4: time.sleep(0.75)
operation='frida_session_status';arguments={'session_id':session,**({'detail':'summary'} if args.mode=='summary' else {})}
elif index==6:
operation='close_frida_session';arguments={'session_id':session}
else:
operation='export_evidence_bundle';arguments={'path':str(args.output/'evidence.json')}
selected=[name for name in names if name == operation];assert len(selected)==1
item={"type":"function_call","id":f"fc_local_{index}","call_id":f"call_local_{index}","name":selected[0],"namespace":"mcp__rea_audit","arguments":json.dumps(arguments),"status":"completed"}
else:
item={"type":"message","id":"msg_local_done","role":"assistant","status":"completed","content":[{"type":"output_text","text":"Scripted local Frida state and capture-coverage task complete.","annotations":[]}]}
response_id=f"resp_local_{http_index}"
base = {"id": response_id, "object": "response", "model": body.get("model"), "status": "completed", "output": [item]}
events = [
{"type": "response.created", "response": {**base, "status": "in_progress", "output": []}},
{"type": "response.output_item.added", "output_index": 0, "item": {**item, "status": "in_progress"}},
{"type": "response.output_item.done", "output_index": 0, "item": item},
{"type": "response.completed", "response": base},
]
wire = "".join("event: " + e["type"] + "\ndata: " + json.dumps(e) + "\n\n" for e in events).encode()
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.send_header("Content-Length", str(len(wire)))
self.end_headers()
self.wfile.write(wire)
endpoint = http.server.HTTPServer(("127.0.0.1", 0), Capture)
endpoint_thread = threading.Thread(target=endpoint.serve_forever, daemon=True)
endpoint_thread.start()
task_path = f"{args.node.parent}:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
mcp_env = {"HOME": str(mcp_home), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1",
"REA_LOG_LEVEL": "silent", "REA_MCP_INPUT_SCHEMA_PROFILE": args.input_profile,
"REA_MCP_OUTPUT_SCHEMA_PROFILE": args.output_profile}
config = 'model_provider = "rea_capture"\nmodel = "rea-wire-probe"\n[analytics]\nenabled = false\n[mcp_servers.rea_audit]\n'
config += f'command = {json.dumps(str(args.node))}\n'
config += f'args = {json.dumps([str(args.runtime / "scripts/rea.mjs"), "mcp"])}\n'
config += 'startup_timeout_sec = 60\ntool_timeout_sec = 60\n'
config += '[mcp_servers.rea_audit.env]\n'
config += ''.join(f'{key} = {json.dumps(value)}\n' for key, value in mcp_env.items())
for name in ['start_frida_session','load_frida_script','resume_frida_session','frida_session_status','close_frida_session','export_evidence_bundle']:
config += f'[mcp_servers.rea_audit.tools.{name}]\napproval_mode = "approve"\n'
if args.budget: config += f'output_token_limit = {args.budget}\n'
config += f'\n[model_providers.rea_capture]\nname = "Local scripted capture"\nbase_url = "http://127.0.0.1:{endpoint.server_port}/v1"\nwire_api = "responses"\nrequires_openai_auth = false\nsupports_websockets = false\nrequest_max_retries = 0\nstream_max_retries = 0\n'
(codex_state / "config.toml").write_text(config)
(args.output / "declared-config.toml").write_text(config)
environment = {"HOME": str(audit_home), "CODEX_HOME": str(codex_state), "PATH": task_path,
"NODE_OPTIONS": "--max-old-space-size=2048", "UV_THREADPOOL_SIZE": "1"}
report = {"question": "Retain a caller-requested native libc export inventory, check session identity/state, queued message count and truncation twice, close instrumentation and export complete Evidence.",
"client": "Codex app-server 0.163.0", "model_turns_started": 1, "backend": "local deterministic SSE fixture; no AI model or paid service",
"actual_model_usage": None, "input_profile": args.input_profile,
"output_profile": args.output_profile, "calls": [], "observations": [],
"unexpected_server_requests": [], "cleanup_failures": []}
client_version = subprocess.check_output([str(args.codex), "--version"], env=environment, cwd=workspace, text=True, timeout=10).strip()
assert client_version == "codex-cli 0.163.0", client_version
transcript = []
pending = {}
completed_turn = asyncio.Event()
next_id = 0
stderr = (args.output / "server.log").open("wb")
process = await asyncio.create_subprocess_exec(str(args.codex), "app-server", "--stdio",
stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=stderr,
cwd=workspace, env=environment, limit=16 * 1024 * 1024)
async def send(message):
encoded = (json.dumps(message, separators=(",", ":")) + "\n").encode()
transcript.append({"direction": "client", "bytes": len(encoded), "message": message})
process.stdin.write(encoded)
await process.stdin.drain()
async def read_messages():
try:
while line := await process.stdout.readline():
value = json.loads(line)
transcript.append({"direction": "server", "bytes": len(line), "message": value})
if value.get("method") == "turn/completed":
report["turn"] = value["params"]["turn"]
completed_turn.set()
if "id" in value and "method" not in value:
future = pending.pop(value["id"], None)
if future and not future.done():
future.set_result(value)
elif "id" in value:
report["unexpected_server_requests"].append(value["method"])
await send({"id": value["id"], "error": {"code": -32601, "message": "Unexpected request in scripted metadata probe"}})
raise RuntimeError("app-server stdout closed")
except Exception as error:
for future in pending.values():
if not future.done():
future.set_exception(error)
reader = asyncio.create_task(read_messages())
async def request(method, params):
nonlocal next_id
assert method in ["initialize", "thread/start", "mcpServerStatus/list", "mcpServer/tool/call", "turn/start"]
next_id += 1
message = {"id": next_id, "method": method, "params": params}
future = asyncio.get_running_loop().create_future()
pending[next_id] = future
start = time.monotonic()
await send(message)
reply = await asyncio.wait_for(future, 90)
report["calls"].append({"method": method, "elapsed_ms": (time.monotonic() - start) * 1000,
"error": reply.get("error")})
if "error" in reply:
raise RuntimeError(json.dumps(reply["error"])[:4000])
return reply["result"]
try:
await request("initialize", {"clientInfo": {"name": "rea_host_probe", "title": "REA host compatibility probe", "version": "1"},
"capabilities": {"experimentalApi": True, "explicitGatewayOauth": True}})
await send({"method": "initialized"})
created = await request("thread/start", {"cwd": str(workspace), "ephemeral": True,
"approvalPolicy": "never", "sandbox": "read-only"})
thread_id = created["thread"]["id"]
assert not created["thread"].get("turns", [])
inventory = await request("mcpServerStatus/list", {"threadId": thread_id,
"serverName": "rea_audit", "detail": "toolsAndAuthOnly", "limit": 10})
assert inventory.get("nextCursor") is None
assert len(inventory["data"]) == 1
server = inventory["data"][0]
assert server["name"] == "rea_audit" and server.get("toolsError") is None
tools = list(server["tools"].values())
names = [tool["name"] for tool in tools]
assert len(names) == len(set(names)) == args.expected_tools
assert "frida_session_status" in names
output_schemas = sum(tool.get("outputSchema") is not None for tool in tools)
assert output_schemas == (args.expected_tools if args.output_profile == "full" else 0)
raw = json.dumps(inventory, separators=(",", ":"), ensure_ascii=False).encode()
(args.output / "catalog.json").write_bytes(raw)
report["catalog"] = {"tools": len(tools), "output_schemas": output_schemas,
"bytes": len(raw), "sha256": hashlib.sha256(raw).hexdigest(),
"server_info": server.get("serverInfo"), "runtime_status": server.get("runtimeStatus")}
await request("turn/start", {"threadId":thread_id,"input":[{"type":"text","text":"Retain a complete libc export inventory using the selected caller-authored script. Check session state and queued message capture count/truncation twice, close Frida instrumentation and export complete Evidence. The local fixture scripts the tools; no AI inference."}]})
await asyncio.wait_for(completed_turn.wait(), 90)
assert report["turn"]["status"] == "completed", report["turn"]
assert state["normal_requests"]==steps+1
assert len(captured)==steps+1+state["compactions"]
outputs=known_outputs
assert len(outputs)==steps
bundle=json.loads((args.output/'evidence.json').read_text())
delivery_complete=True
answers=[]
for index in [4,5]:
delivered_result=outputs[f'call_local_{index}']
answer=verify_frida_delivery(delivered_result,bundle,args.mode,args.budget,outputs['call_local_1'])
if answer is None: delivery_complete=False
else: answers.append(answer)
loaded=next(e for e in bundle['records'] if e['operation']=='load_frida_script')
assert loaded['normalized_result']['source_sha256']==hashlib.sha256(source.encode()).hexdigest()
report['task_completed']=delivery_complete
report['observations']={'answers':answers,'exported':True,'exported_bundle_bytes':(args.output/'evidence.json').stat().st_size,'tool_output_budget':args.budget or 'client default','source_sha256':hashlib.sha256(source.encode()).hexdigest()}
report["captured_requests"]=len(captured)
report["compaction_requests"]=state["compactions"]
report["compaction_backend"]="Deterministic checkpoint of observed identifiers and retained-body location; no AI inference or quality claim."
report["mode"]=args.mode
report["request_tool_counts"] = [len(c["body"].get("tools", [])) for c in captured]
assert not report["unexpected_server_requests"]
report["status"] = "passed" if report["task_completed"] else "expected-incomplete-task"
except Exception:
report["status"] = "failed"
report["failure"] = traceback.format_exc()[-6000:]
finally:
process.stdin.close()
try:
await asyncio.wait_for(process.wait(), 15)
except asyncio.TimeoutError:
report["cleanup_failures"].append("app-server did not exit after stdin close; terminated owned process")
process.terminate()
try:
await asyncio.wait_for(process.wait(), 5)
except asyncio.TimeoutError:
process.kill()
await process.wait()
await reader
# app-server can acknowledge exit before its MCP child finishes EOF
# cleanup. Wait on the allocated resource scope; never target a PID.
try:
groups = [line[3:] for line in Path("/proc/self/cgroup").read_text().splitlines()
if line.startswith("0::")]
assert len(groups) == 1
group = Path("/sys/fs/cgroup") / groups[0].lstrip("/")
assert re.fullmatch(r"rea-efficiency-[0-9a-f]{32}\.scope", group.name)
observer = os.getppid()
observer_args = Path(f"/proc/{observer}/cmdline").read_bytes().split(b"\0")
assert any(Path(os.fsdecode(value)).name == "resource_payload.py" for value in observer_args if value)
expected = {os.getpid(), observer}
until = time.monotonic() + 10
while True:
actors = {int(value) for value in (group / "cgroup.procs").read_text().splitlines()}
assert expected <= actors
remaining = sorted(actors - expected)
if not remaining or time.monotonic() >= until:
break
await asyncio.sleep(0.05)
report["owned_scope_drain"] = {"remaining_processes": remaining}
if remaining:
report["cleanup_failures"].append("owned MCP child did not settle after app-server exit")
except (OSError, ValueError, AssertionError) as error:
report["owned_scope_drain"] = {"status": "unavailable", "reason": type(error).__name__}
report["cleanup_failures"].append("allocated resource-scope cleanup could not be verified")
endpoint.shutdown()
endpoint.server_close()
endpoint_thread.join(5)
stderr.close()
report["appserver_exit_code"] = process.returncode
if process.returncode != 0:
report["cleanup_failures"].append(f"app-server exit code: {process.returncode}")
(args.output / "transcript.json").write_text(json.dumps(transcript) + "\n")
(args.output / "report.json").write_text(json.dumps(report, indent=2) + "\n")
print(json.dumps({key: report.get(key) for key in ["status", "failure", "catalog", "observations", "cleanup_failures", "appserver_exit_code"]}))
return 0 if report["status"] in ["passed","expected-incomplete-task"] and not report["cleanup_failures"] else 1
raise SystemExit(asyncio.run(main()))
@@ -0,0 +1,37 @@
"""Accept only complete client delivery of the selected Frida status question."""
def verify_frida_delivery(delivered, bundle, mode, budget, session):
if '_invalid_json_payload' in delivered:
assert mode == 'complete' and budget == 0, 'A truncated summary never completes the task'
return None
value = delivered['normalized_result']
parent = next(e for e in bundle['records'] if e['evidence_id'] == value['parent_evidence_id']) if mode == 'summary' else delivered
assert parent in bundle['records'] and parent['operation'] == 'frida_session_status'
assert parent['authority'] == 'external-service' and parent['environment'] is None
assert parent['raw_result'] is None and parent['subject'] is None
assert parent['parameters'] == {'session_id': session['session_id']}
facts = parent['normalized_result']
assert facts['state'] == 'running' and not facts['messages_truncated']
assert facts['session_id'] == session['session_id'] and facts['pid'] == session['pid']
payloads = [m['payload'] for m in facts['messages'] if m.get('type') == 'send']
header = next(p for p in payloads if p['kind'] == 'module')
complete = next(p for p in payloads if p['kind'] == 'complete')
count = 0
for chunk in [p for p in payloads if p['kind'] == 'exports']:
assert chunk['offset'] == count
count += len(chunk['items'])
assert count == header['total'] == complete['total'] and count > 1000
assert facts['pid'] == header['pid']
if mode == 'summary':
assert delivered in bundle['records']
assert delivered['subject'] is None and delivered['raw_result'] is None
assert delivered['authority'] == parent['authority']
assert value['parent_operation'] == parent['operation']
assert value['source_parameters'] == parent['parameters'] and value['source_provider'] == parent['provider']
assert value['parent_digest'] == parent['evidence_id'][3:] and value['artifact'] is None
assert value['summary'] == {**{k: v for k, v in facts.items() if k != 'messages'}, 'messages_retained': len(facts['messages'])}
assert value['unknowns'] and value['limitations']
return {'state': facts['state'], 'messages_retained': len(facts['messages']),
'messages_truncated': facts['messages_truncated'], 'exports': count,
'parent_evidence_id': parent['evidence_id']}
@@ -0,0 +1,81 @@
"""A truncated summary must never qualify as a completed investigation."""
from copy import deepcopy
import unittest
from command_observation_delivery import verify_command_delivery
class CommandDelivery(unittest.TestCase):
def test_truncation_is_accepted_only_for_complete_default_baseline(self):
truncated = {'_invalid_json_payload': '{truncated'}
self.assertFalse(verify_command_delivery(truncated, {}, 'complete', 0))
for mode, budget in [
('summary', 0), ('summary', 30000), ('complete', 30000),
]:
with self.subTest(mode=mode, budget=budget):
with self.assertRaises(AssertionError):
verify_command_delivery(truncated, {}, mode, budget)
def test_valid_complete_answer_can_complete_under_either_budget(self):
parent = {'evidence_id': 'parent'}
for budget in [0, 30000]:
self.assertTrue(
verify_command_delivery(parent, parent, 'complete', budget)
)
with self.assertRaises(AssertionError):
verify_command_delivery({'error': 'failed command'}, parent, 'complete', 0)
def test_summary_must_preserve_every_record_and_accurate_stream_counts(self):
report = {
'command': 'disassemble /r entry', 'mi': '7^done',
'records': ['~"🧪\\n"', '=async', 'unknown', '7^done'],
'console': '🧪', 'target': '', 'log': '',
'completion_status': 'complete', 'output_truncated': False,
}
parent = {
'evidence_id': 'parent',
'parameters': {'command': report['command']},
'provider': {'id': 'gdb'}, 'analysis_profile': None,
'normalized_result': report,
}
facts = {
key: value for key, value in report.items()
if key not in ('console', 'target', 'log')
}
selected = {
'evidence_id': 'derived', 'raw_result': None, 'subject': None,
'normalized_result': {
'parent_evidence_id': 'parent', 'artifact': None,
'source_analysis_profile_digest': None,
'source_parameters': parent['parameters'],
'source_provider': parent['provider'],
'summary': {
**facts,
'available_counts': {
'records': 4, 'console_utf16_units': 2,
'target_utf16_units': 0, 'log_utf16_units': 0,
},
},
},
}
self.assertTrue(verify_command_delivery(selected, parent, 'summary', 0))
for damage in ['records', 'counts', 'parameters', 'profile']:
bad = deepcopy(selected)
result = bad['normalized_result']
if damage == 'records':
result['summary']['records'].pop(1)
elif damage == 'counts':
result['summary']['available_counts']['console_utf16_units'] = 1
elif damage == 'profile':
result['source_analysis_profile_digest'] = 'wrong'
else:
result['source_parameters'] = {}
with self.subTest(damage=damage):
with self.assertRaises(AssertionError):
verify_command_delivery(bad, parent, 'summary', 0)
parent['analysis_profile'] = {'digest': 'recorded-profile'}
selected['normalized_result']['source_analysis_profile_digest'] = 'recorded-profile'
self.assertTrue(verify_command_delivery(selected, parent, 'summary', 0))
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,48 @@
"""Reject incomplete status delivery and incorrect retained-source claims."""
from copy import deepcopy
import unittest
from frida_status_delivery import verify_frida_delivery
class FridaDelivery(unittest.TestCase):
def fixture(self):
session = {'session_id': 'selected', 'pid': 12}
payloads = [{'kind': 'module', 'pid': 12, 'total': 1001}, {'kind': 'exports', 'offset': 0, 'items': [None] * 1001}, {'kind': 'complete', 'total': 1001}]
facts = {**session, 'state': 'running', 'messages_truncated': False, 'messages': [{'type': 'send', 'payload': p} for p in payloads]}
parent = {'evidence_id': 'ev_parent', 'operation': 'frida_session_status', 'authority': 'external-service', 'environment': None, 'raw_result': None, 'subject': None, 'parameters': {'session_id': 'selected'}, 'provider': {'id': 'frida'}, 'normalized_result': facts}
summary = {'evidence_id': 'ev_summary', 'subject': None, 'raw_result': None, 'authority': 'external-service', 'normalized_result': {
'parent_evidence_id': 'ev_parent', 'parent_digest': 'parent', 'parent_operation': 'frida_session_status', 'artifact': None,
'source_parameters': parent['parameters'], 'source_provider': parent['provider'], 'unknowns': ['historical snapshot'], 'limitations': ['parent retained'],
'summary': {**{k: v for k, v in facts.items() if k != 'messages'}, 'messages_retained': 3}}}
return session, parent, summary
def test_truncated_summary_never_qualifies(self):
truncated = {'_invalid_json_payload': 'truncated'}
self.assertIsNone(verify_frida_delivery(truncated, {}, 'complete', 0, {}))
for mode, budget in [('summary', 0), ('summary', 300000), ('complete', 300000)]:
with self.subTest(mode=mode, budget=budget), self.assertRaises(AssertionError):
verify_frida_delivery(truncated, {}, mode, budget, {})
def test_complete_and_summary_require_correct_facts_and_source(self):
session, parent, summary = self.fixture()
for mode, delivered in [('complete', parent), ('summary', summary)]:
self.assertEqual(verify_frida_delivery(delivered, {'records': [parent, summary]}, mode, 300000, session)['exports'], 1001)
for field in ['messages_retained', 'state', 'messages', 'source_parameters', 'parent_digest', 'source_provider']:
broken = deepcopy(summary)
target = broken['normalized_result']['summary'] if field in ['messages_retained', 'state', 'messages'] else broken['normalized_result']
target[field] = 'incorrect'
with self.subTest(field=field), self.assertRaises(AssertionError):
verify_frida_delivery(broken, {'records': [parent, broken]}, 'summary', 0, session)
def test_missing_native_chunk_and_wrong_session_fail(self):
session, parent, _ = self.fixture()
broken = deepcopy(parent)
broken['normalized_result']['messages'][1]['payload']['items'].pop()
with self.assertRaises(AssertionError):
verify_frida_delivery(broken, {'records': [broken]}, 'complete', 300000, session)
with self.assertRaises(AssertionError):
verify_frida_delivery(parent, {'records': [parent]}, 'complete', 300000, {**session, 'session_id': 'other'})
if __name__ == '__main__':
unittest.main()
+8 -2
View File
@@ -119,7 +119,13 @@
"precheck:pr": "npm run deps:check",
"check:pr": "node scripts/run-exclusive.mjs artifacts turbo run typecheck lint:built format:check knip metadata:check test:run && npm run test:python && npm run docs:check && npm run jscpd && npm run scan:todos",
"precheck:ci": "npm run deps:check",
"check:ci": "node scripts/run-exclusive.mjs artifacts npm run check:ci:tasks && npm run verify:architecture-guards",
"precheck:ci:fast": "npm run deps:check",
"precheck:ci:deep": "npm run deps:check",
"check:ci": "npm run check:ci:fast && npm run check:ci:deep",
"check:ci:fast": "node scripts/run-exclusive.mjs artifacts npm run check:ci:fast:tasks",
"check:ci:fast:tasks": "turbo run typecheck lint:built format:check",
"check:ci:deep": "node scripts/run-exclusive.mjs artifacts npm run check:ci:deep:tasks && npm run verify:architecture-guards && npm run verify:test-discovery && npm run test:python",
"check:ci:deep:tasks": "turbo run knip metadata:check jscpd scan:todos",
"check:ci:tasks": "turbo run typecheck lint:built format:check knip metadata:check jscpd scan:todos",
"ci:static-context": "node scripts/ci/static-task-context.mjs",
"build:cached": "node scripts/check-dependency-install.mjs && node scripts/run-exclusive.mjs artifacts turbo run artifacts:skills",
@@ -131,7 +137,7 @@
"lint:details": "oxlint && npm run verify:module-boundaries",
"lint:built": "npm run lint",
"lint:fix": "oxlint --fix && npm run verify:module-boundaries",
"knip": "knip",
"knip": "knip --cache",
"jscpd": "node scripts/ci/static-check-report.mjs duplicates",
"jscpd:details": "jscpd src/ tests/ --no-tips",
"scan:todos": "node scripts/scan-todos.mjs",
+1
View File
@@ -49,6 +49,7 @@ export function selectStaticCheckArtifact(
page: unknown,
run: { id?: string | number; head_sha?: string },
runAttempt: number,
name?: string,
): unknown | null;
export function readStaticCheckArtifact(options: {
+14 -5
View File
@@ -7,6 +7,11 @@ import { promisify } from "node:util";
import { parseStaticTaskContext } from "./static-task-context.mjs";
const execute = promisify(execFile);
/** Job display name owns the artifact that contains that gate's diagnostics. */
export const STATIC_EVIDENCE_ARTIFACTS = {
"Static checks": "static-check-reports",
"Deep static checks": "static-deep-check-reports",
};
const MAX_ARCHIVE_BYTES = 32 * 1024 * 1024;
const MAX_MEMBER_BYTES = 4 * 1024 * 1024;
const MAX_DIAGNOSTICS = 10_000;
@@ -43,7 +48,8 @@ export function classifyFailure({
"run_event_or_repository_identity_unverified",
],
() => [
job?.name !== mainJob?.name || job?.name !== "Static checks",
job?.name !== mainJob?.name ||
!Object.hasOwn(STATIC_EVIDENCE_ARTIFACTS, job?.name ?? ""),
"failed_job_not_the_same_static_gate",
],
() => [
@@ -134,7 +140,12 @@ export function safeSha(value) {
? value.toLowerCase()
: null;
}
export function selectStaticCheckArtifact(page, run, runAttempt) {
export function selectStaticCheckArtifact(
page,
run,
runAttempt,
name = STATIC_EVIDENCE_ARTIFACTS["Static checks"],
) {
if (runAttempt !== 1) return null;
if (
!Array.isArray(page?.artifacts) ||
@@ -143,9 +154,7 @@ export function selectStaticCheckArtifact(page, run, runAttempt) {
page.artifacts.length >= 100
)
return null;
const matches = page.artifacts.filter(
(artifact) => artifact.name === "static-check-reports",
);
const matches = page.artifacts.filter((artifact) => artifact.name === name);
if (matches.length !== 1) return null;
const [artifact] = matches;
if (
+61
View File
@@ -0,0 +1,61 @@
export interface RetainedCacheEntry {
readonly id: number;
readonly key: string;
readonly createdAt: number;
readonly accessedAt: number;
readonly size: number;
}
export interface CacheListRecord {
readonly id: number;
readonly key: string;
readonly version: string;
readonly ref: string;
readonly created_at: string;
readonly last_accessed_at?: string;
readonly size_in_bytes: number;
}
export interface RetentionFamily {
readonly prefix: string;
readonly keepKey?: string;
}
export type CacheRequest = (
method: string,
endpoint: string,
) => Promise<unknown>;
export function assertRetentionPrefix(prefix: string): void;
export function selectPrefixDeletions(
caches: readonly unknown[],
family?: RetentionFamily,
keepGenerations?: number,
): RetainedCacheEntry[];
export function selectNpmDownloadDuplicateDeletions(
caches: readonly unknown[],
): RetainedCacheEntry[];
export function listAllCaches(
request: CacheRequest,
repository: string,
prefix: string,
): Promise<readonly CacheListRecord[]>;
export function retainCaches(options: {
repository: string;
request: CacheRequest;
deleteCache: (id: number) => Promise<void>;
families?: readonly RetentionFamily[];
dedupeNpmDownloads?: boolean;
dryRun?: boolean;
}): Promise<{
dryRun: boolean;
plan: readonly {
prefix: string;
deletions: readonly RetainedCacheEntry[];
}[];
npmDeletions: readonly RetainedCacheEntry[];
}>;
+358
View File
@@ -0,0 +1,358 @@
import { execFile } from "node:child_process";
import { realpathSync } from "node:fs";
import { pathToFileURL } from "node:url";
import { promisify } from "node:util";
const execFileAsync = promisify(execFile);
const MAX_BUFFER = 8 * 1024 * 1024;
const REQUEST_TIMEOUT_MS = 20_000;
const MAX_PAGES = 10;
const PAGE_SIZE = 100;
// Keep the newest snapshots, not only the key just saved. A concurrent job may
// still be restoring the previous generation; deleting that id fails its restore.
// The following trusted run drops anything older than those two.
const RETAINED_CACHE_GENERATIONS = 2;
const PREFIX_PATTERNS = [
/^rea-static-v1-[A-Za-z0-9]+-[A-Za-z0-9]+-$/u,
/^rea-static-deep-v1-[A-Za-z0-9]+-[A-Za-z0-9]+-$/u,
/^rea-portable-build-v2-[A-Za-z0-9]+-[A-Za-z0-9]+-$/u,
/^knip-meta-v1-[A-Za-z0-9]+-$/u,
];
/** Reject prefixes that could match npm downloads or another platform. */
export function assertRetentionPrefix(prefix) {
if (
typeof prefix !== "string" ||
!PREFIX_PATTERNS.some((pattern) => pattern.test(prefix))
)
throw new Error(
"Cache retention prefix must be one Turbo family for one OS and architecture, or one Knip metadata family for one OS",
);
}
function normalizeCache(cache, index) {
if (typeof cache !== "object" || cache === null || Array.isArray(cache))
throw new Error(`Cache entry ${String(index + 1)} is malformed`);
const { id, key, size_in_bytes: size } = cache;
if (!Number.isSafeInteger(id) || id <= 0)
throw new Error(`Cache entry ${String(index + 1)} has an invalid id`);
if (typeof key !== "string" || key.length === 0 || /[\r\n]/u.test(key))
throw new Error(`Cache entry ${String(index + 1)} has an invalid key`);
const createdAt = Date.parse(cache.created_at);
const accessedAt = Date.parse(cache.last_accessed_at ?? cache.created_at);
if (!Number.isFinite(createdAt) || !Number.isFinite(accessedAt))
throw new Error(
`Cache entry ${String(index + 1)} has an invalid timestamp`,
);
if (!Number.isSafeInteger(size) || size < 0)
throw new Error(`Cache entry ${String(index + 1)} has an invalid size`);
// actions/cache restores a key only when this metadata hash matches.
const version = cache.version;
if (typeof version !== "string" || !/^[a-f0-9]{64}$/u.test(version))
throw new Error(`Cache entry ${String(index + 1)} has an invalid version`);
// The git ref is the cache's access scope. Another branch cannot use it.
const ref = cache.ref;
if (typeof ref !== "string" || !/^refs\/[^\s]+$/u.test(ref))
throw new Error(`Cache entry ${String(index + 1)} has an invalid ref`);
return { id, key, version, ref, createdAt, accessedAt, size };
}
function sameStableIdentity(left, right) {
return (
left.key === right.key &&
left.version === right.version &&
left.ref === right.ref &&
left.size === right.size &&
left.createdAt === right.createdAt
);
}
// The list API orders by last access. A restore between pages can place one
// id on two pages. That is still one cache: collapse it, or abort when the
// pages disagree, before any id can be both kept and deleted.
function uniqueById(entries) {
const byId = new Map();
for (const entry of entries) {
const existing = byId.get(entry.id);
if (existing === undefined) {
byId.set(entry.id, entry);
continue;
}
if (!sameStableIdentity(existing, entry))
throw new Error(
`Cache id ${String(entry.id)} was listed with inconsistent fields`,
);
if (entry.accessedAt > existing.accessedAt) byId.set(entry.id, entry);
}
return [...byId.values()];
}
function byNewest(left, right) {
if (left.createdAt !== right.createdAt)
return right.createdAt - left.createdAt;
return right.id - left.id;
}
/**
* Choose Turbo snapshot ids to delete. `keepKey` is protected even when it is
* older than the retained generations. Keys outside the prefix abort retention.
*/
export function selectPrefixDeletions(
caches,
{ prefix, keepKey } = {},
keepGenerations = RETAINED_CACHE_GENERATIONS,
) {
assertRetentionPrefix(prefix);
if (!Number.isSafeInteger(keepGenerations) || keepGenerations < 1)
throw new Error("Cache retention must keep at least one generation");
if (keepKey !== undefined) {
if (
typeof keepKey !== "string" ||
!keepKey.startsWith(prefix) ||
keepKey === prefix
)
throw new Error("Cache retention keep key must extend its prefix");
}
const entries = uniqueById(caches.map(normalizeCache));
for (const entry of entries)
if (!entry.key.startsWith(prefix))
throw new Error(
`Refusing to evaluate a cache outside ${prefix}: ${entry.key}`,
);
const protectedIds = new Set(
[...entries]
.sort(byNewest)
.slice(0, keepGenerations)
.map((entry) => entry.id),
);
if (keepKey !== undefined)
for (const entry of entries)
if (entry.key === keepKey) protectedIds.add(entry.id);
return entries.filter((entry) => !protectedIds.has(entry.id));
}
/**
* Drop extra copies of one npm restore target. Records that share a user key
* but differ by cache version or git ref stay, because a workflow can restore
* only a matching version inside its visible scope.
*/
export function selectNpmDownloadDuplicateDeletions(caches) {
const entries = uniqueById(caches.map(normalizeCache));
for (const entry of entries)
if (!entry.key.startsWith("npm-download-"))
throw new Error(
`Refusing to evaluate a non-download cache: ${entry.key}`,
);
const groups = new Map();
for (const entry of entries) {
const identity = `${entry.version}\n${entry.ref}\n${entry.key}`;
const group = groups.get(identity) ?? [];
group.push(entry);
groups.set(identity, group);
}
const deletions = [];
for (const group of groups.values()) {
if (group.length < 2) continue;
const ranked = [...group].sort((left, right) => {
if (left.accessedAt !== right.accessedAt)
return right.accessedAt - left.accessedAt;
if (left.createdAt !== right.createdAt)
return right.createdAt - left.createdAt;
return right.id - left.id;
});
deletions.push(...ranked.slice(1));
}
return deletions;
}
export async function listAllCaches(request, repository, prefix) {
const caches = [];
for (let page = 1; page <= MAX_PAGES; page += 1) {
const body = await request(
"GET",
`repos/${repository}/actions/caches?per_page=${String(PAGE_SIZE)}&page=${String(page)}&key=${encodeURIComponent(prefix)}`,
);
if (
typeof body !== "object" ||
body === null ||
Array.isArray(body) ||
!Array.isArray(body.actions_caches)
)
throw new Error("GitHub cache list is malformed");
caches.push(...body.actions_caches);
if (body.actions_caches.length < PAGE_SIZE) return caches;
}
throw new Error(
`Cache list for ${prefix} exceeded ${String(MAX_PAGES)} pages; deleted nothing`,
);
}
/**
* List every requested family before deleting. A malformed list or an unsafe
* prefix leaves the repository caches unchanged.
*/
export async function retainCaches({
repository,
request,
deleteCache,
families = [],
dedupeNpmDownloads = false,
dryRun = false,
}) {
if (
typeof repository !== "string" ||
!/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/u.test(repository)
)
throw new Error(
"CI_REPOSITORY must be an owner/name repository identifier",
);
for (const family of families) {
assertRetentionPrefix(family.prefix);
if (
family.keepKey !== undefined &&
(typeof family.keepKey !== "string" ||
!family.keepKey.startsWith(family.prefix) ||
family.keepKey === family.prefix)
)
throw new Error("Cache retention keep key must extend its prefix");
}
const plan = [];
for (const family of families) {
const caches = await listAllCaches(request, repository, family.prefix);
plan.push({
prefix: family.prefix,
deletions: selectPrefixDeletions(caches, family),
});
}
const npmDeletions = dedupeNpmDownloads
? selectNpmDownloadDuplicateDeletions(
await listAllCaches(request, repository, "npm-download-"),
)
: [];
if (!dryRun)
for (const entry of [
...plan.flatMap((family) => family.deletions),
...npmDeletions,
])
await deleteCache(entry.id);
return { dryRun, plan, npmDeletions };
}
function repositoryFromEnv(env) {
const repository = env.CI_REPOSITORY;
if (
typeof repository !== "string" ||
!/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/u.test(repository)
)
throw new Error(
"CI_REPOSITORY must be an owner/name repository identifier",
);
return repository;
}
function familyWithKeepKey(env, prefixName, keepName) {
const prefix = env[prefixName];
const keepKey = env[keepName];
const hasPrefix = typeof prefix === "string" && prefix.length > 0;
const hasKeepKey = typeof keepKey === "string" && keepKey.length > 0;
if (!hasPrefix && !hasKeepKey) return undefined;
if (!hasPrefix || !hasKeepKey)
throw new Error(`${keepName} is required with ${prefixName}`);
return { prefix, keepKey };
}
function familiesFromEnv(env) {
const families = [
familyWithKeepKey(env, "CACHE_PREFIX", "CACHE_KEEP_KEY"),
familyWithKeepKey(env, "CACHE_DEEP_PREFIX", "CACHE_DEEP_KEEP_KEY"),
familyWithKeepKey(env, "CACHE_KNIP_PREFIX", "CACHE_KNIP_KEEP_KEY"),
typeof env.CACHE_PORTABLE_PREFIX === "string" &&
env.CACHE_PORTABLE_PREFIX.length > 0
? { prefix: env.CACHE_PORTABLE_PREFIX }
: undefined,
].filter((family) => family !== undefined);
if (families.length === 0 && env.DEDUPE_NPM_DOWNLOADS !== "true")
throw new Error(
"Cache retention requires CACHE_PREFIX or CACHE_PORTABLE_PREFIX or npm dedupe",
);
return families;
}
function failureText(error) {
if (typeof error !== "object" || error === null) return String(error);
const stderr =
"stderr" in error && error.stderr !== undefined ? String(error.stderr) : "";
const message = error instanceof Error ? error.message : "";
return `${message}\n${stderr}`;
}
async function githubRequest(method, endpoint, env) {
const { stdout } = await execFileAsync(
"gh",
["api", "--method", method, endpoint],
{
env,
encoding: "utf8",
maxBuffer: MAX_BUFFER,
timeout: REQUEST_TIMEOUT_MS,
},
);
if (method === "DELETE") return undefined;
try {
return JSON.parse(stdout);
} catch (cause) {
throw new Error("GitHub cache API returned malformed JSON", { cause });
}
}
async function main() {
const env = process.env;
const dryRun = env.CACHE_RETENTION_DRY_RUN === "true";
const report = await retainCaches({
repository: repositoryFromEnv(env),
request: (method, endpoint) => githubRequest(method, endpoint, env),
deleteCache: async (id) => {
try {
await githubRequest(
"DELETE",
`repos/${repositoryFromEnv(env)}/actions/caches/${String(id)}`,
env,
);
} catch (error) {
if (/404|not found/iu.test(failureText(error))) return;
throw error;
}
},
families: familiesFromEnv(env),
dedupeNpmDownloads: env.DEDUPE_NPM_DOWNLOADS === "true",
dryRun,
});
const summary = {
dryRun: report.dryRun,
families: report.plan.map((family) => ({
prefix: family.prefix,
deleteCount: family.deletions.length,
deleteBytes: family.deletions.reduce((sum, entry) => sum + entry.size, 0),
})),
npmDuplicateDeleteCount: report.npmDeletions.length,
npmDuplicateDeleteBytes: report.npmDeletions.reduce(
(sum, entry) => sum + entry.size,
0,
),
};
process.stdout.write(`${JSON.stringify(summary)}\n`);
}
function isDirectRun() {
const entry = process.argv[1];
if (!entry) return false;
try {
return import.meta.url === pathToFileURL(realpathSync(entry)).href;
} catch {
return false;
}
}
if (isDirectRun()) await main();
+9 -2
View File
@@ -11,6 +11,7 @@ import {
readStaticCheckArtifact,
safeSha,
selectStaticCheckArtifact,
STATIC_EVIDENCE_ARTIFACTS,
validRepoPath,
} from "./artifacts.mjs";
import { createGhApiRequest } from "./github-api.mjs";
@@ -366,12 +367,18 @@ export function createGitHubApi({
return page.jobs;
},
async getStaticEvidence(repository, run, job) {
if (job.name !== "Static checks") return null;
const artifactName = STATIC_EVIDENCE_ARTIFACTS[job.name];
if (!artifactName) return null;
if (run.run_attempt !== 1) return null;
const page = await request(
`repos/${repository}/actions/runs/${positiveId(run.id)}/artifacts?per_page=100&page=1`,
);
const artifact = selectStaticCheckArtifact(page, run, run.run_attempt);
const artifact = selectStaticCheckArtifact(
page,
run,
run.run_attempt,
artifactName,
);
if (!artifact) return null;
return await readStaticCheckArtifact({
repository,
+33 -2
View File
@@ -42,8 +42,18 @@ const matchesAny = (file, patterns) =>
patterns.some((pattern) => matchesGlob(file, pattern));
const selected = Object.fromEntries(flags.map((flag) => [flag, false]));
const reasons = [];
// Shared process, contract, CLI, and CI-script edits are not a specific provider.
// Fast qualification keeps the portable suite and the Linux x64 runtime
// baseline. Full qualification still selects every lane.
const SHARED_BOUNDARY =
/^(src\/process\/|src\/filesystem\/|src\/contracts\/tool|src\/server\/createServer|src\/cli\.ts|scripts\/ci\/)/;
const full = (reason) => {
for (const flag of flags) selected[flag] = true;
if (tier === "full") {
for (const flag of flags) selected[flag] = true;
} else {
selected.code = true;
selected.runtime = true;
}
reasons.push(reason);
};
const git = (...args) =>
@@ -196,6 +206,28 @@ if (tier === "reused") {
selected.code = true;
continue;
}
// Timing and report helpers stay on the portable suite. They are not a
// runtime or provider change.
if (PORTABLE_CI_REPORTERS.has(file)) {
selected.code = true;
continue;
}
// A shared boundary is not evidence that every provider or host changed.
// One matching owner is path evidence. Several owners means the same
// registration file is listed by many lanes; fast qualification does not
// fan those out or start the multi-host package matrix.
if (tier !== "full" && SHARED_BOUNDARY.test(file)) {
selected.code = true;
selected.runtime = true;
const owners = Object.entries(SCOPES)
.filter(([, patterns]) =>
patterns.some((pattern) => matchesGlob(file, pattern)),
)
.map(([owner]) => owner);
if (owners.length === 1) selected[owners[0]] = true;
reasons.push(`Shared boundary on fast tier: ${file}`);
continue;
}
const owners = Object.entries(SCOPES)
.filter(([, patterns]) =>
patterns.some((pattern) => matchesGlob(file, pattern)),
@@ -226,7 +258,6 @@ if (tier === "reused") {
}
if (matchesAny(file, READINESS_SOURCES)) selected.readiness = true;
if (file.startsWith("tests/")) continue;
if (PORTABLE_CI_REPORTERS.has(file)) continue;
if (
/^(src\/process\/|src\/filesystem\/|src\/contracts\/tool|src\/server\/createServer|src\/cli\.ts|scripts\/ci\/)/.test(
file,
+1
View File
@@ -29,6 +29,7 @@ if (
if (outputs.tier === "full" && selectedScopes.length !== CI_FLAGS.length)
throw new Error("Full qualification requires every canonical CI scope");
const required = reused ? ["changes"] : ["changes", "static", "test"];
if (outputs.tier === "full") required.push("static-deep");
const lanes = {
code: [
"build",
+1
View File
@@ -15,4 +15,5 @@ export function recordStaticTaskContext(options?: {
root?: string;
npmCli?: string;
outputDirectory?: string;
script?: string;
}): Promise<StaticTaskContext>;
+8 -1
View File
@@ -6,6 +6,10 @@ import { promisify } from "node:util";
import { z } from "zod";
const execute = promisify(execFile);
const STATIC_CONTEXT_SCRIPTS = new Set([
"check:ci:fast:tasks",
"check:ci:deep:tasks",
]);
const contextSchema = z.object({
version: z.literal(1),
sourceSha: z.string().regex(/^[a-f0-9]{40}$/u),
@@ -30,7 +34,10 @@ export async function recordStaticTaskContext({
root = process.cwd(),
npmCli = process.env.npm_execpath,
outputDirectory = join(root, ".cache/static-checks"),
script = process.env.CI_STATIC_CONTEXT_SCRIPT ?? "check:ci:fast:tasks",
} = {}) {
if (!STATIC_CONTEXT_SCRIPTS.has(script))
throw new Error("Static context script is not a known check");
if (typeof npmCli !== "string" || !npmCli)
throw new Error(
"Run static context recording through npm run ci:static-context",
@@ -44,7 +51,7 @@ export async function recordStaticTaskContext({
const [graph, source, status, npmVersion, turbo] = await Promise.all([
execute(
process.execPath,
[npmCli, "run", "--silent", "check:ci:tasks", "--", "--dry=json"],
[npmCli, "run", "--silent", script, "--", "--dry=json"],
options,
),
execute("git", ["rev-parse", "HEAD"], options),
+14 -3
View File
@@ -58,8 +58,7 @@ export async function artifactCliEvidence(
return evidence;
}
/** Run a real stdio MCP subprocess, owning and closing its transport. */
export async function withArtifactMcp(target, verify) {
const withOpenedArtifactMcp = async (target, openTool, verify) => {
const transport = new StdioClientTransport({
command: process.execPath,
args: [fileURLToPath(new URL("../rea.mjs", import.meta.url)), "mcp"],
@@ -70,15 +69,27 @@ export async function withArtifactMcp(target, verify) {
try {
await client.connect(transport);
const opened = await client.callTool({
name: "open_binary",
name: openTool,
arguments: { path: target },
});
assert.notEqual(opened.isError, true, JSON.stringify(opened));
if (openTool === "open_bundle_resources")
assert.equal(opened.structuredContent?.result?.caller_path, target);
await verify(client);
} finally {
await client.close();
await transport.close();
}
};
/** Run a real stdio MCP subprocess with an explicitly opened binary target. */
export async function withArtifactMcp(target, verify) {
return withOpenedArtifactMcp(target, "open_binary", verify);
}
/** Run a real stdio MCP subprocess with an explicitly opened bundle resource root. */
export async function withBundleResourcesMcp(target, verify) {
return withOpenedArtifactMcp(target, "open_bundle_resources", verify);
}
/** Require a successful structured MCP tool result rather than its text rendering. */
+24 -1
View File
@@ -94,7 +94,9 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
const cliArgs = normalizeCliArguments(args);
const outputArguments = validateCliOutputArguments(cliArgs);
const configurationError = cliConfigurationError(process.env);
if (configurationError !== undefined) {
// Doctor is the readiness report for this failure. Other commands still stop
// here so they do not run with an unusable log level.
if (configurationError !== undefined && !isDoctorCommand(cliArgs)) {
process.stderr.write(`${configurationError}\n`);
process.exitCode = 1;
} else if (!outputArguments.ok) {
@@ -135,6 +137,27 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
}
}
/** True when the first positional command is `doctor`, after global output flags. */
function isDoctorCommand(arguments_) {
const valuedFlags = new Set([
"--format",
"--filter-output",
"--token-limit",
"--token-offset",
]);
for (let index = 0; index < arguments_.length; index += 1) {
const argument = arguments_[index];
if (argument === "--") return arguments_[index + 1] === "doctor";
if (valuedFlags.has(argument)) {
index += 1;
continue;
}
if (argument.startsWith("-")) continue;
return argument === "doctor";
}
return false;
}
/**
* A reader that closes the pipe early, as `rea ... | head` does, has declined
* the rest of the output. Stop writing without reporting a failure, and exit
+31 -5
View File
@@ -6,6 +6,8 @@ import { fileURLToPath } from "node:url";
import { firstProcedureAddress } from "../dist/application/RealHopperAssertions.js";
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import { ArtifactResourceScope } from "../dist/artifacts/ArtifactResourceScope.js";
import { AnalysisCleanupLease } from "../dist/artifacts/AnalysisCleanupLease.js";
import { resolveHopperAnalysisProfile } from "../dist/hopper/HopperAnalysisProfile.js";
import { HopperApplicationLauncher } from "../dist/hopper/BridgeLauncher.js";
import { HopperClient } from "../dist/hopper/HopperClient.js";
@@ -26,6 +28,16 @@ const root = await realpath(
await mkdtemp(join(tmpdir(), "rea-hopper-deadlines-")),
);
const target = join(root, "deadline-fixture");
const profileResources = new ArtifactResourceScope();
const profileResourceScope = {
retain: (owner, resource) =>
profileResources.retain({
kind: "lease",
resource,
lease: AnalysisCleanupLease.forCloseOwner(owner, resource),
}),
run: (operation) => profileResources.run(operation),
};
let client;
let primaryFailure;
try {
@@ -39,6 +51,7 @@ try {
launcherPath,
loaderArgsOverride: [],
provider: HOPPER_PROVIDER_IDENTITY,
resourceScope: profileResourceScope,
});
if (!resolved.ok) throw resolved.error;
const profile = resolved.value.profile;
@@ -138,15 +151,28 @@ try {
primaryFailure = cause;
throw cause;
} finally {
let cleanupFailure;
if (client !== undefined) {
const closed = await client.close();
if (!closed.ok)
throw primaryFailure === undefined
? closed.error
if (!closed.ok) cleanupFailure = closed.error;
}
try {
await profileResources.close();
} catch (cause) {
cleanupFailure =
cleanupFailure === undefined
? cause
: new AggregateError(
[primaryFailure, closed.error],
"Hopper verification and cleanup failed",
[cleanupFailure, cause],
"Hopper profile and client cleanup failed",
);
}
await rm(root, { recursive: true, force: true });
if (cleanupFailure !== undefined)
throw primaryFailure === undefined
? cleanupFailure
: new AggregateError(
[primaryFailure, cleanupFailure],
"Hopper verification and cleanup failed",
);
}
+16 -1
View File
@@ -10,6 +10,7 @@ import {
artifactCli,
artifactMcpResult,
withArtifactMcp,
withBundleResourcesMcp,
} from "../../lib/artifact-e2e.mjs";
if (process.platform !== "darwin")
@@ -68,6 +69,9 @@ try {
const sha256 = createHash("sha256")
.update(await readFile(join(resources, "Assets.car")))
.digest("hex");
const executableSha256 = createHash("sha256")
.update(await readFile(join(app, "Contents", "MacOS", "Fixture")))
.digest("hex");
const cli = await artifactCli("inspect-asset-catalog", app);
assert.deepEqual(
cli.records.map(({ metadata }) => metadata),
@@ -80,7 +84,7 @@ try {
asset_name === "FixtureColor" && metadata.AssetType === "Color",
),
);
await withArtifactMcp(app, async (client) => {
await withBundleResourcesMcp(app, async (client) => {
assert.deepEqual(
await artifactMcpResult(client, "inspect_asset_catalog"),
cli,
@@ -99,6 +103,17 @@ try {
});
assert.equal(invalid.isError, true);
});
const { resource_root, target_sha256, ...resourcePayload } = cli;
assert.equal(resource_root.executable_sha256, null);
assert.equal(target_sha256, null);
await withArtifactMcp(app, async (client) => {
const binary = await artifactMcpResult(client, "inspect_asset_catalog");
assert.equal(binary.target_sha256, executableSha256);
assert.deepEqual(binary, {
...resourcePayload,
target_sha256: executableSha256,
});
});
report = {
ok: true,
mocked: false,
+3 -1
View File
@@ -30,7 +30,7 @@ const LINK_MODES = [
{
mode: "dyld-info",
flags: ["-Wl,-no_fixup_chains"],
fixups: "LC_DYLD_INFO rebase opcodes were not decoded",
fixups: "LC_DYLD_INFO rebase opcodes",
swift: true,
},
{ mode: "chained", flags: [], fixups: "chained fixups:", swift: true },
@@ -70,6 +70,7 @@ const inspect = async (path) => {
};
const checkObjc = (metadata, mode) => {
assert.equal(fixupCoverage(metadata)?.status, "complete");
const fixture = metadata.objc_classes.find(
({ name, is_meta_class: meta }) => name === "ReaDispatchFixture" && !meta,
);
@@ -159,6 +160,7 @@ const checkObjc = (metadata, mode) => {
};
const checkSwift = (metadata, mode) => {
assert.equal(fixupCoverage(metadata)?.status, "complete");
if (
!metadata.swift_dispatch_slots.some(
(slot) =>
+20 -2
View File
@@ -1,8 +1,9 @@
#!/usr/bin/env node
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { execFile } from "node:child_process";
import { mkdtemp, mkdir, rm, writeFile } from "node:fs/promises";
import { mkdtemp, mkdir, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
@@ -12,6 +13,7 @@ import {
artifactCliEvidence,
artifactMcpResult,
withArtifactMcp,
withBundleResourcesMcp,
} from "../../lib/artifact-e2e.mjs";
const exec = promisify(execFile);
@@ -49,7 +51,10 @@ try {
appPath,
);
const decoded = evidence.normalized_result;
await withArtifactMcp(appPath, async (client) => {
const executableSha256 = createHash("sha256")
.update(await readFile(executablePath))
.digest("hex");
await withBundleResourcesMcp(appPath, async (client) => {
const mcp = await artifactMcpResult(client, "decode_interface_builder");
assert.deepEqual(
mcp,
@@ -57,6 +62,19 @@ try {
"CLI and MCP compiled archive results differ",
);
});
const { resource_root, target_sha256, ...resourcePayload } = decoded;
assert.equal(resource_root.executable_sha256, null);
assert.equal(target_sha256, null);
await withArtifactMcp(appPath, async (client) => {
const binary = await artifactMcpResult(client, "decode_interface_builder");
assert.equal(binary.target_sha256, executableSha256);
assert.equal(binary.graph.target_sha256, executableSha256);
assert.deepEqual(binary, {
...resourcePayload,
target_sha256: executableSha256,
graph: { ...decoded.graph, target_sha256: executableSha256 },
});
});
assert.equal(decoded.documents.length, 1);
assert.equal(
decoded.documents[0]?.relative_path,
+5 -1
View File
@@ -140,8 +140,12 @@ const verifyParserConformance = async (app, trace) => {
join(app, image.path),
slice.architecture,
);
// otool -l prints LC_LAZY_LOAD_DYLIB_INFO without its linkedit install
// name, so this comparison cannot expect that edge. The structural
// reader tests own the payload.
const edges = trace.edges.filter(
({ loader, architecture, root }) =>
({ loader, architecture, root, command }) =>
command !== "LC_LAZY_LOAD_DYLIB_INFO" &&
loader === image.path &&
architecture === slice.architecture &&
root ===
+66 -1
View File
@@ -6,6 +6,7 @@ import { toolContract } from "../../../dist/contracts/toolContracts.js";
import {
artifactCliEvidence,
withArtifactMcp,
withBundleResourcesMcp,
} from "../../lib/artifact-e2e.mjs";
/** Inspect the compiled bundle with real native utilities through both adapters. */
@@ -16,15 +17,79 @@ export async function verifyNativeProvenance(app, executable) {
const operations = [
["inspect_macho", []],
["inspect_signature", []],
["inspect_plist", []],
["list_architectures", []],
["demangle_swift", symbols],
];
const observations = [];
const resourcePlistCli = toolContract("inspect_plist").outputSchema.parse(
await artifactCliEvidence("inspect-plist", app),
);
assert.equal(resourcePlistCli.subject, null);
assert.equal(resourcePlistCli.raw_result, null);
await withBundleResourcesMcp(app, async (client) => {
const response = await client.callTool({
name: "inspect_plist",
arguments: {},
});
assert.notEqual(response.isError, true, JSON.stringify(response));
const mcp = toolContract("inspect_plist").outputSchema.parse(
response.structuredContent,
);
assert.equal(mcp.subject, null);
assert.equal(mcp.raw_result, null);
assert.deepEqual(mcp.normalized_result, resourcePlistCli.normalized_result);
assert.deepEqual(
JSON.parse(response.content.find((item) => item.type === "text").text),
response.structuredContent,
);
await client.ping();
observations.push({
operation: "inspect_plist_resource_root",
commands: resourcePlistCli.normalized_result.provenance.length,
cli_bytes: Buffer.byteLength(JSON.stringify(resourcePlistCli)),
mcp_bytes: Buffer.byteLength(JSON.stringify(response)),
});
});
const binaryPlistCli = toolContract("inspect_plist").outputSchema.parse(
await artifactCliEvidence("inspect-plist", executable),
);
assert.equal(binaryPlistCli.subject?.digest.sha256, sha256);
assert.equal(binaryPlistCli.raw_result, null);
assert.equal(binaryPlistCli.normalized_result.resource_root, undefined);
await withArtifactMcp(app, async (client) => {
const definitions = new Map(
(await client.listTools()).tools.map((tool) => [tool.name, tool]),
);
const plistResponse = await client.callTool(
{ name: "inspect_plist", arguments: {} },
{ toolDefinition: definitions.get("inspect_plist") },
);
assert.notEqual(plistResponse.isError, true, JSON.stringify(plistResponse));
const binaryPlistMcp = toolContract("inspect_plist").outputSchema.parse(
plistResponse.structuredContent,
);
assert.equal(binaryPlistMcp.subject?.digest.sha256, sha256);
assert.equal(binaryPlistMcp.raw_result, null);
assert.equal(binaryPlistMcp.normalized_result.resource_root, undefined);
assert.deepEqual(
binaryPlistMcp.normalized_result,
binaryPlistCli.normalized_result,
);
assert.deepEqual(
JSON.parse(
plistResponse.content.find((item) => item.type === "text").text,
),
plistResponse.structuredContent,
);
await client.ping();
observations.push({
operation: "inspect_plist_binary_bound",
commands: binaryPlistCli.normalized_result.provenance.length,
cli_bytes: Buffer.byteLength(JSON.stringify(binaryPlistCli)),
mcp_bytes: Buffer.byteLength(JSON.stringify(plistResponse)),
});
for (const [operation, arguments_] of operations) {
const cli = toolContract(operation).outputSchema.parse(
await artifactCliEvidence(
+37
View File
@@ -1,4 +1,5 @@
import { expect, it, vi } from "vitest";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import {
ExecFileOutputCancelledError,
execFileBytes,
@@ -183,6 +184,42 @@ it("does not infer cancellation from a foreign ABORT_ERR code", async () => {
});
});
it.each([true, false])(
"projects a truncated adb capture as retryable only for volatile output (%s)",
async (volatileOutput) => {
vi.mocked(execFileBytes).mockRejectedValueOnce(
Object.assign(new Error("too much"), {
code: 1,
stdout: "",
stderr: "",
signal: null,
killed: false,
outputTruncated: true,
}),
);
const result = await runAdbCapture(
volatileOutput ? "read_adb_logcat" : "install_adb_package",
{ device: "device" },
"/host/adb",
volatileOutput ? ["logcat", "-d"] : ["install", "app.apk"],
{ timeoutMs: 100, maxOutputBytes: 1024, volatileOutput },
undefined,
);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Expected a capture allowance refusal");
const projected = projectAnalysisError(result.error);
expect(projected.retryable).toBe(volatileOutput);
expect(projected.remediation.action).toContain(
"The adb output capture allowance",
);
expect(projected.remediation.action).toContain(
volatileOutput
? "live output may change between requests"
: "Repeating the unchanged request fails the same way.",
);
},
);
it("keeps a physical failure even when the later abort reason is that same error", async () => {
const controller = new AbortController();
const failure = new Error("physical command failure");
+15 -2
View File
@@ -9,6 +9,7 @@ import type {
AnalysisCapturedOutput,
AnalysisError,
} from "../domain/analysisErrorBase.js";
import { fixedAllowanceOptions } from "../domain/fixedAllowanceRemediation.js";
import { err, ok, type Result } from "../domain/result.js";
import {
execFileBytes,
@@ -86,6 +87,11 @@ export const adbArguments = (
export interface AdbRunLimits {
readonly timeoutMs: number;
readonly maxOutputBytes: number;
/**
* Live device observations can shrink between calls. Mutation commands omit
* this so a truncated result stays non-retryable.
*/
readonly volatileOutput?: boolean;
}
/** Keep the selected device separate from the file addressed by the command. */
@@ -139,7 +145,14 @@ export const runAdbCapture = async (
"memory",
`adb output exceeded the ${String(limits.maxOutputBytes)}-byte capture budget`,
{ max_output_bytes: limits.maxOutputBytes },
{ cause },
{
cause,
...fixedAllowanceOptions(
"The adb output capture allowance",
limits.volatileOutput === true ? "live" : "capture",
),
...(limits.volatileOutput === true ? { retryable: true } : {}),
},
),
);
const stderr = failure.stderr.trim();
@@ -268,7 +281,7 @@ export const probeAdbPathExistence = async (
{ device: serial, path },
binary,
adbArguments(serial, ["shell", "test", "-e", path]),
limits,
{ ...limits, volatileOutput: true },
signal,
);
if (probed.ok) {
+8 -1
View File
@@ -12,6 +12,7 @@ import {
} from "../process/ExecFileOutput.js";
import type { ProviderAvailability } from "../application/AnalysisProvider.js";
import type { ProviderRejectionCode } from "../contracts/providerSelection.js";
import { fixedAllowanceOptions } from "../domain/fixedAllowanceRemediation.js";
import type { JsonValue } from "../domain/jsonValue.js";
import { adbCommandCapturedOutput } from "./AdbCommand.js";
@@ -146,7 +147,13 @@ export const inspectAdbClient = async (
"memory",
`adb version output exceeded the ${String(VERSION_MAX_OUTPUT_BYTES)}-byte capture budget`,
{ max_output_bytes: VERSION_MAX_OUTPUT_BYTES },
options,
{
...options,
...fixedAllowanceOptions(
"The adb version-output capture allowance",
"capture",
),
},
);
if (failure?.code === "ETIMEDOUT")
throw new AnalysisTimeoutError(
+15 -3
View File
@@ -1326,7 +1326,7 @@ export class AdbProvider {
"serial" in request.input ? request.input.serial : undefined,
["shell", ...arguments_],
),
limits,
{ ...limits, volatileOutput: true },
signal,
);
switch (operation) {
@@ -1346,6 +1346,7 @@ export class AdbProvider {
{
timeoutMs: DEVICES_TIMEOUT_MS,
maxOutputBytes: DEVICES_MAX_OUTPUT_BYTES,
volatileOutput: true,
},
signal,
);
@@ -1382,6 +1383,7 @@ export class AdbProvider {
{
timeoutMs: SHELL_TIMEOUT_MS,
maxOutputBytes: GETPROP_MAX_OUTPUT_BYTES,
volatileOutput: true,
},
signal,
);
@@ -1631,6 +1633,7 @@ export class AdbProvider {
{
timeoutMs: LISTING_TIMEOUT_MS,
maxOutputBytes: LISTING_MAX_OUTPUT_BYTES,
volatileOutput: true,
},
signal,
);
@@ -1917,6 +1920,7 @@ export class AdbProvider {
{
timeoutMs: PM_PATH_TIMEOUT_MS,
maxOutputBytes: PM_PATH_MAX_OUTPUT_BYTES,
volatileOutput: true,
},
signal,
);
@@ -2704,7 +2708,11 @@ export class AdbProvider {
"android.intent.category.LAUNCHER",
input.package,
]),
{ timeoutMs: SHELL_TIMEOUT_MS, maxOutputBytes: PM_PATH_MAX_OUTPUT_BYTES },
{
timeoutMs: SHELL_TIMEOUT_MS,
maxOutputBytes: PM_PATH_MAX_OUTPUT_BYTES,
volatileOutput: true,
},
signal,
);
if (!resolved.ok) return resolved;
@@ -2918,7 +2926,11 @@ export class AdbProvider {
{ device: input.serial },
selection.binary,
adbArguments(input.serial, ["exec-out", "screencap", "-p"]),
{ timeoutMs: SCREEN_TIMEOUT_MS, maxOutputBytes: SCREEN_MAX_OUTPUT_BYTES },
{
timeoutMs: SCREEN_TIMEOUT_MS,
maxOutputBytes: SCREEN_MAX_OUTPUT_BYTES,
volatileOutput: true,
},
signal,
);
if (!captured.ok) return captured;
+5
View File
@@ -45,6 +45,7 @@ import {
AnalysisUnsupportedTargetError,
} from "../domain/analysisErrorCore.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { fixedAllowanceOptions } from "../domain/fixedAllowanceRemediation.js";
import { err, ok, type Result } from "../domain/result.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
@@ -194,6 +195,10 @@ const apktoolCommandFailure = (
{
cause,
...(capturedOutput === undefined ? {} : { capturedOutput }),
...fixedAllowanceOptions(
"The apktool output capture allowance",
"capture",
),
},
);
// A physical launcher/capture failure does not establish target refusal.
+30 -18
View File
@@ -68,7 +68,11 @@ export interface AnalysisExecution {
*/
readonly referencesTruncated?: boolean;
readonly locations: readonly EvidenceLocation[];
readonly subject: EvidenceSubjectTarget | null;
/**
* Missing means the selected operation's default subject should be bound by
* its caller. Explicit null means the observation has no file subject.
*/
readonly subject?: EvidenceSubjectTarget | null;
}
interface CreateAnalysisExecutionOptions {
@@ -79,7 +83,7 @@ interface CreateAnalysisExecutionOptions {
readonly relationshipCoverage?: RelationshipCoverage;
readonly referencesTruncated?: boolean;
readonly locations?: readonly EvidenceLocation[];
readonly subject?: EvidenceSubjectTarget;
readonly subject?: EvidenceSubjectTarget | null;
}
/**
@@ -129,7 +133,7 @@ export function createAnalysisExecution(
? {}
: { referencesTruncated: options.referencesTruncated }),
locations: [...(options.locations ?? [])],
subject: options.subject ?? null,
...(options.subject === undefined ? {} : { subject: options.subject }),
};
}
@@ -143,21 +147,27 @@ export const createAnalysisExecutionEvidence = (
execution: AnalysisExecution,
confidence: "observed" | "derived",
): Evidence =>
createEvidence(execution.subject ?? undefined, execution.provider, {
operation,
parameters: jsonObjectSchema.parse(input),
result: execution.result,
rawResult: execution.rawResult,
limitations: execution.limitations,
...(execution.analysisProfile === undefined
? {}
: { analysisProfile: execution.analysisProfile }),
...(execution.relationshipCoverage === undefined
? {}
: { relationshipCoverage: execution.relationshipCoverage }),
locations: execution.locations,
confidence,
});
createEvidence(
execution.subject === undefined
? undefined
: (execution.subject ?? undefined),
execution.provider,
{
operation,
parameters: jsonObjectSchema.parse(input),
result: execution.result,
rawResult: execution.rawResult,
limitations: execution.limitations,
...(execution.analysisProfile === undefined
? {}
: { analysisProfile: execution.analysisProfile }),
...(execution.relationshipCoverage === undefined
? {}
: { relationshipCoverage: execution.relationshipCoverage }),
locations: execution.locations,
confidence,
},
);
/** Provider-neutral application capability for one analysis operation. */
export interface AnalysisOperationPort {
@@ -206,6 +216,8 @@ export interface AnalysisClient extends AnalysisOperationPort {
close(
options?: Pick<ExecutionOptions, "progress"> & {
readonly retainDocument?: boolean;
/** Report each owned cleanup failure as soon as its owner observes it. */
readonly onCleanupFailure?: (failure: AnalysisError) => void;
},
): Promise<Result<null, AnalysisError>>;
}
+58
View File
@@ -14,6 +14,7 @@ import {
import { decodeXmlPlistText } from "../domain/propertyListXmlText.js";
import { hasBinaryPlistSignature } from "../domain/apple/appleArchiveFormat.js";
import { err, ok, type Result } from "../domain/result.js";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
/** Decoded representation bound for one binary Info.plist. */
const BINARY_PLIST_DECODE_BYTES = 16 * 1024 * 1024;
@@ -42,6 +43,18 @@ export interface ResolvedAppBundle {
readonly infoPlist?: string;
}
/** Resource-only identity for an app directory; it is not a BinaryTarget. */
export interface ResolvedAppBundleResourceRoot {
/** Path selected by the caller, retained even when it is a symlink alias. */
readonly callerPath: string;
/** Canonical directory whose resources are inspected. */
readonly canonicalPath: string;
readonly infoPlistPath: string;
readonly layout: "contents" | "root";
/** No executable was opened or hashed by resource-root discovery. */
readonly executableSha256: string | null;
}
/** Filesystem operations used by the resolver, injectable for boundary tests. */
export interface AppBundleFileSystem {
readonly readFile: (path: string) => Promise<Buffer>;
@@ -90,6 +103,51 @@ export const resolveAppBundleExecutable = async (
);
};
/**
* Resolve only the root and Info.plist layout needed for read-only resource
* inspection. This intentionally does not require or guess CFBundleExecutable.
*/
export const resolveAppBundleResourceRoot = async (
path: string,
options: { readonly cwd?: string } = {},
): Promise<Result<ResolvedAppBundleResourceRoot, BinaryTargetError>> => {
const callerPath = absoluteCallerPath(
path,
options.cwd === undefined ? {} : { cwd: options.cwd },
);
try {
const canonicalInput = await realpath(callerPath);
const directLayout = await presentLayout(canonicalInput);
const wrapped =
directLayout === undefined
? await wrappedBundle(canonicalInput)
: undefined;
const root = wrapped ?? canonicalInput;
const layout = directLayout ?? (await presentLayout(root));
if (layout === undefined)
return err(
new BinaryTargetError(
callerPath,
`app resource root has no supported Info.plist layout: ${callerPath}`,
),
);
const canonicalPath = await realpath(root);
return ok({
callerPath,
canonicalPath,
infoPlistPath: join(canonicalPath, ...layout.plist),
layout: layout.plist[0] === "Contents" ? "contents" : "root",
executableSha256: null,
});
} catch (cause: unknown) {
return err(
new BinaryTargetError(callerPath, errorReason(cause, callerPath), {
cause,
}),
);
}
};
const presentLayout = async (
bundle: string,
): Promise<BundleLayout | undefined> => {
@@ -0,0 +1,101 @@
import { mkdir, mkdtemp, rm, symlink, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { expect, it } from "vitest";
import {
createAnalysisExecution,
createAnalysisExecutionEvidence,
} from "./AnalysisProvider.js";
import { resolveAppBundleResourceRoot } from "./AppBundleExecutable.js";
import { bindExecutionTarget } from "./binary/BinarySessionExecution.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
const provider = { id: "test", name: "test", version: null } as const;
const target = {
path: "/Fixture.app/Contents/Info.plist",
sha256: "a".repeat(64),
kind: "artifact",
format: "plist",
} satisfies BinaryTarget;
it("resolves resource roots without requiring or hashing CFBundleExecutable", async () => {
const temp = await mkdtemp(join(tmpdir(), "rea-app-resource-root-"));
try {
const bundle = join(temp, "Fixture.app");
const contents = join(bundle, "Contents");
const alias = join(temp, "Selected.app");
await mkdir(contents, { recursive: true });
await writeFile(join(contents, "Info.plist"), "<plist><dict/></plist>");
await symlink(bundle, alias);
const resolved = await resolveAppBundleResourceRoot(alias);
expect(resolved).toEqual({
ok: true,
value: {
callerPath: alias,
canonicalPath: bundle,
infoPlistPath: join(bundle, "Contents", "Info.plist"),
layout: "contents",
executableSha256: null,
},
});
} finally {
await rm(temp, { recursive: true, force: true });
}
});
it("resolves a flat bundle resource root without inferring executable identity", async () => {
const temp = await mkdtemp(join(tmpdir(), "rea-ios-resource-root-"));
try {
const bundle = join(temp, "Fixture.app");
await mkdir(bundle);
await writeFile(join(bundle, "Info.plist"), "<plist><dict/></plist>");
const resolved = await resolveAppBundleResourceRoot(bundle);
expect(resolved).toMatchObject({
ok: true,
value: {
callerPath: bundle,
canonicalPath: bundle,
infoPlistPath: join(bundle, "Info.plist"),
layout: "root",
executableSha256: null,
},
});
} finally {
await rm(temp, { recursive: true, force: true });
}
});
it("distinguishes default target binding from an explicitly unbound observation", () => {
const defaultSubject = createAnalysisExecution({ ok: true }, provider);
const unbound = createAnalysisExecution({ ok: true }, provider, {
subject: null,
});
const bound = bindExecutionTarget(
{ ok: true, value: defaultSubject },
"inspect_plist",
target,
);
const preserved = bindExecutionTarget(
{ ok: true, value: unbound },
"inspect_plist",
target,
);
expect(bound.ok && bound.value.subject).toEqual({
path: target.path,
sha256: target.sha256,
format: "plist",
});
expect(preserved.ok && preserved.value.subject).toBeNull();
expect(
createAnalysisExecutionEvidence("inspect_plist", {}, unbound, "observed")
.subject,
).toBeNull();
});
+173 -12
View File
@@ -3,9 +3,19 @@ import type { BrowserObservationPort } from "./BrowserObservationPort.js";
import { listBrowserTargets } from "./BrowserObservationService.js";
import type { BrowserTargetList } from "../domain/browserObservationSchemas.js";
import { parseEvidence } from "../domain/evidence.js";
import { AnalysisOutputError } from "../domain/analysisErrorCore.js";
import {
AnalysisCancelledError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import {
expectDroppedPartial,
expectKeptPartial,
expectOutputError,
expectOwnerError,
} from "./ownerObservationExpectation.fixture.js";
import { BrowserObservationError } from "../domain/browserObservationError.js";
import { ok } from "../domain/result.js";
import { err, ok } from "../domain/result.js";
import { passiveNetworkCapture } from "../../tests/fixtures/passiveNetworkCapture.js";
import { inspectWebPage } from "./BrowserObservationService.js";
import { analyzeWebBundleInputSchema } from "../domain/webBundleAnalysis.js";
@@ -71,11 +81,7 @@ it("does not let late cancellation hide invalid completed browser output", async
provider(async () => ok(malformed)),
input,
);
expect(output).toMatchObject({
ok: false,
error: { _tag: "AnalysisOutputError" },
});
if (!output.ok) expect(output.error).toBeInstanceOf(AnalysisOutputError);
expectOutputError(output);
const controller = new AbortController();
const late = await listBrowserTargets(
@@ -86,11 +92,7 @@ it("does not let late cancellation hide invalid completed browser output", async
input,
{ signal: controller.signal },
);
expect(late).toMatchObject({
ok: false,
error: { _tag: "AnalysisOutputError" },
});
if (!late.ok) expect(late.error).toBeInstanceOf(AnalysisOutputError);
expectOutputError(late);
});
it("converts a raw page inspection retained after browser cleanup failure", async () => {
@@ -215,3 +217,162 @@ it("preserves a prior caller reason without admitting browser work", async () =>
expect(result.error._tag).toBe("AnalysisCancelledError");
expect(result.error.cause).toBe(reason);
});
it("keeps page cancellation when the retained inspection is incomplete", async () => {
const reason = { requested: "stop during page inspection" };
const partial = {
target: { target_id: "page" },
network: {},
capture_window: {},
};
const failure = new AnalysisCancelledError("inspect_web_page", {
cause: reason,
partialObservation: partial as never,
});
const browser = {
...provider(async () => ok(report)),
inspectPage: async () => err(failure),
} as unknown as BrowserObservationPort;
const pageInput = inspectWebPageInputSchema.parse({
cdp_endpoint: input.cdp_endpoint.recorded,
allowed_origins: [],
target_id: "page",
observation_ms: 0,
include_script_sources: false,
});
const result = await inspectWebPage(browser, pageInput);
expectKeptPartial(expectOwnerError(result, failure, reason), partial);
});
it("keeps bundle cancellation and an earlier failure when retained facts are incomplete", async () => {
const reason = { requested: "stop during bundle analysis" };
const partial = { observations: [], capture: { status: "partial" } };
const cancelled = new AnalysisCancelledError("analyze_web_bundle", {
cause: reason,
partialObservation: partial as never,
});
const timedOut = new BrowserObservationError(
"analyze_web_bundle",
"timeout",
{ partialObservation: partial as never },
);
const selected = analyzeWebBundleInputSchema.parse({
cdp_endpoint: input.cdp_endpoint.recorded,
target_id: "page",
observation_ms: 0,
});
for (const failure of [cancelled, timedOut]) {
const browser = {
...provider(async () => ok(report)),
analyzeBundle: async () => err(failure),
} as unknown as BrowserObservationPort;
const result = await analyzeWebBundle(browser, selected);
expectKeptPartial(
expectOwnerError(result, failure, failure.cause),
partial,
failure === timedOut ? "provider_timeout" : "cancelled",
);
}
expect(projectAnalysisError(cancelled).code).toBe("cancelled");
});
const pageInput = () =>
inspectWebPageInputSchema.parse({
cdp_endpoint: input.cdp_endpoint.recorded,
allowed_origins: [],
target_id: "page",
observation_ms: 0,
include_script_sources: false,
});
const bundleInput = () =>
analyzeWebBundleInputSchema.parse({
cdp_endpoint: input.cdp_endpoint.recorded,
target_id: "page",
observation_ms: 0,
});
const browserReturning = (
failure: BrowserObservationError | AnalysisCancelledError,
) =>
({
...provider(async () => ok(report)),
inspectPage: async () => err(failure),
analyzeBundle: async () => err(failure),
}) as unknown as BrowserObservationPort;
it.each([
{
name: "bundle bytes",
operation: "analyze_web_bundle" as const,
partial: { observations: [], capture: { bytes: Number.NaN } },
},
{
name: "bundle observations",
operation: "analyze_web_bundle" as const,
partial: { observations: [undefined], capture: {} },
},
{
name: "page requests",
operation: "inspect_web_page" as const,
partial: {
target: { target_id: "page" },
network: { requests: Number.NaN },
capture_window: {},
},
},
])(
"drops a non-JSON $name partial and keeps cancellation primary",
async ({ operation, partial }) => {
const reason = { requested: `stop ${operation}` };
const failure = new AnalysisCancelledError(operation, {
cause: reason,
partialObservation: partial as never,
});
const result =
operation === "inspect_web_page"
? await inspectWebPage(browserReturning(failure), pageInput())
: await analyzeWebBundle(browserReturning(failure), bundleInput());
const error = expectOwnerError(result, failure, reason);
expectDroppedPartial(
error,
"Browser provider retained a non-JSON incomplete observation.",
);
expect(projectAnalysisError(error).details).not.toHaveProperty(
"partial_observation",
);
},
);
it.each([
{
name: "schema-shaped page",
partial: {
target: { target_id: "page" },
network: { requests: Number.NaN },
capture_window: {},
},
},
{
name: "shapeless page",
partial: { requests: Number.NaN },
},
])(
"drops a non-JSON $name partial and keeps timeout primary",
async ({ partial }) => {
const failure = new BrowserObservationError("inspect_web_page", "timeout", {
partialObservation: partial as never,
});
const error = expectOwnerError(
await inspectWebPage(browserReturning(failure), pageInput()),
failure,
undefined,
);
expectDroppedPartial(
error,
"Browser provider retained a non-JSON incomplete observation.",
"provider_timeout",
);
expect(projectAnalysisError(error).category).toBe("timeout");
},
);
+41 -53
View File
@@ -25,9 +25,11 @@ import type { ExecutionOptions } from "./AnalysisProvider.js";
import type { BrowserObservationPort } from "./BrowserObservationPort.js";
import { createBrowserEvidence } from "./BrowserEvidence.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import { completeProviderEvidence } from "./ProviderOperationExecution.js";
import {
completeProviderEvidence,
keepIncompleteObservation,
} from "./ProviderOperationExecution.js";
import { webPageInspectionSchema } from "../domain/browserObservationSchemas.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
/** List browser targets within the endpoint and origin scope in the request. */
export const listBrowserTargets = async (
@@ -275,7 +277,10 @@ const browserEvidence = (
);
};
/** Retain the completed capture or bundle facts under their actual operation. */
const NON_JSON_BROWSER_OBSERVATION =
"Browser provider retained a non-JSON incomplete observation.";
/** Promote completed page or bundle facts; keep only a JSON-safe incomplete observation. */
const retainBrowserObservation = (
error: AnalysisError,
input: InspectWebPageInput | AnalyzeWebBundleInput,
@@ -291,59 +296,42 @@ const retainBrowserObservation = (
"capture" in partial
) {
const bundle = webBundleAnalysisSchema.safeParse(partial);
if (!bundle.success) {
const malformed = new AnalysisOutputError(
errorOperation,
"Browser provider retained malformed completed bundle facts.",
{ cause: bundle.error },
);
return err(
error.cleanupIncomplete
? analysisErrorWithCleanupFailure(
malformed,
error,
errorOperation,
).bindCleanupDependencies([malformed, error])
: malformed,
if (bundle.success) {
error.partialObservation = createBrowserEvidence(
"analyze_web_bundle",
input,
bundle.data,
provider.identity(),
);
return err(error);
}
error.partialObservation = createBrowserEvidence(
"analyze_web_bundle",
input,
bundle.data,
provider.identity(),
);
return err(error);
}
if (
partial === undefined ||
"evidence_id" in partial ||
!("target" in partial) ||
!("network" in partial) ||
!("capture_window" in partial)
)
return err(error);
const inspection = webPageInspectionSchema.safeParse(partial);
if (!inspection.success) {
if (error instanceof AnalysisCancelledError)
return err(
new AnalysisOutputError(
errorOperation,
"Browser provider retained a malformed completed page inspection.",
{ cause: inspection.error },
),
} else if (
partial !== undefined &&
!("evidence_id" in partial) &&
"target" in partial &&
"network" in partial &&
"capture_window" in partial
) {
const inspection = webPageInspectionSchema.safeParse(partial);
if (inspection.success) {
const pageInput: InspectWebPageInput =
"fetch_source_maps" in input
? (({ fetch_source_maps: _fetchSourceMaps, ...page }) => page)(input)
: input;
error.partialObservation = createBrowserEvidence(
"inspect_web_page",
pageInput,
inspection.data,
provider.identity(),
);
return err(error);
return err(error);
}
}
const pageInput: InspectWebPageInput =
"fetch_source_maps" in input
? (({ fetch_source_maps: _fetchSourceMaps, ...page }) => page)(input)
: input;
error.partialObservation = createBrowserEvidence(
"inspect_web_page",
pageInput,
inspection.data,
provider.identity(),
return err(
keepIncompleteObservation(
error,
errorOperation,
NON_JSON_BROWSER_OBSERVATION,
),
);
return err(error);
};
@@ -1,13 +1,26 @@
import { expect, it } from "vitest";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import { BrowserObservationError } from "../domain/browserObservationError.js";
import { parseEvidence } from "../domain/evidence.js";
import { browserScenarioInputSchema } from "../domain/browserScenario.js";
import { err } from "../domain/result.js";
import type {
BrowserScenarioCapture,
BrowserScenarioPartialObservation,
} from "../domain/browserScenarioCapture.js";
import { err, ok } from "../domain/result.js";
import { scriptScenarioFixture } from "../../tests/fixtures/webScriptCapture.js";
import type { BrowserScenarioCapturePort } from "./BrowserScenarioCapturePort.js";
import { captureBrowserScenario } from "./BrowserScenarioCaptureService.js";
import {
expectDroppedPartial,
expectKeptPartial,
expectOwnerError,
} from "./ownerObservationExpectation.fixture.js";
const scenario = browserScenarioInputSchema.parse({
browser: { mode: "launch", executable_path: "/fixture/chrome" },
@@ -15,6 +28,34 @@ const scenario = browserScenarioInputSchema.parse({
actions: [{ step_id: "settle", action: "wait_for_timeout", duration_ms: 1 }],
});
const capture = scriptScenarioFixture();
const scenarioIdentity = {
id: "scenario-fixture",
name: "Scenario fixture",
version: "1",
};
const runScenario = (
failure: AnalysisError,
id = { id: "fixture", name: "fixture", version: "1" },
) =>
captureBrowserScenario(
{
identity: () => id,
captureScenario: async () => err(failure),
},
scenario,
);
const expectRetainedCapture = (
result: Awaited<ReturnType<typeof runScenario>>,
failure: AnalysisError,
) => {
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Expected the retained capture");
expect(result.error).toBe(failure);
expect(parseEvidence(result.error.partialObservation)).toMatchObject({
operation: "capture_browser_scenario",
normalized_result: capture,
});
};
it("converts a retained raw scenario capture into partial Evidence", async () => {
const failure = new AnalysisCancelledError("capture_browser_scenario", {
@@ -23,23 +64,7 @@ it("converts a retained raw scenario capture into partial Evidence", async () =>
capture,
},
});
const provider: BrowserScenarioCapturePort = {
identity: () => ({
id: "scenario-fixture",
name: "Scenario fixture",
version: "1",
}),
captureScenario: async () => err(failure),
};
const result = await captureBrowserScenario(provider, scenario);
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.error).toBe(failure);
expect(parseEvidence(result.error.partialObservation)).toMatchObject({
operation: "capture_browser_scenario",
normalized_result: capture,
});
}
expectRetainedCapture(await runScenario(failure, scenarioIdentity), failure);
});
it("converts a raw scenario capture retained after cleanup failure", async () => {
@@ -57,24 +82,9 @@ it("converts a raw scenario capture retained after cleanup failure", async () =>
},
},
);
const provider: BrowserScenarioCapturePort = {
identity: () => ({
id: "scenario-fixture",
name: "Scenario fixture",
version: "1",
}),
captureScenario: async () => err(failure),
};
const result = await captureBrowserScenario(provider, scenario);
expect(result.ok).toBe(false);
if (!result.ok) {
expect(result.error).toBe(failure);
expect(result.error.cleanupIncomplete).toBe(true);
expect(parseEvidence(result.error.partialObservation)).toMatchObject({
operation: "capture_browser_scenario",
normalized_result: capture,
});
}
const result = await runScenario(failure, scenarioIdentity);
expectRetainedCapture(result, failure);
if (!result.ok) expect(result.error.cleanupIncomplete).toBe(true);
});
it.each([
@@ -103,3 +113,63 @@ it.each([
expect(calls).toBe(0);
},
);
it("keeps startup cancellation and its incomplete capture", async () => {
const reason = { requested: "stop during initial navigation" };
const completed = scriptScenarioFixture();
const {
completeness: _completeness,
steps: _steps,
browser,
...retained
} = completed;
const partial: BrowserScenarioPartialObservation = {
kind: "browser-scenario-observation",
capture: {
...retained,
browser: { ...browser, cleanup: "incomplete" },
steps: [],
},
};
const failure = new AnalysisCancelledError("capture_browser_scenario", {
cause: reason,
partialObservation: partial,
});
const result = await runScenario(failure);
expectKeptPartial(expectOwnerError(result, failure, reason), partial);
});
it.each([{ steps: [undefined] }, { started_at: Number.NaN }])(
"drops a non-JSON incomplete scenario capture and keeps cancellation primary",
async (capturePartial) => {
const reason = { requested: "stop during initial navigation" };
const partial: BrowserScenarioPartialObservation = {
kind: "browser-scenario-observation",
capture: capturePartial as never,
};
const failure = new AnalysisCancelledError("capture_browser_scenario", {
cause: reason,
partialObservation: partial,
});
const result = await runScenario(failure);
expectDroppedPartial(
expectOwnerError(result, failure, reason),
"Browser scenario provider retained a non-JSON incomplete observation.",
);
},
);
it("rejects a successful capture that is not a completed scenario", async () => {
const provider: BrowserScenarioCapturePort = {
identity: () => ({ id: "fixture", name: "fixture", version: "1" }),
captureScenario: async () => ok({} as BrowserScenarioCapture),
};
const result = await captureBrowserScenario(provider, scenario);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Expected a completed-capture refusal");
expect(result.error).toBeInstanceOf(AnalysisOutputError);
expect(result.error.message).toContain(
"returned a malformed completed capture",
);
expect(result.error.partialObservation).toBeUndefined();
});
@@ -11,7 +11,10 @@ import type { Evidence } from "../domain/evidence.js";
import type { BrowserScenario } from "../domain/browserScenario.js";
import { err, type Result } from "../domain/result.js";
import { browserScenarioCaptureSchema } from "../domain/browserScenarioCapture.js";
import { completeProviderEvidence } from "./ProviderOperationExecution.js";
import {
completeProviderEvidence,
keepIncompleteObservation,
} from "./ProviderOperationExecution.js";
const OPERATION = "capture_browser_scenario" as const;
const PROVIDER_ID = "rea-playwright-browser-scenario";
@@ -43,6 +46,8 @@ export const captureBrowserScenario = async (
partial.kind === "browser-scenario-observation"
) {
const report = browserScenarioCaptureSchema.safeParse(partial.capture);
// Only a completed capture becomes Evidence. A JSON-safe incomplete
// observation stays on the owner error, including typed cancellation.
if (
report.success &&
report.data.steps.every(({ status }) => status === "completed")
@@ -52,19 +57,16 @@ export const captureBrowserScenario = async (
report.data,
provider.identity(),
);
} else if (!report.success) {
if (captured.error instanceof AnalysisCancelledError)
return err(
new AnalysisOutputError(
OPERATION,
"Browser scenario provider retained a malformed completed capture.",
{ cause: report.error },
),
);
return captured;
}
}
return captured;
return err(
keepIncompleteObservation(
captured.error,
OPERATION,
"Browser scenario provider retained a non-JSON incomplete observation.",
),
);
}
const report = browserScenarioCaptureSchema.safeParse(captured.value);
if (!report.success)
+1 -1
View File
@@ -478,7 +478,7 @@ const workflowAvailabilityFor = ({
: {
reason: "provider_missing",
remediation:
"On Linux, supply TShark 4.x via absolute REA_TSHARK_COMMAND and util-linux prlimit (REA_TSHARK_PRLIMIT_COMMAND, default /usr/bin/prlimit). No active binary target or hardware is required; the tool is checked at execution.",
"On Linux, supply TShark 4.x on PATH or through absolute REA_TSHARK_COMMAND, and util-linux prlimit (REA_TSHARK_PRLIMIT_COMMAND, default /usr/bin/prlimit). No active binary target or hardware is required; the tool is checked at execution.",
};
if (name === "decode_signal_recording" || name === "inspect_signal_decoders")
return policy.signalRecordingEnabled === true
@@ -74,7 +74,10 @@ export const retryClientConfigurationCleanup = async (
if (retried.cleanup !== undefined) observations.push(retried.cleanup);
primary = retried.primary;
const last = retried.failures.at(-1)!;
nested = last.outcome.kind === "failed" ? last.outcome.cause : undefined;
nested =
last.operationFailure === undefined && last.outcome.kind === "failed"
? last.outcome.cause
: undefined;
} else {
const released = await resources.release({
kind: "lease",
+42 -4
View File
@@ -24,10 +24,14 @@ import {
} from "../domain/evidenceErrors.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { access } from "node:fs/promises";
import { access, stat } from "node:fs/promises";
import { absoluteCallerPath } from "../filesystem/callerPath.js";
import { readAnalysisSnapshot } from "./binary/AnalysisSnapshotFiles.js";
import { requiresNativeSourceAdmissionBeforeReplay } from "./binary/AnalysisSnapshotCache.js";
import { parseBinaryTarget } from "./BinaryTargetResolver.js";
import {
classifyBinaryTargetFilesystemFailure,
parseBinaryTarget,
} from "./BinaryTargetResolver.js";
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
snapshotEvidenceForQuery,
@@ -56,11 +60,13 @@ import type { AnalysisProviderSelector } from "../contracts/providerSelection.js
import { artifactInspectionResultSchema } from "../domain/artifactInspection.js";
import { resolveXrefsAddress } from "./XrefsAddressResolution.js";
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
import type { BinaryTargetError } from "../domain/configurationErrors.js";
import {
directAnalysisCleanupFailure,
managedAnalysisCleanupFailure,
withSessionCleanup,
} from "./DirectAnalysisCleanup.js";
import { BUNDLE_RESOURCE_OPERATIONS } from "./binary/BundleResourceSession.js";
type DirectAnalysisTool =
| "annotate_native_function"
@@ -97,7 +103,6 @@ const NATIVE_INSPECTION_OPERATIONS: ReadonlySet<string> = new Set([
const ARTIFACT_ANALYSIS_OPERATION_SET: ReadonlySet<string> = new Set(
ARTIFACT_ANALYSIS_OPERATIONS,
);
/**
* Open one binary, execute one tool, and always release provider resources.
* Unlike MCP mode, every CLI invocation is intentionally isolated and does not
@@ -249,6 +254,24 @@ const runAnalysis = async (
return withSessionCleanup(
session,
async () => {
let isDirectory = false;
if (BUNDLE_RESOURCE_OPERATIONS.has(tool)) {
const inspected = await inspectBundleResourceInput(path);
if (!inspected.ok) return cliError(inspected.error);
isDirectory = inspected.value;
}
if (BUNDLE_RESOURCE_OPERATIONS.has(tool) && isDirectory) {
const opened = await session.openBundleResources(path, { signal });
if (!opened.ok) return cliError(opened.error);
const execution = await session.execute(tool, arguments_, { signal });
if (!execution.ok) return cliError(execution.error);
return createAnalysisExecutionEvidence(
tool,
arguments_,
execution.value,
"observed",
);
}
const prepared = await prepareSnapshot({
path,
snapshotPath,
@@ -493,7 +516,9 @@ const executeAnalysisTool = async (input: {
}
if (!result.ok) return { output: cliError(result.error) };
const evidence = createEvidence(
result.value.subject ?? input.openedTarget,
result.value.subject === undefined
? input.openedTarget
: (result.value.subject ?? undefined),
result.value.provider,
{
operation: tool,
@@ -577,6 +602,19 @@ const prepareSnapshot = async (options: {
return ok({ snapshot: loaded.value, target: target.value });
};
const inspectBundleResourceInput = async (
path: string,
): Promise<Result<boolean, BinaryTargetError>> => {
const callerPath = absoluteCallerPath(path);
try {
return ok((await stat(callerPath)).isDirectory());
} catch (cause: unknown) {
const failure = classifyBinaryTargetFilesystemFailure(callerPath, cause);
if (failure !== undefined) return err(failure);
throw cause;
}
};
const allowsSnapshotReplay = (
route: SessionProviderRoute,
tool:
@@ -0,0 +1,117 @@
import { afterEach, expect, it, vi } from "vitest";
const statFailures = vi.hoisted(() => new Map<string, unknown>());
vi.mock("node:fs/promises", async (importOriginal) => {
const original = await importOriginal<typeof import("node:fs/promises")>();
return {
...original,
stat: vi.fn(async (...args: Parameters<typeof original.stat>) => {
const failure = statFailures.get(String(args[0]));
if (failure !== undefined) throw failure;
return original.stat(...args);
}),
};
});
import { parseConfig } from "../config/parseConfig.js";
import { classifyBinaryTargetFilesystemFailure } from "./BinaryTargetResolver.js";
import type { BinarySession } from "./binary/BinarySession.js";
import type { DirectAnalysisDependencies } from "./DirectAnalysisDependencies.js";
import { runProviderAnalysis } from "./DirectAnalysis.js";
afterEach(() => statFailures.clear());
it("retains the original filesystem cause on the shared typed error", () => {
const cause = Object.assign(new Error("permission denied"), {
code: "EACCES",
});
const failure = classifyBinaryTargetFilesystemFailure(
"/tmp/rea-stat-failure-cause.app",
cause,
);
expect(failure?.cause).toBe(cause);
expect(failure?.systemCode).toBe("EACCES");
});
it.each([
{ code: "ENOENT", projectedCode: "target_unavailable" },
{ code: "ENOTDIR", projectedCode: "target_unavailable" },
{ code: "EACCES", projectedCode: "access_denied", systemCode: "EACCES" },
{ code: "EPERM", projectedCode: "access_denied", systemCode: "EPERM" },
{ code: "EIO", projectedCode: "target_unavailable" },
])(
"stops CLI resource dispatch on stat failure $code and preserves its path",
async ({ code, projectedCode, systemCode }) => {
const path = `/tmp/rea-stat-failure-${code}.app`;
const cause = Object.assign(new Error(`${code}: injected stat failure`), {
code,
});
statFailures.set(path, cause);
const session = {
close: vi.fn(async () => ({ ok: true as const, value: null })),
open: vi.fn(),
openBundleResources: vi.fn(),
execute: vi.fn(),
} as unknown as BinarySession;
const dependencies: DirectAnalysisDependencies = {
readConfiguration: () => parseConfig({}),
createBinarySession: () => session,
createNativeDispatchBinarySession: () => session,
createArtifactBinarySession: () => session,
createNativeBinarySession: () => session,
createManagedBinarySession: () => session,
};
const output = await runProviderAnalysis(
dependencies,
path,
"inspect_keyed_archive",
{ path: "Resources/Archive.plist" },
);
expect(output).toMatchObject({
code: projectedCode,
details: {
path,
reason: expect.stringContaining(cause.message),
...(systemCode === undefined ? {} : { system_code: systemCode }),
},
});
expect(session.open).not.toHaveBeenCalled();
expect(session.openBundleResources).not.toHaveBeenCalled();
expect(session.execute).not.toHaveBeenCalled();
expect(session.close).toHaveBeenCalledOnce();
},
);
it("rethrows an unclassified stat failure instead of treating it as a file", async () => {
const path = "/tmp/rea-stat-failure-opaque.app";
const cause = new Error("opaque filesystem failure");
statFailures.set(path, cause);
const session = {
close: vi.fn(async () => ({ ok: true as const, value: null })),
open: vi.fn(),
openBundleResources: vi.fn(),
execute: vi.fn(),
} as unknown as BinarySession;
const dependencies: DirectAnalysisDependencies = {
readConfiguration: () => parseConfig({}),
createBinarySession: () => session,
createNativeDispatchBinarySession: () => session,
createArtifactBinarySession: () => session,
createNativeBinarySession: () => session,
createManagedBinarySession: () => session,
};
await expect(
runProviderAnalysis(dependencies, path, "inspect_keyed_archive", {
path: "Resources/Archive.plist",
}),
).rejects.toBe(cause);
expect(session.open).not.toHaveBeenCalled();
expect(session.openBundleResources).not.toHaveBeenCalled();
expect(session.execute).not.toHaveBeenCalled();
expect(session.close).toHaveBeenCalledOnce();
});
+268 -1
View File
@@ -89,7 +89,7 @@ describe("doctor Hopper startup timeout", () => {
},
);
it("keeps an invalid Hopper timeout informational outside the Hopper scope", async () => {
it("keeps an invalid Hopper timeout informational when no provider is selected", async () => {
const selectedHost = timeoutHost("0");
const result = await runDoctor(undefined, selectedHost, { providers: [] });
expect(result.healthy).toBe(true);
@@ -100,6 +100,273 @@ describe("doctor Hopper startup timeout", () => {
}),
);
});
it("requires an invalid Hopper timeout for a ready non-Hopper provider", async () => {
const ghidra = {
providerInspections: () =>
Promise.resolve([
{
id: "ghidra" as const,
configured: true,
available: true,
providerVersion: "12.1.4",
registrationEnvironment: { GHIDRA_INSTALL_DIR: "/tools/ghidra" },
checks: [
{
name: "configuration",
ok: true,
code: null,
detail: "/tools/ghidra",
remediation: null,
classification: "config_drift" as const,
},
],
},
]),
};
const blocked = await runDoctor(undefined, timeoutHost("abc", ghidra), {
providers: ["ghidra"],
});
expect(blocked.healthy).toBe(false);
expect(blocked.scope_checks).toContainEqual(
expect.objectContaining({
name: "hopper-startup-timeout",
ok: false,
classification: "config_drift",
}),
);
expect(
blocked.scope_checks.filter(
({ name }) => name === "hopper-startup-timeout",
),
).toHaveLength(1);
const ready = await runDoctor(undefined, timeoutHost("120000", ghidra), {
providers: ["ghidra"],
});
expect(ready.healthy).toBe(true);
expect(ready.scope_checks).toContainEqual(
expect.objectContaining({ name: "hopper-startup-timeout", ok: true }),
);
});
});
describe("doctor native analysis configuration", () => {
const configurationHost = (
environment: Readonly<NodeJS.ProcessEnv>,
): DoctorHost => ({
...systemDoctorHost({ environment: { ...environment } }),
...host({
installationPaths: () => Promise.resolve([]),
clientRegistrations: () => Promise.resolve([]),
}),
});
it.each([
["REA_ANALYSIS_PROVIDER", "Hopper"],
["REA_GHIDRA_STARTUP_TIMEOUT_MS", "abc"],
["REA_SWIFTC_COMPILE_TIMEOUT_MS", "0"],
["REA_LOG_LEVEL", "verbose"],
["REA_GHIDRA_COMPILER_SPEC_ID", "gcc"],
] as const)(
"rejects %s before scoped Hopper readiness reports success",
async (setting, value) => {
const result = await runDoctor(
undefined,
configurationHost({ [setting]: value }),
{ providers: ["hopper"] },
);
expect(result.healthy).toBe(false);
expect(result.environment_healthy).toBe(false);
expect(result.scope_checks).toContainEqual(
expect.objectContaining({
name: `runtime-configuration:${setting}`,
ok: false,
classification: "config_drift",
detail: expect.stringContaining(setting),
remediation: expect.stringContaining(setting),
details: { setting },
}),
);
},
);
it("keeps a lowercase provider preference ready for the selected engine", async () => {
const result = await runDoctor(
undefined,
configurationHost({ REA_ANALYSIS_PROVIDER: "hopper" }),
{ providers: ["hopper"] },
);
expect(result.healthy).toBe(true);
expect(
result.checks.some(({ name }) =>
name.startsWith("runtime-configuration:"),
),
).toBe(false);
});
it("keeps an invalid native setting informational when nothing starts MCP", async () => {
const result = await runDoctor(
undefined,
configurationHost({ REA_GHIDRA_STARTUP_TIMEOUT_MS: "abc" }),
{ providers: [] },
);
expect(result.healthy).toBe(true);
expect(result.environment_healthy).toBe(false);
expect(result.scope_checks.map(({ name }) => name)).not.toContain(
"runtime-configuration:REA_GHIDRA_STARTUP_TIMEOUT_MS",
);
expect(result.informational_checks).toContainEqual(
expect.objectContaining({
name: "runtime-configuration:REA_GHIDRA_STARTUP_TIMEOUT_MS",
ok: false,
}),
);
});
it("does not treat a JEB-only timeout as native analysis configuration", async () => {
const result = await runDoctor(
undefined,
configurationHost({ REA_JEB_MCP_TIMEOUT_MS: "abc" }),
{ providers: ["hopper"] },
);
expect(result.healthy).toBe(true);
expect(
result.checks.some(({ name }) =>
name.startsWith("runtime-configuration:"),
),
).toBe(false);
});
});
describe("doctor MCP startup readiness", () => {
const configurationHost = (
environment: Readonly<NodeJS.ProcessEnv>,
): DoctorHost => ({
...systemDoctorHost({ environment: { ...environment } }),
...host({
installationPaths: () => Promise.resolve([]),
clientRegistrations: () => Promise.resolve([]),
}),
});
const alignedCodex: ClientRegistrationStatus = {
client: "codex",
config_path: "/home/user/.codex/config.toml",
command: ["/current/rea", "mcp"],
state: "aligned",
remediation: null,
};
const selectedHost = (
environment: Readonly<NodeJS.ProcessEnv>,
): DoctorHost => ({
...configurationHost(environment),
clientRegistrations: () => Promise.resolve([alignedCodex]),
});
it("requires an invalid log level for a client and for the skill", async () => {
const client = await runDoctor(
undefined,
selectedHost({
REA_LOG_LEVEL: "verbose",
REA_GHIDRA_STARTUP_TIMEOUT_MS: "abc",
}),
{ clients: ["codex"] },
);
expect(client.healthy).toBe(false);
expect(client.scope_checks).toContainEqual(
expect.objectContaining({
name: "runtime-configuration:REA_LOG_LEVEL",
ok: false,
details: { setting: "REA_LOG_LEVEL" },
}),
);
expect(client.scope_checks).toContainEqual(
expect.objectContaining({
name: "runtime-configuration:REA_GHIDRA_STARTUP_TIMEOUT_MS",
ok: false,
}),
);
expect(client.scope_checks).toContainEqual(
expect.objectContaining({ name: "registration:codex", ok: true }),
);
const skill = await runDoctor(
undefined,
selectedHost({
REA_LOG_LEVEL: "verbose",
REA_GHIDRA_STARTUP_TIMEOUT_MS: "abc",
}),
{ skill: true },
);
expect(skill.healthy).toBe(false);
expect(skill.scope_checks).toContainEqual(
expect.objectContaining({
name: "runtime-configuration:REA_LOG_LEVEL",
ok: false,
}),
);
expect(skill.scope_checks.map(({ name }) => name)).not.toContain(
"runtime-configuration:REA_GHIDRA_STARTUP_TIMEOUT_MS",
);
});
it("requires every MCP startup failure for an aligned client", async () => {
const ghidraTimeout = await runDoctor(
undefined,
selectedHost({ REA_GHIDRA_STARTUP_TIMEOUT_MS: "abc" }),
{ clients: ["codex"] },
);
expect(ghidraTimeout.healthy).toBe(false);
expect(ghidraTimeout.scope_checks).toContainEqual(
expect.objectContaining({
name: "runtime-configuration:REA_GHIDRA_STARTUP_TIMEOUT_MS",
ok: false,
}),
);
expect(ghidraTimeout.scope_checks).toContainEqual(
expect.objectContaining({ name: "registration:codex", ok: true }),
);
const hopperTimeout = await runDoctor(
undefined,
selectedHost({ REA_HOPPER_STARTUP_TIMEOUT_MS: "abc" }),
{ clients: ["codex"] },
);
expect(hopperTimeout.healthy).toBe(false);
expect(
hopperTimeout.scope_checks.filter(
({ name }) => name === "hopper-startup-timeout",
),
).toHaveLength(1);
const hopperAndClient = await runDoctor(
undefined,
selectedHost({ REA_HOPPER_STARTUP_TIMEOUT_MS: "abc" }),
{ clients: ["codex"], providers: ["hopper"] },
);
expect(
hopperAndClient.scope_checks.filter(
({ name }) => name === "hopper-startup-timeout",
),
).toHaveLength(1);
const skill = await runDoctor(
undefined,
selectedHost({ REA_GHIDRA_STARTUP_TIMEOUT_MS: "abc" }),
{ skill: true },
);
expect(skill.healthy).toBe(true);
expect(skill.informational_checks).toContainEqual(
expect.objectContaining({
name: "runtime-configuration:REA_GHIDRA_STARTUP_TIMEOUT_MS",
ok: false,
}),
);
});
});
describe("doctor runtime check", () => {
+12
View File
@@ -11,7 +11,10 @@ import { constants } from "node:fs";
import { access, readdir, stat } from "node:fs/promises";
import { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import type { RetainedResourceScope } from "./AnalysisProvider.js";
import { nativeAnalysisConfigurationError } from "../config/parseConfig.js";
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
import type { ConfigurationSettingIssue } from "../domain/configurationErrors.js";
import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
import { join } from "node:path";
import { analysisErrorRemediationAction } from "../domain/analysisErrorPresentation.js";
@@ -109,6 +112,11 @@ export interface DoctorHost {
readonly homeDirectory: string;
readonly configuredHopperPath?: string;
readonly configuredHopperStartupTimeout?: string;
/**
* Settings `parseConfig` rejects before native analysis starts.
* Undefined means this host did not run that admission.
*/
readonly configurationIssues?: readonly ConfigurationSettingIssue[];
readonly configuredIlspyCmdPath?: string;
macosVersion(): Promise<string | undefined>;
linuxDistribution(): Promise<LinuxDistribution | undefined>;
@@ -508,6 +516,9 @@ export const systemDoctorHost = (
const platform = options.platform ?? process.platform;
const architecture = options.architecture ?? process.arch;
const environment = options.environment ?? process.env;
const configurationError = nativeAnalysisConfigurationError(
snapshotEnvironment(environment, platform),
);
const hostExecFileOutput = options.execFileOutput ?? execFileOutput;
const homeDirectory = homeDirectoryFromEnvironment(environment, platform);
const commandEnvironment = { env: environment };
@@ -548,6 +559,7 @@ export const systemDoctorHost = (
configuredHopperStartupTimeout:
environment.REA_HOPPER_STARTUP_TIMEOUT_MS,
}),
configurationIssues: configurationError?.settings ?? [],
...(environment.REA_ILSPY_CMD_PATH === undefined
? {}
: { configuredIlspyCmdPath: environment.REA_ILSPY_CMD_PATH }),
+39
View File
@@ -37,6 +37,7 @@ export const collectDoctorDiagnostics = async (
checks.push(hopperCheck(host, hopperPath));
const hopperTimeoutCheck = optionalHopperStartupTimeoutCheck(host);
if (hopperTimeoutCheck !== undefined) checks.push(hopperTimeoutCheck);
checks.push(...runtimeConfigurationChecks(host));
const hopperVersionCheck = await optionalHopperVersionCheck(host, hopperPath);
if (hopperVersionCheck !== undefined) checks.push(hopperVersionCheck);
@@ -230,6 +231,44 @@ const optionalHopperStartupTimeoutCheck = (
);
};
/**
* Project every native-analysis admission failure except the Hopper timeout,
* which already has its own check. Provider scopes include engine settings.
* Settings that block process start are marked for every doctor scope.
*/
const runtimeConfigurationChecks = (
host: DoctorHost,
): readonly DoctorCheck[] => {
const issues = host.configurationIssues;
if (issues === undefined || issues.length === 0) return [];
const grouped: { setting: string; constraints: string[] }[] = [];
for (const issue of issues) {
if (issue.setting === "REA_HOPPER_STARTUP_TIMEOUT_MS") continue;
const existing = grouped.find((item) => item.setting === issue.setting);
if (existing === undefined)
grouped.push({ setting: issue.setting, constraints: [issue.constraint] });
else existing.constraints.push(issue.constraint);
}
return grouped.map(({ setting, constraints }) => ({
...check(
`runtime-configuration:${setting}`,
false,
constraints
.map((constraint) =>
constraint.includes(setting)
? constraint.replace(/\.$/u, "")
: `${setting}: ${constraint}`,
)
.join("; "),
{
classification: "config_drift",
remediation: `Correct ${setting} in the environment that starts REA (your shell or the MCP client's registration), then rerun.`,
},
),
details: { setting },
}));
};
const optionalHopperVersionCheck = async (
host: DoctorHost,
hopperPath: string | undefined,
+27
View File
@@ -1,4 +1,5 @@
import { uniqueSorted } from "../domain/canonicalOrdering.js";
import { blocksProcessStart } from "../logger.js";
import type {
ClientRegistrationStatus,
UnhealthyClientRegistrationStatus,
@@ -80,8 +81,33 @@ export const scopeDoctorChecks = (
const hopperHostChecks = input.scope.providers.includes("hopper")
? input.checks.filter(({ name }) => name === "host")
: [];
// parseConfig rejects these before the MCP server starts. Hopper's own
// scope already includes hopper-startup-timeout through its provider checks.
const blocksMcpStartup = (check: DoctorCheck): boolean =>
check.name.startsWith("runtime-configuration:") ||
check.name === "hopper-startup-timeout";
const blocksProcess = (check: DoctorCheck): boolean => {
const setting = check.details?.setting;
return (
check.name.startsWith("runtime-configuration:") &&
typeof setting === "string" &&
blocksProcessStart(setting)
);
};
const exercisesMcpStartup =
input.scope.clients.length > 0 || input.scope.providers.length > 0;
const startupChecks = (
exercisesMcpStartup
? input.checks.filter(blocksMcpStartup)
: input.checks.filter(blocksProcess)
).filter(
(check) =>
check.name !== "hopper-startup-timeout" ||
!input.scope.providers.includes("hopper"),
);
const scopeChecks = [
...coreChecks,
...startupChecks,
...hopperHostChecks,
...selectedProviderChecks,
...clientChecks,
@@ -91,6 +117,7 @@ export const scopeDoctorChecks = (
return {
healthy:
coreChecks.every(({ ok }) => ok) &&
startupChecks.every(({ ok }) => ok) &&
hopperHostChecks.every(({ ok }) => ok) &&
clientChecks.every(({ ok }) => ok) &&
skillChecks.every(({ ok }) => ok) &&
@@ -0,0 +1,446 @@
import { describe, expect, it, vi } from "vitest";
import {
createAnalysisExecution,
type AnalysisOperation,
type AnalysisOperationPort,
} from "./AnalysisProvider.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import { EnhancedTools } from "./EnhancedTools.js";
import type { BinaryArchitecture } from "../domain/binaryTargetTypes.js";
import { objcSwiftMetadataSchema } from "../domain/native/objcSwiftMetadata.js";
import { err, ok } from "../domain/result.js";
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
import {
AnalysisOutputError,
AnalysisCancelledError,
} from "../domain/analysisErrorCore.js";
import { createAnalysisProfile } from "../domain/analysisProfile.js";
import type { MachoTargetSelection } from "../domain/apple/machoTargetSelection.js";
const sha256 = "a".repeat(64);
const provider = { id: "native", name: "Native", version: "1" };
const namesProvider = { id: "hopper", name: "Hopper", version: "2" };
const selection = {
kind: "macho-selected-image-v1" as const,
container: "thin" as const,
offset: 0,
byte_length: 4096,
cpu_type: 0x0100000c,
cpu_subtype: 0x80000002,
container_table_sha256: null,
};
const profile = createAnalysisProfile(namesProvider, {
target_selection: selection,
});
const subject = {
sha256,
path: "/fixture",
format: "mach-o" as const,
architecture: "arm64" as const,
};
const createPort = (
nativeOverrides: Partial<{
sha256: string;
architecture: BinaryArchitecture;
names: Array<{ address: string; value: string }>;
nameLimitations: string[];
metadataFailure: AnalysisError;
bindings: Array<{
requested_address: string;
symbol: string;
binding_source: "chained" | "bind" | "weak" | "lazy";
library_ordinal: number;
library: string | null;
weak: boolean;
addend: string;
}>;
targetSelection: MachoTargetSelection | null;
nameProfile: boolean;
}> = {},
): AnalysisOperationPort => ({
execute: async (operation: AnalysisOperation, parameters) => {
if (operation === "list_names")
return ok(
createAnalysisExecution(
nativeOverrides.names ?? [
{ address: "0x1000", value: "OBJC_CLASS_$_Document$GOT" },
{ address: "0x1000", value: "OBJC_CLASS_$_Document" },
],
namesProvider,
{
subject,
limitations: nativeOverrides.nameLimitations ?? [],
...(nativeOverrides.nameProfile === false
? {}
: { analysisProfile: profile }),
},
),
);
if (operation !== "inspect_native_dispatch_metadata")
return err(
new AnalysisCapabilityUnavailableError(
provider.id,
operation,
"Unavailable in fixture",
),
);
expect(parameters).toMatchObject({
max_records: 20_000,
objc_class_import_addresses: [
...new Set(
(
nativeOverrides.names ?? [
{ address: "0x1000", value: "OBJC_CLASS_$_Document$GOT" },
{ address: "0x1000", value: "OBJC_CLASS_$_Document" },
]
).map(({ address }) => address),
),
],
objc_class_import_addresses_complete: true,
});
if (nativeOverrides.metadataFailure !== undefined)
return err(nativeOverrides.metadataFailure);
const result = objcSwiftMetadataSchema.parse({ db_save_result: null });
if (nativeOverrides.targetSelection !== null)
result.target_selection = nativeOverrides.targetSelection ?? selection;
result.objc_class_import_bindings = nativeOverrides.bindings ?? [
{
requested_address: "0x1000",
symbol: "_OBJC_CLASS_$_Document",
binding_source: "chained",
library_ordinal: 1,
library: "/System/Library/Frameworks/AppKit.framework/AppKit",
weak: false,
addend: "0",
},
];
result.objc_class_import_binding_coverage = {
facet: "objc_class_import_bindings",
status: "complete",
reason: null,
examined: 1,
decoded: result.objc_class_import_bindings.length,
};
return ok(
createAnalysisExecution(
{
target_sha256: nativeOverrides.sha256 ?? sha256,
target_architecture: nativeOverrides.architecture ?? "arm64",
provider,
analysis_profile_digest: null,
result,
},
provider,
{
subject: {
...subject,
...(nativeOverrides.sha256 === undefined
? {}
: { sha256: nativeOverrides.sha256 }),
...(nativeOverrides.architecture === undefined
? {}
: { architecture: nativeOverrides.architecture }),
},
},
),
);
},
});
it.each([
["list_names", "omitted"],
["list_names", "unbound"],
["inspect_native_dispatch_metadata", "omitted"],
["inspect_native_dispatch_metadata", "unbound"],
] as const)(
"retains class candidates when %s subject is %s",
async (operation, state) => {
const port = createPort();
const tools = new EnhancedTools({
execute: async (...args) => {
const result = await port.execute(...args);
if (!result.ok || args[0] !== operation) return result;
const value = { ...result.value };
if (state === "omitted") delete value.subject;
else value.subject = null;
return ok(value);
},
});
await expect(tools.execute("get_objc_classes", {})).resolves.toMatchObject({
ok: true,
value: {
count: 2,
classes: [
{ address: "0x1000", name: "OBJC_CLASS_$_Document$GOT" },
{ address: "0x1000", name: "OBJC_CLASS_$_Document" },
],
identity_coverage: { status: "unknown" },
},
});
},
);
describe("Objective-C class import identity composition", () => {
it("joins only exact same-target bind slots and preserves literal filter selection", async () => {
const tools = new EnhancedTools(createPort());
await expect(tools.execute("get_objc_classes", {})).resolves.toMatchObject({
ok: true,
value: {
count: 1,
classes: [
{
address: "0x1000",
name: "OBJC_CLASS_$_Document",
binding_identity: {
symbol: "_OBJC_CLASS_$_Document",
slot_addresses: ["0x1000"],
},
},
],
identity_coverage: { status: "complete", reason: null },
},
});
await expect(
tools.execute("get_objc_classes", { pattern: "$GOT" }),
).resolves.toMatchObject({
ok: true,
value: {
count: 1,
classes: [{ address: "0x1000", name: "OBJC_CLASS_$_Document$GOT" }],
},
});
});
it("keeps rows unjoined when the native observation has a different subject", async () => {
const tools = new EnhancedTools(createPort({ sha256: "b".repeat(64) }));
await expect(tools.execute("get_objc_classes", {})).resolves.toMatchObject({
ok: true,
value: {
count: 2,
classes: [
{ address: "0x1000", name: "OBJC_CLASS_$_Document$GOT" },
{ address: "0x1000", name: "OBJC_CLASS_$_Document" },
],
identity_coverage: { status: "unknown" },
},
});
});
it("keeps identity unknown when exact selected-image evidence is missing or mismatched", async () => {
const cases = [
{ nameProfile: false },
{ targetSelection: null },
{ targetSelection: { ...selection, cpu_type: 0x01000007 } },
{ targetSelection: { ...selection, cpu_subtype: 2 } },
{
targetSelection: {
...selection,
container: "fat" as const,
offset: 4096,
container_table_sha256: "b".repeat(64),
},
},
];
for (const overrides of cases) {
const tools = new EnhancedTools(createPort(overrides));
await expect(
tools.execute("get_objc_classes", {}),
).resolves.toMatchObject({
ok: true,
value: { count: 2, identity_coverage: { status: "unknown" } },
});
}
});
it("marks identity coverage partial when the selected names inventory is limited", async () => {
const tools = new EnhancedTools(
createPort({ nameLimitations: ["provider capped the name inventory"] }),
);
await expect(tools.execute("get_objc_classes", {})).resolves.toMatchObject({
ok: true,
value: {
count: 1,
identity_coverage: {
status: "partial",
reason: "list_names: provider capped the name inventory",
},
},
});
});
it("keeps names with unknown identity when binding inspection is unavailable", async () => {
const tools = new EnhancedTools(
createPort({
metadataFailure: new AnalysisCapabilityUnavailableError(
provider.id,
"inspect_native_dispatch_metadata",
"No static Mach-O reader",
),
}),
);
await expect(tools.execute("get_objc_classes", {})).resolves.toMatchObject({
ok: true,
value: {
count: 2,
identity_coverage: { status: "unknown" },
},
});
});
it("preserves non-capability binding failures", async () => {
const failure = new AnalysisOutputError(
"inspect_native_dispatch_metadata",
"fixture failed",
);
const tools = new EnhancedTools(createPort({ metadataFailure: failure }));
await expect(tools.execute("get_objc_classes", {})).resolves.toEqual({
ok: false,
error: failure,
});
});
});
it("preserves same-named rows whose exact import slots have different identities", async () => {
const addresses = ["0x1000", "0x2000", "0x3000", "0x4000"];
const name = "OBJC_CLASS_$_Document";
const tools = new EnhancedTools(
createPort({
names: [
...addresses.map((address) => ({ address, value: name })),
{ address: "0x1000", value: name },
],
bindings: [
{
requested_address: "0x1000",
symbol: "_OBJC_CLASS_$_Document",
binding_source: "chained",
library_ordinal: 1,
library: "/System/Library/Frameworks/AppKit.framework/AppKit",
weak: false,
addend: "0",
},
{
requested_address: "0x2000",
symbol: "_OBJC_CLASS_$_Document",
binding_source: "chained",
library_ordinal: 1,
library: "/System/Library/Frameworks/UIKit.framework/UIKit",
weak: false,
addend: "0",
},
{
requested_address: "0x3000",
symbol: "_OBJC_CLASS_$_Document",
binding_source: "chained",
library_ordinal: 1,
library: "/System/Library/Frameworks/AppKit.framework/AppKit",
weak: true,
addend: "0",
},
{
requested_address: "0x4000",
symbol: "_OBJC_CLASS_$_Document",
binding_source: "chained",
library_ordinal: 1,
library: "/System/Library/Frameworks/AppKit.framework/AppKit",
weak: false,
addend: "1",
},
],
}),
);
await expect(tools.execute("get_objc_classes", {})).resolves.toMatchObject({
ok: true,
value: {
count: 4,
classes: addresses.map((address, index) => ({
address,
name,
binding_identity: {
symbol: "_OBJC_CLASS_$_Document",
binding_source: "chained",
library_ordinal: 1,
library:
index === 1
? "/System/Library/Frameworks/UIKit.framework/UIKit"
: "/System/Library/Frameworks/AppKit.framework/AppKit",
weak: index === 2,
addend: index === 3 ? "1" : "0",
slot_addresses: [address],
},
})),
identity_coverage: { status: "complete", reason: null },
},
});
});
describe("EnhancedTools cancellation ownership", () => {
it("preserves the caller reason at admission", async () => {
const reason = { request: "cancel-before-observation" };
const controller = new AbortController();
controller.abort(reason);
const execute = vi.fn(createPort().execute);
const tools = new EnhancedTools({ execute });
const result = await tools.execute(
"get_objc_classes",
{},
controller.signal,
);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Pre-cancelled observation was accepted");
expect(result.error).toBeInstanceOf(AnalysisCancelledError);
expect(result.error.cause).toBe(reason);
expect(execute).not.toHaveBeenCalled();
});
it("preserves a post-success caller reason without replacing provider failures", async () => {
const reason = { request: "cancel-after-provider-success" };
const controller = new AbortController();
const successfulPort = createPort();
const successTools = new EnhancedTools({
execute: async (...args) => {
const result = await successfulPort.execute(...args);
if (args[0] === "list_names" && result.ok) controller.abort(reason);
return result;
},
});
const cancelled = await successTools.execute(
"get_objc_classes",
{},
controller.signal,
);
expect(cancelled.ok).toBe(false);
if (cancelled.ok)
throw new Error("Post-cancelled observation was accepted");
expect(cancelled.error).toBeInstanceOf(AnalysisCancelledError);
expect(cancelled.error.cause).toBe(reason);
const physicalFailure = new AnalysisOutputError(
"list_names",
"fixture read failed",
);
const failingController = new AbortController();
const failureTools = new EnhancedTools({
execute: async (...args) => {
if (args[0] === "list_names") {
failingController.abort(reason);
return err(physicalFailure);
}
return successfulPort.execute(...args);
},
});
const failed = await failureTools.execute(
"get_objc_classes",
{},
failingController.signal,
);
expect(failed).toEqual({ ok: false, error: physicalFailure });
});
});
+147 -7
View File
@@ -9,6 +9,7 @@ import type {
} from "./AnalysisProvider.js";
import {
enhancedInputSchemas,
MAX_ENHANCED_RECORDS,
type CallRelationshipDirection,
type EnhancedToolName,
} from "../contracts/enhancedInputs.js";
@@ -30,15 +31,24 @@ import {
import { err, ok, type Result } from "../domain/result.js";
import {
categorizeSwiftTypes,
discoverObjcClasses,
discoverObjcProtocols,
projectObjcClassCandidates,
selectObjcClassCandidates,
} from "../domain/symbolAnalysis.js";
import { inspectNativeDispatch } from "./native/NativeDispatchMetadataInspection.js";
import { traceNativeValues } from "./native/NativeValueTrace.js";
import { traceNativeUiAction } from "./native/NativeUiActionTrace.js";
import { jsonValueSchema, type JsonValue } from "../domain/jsonValue.js";
import { addressedValueSchema } from "../domain/native/nativeInventory.js";
import { nativeDispatchMetadataResultSchema } from "../domain/native/objcSwiftMetadata.js";
import { isMachOFormat } from "../domain/binaryTargetTypes.js";
import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
import {
machoTargetSelectionMatchesArchitecture,
machoTargetSelectionMatchesFormat,
profileMachoTargetSelection,
sameMachoTargetSelection,
} from "../domain/apple/machoTargetSelection.js";
import {
invalidEnhancedInput,
@@ -187,6 +197,8 @@ export class EnhancedTools {
return this.#inspectNativeDispatchMetadata(
call.input.max_records,
signal,
call.input.objc_class_import_addresses,
call.input.objc_class_import_addresses_complete,
);
case "get_objc_classes":
return this.#objcClasses(call.input.pattern, signal);
@@ -286,15 +298,137 @@ export class EnhancedTools {
}
async #objcClasses(pattern: string, signal?: AbortSignal): EnhancedResult {
const names = await this.#allAddressed("list_names", signal);
return names.ok ? ok(discoverObjcClasses(names.value, pattern)) : names;
const namesExecution = await this.#observe("list_names", {}, signal);
if (!namesExecution.ok) return namesExecution;
const names = this.#parseAddressed(
"list_names",
namesExecution.value.result,
);
if (!names.ok) return names;
const candidates = selectObjcClassCandidates(names.value, pattern);
if (candidates.length === 0)
return ok(
projectObjcClassCandidates(candidates, {
bindings: [],
coverage: { status: "not_applicable", reason: null },
}),
);
const addresses = [...new Set(candidates.map(({ address }) => address))];
const scannedAddresses = addresses.slice(0, MAX_ENHANCED_RECORDS);
const bindingsExecution = await this.#observe(
"inspect_native_dispatch_metadata",
{
max_records: MAX_ENHANCED_RECORDS,
objc_class_import_addresses: scannedAddresses,
objc_class_import_addresses_complete:
addresses.length <= MAX_ENHANCED_RECORDS,
},
signal,
);
if (!bindingsExecution.ok) {
if (bindingsExecution.error._tag === "AnalysisCancelledError")
return bindingsExecution;
if (bindingsExecution.error._tag !== "AnalysisCapabilityUnavailableError")
return bindingsExecution;
return ok(
projectObjcClassCandidates(candidates, {
bindings: [],
coverage: {
status: "unknown",
reason: `Native class-import binding evidence is unavailable (${bindingsExecution.error._tag}).`,
},
}),
);
}
const native = nativeDispatchMetadataResultSchema.safeParse(
bindingsExecution.value.result,
);
const nameSubject = namesExecution.value.subject;
const bindingSubject = bindingsExecution.value.subject;
const namesSelection = profileMachoTargetSelection(
namesExecution.value.analysisProfile,
);
const nativeSelection = native.success
? native.data.result.target_selection
: undefined;
if (
!native.success ||
nameSubject === null ||
nameSubject === undefined ||
bindingSubject === null ||
bindingSubject === undefined ||
nameSubject.sha256 !== bindingSubject.sha256 ||
nameSubject.architecture === undefined ||
bindingSubject.architecture === undefined ||
nameSubject.architecture !== bindingSubject.architecture ||
!isMachOFormat(nameSubject.format) ||
nameSubject.format !== bindingSubject.format ||
native.data.target_sha256 !== nameSubject.sha256 ||
native.data.target_architecture !== nameSubject.architecture ||
native.data.provider.id !== bindingsExecution.value.provider.id ||
namesExecution.value.analysisProfile?.provider.id !==
namesExecution.value.provider.id ||
nativeSelection === undefined ||
!machoTargetSelectionMatchesArchitecture(
nativeSelection,
nameSubject.architecture,
) ||
!machoTargetSelectionMatchesFormat(nativeSelection, nameSubject.format) ||
!sameMachoTargetSelection(namesSelection, nativeSelection) ||
native.data.result.objc_class_import_bindings === undefined ||
native.data.result.objc_class_import_binding_coverage === undefined
)
return ok(
projectObjcClassCandidates(candidates, {
bindings: [],
coverage: {
status: "unknown",
reason:
"Native class-import facts did not prove matching artifact, architecture, and complete producer identity.",
},
}),
);
const coverage = native.data.result.objc_class_import_binding_coverage;
const inventoryLimitations = namesExecution.value.limitations;
const inventoryIncomplete = inventoryLimitations.length > 0;
const coverageReason = [
coverage.reason,
...inventoryLimitations.map((limitation) => `list_names: ${limitation}`),
]
.filter((reason): reason is string => reason !== null)
.join("; ");
return ok(
projectObjcClassCandidates(candidates, {
bindings: native.data.result.objc_class_import_bindings,
coverage: {
status:
coverage.status === "not_requested"
? "unknown"
: coverage.status === "complete" && inventoryIncomplete
? "partial"
: coverage.status,
reason: coverageReason.length === 0 ? null : coverageReason,
},
}),
);
}
#inspectNativeDispatchMetadata(
maxRecords: number,
signal?: AbortSignal,
objcClassImportAddresses?: readonly string[],
objcClassImportAddressesComplete?: boolean,
): EnhancedResult {
return inspectNativeDispatch(this.analysis, maxRecords, signal);
return inspectNativeDispatch(
this.analysis,
maxRecords,
signal,
objcClassImportAddresses,
objcClassImportAddressesComplete,
);
}
async #objcProtocols(signal?: AbortSignal): EnhancedResult {
@@ -677,7 +811,11 @@ export class EnhancedTools {
) {
const result = await this.#call(tool, {}, signal);
if (!result.ok) return result;
const parsed = z.array(addressedValueSchema).safeParse(result.value);
return this.#parseAddressed(tool, result.value);
}
#parseAddressed(tool: "list_names" | "list_procedures", value: JsonValue) {
const parsed = z.array(addressedValueSchema).safeParse(value);
return parsed.success
? ok(
parsed.data.map(({ address, value: name, objc_class_label }) => ({
@@ -715,13 +853,15 @@ export class EnhancedTools {
signal?: AbortSignal,
selectorOrigin?: ExecutionOptions["selectorOrigin"],
): Promise<Result<AnalysisExecution, AnalysisError>> {
if (isAborted(signal)) return err(new AnalysisCancelledError(name));
if (isAborted(signal))
return err(new AnalysisCancelledError(name, { cause: signal?.reason }));
const execution = await this.analysis.execute(name, arguments_, {
...(signal === undefined ? {} : { signal }),
...(selectorOrigin === undefined ? {} : { selectorOrigin }),
});
if (!execution.ok) return execution;
if (isAborted(signal)) return err(new AnalysisCancelledError(name));
if (isAborted(signal))
return err(new AnalysisCancelledError(name, { cause: signal?.reason }));
return execution;
}
}
+20 -17
View File
@@ -22,7 +22,6 @@ import {
canonicalJsonParts,
} from "../domain/jsonSerialization.js";
import { readJsonFile, writeTextParts } from "./JsonFiles.js";
import { RegularFileCleanupFailure } from "./RegularFileRead.js";
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
type EvidenceReadFailure =
@@ -58,23 +57,27 @@ const readBundle = async (
resources?: ArtifactResourceScope,
): Promise<Result<EvidenceBundle, EvidenceReadFailure>> => {
if (signal?.aborted)
return err(new AnalysisCancelledError("import_evidence_bundle"));
const loaded = await readJsonFile(path, signal, resources).catch(
(cause: unknown) => {
// A failed close transfers a live handle owner; preserve that failure.
if (
signal?.aborted === true &&
!(cause instanceof RegularFileCleanupFailure)
)
return err(
new AnalysisCancelledError("import_evidence_bundle", { cause }),
);
throw cause;
},
);
if (!loaded.ok) return loaded;
return err(
new AnalysisCancelledError("import_evidence_bundle", {
cause: signal.reason,
}),
);
const loaded = await readJsonFile(path, signal, resources);
if (!loaded.ok) {
if (loaded.error instanceof AnalysisCancelledError)
return err(
new AnalysisCancelledError("import_evidence_bundle", {
cause: loaded.error.cause,
}),
);
return loaded;
}
if (signal?.aborted)
return err(new AnalysisCancelledError("import_evidence_bundle"));
return err(
new AnalysisCancelledError("import_evidence_bundle", {
cause: signal.reason,
}),
);
let bundle: EvidenceBundle;
try {
bundle = parseBundle(loaded.value);
+20 -14
View File
@@ -39,19 +39,25 @@ export const executeFunctionAnalysisEvidence = async (
if (!dossier.ok) return dossier;
const observation = execution.value;
return ok(
createEvidence(observation.subject ?? target, observation.provider, {
operation: "analyze_function",
parameters: arguments_,
result: dossier.value,
rawResult: observation.rawResult,
limitations: observation.limitations,
locations: observation.locations,
...(observation.relationshipCoverage === undefined
? {}
: { relationshipCoverage: observation.relationshipCoverage }),
...(observation.analysisProfile === undefined
? {}
: { analysisProfile: observation.analysisProfile }),
}),
createEvidence(
observation.subject === undefined
? target
: (observation.subject ?? undefined),
observation.provider,
{
operation: "analyze_function",
parameters: arguments_,
result: dossier.value,
rawResult: observation.rawResult,
limitations: observation.limitations,
locations: observation.locations,
...(observation.relationshipCoverage === undefined
? {}
: { relationshipCoverage: observation.relationshipCoverage }),
...(observation.analysisProfile === undefined
? {}
: { analysisProfile: observation.analysisProfile }),
},
),
);
};
+44 -3
View File
@@ -19,6 +19,8 @@ import {
} from "../domain/analysisErrorCore.js";
import { err, ok, type Result } from "../domain/result.js";
import {
JsonInputPhysicalReadFailure,
JsonInputCancellationFailure,
readJsonInputFile,
readJsonObjectFile,
type JsonObjectMemberReader,
@@ -46,7 +48,10 @@ export const readJsonFile = async (
signal?: AbortSignal,
resources?: ArtifactResourceScope,
): Promise<
Result<unknown, EvidenceFileError | AnalysisResourceConstraintError>
Result<
unknown,
EvidenceFileError | AnalysisResourceConstraintError | AnalysisCancelledError
>
> =>
readJsonFileInput(
path,
@@ -93,7 +98,10 @@ const readJsonFileInput = async <Value>(
>,
signal?: AbortSignal,
): Promise<
Result<Value, EvidenceFileError | AnalysisResourceConstraintError>
Result<
Value,
EvidenceFileError | AnalysisResourceConstraintError | AnalysisCancelledError
>
> => {
const requestedPath = absoluteCallerPath(path);
try {
@@ -116,7 +124,25 @@ const readJsonFileInput = async <Value>(
} catch (cause: unknown) {
if (cause instanceof RegularFileCleanupFailure) throw cause;
if (cause instanceof AnalysisResourceConstraintError) return err(cause);
if (signal?.aborted === true) throw cause;
if (cause instanceof JsonInputCancellationFailure)
return err(
new AnalysisCancelledError("read_evidence_file", {
cause: cause.reason,
}),
);
if (cause instanceof JsonInputPhysicalReadFailure)
return err(
new EvidenceFileError("read", "io", {
cause: cause.physicalCause,
path: requestedPath,
}),
);
if (isJsonReadCancellation(cause, signal))
return err(
new AnalysisCancelledError("read_evidence_file", {
cause: signal?.reason,
}),
);
return err(
new EvidenceFileError(
"read",
@@ -132,6 +158,21 @@ const readJsonFileInput = async <Value>(
}
};
const isJsonReadCancellation = (
cause: unknown,
signal?: AbortSignal,
): boolean => {
if (signal?.aborted !== true) return false;
if (cause === signal.reason) return true;
return (
cause instanceof Error &&
cause.name === "AbortError" &&
"code" in cause &&
cause.code === "ABORT_ERR" &&
cause.cause === signal.reason
);
};
/** Atomically write text to the caller-supplied path. */
export const writeTextFile = async (
encoded: string,
+223 -107
View File
@@ -1,5 +1,6 @@
import { constants } from "node:buffer";
import { createHash } from "node:crypto";
import type { Stats } from "node:fs";
import type { FileHandle } from "node:fs/promises";
import { pipeline } from "node:stream/promises";
import { getHeapStatistics } from "node:v8";
@@ -10,10 +11,17 @@ import parser from "stream-json/parser.js";
import streamValues from "stream-json/streamers/stream-values.js";
import streamArray from "stream-json/streamers/stream-array.js";
import { RegularFileChangedError } from "../filesystem/RegularFile.js";
import {
assertRegularFileActive,
RegularFileChangedError,
} from "../filesystem/RegularFile.js";
import type { ArtifactResourceScope } from "../artifacts/ArtifactResourceScope.js";
import { sealOwnedJsonInput } from "./OwnedJsonInput.js";
import { withRegularFile } from "./RegularFileRead.js";
import {
RegularFileReadFailure,
withRegularFile,
type RegularFileReadOptions,
} from "./RegularFileRead.js";
import {
JSON_BYTE_ORDER_MARK_MESSAGE,
JSON_INPUT_RESOURCE_REMEDIATION,
@@ -34,6 +42,74 @@ const PACKED_TOKEN_PART_CODE_UNITS = 64 * 1024;
// This selects an implementation; larger files remain accepted through streaming.
const NATIVE_PARSE_READ_BUDGET = 8 * 1024 * 1024;
/** Physical descriptor failure remains primary if cancellation arrives afterward. */
export class JsonInputPhysicalReadFailure extends Error {
readonly _tag = "JsonInputPhysicalReadFailure";
readonly physicalCause: unknown;
constructor(physicalCause: unknown) {
super("JSON input descriptor read failed");
this.physicalCause = physicalCause;
}
}
/** Explicit cancellation fact; the caller's value is opaque and retained as data. */
export class JsonInputCancellationFailure extends Error {
readonly _tag = "JsonInputCancellationFailure";
constructor(readonly reason: unknown) {
super(
reason instanceof Error ? reason.message : "JSON input read cancelled",
);
}
}
interface JsonInputPhysicalReadObservation {
failure?: JsonInputPhysicalReadFailure;
}
const readPhysicalJsonInput = async <Value>(
observation: JsonInputPhysicalReadObservation,
read: () => Promise<Value>,
): Promise<Value> => {
try {
return await read();
} catch (physicalCause: unknown) {
const failure = new JsonInputPhysicalReadFailure(physicalCause);
observation.failure = failure;
throw failure;
}
};
const withJsonInputFile = <Value>(
path: string,
read: (handle: FileHandle, stats: Stats) => Promise<Value>,
options: Pick<RegularFileReadOptions, "signal" | "resources">,
): Promise<Value> =>
withRegularFile(path, read, {
...options,
cancellationFailure: (reason) => new JsonInputCancellationFailure(reason),
}).catch((cause: unknown) => {
if (cause instanceof RegularFileReadFailure) throw cause.operationCause;
throw cause;
});
const withJsonInputPipeline = async <Value>(
signal: AbortSignal | undefined,
run: (signal?: AbortSignal) => Promise<Value>,
): Promise<Value> => {
if (signal === undefined) return run();
const controller = new AbortController();
const cancel = (): void => {
controller.abort(new JsonInputCancellationFailure(signal.reason));
};
if (signal.aborted) cancel();
else signal.addEventListener("abort", cancel, { once: true });
try {
return await run(controller.signal);
} finally {
signal.removeEventListener("abort", cancel);
}
};
/** Heap a whole-file native JSON.parse may need: the assembler's 64x container expansion plus byte and text copies. */
export const nativeJsonParseRequiredBytes = (
inputBytes: number,
@@ -66,10 +142,11 @@ export const readJsonObjectFile = (
signal?: AbortSignal,
resources?: ArtifactResourceScope,
): Promise<Result<void, JsonInputFileFailure>> =>
withRegularFile(
withJsonInputFile(
path,
async (handle, stats) => {
let previousDigest: string | undefined;
const physicalRead: JsonInputPhysicalReadObservation = {};
try {
const native = await readNativeJson(
handle,
@@ -77,6 +154,8 @@ export const readJsonObjectFile = (
path,
operation,
signal,
false,
physicalRead,
);
if (native !== undefined) {
if (!native.ok) return native;
@@ -92,7 +171,7 @@ export const readJsonObjectFile = (
for (const members of scans)
for (const [key, member] of members) {
if (!Object.hasOwn(value, key)) continue;
signal?.throwIfAborted();
assertRegularFileActive(signal);
member.begin();
const selected: unknown = Reflect.get(value, key);
if (member.mode === "array" && !Array.isArray(selected)) {
@@ -105,7 +184,7 @@ export const readJsonObjectFile = (
Array.isArray(selected)
? selected
: [selected]) {
signal?.throwIfAborted();
assertRegularFileActive(signal);
try {
await member.accept(item);
} catch (cause: unknown) {
@@ -123,68 +202,80 @@ export const readJsonObjectFile = (
let consume:
| ((token: parser.Token) => typeof none | Promise<typeof none>)
| undefined;
await pipeline(
decodedChunks(handle, stats.size, path, signal, (bytes) => {
hash.update(bytes);
}),
asStream(
fun(
parser({
jsonStreaming: false,
streamValues: false,
packKeys: false,
packStrings: false,
packNumbers: false,
}),
packJsonScalars(operation, stats.size, path),
(token: parser.Token): typeof none | Promise<typeof none> => {
const opens =
token.name === "startObject" || token.name === "startArray";
const closes =
token.name === "endObject" || token.name === "endArray";
if (depth === 0) {
rootObject = token.name === "startObject";
} else if (
rootObject &&
depth === 1 &&
token.name === "keyValue"
) {
key = token.value;
} else if (rootObject && depth === 1 && !closes) {
const member =
key === undefined ? undefined : members.get(key);
consume = undefined;
if (member !== undefined) {
member.begin();
if (
member.mode === "array" &&
token.name !== "startArray"
) {
member.reject(
new TypeError(`JSON member ${key} must be an array`),
);
} else {
consume = memberConsumer(
member,
operation,
stats.size,
path,
);
}
}
key = undefined;
}
const selected = consume;
if (opens) depth += 1;
if (closes) depth -= 1;
if (rootObject && depth === 1 && token.name !== "keyValue")
consume = undefined;
if (selected === undefined) return none;
return selected(token);
await withJsonInputPipeline(signal, (pipelineSignal) =>
pipeline(
decodedChunks(
handle,
stats.size,
path,
signal,
(bytes) => {
hash.update(bytes);
},
physicalRead,
),
asStream(
fun(
parser({
jsonStreaming: false,
streamValues: false,
packKeys: false,
packStrings: false,
packNumbers: false,
}),
packJsonScalars(operation, stats.size, path),
(token: parser.Token): typeof none | Promise<typeof none> => {
const opens =
token.name === "startObject" ||
token.name === "startArray";
const closes =
token.name === "endObject" || token.name === "endArray";
if (depth === 0) {
rootObject = token.name === "startObject";
} else if (
rootObject &&
depth === 1 &&
token.name === "keyValue"
) {
key = token.value;
} else if (rootObject && depth === 1 && !closes) {
const member =
key === undefined ? undefined : members.get(key);
consume = undefined;
if (member !== undefined) {
member.begin();
if (
member.mode === "array" &&
token.name !== "startArray"
) {
member.reject(
new TypeError(
`JSON member ${key} must be an array`,
),
);
} else {
consume = memberConsumer(
member,
operation,
stats.size,
path,
);
}
}
key = undefined;
}
const selected = consume;
if (opens) depth += 1;
if (closes) depth -= 1;
if (rootObject && depth === 1 && token.name !== "keyValue")
consume = undefined;
if (selected === undefined) return none;
return selected(token);
},
),
),
pipelineSignal === undefined ? {} : { signal: pipelineSignal },
),
signal === undefined ? {} : { signal },
);
if (!rootObject)
return err(
@@ -197,6 +288,7 @@ export const readJsonObjectFile = (
}
return ok(undefined);
} catch (cause: unknown) {
if (physicalRead.failure !== undefined) throw physicalRead.failure;
const failure = jsonReadFailure(cause, operation, stats.size, path);
if (failure !== undefined) return err(failure);
throw cause;
@@ -256,11 +348,12 @@ export const readJsonInputFile = (
resources?: ArtifactResourceScope,
snapshot = false,
): Promise<Result<unknown, JsonInputFileFailure>> =>
withRegularFile(
withJsonInputFile(
path,
async (handle, stats) => {
let found = false;
let value: unknown;
const physicalRead: JsonInputPhysicalReadObservation = {};
try {
const native = await readNativeJson(
handle,
@@ -269,44 +362,59 @@ export const readJsonInputFile = (
operation,
signal,
snapshot,
physicalRead,
);
if (native !== undefined)
return native.ok && snapshot
? await sealOwnedJsonInput(native.value, operation, signal)
: native;
await pipeline(
decodedChunks(handle, stats.size, path, signal),
// Fuse synchronous token stages so only file chunks cross the async
// stream boundary. Admission still precedes scalar/value assembly.
asStream(
fun(
parser({
jsonStreaming: false,
streamValues: false,
packKeys: false,
packStrings: false,
packNumbers: false,
}),
packJsonScalars(operation, stats.size, path),
admitJsonAssembly(operation, stats.size, path, snapshot ? 2 : 1),
streamValues(),
(row: unknown): typeof none => {
if (
found ||
typeof row !== "object" ||
row === null ||
!("key" in row) ||
row.key !== 0 ||
!("value" in row)
)
throw new Error("Unexpected JSON parser result");
found = true;
value = row.value;
return none;
},
await withJsonInputPipeline(signal, (pipelineSignal) =>
pipeline(
decodedChunks(
handle,
stats.size,
path,
signal,
undefined,
physicalRead,
),
// Fuse synchronous token stages so only file chunks cross the async
// stream boundary. Admission still precedes scalar/value assembly.
asStream(
fun(
parser({
jsonStreaming: false,
streamValues: false,
packKeys: false,
packStrings: false,
packNumbers: false,
}),
packJsonScalars(operation, stats.size, path),
admitJsonAssembly(
operation,
stats.size,
path,
snapshot ? 2 : 1,
),
streamValues(),
(row: unknown): typeof none => {
if (
found ||
typeof row !== "object" ||
row === null ||
!("key" in row) ||
row.key !== 0 ||
!("value" in row)
)
throw new Error("Unexpected JSON parser result");
found = true;
value = row.value;
return none;
},
),
),
pipelineSignal === undefined ? {} : { signal: pipelineSignal },
),
signal === undefined ? {} : { signal },
);
return found
? snapshot
@@ -314,6 +422,7 @@ export const readJsonInputFile = (
: ok(value)
: err(invalidJson(operation, "JSON document is empty"));
} catch (cause: unknown) {
if (physicalRead.failure !== undefined) throw physicalRead.failure;
const failure = jsonReadFailure(cause, operation, stats.size, path);
if (failure !== undefined) return err(failure);
throw cause;
@@ -526,6 +635,7 @@ async function* decodedChunks(
path: string,
signal: AbortSignal | undefined,
observe?: (bytes: Uint8Array) => void,
physicalRead: JsonInputPhysicalReadObservation = {},
): AsyncGenerator<string> {
const decoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
let leading = true;
@@ -539,14 +649,16 @@ async function* decodedChunks(
const bytes = Buffer.allocUnsafe(READ_CHUNK_BYTES);
let observedBytes = 0;
while (true) {
signal?.throwIfAborted();
const read = await handle.read(
bytes,
0,
Math.min(bytes.length, admittedBytes - observedBytes + 1),
observedBytes,
assertRegularFileActive(signal);
const read = await readPhysicalJsonInput(physicalRead, () =>
handle.read(
bytes,
0,
Math.min(bytes.length, admittedBytes - observedBytes + 1),
observedBytes,
),
);
signal?.throwIfAborted();
assertRegularFileActive(signal);
if (read.bytesRead === 0) break;
observedBytes += read.bytesRead;
if (observedBytes > admittedBytes) throw new RegularFileChangedError(path);
@@ -569,13 +681,16 @@ const readPrefix = async (
handle: FileHandle,
size: number,
signal: AbortSignal | undefined,
physicalRead: JsonInputPhysicalReadObservation,
): Promise<{ readonly bytes: Buffer; readonly complete: boolean }> => {
const bytes = Buffer.allocUnsafe(size);
let offset = 0;
while (offset < bytes.length) {
signal?.throwIfAborted();
const read = await handle.read(bytes, offset, bytes.length - offset, null);
signal?.throwIfAborted();
assertRegularFileActive(signal);
const read = await readPhysicalJsonInput(physicalRead, () =>
handle.read(bytes, offset, bytes.length - offset, null),
);
assertRegularFileActive(signal);
if (read.bytesRead === 0)
return { bytes: bytes.subarray(0, offset), complete: true };
offset += read.bytesRead;
@@ -591,9 +706,10 @@ const readNativeJson = async (
operation: string,
signal?: AbortSignal,
snapshot = false,
physicalRead: JsonInputPhysicalReadObservation = {},
): Promise<Result<unknown, JsonInputFileFailure> | undefined> => {
if (!nativeParseFits(size, snapshot)) return undefined;
const prefix = await readPrefix(handle, size + 1, signal);
const prefix = await readPrefix(handle, size + 1, signal, physicalRead);
if (!prefix.complete || prefix.bytes.length !== size)
throw new RegularFileChangedError(path);
const parsed = parseUtf8Json(
+16 -2
View File
@@ -7,7 +7,11 @@ import {
ArtifactResourceScope,
type ArtifactResourceOwner,
} from "../artifacts/ArtifactResourceScope.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import type { AnalysisCleanupLease } from "../artifacts/AnalysisCleanupLease.js";
import {
ArtifactReaderFailure,
artifactPrimaryCleanupLeases,
} from "../artifacts/ArtifactReader.js";
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
import { err, ok, type Result } from "../domain/result.js";
import {
@@ -922,7 +926,9 @@ export class LinuxHopperPublisher {
"integrity",
"Retained Hopper evidence exceeds its resource byte budget.",
);
const bytes = await readBoundedFileBytes(handle, stats.size, signal);
const bytes = await readBoundedFileBytes(handle, stats.size, {
signal,
});
if (bytes === undefined)
throw new HopperReleaseError(
"integrity",
@@ -1061,12 +1067,17 @@ export class LinuxHopperPublisher {
export const retryLinuxHopperPublisherCleanup = async (): Promise<void> => {
const causes: unknown[] = [];
const observations = [];
const cleanupLeases = new Set<AnalysisCleanupLease>();
let cleanupLeaseAuthorityKnown = true;
for (const scope of pendingCleanup) {
try {
await scope.close();
pendingCleanup.delete(scope);
} catch (cause: unknown) {
causes.push(cause);
const leases = artifactPrimaryCleanupLeases(cause);
if (leases === undefined) cleanupLeaseAuthorityKnown = false;
else for (const lease of leases) cleanupLeases.add(lease);
observations.push(
ArtifactReaderFailure.cleanupObservation(
cause,
@@ -1088,6 +1099,9 @@ export const retryLinuxHopperPublisherCleanup = async (): Promise<void> => {
{
cause: new AggregateError(causes, "Hopper publisher cleanup failures"),
cleanup,
cleanupLeases: cleanupLeaseAuthorityKnown
? [...cleanupLeases]
: undefined,
},
);
}
@@ -1,8 +1,18 @@
import { expect, it, vi } from "vitest";
import { z } from "zod";
import { AnalysisInputError } from "../domain/analysisErrorCore.js";
import { executeProviderOperation } from "./ProviderOperationExecution.js";
import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import { analysisErrorWithCleanupFailure } from "../domain/analysisErrorCleanup.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import {
executeProviderOperation,
keepIncompleteObservation,
} from "./ProviderOperationExecution.js";
it("projects request-schema issues without exposing the request envelope", async () => {
const execute = vi.fn();
@@ -87,3 +97,84 @@ it("preserves operation correction outside the input envelope", async () => {
]);
expect(execute).not.toHaveBeenCalled();
});
const cancelledScenario = (partial: unknown) =>
new AnalysisCancelledError("capture_browser_scenario", {
cause: { requested: "stop during capture" },
partialObservation: partial as never,
});
const wrapScenarioCleanup = (primary: AnalysisCancelledError) =>
analysisErrorWithCleanupFailure(
primary,
new ProviderCleanupError("fixture", ["browser"], {}),
"capture_browser_scenario",
);
it("clears a non-JSON partial from the cleanup primary projection returns after retry", () => {
const partial = { steps: [] as unknown[] };
const primary = cancelledScenario(partial);
const composed = wrapScenarioCleanup(primary);
partial.steps.push(undefined);
const reason =
"Browser scenario provider retained a non-JSON incomplete observation.";
const retained = keepIncompleteObservation(
composed,
"capture_browser_scenario",
reason,
);
expect(retained).toBe(composed);
expect(composed.partialObservation).toBeUndefined();
expect(primary.partialObservation).toBeUndefined();
expect(composed.secondaryFailures).toMatchObject([
{ _tag: "AnalysisOutputError", reason },
]);
expect(primary.secondaryFailures[0]).toBeInstanceOf(AnalysisOutputError);
expect(projectAnalysisError(composed).code).toBe("cleanup_incomplete");
composed.resolveCleanup();
expect(projectAnalysisError(composed)).toMatchObject({
code: "cancelled",
category: "cancelled",
});
expect(projectAnalysisError(composed).details).not.toHaveProperty(
"partial_observation",
);
});
it("freezes one shared snapshot for a cleanup wrapper and its primary", () => {
const partial = { steps: ["kept"] };
const primary = cancelledScenario(partial);
const composed = wrapScenarioCleanup(primary);
keepIncompleteObservation(
composed,
"capture_browser_scenario",
"unused because the partial is JSON",
);
expect(composed.partialObservation).toBe(primary.partialObservation);
expect(composed.partialObservation).toEqual({ steps: ["kept"] });
expect(Object.isFrozen(composed.partialObservation)).toBe(true);
expect(composed.secondaryFailures).toHaveLength(0);
expect(primary.secondaryFailures).toHaveLength(0);
});
it("freezes a retained partial so a later mutation cannot break projection", () => {
const partial = { steps: [] as unknown[] };
const primary = cancelledScenario(partial);
keepIncompleteObservation(
primary,
"capture_browser_scenario",
"unused because the partial is JSON",
);
partial.steps.push(undefined);
expect(primary.partialObservation).toEqual({ steps: [] });
expect(Object.isFrozen(primary.partialObservation)).toBe(true);
expect(primary.secondaryFailures).toHaveLength(0);
expect(projectAnalysisError(primary)).toMatchObject({
code: "cancelled",
details: { partial_observation: { steps: [] } },
});
});
@@ -2,12 +2,19 @@ import { z } from "zod";
import type {
AnalysisError,
AnalysisErrorOptions,
AnalysisPartialObservation,
} from "../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisInputError,
AnalysisOutputError,
} from "../domain/analysisErrorCore.js";
import type { Evidence } from "../domain/evidence.js";
import {
freezeJsonSnapshot,
isImmutableJsonSnapshot,
} from "../domain/immutableJson.js";
import { jsonValueSchema } from "../domain/jsonValue.js";
import { projectInputIssues } from "../domain/inputIssueProjection.js";
import { err, ok, type Result } from "../domain/result.js";
import {
@@ -92,3 +99,49 @@ export const completeProviderEvidence = (
);
return ok(evidence);
};
/**
* Retain an incomplete observation when projection can serialize it.
* A JSON partial is replaced by one frozen snapshot, so a later mutation
* cannot make projection throw and projection does not copy it again.
* A partial that is not JSON is cleared from the returned error and from
* every cleanup primary projection can select later. Each cleared error
* records an output failure, leaving cancellation or timeout primary.
*/
export const keepIncompleteObservation = (
error: AnalysisError,
operation: string,
reason: string,
): AnalysisError => {
const seen = new Set<AnalysisError>();
// One source object can sit on a wrapper and its cleanup primary.
const judged = new Map<object, AnalysisPartialObservation | undefined>();
let current: AnalysisError | undefined = error;
while (current !== undefined && !seen.has(current)) {
seen.add(current);
const partial = current.partialObservation;
if (
partial !== undefined &&
!isImmutableJsonSnapshot(partial) &&
typeof partial === "object"
) {
// safeParse clones only after validation succeeds. Freeze that copy once.
let snapshot = judged.get(partial);
if (!judged.has(partial)) {
const parsed = jsonValueSchema.safeParse(partial);
snapshot = parsed.success
? (freezeJsonSnapshot(parsed.data) as AnalysisPartialObservation)
: undefined;
judged.set(partial, snapshot);
}
if (snapshot === undefined) {
current.partialObservation = undefined;
current.retainSecondaryFailure(
new AnalysisOutputError(operation, reason),
);
} else current.partialObservation = snapshot;
}
current = current.cleanupPrimary;
}
return error;
};
+48 -10
View File
@@ -5,6 +5,7 @@ import ignore from "ignore";
import {
readRegularFileText,
RegularFileCleanupFailure,
RegularFileReadFailure,
retryRegularFileCleanup,
} from "./RegularFileRead.js";
@@ -17,6 +18,7 @@ import {
type ReferenceSourceImportOptions,
} from "./ReferenceSourceImportTypes.js";
import { rootFilesystemFailure } from "../reference/ReferenceSourceReaderErrors.js";
import { regularFileCancellation } from "../filesystem/OwnedRegularFile.js";
/** Validated inputs ready for filesystem traversal. */
export interface PreparedReferenceSourceImport {
@@ -34,6 +36,25 @@ const failure = (
message,
});
class IgnorePolicyReadCancelled extends Error {
constructor(readonly reason: unknown) {
super("Reference source ignore policy read was cancelled", {
cause: reason,
});
this.name = "IgnorePolicyReadCancelled";
}
}
const isIgnorePolicyCancellation = (cause: unknown): boolean =>
cause instanceof IgnorePolicyReadCancelled;
const ignorePolicyCause = (cause: unknown): unknown =>
cause instanceof IgnorePolicyReadCancelled
? cause.reason
: cause instanceof RegularFileReadFailure
? cause.operationCause
: cause;
const isAborted = (signal?: AbortSignal): boolean => signal?.aborted === true;
const addMarkedPatterns = (
@@ -102,30 +123,46 @@ const buildIgnored = async (
throw cause;
present = false;
}
signal?.throwIfAborted();
if (signal?.aborted === true)
throw new IgnorePolicyReadCancelled(signal.reason);
if (present) {
let text: string;
try {
text = await readRegularFileText(policyPath, { signal });
text = await readRegularFileText(policyPath, {
signal,
cancellationFailure: (reason) => new IgnorePolicyReadCancelled(reason),
});
} catch (cause: unknown) {
if (signal?.aborted === true && cause === signal.reason) throw cause;
if (cause instanceof IgnorePolicyReadCancelled) throw cause;
if (!(cause instanceof RegularFileCleanupFailure)) throw cause;
const retry = await retryRegularFileCleanup(cause, resources, signal);
const operationCancellation =
cause.outcome.kind === "failed"
? regularFileCancellation(cause)
: undefined;
if (retry.cleanup !== undefined) {
const primary =
retry.outcome.kind === "failed"
? retry.outcome.cause
: cause.cleanupCause;
const primaryCause = ignorePolicyCause(primary);
return err({
...failure(
signal?.aborted === true ? "cancelled" : "io",
`Reference source ignore policy could not be read at ${policyPath}: ${errorMessage(primary)}`,
isIgnorePolicyCancellation(primary) ||
operationCancellation !== undefined
? "cancelled"
: "io",
`Reference source ignore policy could not be read at ${policyPath}: ${errorMessage(primaryCause)}`,
),
cleanup: retry.cleanup,
cause: primary,
cause: primaryCause,
});
}
if (retry.outcome.kind === "failed") throw retry.outcome.cause;
if (retry.outcome.kind === "failed") {
if (operationCancellation !== undefined)
throw new IgnorePolicyReadCancelled(operationCancellation.reason);
throw retry.outcome.cause;
}
text = retry.outcome.value;
}
addMarkedPatterns(ignored, [text], "project-ignored");
@@ -166,12 +203,13 @@ export const prepareReferenceSourceImport = async (
secrets: ignore().add([...options.policy.secretPatterns]),
});
} catch (cause: unknown) {
const causeValue = ignorePolicyCause(cause);
return err({
...failure(
options.signal?.aborted === true ? "cancelled" : "io",
`Reference source ignore policy could not be read at ${join(root.value, ".gitignore")}: ${errorMessage(cause)}`,
isIgnorePolicyCancellation(cause) ? "cancelled" : "io",
`Reference source ignore policy could not be read at ${join(root.value, ".gitignore")}: ${errorMessage(causeValue)}`,
),
cause,
cause: causeValue,
});
}
});
+90 -23
View File
@@ -10,7 +10,9 @@ import {
import {
RegularFileCleanupFailure,
RegularFileCancelled,
regularFileCancellation,
withOwnedRegularFile,
OwnedRegularFileOperationFailure,
type RegularFileOperationOutcome,
} from "../filesystem/OwnedRegularFile.js";
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
@@ -24,6 +26,18 @@ export interface RegularFileReadOptions {
readonly resources?: ArtifactResourceScope;
/** Runs on the opened file's own state before any byte is read; throw to refuse the read. */
readonly admit?: (state: Stats) => void;
/** Tags cancellation at the bounded reader's cooperative checkpoints. */
readonly cancellationFailure?: (reason: unknown) => Error;
}
/** Preserves a physical descriptor-read cause until its owner has finalized. */
export class RegularFileReadFailure extends OwnedRegularFileOperationFailure {
override readonly _tag = "RegularFileReadFailure";
constructor(operationCause: unknown) {
super(operationCause);
this.name = this._tag;
}
}
export class InvalidUtf8FileError extends TypeError {
@@ -52,6 +66,7 @@ export const retryRegularFileCleanup = async <Value>(
}> => {
const outcome: RegularFileReadOutcome<Value> =
failure.outcome.kind === "failed" &&
failure.operationFailure === undefined &&
failure.outcome.cause instanceof RegularFileOperationCancelled &&
!(signal?.aborted === true && failure.outcome.cause === signal.reason)
? { kind: "failed", cause: failure.outcome.cause.reason }
@@ -93,7 +108,10 @@ export const retryRegularFileCleanupFailures = async (
? retried.outcome.cause
: current.cleanupCause;
const nested =
current.outcome.kind === "failed" ? current.outcome.cause : undefined;
current.operationFailure === undefined &&
current.outcome.kind === "failed"
? current.outcome.cause
: undefined;
if (!(nested instanceof RegularFileCleanupFailure)) break;
current = nested;
}
@@ -153,21 +171,51 @@ export const withRegularFile = async <Value>(
// Caller reasons are opaque, including objects shaped like file failures.
// They do not transfer ownership of any descriptor to this operation.
if (signal?.aborted === true && cause === signal.reason) throw cause;
if (cause instanceof RegularFileOperationCancelled) throw cause.reason;
if (cause instanceof RegularFileReadFailure) throw cause;
if (
cause instanceof RegularFileCancelled &&
cause.owner.handle === admitted
)
throw cause.reason;
if (
!(cause instanceof RegularFileCleanupFailure) ||
resources === undefined
cause.owner.handle !== admitted
)
throw cause;
const retry = await retryRegularFileCleanup(cause, resources, signal);
if (retry.cleanup !== undefined) throw cause;
if (retry.outcome.kind === "failed") throw retry.outcome.cause;
return retry.outcome.value;
if (cause instanceof RegularFileCleanupFailure) {
if (resources === undefined) throw cause;
const retry = await retryRegularFileCleanup(cause, resources, signal);
if (retry.cleanup !== undefined) throw cause;
if (retry.outcome.kind === "failed") {
if (
cause.operationFailure === undefined &&
signal?.aborted === true &&
retry.outcome.cause === signal.reason
)
throw signal.reason;
const cancellation =
cause.operationFailure === undefined
? (regularFileCancellation(retry.outcome.cause) ??
regularFileCancellation(cause))
: undefined;
if (cancellation !== undefined) {
if (options.cancellationFailure !== undefined)
throw options.cancellationFailure(cancellation.reason);
throw cancellation.reason;
}
if (
cause.operationFailure !== undefined &&
options.cancellationFailure !== undefined
)
throw cause.operationFailure;
throw retry.outcome.cause;
}
return retry.outcome.value;
}
const cancellation = regularFileCancellation(cause);
if (cancellation !== undefined) {
if (options.cancellationFailure !== undefined)
throw options.cancellationFailure(cancellation.reason);
throw cancellation.reason;
}
if (options.cancellationFailure !== undefined)
throw new RegularFileReadFailure(cause);
throw cause;
}
};
const value = await (resources === undefined
@@ -175,6 +223,8 @@ export const withRegularFile = async <Value>(
: resources.run(operation));
// A retained close may have recovered after the descriptor owner's outcome
// was captured. Include that retry and the scope's finalization in cancellation.
if (signal?.aborted === true && options.cancellationFailure !== undefined)
throw options.cancellationFailure(signal.reason);
signal?.throwIfAborted();
return value;
};
@@ -221,19 +271,36 @@ const readRegularFileContent = <Value>(
path: string,
decode: (bytes: Buffer, state: Stats) => Value,
options: RegularFileReadOptions,
): Promise<Value> =>
withRegularFile(
): Promise<Value> => {
const operation = withRegularFile(
path,
async (handle, stats) => {
options.admit?.(stats);
const bytes = await readBoundedFileBytes(
handle,
stats.size,
options.signal,
);
if (bytes === undefined || bytes.length !== stats.size)
throw new RegularFileChangedError(path);
return decode(bytes, stats);
try {
options.admit?.(stats);
const bytes = await readBoundedFileBytes(handle, stats.size, {
...(options.signal === undefined ? {} : { signal: options.signal }),
cancellationFailure: (reason) =>
new RegularFileOperationCancelled(reason),
operationFailure: (cause) => new RegularFileReadFailure(cause),
});
if (bytes === undefined || bytes.length !== stats.size)
throw new RegularFileChangedError(path);
return decode(bytes, stats);
} catch (cause: unknown) {
if (
options.cancellationFailure !== undefined &&
!(cause instanceof RegularFileOperationCancelled) &&
!(cause instanceof RegularFileReadFailure)
)
throw new RegularFileReadFailure(cause);
throw cause;
}
},
options,
);
if (options.cancellationFailure !== undefined) return operation;
return operation.catch((cause: unknown) => {
if (cause instanceof RegularFileReadFailure) throw cause.operationCause;
throw cause;
});
};
+3 -6
View File
@@ -53,12 +53,9 @@ const readSkillFileState = async (
withRegularFile(
path,
async (handle, stats) => {
const bytes = await readBoundedFileBytes(
handle,
stats.size,
undefined,
stats.size,
);
const bytes = await readBoundedFileBytes(handle, stats.size, {
expectedSize: stats.size,
});
if (bytes === undefined || bytes.length !== stats.size)
throw new RegularFileChangedError(path);
return { bytes, mode: stats.mode & 0o7777 };
Loaded 100 of 457 files, more files were not shown because too many files have changed in this diff. Show more