Compare commits

...
Author SHA1 Message Date
790cdfdefa test(experiment): freeze client wait guidance diagnostic
Keep identical analysis contracts, question, engine and complete catalogs. Test common opt-in wait guidance while retaining polling and native latency in whole-task costs.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 16:02:44 +00:00
c795081e78 docs(experiment): record real-agent selected-case results
Report both correct answers and complete task costs. Preserve the native latency and client polling confounds; no general Code Mode advantage is established.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 15:58:53 +00:00
f44dca9c37 feat(experiment): expose native case-to-edge facts
Share exact jump-table metadata and paged case relationships across CLI,
MCP and Code Mode. Keep labels, row identities, recovered edges and machine
defaults separate; reclaim independent facts under output capacity.

Verify all 182 SQLite table targets against ELF bytes through both MCP
adapters and retain cases/calls when a real 2 MiB native budget omits IR.
Freeze the next analyst question and oracle before real-model trials.

[AUTOMATED] Experimental native patch; no upstream integration or PR.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 15:32:12 +00:00
29e0472812 test(experiment): measure queued and repeated native reads
Compare sequential and overlapping requests through both real MCP adapters
with the same preserved engine, selected instruction windows and full graph.
Retain the unexplained model-run anomaly without claiming a speed fix.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 14:25:22 +00:00
4882f6b5dc docs(experiment): retain guided model comparisons and limits
Verify both repeated answers and exact discovery calls against unchanged
source facts. Record complete task costs, overlapping requests, unexplained
capture-time variation and the client executor available to both lanes.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 14:14:27 +00:00
7a7c4a6d7a docs(experiment): record real-agent control-flow evidence
Review both complete answers against the precommitted oracle and independent
ELF instruction bytes. Retain all costs, validation retries, discovery noise
and the limits of this single observational pair.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 14:01:07 +00:00
e2205313a5 feat(experiment): expose bounded full recovered graphs
Preserve complete graph facts with a bounded lossless native wire codec and
shared paged Evidence views. Verify both MCP adapters and retained legacy
records, then freeze a control/data-flow question and oracle for real agents.

[AUTOMATED] Exploratory native patch; no upstream integration or PR.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 13:47:33 +00:00
7d0da075e7 perf(experiment): measure native analysis phase costs
[AUTOMATED] Add a gated diagnostic overlay and production-owner profiling runner. Preserve actual phase/resource results, disabled-output control and lossless full-graph encoding feasibility without claiming a public codec or model speedup.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 12:41:35 +00:00
4986b8644f docs(experiment): record indexed discovery model costs
Retain both correctness-equal wide-PHI pairs, complete discovery costs and the remaining whole-function native computation boundary.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 12:22:02 +00:00
a440049026 feat(experiment): expose a schema-free method index
Discover declared methods through the shared reference owner before requesting named contracts; preserve the existing default and runtime capability semantics.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 12:15:10 +00:00
e678be3f9d docs(experiment): record selected operand model results
Both adapters satisfy the fixed wide-PHI oracle with one capture and selected slots. Preserve mixed timing/token costs and the observed discovery overhead rather than inferring a Code Mode advantage.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 12:12:09 +00:00
e38a4c4394 feat(experiment): page native SSA operand relationships
Keep operation and slice defaults compact; preserve exact input slots and phi edges, distinguish unretained seeds, and align advertised relation schemas with runtime validation. Pin a seeded wide-PHI model task after real public MCP verification.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 12:05:15 +00:00
a225d7c552 fix(experiment): retain direct analysis failures in trial ledgers
Record the second PHI model pair, including the native timeout and successful retry; keep adapter conclusions limited to observed whole-task costs.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 11:57:25 +00:00
14bc2bca21 fix(experiment): align named reference lookup with input contracts
Share displayed method names with direct registration, support explicit multi-name selection and unmatched terms, and cache immutable schema conversions by schema identity. Export input-side schemas so defaulted fields stay optional; verify both actual MCP catalogs and named contracts before repeating the unchanged model task.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 11:41:48 +00:00
71be07de42 docs(experiment): record real-model phi projection results
Both answers satisfy the fixed four-alternative oracle with one VM capture and scoped facts. Preserve whole-task costs and source checks, including the direct reference-name lookup miss that needs a shared discovery fix.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 11:30:18 +00:00
8114bd675e feat(experiment): expose bounded call-rooted definition facts
Share native membership and operand ownership across full and projected facts. Preserve exact phi bindings and definition frontiers when the full graph is unavailable, keep CFG/use coverage explicit, and expose selected reads through the same CLI/SDK/MCP contracts. Verify real SQLite/zlib fidelity, encoded fallback, bounded metadata, and cache reuse; pin the next real-model question and oracle.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 11:16:26 +00:00
4d7fccbce6 docs(experiment): measure bounded native definition projections
Preserve a diagnostic-only patch and reproducible lossless encoding/projection probe. The real VM graph exceeds several public capacities; selected SSA neighborhoods are much smaller, with explicit frontiers. Record actual resource costs and keep public engine/API limits unchanged.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 10:30:19 +00:00
af53e0fbe5 fix(experiment): retain ownership after tagged cleanup failures
Check the native cleanup Result before clearing the lease and preserve cleanup resources through the MCP error adapter. Reproduce the old false success, then verify real SDK close/replacement ownership and both MCP retry paths with explicit fault injection.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 10:25:17 +00:00
f08c191ed3 docs(experiment): compare compact call counts in real model tasks
Retain the successful answer and source checks from both adapters and all measured task costs. Both avoid call-set paging; direct beats Code Mode in the new sample and before/after effects remain mixed.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 10:07:36 +00:00
da6f5470be perf(experiment): summarize retained calls before paging
Expose a compact call distribution in snapshot metadata through the shared validated owner. Preserve coverage, unknown targets, dead rows, immutable selected reads, and empty versus unavailable semantics. Retain the same boundary question and oracle for paired real-model comparison.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 10:00:04 +00:00
fb9adbf384 docs(experiment): record real-model call boundary results
Both lanes satisfy the pre-pinned SQLite task with one retained VM analysis. Record full answer/source review, actual model and resource counters, and mixed context costs without a general efficiency claim.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 09:53:53 +00:00
f9f9dc209d feat(experiment): retain call facts beyond full graph capacity
Separate native call and boundary-value ownership from the bounded recovered graph. Validate facets independently and preserve shared identities, source evidence, and explicit partial coverage. Pin the SQLite boundary task and oracle before paired real-model trials.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 09:46:42 +00:00
04c081fc61 experiment(re): verify model reference aggregation against machine calls
Record both completed model workflows, whole-task costs and source/identity checks. Compare every VM direct-call pair with independent disassembly, preserve indirect-call limits, verify unpatched stock through the shared CLI/API, and retain failed/partial research costs.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 09:01:54 +00:00
3aa685db39 feat(experiment): compose shallow outgoing references with explicit scope
Share one native parser for both directions, preserve occurrence identity and unknown function applicability, push kind filters to Kuna and reuse sources across fields/pages/scopes. Verify real SQLite machine references independently, including calls omitted from optimized IR and VM queries beyond graph capacity. Pin the aggregate model task before its paired runs.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 08:48:24 +00:00
56182b2217 experiment(re): verify cross-function model answers and mixed adapter costs
Record the pre-pinned explicit-return-path pair with source and identity validation. Both models complete the context-flow task; Code Mode has fewer MCP requests without a demonstrated overall efficiency advantage. Preserve large-function and shallow-reference coverage limits.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 08:20:24 +00:00
e9348df0c3 experiment(re): align cross-function question with pinned return-path oracle
Retain both initial real-model answers and whole-task costs. Both prove the core context connection but omit the oracle-required return-code distinctions; clarify that requirement before the next pair without changing the oracle or supplying addresses.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 08:10:41 +00:00
01e5c59551 fix(experiment): validate native engine pins before model trials
Preserve the rejected cross-function launch caused by a non-embedded Cargo test build replacing the mutable CLI. Check explicit engine identity before profile creation and after the task; use an independent executable copy for measured runs.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 08:03:30 +00:00
33c4282b8a feat(experiment): expose structured native call and argument relations
Keep recovered callee identities separate from fspec annotations and SSA argument values separate from passing storage. Add selected calls and function-input reads through the shared fact owner, with real SQLite MCP coverage and a pre-pinned cross-function model task.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 07:58:01 +00:00
f34425dc24 fix(experiment): enforce Code Mode payload budgets on guest errors
Reject oversized error delivery with explicit omission metadata, account for JSON escaping, and retain source observations and their lookup after failure. Reproduce and verify the real MCP path without claiming model-token savings from byte counts.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 07:33:09 +00:00
dad0228df8 experiment(re): verify model composition of class and dispatch tables
Record matching six-byte table oracles, real byte-tool use, model response counts and whole-task costs without generalizing a single paired observation.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 07:23:40 +00:00
73db4fad54 feat(experiment): compose bounded native byte windows with RE facts
Add shared SDK/CLI/MCP byte reads over the existing Kuna data protocol, preserve partial batches and exact u64 ranges, and retain complete source Evidence. Reuse named read lifecycles, bound aggregate hex expansion, and verify stock/native/direct MCP paths plus a shipped-file byte oracle before pinning a table-dispatch model task.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 07:17:18 +00:00
c53c2a9b90 experiment(re): record equal-coverage model use of retained Pragma code
Keep actual response/MCP counts, whole-task token usage and resource receipts separate. Both adapters complete the local task with explicit missing IR; no general speedup is inferred.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 06:58:46 +00:00
9e1b39358a experiment(re): pin a model task over recovered Pragma code
Exercise the restored code facet with a question fixed before trial and independent instruction anchors, using identical envelope guidance in both atomic calling modes.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 06:52:03 +00:00
1346e97dca fix(experiment): preserve code when encoded native graphs exceed capacity
Bound compact native JSON with the existing writer, keep a valid code facet when the graph cannot fit, and pass the supervised byte budget explicitly. Verify real SQLite Parser/Pragma captures, exact preserved C/mappings/graph facts and public MCP reads. Keep string inventory parsing/projection under its own owner.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 06:50:20 +00:00
dc022bcd0f experiment(re): separate model rounds from MCP requests in fair trials
Validate per-response usage totals, retain the negative large-VM comparison and qualify the client executor, cache and investigation-depth differences. Add optional common envelope/stopping guidance without relabeling historical runs.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 06:29:26 +00:00
5aa78c873e experiment(re): pin coverage criteria for a large SQLite VM task
Record independent machine-code anchors before model trials, separate cache regimes and incomplete answers, and exercise real recovered-IR capacity failure without hiding retained code.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 06:09:46 +00:00
15d75ea1ea fix(experiment): preserve cached fact ownership for SDK callers
Freeze parsed inventories once at their owning boundary and share the rule with native IR and function views. Verify the real SDK mutation regression and retain corrected equal-depth SQLite model costs, including persistent-cache limitations.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 06:07:46 +00:00
e9b9a8dd0e experiment(re): describe direct atomic trials without a Code Mode tool
Keep the direct adapter benchmark instructions consistent with its actual registered tools; retain historical trial profiles unchanged.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 05:48:57 +00:00
f3f209d03f experiment(re)!: preserve partial captures and resolve function entries
Keep validated code available when native graph extraction is unavailable, bind output to the actual input/entry, preserve cancellation observations, and share batch source/error semantics. Add filtered function/alias reads and slice field selection across adapters.

Verified 38 focused tests, targeted lint, real stock/native MCP workflows, large SQLite graph-capacity fallback, and real cold/warm MCP reuse. Retain actual model costs and unequal-depth/selection limitations instead of claiming general superiority.

BREAKING CHANGE: Experimental native snapshots may succeed with readable code and an unavailable recovered_ir facet. Inspect facets before graph reads; unavailable graph counts are null/unknown.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 05:48:03 +00:00
4f13bd6733 experiment(re)!: separate IR origins from machine instruction facts
A real model conflated an optimized CFG edge with the bytes at its instruction origin. Make provenance explicit and add bounded, batched instruction windows from the existing Kuna CLI, preserving exact bytes, per-window failures and Evidence. Record the task-cost comparison without treating the incomplete machine-path answer as an equal-quality win.

Verified 33 focused tests, targeted lint, real stock/patched MCP decoding with mixed success, and a real 4096-row capacity run under the resource observer.

BREAKING CHANGE: Experimental operation.address/addresses and block.start become origin_address/origin_addresses and origin_start. Printed CFG target_address becomes target_origin; historical source captures are unchanged.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 04:51:55 +00:00
f2f029c5dd experiment(re): retain native model trace and task-cost limits
The real model uses recovered controllers and indexed phi/value relations, but the first native-IR trial still does not beat the traditional local-question workflow. Record actual whole-task usage and prepare value-origin and small-function counterexamples.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 04:33:55 +00:00
a7e9f8843b experiment(re): expose native CFG and value identities for composition
[AUTOMATED] Preserve the opt-in pinned Kuna exporter as a reproducible source patch. Share typed graph/value reads and derived control dependence through the SDK, CLI and MCP adapters, retaining unknown runtime and memory semantics.

Validated 29 focused tests, lint, actual diagnostic/native MCP workflows and a real zlib CLI composition. C and line mappings match the prior stock capture. Full upstream corpus/workspace gates and revised real-model costs remain unverified.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 04:28:22 +00:00
fcfc87b2b4 experiment(re): record actual three-way model task costs
Retain failed attempts, cached input accounting and payload-exit process-family counters. The first comparison favors the traditional workflow; keep the negative finding explicit.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 04:00:19 +00:00
c5cefd622b experiment(re): refine composition contracts from a real model trace
Record genuine Code Mode use and full model usage rather than response-only estimates. Describe return shapes, permit byte-budgeted internal C reads, separate guest CPU from analysis deadlines, and move subsequent model profiles outside the source tree. Verify lease, admission, cleanup-retry and cancellation semantics.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 03:28:30 +00:00
5aa1c5c64f experiment(re): expose shared atomic facts through isolated Code Mode
Add a compact rea-codemode MCP server, bounded QuickJS execution and shared SDK/direct adapters. Reuse native target and process owners; retain evidence and lease-scoped snapshots. Extend the real-model trial runner with an isolated experimental MCP registration. Capability and evaluation work remains in progress.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 03:17:05 +00:00
2e11fc23fd experiment(re): validate guard slices and record token tradeoffs
Verify three repaired zlib mappings, a SQLite call guard and all ordinary 16-bit header values against retained native evidence. Record producer omissions, failed-run costs and the small-function context counterexample. Preserve existing capture outputs with exclusive writes.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 02:59:46 +00:00
fd79a273f3 experiment(re): compose native SSA uses and branch dependencies
Retain bounded native snapshots and public MCP evidence, then query explicit SSA relations without interpreting reconstructed C. Preserve unknown flow, cloned block identity and dependency frontiers.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 02:59:46 +00:00
d711c55aed experiment(re): distinguish point facts from object selectors
Apply the interior-reference counterexample to the facade: label exact-address scope and emit location endpoints while preserving caller selectors separately. Carry canonical target-content provenance through experimental MCP replies and verify it on the real client. Keep original measurements pinned to the v1 response shape. Nine experiment checks and real Kuna lifecycle/context checks pass.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 02:59:46 +00:00
72b24433ea experiment(re): record native sharing and composition counterexamples
Measure twelve equivalent MCP navigation workflows, native reference plans on SQLite/zlib, a compiled interior-reference oracle and a failed exact code-mapping composition. Preserve failed attempts and measurement limits, including unavailable live-agent effectiveness evidence. Add bounded native probes and deterministic mapping tests. All 191 Kuna cases pass serially; nine experiment tests and one-checker typecheck pass.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 02:59:46 +00:00
2e19ece025 experiment(kuna): preserve typed reference facts at their owner
Expose source-owned reference kind, endpoints, function identities and instruction text for experiments. Keep the existing address-only public result and retained-cache shape; materialize richer facts only when explicitly requested. Record prior art, composition rules and the distinction between fact, transport and compute units. Verified 18 producer boundary cases and an independently checked native interior-reference example.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 02:59:46 +00:00
27f981352d experiment(re): compare fact granularity through real MCP
Add a non-shipping MCP facade over the existing admitted analysis port, real-provider comparison runner and offline token accounting. Keep the full catalog, source Evidence and per-input failures. Separate verification runs from equivalent-work timings. Kuna zlib comparisons pass; broader comparisons remain in progress.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 02:59:45 +00:00
0d99e8c5c5 experiment(re): define composable facts and context boundaries
Start a non-integrating experiment around fact identity, per-input outcomes, paging and shared primitive composition. Include deterministic boundary checks; real-provider comparisons remain in progress.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 02:59:45 +00:00
Ysopwnable 42fc6c53ae fix(javascript): preserve incoming callback target uncertainty (#4243)
Preserve incoming callback uncertainty and resolve proven preceding writes.
Share read-position and initializer identity facts with returned-reference
analysis; cache completed projected work with actual cycle dependencies.

Fixes #3880
2026-10-16 10:09:22 +08:00
morluto 1bd5c5c87e fix(adb): retain and resolve owned hash cleanup
Keep exact returned hash and directory failures linked to their cleanup owners. Retain input staging while its descriptor remains unresolved, and resolve the linked errors only after successful release. Preserve typed cleanup and partial acquisition facts through file, package, and bugreport consumers instead of wrapping or flattening them away.

Validated 189 affected tests across 13 files, build, typecheck, lint, and docs. Three larger integration cases were excluded to respect the CPU limit. New real-file fault controls reproduce on the old source and verify persistent failure, owner identity, cancellation, retained output, and recovery. No physical ADB device or real JADX engine verification. Refs #4266, #4274.
2026-10-16 09:06:26 +08:00
morluto 6e74704f93 fix(android): resolve hash cleanup through its owner
Bind the typed hash failure to its retained descriptor and resolve cleanup uncertainty only when that exact owner closes. Preserve the original failure through immediate retirement and later shutdown recovery. Controlled real-file regressions cover immediate and persistent close faults; no real JADX engine was launched. Refs #4266, #4274.
2026-10-16 09:06:19 +08:00
morluto 3ed925e547 fix(errors): exclude recovered cleanup resources
Compose only unresolved primary resources into a later cleanup failure, retaining both original causes and diagnostics. Document exact-owner recovery and cover resolved and unresolved composition controls. Refs #4266, #4274.
2026-10-16 09:06:13 +08:00
morluto ca7ee018b2 fix(providers): own shared startup and retained cleanup
Give concurrent Ghidra and Hopper callers independent waits over one owned startup. Join final-caller cleanup, preserve failed cleanup owners, and derive readiness from the current startup result and live transport. Distinguish provider closure from caller cancellation and clean up interrupted final lineage observation.

Reuse the Ghidra socket fixture, replace stale cancellation health assertions with cold and warm reuse controls, and cover public MCP peer cancellation. Validation: 315 affected tests, build, typecheck, lint, docs, and 25 relocated startup controls passed. Controlled child bridges only; real Ghidra JVM and Hopper GUI verification remains unperformed. Refs #3983, #4227, #4274.
2026-10-16 08:51:11 +08:00
morlutoandVinícius DS Andrade 808339f0b1 fix(native): distinguish undecoded Swift spelling adjustments
Recognize known undecoded producer spellings without treating them as successful demangles. Preserve actual producer output and derive symbol types from the domain contract instead of duplicating its validation schema.

Compare retained output with the real demangler through MCP, while preserving CLI malformed-input and line-ending coverage.

Refs: #4215, #4287

Co-authored-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>
2026-10-16 08:26:19 +08:00
morlutoandHenglie 2f6b9ecdc6 fix(native): preserve slice identity when selecting UUID rows
Carry the selected slice name and the existing tool owner's dwarfdump spelling separately. Compare observed row labels exactly, preserving explicit ABI versions and uncertainty for missing rows.

Correct the production-boundary fixture and retain wrong-version and mixed-row controls. Real Apple multi-version selection behavior remains unverified.

Refs: #4195, #4288

Co-authored-by: Henglie <ebhenglie@gmail.com>
2026-10-16 08:26:18 +08:00
morluto 5f0d18a534 fix(ghidra): retain request limits and startup cleanup
Preserve the full request options through the FIFO and settle interrupted or rejected final lineage observation through owned startup cleanup. Keep the startup diagnostic checkpoint until finalization succeeds.

Validate typed expired ping, cancellation, child exit, runtime removal, retry, and MCP recovery without running a native Ghidra instance.

Refs: #4289, #4274
2026-10-16 08:26:18 +08:00
morlutoandHemang Varshney 018ab2c5d9 fix(javascript): retain unresolved network targets with provenance
Resolve stable lexical and URL targets using shared effects, and preserve unresolved request expressions as partial facts. Carry their uncertainty through workers, artifact graphs, and the CLI. Scope partial endpoint identity to the source site.

Refs: #3451, #3534

Co-authored-by: Hemang Varshney <218273197+hemv-857@users.noreply.github.com>
2026-10-16 08:14:52 +08:00
morlutoandGeronimo 1b9b725c4c fix(javascript): unify loader binding and mutation ownership
Model the CommonJS wrapper binding and query shared lexical identity, mutation, and evaluation timing at each call. Remove duplicate importer interpretation and preserve uncertainty for with scopes, prototypes, repeated writes, and opaque object escapes.

Validate generated fixtures against bounded Node oracles; no analyzed target execution is added.

Refs: #3982, #4072

Co-authored-by: Geronimo <varungore1@gmail.com>
2026-10-16 08:14:52 +08:00
morlutoandGeekwolf b67baede2e fix(electron): unwrap transparent inline window options
Use the shared expression owner for BrowserWindow options and webPreferences. Retain overwrite and spread uncertainty through CLI and MCP evidence.

Refs: #4250, #4281

Co-authored-by: Geekwolf <geekwolf@163.com>
2026-10-16 08:14:51 +08:00
morlutoandGeekwolf 38f92d108e fix(browser): match exact WebMCP receiver segments
Inspect AST property segments instead of flattened suffixes. Reject dotted-key and suffix collisions while retaining page-declared trust and source evidence through the CLI.

Refs: #4205, #4279

Co-authored-by: Geekwolf <geekwolf@163.com>
2026-10-16 08:14:51 +08:00
b3389a5b08 fix(mcp): preserve nested JSON values in compact array schemas
Give array unions an unrestricted item schema and validate converted tools/list schemas for both advertised profiles. Keep client constraints separate from JSON Schema validity.

Refs: #3838, #3921, #3842

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: 冰水泡枸杞 <xingzihai@users.noreply.github.com>
2026-10-16 08:14:50 +08:00
morlutoandalfredo1491 4fd28fc9e0 fix(setup): preserve JSONC trivia when removing registrations
Remove properties and their separators using parsed JSONC token spans. Preserve unrelated comments, BOM, CRLF, backups, and repeated removal behavior.

Refs: #4244, #4284

Co-authored-by: alfredo1491 <208325033+alfredo1491@users.noreply.github.com>
2026-10-16 08:14:50 +08:00
morlutoandSyed Rishad d52a488476 fix(ghidra): retain scoped failures through process finalization
Drain output within caller bounds at asynchronous completion owners and classify JVM exhaustion from the failing operation only. Preserve raw-byte scope, credential redaction, configured heap provenance, cancellation and exact cleanup failures. Validation uses controlled processes and provider fixtures; native Ghidra OOM execution was not repeated.

Refs #4241, #4275

Co-authored-by: Syed Rishad <rishadzzzz@gmail.com>
2026-10-16 07:25:23 +08:00
morlutoandLiu Jie 43bc6e1d23 fix(javascript): preserve canonical rest-array property keys
Reuse the semantic array-index parser so leading-zero keys stay unresolved instead of selecting positional arguments. Preserve canonical indices, literal fallback and dynamic-key uncertainty.

Refs #4242, #4272

Co-authored-by: Liu Jie <maxazure@gmail.com>
2026-10-16 07:25:23 +08:00
morlutoandogutdgn 89dd71f0f0 fix(analysis-view): preserve native relationship coverage
Carry the parent native coverage facts through inline and retained views independently of row-page exhaustion. Preserve uncertainty, optional diagnostics and source Evidence.

Refs #3891, #4263

Co-authored-by: ogutdgn <79801848+ogutdgn@users.noreply.github.com>
2026-10-16 07:25:22 +08:00
morlutoandY0UW1SH a05092726f fix(go): distinguish dependency coverage from parse completeness
Explain unknown dependency coverage when embedded metadata has no main-module record. Preserve decoded dependencies and distinguish GOPATH-style omissions from observed module-mode empty lists.

Refs #4191, #4267

Co-authored-by: Y0UW1SH <29498139+Y0UW1SH@users.noreply.github.com>
2026-10-16 07:25:22 +08:00
morlutoandPolly Labs 79e83cbd67 fix(setup): resolve empty Claude profile paths consistently
Use the default home registration and personal skill roots for an empty profile override. Share marker and skill path resolution while preserving nonempty caller paths, backups and idempotence.

Refs #4262, #4270

Co-authored-by: Polly Labs <pollychen.lab@gmail.com>
2026-10-16 07:25:21 +08:00
morlutoandLiu Jie 862a0b2631 fix(artifacts): reject colliding extraction paths before writing
Preserve duplicate archive occurrences in inventory and reject an ambiguous materialization before acquiring readers or output resources. Keep path collisions distinct from integrity failures.

Refs #4271, #4273

Co-authored-by: Liu Jie <maxazure@gmail.com>
2026-10-16 07:25:21 +08:00
morluto 839410b663 fix(session): report retained file and candidate cleanup
Report cleanup from the owning snapshot and provider-candidate resource scopes without retrying or starting work. Keep status accurate before a client exists and through cleanup retry.

Refs #4277, #4274
2026-10-16 07:25:20 +08:00
morluto 93c620c3e7 fix(browser): own transport cleanup through finalization
Own CDP transports from admission through observed close, retain failed
owners for retry, and keep active peers separate from pending cleanup.
Preserve primary errors and completed facts through final cancellation.
Forward passive Electron CLI cancellation and dispose browser, runtime and
scenario owners from CLI and MCP shutdown.

Refs #4255, #4256, #4257
2026-10-16 06:46:26 +08:00
morluto 3b698645a0 fix(evidence): retain completed observations on late cancellation
Validate completed reports before cancellation projection, retain valid
Evidence and captured command diagnostics, and convert raw cleanup partials
at their application owner. Record partial Evidence consistently across MCP
handlers. Keep page inspection separate from unfinished bundle analysis.

Refs #4261
2026-10-16 06:46:25 +08:00
morluto 131fd5ddcd fix(filesystem): retain stable hash cleanup owners across callers
Preserve completed digests and primary failures when descriptor close fails.
Carry the exact handle owner through provider, readiness, cache, doctor and
verifier boundaries, and retry cleanup without rehashing or retrying a raw fd.
Close candidate-owned resources even when no target client was opened.
Keep verifier workspaces when provider shutdown remains incomplete.

Refs #4266
2026-10-16 06:46:25 +08:00
morluto 034293df14 fix(docs): resolve link roots before containment checks
Normalize repository and generated-site roots at the owning boundary so
symlink aliases do not reject valid local links. Preserve missing-fragment
and escaping-path rejection.

Refs #4264
2026-10-16 06:46:24 +08:00
morluto 139d86b05c fix(verifiers): report adb operation and cleanup failures together
Close the provider on every verifier path and report primary and cleanup
failures before printing success. Preserve operation failures when shutdown
also fails instead of swallowing cleanup or exiting before it completes.

Refs #4260
2026-10-16 06:46:24 +08:00
morlutoandDeryFerd c7b70896b1 fix(dev): retain exclusive ownership through command shutdown
Terminate the directly owned Windows command on cancellation and retain the
SQLite lock until child close. Record post-spawn errors without releasing
ownership early or leaving later error events unhandled.

Refs #4258

Co-authored-by: DeryFerd <deryferdikao125@gmail.com>
2026-10-16 06:46:14 +08:00
morlutoandCodex 6183544e8b fix(javascript): select timer handles at cancellation reads
Use the shared binding-at-read owner instead of grouping schedules across the
whole binding lifetime. Match selected initializer nodes, retain uncertain
origins as candidate links, and preserve immutable closure captures.

Adapt the diagnosis in PR #3794 by @Xinglan233, whose source commit was authored
by Codex. Verify stale writes, later schedules, replacements and CLI graph links.

Refs #3358, #3794

Co-authored-by: Codex <codex@openai.com>
2026-10-16 06:06:11 +08:00
morlutoandabdooo2235 ebd5a82c43 fix(web): read WebMCP inputs from schema properties
Remove the fallback that interpreted schema metadata as parameter names.
Preserve exact literal property keys, including empty names, without inventing
names for missing or dynamic properties. Keep this an observed name projection.

Refs #3331, #3765

Co-authored-by: abdooo2235 <abdalrhmantheiv15@gmail.com>
2026-10-16 06:06:07 +08:00
396b89c5e2 fix(javascript): attribute service workers to intact platform paths
Check the complete navigator.serviceWorker.register identity through the shared
ambient-write owner, including replaced methods and aliased receiver mutations.
Reuse the module URL reader for intact registrations while preserving unknown
document-relative strings. Cover the unresolved review case and the CLI graph.

Refs #3841

Co-authored-by: Chandan Chaudhary <chandankkumar156@gmail.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 06:06:06 +08:00
morlutoandYi Han 880b3a510b fix(javascript): bound static getter reference expansion
Replace recursive getter histories with a query-owned worklist and memo. Bound
precise expansion at 512 states and widen growing receiver cycles; retain
escaped roots and constructed children rather than silently dropping suffixes.
Carry the resource limit through semantic coverage, worker transport and graph
Evidence while preserving direct writes, imports and unrelated literal facts.

The small source probe reduced 65/326 paths to one widened reference for five
and six getters. The original large Claude artifact was not rerun. Validate
receiver/alias/mutation controls and compiled CLI/MCP consumers.

Refs #3839, #3848

Co-authored-by: Yi Han <yhan606@aucklanduni.ac.nz>
2026-10-16 06:06:00 +08:00
morluto 461eb16128 fix(adb): retain cancellation at screen publication
Stop before publishing when screencap returns after cancellation. If the file
has already committed, preserve its path, digest and dimensions in cancellation
Evidence. Reuse the execution-to-Evidence owner and remove a duplicate check.

Validated both sides of publication using a valid PNG and real temporary
filesystem writes; provider and process replies are simulated.
Refs #4251.
2026-10-16 05:44:20 +08:00
morluto 1342127e91 fix(adb): bind transfer evidence to consumed snapshots
Push and install now consume private copies whose stable hashes identify the
transferred bytes. Preserve original caller paths, remote side-effect receipts
and cancellation through final cleanup. Retain failed snapshot owners in the
existing directory cleanup lifecycle and remove redundant stored identity.

Validated path replacement through real subprocess stubs and a public CLI
negative/positive reproduction, plus capacity, cancellation and cleanup retry.
The small cold/warm CLI comparison does not establish a speed improvement.
Real Android devices and Windows private-root behavior were not exercised.
Refs #4252.
2026-10-16 05:44:12 +08:00
morluto 8873b39770 fix(mcp): retain shutdown signals until cleanup succeeds
Give process signal registration a named owner and keep SIGINT/SIGTERM
listeners until successful cleanup unregisters them. Reuse existing shutdown
coalescing and failed-owner retry behavior.

Validated repeated signals with mixed cleanup results through the production
registration boundary. No real OS signal integration claim.
Refs #4253.
2026-10-16 05:43:39 +08:00
morluto 321cacb84a fix(wasm): retain cancellation through final cleanup
Observe cancellation after releasing the private runtime root and preserve
completed Evidence. Keep cleanup failure, cancellation and retry ownership
separate. Construct partial Evidence only when the outcome needs it.

Validate both successful cleanup and abort plus cleanup failure/retry. Type
the scenario tag map and give cleanup assertions live counters to prevent
missing-tag and stale-counter test debt. Tests use simulated WABT executables.
Refs #4254.
2026-10-16 05:43:31 +08:00
morlutoandLiu Jie 6b991d3aa7 fix(electron): snapshot script frame identity at ingestion
Keep each parsed script's observed frame after context destruction or ID reuse.
Remove the later context lookup so record identity and deduplication use the
same historical fact. Unknown and unauthorized frame mappings remain null.

Validated event-to-record behavior with real local files and simulated CDP
events, plus existing provider tests; no real Electron engine was exercised.
Refs #3823.

Co-authored-by: Liu Jie <maxazure@gmail.com>
2026-10-16 05:43:24 +08:00
morlutoandShitK 0ef938f4c5 fix(javascript): version syntax-aware structural fingerprints
Retain bigint/directive values, assignment operators, fixed public keys and
execution-affecting syntax flags in v3. Preserve local/private binding renames
and isolate structural matching by algorithm version. Remove the redundant
test grouping and document structural matching as heuristic evidence.

Validated AST, worker admission, archived-version matching, reconstruction and
public CLI comparisons. Auto-accessors are tested at the AST boundary because
the production JavaScript parser does not enable their Babel plugin.
Refs #3952, #4020.

Co-authored-by: ShitK <1724059440@qq.com>
2026-10-16 05:43:14 +08:00
c0d550b231 fix(reference): resolve runtime imports from their owning syntax
Distinguish call-form require from TypeScript import-equals before resolving
inventory paths. Probe CommonJS files before indexes and honor package main
without requiring an index. Retain selected manifest directory identities and
reuse the path lookup owner for canonical ambiguity.

Validated with source/reader tests and a public CLI fixture controlled by the
pinned Node loader. Windows no-follow reader behavior remains unverified.
Refs #3895, #3896, #3973, #4017.

Co-authored-by: hhang <169027046+xu91102@users.noreply.github.com>
Co-authored-by: IssueHunter Worker <issuehunter@users.noreply.github.com>
2026-10-16 05:43:04 +08:00
morlutoandhhang 2202e3f0ba fix(apple): preserve exact Interface Builder plist integers
Reuse byte-derived numeric metadata after graph classification so emitted
attributes preserve exact integers without changing UID links or identities.
Reserve metadata and attribute projection within the shared decode budget.

Validated six CLI cases covering XML, binary, collisions, real values and
incomplete metadata, plus decode-budget regressions. A small cold/warm CLI
comparison showed similar costs; it does not establish a speed improvement.
Refs #3876, #3972.

Co-authored-by: hhang <169027046+xu91102@users.noreply.github.com>
2026-10-16 05:17:59 +08:00
morlutoandLiu Jie 0123d59768 fix(browser): compare accessibility only with observed coverage
Return unknown before fingerprinting incomplete accessibility trees. Keep
known structural changes comparable when only text permission is absent,
and preserve independent changes in other dimensions.

Validated through seven CLI cases and existing network comparison semantics.
Refs #3385, #3824.

Co-authored-by: Liu Jie <maxazure@gmail.com>
2026-10-16 05:17:51 +08:00
4cc044be68 fix(browser): retain malformed inventory uncertainty
Distinguish missing CDP inventories from observed empty arrays, retain valid
accessibility and worker siblings, and admit required fields before scope
inference. Stop interpreting DOM documents without a usable string table.

Validated producer semantics and simulated CDP provider boundaries; this does
not establish real Chrome or Electron engine behavior.
Refs #3376, #3379, #3850.

Co-authored-by: 李云周 <154221037+YunzhouLi-hub@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 05:17:50 +08:00
morlutoandSriram-PR 5a590e61b1 fix(apple): scan keyed archive root values in context
Treat top-level names separately from encoded values so scalar roots named
UID or CF$UID cannot become references. Keep one shared root traversal budget
and deterministic reference paths.

Validated projection and filesystem reader regressions. The aggregate-budget
stress regression is retained for CI and was not executed locally.
Refs #3913, #4019.

Co-authored-by: Sriram-PR <160517347+Sriram-PR@users.noreply.github.com>
2026-10-16 05:17:42 +08:00
morlutoandRudy Celekli 153017d750 fix(reconstruction): preserve residual replay uncertainty
Keep overall verification unknown when an unchanged process comparison cannot
prove the absence of divergence. Preserve independently complete dimensions
and known changes. Let the named comparison schema own status consistency.

Validated through stdio MCP comparison and verification, plus domain tests.
Refs #3980, #4160.

Co-authored-by: Rudy Celekli <rudy@gradiahq.com>
2026-10-16 05:17:36 +08:00
morluto 0e1f0bf2d0 fix(lifecycle): preserve ownership, cancellation, and cleanup outcomes (#4238)
* fix(cleanup): preserve owner recovery instructions

Keep cleanup remediation through combined errors and direct or managed one-shot analysis while retaining primary diagnostics.

Refs: #4227

* fix(hopper): recover late startup and retained cleanup

Authenticate initial and cleanup handshakes through the same identity owner. Expose startup and cleanup activity, retain health after target retirement, and expire abandoned unauthenticated listeners without timing out authenticated work.

Refs: #4227

* fix(providers): retain composite lifecycle facts across retries

Preserve observed child health with incomplete peer coverage, retry only unresolved clients, and retain all child cleanup recovery actions.

Refs: #4233

* fix(process): honor cancellation through capture cleanup

Check the request signal after cleanup and before final publication, retaining completed observations under typed cancellation.

Refs: #4005

* fix(native): preserve failed LLDB launch ownership and exit proof

Publish failed-launch process identity and observed exit separately from object validity. Retire only the exact owned process and retain uncertainty when exit cannot be established.

Refs: #4232

* fix(native): honor cancellation after trace finalization

Check cancellation after runtime-root cleanup and retain collected trace observations as partial cancellation evidence.

Refs: #4009

* fix(cli): preserve analysis results when provider cleanup fails

Reuse the existing one-shot finalizer across thirteen command families to retain completed Evidence and primary diagnostics on close failure. Cover success, denied cleanup, and combined primary/cleanup failure through the public CLI.

Refs: #3935

* docs(lifecycle): define completion invariants and regression scenarios

Document distinct ownership/readiness/exit facts, late cancellation and atomic publication boundaries, partial child cleanup, and diagnostic retention. Link representative regressions and route contributor guidance through AGENTS.md.

Refs: #4227

* fix(session): retain run identity through pending cleanup

* fix(cli): preserve original cleanup resource diagnostics

* fix(native): retain completed facts through finalization failure

* fix(providers): report retained cleanup paths at their owners
2026-10-16 05:09:35 +08:00
morluto 25c22222ab fix(reference): retain unverified symlink target provenance
Keep the observed target path and failed directory in the unreadable-link diagnostic without promoting either to proven graph identity. Verify the retained path through the importer boundary.

Refs: #4013, #4053
2026-10-16 04:52:19 +08:00
morluto 921fa03fca docs(browser): explain CDP detach effects on caller emulation
Describe Chromium platform-emulation reset across connected page workflows and recovery by the owning client. Browser survival does not imply state preservation; the upstream limitation remains unresolved.

Refs: #4089
2026-10-16 04:41:40 +08:00
morlutoandTomas Galmarini f25c41b832 fix(dmg): classify localized verification failures safely
Use untranslated checksum status and nonblocking file admission for localized verification failures. Keep ambiguous readable targets unknown and bounded retries intact; retain failed descriptor ownership and cancellation.

Refs: #4146

Co-authored-by: Tomas Galmarini <tomigalmarini@gmail.com>
2026-10-16 04:41:40 +08:00
morlutoandXY-code1 d6e4fe4a4d fix(reference): preserve observed directories and unreadable links
Continue validating child directories observed before a parent enumeration failure. Reconcile unobserved link targets at the reader using canonical path identity and the nearest failed ancestor, preserving exclusions and cancellation.

Refs: #4013, #4053, #4060

Co-authored-by: XY-code1 <1611761391@qq.com>
2026-10-16 04:41:40 +08:00
morlutoandTotopo27 ee250816e1 fix(reconstruction): share artifact inspection source validation
Accept inspection Evidence through the inventory semantic owner, retaining nested substep, subject and page validation. Verify the complete CLI/MCP comparison workflow.

Refs: #3984, #4137

Co-authored-by: Totopo27 <35828647+Totopo27@users.noreply.github.com>
2026-10-16 04:41:39 +08:00
morlutoandrudrakshkarpe 10e9448d89 fix(evidence): anchor deterministic replay to unknown creation
Match repeated derived Evidence against the immutable creation revision while retaining analyst updates, conflict rejection and imported history.

Refs: #3925, #4159

Co-authored-by: rudrakshkarpe <rudraksh.karpe@gmail.com>
2026-10-16 04:41:39 +08:00
morlutoandVinícius DS Andrade 4f87f02bb1 fix(wasm): retain cooked directive literals in glue references
Collect leading directive literals using Babel cooked values, preserving source locations and selected candidate identities. Exercise escaped directives through the existing provider workflow.

Refs: #4001, #4149

Co-authored-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>
2026-10-16 04:41:26 +08:00
Ysopwnable 0cc56493be fix(javascript): preserve values captured before later assignments (#4196)
Select binding facts at reads without mixing whole-binding and temporal values. Share conditional activation rules, preserve partial expression order, and bound immutable ancestry work. Memoize temporal results within each query using initializer identity, capture point, budget and dependency proofs; preserve depth limits and source Evidence.

Fixes #3881
2026-10-16 04:35:54 +08:00
morluto e2db768012 test(pcap): verify timestamps through the named summary contract
Move the capinfos oracle to inspect_pcap_capture, whose summary retains timestamps. The field-inspection contract does not expose them. Check both CLI and MCP results against the independently observed producer values.
2026-10-16 04:05:12 +08:00
morluto edfd5025b0 test(pcap): align timestamp oracle with the provider locale
Use the same C locale as the production capinfos command so the independent real-tool oracle compares equivalent producer output.
2026-10-16 03:59:25 +08:00
morlutoandKET ⚡ 95957d748d fix(pcap): retain timestamps from supported Wireshark 4.x labels
Accept the First/Last labels used by Wireshark 4.0 and 4.2 at the capinfos metadata owner. Preserve producer values and omitted timestamps, and verify real MCP timestamp projection against direct capinfos output.

Refs #4186, #4180

Co-authored-by: KET ⚡ <ket.patil77@gmail.com>
2026-10-16 03:58:32 +08:00
morlutoandVinícius DS Andrade d9aa06d93d fix(gdb): validate complete thread-group inventories before cleanup
Parse quoted MI fields and container syntax once before interpreting live PIDs. Keep malformed or duplicate inventories unknown, preserve compatible extension fields, and bound parser state rather than treating partial syntax as proof of no inferiors.

Refs #4150

Co-authored-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>
2026-10-16 03:49:44 +08:00
morluto 94d9bffaf6 test(windows): hold sharing conflict through first close attempt
Keep the delete-sharing-denied reader held until the next event-loop turn and assert close is still pending before releasing it. This distinguishes an internal retry from a holder that releases before cleanup starts.
2026-10-16 03:49:44 +08:00
306d531f6f fix(packaging): distinguish tar line endings from filename bytes
Parse one listing separator so CRLF output matches packaged bridge paths while literal CR bytes in LF listings remain part of filenames. Retain generated-bytecode rejection.

Refs #4154

Co-authored-by: abdooo2235 <abdalrhmantheiv15@gmail.com>
Co-authored-by: Ysopwnable <161784452+N0zoM1z0@users.noreply.github.com>
2026-10-16 03:49:44 +08:00
morlutoandjackie_cqz c209dcb209 fix(artifacts): classify ZIP offsets beyond EOF as malformed
Reject negative bounded reads before allocation while retaining short-read and exact-EOF behavior. Exercise the corrupted central-directory offset through the public CLI.

Refs #4006, #3953

Co-authored-by: jackie_cqz <88996311+jackie-cqz@users.noreply.github.com>
2026-10-16 03:49:44 +08:00
morlutoandVinícius DS Andrade bfacfbac30 fix(ida): share string-page failure interpretation
Preserve nullable producer errors in the shared page contract and reject explicit failures before returning matches. Retain both successful and failed replies as partial evidence.

Refs #4151, #3990

Co-authored-by: Vinícius DS Andrade <95101505+viniciusdsandrade@users.noreply.github.com>
2026-10-16 03:49:44 +08:00
bc51d0ea5d fix(sqlite): preserve snapshot admission failure semantics
Refuse oversized database and WAL inputs before acquisition with exact observed bytes and export guidance. Keep initial path failures separate from file-set changes and retain cleanup and partial observations on late growth.

Refs #4208, #4188, #4141, #4127

Co-authored-by: hima <1ep23ic023@eastpoint.ac.in>
Co-authored-by: Royce <roycelam@umich.edu>
2026-10-16 03:49:44 +08:00
YsopwnableandKaoru0822 85eac7b92b fix(javascript): preserve analysis applicability after interruption (#4207)
Preserve applicable artifact processing phases and entered/unvisited status after interruption. Centralize JavaScript semantic applicability for full and incremental builders; retain native byte identity, actual JavaScript unknowns and structured-file provenance.

Fixes #3948. Part of #4027.

Validation:78 distinct focused cases across10files, full static/docs, and real Joplin timeout/default public CLI/MCP controls passed. Source pins, every attempt/outlier and compact resource/token projections are retained. No target/native provider execution or speedup claim.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-16 03:46:00 +08:00
morlutoandSwatantra Yadav 663d07f02c fix(server): retain session admission through investigation publication
Admit all four derived investigation workflows until Evidence recording settles, so close drains earlier work before clearing the ledger. Honor cancellation after awaited terminal progress before publishing a result.

Refs #4002, #4174

Co-authored-by: Swatantra Yadav <maverickswatantra@gmail.com>
2026-10-16 03:13:20 +08:00
morlutoandHarsH a2e30168a0 fix(mcp): relax Boolean literals without narrowing valid inputs
Normalize only advertised schema positions while keeping canonical validation. Preserve branch guidance and relax nonmonotone constraints as a unit. Verify full/compact catalogs and canonical examples after SDK conversion.

Refs #4025, #4087

Co-authored-by: HarsH <harshasalkar7@gmail.com>
2026-10-16 03:13:14 +08:00
morlutoandDeryFerd 0679a7972b fix(windows): retry owned runtime cleanup on sharing conflicts
Bound retries of native close conflicts at the retained runtime owner. Preserve exact failures and later retry authority after exhaustion. Attempt all verifier cleanup while retaining primary failures; extend held-file Windows verification.

Refs #4136, #4185

Co-authored-by: DeryFerd <deryferdikao125@gmail.com>
2026-10-16 03:13:10 +08:00
morlutoandArjun Prabhulal 8d8257928b fix(cli): validate integer tokens before coercion
Validate decimal tokens at the CLI owner before safe-number parsing. Apply the same rule to positional indexes/generations and repeated record selectors, retaining leading-zero decimals and canonical MCP inputs.

Refs #4182

Co-authored-by: Arjun Prabhulal <arjun.prabhulal@gmail.com>
2026-10-16 03:13:06 +08:00
morluto 08a897b2cd chore(ci): remove the redundant exact-test runner entry
The workflow action-pinning gate now discovers this runner through its explicit command, so Knip no longer needs a duplicate manual entry. Preserve its normal unused-code checks.
2026-10-16 02:48:45 +08:00
morluto 2ff086796a fix(browser): disclose connected platform-emulation teardown limits
Preserve the CDP detach limitation through successful and retained partial observations at their shared projector. Do not infer or replay unavailable caller user-agent/client-hint state.

This documents the reported Chromium platform reset; it does not prevent it. Source composition tests ran, without a new real-browser reproduction. Issue #4089 remains open.
2026-10-16 02:47:04 +08:00
morlutoandGabriel Dias 045c84db76 fix(pe): retain empty resources at virtual section endpoints
Map a zero-byte payload at a section endpoint only when no section contains that RVA, preserving adjacent-section ownership and raw backing checks. Extend the real CLI and stdio MCP workflow across PE32/PE32+ empty and nonempty resources.

The portable parser and public adapter regressions ran locally; Windows API loader behavior and GNU-produced images were not independently rerun.

Refs #3970

Co-authored-by: Gabriel Dias <gabrieldiasnevess@outlook.com>
2026-10-16 02:47:04 +08:00
morlutoandAdkid-Zephyr 6b1c36e336 fix(artifacts): normalize ZIP current-directory prefixes at the reader
Preserve original member spellings and central-directory identities while omitting actual root directory markers. Derive entry kind once so Unix symlink facts and size metadata agree; retain traversal and non-directory root rejection.

Verify public inspect/extract workflows for prefixed and root-only archives, distinct duplicate occurrences, and malformed/symlink controls.

Refs #4124

Co-authored-by: Adkid-Zephyr <laufs0310@163.com>
2026-10-16 02:47:04 +08:00
3406dc4d85 ci: pin Cutter actions and guard workflow-only changes (#4199)
Pin the remaining Cutter upload/download actions and enforce immutable GitHub/Docker references at actual action call sites. Run the same exact guard for workflow-only changes without preparing runtime artifacts.

Co-authored-by: Govindu Sasanka <31107592+GovinduSasanka@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 18:38:19 +00:00
morlutoandakushonkamen 5e30969cc5 fix(mcp): advertise self-contained tool inputs
Inline reused input schemas at their projection owner and remove the superseded reference-budget fallback. Validate actual SDK input and output schemas while preserving canonical parsing, output references, and projection cache options.

Claude Desktop and affected Windows connector behavior remain unverified; the SDK catalog has no schema-local input references.

Co-authored-by: akushonkamen <akushonkamen@163.com>
2026-10-16 02:23:25 +08:00
morlutoandgongfpp 5fb6edc34b fix(target): confine app program directories to their canonical bundle
Validate the canonical programs directory before resolving its executable, preserving containment failures even when the external child is missing. Retain in-bundle directory and bundle aliases.

Co-authored-by: gongfpp <22683417+gongfpp@users.noreply.github.com>
2026-10-16 02:23:25 +08:00
morlutoandArjun Prabhulal fb51960d7d fix(native): preserve Mach-O operand boundaries and integer limits
Consume signed and unsigned LEB operands through their terminators while validating 64-bit payloads and bounded extension. Preserve independent decoded metadata when a malformed stream fails.

Co-authored-by: Arjun Prabhulal <arjun.prabhulal@gmail.com>
2026-10-16 02:23:25 +08:00
claudetrail2026-droidandKaoru0822 1b828dbb10 ci(ghidra): compile the packaged bridge source cohort (#3142)
Compile the complete packaged Java source cohort against the verified SDK in selected CI. Share release and source ownership with runtime/package consumers, fail closed on drift, and remove compiler scratch on failure.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 18:11:19 +00:00
YsopwnableandKaoru0822 c7f6b8bad1 fix(javascript): bind native handoffs to canonical byte identity
Require the native node owner's content digest instead of borrowing an
observation artifact digest, which can identify the containing ASAR. Keep
unavailable nodes and their path/provenance/partial facts in the trace,
without a false actionable target or native Evidence link. Remove the old
fallback and allocate the query index only for eligible native targets.

Verify real public CLI/stdio MCP workflows, known/missing companion inputs
and shipped Joplin controls. Preserve reproducible scripts, source pins and
compact correctness/resource/token records without archiving applications.

Fixes #3926. Part of #4027.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 17:08:49 +00:00
ShuaixinHuang 633e4ff434 fix(javascript): retain static paths through TypeScript wrappers (#3998)
Normalize erased TypeScript wrappers through existing expression/string owners for static paths, call targets, collection/configuration receivers and global aliases. Preserve binding/mutation guards and explicit unknown targets, with CLI and application-boundary regressions.

Original contribution by @ShuaixinHuang.

Co-authored-by: ShuaixinHuang <1836656514@qq.com>
2026-10-16 01:04:24 +08:00
ShuaixinHuang d67d895c26 fix(jeb): isolate invalid timeout from shared startup (#3964)
Keep malformed JEB-only timeouts isolated from shared CLI/MCP startup while strict registration and JEB configuration checks remain authoritative. Cover built CLI discovery and the production stdio MCP refusal and peer requests.

Original contribution by @ShuaixinHuang.

Co-authored-by: ShuaixinHuang <1836656514@qq.com>
2026-10-16 01:03:39 +08:00
af6845cddb test(browser): preserve resource uncertainty through MCP (#3416)
Preserve resource uncertainty through production MCP registration and the real SDK. Test surviving resource fields/exclusions, forward/reverse/self comparisons, and complete-empty controls. Production behavior is unchanged.

Baron approved the unchanged head 07ea877a388ee2cd0586f04b1fb75bd104b484be; current required CI passes.

Original contribution by @ShuaixinHuang.

Co-authored-by: ShuaixinHuang <1836656514@qq.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-16 01:01:18 +08:00
7b86d5caf6 fix(process): preserve unrelated trace comparison uncertainty (#4178)
Only apply a different trace verdict to its attributed dimensions. Preserve
unrelated unknown coverage and independently observed additions/removals in
the shared CLI/MCP comparison owner.

Fixes #3931

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>
2026-10-16 00:56:55 +08:00
Kaoru0822andClaude Opus 5.5 62bf09809e fix(kuna): answer a repeated single-function request without re-running Kuna (#4169)
* fix(kuna): answer a repeated single-function request without re-running Kuna

A session dropped each single-function run once it settled, so a repeat
request for the same procedure re-ran `kuna decompile --addr`, which
analyzes the whole image again. After a failed export every pseudocode
request uses that path for the rest of the session.

Keep each completed text by entry address, within a 16 MiB least recently
used budget, until the export's reader is adopted or the session closes.
Failed and cancelled runs are not kept. The export's text still supersedes
a kept one.

Refs #1957

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(kuna): pin that a kept text's hit refreshes its recency

With two 9 MiB texts only one fit the budget, so a repeated lookup could
not change which entry was dropped. Use three 6 MiB texts: re-reading main
leaves parse least recently used, so keeping broken must drop parse.

Refs #1957

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:36:05 -07:00
Kaoru0822andClaude Opus 5.5 8686b32209 fix(browser): omit erased type-only import-equals from source-map dependencies (#4165)
Source-map original-source parsing reported `import type X = require("m")`
as a require dependency, although TypeScript erases the declaration. The
fetcher re-derived import-equals targets instead of using the existing
value-only rule. That rule now lives with the other type-only module
declaration predicates in javascriptModuleDeclaration, and the fetcher,
static analysis, and semantic analysis all use it.

Fixes #3932

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:34:18 -07:00
Kaoru0822andClaude Opus 5.5 0cfced12ea fix(reference): resolve extensionless CommonJS requires to JSON files (#4164)
The reference-source resolver probed only JavaScript and TypeScript
extensions, so require("./data") stayed unresolved when the tree held
data.json, which Node's CommonJS loader selects. Resolution now uses
extension slots: a requires edge also tries X.json after X.js and
before any directory index, and X/index.json after X/index.js,
including a package main. ESM and TypeScript imports still need an
explicit .json extension.

Fixes #3915

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:34:00 -07:00
Kaoru0822andClaude Opus 5.5 d816470bcb fix(apple): count Interface Builder hierarchy records beyond the object limit (#4162)
The ibtool dictionary parser cut the hierarchy array at a fixed 20,000
entries without counting what it discarded, so an identified record past
that position vanished while the document still reported complete
hierarchy coverage. The graph walker already owns the hierarchy limit;
the parser now passes the decoded array through.

The walker applies max_objects per identified record and keeps walking
past the limit to count omitted records, instead of stopping on a
pending structural entry. Coverage now reports the omitted count, and a
hierarchy that exactly reaches the limit is no longer marked partial.

Fixes #3950

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 09:33:02 -07:00
Guilherme Dias FusariandKaoru0822 89faa863ed test(applications): import capture fixtures with file URLs (#3050)
Convert the selected filesystem path at the fixture execution boundary. Exercise spaces and hash characters without changing real execution, CLI/MCP comparisons or the test timeout. Verified the full Linux and native Windows files with real ESM-loader negative controls.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 16:18:04 +00:00
Guilherme Dias FusariandKaoru0822 5b16f24310 test(applications): pass Evidence requests to the CLI through files (#3049)
Use existing test-owned request files for the remaining full Evidence CLI inputs. Preserve complete inputs, semantic assertions and inline transport coverage. Verified the full Linux file and native Windows baseline failures plus corrected public CLI cases.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 15:51:03 +00:00
YsopwnableandKaoru0822 9f8659e053 fix(javascript): enforce semantic byte bounds for literals
Fixes #3933. Part of #4027.

Normalize exact literals and configuration defaults through the shared primitive owner. Preserve tagged limits through member/destructuring projection and record encountered bounds in per-analysis state independently of later selected values. Remove the retained-value rescan and duplicate default reader. Preserve source Evidence, bounded/null values, aggregate wire capacity and complete endpoint lookup identities for query/trace/comparison.

Validated 66 focused and 932 semantic tests on the restored identical runtime; final public selection 7/7 adds an augmented large-endpoint acceptance (67 distinct focused cases across invocations). Full static/docs and fresh real Obsidian controls passed. Compact measurements and rejected attempts remain recorded.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 23:43:05 +08:00
KenmegeandKaoru0822 40f73f62bc test(rizin): own fake provider cleanup in shared fixtures (#3735)
Retain native fixture ownership through observed bounded cleanup and preserve files and handles when cleanup cannot be confirmed. Verified all seven files on Linux and native macOS, including cleanup failure and exit-wait controls.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 15:24:55 +00:00
Ysopwnable 6ae59f3d19 fix(verifiers): launch npm through Node on Windows (#4153)
* fix(verifiers): launch npm through Node on Windows

* test(verifiers): cover Windows npm fallback
2026-10-15 23:12:29 +08:00
Kaoru0822andClaude Opus 5.5 9d6651e5b5 fix(artifacts): never treat a directory named *.map as a source map (#4131)
Artifact inventory decided source-map edges from the `.map` path suffix,
so an archive directory such as `main.js.map/` received a maps-source
edge whose parent was the sibling `main.js`, although the same
occurrence was classified as a directory container. Edge construction
now uses the occurrence's own artifact_kind, and the duplicate suffix
rule in relationFor is removed; such directories are contained by their
parent, and real source-map files keep maps-source.

Fixes #3909

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:41:04 -07:00
Kaoru0822andClaude Opus 5.5 a90c9eac6a fix(apple): keep Interface Builder objects whose archived class is unresolved (#4118)
* fix(apple): keep Interface Builder objects whose archived class is unresolved

Interface Builder keyed-archive projection skipped every object-table
entry without a resolved class name, so an archived object whose $class
reference dangled vanished from the objects, its count and its coverage,
which still reported complete; as a hierarchy root it lost its label.
Entries that carry their own $class are archived objects: they are now
kept with class_name null and the generic kind, their fields and labels
preserved. Scalars and class descriptors remain excluded.

Fixes #3958

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(apple): keep the dangling class reference of an unknown IB object

Review follow-up. An archived object kept with an unresolved class lost
its original `$class` reference, because class resolution replaces it
with a null class name. The object's attributes now retain that raw
reference when the class cannot be resolved. A graph-level test covers
the reported hierarchy-root case: the unresolved root keeps its label,
unknown kind, containment edge, object count and coverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:40:10 -07:00
Kaoru0822andClaude Opus 5.5 cfd3c35a50 fix(kuna): keep the real lane's wide-string marker independent of layout (#4116)
On macOS arm64 the lane's UTF-8 marker ends right before the UTF-16 array,
and its last byte plus NUL decode as the UTF-16LE unit 'r'. Kuna therefore
reports "rREA wide marker", and the lane, which requires the exact text,
failed with "Required string \"REA wide marker\" was not recovered". A
leading NUL unit now ends any run the preceding bytes start, so the marker
is recovered exactly on any layout.

Refs #1957

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:38:33 -07:00
Kaoru0822andClaude Opus 5.5 3776f218e5 fix(apple): bound Interface Builder XML plists by shared value depth (#4114)
* fix(apple): bound Interface Builder XML plists by shared value depth

The Interface Builder preflight rejected XML archives whose raw element
depth exceeded 128, but the shared plist parser bounds zero-based value
depth at 128; the plist envelope and dictionary keys add elements, not
value depth. Archives whose deepest value sat at depth 127 or 128 were
therefore omitted as over budget although keyed-archive inspection
decodes them. The preflight's duplicate depth rule is removed; the
shared parser's depth limit is mapped to the existing budget omission,
so deeper archives stay explicitly omitted.

Fixes #3949

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(apple): return the decode reservation of a depth-rejected archive

Review follow-up. The removed element-depth preflight rejected an archive
before its plist representation was reserved; the shared depth limit now
rejects it during decoding, after the reservation. A large over-depth
archive therefore kept its share of the aggregate budget and could
starve later valid archives. A document omitted by the budget after
reserving now releases that reservation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:38:07 -07:00
Kaoru0822andClaude Opus 5.5 f7250fe35f fix(kuna): say why Kuna cannot answer instead of a generic message (#4113)
Kuna capability gaps such as "no code at this address" or "no function node
at this address" were built without a user message, so MCP and the CLI
showed only "This analysis is unavailable for the current target. Choose
another analysis or target." and kept the reason in details. One Kuna
helper now presents the reason as the message for every such gap, and the
unsupported-operation reason names call relationships and xrefs, which Kuna
now answers.

Refs #1957

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:37:20 -07:00
Ysopwnable cd64e24ff2 fix(verifiers): align WABT operation timeout (#4117)
* fix(verifiers): align WABT MCP request timeout

* fix(verifiers): bound packaged WABT workflow
2026-10-15 22:29:15 +08:00
Abdul Samad SiddiquiandKaoru0822 a40ee741c2 test(ghidra): align MIPS verifier ABI and snapshot expectations
Fixes #3415. Retain independently inspected GNU attribute facts and compare exact snapshot wire fields. Preserve strict metadata comparisons, specific incomplete-ABI refusals and source immutability checks.

Original implementation by @AbdulSamad94.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 22:00:16 +08:00
619 changed files with 55333 additions and 5669 deletions

No files matched your search

+13
View File
@@ -60,6 +60,7 @@ jobs:
web_runtime: ${{ steps.plan.outputs.web_runtime }}
web_source_map: ${{ steps.plan.outputs.web_source_map }}
nativeaot: ${{ steps.plan.outputs.nativeaot }}
ghidra_bridge: ${{ steps.plan.outputs.ghidra_bridge }}
website: ${{ steps.plan.outputs.website }}
matrix: ${{ steps.plan.outputs.matrix }}
test_selection: ${{ steps.plan.outputs.test_selection }}
@@ -139,6 +140,11 @@ jobs:
- name: Validate workflow syntax and expressions
if: needs.changes.outputs.workflows == 'true'
run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes=
# Code tiers run this source-only policy test in their existing shards.
# Workflow-only changes need the guard, without preparing runtime artifacts.
- name: Check action pins for workflow-only changes
if: needs.changes.outputs.workflows == 'true' && needs.changes.outputs.code != 'true'
run: node scripts/run-exact-tests.mjs run tests/boundary/filesystem/workflowActionPinning.test.ts
- name: Record static task source and execution identity
continue-on-error: true
if: needs.changes.outputs.code == 'true'
@@ -741,6 +747,7 @@ jobs:
tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts
tests/boundary/filesystem/webScriptManifestPaths.test.ts
tests/boundary/process/ProcessEvidence.host.test.ts
tests/boundary/process/packageNpmInvocation.test.ts
tests/boundary/process/windowsOutputCapture.test.ts
tests/acceptance/webModuleCli.test.ts
tests/acceptance/webSourceLocationCli.test.ts
@@ -898,6 +905,11 @@ jobs:
if: needs.changes.outputs.nativeaot == 'true'
uses: ./.github/workflows/nativeaot-fixtures.yml
ghidra_bridge:
needs: [changes, static]
if: needs.changes.outputs.ghidra_bridge == 'true'
uses: ./.github/workflows/ghidra-bridge-sources.yml
website:
needs: [changes, static]
if: needs.changes.outputs.website == 'true'
@@ -933,6 +945,7 @@ jobs:
web_runtime,
web_source_map,
nativeaot,
ghidra_bridge,
website,
]
runs-on: ubuntu-24.04
@@ -0,0 +1,48 @@
name: Ghidra bridge source compilation
on:
workflow_dispatch:
workflow_call:
permissions:
contents: read
env:
HUSKY: "0"
jobs:
ghidra-bridge-sources:
name: Compile packaged Ghidra bridge sources
runs-on: ubuntu-24.04
timeout-minutes: 15
env:
GHIDRA_RELEASE_URL: https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_12.1.4_build/ghidra_12.1.4_PUBLIC_20260921.zip
GHIDRA_RELEASE_SHA256: ddac49f903da9d5bac833e5cc79395098b9c33cfd3279be5f31bd00387d2d4db
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Validate the SDK pin against the provider's verified release
run: node scripts/verify-ghidra-bridge-sources.mjs --check-sdk-pin >> "$GITHUB_ENV"
# Only the jars and application.properties are extracted: the check
# compiles against them and never launches Ghidra or imports a target.
- name: Acquire checksum-verified Ghidra jars in isolated runner storage
shell: bash
run: |
ghidra_root="${RUNNER_TEMP}/rea-ghidra"
mkdir -p "${ghidra_root}"
curl -fsSL --retry 3 --write-out 'ghidra_download_bytes=%{size_download} ghidra_download_seconds=%{time_total}\n' -o "${ghidra_root}/ghidra.zip" "${GHIDRA_RELEASE_URL}"
echo "${GHIDRA_RELEASE_SHA256} ${ghidra_root}/ghidra.zip" | sha256sum -c -
unzip -q "${ghidra_root}/ghidra.zip" '*/Ghidra/*.jar' '*/Ghidra/application.properties' -d "${ghidra_root}"
rm "${ghidra_root}/ghidra.zip"
properties="$(find "${ghidra_root}" -path '*/Ghidra/application.properties' -print -quit)"
test -n "${properties}"
echo "GHIDRA_INSTALL_DIR=$(dirname "$(dirname "${properties}")")" >> "${GITHUB_ENV}"
# JAVA_HOME_<major>_X64 comes from the runner image, not the env context.
- name: Compile bridge sources with the verified runner JDK
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
test -n "${!java_home_variable}"
JAVA_HOME="${!java_home_variable}" npm run verify:ghidra:bridge-sources -- --verified-sdk
+3 -3
View File
@@ -22,7 +22,7 @@ jobs:
sudo apt-get update
sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
- run: npm run build:windows-native
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: cutter-windows-native
path: native/windows/build/
@@ -53,7 +53,7 @@ jobs:
uses: ./.github/actions/npm-cache
- run: npm ci
- if: matrix.platform == 'Windows'
uses: actions/download-artifact@v4
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: cutter-windows-native
path: native/windows/build/
@@ -67,7 +67,7 @@ jobs:
env:
EXPECTED_PLATFORM: ${{ matrix.platform }}
run: npm run verify:cutter
- uses: actions/upload-artifact@v4
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: always()
with:
name: cutter-upstream-${{ matrix.platform }}
+2
View File
@@ -20,6 +20,8 @@ Give each interpretation and resource a clear owner. Consumers must use that own
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
Assess brittleness in touched code during implementation, debugging, review, and maintenance. Fix the producing representation or ownership rule before adding downstream repairs. Normalize once at the owning boundary using format-aware parsers. Remove superseded representations, aliases, wrappers, and redundant tests after verifying affected consumers. Introduce abstractions only when they remove observed complexity.
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
+9 -1
View File
@@ -1057,6 +1057,10 @@ def _serve_connection(connection):
):
raise PermissionError("Invalid bridge capability")
authenticated = True
# The connection grace period only bounds unauthenticated idle
# clients. Native analysis and a connected owner's idle time
# remain unbounded after capability validation.
connection.settimeout(None)
_write_message(file, {"id": request_id, "event": {
"type": "progress",
"phase": "hopper_bridge",
@@ -1148,13 +1152,17 @@ def _run():
server.bind(REA_SOCKET)
os.chmod(REA_SOCKET, 0o600)
server.listen(1)
# A bridge can start after its caller's startup/cleanup deadline. Bound
# the first accept as well as reconnects so an orphan cannot pin Hopper's
# dedicated Python thread and prevent later documents from starting REA.
server.settimeout(REA_RECONNECT_TIMEOUT_SECONDS)
try:
while True:
try:
connection, _ = server.accept()
except socket.timeout:
break
connection.settimeout(None)
connection.settimeout(REA_RECONNECT_TIMEOUT_SECONDS)
if _serve_connection(connection):
break
# Match REA's cleanup handshake budget without pinning Hopper's
+48 -4
View File
@@ -555,6 +555,38 @@ def _failed_launch_description(process):
return None
def _retire_failed_launch(process, config, listener, stdout, stderr):
"""Reconcile an SBProcess returned alongside a failed Launch error."""
if not process.IsValid():
stdout.close()
stderr.close()
return False
pid = process.GetProcessID()
if pid > 0:
# Even an already-exited process needs its PID published so the host
# can distinguish a completed launch failure from an unknown target.
with open(config["pid_path"], "w") as handle:
handle.write(str(pid))
if process.GetState() != lldb.eStateExited:
stdout.mark_launched()
stderr.mark_launched()
if pid > 0:
# Hold a surviving target until the host has captured its start
# identity. If that proof is unavailable, Kill still addresses
# this exact SBProcess created by the current launch request.
_wait_for_identity_ack(config)
process.Kill()
_wait_for_exit(listener, process, 5)
exit_observed = process.GetState() == lldb.eStateExited
expected_exit = exit_observed
stdout.close(expected_exit=expected_exit)
stderr.close(expected_exit=expected_exit)
return exit_observed
def trace(debugger, config):
global _TRACE_CONTEXT
debugger.SetAsync(True)
@@ -630,8 +662,9 @@ def trace(debugger, config):
)
process = target.Launch(_launch_info(target, config, listener), error)
if not error.Success() or not process.IsValid():
stdout.close()
stderr.close()
exit_observed = _retire_failed_launch(
process, config, listener, stdout, stderr
)
return {
"status": "launch-error",
"error": error.GetCString(),
@@ -640,15 +673,26 @@ def trace(debugger, config):
if process.IsValid()
else {}
),
**(
{"target_process_exit_observed": True}
if exit_observed
else {}
),
}
launch_failure = _failed_launch_description(process)
if launch_failure is not None:
stdout.close(expected_exit=True)
stderr.close(expected_exit=True)
exit_observed = _retire_failed_launch(
process, config, listener, stdout, stderr
)
return {
"status": "launch-error",
"error": launch_failure,
"target_process_created": True,
**(
{"target_process_exit_observed": True}
if exit_observed
else {}
),
}
stdout.mark_launched()
stderr.mark_launched()
+132
View File
@@ -15,6 +15,8 @@ fake_lldb = sys.modules.setdefault("lldb", types.ModuleType("lldb"))
fake_lldb.LLDB_INVALID_ADDRESS = -1
fake_lldb.eLaunchFlagDebug = 2
fake_lldb.eLaunchFlagStopAtEntry = 4
fake_lldb.eStateExited = 10
fake_lldb.SBEvent = lambda: object()
spec = importlib.util.spec_from_file_location(
"rea_lldb_tracer", os.path.join(os.path.dirname(__file__), "rea_lldb_tracer.py")
)
@@ -23,6 +25,136 @@ spec.loader.exec_module(tracer)
class NativeTraceBudgetTests(unittest.TestCase):
def test_failed_launch_does_not_infer_exit_from_a_failed_kill(self):
class Process:
def IsValid(self):
return True
def GetProcessID(self):
return 12345
def GetState(self):
return 6
def Kill(self):
return None
class Sink:
close_exit = None
def mark_launched(self):
pass
def close(self, expected_exit=False):
self.close_exit = expected_exit
with tempfile.TemporaryDirectory() as directory:
stdout, stderr = Sink(), Sink()
with patch.object(tracer, "_wait_for_identity_ack", return_value="unavailable"), patch.object(
tracer, "_wait_for_exit", return_value=None
):
exited = tracer._retire_failed_launch(
Process(), {"pid_path": os.path.join(directory, "pid")}, object(), stdout, stderr
)
self.assertFalse(exited)
self.assertFalse(stdout.close_exit)
self.assertFalse(stderr.close_exit)
def test_failed_launch_publishes_pid_and_retires_live_process(self):
class Process:
state = 6
killed = False
def IsValid(self):
return True
def GetProcessID(self):
return 12345
def GetState(self):
return self.state
def Kill(self):
self.killed = True
self.state = tracer.lldb.eStateExited
class Sink:
launched = False
close_exit = None
def mark_launched(self):
self.launched = True
def close(self, expected_exit=False):
self.close_exit = expected_exit
with tempfile.TemporaryDirectory() as directory:
pid_path = os.path.join(directory, "pid")
process = Process()
stdout, stderr = Sink(), Sink()
def acknowledge(config):
with open(config["pid_path"], "r") as pid_file:
self.assertEqual(pid_file.read(), "12345")
return "readable"
with patch.object(tracer, "_wait_for_identity_ack", acknowledge):
exited = tracer._retire_failed_launch(
process,
{"pid_path": pid_path},
object(),
stdout,
stderr,
)
self.assertTrue(exited)
self.assertTrue(process.killed)
self.assertTrue(stdout.launched)
self.assertTrue(stderr.launched)
self.assertTrue(stdout.close_exit)
self.assertTrue(stderr.close_exit)
def test_failed_launch_records_when_valid_process_is_already_exited(self):
class Process:
def IsValid(self):
return True
def GetProcessID(self):
return 12345
def GetState(self):
return tracer.lldb.eStateExited
def Kill(self):
raise AssertionError("an exited process must not be killed")
class Sink:
close_exit = None
def mark_launched(self):
raise AssertionError("an exited process has no output to drain")
def close(self, expected_exit=False):
self.close_exit = expected_exit
with tempfile.TemporaryDirectory() as directory:
pid_path = os.path.join(directory, "pid")
stdout, stderr = Sink(), Sink()
with patch.object(
tracer,
"_wait_for_identity_ack",
side_effect=AssertionError("exit is already observed"),
):
exited = tracer._retire_failed_launch(
Process(), {"pid_path": pid_path}, object(), stdout, stderr
)
with open(pid_path, "r") as pid_file:
self.assertEqual(pid_file.read(), "12345")
self.assertTrue(exited)
self.assertTrue(stdout.close_exit)
self.assertTrue(stderr.close_exit)
def test_pre_target_integrity_failure_reports_no_process_created(self):
original_debugger = getattr(tracer.lldb, "SBDebugger", None)
+10
View File
@@ -108,6 +108,13 @@ The `replaced` result is `null`: the successful install response does not say
whether the package was installed fresh or replaced, and REA does not query the
device first. The install verdict and host APK digest remain available.
Push and install consume a private snapshot of the selected local input. The
reported size and SHA-256 identify those consumed bytes; `local_path` and
`apk_path` retain the original input path, whose contents may change later.
Snapshot creation needs temporary disk space proportional to the input size.
Failed snapshot cleanup remains owned and is retried before later operations
and during shutdown.
## Interpret the result
- **Directory names retain whitespace and newlines.** Symlink targets do too,
@@ -168,6 +175,9 @@ device first. The install verdict and host APK digest remain available.
- Cancellation during a package pull preserves caller-selected output. The
error includes partial observations identifying completed artifacts and the
interrupted local path; remaining APKs were not attempted.
- Screen capture checks cancellation before publishing its exclusive output.
If cancellation arrives after publication, the error retains the committed
file's path, digest and dimensions; the file remains available as evidence.
- Bugreports collect into isolated staging and publish with exclusive creation;
filename collisions select the next numeric suffix. If publication fails, the
completed archive remains in staging. The error's partial observation gives
+18
View File
@@ -147,6 +147,24 @@ received signal alone does not establish its exact cause. Dynamic tags require
a complete DT_NULL inside PT_DYNAMIC; interpreter names and ranges end at the
first NUL, with any padding still represented by the original segment range.
## Ghidra process failures
When output from the failing Ghidra startup or operation reports
`java.lang.OutOfMemoryError`, REA returns `resource_constraint`. Java heap or GC
overhead exhaustion includes heap-specific recovery advice. Metaspace and array
size failures do not recommend increasing the heap; native-thread exhaustion is
reported as a capacity constraint with unknown limits.
`configured_jvm_max_heap` records the explicit `-Xmx` value selected by REA's
launch builder. It is a configuration fact, not measured memory use or proof of
the effective JVM limit; JVM environment options may affect that limit.
When a Windows script wrapper selects the JVM, this value remains unknown.
Historical stdout and stderr remain available, but classification uses only the
failing operation's `failure_output`. Output draining waits at most one second,
subject to earlier cancellation or the startup deadline. `output_incomplete`
marks a snapshot taken before stream closure; cleanup ownership remains with
the session until release succeeds.
## Portable PE resources
`inspect_pe_resources` / `inspect-pe-resources` inspects an explicit PE32 or
+19 -8
View File
@@ -66,14 +66,25 @@ Electron `file://` pages use a separate provider and target boundary; see [elect
## Provider authorities
CDP page capture, V8 Inspector observation, and Playwright scenarios stay
separate providers because their authorities differ: CDP and V8 attach to an
already-running target and never launch, drive, evaluate, or mutate it, while
Playwright providers own and drive the runtime (launch browsers, run actions,
capture step snapshots). Merging them would mix attach-only and owned-process
lifecycles in one contract. What they share instead is one exclusion
vocabulary (`BrowserExclusionReason`): every denial reason in every stack maps
onto it, even where a wire schema keeps a narrower historical bucket.
CDP page capture and V8 Inspector observation attach to an already-running
target without launching it, driving actions, or evaluating target code.
Playwright scenarios drive actions and capture step snapshots in either a
provider-owned browser or a caller-selected external browser. Separate providers
preserve these authority and lifecycle differences. They share one exclusion
vocabulary (`BrowserExclusionReason`): every denial reason maps onto it, even
where a wire schema keeps a narrower historical bucket.
CDP attachment is not a guarantee that the browser preserves all state.
Chromium can clear another client's `navigator.platform` emulation when REA's
page session disconnects, even if that session sent no emulation commands.
This affects page inspection, bundle capture, session observation, screenshots,
WebMCP discovery, and Electron page inspection; target listing alone does not
attach a page session. REA cannot read the other client's complete
user-agent/client-hint configuration and does not replay guessed settings.
Use a disposable browser when preserving externally configured emulation is
required, or have the owning client reapply its complete configuration afterward.
See [the reproduction and Chromium ownership limitation](https://github.com/morluto/rea/issues/4089).
Existing static application Evidence and passive web/Electron captures can be
combined later through
[JavaScript static/runtime reconciliation](javascript-runtime-reconciliation.md).
+9
View File
@@ -144,6 +144,15 @@ distinction: it closes and deletes provider-owned profiles, but only disconnects
from an external CDP browser. Real-browser verification checks both outcomes and
confirms that an attached external browser remains alive.
Keeping that browser alive does not guarantee its emulation state survives.
Chromium can clear caller-owned `navigator.platform` emulation when a page CDP
session disconnects, including sessions that sent no emulation commands. REA
cannot snapshot the other client's complete user-agent/client-hint settings and
does not replay a partial reconstruction. Use launch mode to keep this effect
inside a provider-owned browser, or have the external owning client reapply its
complete configuration after capture. Connected captures report this limitation;
see [issue #4089](https://github.com/morluto/rea/issues/4089).
`environment.locale` sets Intl formatting in both modes. Launch also sets
`navigator.language`, `navigator.languages`, and the request `Accept-Language`.
Connect preserves the caller's page and request languages; every attached
+6
View File
@@ -168,6 +168,12 @@ be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
for import, client timeouts and recovery.
If one-shot analysis fails and closing its session also fails, the original
error details remain inline and `details.cleanup_error` describes the later
close failure. Each diagnostic retains its own reported resources; the later
failure does not establish whether an earlier resource was released. Captured
output and partial observations remain in the original details.
## Save and reuse analysis snapshots
A snapshot retains successful analysis results for later queries. REA reuses
+6
View File
@@ -2,6 +2,12 @@
REA can attach to an Electron/Chromium CDP endpoint and inspect existing `file://` renderer pages without evaluating JavaScript or invoking Electron APIs. The request supplies the endpoint directly; REA accepts only literal-port loopback HTTP endpoints. Selecting an endpoint exposes every Electron target it serves, including local file paths and page metadata.
Disconnecting the inspection's page session can clear caller-owned
`navigator.platform` emulation inside Chromium. REA cannot restore another
client's complete user-agent/client-hint configuration. Use a disposable target
or let the owning client reapply its configuration; see the shared
[CDP state limitation](browser-observation.md#provider-authorities).
This passive runtime surface is distinct from the target-free static
[`analyze_javascript_application`](javascript-artifact-reconstruction.md)
workflow. Static analysis reads a local ASAR or extracted directory without
+6
View File
@@ -139,6 +139,12 @@ overview merely because its source artifact is unchanged.
## Admitted function-analysis semantics
Concurrent callers share one startup while retaining independent cancellation.
Only the final departing caller abandons startup and waits for its cleanup.
New attempts wait for that cleanup; failed cleanup retains its original error
and owned directories until a successful close. Closing the provider reports a
provider-state failure to live callers rather than cancelling their signals.
| Concern | Ghidra contract |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Decompiler lifetime | One persistent `DecompInterface` is opened for the imported Program and disposed during bridge shutdown. Native decompilation has no fixed per-function deadline. External functions or functions without bodies return `null`; cancellation and native failure remain distinct. |
+9 -2
View File
@@ -39,8 +39,8 @@ container format, architecture, word size and byte order. A recognized
- `module.dependencies`, with the same fields and their original order;
- `module.settings`, retaining ordered key/value entries and duplicate keys;
- `module.unparsed_lines`, `module.unparsed_line_bytes_base64` and
`module.complete`, so unfamiliar, malformed, conflicting or undecodable records
do not disappear behind a complete result.
`module.complete`, which says whether every present module-text line was
parsed. It does not establish that the dependency inventory is complete.
Missing checksums are `null`; empty fields and development/local replacement
versions remain unchanged. Build settings such as `GOOS`, `GOARCH` or `vcs.revision`
@@ -50,6 +50,13 @@ base64 while other decodable module records are still reported; the reader does
not insert replacement characters or reject the entire artifact for those
strings.
When no main-module (`mod`) record was decoded, `module.main` is `null` and
dependency coverage is unknown. Successfully decoded dependency records remain
visible. GOPATH-mode builds may contain only a package `path` and build settings;
their empty dependency array does not establish the absence of dependencies.
The result's limitations state this explicitly. A module-mode build with a main
module record and no dependency records has an observed empty dependency list.
## Supported boundary
The built-in TypeScript reader supports ELF, PE and thin Mach-O images with Go's
+24 -6
View File
@@ -179,6 +179,8 @@ as well, so it is broader than this Claude-only migration.
history under `CLAUDE_CONFIG_DIR`. Setup follows the same
variable: it writes the registration to `$CLAUDE_CONFIG_DIR/.claude.json` and
the skill to `$CLAUDE_CONFIG_DIR/skills`.
For REA setup, an empty `CLAUDE_CONFIG_DIR` is treated as unset: registration
goes to `~/.claude.json` and the skill goes to `~/.claude/skills`.
```bash
CLAUDE_CONFIG_DIR=~/.claude-rea npx rea-agents setup --client claude_code
@@ -635,6 +637,11 @@ bridge readiness. For large targets, set `REA_HOPPER_STARTUP_TIMEOUT_MS` to a
decimal integer from 1 to 2,147,483,647 milliseconds in the environment that
starts REA. CLI and MCP use the same setting; invalid values fail configuration
validation. An MCP client's own request timeout must also allow enough time.
`open_binary` validates and selects a target without waiting for native provider
readiness. During Hopper startup, `binary_session.provider_operation_health`
reports `busy` at stage `launch`; `analysis_activity` describes bridge requests
and can remain idle until the socket is ready. Cleanup work reports `busy` at
stage `connection`.
If macOS denies Apple Events, allow the application that starts REA under
**System Settings → Privacy & Security → Automation**, then retry. REA releases
@@ -644,9 +651,12 @@ Arguments containing quotes, backslashes, or line breaks remain unchanged but
cannot establish that a failed handoff opened no document. Other failed handoffs
retain ownership until document closure is confirmed. If a timed-out bridge
becomes ready later, `close_binary` connects to that run, authenticates its
identity, and requests document shutdown. After a connection drops, the bridge
allows 30 seconds for an authenticated cleanup reconnect, then releases Hopper's
Python execution thread. Expiry does not establish document closure, so REA
identity, and requests document shutdown. Once its listener is ready, the bridge
allows 30 seconds for the first connection and capability validation. After a
connection drops, it allows the same grace period for an authenticated cleanup
reconnect, then releases Hopper's Python execution thread. Initial analysis is
outside this listener grace period, and authenticated native work is not bounded
by it. Expiry does not establish document closure, so REA
retains unconfirmed ownership. Retry close while the bridge remains available.
If a Linux launcher exists but cannot start, inspect its shared libraries:
@@ -736,9 +746,17 @@ Once document and process closure are confirmed, the lease can be released even
if temporary-file cleanup fails. Another client can then launch, but the owning
client must finish its retained cleanup before starting again.
An unconfirmed external document can be retried while its authenticated bridge
remains connected. If that bridge has disconnected, another close cannot confirm
the document: inspect and close the reported document in Hopper before ending
the owning REA connection.
remains available. The same authentication and process-identity capture apply
when the first successful handshake occurs during cleanup after a startup
timeout. REA can confirm closure if that exact authenticated process exits,
including while a shutdown request is in flight. A helper exit alone is
insufficient, and an application that never authenticated has no verified
process identity. If no matching bridge becomes available, inspect and close
the reported document in Hopper before ending the owning REA connection.
Export the Evidence bundle before `close_binary` clears session records.
`binary_session.cleanup_pending` remains true while provider cleanup is retained,
and provider health remains visible even with `open: false`; that flag alone
does not establish successful cleanup.
### Hopper in CI
+11
View File
@@ -68,6 +68,17 @@ The node vocabulary covers:
- runtime script instance;
- an explicit `unknown` kind when classification is not justified.
Static network endpoints can have `resolution: "partial"` when the analyzer
recognizes a request call but cannot resolve its target to one exact value.
The observation's `value` then preserves the source expression, and its
evidence points to the call site. Partial and exact endpoints have distinct
identities even when their displayed values match. Partial identities also retain
the source site: matching expression text at different sites does not establish
the same runtime target. This records observed source syntax; it does not
establish that the call ran or contacted that endpoint.
Graph coverage describes the analyzed artifact scope and resource limits, not
runtime execution. A partial target alone does not imply omitted source.
The directed relation vocabulary is `contains`, `loads`, `imports`, `maps_to`,
`exposes`, `sends`, `invokes`, `handles`, `calls`, `persists_to`, `observed_as`,
and `changed_from`. Provider names are deliberately absent from both
+33 -2
View File
@@ -95,6 +95,14 @@ matches exactly. Otherwise the result recommends provider-neutral follow-up
tools and reports `requires-provider-analysis`; it never starts or switches a
provider implicitly.
A handoff requires the add-on's exact content identity. Unavailable unpacked
companions remain visible in the trace graph with their canonical path,
availability and container-backed Evidence. They yield no actionable handoff
until their bytes are available and the application is reanalyzed; the archive
digest does not identify the missing add-on. An empty handoff list does not prove
that the trace contains no native components. Source partial coverage remains
visible independently of handoff eligibility.
## Semantic relation tracing
`trace_javascript_semantics` queries the authenticated companion graph without
@@ -117,6 +125,17 @@ registrations/removals and dispatch, Node timers and cancellation handles, async
defaults, request construction and response consumers, parse/coercion/
validation boundaries, and built-in resource acquisition/release.
Local callback candidates do not establish a complete target set when a
conditional write may leave a parameter, catch binding, or uninitialized
variable unchanged. These calls retain candidate edges and dynamic uncertainty;
default traces remain ambiguous with partial coverage. Opt into
`include_ambiguous_dynamic_edges` to traverse the candidates. Proven preceding
unconditional assignments, including writes dominating a read in the same
conditional block, can resolve a target; later writes do not replace an
earlier read or captured alias. Unproved ordering across branches or function
activations remains conservative. Returned-reference effects use the same
read-position facts as call resolution.
Mutation tracking covers explicit member assignments, updates, deletion, and
loop assignment targets, including the supported local aliases and shared
nested references. Calls conservatively invalidate exposed mutable references;
@@ -166,8 +185,13 @@ graph and read literal values from `properties.value`, rather than decoding
role keys or labels. Previously stored graphs and their original IDs remain
valid inputs.
Endpoint nodes keep the exact endpoint in each observation's `properties.value`;
semantic request and response nodes keep it in `properties.endpoint`. Endpoint
Endpoint observations keep their value in `properties.value`. When
`properties.resolution` is `partial`, that value is the unresolved source
expression, with an identity distinct from an exact endpoint of the same text.
Semantic request and response nodes keep their endpoint in `properties.endpoint`. Endpoint
lookup identities are preserved independently of the primitive-value lattice's
byte budget. An oversized normalized literal can become unknown while the same
string remains an exact endpoint for tracing and comparison. Endpoint
identity keys use `value-sha256:<digest>` over the canonical JSON string when
that is shorter, or when the original value starts with the reserved digest
prefix. Other short endpoint keys remain unchanged. Endpoint labels use the
@@ -248,6 +272,13 @@ Direct return expressions, including expression-bodied arrows, are evaluated
through an execution-free value lattice. Literal object fields and direct
return sites are represented in the result. Calls, dynamic spreads, computed
keys, and parser recovery remain partial or unknown.
Rebound values are evaluated at their read position: saving an incoming parameter
before a later assignment preserves caller-dependent uncertainty, while a proven
preceding assignment can supply a constant. Unresolved ordering remains unknown.
Supported reads within object and array literals follow left-to-right evaluation;
an unproved relative order remains unknown. A later write in a separate conditional
block can still leave an earlier capture unknown because ordering across those
blocks is not proven.
Objects and arrays passed to calls, constructors, or tagged templates, used
as method receivers, or stored through property targets are not assumed
unchanged afterward. Property stores conservatively retain reference escape
+50 -12
View File
@@ -182,6 +182,18 @@ partial Evidence. Cancellation after source analysis begins can also retain
completed facts, including when interrupted during cleanup, graph construction,
or Evidence sealing. Unexamined relationships remain unknown.
Per-file gaps identify the applicable processing phase: JavaScript source analysis
uses AST authority, while package/JSON, HTML-reference and source-map processing
use artifact-byte authority. File-analysis interruption gaps carry `analysis_status`:
`interrupted` means the phase was entered (including source-worker startup),
and `not-started` means interruption prevented that phase from beginning.
Neither state establishes that parsing completed. Remaining parsers are not run
after interruption just to populate a result.
Native add-ons remain metadata-only and acquire no JavaScript parse or analysis
gap. This also applies when their paths are outside `source_paths`; their inventory
and caller selection remain available without implying applicable AST work.
MCP returns a `partial_observation` retained reference after recording succeeds;
open it with `inspect_analysis_view` on the same connection. If the SDK stopped
waiting before receiving the reply, follow [MCP cancellation recovery](mcp-contracts.md#progress-and-cancellation)
@@ -299,13 +311,25 @@ suffix. Utility calls such as `useUtils().invoice.list.fetch()`, `useQueries`,
and links whose callee was renamed by a minifier are not recognized.
Each recovered bundle module retains the exact factory-source digest. A complete
bounded AST also receives a `babel-ast-v2` structural fingerprint that ignores
ordinary identifier names while retaining syntax, literals, operators, object
keys, and member properties. Computed member identifiers retain their access
syntax without treating a local variable's spelling as a fixed property; template
literal contents remain part of the fingerprint. Version comparison keeps the
algorithm in its matching key, so retained `babel-ast-v1` fingerprints cannot match
these new fingerprints. Re-analyze both snapshots to compare their new structure.
bounded AST also receives a `babel-ast-v3` structural fingerprint. It ignores
ordinary identifier spellings while retaining literal and directive values,
operators, fixed property names, and selected syntax flags such as computed
keys, declaration kind, function async/generator state, update-expression
prefix, method kind, and class static state. Local names used in computed keys
remain rename-resistant; private member names are also ignored while their
method kind and static state remain represented.
This fingerprint is structural evidence for candidate matching, not proof that
two modules behave equivalently. Its scope follows the AST facts listed above;
unrepresented syntax can still collide. The exact source digest remains
available to show byte-level change, and comparison reports content changes even
when structural matching succeeds.
Version comparison includes the algorithm in its structural matching key.
Retained `babel-ast-v1` results can still match other v1 results, and v2 results
can still match v2 results, but v1/v2 results cannot structurally match v3
results. Other matching tiers remain available across algorithm versions.
Re-analyze both snapshots to compare using the v3 rules.
If fingerprint construction reaches its structural
bound, the fingerprint is unavailable and its status is `truncated`; a bounded
prefix is never presented as a complete stable fingerprint.
@@ -402,10 +426,13 @@ For `.node` bindings, member names mean “requested by JavaScript syntax.” A
resolved add-on path does not convert them into verified native exports. Native
symbol verification remains a separate deep-analysis claim.
Endpoint observations preserve useful local diagnostics, including query values
and fragments, while removing only URL username/password credentials. Artifact
paths, digests, parse locations, and analysis metadata remain actionable
because REA is local-only.
Exact endpoint values preserve useful local diagnostics, including query values
and fragments, while removing URL username/password credentials when the value
parses as a URL. When a recognized request target cannot be resolved exactly,
the endpoint's partial resolution retains the original source expression with
its call-site provenance. This does not establish that the request executed.
Artifact paths, digests, parse locations, and analysis metadata remain
actionable because REA is local-only.
## Safety boundary
@@ -437,7 +464,18 @@ retained candidate strings and temporary canonical-key serialization. This is
an in-memory semantic allocation bound independent of CLI or MCP transport
budgets. Source literal bytes remain in the parsed source and artifact evidence;
when a normalized semantic value exceeds the bound, its value is unknown and
coverage is partial. Expression
coverage is partial. Direct string literals, cooked templates without
substitutions, and derived strings all use this normalization boundary. A
single exact primitive is validated without allocating deduplication or sort
state or serializing a canonical key. Configuration defaults use the same
normalizer; limited defaults retain `value_resource_limit` and an anchored
resource-limit unknown. Known null, false, zero, and empty-string defaults stay
exact. This primitive-value bound does not truncate lookup identities such as
source paths, field names or request endpoints. Full endpoint properties remain
available for endpoint queries, tracing and comparison. Each analysis records encountered limit classifications independently of
later projections: a selected healthy sibling can remain exact while whole-file
coverage reports the encountered bound. Projection preserves limiting values
from the shared property-slot owner. Expression
evaluation and provenance walks stop after 256 nested levels. Union products
and string lengths are checked before allocating combinations or concatenated
strings. When a boundary is reached, the value stays unknown, semantic coverage
+1
View File
@@ -48,6 +48,7 @@ The first text or assembly request in a session looks for this target's export i
- REA starts `kuna decompile-project` in the background on a private copy of the target.
- Meanwhile, pseudocode comes from one `kuna decompile --addr` run. Name and address lookups, and listings, use the inventory the export takes with `kuna functions --json` before it decompiles. The session runs `kuna functions` itself, under `REA_KUNA_TIMEOUT_MS`, only when no export is building (an inventory-only request does not start one) or the export ends without an inventory.
- Requests for the same function share one run; it stops when its last waiting request is cancelled. Each run analyzes the whole image again in its own process, so one REA process runs at most as many at once as the host has cores beyond `REA_KUNA_JOBS` (at least one). Further requests wait in order, and a waiting request is answered from the export if it finishes first. A run whose stop REA cannot prove keeps its slot until `close_binary` proves it stopped.
- The session keeps each completed single-function text, up to 16 MiB in all with the least recently used dropped first, and answers repeats from it until the export can. After a failed export that lasts for the session.
- Assembly and data names wait for the export; progress notifications report the wait every ten seconds. A session does not rebuild a failed export: requests that need it fail with `retryable: false`, the export's failure, its captured Kuna output and what to change before reopening the target. A setting such as `REA_KUNA_PROJECT_TIMEOUT_MS` or `REA_KUNA_JOBS` is read when REA starts, so changing it also needs a restart. Pseudocode keeps using single-function runs.
- A single-function answer says so in its limitations. It lacks the export's whole-program prototypes, so a later request can return different text for the same function.
+32 -3
View File
@@ -393,15 +393,26 @@ concrete version, selection source, and complete analysis profile through
choices return typed selection details. A selected provider is never replaced
automatically after a runtime failure.
Every successful target transition allocates `binary_session.analysis_run.run_id`
before any provider startup. `process_lineage` is `not_observed` until a dynamic
provider starts, then becomes `snapshots` with every started provider's identity
Each provider client receives `binary_session.analysis_run.run_id` before any
provider startup, including clients whose startup fails. `process_lineage` is
`not_observed` until a dynamic provider starts, then becomes `snapshots` with
every started provider's identity
and retained ownership observation. Each observation records `observed_at` and
is `unavailable` with a reason when ownership could not be revalidated, or
`verified` with launcher PID, parent PID, process group, and descendants observed
at that bounded check. These are historical snapshots, not live process
inventories, and do not claim that no short-lived descendant existed.
`binary_session.cleanup_pending` reports whether the session retains a provider
client, session file resource, or provider-candidate resource whose cleanup has
not succeeded. It can be true with `open: false`; target retirement does not
establish resource release. Health, activity, and lineage remain available
under a retained client's original `analysis_run.run_id` where the provider
supplies them. Successful client cleanup clears the retained run; a replacement
client receives a fresh run ID. Follow the cleanup error's reported resources
and recovery action before reopening; a missing health observation does not
establish an idle provider.
## Progress and cancellation
REA accepts ordinary `tools/call` progress tokens. The numeric `progress` counts
@@ -428,6 +439,10 @@ Native call tracing and process capture retain available observations in
`details.partial_observation` on failure, including when cleanup succeeds.
The observation reports its partial coverage; cleanup details describe host
state separately from the execution failure.
For these tools, cancellation observed during result finalization or owned
cleanup also returns typed cancellation with the available observations.
A normal target exit does not establish request completion. Incomplete cleanup
still takes precedence and retains its resource diagnostics.
A process capture that fails during a finalization interval keeps every signal
attempt and its delivery result in `details.partial_observation`, with
`elapsed_ms` and `exit_code` null when no exit was observed.
@@ -694,6 +709,20 @@ after SDK conversion and check advertised validation and actual calls.
Individual model APIs can impose additional nesting limits; complete producer
captures can exceed ten structural levels.
Boolean literal constraints in input schemas are relaxed only in this
advertisement. For example, a boolean `const` is presented as `type: boolean`;
an affected `not` is omitted, conditional predicates and their branches are
omitted together, pure boolean-literal `oneOf` branches collapse to a boolean,
and other affected `oneOf` branches are presented as `anyOf` to retain branch
guidance without making overlapping widened branches reject inputs.
This can make the advertised schema accept inputs the canonical contract
rejects. The MCP handler still validates every call with the canonical
contract. The Gemini CLI verifier checks the outgoing
`parametersJsonSchema` and canonical contract examples against that declaration
using a loopback model fixture; it does not establish live Google API or
OpenCode client behavior. Set `REA_VERIFY_GEMINI_SCHEMA_PROFILE=compact` to
verify the compact advertisement through the same client boundary.
### Compact input schema profile
Some function-calling providers reject a request while any single tool's
+22
View File
@@ -451,6 +451,11 @@ The run ends when the process exits, `max_events` entries are recorded,
Bounded termination reports partial coverage. REA checks process termination
using a launch-time OS start identity; if ownership or cleanup cannot be
verified, the result or error preserves that uncertainty. The result reports:
Launch errors keep the fact that LLDB created a process separate from whether
it remains alive. An exited `SBProcess` is reported as exited only after LLDB
confirms that state; unresolved ownership remains retained for cleanup retry.
Cancellation observed while output and Evidence are finalized returns partial
cancellation Evidence, even if the target has already exited.
- the outcome, exit status and captured stdout/stderr (1 MiB kept per stream
while excess bytes are drained and counted without growing capture files).
@@ -481,6 +486,11 @@ with partial coverage. Available output prefixes remain inline; their byte
counts are observed lower bounds when final drain counters are unavailable.
The original failure classification and cleanup uncertainty remain separate.
An interrupted journal row is ignored with an explicit limitation.
If the trace completed before cancellation or runtime-directory cleanup failed,
the retained observation also keeps its exit status, termination proof, launch
identity, breakpoint results and final stream counters. Those fields remain
absent when the interrupted producer did not report them; partial coverage alone
does not mean that every retained fact is incomplete.
Only entries are observed: return values, floating-point and stack arguments,
and inlined or `objc_direct` calls are not. The target runs with the current
@@ -577,6 +587,18 @@ Exact and ambiguous value counts describe emitted root/object value instances;
a root and an object may contain the same value. Archive digests identify the
original bytes.
Interface Builder's XML and binary plist archives use the same number metadata
and exact-decimal representation for emitted node attributes. Projection follows
object and connection classification, so archive IDs, UID links and graph counts
keep their meaning. Precision limitations name the source document. Unsafe
integer-literal counts cover the archive, including fields not emitted as node
attributes; exact and ambiguous projection counts cover emitted attribute values.
Integral reals stay numbers, and ambiguous or incompletely observed values retain
explicit precision limitations. Before plist decoding, the existing aggregate
decode budget reserves estimated room for number metadata and attribute
projection; these estimates are not a hard process-memory bound. Native
NIBArchive decoding is unchanged by this plist-specific representation.
Dates are reported as `{ "$plist_type": "date", "iso8601": "<timestamp>" }`. A
date the decoder cannot represent, such as an XML date with a year beyond 9999
or a negative year, is reported as `{ "$plist_type": "date", "iso8601": null }`
+3 -1
View File
@@ -425,7 +425,9 @@ results for unaffected dimensions and mark affected dimensions unknown; an
incomplete capture cannot locate the first divergence across all dimensions.
Trace assertions use coverage for the sources they select, so an assertion
about complete raw terminal output need not fail because rendered snapshots
were omitted. Captures missing `truncation_details` are rejected on import;
were omitted. A trace difference applies to the dimensions identified by its
diagnostic; unrelated dimensions retain their own comparison, including unknown
coverage. Captures missing `truncation_details` are rejected on import;
preserve them as historical files and recapture for current comparisons.
`limits.file_bytes` is already the cumulative whole-file hashing budget,
independent of `limits.output_bytes`, which bounds retained terminal content.
+14
View File
@@ -105,6 +105,20 @@ overall native heap and total resident memory are not capped. Row limits do not
override these byte and resource limits. Resource-limit failures do not return
an incomplete schema as successful complete evidence.
If the combined database and WAL exceed the snapshot budget, the refusal is a
`resource_constraint` and reports both the configured byte limit and the exact
observed combined size as a decimal string. The remediation directs callers to
create a smaller consistent offline snapshot containing only relevant data
through the application's supported export workflow. A generic backup or copy
does not guarantee a smaller result; REA does not truncate or modify the
selected source.
On hosts that cannot address the derived WAL or rollback-journal path for a
long selected database name, REA reports that the SQLite capability is
unavailable. It does not assume an inaccessible sidecar is absent. Use the
application's supported consistent offline export or copy workflow to place
the complete known file set at a shorter path.
MCP also applies its configured response budget to the complete Evidence
envelope. If delivery exceeds that budget, it reports a transport constraint
with guidance for exporting retained Evidence, requesting a matching larger
+51 -1
View File
@@ -199,6 +199,44 @@ Measure slow cases before pruning capacity coverage. Retain inputs beyond the
formerly failing size/depth and distinguish cold runs, warm builds, coverage,
and toolchain conditions when comparing timings.
## Lifecycle regressions
For changes to resource acquisition, readiness, cancellation, or cleanup, select
the affected owners and applicable scenarios below. Include temporary snapshots
and one-shot commands. Use public CLI/MCP workflows where practical and a
controlled production seam for interleavings that cannot be reproduced reliably.
| Scenario | Assertion and control |
| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Target admitted; startup still pending | Status preserves the readiness distinction and observed startup activity; readiness requires the provider's documented proof. |
| First handshake is late, absent, invalid, or recovered during close | Initial and recovery paths retain the same authenticated identity facts. Foreign run/token controls cannot authorize cleanup; authenticated idle/native work survives the unauthenticated listener deadline. |
| Launch returns an error with a valid process handle | Cover already-exited and still-live owned processes. Verify observed exit separately from creation; failed termination or unavailable identity remains unresolved. |
| Cancellation arrives after workload exit, during result reads or temporary-root removal | Return typed cancellation with available partial observations and actual cleanup status. The non-aborted control succeeds; existing primary/cleanup failures keep their precedence. |
| Children have mixed health or cleanup outcomes | Missing peer health does not erase an observed failure or activity. Idle plus unknown stays unknown. Retry a failed child without closing a successful peer again, including after a close-option change. |
| Cleanup fails after success or primary failure | Public output retains the completed observation or primary diagnostic, remaining resources, and owner recovery action. Restore the denied resource and verify the owning cleanup retry. |
| Target exits without an acknowledgement, or transport disappears while it stays alive | Positive owned exit proof resolves target lifetime; other resources still need cleanup. Lost transport alone does not prove exit. Verify unrelated/attached targets survive and same-connection recovery works when supported. |
Synchronize cancellation with the finalization boundary being tested; a fixed
delay after dispatch does not establish that phase. Use markers, barriers, or
owned cleanup seams while preserving real cleanup where feasible. Check server
settlement and retained resources even if the MCP SDK rejects the client promise
before the handler finishes. A cancellation during active execution does not
cover cancellation during final cleanup.
For atomic writes, exercise cancellation on both sides of the documented commit
point and preserve any publication receipt. A late interruption does not prove
that no output was published.
Representative regressions include
[process cleanup-time SIGINT](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/process/processCaptureCancellationAdapters.test.ts),
[late authenticated cleanup](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/providers/hopper/hopperClient.handshakeRecovery.test.ts),
[LLDB launch and finalization](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/providers/native/lldbCallTracerFailures.test.ts),
[composed child ownership](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/src/application/binary/CompositeProvider.test.ts),
and [public CLI cleanup denial](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/cli/wasmCleanupCli.test.ts).
These controlled probes do not replace the matching real-provider lane. Record
the base failure, passing control, exact host/provider, verified cleanup, and
unavailable coverage; a missing engine is not a pass.
## Behavioral depths
| Depth | Location | What it proves |
@@ -989,7 +1027,19 @@ the Apple or Windows curated lanes invoke explicitly, which also select that
lane. Edits to verifier entrypoints run only by manual, licensed or self-hosted
workflows run static checks and the Linux suite. Website changes run website
checks. Provider workflow changes validate the workflow and run that provider's
verifier. Manual-provider/Android workflow changes run workflow validation only.
verifier. Changes under `bridge/ghidra/` also select the `ghidra_bridge` lane,
which compiles the packaged bridge sources against the pinned Ghidra 12.1.4
jars with `npm run verify:ghidra:bridge-sources`. It never launches Ghidra or
imports a target, so a missing import fails the PR instead of every Ghidra
session. The launcher, compiler and package checks share named required source
roles. The manifest must match the complete Java cohort, so deleting a helper
or adding an unregistered dependency cannot silently change the shipped set.
The preflight reuses the provider's Java Properties parser with the pinned Node
toolchain's native type stripping, without an npm install or runtime build.
The shared release policy owns the verified version and JDK major. CI checks
the archive pin before downloading and verifies the extracted SDK version;
changes to these source owners also select the compile lane.
Manual-provider/Android workflow changes run workflow validation only.
The four-version SQLite compatibility matrix has its own `sqlite` scope for
SQLite implementation, contracts, CLI/MCP registration, fixtures, and shared
runtime boundaries. SQLite and server-lifecycle test edits and the SQLite npm
+48
View File
@@ -68,6 +68,54 @@ satisfy. A plan without execution does not establish runtime behavior.
constraints. Report pagination, truncation, cancellation, and partial failure.
An assumed agent budget is insufficient justification for a limit.
## Own the lifecycle
For tools that acquire resources, identify the owner and the observations that
establish each transition. Include temporary files and one-shot commands, as
well as persistent engine sessions. Use the existing provider contracts;
providers need not expose identical state machines.
- Keep target admission, provider readiness, active work, observed process exit,
and resource release separate. A valid process handle, successful launcher,
or selected target establishes only its own fact. Publish available identity
and exit observations even when launch fails; retain unknown ownership.
- Authenticate and capture identity through the same boundary on initial and
recovery handshakes. Bound abandoned unauthenticated listeners where the
protocol permits it; that deadline must not expire authenticated idle sessions
or long-running native work. Listener expiry does not prove target closure.
- Shared startup has one producer and independently cancellable caller waits.
Keep it running while any caller still needs it. The final departing caller
waits for owned cleanup and retains its failure. Provider closure is distinct
from caller cancellation. Retained cleanup capabilities block replacement;
derive readiness from the current startup's completion and live connection.
- For cancellable observations, carry cancellation through result consumption
and awaited cleanup. Before returning success, check the signal after the
last awaited finalization step.
Retain collected observations under typed cancellation. Preserve an existing
primary failure and let incomplete cleanup retain its own classification.
Client cancellation alone does not prove native work stopped.
For atomic publication, document the cancellation cutoff and preserve any
committed output's publication receipt.
- Release only owned resources. Attaching owns a connection or session unless
the provider contract explicitly establishes target ownership. A helper exit,
missing reply, or missing PID does not establish target exit.
- Retain successful child cleanup and retry only unresolved children, including
when close options change. Preserve observed child failure or activity when
another child's health is unavailable; idle plus missing coverage stays
unknown.
Bind cleanup failures to their exact resource owner. Resolve current cleanup
uncertainty only after that owner confirms release, while preserving the
original failure and completed observations. Later failure composition must
report only resources whose cleanup remains unresolved.
- Await one-shot cleanup without discarding the completed result or primary
diagnostic. Preserve partial observations, remaining resources, and the cleanup
owner's recovery instructions. CLI adapters should reuse
[`finishAnalysisCommand`](https://github.com/morluto/rea/blob/main/src/cli/finishAnalysisCommand.ts) where its contract
applies rather than duplicate close/finally handling.
Exercise the applicable [lifecycle regression scenarios](testing.md#lifecycle-regressions)
through the producing owner and affected CLI/MCP consumers.
## Implement and evaluate
Put provider protocols/parsing in adapters and shared CLI/MCP workflows in the
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,245 @@
# Interrupted application processing: applicability and Evidence
Tracks #3948 under #4027. Measured baseline:
`c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb`.
The question is which processing phases remain unavailable after a source
interruption, while keeping native inventory and already collected facts.
Native identity and provider linkage must stay independent of application-wide
partial coverage. This is static artifact analysis; no target application or
native provider runs.
## Observed failure and correction
The interruption producer previously attached one generic failure to every
remaining relevant file. Graph projection interpreted it as JavaScript AST work.
Real Joplin reproduced an `analyze-javascript-source` / `ast-static-analysis`
unavailable scope on keytar's native binary and on unvisited package, HTML and
source-map files. Public CLI/MCP controls reproduced the same applicability
problem. Deterministic cancellation before native metadata additionally produced
a phantom JavaScript semantic module for the binary.
The producer now records the applicable operation/authority and whether the
phase was entered. `analysis_status: interrupted` includes worker startup and
does not imply parsing completed; `not-started` identifies unvisited phases.
Native add-ons need no post-inventory source-processing phase, so neither interrupted
nor caller-excluded native files acquire AST scopes or semantic modules.
Actual JavaScript failures, structured-file unknowns, caller selection and byte
identity remain available. The graph consumes these produced facts.
The AST-only unavailable Evidence helper is replaced by an explicitly typed
static helper. Normal package/JSON/map gaps also retain their artifact-byte
authority. Per-file progress uses the same applicability owner: JavaScript keeps
its source phase, and metadata uses `process_javascript_artifact_metadata`.
Remaining parsers do not run after interruption. Failure sharing is limited to
five applicable phases during one interruption; no persistent cache is added.
A second review found the alternate full-analysis semantic builder still
created modules for every excluded file kind. An actual filesystem-producer
regression on revision `323fae8cbf3e36677bcb7b22ba31fa1a7e3c5cd4` reproduced
phantom native/JSON/HTML/package/source-map modules, while selected and excluded
JavaScript were positive controls. Semantic applicability now belongs to the
shared `recordUnavailableFile` projection entry point, before retaining limits,
nodes, roots or unknowns. Three duplicate caller checks are removed. Both full
and incremental construction use this boundary; actual excluded JavaScript
continues to carry source-bound unknowns and partial semantic coverage.
## Real input and reproduction
Use the [official Joplin v3.7.21 acquisition and extraction instructions](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/docs/verification/javascript-native-handoff-identity-2026-10-15.md).
The AppImage SHA-256 is
`5d202dcaca31e14206826aa3f04125e7c218dbc3dc3d98479d811efbd6b856c1`;
ASAR SHA-256 is
`767df409b18effe60421fbefcca873de237c7e813eac744ecdeebbc5e825a33a`.
The 6,324-file inventory includes 46 unpacked entries. Both cohorts stage all
three native companions; the other 43 unpacked entries remain unavailable.
Build with pinned Node 24.18.0/npm 11.16.0 and locked dependencies. The extended
existing verifier also runs against the baseline runtime; copy the verifier and
`scripts/lib/javascript-file-scope-oracle.mjs` from the fixed revision into a
built baseline checkout when reproducing the failure.
```bash
artifact_path="/absolute/path/to/cases/available/app.asar"
common=(
--artifact "$artifact_path"
--addon-path node_modules/keytar/build/Release/keytar.node
--native-sha256 0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a
--source-path ElectronAppWrapper.js
--source-path ElectronAppWrapper.js.map
--source-path index.html
--source-path package.json
--source-path node_modules/keytar/build/Release/keytar.node
--structured-path ElectronAppWrapper.js.map
--structured-path index.html
--structured-path package.json
--structured-expectations '{"ElectronAppWrapper.js.map":{"operation":"parse-local-source-map","analysis_status":"not-started"},"index.html":{"operation":"parse-html-script-references","analysis_status":"not-started"},"package.json":{"operation":"parse-package-json","analysis_status":"not-started"}}'
--rounds 3 --cli-parity
--tokenizer-python /absolute/path/to/python-with-tiktoken-0.12.0
)
node scripts/verify/javascript/native-handoffs.mjs "${common[@]}" \
--analysis-timeout-ms 1 --expect-timeout --require-safe
node scripts/verify/javascript/native-handoffs.mjs "${common[@]}" \
--validate-structured-from-export --require-safe
```
Omit `--require-safe` on the baseline timeout run to record its incorrect
applicability. Its diagnostic exit 0 is not a passing analyst task. A supported
1 ms deadline includes worker startup; the verifier requires a typed timeout
and retained partial reference for that cohort. Each structured path supplies an
independent expected operation and phase-entry status; the safety oracle matches
all gaps exactly, including unavailable state and unknown confidence. Missing or
conflicting expectations do not pass.
The native path is explicitly selected. An earlier structured-only run excluded
it and therefore did not reproduce the native interruption. Input selections
and that preliminary result remain recorded. Both final cohorts preserve
partial coverage for unselected/unavailable scopes.
The supplied native Evidence is simulated solely to check exact subject linkage.
The verifier checks physical companion bytes and the exported parent's root
digest, validates public CLI/MCP native-trace parity, and reports only after
owned transport/command/private-file cleanup. Applications and large Evidence
are regenerable and are not archived.
## Two distinct task projections
The **timeout task** includes the expected typed-timeout preparation, one inline
native trace and three inline structured-path traces. It checks that native has
no AST failure and that each unvisited structured phase remains unavailable with
the appropriate artifact-byte authority and `not-started` status.
The **default-deadline control** models the summary plus one inline native trace.
Its named structured paths are additionally validated from the already exported
complete parent, explicitly labelled `exported-parent-validation`. Those checks
are diagnostic overhead outside the modeled native-task tokens and inside total
verifier resources. They are not presented as inline metadata answers.
A control that also requested the complete outgoing `package.json` feature trace
hit stdio admission twice: at least 10,587,373 bytes against a 10,484,736-byte
result budget, with identical retained parent/trace identities. It is not a
completed equivalent task or an efficiency baseline. [#4190](https://github.com/morluto/rea/issues/4190)
records the separate need for exact retained metadata/native fact views. Complete
export remains available. Do not compare costs across these different tasks.
## Measurements
| Cohort | Revision | Wall s | CPU s | Peak cgroup bytes | Applicability |
| ------------------------------ | -------- | ------: | ---------: | ----------------: | ------------- |
| Timeout, four inline traces | baseline | 60.738 | 60.289464 | 3,092,045,824 | wrong scope |
| Timeout, four inline traces | final | 57.515 | 57.089883 | 2,763,276,288 | pass |
| Default control, native inline | baseline | 61.542 | 61.135522 | 2,808,336,384 | pass |
| Default control, native inline | final | 186.085 | 142.371100 | 2,959,585,280 | pass |
| Defined task | Selected catalog + parsed/structured replies BPE | Selected catalog + complete tool replies BPE |
| -------------- | -----------------------------------------------: | -------------------------------------------: |
| Timeout | 21,136 → 20,161 | 36,468 → 34,471 |
| Native control | 11,776 → 11,720 | 17,738 → 17,632 |
The full-catalog alternatives and per-query samples are preserved in JSON.
Native content identity, observed byte provenance, source selection and handoff
provider Evidence remain equal. Corrected global provenance changes derived
parent/trace identities; null partial-Evidence subjects remain null. Both final
runs have exact CLI/MCP native-trace parity and match the earlier post-fix
controls in every recorded non-timing field.
The whole verifier includes input hashing, analysis, three native trace rounds,
applicable structured queries, export, CLI parity and tokenization. Every heavy
run uses the preserved [resource observer](https://github.com/morluto/rea/blob/b3bdabab694af1f9801eff7caf75001b92314a58/experiments/analysis-efficiency/resource-run.py):
one CPU, 8 GiB hard cap, 6 GiB early stop and no swap. Cgroup peaks include charged
file cache, not summed RSS. Counters are observed at payload exit while the
observer remains alive, not final whole-scope counters. Completed runs verify
owned cleanup and report no OOM.
Each MCP process is fresh, but input hashing warms its file cache. Build/OS caches
and shared-host scheduling are uncontrolled. First and repeated retained-query
wall samples remain separate; isolated query CPU/memory are unmeasured. An
earlier fixed timeout took 150.173 s wall / 119.468013 s CPU; its exact unchanged
runtime repeat took 58.827 s / 58.482996 s, with identical non-timing facts,
Evidence and BPE. Both receipts are retained. The final default control also took 186.085 s wall / 142.371100 s CPU; its
unchanged-runtime repeat took 62.651 s / 62.240964 s with identical non-timing
results. The slow final row remains reported. Attribution remains unresolved;
these runs establish no speedup or performance-equivalence bound.
The stricter post-review verifier reruns also passed without changing production
code: timeout 162.645 s wall / 125.876114 s CPU / 2,809,643,008 B peak; default
control 61.367 s / 60.970077 s / 2,526,949,376 B. Both preserve the original final
analytical fields, identities, coverage and BPE exactly; only timings and the new
explicit oracle-input field differ. These additional receipts do not replace the
slower rows above.
After centralizing semantic applicability, both real public controls passed again:
timeout 60.443 s wall / 59.970344 s CPU / 3,078,295,552 B peak; default control
63.574 s / 63.177884 s / 3,189,719,040 B. Analytical results, identities, coverage,
BPE and exact CLI/MCP parity remain unchanged. JSON identifies the runtime and
verifier source manifests for each cohort and keeps all earlier receipts.
Static `cl100k_base`/tiktoken 0.12.0 projections include the selected or complete
name/description/inputSchema catalog, requests and structured Evidence or parsed
text errors. Complete-tool-reply alternatives include MCP text/structured
duplication. The selected two-tool catalog is 5,570 BPE; the complete projection
is 195,522. Output schemas, host rendering/replay, notifications, model reasoning,
final answers and billing are unmeasured. Repeated rounds, export, CLI parity,
tokenization and exported-parent diagnostics are validation overhead outside the
defined task tokens but inside resource receipts. Removing unsupported claims
from the incorrect timeout baseline earns no efficiency credit.
## Verification and limitations
The initial production fix had 49 distinct passing cases across eight files. The initial
pre-progress-cleanup seven-file selection passed 44/44. After progress cleanup,
the same selection passed 43 and hit the existing 30-second test deadline in one
native identity case; an unchanged two-file rerun passed both native cases and
five public CLI cancellation cases. The original deadline was not increased.
The runtime verifier collects interruption-bearing scopes; separate regressions
cover native caller exclusion and malformed structured data. Source selection,
malformed structures, actual worker resource constraints,
retained large partial observations, cancellation/publication/cleanup, native
identity and both public adapters remain covered.
The first review found that the real verifier previously accepted any non-JavaScript
operation with artifact-byte authority and ignored phase-entry status. The
scenario now supplies exact operation/status expectations independently of the
production phase map. Six additional oracle cases use the actual retained
Joplin snapshots and reject wrong metadata operations, entered or missing
statuses, and absent or conflicting gaps. They passed, bringing the distinct
coverage to 55 cases across nine files. The production implementation did not
change during this review correction.
After the shared semantic-boundary correction from the second review, 78
distinct cases passed across ten files: the nine-file selection passed 65/65,
and the original root reconstruction file passed 13/13. This includes the
alternate-builder owning test file's 23 cases. The new producer-boundary
regression checks both construction paths, all five excluded metadata/native
kinds, and selected/excluded JavaScript controls. Real Joplin controls exercise
the public incremental workflow; the alternate full-builder path has filesystem
boundary verification rather than a separately measured real-app workflow.
Recorded attempts also include the verifier's initial non-null-subject assumption
and an invalid top-level semantic-module-path absence check. Both were corrected
before their assertions were used as proof. Corrected cancellation reproduction
shows the actual native AST scope and phantom module on the baseline. Final
source and execution-driver versions, source pins, input digests, prior attempts
and resource receipts are in the [compact JSON record](javascript-interruption-applicability-2026-10-15.json).
Before review, `npm run check && npm run docs:check` passed, including five consistency
tests: 165.553 s wall / 150.247257 s CPU /
4,232,491,008 B peak cgroup memory, clean owned exit and no OOM.
The first full check stopped on a 172-line test callback exceeding the 150-line
rule. A named assertion helper now shares scope lookup across all four public
results; the strengthened six-case file passed again before the final gates.
After the review correction, the full static/docs gate passed again, including
the five consistency tests: 111.395 s wall / 110.979710 s CPU /
4,347,125,760 B peak, with clean owned exit and no OOM.
After centralizing semantic applicability, the final full static/docs gate also
passed, including five consistency tests: 117.636 s wall / 117.194118 s CPU /
4,263,153,664 B peak, clean owned exit and no OOM.
At the initial PR revision, 36 CI checks passed; macOS `hdiutil create` failed
while creating its DMG fixture before REA verification, also failing the aggregate
check. No Apple fixture/workflow code changed. That attempt provides no passing
Apple runtime evidence.
Live Electron/application execution, real native-provider behavior, other-platform
runtime coverage and hosted-agent costs are **NOT_ENOUGH_INFO**. The verification
does not claim them.
@@ -0,0 +1,547 @@
{
"base_revision": "e9a71efa31d6c3675b67938b8a79fa4ddf01604c",
"scope": "Semantic per-value string-byte normalization; real static shipped-application control, no hosted-model performance claim",
"runtime": {
"platform": "Linux x64",
"node": "24.18.0",
"npm": "11.16.0",
"tiktoken": "0.12.0"
},
"baseline_source_sha256": {
"src/domain/javascript/javascriptSemanticValues.ts": "df3fc30b5ba91901a85fe56f200355930da32d054bb026a747e3414af233b6b5",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "c12b29f0faf4b6511777cba7ed8ac007c6d6170d657f01e43e194feacb3e0601",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0203d207e0d6bb8aaa17746090f008876a11653afd660fce3dc44f3fda646a6c",
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1"
},
"final_source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "844d4639401a9f050ce42c80192572a6d0d74fe4f81b37e93a2c20aaca9fb9a6",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a3f802ca3446c2b5376cc58b19d818eff318e6775bd47752793c3f649994f8a2",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "2e78717d30e637b5fc5bab6d0eff063f7a62e0199d33697c1cdfaf8317baa698",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "8f9cf1a36f05b03dd816eaf9a6cdba0160d34ad7d17a961206e1bc32444d2ccc",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "e69408f2f5910a0107fd5d0cae0ff60ed706448857bc8e535c37cac1a0cd4d37",
"src/domain/javascript/javascriptSemanticIr.ts": "68dad360d0a9e6bd72108b2717cae756b84ca1efb48455248ded2ff699d2c113",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0993a397f2a2a48c2b6c35494e940b3fce5c2c9032cfe33c7cf0401128c79dde",
"src/domain/javascript/javascriptSemanticState.ts": "bfcface5dcda22594f47ed390efa15f9140fe41cb2d5995896e52dd0ee31b978",
"src/domain/javascript/javascriptSemanticValues.ts": "a4ba51cc60fff38ef11fd5af35a2e1ad753cdb2dea9d44895f91d89320c5ddeb",
"tests/acceptance/applications/javascriptEndpointPayload.test.ts": "b3cfcde1c5b5e0698f84afd51f4b53e42e56101267707c20c0d038320f5cfbd2",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "070106115b482269794799e448bd20b8e02df1f511f4ceb2b8fa75612c7f78e4",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"artifact": {
"path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f"
},
"selected_source_paths": ["package.json", "main.js"],
"where": {
"path": "main.js",
"kind": "javascript-module"
},
"identical_parent_evidence_id": "ev_73ef06e3b9b944f54c5257f7960279127d8518d2a73cd8618bd045dbe44e6b5a",
"cli_parity": {
"evidence_id": "ev_b3586dfdf7d4f01157f8f2978326aaa7103987110f2df4fa5bf3331ff2898749",
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"exact_cli_mcp_view_parity": true
},
"unchanged_query_shapes": {
"legacy": {
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
"references": {
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
}
},
"unchanged_token_costs": {
"tokenizer": "cl100k_base",
"tiktoken_version": "0.12.0",
"costs": {
"legacy": {
"selected_catalog": 7977,
"full_catalog": 195522,
"common_request": 53,
"common_summary": 2675,
"requests": 2836,
"structured_replies": 71761,
"selected_catalog_task_total": 85302,
"full_catalog_task_total": 272847
},
"references": {
"selected_catalog": 7977,
"full_catalog": 195522,
"common_request": 53,
"common_summary": 2675,
"requests": 102,
"structured_replies": 5741,
"selected_catalog_task_total": 16548,
"full_catalog_task_total": 204093
}
}
},
"cohorts": {
"baseline": {
"preparation_ms": 19089.119622,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 383.8260569999984
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 15.778886999996757
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 13.15173700000014
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 240.44363500000327
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 346.04796499999793
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 13.727306999997381
}
],
"resources": {
"elapsed_seconds": 33.394,
"cpu_seconds": 33.016802,
"peak_cgroup_bytes": 1979863040,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"final": {
"preparation_ms": 19061.292544,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 344.6316079999997
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 14.438351000000694
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 11.38763100000142
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 298.41579600000114
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 211.45682099999976
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.275370999999723
}
],
"resources": {
"elapsed_seconds": 32.949,
"cpu_seconds": 32.559187,
"peak_cgroup_bytes": 1880096768,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
}
},
"conditions": "First query is after shared artifact analysis; subsequent queries share one MCP session. Legacy item reads are serial; a concurrent legacy baseline is unmeasured. Filesystem/build warmness uncontrolled. Token costs count one post-change catalog, preparation request/summary and each request/structured reply once; client rendering, replay, reasoning, final answer and model latency are unmeasured. Preparation and parity/tokenization overhead are outside per-query elapsed times; external family-resource receipts include the complete verifier and children. Before/after used separate fresh MCP sessions. CPU/memory includes analysis, query arms, export, CLI parity and tokenization; cgroup memory includes file cache, not summed RSS. Counters are at payload exit while observer lives, not final scope counters. One before/after control establishes no speedup or equivalence bound. Catalog BPE projection excludes annotations/output schemas.",
"resource_policy": {
"memory_max_bytes": 8589934592,
"early_stop_bytes": 6442450944,
"cpu_quota_percent": 100,
"swap": false
},
"observer_sources_sha256": {
"resource-run.py": "7728862171415e7dee2c11af2ead2b9a54265fccaf82a7113956d36b29065e33",
"resource_payload.py": "6f089e645459eba80c5d73e39c21606c78173a44daf49fabc3b88c4631ce51c6"
},
"regressions": {
"original_main": "Quoted/template oversized literals failed through public CLI/MCP; addition/small controls passed. Corrected domain boundary reproduced 2 failures/16 passes.",
"review_head": "8 failed/25 passed before fixes for projection tags, encountered-but-discarded limits and bare quoted/template defaults; includes exact null default control.",
"interim_review_fix": "65/66 focused passed; old doubling-chain expectation omitted a genuine expression-depth limit now observed. Adjusted that assertion to preserve the byte-limit requirement while allowing other observed classes.",
"final_focused": "66 passed across five files on the restored 38d263a runtime. Final corrected public selection passed 7/7 in two files: the six literal/default/member cases plus the augmented original large-endpoint acceptance. 67 distinct focused cases across invocations; not a single 67-case invocation.",
"final_semantic_suite": "932 passed in 36 files on 38d263a. Final production src tree (including these domain tests) verified byte-identical to that revision; no repeated semantic run claimed.",
"arithmetic": "Computed 174762*6+2=1048574 and 174763*6+2=1048580. Corrected the erroneous published 1048578 total.",
"final_grouped_domain": "27 passed after grouping observations/defaults/allocation within function-size rules; no scenarios removed.",
"endpoint_contract_correction": "Restored exact endpoint lookup identities rather than shrinking the existing large-endpoint fixture. Added the identical oversized string as an exported primitive in that acceptance workflow: literal normalization must report its bound while full endpoint lookup/feature trace/semantic trace/version comparison continue to work. Production src tree restored byte-identical to 38d263a; 932 semantic tests from that state remain applicable. Final corrected public tests/gates recorded after execution.",
"final_corrected_gates": "Full static gate and docs gate (5 consistency tests) passed after endpoint identity restoration and strengthened unchanged-large-input acceptance. Production src remains byte-identical to 38d263a; final public 7/7 and fresh real control passed."
},
"prior_attempts": [
{
"stage": "Initial literal normalization, incomplete for member/default paths identified in review",
"revision": "40b9d3c5b9cbeae7661d8e351e1b5ef6e5338931",
"source_sha256": {
"src/domain/javascript/javascriptSemanticValues.ts": "2debbb4f32d7d189819f2413f0f7d05b9bdcf497451a1c578475e3f2ec6c8554",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0203d207e0d6bb8aaa17746090f008876a11653afd660fce3dc44f3fda646a6c",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "747d893d6ddd91cf24d3b62b7eb50e0af9b1715ab355bbfc2e10ed0dc171ec17",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "46ceb5a169de3eec3a335b05ef4aa7b1b341a5ac06ad325bd191e862c5b6be70",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50",
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1"
},
"real_control_cohort": {
"preparation_ms": 18876.764036,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 346.7870870000006
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 18.34765800000241
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 11.56626999999935
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 227.17703500000061
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 251.5148330000011
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.888227999999799
}
],
"resources": {
"elapsed_seconds": 31.347,
"cpu_seconds": 31.05517,
"peak_cgroup_bytes": 1589915648,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"validation": {
"baseline_public": "Quoted and cooked-template cases retained 200000-unit normalized literals and failed; addition and small exact controls passed. Aggregate-capacity test passed.",
"baseline_domain_corrected": "2 over-bound failures, 16 passed. Initial combined-run domain assertions had incorrectly required an empty optional resourceLimits array; corrected before using the domain reproduction as evidence.",
"fixed_focused": "55 passed across five files.",
"semantic_suite": "923 passed across 36 files.",
"capacity_after_fixture_cleanup": "1 passed; all 16 bounded values, aggregate size >= original, typed identities and retained tracing preserved.",
"interim_static_failure": "Typecheck rejected resume() on the new test SDK stderr Stream declaration. Switched to its declared data event, matching existing typed tests; analyzer source unchanged.",
"public_boundary_after_stream_type_fix": "4 passed across actual CLI and stdio MCP; analyzer source unchanged."
}
},
{
"stage": "Projection/default correction; final review subsequently found unnormalized request endpoints",
"revision": "38d263a369e7853ccfae1c8e0a3bd387f920c7cf",
"source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "844d4639401a9f050ce42c80192572a6d0d74fe4f81b37e93a2c20aaca9fb9a6",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a3f802ca3446c2b5376cc58b19d818eff318e6775bd47752793c3f649994f8a2",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "2e78717d30e637b5fc5bab6d0eff063f7a62e0199d33697c1cdfaf8317baa698",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "8f9cf1a36f05b03dd816eaf9a6cdba0160d34ad7d17a961206e1bc32444d2ccc",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "e69408f2f5910a0107fd5d0cae0ff60ed706448857bc8e535c37cac1a0cd4d37",
"src/domain/javascript/javascriptSemanticIr.ts": "68dad360d0a9e6bd72108b2717cae756b84ca1efb48455248ded2ff699d2c113",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0993a397f2a2a48c2b6c35494e940b3fce5c2c9032cfe33c7cf0401128c79dde",
"src/domain/javascript/javascriptSemanticState.ts": "bfcface5dcda22594f47ed390efa15f9140fe41cb2d5995896e52dd0ee31b978",
"src/domain/javascript/javascriptSemanticValues.ts": "a4ba51cc60fff38ef11fd5af35a2e1ad753cdb2dea9d44895f91d89320c5ddeb",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "070106115b482269794799e448bd20b8e02df1f511f4ceb2b8fa75612c7f78e4",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"real_control_cohort": {
"preparation_ms": 19599.643274,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 396.9265450000021
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 15.658120000000054
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 12.24905100000251
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 237.03392900000108
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 357.78818299999693
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 14.37622100000226
}
],
"resources": {
"elapsed_seconds": 33.969,
"cpu_seconds": 33.499223,
"peak_cgroup_bytes": 1893015552,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"validation": {
"original_main": "Quoted/template oversized literals failed through public CLI/MCP; addition/small controls passed. Corrected domain boundary reproduced 2 failures/16 passes.",
"review_head": "8 failed/25 passed before fixes for projection tags, encountered-but-discarded limits and bare quoted/template defaults; includes exact null default control.",
"interim_review_fix": "65/66 focused passed; old doubling-chain expectation omitted a genuine expression-depth limit now observed. Adjusted that assertion to preserve the byte-limit requirement while allowing other observed classes.",
"final_focused": "66 passed in five files.",
"final_semantic_suite": "932 passed in 36 files.",
"arithmetic": "Computed 174762*6+2=1048574 and 174763*6+2=1048580. Corrected the erroneous published 1048578 total.",
"final_grouped_domain": "27 passed after grouping observations/defaults/allocation within function-size rules; no scenarios removed.",
"endpoint_review_baseline": "Production at 38d263a: 7 failed/14 passed across 13 new domain and 8 CLI/MCP cases before endpoint normalization. The later public linked-response case was not in this baseline run. Regression tests were uncommitted; final formatted test hashes are preserved, not claimed as the baseline test bytes.",
"endpoint_selection_attempt": "First six-file selection exited before execution because a signed-zero test filename was mistyped; corrected the exact path. No test result claimed for that attempt."
}
},
{
"stage": "Rejected endpoint normalization: violated existing full-endpoint lookup/trace contract; reverted after CI acceptance failure",
"revision": "580887209c4fceda432eac4083fac82a0dd78f15",
"source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "a648f97e631985edcb5259ff4f1fdf0332dca18b8c10b57e050e36a785c1c524",
"src/domain/javascript/javascriptSemanticAnalysis.requestBudget.test.ts": "8e84ffa70d834f5589d04118f0aaf33b0383b15ea97545ff69c66a74473aca73",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a3f802ca3446c2b5376cc58b19d818eff318e6775bd47752793c3f649994f8a2",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "2e78717d30e637b5fc5bab6d0eff063f7a62e0199d33697c1cdfaf8317baa698",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "8f9cf1a36f05b03dd816eaf9a6cdba0160d34ad7d17a961206e1bc32444d2ccc",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "eed961e0b1d8a320e76bbba933408e950b4a4a446110401e8cf470f1e74d8315",
"src/domain/javascript/javascriptSemanticIr.ts": "3b2ede73dbcfc74c4014e4d1affcbf00ecccc29a422ed495a7a3f2f517056c9b",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0993a397f2a2a48c2b6c35494e940b3fce5c2c9032cfe33c7cf0401128c79dde",
"src/domain/javascript/javascriptSemanticState.ts": "bfcface5dcda22594f47ed390efa15f9140fe41cb2d5995896e52dd0ee31b978",
"src/domain/javascript/javascriptSemanticValues.ts": "a4ba51cc60fff38ef11fd5af35a2e1ad753cdb2dea9d44895f91d89320c5ddeb",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "61319ea954de7f256841f43ca9355a1706ce8e93d30f2a8ba851d5e717bd19e7",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"real_control_cohort": {
"preparation_ms": 19939.160272,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 348.3974309999976
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 13.97052899999835
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 12.060379000002285
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 212.01217800000086
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 223.73954800000138
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.450989999997546
}
],
"resources": {
"elapsed_seconds": 33.062,
"cpu_seconds": 32.71624,
"peak_cgroup_bytes": 1897406464,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"validation": {
"original_main": "Quoted/template oversized literals failed through public CLI/MCP; addition/small controls passed. Corrected domain boundary reproduced 2 failures/16 passes.",
"review_head": "8 failed/25 passed before fixes for projection tags, encountered-but-discarded limits and bare quoted/template defaults; includes exact null default control.",
"interim_review_fix": "65/66 focused passed; old doubling-chain expectation omitted a genuine expression-depth limit now observed. Adjusted that assertion to preserve the byte-limit requirement while allowing other observed classes.",
"final_focused": "81 passed in six files; the same CLI/MCP file then passed 9/9 after adding a linked-response case. That follow-up replaces its earlier 8-case coverage; 82 distinct cases across the final six-file selection, not one 82-case invocation.",
"final_semantic_suite": "945 passed in 37 files, overlapping focused domain coverage.",
"arithmetic": "Computed 174762*6+2=1048574 and 174763*6+2=1048580. Corrected the erroneous published 1048578 total.",
"final_grouped_domain": "27 passed after grouping observations/defaults/allocation within function-size rules; no scenarios removed.",
"endpoint_review_baseline": "Production at 38d263a: 7 failed/14 passed across 13 new domain and 8 CLI/MCP cases before endpoint normalization. The later public linked-response case was not in this baseline run. Regression tests were uncommitted; final formatted test hashes are preserved, not claimed as the baseline test bytes.",
"endpoint_selection_attempt": "First six-file selection exited before execution because a signed-zero test filename was mistyped; corrected the exact path. No test result claimed for that attempt.",
"endpoint_typecheck_attempt": "Full static gate found the inferred optional dataField in the public test computed key. Replaced the in-property check with an explicit bare === true discriminant; scenario inputs and runtime implementation unchanged. Corrected static/public checks recorded separately.",
"final_gates": "Full static gate passed after the explicit test discriminant; final public file passed 9/9 again, then docs gate passed including 5 consistency tests. Runtime implementation unchanged after the 945-test semantic suite and real control."
},
"ci_failure": {
"run_id": 38758262643,
"job_id": 116503076738,
"test": "tests/acceptance/applications/javascriptEndpointPayload.test.ts",
"result": "1 failed/1573 passed/19 skipped; request endpoint no longer matched its complete source value. Other local tests and real Obsidian control passed but did not establish this compatibility contract.",
"aggregate_result": "Merged source/built report: 1 failed/15625 passed/138 skipped. Portable tests and CI required failed as aggregates of the same endpoint assertion; no separate additional failed test."
}
}
],
"review_baseline_revision": "40b9d3c5b9cbeae7661d8e351e1b5ef6e5338931",
"review_baseline_source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "ee541e634264363822bf4ce05c42c72272029c7e8da72bbeed643ad1341d0756",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a1cf791a61ee4742c351c006b69b64eba6bef3068f7a8450ed8f9366bfe4ff88",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "747d893d6ddd91cf24d3b62b7eb50e0af9b1715ab355bbfc2e10ed0dc171ec17",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "5ee7af18dd847726d8fe1828ea25f386ff9ef317d996a3f7969253bf73973cfa",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "4c62462016246bc7198294a74a60e87598fc009c53f1601abc2d4662d41b5bd1",
"src/domain/javascript/javascriptSemanticIr.ts": "14955cb4b7912ed6a4f26f804a2480f841dcb6084dbddd15856375fa62cdcfa1",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0203d207e0d6bb8aaa17746090f008876a11653afd660fce3dc44f3fda646a6c",
"src/domain/javascript/javascriptSemanticState.ts": "d94c5a88bb105284fab192406a8c88860d59f80c93394ed3ee7264969df254a6",
"src/domain/javascript/javascriptSemanticValues.ts": "2debbb4f32d7d189819f2413f0f7d05b9bdcf497451a1c578475e3f2ec6c8554",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "46ceb5a169de3eec3a335b05ef4aa7b1b341a5ac06ad325bd191e862c5b6be70",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"implementation": "Per-analysis state records encountered resource classifications at expression/default normalization; final coverage consumes those facts. Slot projection preserves tagged unknown values. Removed retained-value recursive coverage scan and duplicate raw default reader. Defaults retain optional valueResourceLimit and graph value_resource_limit plus an anchored resource-limit unknown; known null/false/0/empty remain exact. Endpoint identities intentionally remain exact lookup values, outside the normalized primitive lattice; rejected endpoint normalization and its CI failure are preserved separately.",
"restored_production_revision": "38d263a369e7853ccfae1c8e0a3bd387f920c7cf",
"final_corrected_resource_receipts": {
"public_7": {
"elapsed_seconds": 43.621,
"cpu_seconds": 43.097453,
"peak_cgroup_bytes": 1520324608,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
},
"static_and_docs": {
"elapsed_seconds": 111.403,
"cpu_seconds": 110.830839,
"peak_cgroup_bytes": 4268244992,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
}
}
@@ -0,0 +1,86 @@
# Semantic literal byte-budget verification
Tracks #3933 under #4027. All normalized semantic literals and configuration
defaults use the primitive owner. Its singleton path validates strings/nonfinite
numbers/negative zero without a Set, sorting or canonical-key serialization.
Each source analysis owns a bounded set of encountered limit classifications;
coverage consumes these observations after evaluation. This replaces a recursive
scan that only saw retained binding/return values. No persistent cache is added.
## Correctness and coverage
Public CLI and actual stdio MCP cases cover quoted strings, cooked templates,
equivalent concatenation, member receivers, a configuration-only expression
statement and a small exact control. Direct/default oversized values become
`resource-limit` unknowns with partial application coverage and
`javascript_semantic_primitive_bytes = 1048576`. Defaults carry
`value_resource_limit` inline and an unknown anchored to their config node.
Known null/false/zero/empty defaults remain exact. Original source/digests and
CLI/MCP identity parity are preserved.
Member and destructuring projections use the shared slot owner's tagged value.
An encountered limit stays visible even when a subsequent projection selects a
healthy sibling: the selected value remains exactly `7`, while file coverage is
partial. This avoids inferring completeness from the final retained values alone.
The duplicate raw default reader and superseded retained-value scan are removed.
Boundary controls admit 174,762 UTF-16 units (1,048,574 estimated JSON bytes)
and reject 174,763 (1,048,580 bytes). Actual arithmetic corrected the earlier
published 1,048,578 total; the unit-count boundary assertions were already correct.
The estimate counts six bytes per unit plus two quotes. It does not establish
actual ASCII JSON size above 1 MiB or an observed OOM.
The initial main baseline reproduced both public literal failures, with addition
and small controls passing. A mistaken optional-limit-array control was corrected
before the separate domain reproduction (2 failed/16 passed). Review head
`40b9d3c5` then reproduced the additional projection/default gaps (8 failed,
25 passed). The first review correction passed 65/66 focused tests; a doubling
chain now truthfully retains an additional encountered expression-depth limit,
so its byte-growth assertion was updated to allow other recorded limits.
Prior validation on the restored runtime: **66 focused tests** in five files and **932 semantic tests**
in 36 files, with overlapping domain coverage. The large-wire fixture retains
16 individually admitted values and at least the former aggregate UTF-16 payload,
plus complete CLI/MCP delivery, typed identities, empty strings and retained
tracing. Inputs regenerate from the tests.
## Endpoint identity compatibility
An attempted endpoint normalization at `5808872` passed its new local cases and real Obsidian control, but failed the existing CI endpoint-payload acceptance workflow (1 failed/1,573 passed/19 skipped in that shard). It violated the established full endpoint lookup/trace contract. This extension was reverted; endpoint values are lookup identities, outside the normalized primitive lattice. The original large endpoint is preserved. Corrected public validation passed **7/7** across two files; this adds one acceptance workflow to the previous 66-case focused selection (**67 distinct cases**, not one 67-case invocation). Its acceptance workflow now also normalizes that identical string as an exported primitive, requiring the typed value bound while endpoint lookup, both trace workflows and version comparison remain usable. Prior measurements and failed attempts remain in the JSON record. Production source is restored byte-identical to `38d263a`; the 932-test semantic run belongs to that state. Final full static and documentation gates (five consistency tests) passed after the compatibility correction.
## Real application control and costs
Official Obsidian 1.14.4 ASAR, selected `package.json` and `main.js`. Follow the
acquisition/digest and existing verifier command in
[module-reference verification](javascript-module-references-2026-10-15.md).
Use three alternating-order query pairs with CLI parity on each revision.
Application code is not executed.
The final correction retains the exact baseline full parent/view Evidence IDs,
23-row fact digest, source coverage and unknowns. Static `cl100k_base` task costs
remain 16,548 with selected name/description/inputSchema contracts, or 204,093
with the full catalog projection, for the reference workflow. Serial legacy
reads also keep their original costs. The query projection improvement belongs
to #4097; these are controls for semantic normalization and coverage.
| Complete real verifier | Before | Final correction |
| ---------------------- | ------------: | ---------------: |
| Elapsed seconds | 33.394 | 32.949 |
| CPU seconds | 33.016802 | 32.559187 |
| Peak cgroup bytes | 1,979,863,040 | 1,880,096,768 |
Includes initial analysis, both query arms, export, CLI parity and tokenizer.
One CPU, 8 GiB hard cap, 6 GiB early stop, no swap; cleanup verified and no OOM.
Memory includes charged file cache, not summed RSS. Counters are at payload exit
while the observer remains alive. Fresh MCP sessions do not establish OS-cold
cache conditions. Shared-host scheduling and filesystem/build warmness were
uncontrolled. This one before/after control establishes neither a speedup nor
an equivalence bound. Static BPE excludes annotations/output schemas, client
rendering/replay, reasoning and final answers; no hosted agent was run. Linux
static ASAR analysis was exercised, not live Electron or other platforms.
[Compact records, samples, source hashes and prior attempts](javascript-literal-byte-budget-2026-10-15.json)
preserve the initial incomplete correction separately from the final measurements.
No downloadable application or complete bundle is archived. Reproduce boundary
outcomes through the listed public tests and reuse the existing real verifier
and preserved 8 GiB observer linked above. Observer hashes and result-affecting
touched source hashes are pinned in the records.
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,179 @@
# Native handoff byte-identity verification
Tracks #3926 under #4027. Baseline: `9f8659e0536e4e50a438f5b9d5661f3c9624f6ac`.
Final integration base: `62bf09809ef7bb224028916424f8f07652a20f4f`.
The analyst already knows an ASAR-relative native path and supplies prior native
Evidence. The question is which Evidence can be linked to that component's exact
bytes. Discovery and native-provider execution are outside this task.
## Observed behavior and correction
The native-node producer distinguishes a content digest from a canonical path
whose bytes are unavailable. Its unavailable observation legitimately cites the
containing archive. The handoff consumer previously treated that supporting
artifact digest as the native component's digest.
Public CLI/stdio MCP regression: the missing-companion case failed and the
available-byte control passed. Real Joplin reproduced the same result: missing
keytar bytes produced an `evidence-linked` handoff with the ASAR digest and the
simulated container observation. CLI/MCP agreed on that incorrect answer.
The consumer now uses only the node owner's content-digest identity. The fallback
observation scan is removed. Unavailable native nodes retain their canonical
path, availability, container-backed provenance and partial coverage in the
inline graph; they yield no false actionable handoff or native Evidence link.
An empty handoff list describes eligibility, not absence of native components.
The per-query node index is allocated only after an eligible target exists; no
persistent cache, schema, tool or target authority is added.
Final real checks passed both cases with exact CLI/MCP trace parity. The available
control preserved the complete parent/trace Evidence IDs, native facts, links,
coverage and all token projections. The missing control preserved its parent,
native node, source provenance and coverage, while removing the wrong handoff
and container Evidence link.
## Real input and scope
[Official Joplin v3.7.21 Linux AppImage](https://github.com/laurent22/joplin/releases/download/v3.7.21/Joplin-3.7.21.AppImage),
published 2026-09-25, 201,462,481 bytes, SHA-256
`5d202dcaca31e14206826aa3f04125e7c218dbc3dc3d98479d811efbd6b856c1`.
Read-only 7z 26.02 inspection found 6,324 ASAR files, 46 unpacked entries and
three `.node` components. Extract only the ASAR and those three binaries:
```sh
input_root="$(mktemp -d -t rea-native-handoff-XXXXXX)"
curl --fail --location --output "$input_root/Joplin-3.7.21.AppImage" \
https://github.com/laurent22/joplin/releases/download/v3.7.21/Joplin-3.7.21.AppImage
sha256sum "$input_root/Joplin-3.7.21.AppImage"
7z x -mmt=1 "-o$input_root/extracted" "$input_root/Joplin-3.7.21.AppImage" \
resources/app.asar \
resources/app.asar.unpacked/node_modules/keytar/build/Release/keytar.node \
resources/app.asar.unpacked/node_modules/sqlite3/lib/binding/napi-v6-linux-glibc-x64/node_sqlite3.node \
resources/app.asar.unpacked/node_modules/onnxruntime-node/bin/napi-v6/linux/x64/onnxruntime_binding.node
python3 - "$input_root" <<'PY'
import os, sys
from pathlib import Path
root = Path(sys.argv[1])
source = root / 'extracted/resources'
for mode in ('available', 'missing'):
case = root / 'cases' / mode
case.mkdir(parents=True)
os.link(source / 'app.asar', case / 'app.asar')
if mode == 'available':
for native in (source / 'app.asar.unpacked').rglob('*.node'):
target = case / native.relative_to(source)
target.parent.mkdir(parents=True, exist_ok=True)
os.link(native, target)
PY
```
Verify the asset digest above before extraction. The ASAR is 233,575,014 bytes,
SHA-256 `767df409b18effe60421fbefcca873de237c7e813eac744ecdeebbc5e825a33a`.
Keytar is 76,960 bytes, SHA-256
`0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a`.
All extracted component hashes are in the JSON record. 7z reported one
trailing-data warning for the AppImage's embedded SquashFS and exited 0.
Both cases select only `package.json` for static source analysis while retaining
the inventory. The available case stages all three native companions; the other
43 unpacked entries are not staged. The missing case stages none. Whole-application
coverage is therefore partial in both cases, independently of the selected
native component's known or unknown byte identity. Neither the application nor
a native analysis provider is executed. Native Evidence is explicitly simulated
to test exact subject linkage. Live Electron, real Ghidra/Hopper/IDA behavior and
Windows/macOS verification are **NOT_ENOUGH_INFO**.
## Reproduce the public workflow
Use pinned Node 24.18.0/npm 11.16.0, `npm ci`, then `npm run build:cached`.
The verifier returns compact records and deletes its private exported parent and
CLI input after owned cleanup. No downloadable application or large Evidence is
archived.
```sh
for mode in available missing; do
node scripts/verify/javascript/native-handoffs.mjs \
--artifact "$input_root/cases/$mode/app.asar" \
--addon-path node_modules/keytar/build/Release/keytar.node \
--native-sha256 0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a \
--source-path package.json --rounds 3 --cli-parity --require-safe
done
```
Add `--tokenizer-python /absolute/path/to/python-with-tiktoken-0.12.0` for the
reported BPE projections. The npm alias `verify:javascript:native-handoffs`
builds first; recorded costs use the direct Node invocation above after building.
For a pre-fix observation, run the saved verifier on the baseline build without
`--require-safe`; `safe_identity: false` is an incorrect answer, not a passing
analyst task. Paths affect serialized requests and some Evidence identities.
The exact resource runs additionally used the preserved
[8 GiB observer](https://github.com/morluto/rea/blob/b3bdabab694af1f9801eff7caf75001b92314a58/experiments/analysis-efficiency/resource-run.py)
and adjacent `resource_payload.py`; both hashes are pinned in the JSON.
## Completed workflow costs and limits
| Case | Revision | Wall s | CPU s | Peak cgroup bytes | Identity oracle |
| ---------------------- | -------- | ------: | ---------: | ----------------: | -------------------------- |
| Native bytes available | Before | 60.369 | 59.806670 | 2,662,629,376 | pass |
| Native bytes available | After | 258.885 | 106.221003 | 2,678,456,320 | pass, exact baseline facts |
| Native bytes missing | Before | 57.930 | 57.479109 | 2,719,457,280 | wrong identity/linkage |
| Native bytes missing | After | 59.738 | 59.254776 | 2,571,235,328 | pass, unknown retained |
Whole-verifier resources include input hashing, analysis, three retained trace
queries, export, CLI parity and tokenization. One CPU, 8 GiB hard cap, 6 GiB early
stop, no swap; cleanup verified and no OOM. Memory includes charged file cache,
not summed RSS. Counters are at payload exit while the observer remains alive.
The available control's final wall and CPU times are substantially higher than
baseline. Earlier post-fix controls on the original base took 61.657/74.956 s wall
and 61.135962/60.352428 s CPU for available/missing cases; their complete receipts
remain separately recorded. The final missing case took 59.738 s wall. This
variability is unresolved and cannot be attributed to the native consumer change.
These runs establish neither speedup nor performance equivalence. Input hashing warms the ASAR
cache, build/OS caches and shared-host scheduling are uncontrolled. Fresh MCP
sessions do not establish OS-cold conditions. First post-analysis and repeated
warm-query samples are preserved separately; query CPU/memory are unmeasured.
One defined task is one analysis summary plus one retained trace, with the path
and prior Evidence already supplied. Static `cl100k_base`/tiktoken 0.12.0 counts
include both requests, replies and name/description/inputSchema contracts:
| Available-byte task, unchanged before/after | BPE tokens |
| ----------------------------------------------- | ---------: |
| Selected catalog + structured replies | 12,340 |
| Selected catalog + complete tool replies | 18,906 |
| Full catalog projection + structured replies | 202,292 |
| Full catalog projection + complete tool replies | 208,858 |
These are alternative reply/catalog projections. The selected catalog alone is
5,570 BPE; the full projection is 195,522. Output schemas, annotations, host
rendering/replay, reasoning, final answers and model billing are unmeasured.
Repeated queries/export/CLI/tokenization are validation overhead outside this
one-task token count, but inside resource receipts. Missing-case structured-task
counts change 11,372 → 11,103 by removing unsupported claims; the incorrect
baseline earns no efficiency credit.
## Regression and verification record
The initial four-file focused selection passed 24 tests. The two public cases
were rerun after adding named summary-result parsing and correcting an optional
architecture factory input. Omitting that field instead of passing null produced
byte-identical normalized simulated Evidence in all four real-case inputs.
The initial CLI-selector harness error and first typecheck failure are retained
as harness/validation attempts, not product reproductions.
After syncing the reviewed upstream changes, the updated application workflow
CLI acceptance file passed 34 tests, bringing the focused selection to 58 distinct
cases across five files. Both Joplin controls were then repeated on the final
integration base. Canonical byte identity is also used by the related version
matching/content comparison consumers; no equivalent observation-digest fallback
was found there.
Final `npm run check && npm run docs:check` passed on the integration base,
including five documentation consistency tests: 128.732 s wall, 115.518920 s CPU,
4,301,684,736 B peak cgroup memory, clean owned exit and no OOM.
Final public and static/docs outcomes are recorded in the
[compact measurements and source pins](javascript-native-handoff-identity-2026-10-15.json).
+114 -8
View File
@@ -4,6 +4,7 @@ import argparse
import hashlib
import json
import os
import re
from pathlib import Path
import shutil
import signal
@@ -22,7 +23,31 @@ def file_digest(path):
return digest.hexdigest()
def runtime_fingerprint(root):
def pinned_experimental_engine(settings):
expected = settings.get('REA_ATOMIC_NATIVE_IR_ENGINE_SHA256')
if expected is None:
return None
if not re.fullmatch(r'[0-9a-f]{64}', expected):
raise ValueError('Experimental native engine pin must be a SHA-256')
path = Path(settings.get('REA_KUNA_COMMAND', ''))
if not path.is_absolute() or not path.is_file() or not os.access(path, os.X_OK):
raise ValueError('Pinned experimental engine requires an absolute executable file')
actual = file_digest(path)
if actual != expected:
raise ValueError('Experimental engine digest differs from its pin; use a preserved build artifact')
return {'path': str(path), 'sha256': actual}
def experimental_engine_unchanged(receipt):
if receipt is None:
return None
try:
return file_digest(Path(receipt['path'])) == receipt['sha256']
except OSError:
return False
def runtime_fingerprint(root, entrypoint=None):
files = []
for name in ['dist', 'bridge']:
for path in sorted((root / name).rglob('*')):
@@ -30,6 +55,18 @@ def runtime_fingerprint(root):
raise ValueError('Runtime fingerprint does not follow symlinks')
if path.is_file():
files.append({'path': path.relative_to(root).as_posix(), 'sha256': file_digest(path)})
if entrypoint is not None:
# Include the experimental modules and their installed QuickJS JS/WASM,
# plus the shared fact owner; the entrypoint alone does not pin them.
for directory in [entrypoint.parent, root / 'experiments/atomic-re-tools']:
for path in sorted(directory.rglob('*')):
if path.suffix not in {'.mjs', '.cjs', '.js', '.wasm'} and path.name not in {'package.json', 'package-lock.json'}:
continue
if path.is_symlink():
raise ValueError('Experimental runtime fingerprint does not follow symlinks')
if path.is_file():
label = path.relative_to(root).as_posix() if path.is_relative_to(root) else path.as_posix()
files.append({'path': label, 'sha256': file_digest(path)})
files.sort(key=lambda row: row['path'])
text = json.dumps(files, separators=(',', ':'), ensure_ascii=False)
return files, hashlib.sha256(text.encode()).hexdigest()
@@ -39,6 +76,12 @@ def main():
parser = argparse.ArgumentParser()
parser.add_argument('--mode', choices=['direct', 'client-code'], default='direct')
parser.add_argument('--runtime-root', type=Path, required=True)
parser.add_argument('--mcp-entrypoint', type=Path,
help='Caller-selected experimental stdio MCP entrypoint; leaves production dispatcher unchanged')
parser.add_argument('--server-name', default='rea')
parser.add_argument('--protocol-profile', choices=['canonical', 'atomic', 'atomic-direct'], default='canonical')
parser.add_argument('--profile-parent', type=Path,
help='Absolute parent outside the repository for the isolated model workspace')
parser.add_argument('--revision', required=True)
parser.add_argument('--prompt', type=Path, required=True)
parser.add_argument('--artifact-path', help='Replace {{artifact_path}} in a question template')
@@ -54,6 +97,10 @@ def main():
help='Retain owned tool/usage items; excludes account/ambient state')
parser.add_argument('--guide-enabled-tools', action='store_true',
help='Opt-in exact-name discovery guidance for selected REA tools')
parser.add_argument('--normalize-mcp-guidance', action='store_true',
help='Opt-in common envelope/stopping guidance, identical across protocol lanes')
parser.add_argument('--guide-long-waits', action='store_true',
help='Opt-in common client wait guidance; does not change analysis contracts or deadlines')
args = parser.parse_args()
if args.guide_enabled_tools and args.enabled_tools is None:
parser.error('--guide-enabled-tools requires --enabled-tools')
@@ -94,13 +141,19 @@ def main():
if undeclared:
raise ValueError('Declare inherited provider settings explicitly in --providers: '
+ ', '.join(undeclared) + '. Values are not logged.')
# Fail before copying credentials or spending model tokens. Cargo test/build
# can replace a mutable target/release path with a different feature build.
experimental_engine = pinned_experimental_engine(provider_env)
source_auth_home = (Path(os.environ['CODEX_HOME']) if 'CODEX_HOME' in os.environ
else Path.home() / '.codex')
source_auth = source_auth_home / 'auth.json'
args.output.mkdir(parents=True, exist_ok=False, mode=0o700)
runtime_files, runtime_digest = runtime_fingerprint(repository)
runtime_files, runtime_digest = runtime_fingerprint(repository, args.mcp_entrypoint)
(args.output / 'runtime-files.json').write_text(json.dumps(runtime_files))
profile = Path(tempfile.mkdtemp(prefix='profile-', dir=args.output))
if args.profile_parent and (not args.profile_parent.is_absolute()
or args.profile_parent.resolve().is_relative_to(repository.resolve())):
raise ValueError('profile-parent must be an absolute directory outside runtime-root')
profile = Path(tempfile.mkdtemp(prefix='rea-model-profile-', dir=args.profile_parent or args.output))
trial_codex_home = profile / 'codex'
account = profile / 'account'
workspace = profile / 'workspace'
@@ -119,7 +172,12 @@ def main():
'NODE_OPTIONS': '--max-old-space-size=512',
})
target_digest = file_digest(Path(args.artifact_path)) if args.artifact_path else None
command_args = [str(repository / 'scripts/rea.mjs'), 'mcp']
if not args.server_name.replace('-', '').replace('_', '').isalnum():
raise ValueError('server-name must contain only letters, digits, hyphens or underscores')
if args.mcp_entrypoint and not args.mcp_entrypoint.is_absolute():
raise ValueError('mcp-entrypoint must be absolute')
command_args = ([str(args.mcp_entrypoint)] if args.mcp_entrypoint
else [str(repository / 'scripts/rea.mjs'), 'mcp'])
instruction = (
'Use the registered REA MCP server for this investigation. Static analysis '
'only: do not execute target code, fetch URLs, launch a GUI, or modify targets. '
@@ -132,6 +190,23 @@ def main():
'Prefer summary/focused views where appropriate. Do not invent successful facts '
'after a tool error. No other agents. Stop once the requested facts are supported.'
)
if args.protocol_profile in {'atomic', 'atomic-direct'}:
instruction = (
'Use the registered ' + args.server_name + ' MCP server for this static binary investigation. '
'Call reference to inspect API contracts, then compose the supported analysis calls. '
'Do not execute target code, fetch URLs, launch a GUI, modify targets, use shell/file tools, '
'or use other agents. API methods return {ok,context,data} or {ok:false,error}; '
'execute_javascript returns the script value, with script-derived authority. '
'Preserve real evidence_id values, artifact digest, uncertainty and traversal boundaries. '
'Check failures before reading data. Do not infer absence from partial coverage. '
'Do not return complete inventories or source reports when a focused answer suffices. '
'Close the binary when finished. No precomputed answer is available to you.'
)
if args.protocol_profile == 'atomic-direct':
instruction = instruction.replace(
'execute_javascript returns the script value, with script-derived authority. ',
'Call the registered facts_* tools directly; no JavaScript execution tool is registered. '
)
if args.authority == 'browser':
instruction = (
'Use the registered REA tools for the explicitly selected public browser scenario. '
@@ -147,6 +222,13 @@ def main():
'possible and select the requested normalized_result fields before returning '
'intermediate output to model context. Keep real Evidence IDs and limitations.'
)
if args.normalize_mcp_guidance:
instruction += (
' MCP calls return transport envelopes: prefer structuredContent when present; '
'otherwise parse the text content JSON. Do not print both duplicate encodings. '
'Apply the same rule to every tool. Stop once the requested facts are supported, '
'state missing coverage explicitly, and close the binary when finished.'
)
if args.guide_enabled_tools:
instruction += (
' Configured REA tool function names for this trial: '
@@ -155,6 +237,15 @@ def main():
'return only their metadata. Preserve the original analyst question, '
'dependency order and source Evidence.'
)
if args.guide_long_waits:
instruction += (
' When the client executor reports a running cell during a long analysis, '
'resume that same cell with wait and yield_time_ms:30000, within the '
'advertised tool schema, rather than repeated one-second polls. '
'The wait may return early when the result is ready. Do not sleep or '
'start duplicate analysis calls just to check completion. Keep the '
'original analysis dependencies, evidence and stopping requirements.'
)
quote = json.dumps
config = '\n'.join([
'approval_policy = "never"', 'sandbox_mode = "read-only"',
@@ -166,16 +257,16 @@ def main():
'code_mode = ' + ('true' if args.mode == 'client-code' else 'false'),
'code_mode_only = ' + ('true' if args.mode == 'client-code' else 'false'),
'[history]', 'persistence = "none"',
'[mcp_servers.rea]', 'command = ' + quote(node),
'[mcp_servers.' + args.server_name + ']', 'command = ' + quote(node),
'args = ' + quote(command_args), 'startup_timeout_sec = 60',
'tool_timeout_sec = 600',
*([] if enabled_tools is None else ['enabled_tools = ' + quote(enabled_tools)]),
'[mcp_servers.rea.env]',
'[mcp_servers.' + args.server_name + '.env]',
*[key + ' = ' + quote(value) for key, value in provider_env.items()],
]) + '\n'
# MCP read-only hints cover normal static tools. Explicitly authorize only
# the session-opening primitive needed by the requested native tasks.
config += '[mcp_servers.rea.tools.open_binary]\napproval_mode = "approve"\n'
config += '[mcp_servers.' + args.server_name + '.tools.open_binary]\napproval_mode = "approve"\n'
for tool in ['analyze_javascript_application', 'inspect_analysis_view',
'inspect_android_package', 'search_android_classes',
'inspect_android_class', 'inspect_android_method',
@@ -189,7 +280,7 @@ def main():
'inspect_evm_interface', 'get_evidence_bundle', 'export_evidence_bundle',
'capture_browser_scenario',
'resolve_containing_procedure', 'analyze_function', 'close_binary']:
config += '[mcp_servers.rea.tools.' + tool + ']\napproval_mode = "approve"\n'
config += '[mcp_servers.' + args.server_name + '.tools.' + tool + ']\napproval_mode = "approve"\n'
try:
if source_auth.exists():
shutil.copyfile(source_auth, auth)
@@ -209,10 +300,17 @@ def main():
'mode': args.mode, 'model': args.model, 'reasoning': args.reasoning,
'capture_tool_context': args.capture_tool_context,
'guide_enabled_tools': args.guide_enabled_tools,
'normalize_mcp_guidance': args.normalize_mcp_guidance,
'guide_long_waits': args.guide_long_waits,
'driver_commit': subprocess.check_output(
['git', 'rev-parse', 'HEAD'], cwd=driver_repository, text=True).strip(),
'runtime_revision': args.revision,
'runtime_root': str(repository),
'mcp_server_name': args.server_name,
'protocol_profile': args.protocol_profile,
'isolated_profile_parent': str(args.profile_parent) if args.profile_parent else str(args.output),
'mcp_entrypoint': str(args.mcp_entrypoint) if args.mcp_entrypoint else None,
'mcp_entrypoint_sha256': file_digest(args.mcp_entrypoint) if args.mcp_entrypoint else None,
'selected_artifact_path': args.artifact_path,
'selected_artifact_sha256_before': target_digest,
'enabled_tools': enabled_tools,
@@ -230,6 +328,7 @@ def main():
'question_template_sha256': template_digest,
'provider_settings_sha256': hashlib.sha256(
json.dumps(provider_env, sort_keys=True).encode()).hexdigest(),
'experimental_engine': experimental_engine,
'provider_search_path_sha256': hashlib.sha256(
provider_env.get('PATH', os.environ.get('PATH', '')).encode()).hexdigest(),
'provider_environment_policy_sha256': hashlib.sha256(
@@ -282,6 +381,9 @@ def main():
summary = {
'mode': args.mode, 'requested_model': args.model, 'reasoning': args.reasoning,
'runtime_revision': args.revision,
'runtime_inputs_unchanged': runtime_fingerprint(repository, args.mcp_entrypoint)[1] == runtime_digest,
'experimental_engine': experimental_engine,
'experimental_engine_unchanged': experimental_engine_unchanged(experimental_engine),
'capture_tool_context_requested': args.capture_tool_context,
'selected_artifact_sha256_before': target_digest,
'selected_artifact_sha256_after': final_target_digest,
@@ -296,6 +398,10 @@ def main():
}
(args.output / 'summary.json').write_text(json.dumps(summary, indent=2) + '\n')
print(json.dumps(summary))
if not summary['runtime_inputs_unchanged']:
raise RuntimeError('Runtime inputs changed during the trial; retain costs but reject the comparison')
if summary['experimental_engine_unchanged'] is False:
raise RuntimeError('Experimental engine changed during the trial; retain costs but reject the comparison')
finally:
try:
if args.capture_tool_context:
@@ -0,0 +1,46 @@
import hashlib
import importlib.util
from pathlib import Path
import tempfile
import unittest
spec = importlib.util.spec_from_file_location('agent_trial', Path(__file__).with_name('agent-trial.py'))
trial = importlib.util.module_from_spec(spec)
spec.loader.exec_module(trial)
class ExperimentalEnginePinTests(unittest.TestCase):
def test_missing_pin_preserves_other_provider_trials(self):
self.assertIsNone(trial.pinned_experimental_engine({}))
self.assertIsNone(trial.experimental_engine_unchanged(None))
def test_feature_build_replacement_and_removal_invalidate_pinned_engine(self):
with tempfile.TemporaryDirectory() as directory:
engine = Path(directory) / 'kuna'
engine.write_bytes(b'embedded-spec-build')
engine.chmod(0o755)
settings = {'REA_KUNA_COMMAND': str(engine),
'REA_ATOMIC_NATIVE_IR_ENGINE_SHA256': hashlib.sha256(engine.read_bytes()).hexdigest()}
receipt = trial.pinned_experimental_engine(settings)
self.assertTrue(trial.experimental_engine_unchanged(receipt))
engine.write_bytes(b'non-embedded-test-build')
with self.assertRaisesRegex(ValueError, 'differs from its pin'):
trial.pinned_experimental_engine(settings)
self.assertFalse(trial.experimental_engine_unchanged(receipt))
engine.unlink()
self.assertFalse(trial.experimental_engine_unchanged(receipt))
with self.assertRaisesRegex(ValueError, 'absolute executable'):
trial.pinned_experimental_engine(settings)
def test_malformed_pin_and_relative_command_are_rejected(self):
for settings in [
{'REA_ATOMIC_NATIVE_IR_ENGINE_SHA256': 'bad'},
{'REA_ATOMIC_NATIVE_IR_ENGINE_SHA256': '0' * 64, 'REA_KUNA_COMMAND': './kuna'},
]:
with self.assertRaises(ValueError):
trial.pinned_experimental_engine(settings)
if __name__ == '__main__':
unittest.main()
+301
View File
@@ -0,0 +1,301 @@
# Atomic RE through Code Mode
This is an **in-progress experiment**, not an integrated provider release. Branch
`experiment/atomic-codemode-20261016` starts at main
`42fc6c53ae188978deb3bedfe88287b6b316ba22` and carries the previous atomic-facts
prototype. Historical measurements in `../atomic-re-tools/` retain their original
baselines. No PR or merge is intended.
The goal is a real model completing reverse-engineering questions through
composable facts, with Code Mode as the calling adapter. A configured feature flag,
a scripted verifier or a small response does not prove that goal.
The reference is [Hex-Rays IDA MCP](https://github.com/HexRaysSA/ida-mcp/blob/df6de7985b534a964c125d5a86e41b26b17b14f6/ida_mcp/mcp.py):
an on-demand `reference` lookup and an execution entrypoint over a richer analysis
API. This experiment uses JavaScript and a QuickJS WASM environment rather than
Python in the host process. It exposes only declared static analysis capabilities.
It does not add target execution, patching or GUI control.
Three boundaries own the implementation:
- `provider.mjs`, `native-ir.mjs`: existing REA native target/process owners and parsed Kuna
string/reference/SSA facts, retained sources, freshness and cleanup.
- `api.mjs`: named, validated fact/relationship contracts and session admission.
SDK callers, CLI scripts and both MCP adapters use these same methods.
- `executor.mjs`, `server.mjs`, `cli.mjs`: bounded JavaScript composition and
presentation. No analysis is reimplemented in the execution environment.
The complete Code Mode MCP catalog is `open_binary`, `reference`,
`execute_javascript`, `close_binary`. Setting `REA_ATOMIC_ADAPTER=direct` exposes
the same fact API as `facts_*` MCP operations for comparison. This is a distinct
experimental server; the production REA catalog is not filtered or replaced.
Use the pinned Node, an existing REA build and a caller-supplied Kuna 1.833:
```sh
npm ci --prefix experiments/atomic-codemode --ignore-scripts
node --test experiments/atomic-codemode/*.test.mjs
REA_KUNA_COMMAND=/absolute/path/to/kuna node experiments/atomic-codemode/server.mjs
```
The local MCP registration for real model trials is named `rea-codemode`.
`agent-trial.py --mcp-entrypoint /absolute/path/to/server.mjs
--server-name rea-codemode --protocol-profile atomic --profile-parent /tmp` creates an isolated
configuration, home and empty working directory for the model. Shell/file tools,
web search and delegation are disabled. No repository or oracle is placed in that
workspace. The copied authentication file is removed with the isolated profile.
Model output, actual MCP calls, original errors and usage remain in the trial
directory. `REA_ATOMIC_TRACE` and `REA_ATOMIC_EVIDENCE_DIR` retain actual API calls
and complete sources outside the model workspace when explicitly selected by the
evaluator. The ambient environment and credentials are not logged.
Current bounds: 64 MiB guest heap, 32 KiB code and script value/error payloads, 64 host calls per execution,
120 seconds wall deadline and 2 seconds of guest synchronous execution; native requests have existing supervision, an 8 MiB
capture budget and 1 GiB function-memory limit. Retained source storage is 64 MiB
per target, with two resident indexes (each derived from at most 2 MiB diagnostic
or 8 MiB native wire source). Compressed native graphs admit at most 32 MiB decoded
JSON plus independent row/text/relationship bounds. One decoded Evidence text view
up to 32 MiB is also retained per lease; these source bounds are not parsed-heap
size guarantees. Sources are
content-digested and retained until close; explicit evaluator copies survive
close. Capacity failures are explicit, not clipped facts. Leases and snapshot
handles prevent cross-session joins.
Invocation context and observed source IDs are additional bounded receipt
metadata; the script payload bound is not a 32 KiB whole-MCP-wire limit. A real
MCP probe found that a 40,000-byte guest exception bypassed that payload policy
and produced a 40,183-byte response. Oversized errors now return explicit
`output_budget` with omission metadata. Escaping expansion is checked too;
ordinary smaller errors remain intact. `verify-executor.mjs` confirms real MCP
delivery and continued access to source Evidence collected before the exception.
It measures response bytes, not model tokens or an analysis speedup. Targeted JS
tests at this stage totaled 42, alongside the separate real-provider and native-writer checks.
The diagnostic SSA format still has incomplete CFG and phi predecessor coverage.
The opt-in [native recovered IR experiment](native-engine.md) adds full recovered
CFG edges, exact value/operand identities, phi incoming-edge bindings, selected
`values`/`blocks` reads and derived control dependence. It requires an explicitly
pinned patched engine; stock Kuna does not gain those capabilities from a wrapper.
Runtime successors, memory aliases and machine-semantic equivalence remain
unknown. Other providers are not yet wired to this experimental service.
The initial [actual model comparison](trials.md) favors the focused traditional
workflow. The seeded value-origin task has lower cumulative input and wall time
through Code Mode, but initially exposes an IR-to-machine-path error. The revised
trial correctly explains that machine path with lower observed task costs;
broader claims still require more than this single seeded task.
The large SQLite VM diagnostic task also completes when its native graph exceeds
capacity. Atomic direct beats Code Mode on cumulative input and wall time in
that sample, and a traditional reused-project run is fastest. The actual CLI
client already exposes JavaScript dispatch in every lane, regardless of requested
feature flags. MCP request counts therefore do not establish fewer model rounds;
the ledger now records actual model responses and client dispatch/wait calls.
`instructions` reads selected machine byte/decode windows through the same
owner. Recovered operations use `origin_address` and blocks `origin_start`;
these locations cannot be substituted for machine branch destinations. The
value-origin trial demonstrates why this boundary matters. The experiment
now has fourteen fact methods, still behind the same four-tool Code Mode
catalog.
`calls` connects recovered call operations to native target specifications and
separates argument SSA values from passing storage. `values(is_input:true)`
selects retained callee inputs for exact storage comparisons. These candidate
links preserve snapshot identities, missing specifications, unknown storage and
runtime-binding limits. The [cross-function task and oracle](call-oracle.md)
are fixed before its real-model trials; the graph wire format is version 3, with independent version-1 call facts.
`references` now selects incoming exact-address references or outgoing
instruction/function references, with native kind filtering, field selection and
paging. Source Evidence plus `source_row` identifies an occurrence. Function
applicability, excluded/unattributed rows, indirect targets and discovery stay
explicit. `fields:[]` requests only the always-retained identity columns across
projection contracts. The [reference question and independent oracle](reference-oracle.md)
cover real SQLite VM aggregates and a machine call omitted from optimized IR.
`bytes` reads explicit virtual-address ranges as native loaded-image hex without
decoding them as instructions. Batched reads preserve short reads and individual
failures, with 64 KiB aggregate selected-byte capacity to bound hex/bridge growth.
Complete hexdumps remain in Evidence. The original-file-offset mapping and live
runtime memory remain unknown. This closes a direct byte-reading contract gap
for data tables; it does not establish arbitrary memory-alias analysis.
`functions` provides selected name/alias/entry reads. Native snapshots report
code and recovered-IR facets separately, preserving readable code when graph
extraction is unavailable. Real capacity and cold/warm MCP checks are documented
in [native-engine.md](native-engine.md). Whole-model costs and unequal-depth
comparisons remain separate in the trial report.
The corrected SQLite comparison reaches the formatter implementation in all
three interfaces. Code Mode uses fewer calls than atomic direct in that sample;
a traditional run reusing its persistent whole-project cache is faster than
Code Mode's fresh lease. Cache state, completed-answer coverage and actual
uncached tokens must therefore accompany any cost comparison.
SDK views share immutable parsed inventories with their owner. A real SDK probe
initially changed a returned string and made the next query disappear under the
same Evidence ID. Freezing at the cache boundary fixes that producing rule,
without copying or parsing the entire inventory for each read. `verify-sdk.mjs`
checks the real provider's string/metadata/function views and complete retained
source digest. The fixed probe passes under the bounded runner (1.328 seconds,
1.144 family CPU seconds, 110.6 MiB kernel family peak, zero OOM kills and verified
payload cleanup); it is a correctness check, not a model-cost measurement.
Encoded size is another real coverage boundary: two SQLite functions initially
lose even their C because pretty JSON exceeds transport capacity. The native
patch now uses bounded compact serialization and a graph-only failure when
needed. Parser's full graph fits; Pragma retains identical C with an explicit
missing graph. The 8 MiB bound remains. See the exact sizes, rebuilt engine digest
and verification scope in [native-engine.md](native-engine.md).
Completion requires actual model Code Mode use, independently checked task
answers, shared atomic/direct/Code Mode semantics, needed relation coverage,
failure/cancellation/identity tests, and equivalent completed-task measurements
including discovery/retries, actual model usage, cold/warm analysis, CPU and
process-family peak memory. Negative cases such as the small SQLite function
must remain in the comparison.
The outgoing-reference real-model pair now completes with identical required
facts and explicit coverage limits. In its single sample, Code Mode uses 264,680
cumulative input / 93.837 seconds versus direct's 285,086 / 106.183. The shared
CLI/API also verifies the VM aggregates on unpatched stock Kuna 1.833. These are
scoped positive results alongside the earlier mixed adapter comparisons, not
proof of general superiority. Details and retained failed/partial attempts are
in [trials.md](trials.md).
Recovered calls and their boundary values now survive full-graph capacity
failure. The real VM capture preserves 416 call operations and 1,294 boundary
values while its graph remains unavailable. Values clearly state their partial
scope; definition IDs do not imply readable defining operations. See the
[boundary question and oracle](boundary-oracle.md) and its completed real-model
pair in [trials.md](trials.md). Both answers pass, using one VM capture each.
Code Mode's single-sample latency is lower, with mixed context costs; broader
efficiency remains unproven. The focused JS suite at that stage passes 49 tests.
Available snapshot call facets now include `by_opcode` known/unknown-target
counts and their own coverage. The owner derives them once from the same
validated immutable calls used by selected reads, including dead operations;
target knowledge is independent of opcode. Empty available facets report zeros,
while unavailable facets omit counts. This small aggregate addresses an observed
cost in the real boundary pair: paging the entire call set just to count it.
Both subsequent real-model answers satisfy the unchanged question/oracle without
that paging. In the new sample direct is faster and lower on cumulative/uncached
input than Code Mode, reversing the preceding latency ordering. Whole-task
effects remain mixed; see [the complete results](trials.md). This aggregate is a
shared fact-layer improvement, available to either adapter.
The shared owner also checks the native cleanup `Result` before releasing its
lease or clearing retained state. A tagged cleanup failure now stays retryable;
MCP errors retain the lease context and reported resources. The real SDK
failure-injection verifier first reproduces the old false success, then verifies
both close and target replacement retain the staged file/owner until a successful
retry. Both public MCP adapters preserve that error and retry successfully.
These use an injected tagged failure, not an observed engine cleanup incident;
native reads and subsequent cleanup use the real provider. See the recorded
scope and resource receipts in [native-engine.md](native-engine.md).
A [subsequent diagnostic probe](projection-experiment.md) measures the VM's full
17 MB recovered graph and much smaller selected definition neighborhoods. It
supports trying a bounded operation/value facet, while preserving unresolved
definitions and unavailable CFG. The following implementation now makes that
facet callable; the original diagnostic remains separate from public results.
`snapshot.facets.operations` distinguishes a complete recovered graph from a
bounded call-rooted definition projection. `operations`, `values`, selected
definition/use `relations` and `slice` share the same parsed operation owner in
either case. Projection coverage preserves unexpanded definition counts and
selected slice frontier reasons; CFG/controller reads remain unavailable.
Native operation reads retain `is_dead` and literal operand/value identities.
Coverage returns compact counts and bounded issue text, with complete native
details retained in Evidence. [The new question/oracle](projection-oracle.md)
are fixed before real-model trials of the PHI format alternatives.
Both first PHI-format model answers pass the full oracle using one VM analysis
and a six-operation selected trace, with the global graph/use limits intact.
The trace reveals two format sources at each shared call, verified against byte
and machine evidence. [Costs and source review](trials.md) also record a direct
documentation-lookup miss; named discovery needs fixing before rerunning the
same task. No general adapter speedup follows from that first pair.
Named reference lookup now accepts canonical IDs or exact displayed names,
including space/comma-separated lists. One presentation map supplies direct
registration and documentation names; unmatched terms remain explicit. Selected
schemas come from named contracts, use input semantics (defaulted fields stay
optional), and are cached immutably by Zod schema identity for the process.
Validation stays in the API registry; documentation has its own module. Both
actual MCP catalogs and all fourteen reference schemas validate. The unchanged
PHI question/oracle were rerun: both answers pass again, but direct has a native
timeout followed by a successful retry. Complete costs and the unknown timeout
cause remain in [trials.md](trials.md); this does not isolate an adapter speedup.
The next observed atom boundary is a wide PHI. One real VM operation has 156
inputs; its old default single-node slice is about 44 KB and exceeds the guest's
32 KiB result budget. Native operation/slice defaults now return opcode, output
and exact input/definition/constant counts. Explicit detailed fields still work.
`relations({kind:"operands",seeds:[id],input_indices:[0,105,155]})` reads selected
slots; omitting the selector pages all slots. Each slot retains value/definition
and phi predecessor/edge identity, including null values and definition-free
constants/inputs. Counts concern recovered facts, not source function arity.
Full and projected native operations use the same owner. Relationship paging
counts existing groups before materializing only the requested page; identity
reads deduplicate requested IDs in caller order. Diagnostic SSA explicitly lacks
native operand relations. The discriminated relation schema advertises the same
seed types and selector applicability enforced at runtime. The fourteen-method
API and both adapters remain shared; no new engine or output-budget increase is
needed. The [seeded wide-operand task/oracle](operands-oracle.md) is separate from
the earlier PHI-format question and is pinned before model trials.
Both actual wide-operand model answers now pass that fixed oracle, including
distinct edges sharing a value and absent-slot versus unretained-seed semantics.
Each performs one native analysis and reads only selected slots, distinct values
and their immediate definitions. Code Mode takes 95.699 seconds versus direct's
91.817, with nearly equal cumulative input and lower uncached input. Its four
discovery lookups prevent fewer MCP requests from becoming fewer model responses.
These [mixed costs](trials.md#selected-wide-phi-input-relationships) validate the
fact boundary while leaving broad efficiency unproven. The current shared JS
suite passes 57 tests; the Python cost-ledger suite passes three.
Discovery now also offers `reference({view:"index"})`: the complete fourteen
method names and descriptions, without schemas or return contracts. Follow with
`view:"contracts"` and exact names for only the needed inputs. The default
contract view and named/topic lookup stay compatible. Both views use the same
registry and presentation map; index access never constructs unrequested schemas.
The index describes declared methods, while `snapshot` remains the live coverage
authority. This addresses observed blind discovery without inventing aliases or
changing analysis facts; its whole-task effect must be measured on the unchanged
wide-operand question.
The identical question now completes correctly through both indexed adapters.
Code Mode uses the intended index plus one named lookup; discovery drops from
four calls to two, without changing the seven fact reads. The new pair takes
about 98.8 seconds each; cumulative input favors direct, uncached input favors
Code Mode. All four wide-operand answers pass, with no retry or OOM. Costs remain
mixed. A selected-slot read still incurs whole-function native decompilation on
a fresh lease; phase profiling is the next computation question, separate from
the now finer fact/query boundary.
The [native phase study](native-compute-study.md) now measures the VM drive at
about 34 seconds, C rendering at 0.22 seconds and projection at 0.03 seconds.
Repeated new native processes do not gain the API owner's warm result reuse.
Its subsequent [bounded codec implementation](native-graph-transport-study.md)
now returns the complete 21,349-operation VM graph in about 3.79 MB, inside the
unchanged 8 MiB wire budget. Selected definitions/CFG reads and decoded Evidence
work through both real MCP adapters; all graph fields and ordinary C/calls match
the prior complete golden. Same-lease reuse is about 67 ms after a 39.7-second
fresh capture, with resource limits and no OOM. The [next question/oracle](savepoint-control-oracle.md)
start from names/error messages and exercise the new condition/CFG coverage.
Two [real-model pairs](savepoint-control-study.md), including a discovery-guided
repeat, answer both local guards correctly without a whole-report fetch. Code
Mode uses fewer MCP requests, but whole-task costs are mixed and a large capture
time variation remains unexplained. Both clients already provide JavaScript
orchestration. Source fidelity is not machine equivalence, provider parity or a
general speedup. The [native switch experiment](switch-study.md) now exports and independently checks
182 labels over those 149 targets, exposes selected case-to-edge relations and
keeps them readable when a 2 MiB native output budget omits the full graph.
Both real models now answer the selected-case/default question correctly using
one retained capture and focused case/byte reads. Code Mode uses fewer tokens
in this pair, but unequal native capture time and extra client polling confound
the comparison. Whole-task costs and limits are in the same study; a general
efficiency advantage remains unproved.
+420
View File
@@ -0,0 +1,420 @@
// One named contract registry shared by SDK, CLI, direct MCP and Code Mode.
import { z } from "zod";
import { KunaFactOwner } from "./provider.mjs";
import { FactFailure } from "./fact-error.mjs";
import { MAX_BYTE_BATCH_BYTES } from "./limits.mjs";
import { createReference } from "./reference.mjs";
const hex = z.string().regex(/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/u);
const snapshot = z.string().startsWith("snap_");
const byteWindow = z.strictObject({
address: hex,
length: z.number().int().min(1).max(MAX_BYTE_BATCH_BYTES).default(256),
});
const page = {
offset: z.number().int().nonnegative().default(0),
limit: z.number().int().min(1).max(128).default(20),
};
const kinds = z.enum([
"value",
"load",
"store",
"phi",
"call",
"branch",
"switch",
"return",
]);
const fields = z
.array(
z.enum([
"kind",
"statement",
"definitions",
"constants",
"opcode",
"inputs",
"output",
"is_dead",
"input_count",
"definition_count",
"constant_count",
]),
)
.optional();
const definition = (description, input, method, example) => ({
description,
input: z.strictObject(input),
method,
example,
});
const operationSeeds = z.array(z.string()).min(1).max(128);
const relationInput = z.discriminatedUnion("kind", [
z.strictObject({
snapshot,
kind: z.literal("cases"),
seeds: operationSeeds,
case_values: z.array(hex).max(128).optional(),
case_indices: z.array(z.number().int().nonnegative()).max(128).optional(),
...page,
}),
z.strictObject({
snapshot,
kind: z.literal("operands"),
seeds: operationSeeds,
input_indices: z.array(z.number().int().nonnegative()).max(128).optional(),
...page,
}),
z.strictObject({
snapshot,
kind: z.enum(["definitions", "uses"]),
seeds: operationSeeds,
...page,
}),
z.strictObject({
snapshot,
kind: z.enum(["incoming", "outgoing", "controllers"]),
seeds: z.array(z.number().int().nonnegative()).min(1).max(128),
...page,
}),
]);
export const methods = {
switches: definition(
"Inspect recovered indirect branches and their independent native jump-table metadata, without returning all case rows. Selector is the current recovered BRANCHIND input, which may be normalized rather than the machine jump address. Native consumed mask and signed rendering hint are not complete runtime/signedness proofs. Null tables/defaults stay unknown. The native model default out-edge is separate from any machine out-of-range guard destination. Use relations kind:cases for selected raw table rows, labels and their native mapping to recovered edges; row/edge ordinals are not case values. Available even when the full graph is unavailable if this independent facet was captured.",
{
snapshot,
ids: z.array(z.string()).max(128).optional(),
origin_addresses: z.array(hex).max(128).optional(),
fields: z.array(z.enum(["selector", "table"])).optional(),
...page,
},
"switches",
'await rea.switches({snapshot:s,origin_addresses:["0x1000"]})',
),
functions: definition(
"Search retained native callable entries by literal name/alias, exact name, or entry addresses. Names can match several entries, including PLT/import stubs and implementations; preserve all matches and select by address. A name does not establish runtime symbol binding.",
{
query: z.string().default(""),
exact: z.boolean().default(false),
case_sensitive: z.boolean().default(true),
addresses: z.array(hex).max(128).optional(),
fields: z.array(z.enum(["name", "aliases", "size"])).optional(),
...page,
},
"functions",
'await rea.functions({query:"format_error",exact:true})',
),
strings: definition(
"Search retained string inventory by literal text. Paging does not establish artifact completeness.",
{ query: z.string(), ...page },
"strings",
'await rea.strings({query:"incorrect header check"})',
),
references: definition(
"Batch incoming or outgoing lifted-code references without decompilation. Address scope selects exact target (incoming) or source instruction (outgoing); function scope selects outgoing native owning-function references and requires an entry address. Unknown function applicability stays unknown. Kind filtering is pushed to the native query; source rows, endpoints and Evidence identity remain. Recovered IR calls are a separate representation and may omit machine calls. Interior string references require a separate extent query.",
{
addresses: z.array(hex).min(1).max(16),
direction: z.enum(["incoming", "outgoing"]).default("incoming"),
scope: z.enum(["address", "function"]).default("address"),
kinds: z
.array(z.enum(["call", "jump", "data", "read", "write"]))
.min(1)
.optional(),
fields: z
.array(z.enum(["fromFunction", "toFunction", "instruction"]))
.optional(),
...page,
},
"references",
'await rea.references({addresses:["0x2000"]})',
),
snapshot: definition(
"Analyze an explicit function entry once; return a retained capture handle, facets and coverage. Reuses the source within this target lease. Code, recovered calls and values remain independently available when full recovered IR exceeds capacity. A bounded call-rooted operation/definition projection may also be available; inspect facets.operations and coverage before reads. CFG queries still require the complete recovered graph. Available calls include compact by_opcode counts of known/unknown targets over the whole retained call facet; no paging is needed for these counts. Selected calls and complete Evidence remain readable.",
{ entry: hex },
"snapshot",
'await rea.snapshot({entry:"0x1000"})',
),
bytes: definition(
"Read selected native loaded-image bytes without instruction decoding. Batch up to 16 exact virtual-address windows and 64 KiB total selected bytes; hex uses twice that space before provenance. Short reads remain partial. These are not live process memory or an original-file-offset mapping. Global code discovery is not performed.",
{
windows: z.array(byteWindow).min(1).max(16),
},
"bytes",
'await rea.bytes({windows:[{address:"0x2000",length:32}]})',
),
instructions: definition(
"Batch selected machine instruction windows. Read exact bytes and native disassembly to verify actual control transfers; optimized recovered CFG edges can bypass machine blocks. Address+size with the artifact context identifies the byte range. A caller-selected decode start does not prove execution/alignment.",
{
windows: z
.array(
z.strictObject({
address: hex,
count: z.number().int().min(1).max(4096).default(16),
}),
)
.min(1)
.max(16),
fields: z
.array(z.enum(["bytes", "text", "mnemonic", "operands"]))
.optional(),
},
"instructions",
'await rea.instructions({windows:[{address:"0x1000",count:8}],fields:["bytes","text"]})',
),
operations: definition(
"Select SSA operations by IDs, source instruction origins, block IDs, constant literal or kind. Native default rows are compact summaries: opcode/kind/output and input/definition/constant counts; request detailed fields explicitly or page input slots via relations kind:operands. Filters intersect; identities and native is_dead are retained. Availability may be a complete recovered graph or a bounded call-rooted projection; inspect coverage before inferring absence. A literal match is not object identity. origin_addresses selects optimized recovered IR origins; use instructions for actual machine control transfers.",
{
snapshot,
ids: z.array(z.string()).max(128).optional(),
origin_addresses: z.array(hex).max(128).optional(),
block_ids: z.array(z.number().int().nonnegative()).max(128).optional(),
constant: hex.optional(),
kind: kinds.optional(),
fields,
...page,
},
"operations",
'await rea.operations({snapshot:s,constant:"0x2000",fields:["statement"]})',
),
relations: {
description:
"Read native operands, definitions/uses or case rows for operation IDs; incoming/outgoing CFG edges or derived controllers for block IDs. cases pages one relation per case/out-edge, with a stable case_id; an unmapped case has one explicit unknown-edge row. total counts relations, not table rows. cases filters optional case_values (native labels tagged value) and case_indices (original row ordinals); both filters intersect and paging spans deduplicated seeds in caller order. Rows preserve missing/no-label status, original recovered target, and separate recovered edges. Partial/unavailable tables and unmapped rows are explicit; no runtime feasibility claim. operands pages every selected input slot, including constants, function inputs and null slots; input_indices optionally selects slots in caller order. Identity is snapshot + operation + input_index; value_id/definition and exact phi predecessor/edge bindings remain separate. Out-of-range slots of retained operations are absent; unretained_seeds remain unknown. A bounded call-rooted projection supports operands/definitions/uses but no CFG; uses cover retained selected operations only. Diagnostic SSA lacks native operands and has partial CFG/printed occurrences. controllers requires a complete recovered graph and concerns finite paths to recovered exits, not runtime feasibility.",
input: relationInput,
method: "relations",
example:
'await rea.relations({snapshot:s,kind:"operands",seeds:[op.id],input_indices:[0,2]})',
},
values: definition(
"Read identified native SSA values in a batch; scope may be a full graph, bounded call-rooted definition projection, or call-boundary values only; is_input filters retained operand/output values, not an original source signature. Offset is an address-space offset, not a computed arbitrary-width value. Width, constants, function inputs, annotation status and defining operation are separate facts. Requires the pinned native exporter.",
{
snapshot,
ids: z.array(z.string()).max(128).optional(),
is_input: z.boolean().optional(),
fields: z
.array(
z.enum([
"space",
"offset",
"size_bytes",
"is_constant",
"is_input",
"is_annotation",
"definition",
"type_name",
]),
)
.optional(),
...page,
},
"values",
"await rea.values({snapshot:s,ids:op.inputs.map(i=>i.value_id).filter(Boolean)})",
),
calls: definition(
"Select structured recovered call sites and argument bindings. Target comes from native call specifications, never the fspec annotation offset or display text. Each argument separates its SSA value from its passing storage. Calls and their boundary values survive full graph capacity failure. target_value_id retains input zero independently of target resolution; it may be an fspec annotation or an unresolved indirect target value. A null output describes this recovered operation, not proof of a void source signature. Input slots are not original source parameter ordinals; missing storage and indirect/runtime targets remain unknown. To investigate across functions, snapshot the selected callee and select values with is_input:true, comparing exact space/offset/width. Requires the pinned native exporter.",
{
snapshot,
ids: z.array(z.string()).max(128).optional(),
origin_addresses: z.array(hex).max(128).optional(),
fields: z
.array(
z.enum([
"target",
"target_name",
"arguments",
"output",
"prototype",
"stack_offset",
"target_value_id",
"is_dead",
]),
)
.optional(),
...page,
},
"calls",
'await rea.calls({snapshot:s,origin_addresses:["0x1100"]})',
),
blocks: definition(
"Read native recovered blocks by stable snapshot-local ID, with ordered predecessors, successor edges and member operation IDs. origin_start describes optimized IR provenance, not a machine block extent. Requires an available native recovered IR graph. Runtime successors remain unknown.",
{
snapshot,
ids: z.array(z.number().int().nonnegative()).max(128).optional(),
fields: z
.array(
z.enum(["origin_start", "operations", "predecessors", "successors"]),
)
.optional(),
...page,
},
"blocks",
"await rea.blocks({snapshot:s,ids:[op.block]})",
),
slice: definition(
"Compose a bounded backward definition traversal from explicit operation IDs. Native default operation rows are compact summaries; request detailed fields or use paged operand relations for wide nodes. Available over a complete recovered graph or bounded call-rooted projection. Missing defining bodies retain frontier reasons; complete use sets and CFG are unavailable for a projection. Default load/phi cutoffs keep boundaries visible; choose stop_kinds:[] to traverse beyond them. Complete reachability and memory aliases remain unknown.",
{
snapshot,
seeds: z.array(z.string()).min(1).max(64),
max_operations: z.number().int().min(1).max(256).default(48),
stop_at: z.array(z.string()).max(128).default([]),
stop_kinds: z.array(kinds).default(["phi", "load"]),
fields,
},
"slice",
'await rea.slice({snapshot:s,seeds:callIds,stop_kinds:["load"],fields:["opcode","inputs","output","constants"]})',
),
code: definition(
"Read reconstructed code from the same retained function capture, even when its graph facet is unavailable. offset is zero-based. A large limit reads the complete code inside Code Mode for local filtering. The bridge byte budget bounds the result. No additional decompilation.",
{
snapshot,
offset: page.offset,
limit: z.number().int().positive().default(64),
},
"code",
"await rea.code({snapshot:s,offset:100,limit:64})",
),
evidence: definition(
"Read complete retained native source in UTF-16-code-unit pages by evidence_id; no reanalysis. component:source preserves exact original wire JSON. component:recovered_ir reads the decoded graph source, including lossless compressed graphs; source hash remains the original capture and view has its own hash/byte count. This is source inspection, not a claim that malformed graph facts are valid. Available until close_binary.",
{
evidence_id: z.string().startsWith("ev_"),
component: z.enum(["source", "recovered_ir"]).default("source"),
offset: page.offset,
limit: z.number().int().min(1).max(16384).default(4096),
},
"evidence",
"await rea.evidence({evidence_id:id,offset:0,limit:4096})",
),
};
methods.bytes.input = methods.bytes.input.superRefine((value, context) => {
// Pattern failures need not prevent object refinements from running in Zod.
// Only the named window contract may admit values to exact u64 arithmetic.
const windows = value.windows.flatMap((window) => {
const parsed = byteWindow.safeParse(window);
return parsed.success ? [parsed.data] : [];
});
if (
windows.reduce((sum, window) => sum + window.length, 0) >
MAX_BYTE_BATCH_BYTES
)
context.addIssue({
code: "custom",
path: ["windows"],
message:
"Select at most 64 KiB of bytes per batch; use additional windows/pages in later calls",
});
if (
windows.some(
(window) =>
BigInt(window.address) + BigInt(window.length) > 0xffffffffffffffffn,
)
)
context.addIssue({
code: "custom",
path: ["windows"],
message: "Byte range end exceeds the native u64 address format",
});
});
methods.references.input = methods.references.input.superRefine(
(value, context) => {
if (value.direction === "incoming" && value.scope !== "address")
context.addIssue({
code: "custom",
path: ["scope"],
message: "Incoming references require exact address scope",
});
},
);
export class AtomicSession {
#owner;
#tail = Promise.resolve();
constructor(factory = KunaFactOwner.open) {
this.factory = factory;
}
#admit(work) {
const pending = this.#tail.then(work, work);
this.#tail = pending.catch(() => {});
return pending;
}
get context() {
return this.#owner?.context ?? null;
}
abort() {
this.#owner?.cancel();
}
open(path) {
return this.#admit(async () => {
if (this.#owner) await this.#owner.close();
this.#owner = undefined;
this.#owner = await this.factory(path);
return {
ok: true,
context: this.context,
methods: Object.keys(methods),
evidence_lifetime: "current target lease",
};
});
}
close() {
return this.#admit(async () => {
if (this.#owner) await this.#owner.close();
this.#owner = undefined;
return { ok: true, closed: true };
});
}
call(
name,
input,
signal = new AbortController().signal,
lease = this.context?.lease,
) {
return this.#admit(async () => {
try {
if (signal.aborted)
throw new FactFailure(
"cancelled",
"Request was cancelled before admission",
);
if (!this.#owner)
throw new FactFailure(
"no_target",
"Call open_binary before analysis",
);
if (lease !== this.context.lease)
throw new FactFailure(
"stale_context",
"Target lease changed; do not join facts across leases",
);
if (!Object.hasOwn(methods, name))
throw new FactFailure(
"unknown_method",
`Unknown API method: ${name}`,
);
const contract = methods[name];
const parsed = contract.input.parse(input);
const data = await this.#owner[contract.method](parsed, signal);
if (signal.aborted)
throw new FactFailure(
"cancelled",
"Request was cancelled after observations were collected",
{ partial_observation: data },
);
return { ok: true, context: this.context, data };
} catch (error) {
return {
ok: false,
context: this.context,
error: {
code:
error instanceof z.ZodError
? "invalid_input"
: (error.code ?? "analysis_failed"),
message: error.message,
details: error.details ?? {},
},
};
}
});
}
}
export const reference = createReference(methods);
+145
View File
@@ -0,0 +1,145 @@
import assert from "node:assert/strict";
import test from "node:test";
import Ajv2020 from "ajv/dist/2020.js";
import { AtomicSession, methods, reference } from "./api.mjs";
const factory = () => {
let sequence = 0;
const calls = [];
const create = async (path) => ({
context: { lease: `lease-${++sequence}`, selected_path: path },
close: async () => {
calls.push("close");
},
cancel: () => {
calls.push("cancel");
},
strings: async (input) => {
calls.push(input);
return { items: [], source: { evidence_id: "ev_real-owner" } };
},
});
return { create, calls };
};
test("relation identity and operand-only selectors agree in runtime and advertised schemas", () => {
const validate = new Ajv2020({ strict: false }).compile(
reference("relations").methods[0].input_schema,
);
for (const [request, valid] of [
[{ kind: "operands", seeds: ["0x1:0"], input_indices: [0, 155] }, true],
[{ kind: "operands", seeds: ["0x1:0"], input_indices: [] }, true],
[{ kind: "uses", seeds: ["0x1:0"] }, true],
[
{
kind: "cases",
seeds: ["0x1:0"],
case_values: ["0xffffffffffffffff"],
case_indices: [0, 155],
},
true,
],
[{ kind: "cases", seeds: [0] }, false],
[{ kind: "cases", seeds: ["0x1:0"], case_values: ["0X1"] }, false],
[{ kind: "incoming", seeds: [0] }, true],
[{ kind: "incoming", seeds: ["0"] }, false],
[{ kind: "operands", seeds: [0] }, false],
[{ kind: "uses", seeds: ["0x1:0"], input_indices: [0] }, false],
[{ kind: "operands", seeds: ["0x1:0"], input_indices: [-1] }, false],
]) {
const input = { snapshot: "snap_test", ...request };
assert.equal(methods.relations.input.safeParse(input).success, valid);
assert.equal(validate(input), valid, JSON.stringify(validate.errors));
}
});
test("named contracts reject extra keys and stale leases before provider admission", async () => {
const provider = factory();
const session = new AtomicSession(provider.create);
await session.open("first");
const lease = session.context.lease;
const invalid = await session.call("strings", { query: "x", guessed: true });
assert.equal(invalid.error.code, "invalid_input");
const wrongIdentity = await session.call("relations", {
snapshot: "snap_test",
kind: "incoming",
seeds: ["0"],
});
assert.equal(wrongIdentity.error.code, "invalid_input");
assert.equal(provider.calls.length, 0);
await session.open("second");
const stale = await session.call("strings", { query: "x" }, undefined, lease);
assert.equal(stale.error.code, "stale_context");
assert.deepEqual(provider.calls, ["close"]);
await session.close();
});
test("cancellation after a completed read retains observations under a cancelled outcome", async () => {
const controller = new AbortController();
const session = new AtomicSession(async () => ({
context: { lease: "late-cancel" },
close: async () => {},
strings: async () => {
controller.abort();
return {
source: { evidence_id: "ev_observed" },
items: [{ value: "retained" }],
};
},
}));
await session.open("target");
const result = await session.call(
"strings",
{ query: "x" },
controller.signal,
);
assert.equal(result.ok, false);
assert.equal(result.error.code, "cancelled");
assert.equal(
result.error.details.partial_observation.items[0].value,
"retained",
);
assert.equal(result.context.lease, "late-cancel");
await session.close();
});
test("close drains admitted reads and a failed cleanup retains ownership for retry", async () => {
let release;
let reads = 0;
let closes = 0;
const session = new AtomicSession(async () => ({
context: { lease: "owned" },
strings: () => {
reads++;
return new Promise((resolve) => {
release = resolve;
});
},
close: () => {
if (++closes === 1) throw new Error("cleanup failed");
},
cancel() {},
}));
await session.open("target");
const read = session.call("strings", { query: "x" });
await new Promise((resolve) => setImmediate(resolve));
const close = session.close();
assert.equal(closes, 0);
assert.equal(reads, 1);
release({ items: [] });
assert.equal((await read).ok, true);
await assert.rejects(close, /cleanup failed/u);
assert.equal(session.context.lease, "owned");
await session.close();
assert.equal(session.context, null);
});
test("cancellation before admission makes no provider query; advertised reference schemas compile", async () => {
const provider = factory();
const session = new AtomicSession(provider.create);
await session.open("target");
const stop = new AbortController();
stop.abort();
const cancelled = await session.call("strings", { query: "x" }, stop.signal);
assert.equal(cancelled.error.code, "cancelled");
assert.deepEqual(provider.calls, []);
const ajv = new Ajv2020({ strict: false });
for (const method of reference().methods) ajv.compile(method.input_schema);
await session.close();
});
@@ -0,0 +1,53 @@
# Calls and values surviving full-graph capacity
The question is `sqlite-boundary-question.txt`. Fix it and this acceptance oracle
before either real-model run. Both lanes use the same complete fourteen-method
API, preserved engine, model/effort request, common guidance and resource limits.
This task requires VM decompilation once per fresh lease; it is different work
from the earlier shallow-reference question. Do not compare their task times as
an adapter speedup. Actual client JavaScript dispatch is available in both lanes.
Artifact: Debian `libsqlite3-0:amd64` version `3.40.1-2+deb12u2`,
`libsqlite3.so.0.8.6`, SHA-256
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
The selected VM implementation entry is `0xea6d0`, not its same-name PLT entry.
Pre-model public MCP capture with the new exporter establishes:
- Code is available. Full graph is unavailable with `atomic operation capacity
exceeded`. Calls remain available: 416 retained block-member operations,
consisting of 395 CALL and 21 CALLIND. The 395 direct targets are known, all
21 indirect targets remain unknown. Call-boundary values are available (1,294),
covering targets, arguments and outputs only, with 21,349 operations scanned
over 1,229 recovered blocks. These do not supply a complete definition/use graph.
- At `0xf115c`, recovered call ID `0xf115c:6a84` targets `ram:0x296c0`.
Input zero is `v1000118d6`, `fspec:0x102f4`, annotation true, width 8.
Three recovered inputs use passing storage `register:0x38/8`, `0x30/8`,
`0x10/8` (RDI, RSI, RDX in this verified x86-64 lane).
- At `0xf238b`, call ID `0xf238b:7130` has the same resolved target.
Input zero is `v100012a80`, `fspec:0x1030c`, annotation true, width 8.
Two recovered inputs use passing storage `register:0x38/8`, `0x30/8`.
- Both first arguments are the same snapshot-local SSA value `v30000004b`,
`register:0x38/8`, `is_input:true`, definition null. Other retained argument
values have defining operation IDs outside this boundary view. Exact SSA
identity is supported; source parameter ordinals and general aliases are not.
Independent GNU objdump decoding from the exported VM entry verifies the two
machine CALL targets and register setup. The earlier complete direct-call pair
oracle contains both. Of the 396 physical direct immediate calls, the only one
absent from the recovered 395 is `0xf2493 -> 0x29370` (stack check failure).
The question does not require rediscovering this extra call; the recovered-call
count must not be claimed as complete machine/runtime coverage.
Required final facts: correct selected entry and facet availability/scope; both
call-count groups and target-unknown distinction; both resolved targets separated
from annotation offsets; argument slots/locations; shared first-input identity;
selected machine checks; explicit unreadable definition/use and alias/runtime
limits; real Evidence and subject digest. Do not require the model to report
every additional native metadata counter. Check that its API source arguments
show only one VM analysis and all relevant reads reuse its snapshot/Evidence.
Raw public verification is retained as `facets-v3-mcp.json` and its Evidence
directory under `.cache/atomic-codemode/`. The ordinary VM record (C, mappings,
variables and other ordinary metadata) equals the prior captured engine output.
The independent oracle and precomputed answer are absent from the model workspace.
+103
View File
@@ -0,0 +1,103 @@
// Native loaded-image data is not instruction decoding or original-file mapping.
import { z } from "zod";
import { FactFailure } from "./fact-error.mjs";
import { MAX_BYTE_BATCH_BYTES, NATIVE_OUTPUT_BYTES } from "./limits.mjs";
const hex = z.string().regex(/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/u);
const payload = z.string().regex(/^(?:[0-9a-f]{2})*$/u);
const header = z.object({
binary: z.string(),
kind: z.literal("data"),
start_hex: hex,
end_hex: hex,
count: z
.number()
.int()
.nonnegative()
.max(MAX_BYTE_BATCH_BYTES / 16),
bytes: z.number().int().nonnegative().max(MAX_BYTE_BATCH_BYTES),
hex: payload,
truncated: z.boolean(),
notes: z.array(z.string()),
});
const row = z.object({
address_hex: hex,
size: z.number().int().min(1).max(16),
bytes: payload,
});
const requireFact = (valid, message) => {
if (!valid) throw new FactFailure("invalid_provider_output", message);
};
export function parseByteFacts(text, inputPath, request) {
try {
requireFact(
Buffer.byteLength(text) <= NATIVE_OUTPUT_BYTES,
"Byte source exceeds capture capacity",
);
const raw = JSON.parse(text);
requireFact(
Array.isArray(raw?.rows) && raw.rows.length <= MAX_BYTE_BATCH_BYTES / 16,
"Byte rows exceed capacity or have invalid shape",
);
const doc = header.parse(raw);
requireFact(
doc.binary === inputPath && doc.start_hex === request.address,
"Byte source names a different target/window",
);
requireFact(
doc.bytes <= request.length && doc.count === raw.rows.length,
"Byte source exceeds the request or disagrees with row count",
);
let end = BigInt(request.address);
const pieces = raw.rows.map((value, index) => {
const item = row.parse(value);
requireFact(
BigInt(item.address_hex) === end && item.bytes.length === item.size * 2,
"Byte row address/length is inconsistent",
);
requireFact(
index === raw.rows.length - 1 || item.size === 16,
"Nonfinal byte row is not the declared native width",
);
end += BigInt(item.size);
return item.bytes;
});
requireFact(
end === BigInt(doc.end_hex) &&
end - BigInt(request.address) === BigInt(doc.bytes) &&
pieces.join("") === doc.hex &&
doc.hex.length === doc.bytes * 2,
"Byte rows, payload and extent disagree",
);
return {
hex: doc.hex,
returned_bytes: doc.bytes,
end_address: doc.end_hex,
coverage: {
authority: "native loaded-image bytes",
requested_bytes: request.length,
status:
doc.bytes === request.length
? "complete-for-requested-range"
: "partial",
provider_truncated: doc.truncated,
notes: doc.notes,
original_file_mapping: "unknown",
runtime_memory: "not-observed",
},
};
} catch (error) {
if (error instanceof FactFailure) throw error;
throw new FactFailure(
"invalid_provider_output",
"Native byte window does not match the declared format",
{
reason:
error instanceof z.ZodError
? error.issues[0]?.message.slice(0, 256)
: String(error.message).slice(0, 256),
},
);
}
}
@@ -0,0 +1,104 @@
import test from "node:test";
import assert from "node:assert/strict";
import { parseByteFacts } from "./byte-facts.mjs";
import { AtomicSession } from "./api.mjs";
const make = () => ({
binary: "/input",
kind: "data",
start_hex: "0x20000000000001",
end_hex: "0x20000000000005",
count: 1,
bytes: 4,
hex: "00417fff",
truncated: false,
notes: [],
rows: [
{
address_hex: "0x20000000000001",
size: 4,
bytes: "00417fff",
},
],
});
const request = { address: "0x20000000000001", length: 4 };
test("byte facts keep exact u64 extents, null/nontext bytes and loaded-image authority", () => {
const facts = parseByteFacts(JSON.stringify(make()), "/input", request);
assert.equal(facts.hex, "00417fff");
assert.equal(facts.end_address, "0x20000000000005");
assert.equal(facts.coverage.status, "complete-for-requested-range");
assert.equal(facts.coverage.original_file_mapping, "unknown");
assert.equal(facts.coverage.runtime_memory, "not-observed");
const clipped = parseByteFacts(JSON.stringify(make()), "/input", {
...request,
length: 5,
});
assert.equal(clipped.coverage.status, "partial");
const flagged = make();
flagged.truncated = true;
assert.equal(
parseByteFacts(JSON.stringify(flagged), "/input", request).coverage.status,
"complete-for-requested-range",
);
});
test("wrong byte subject, contradictory duplicated payload, gaps and counts fail before publishing facts", () => {
for (const mutate of [
(d) => (d.binary = "/other"),
(d) => (d.kind = "code"),
(d) => (d.hex = "00417f00"),
(d) => (d.rows[0].address_hex = "0x20000000000000"),
(d) => (d.rows[0].size = 3),
(d) => (d.end_hex = "0x20000000000006"),
(d) => (d.count = 2),
(d) => (d.rows = Array(4097).fill(null)),
]) {
const document = make();
mutate(document);
assert.throws(
() => parseByteFacts(JSON.stringify(document), "/input", request),
{ code: "invalid_provider_output" },
);
}
const empty = make();
Object.assign(empty, {
hex: "",
rows: [],
count: 0,
bytes: 0,
end_hex: request.address,
});
assert.equal(
parseByteFacts(JSON.stringify(empty), "/input", request).coverage.status,
"partial",
);
});
test("byte input capacity, malformed addresses and u64 end overflow reject before native admission", async () => {
let reads = 0;
const session = new AtomicSession(async () => ({
context: { lease: "bytes" },
bytes: () => {
reads += 1;
},
close: () => {},
}));
await session.open("/input");
try {
for (const windows of [
[{ address: "invalid", length: 1 }],
[{ address: "0xffffffffffffffff", length: 1 }],
[
{ address: "0x1000", length: 65536 },
{ address: "0x2000", length: 1 },
],
[{ address: "0x1000", length: 0 }],
]) {
const result = await session.call("bytes", { windows });
assert.equal(result.error.code, "invalid_input", JSON.stringify(result));
}
assert.equal(reads, 0);
} finally {
await session.close();
}
});
@@ -0,0 +1,64 @@
# Cross-function context experiment: pinned question and oracle
This is an experiment acceptance oracle, unavailable to the isolated model.
The question is `sqlite-call-question.txt`. Fix this question and the required
facts before launching either model lane. Both adapters expose the same complete
fourteen-method fact API; Code Mode has four MCP tools, direct has seventeen.
Use the same pinned engine, artifact, model request, reasoning effort, common
envelope guidance, fresh target leases and resource limits. Do not change runtime
sources or HEAD during the model batch. One sample per lane cannot establish a
general advantage, and the client itself already offers JavaScript dispatch in
both lanes. This comparison isolates the server calling adapter, not all forms of
composition or a traditional provider workflow.
The artifact is the local Debian ELF `libsqlite3.so.0.8.6`, SHA-256
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
Independent GNU objdump decoding from the function entries covers `0x64060`
through `0x65400`; existing earlier captures independently pin the same flow.
No target execution or source-code answer is used.
Required facts for an equal completed answer:
- Wrapper entry `0x65180`; direct machine CALL `0x65211` targets internal helper
`0x64060`. The recovered call operation has an fspec annotation as input zero;
that annotation's offset is not the target address.
- Caller LEA `0x65201` passes post-prologue `RSP+0x20` in RDI. Structured passing
storage is `register:0x38`, width 8. The SSA argument and the callee's retained
input are separate snapshot-local identities; matching their storage supplies a
candidate connection. The direct machine transfer and pointer setup support
this particular connection. Do not globally infer source parameter ordinal,
alias completeness or runtime binding from a slot number or a name.
- The helper preserves the input context pointer in RBP. Its 32-bit field at
context offset `0x30` is incremented at `0x64693`/`0x64696`/`0x64699` for an
opening parenthesis and decremented at `0x64c92` for a closing parenthesis.
The parser's token/branch context must support the parenthesis interpretation;
an ADD/SUB instruction alone does not identify the meaning of a field.
- In the wrapper this is `[RSP+0x50]`, initialized to zero with the other context
tail fields. After helper return zero, LOAD `0x65268` and TEST/JZ
`0x65270`/`0x65272` check it. Zero skips the malformed block; nonzero frees the
expression at `0x65274`, clears its output pointer at `0x65279`, then reaches
the malformed diagnostic at `0x65281` through formatter call `0x65290`.
- Return-code-one errors can also reach the malformed block, via
`0x65239`/`0x6523c`; return code 18 selects a separate depth-limit diagnostic
at `0x65330`. No demonstrated runtime-feasibility or full memory-alias claim.
The producer verifier additionally checks two formatter call targets:
`0x5f031 -> 0x2adc0` (free PLT) and `0x5f062 -> 0x27e50` (vmprintf PLT), whereas
their input-zero fspec offsets are `0x10000`/`0x10001`. Resolving these PLT entries
to runtime implementations is outside this task.
Native producer fidelity is checked against previously retained captures of all
three functions: C, ordinary metadata/line mappings, blocks, operations, values
and coverage are identical. Only schema version and structured call records are
added. Older sources used pretty JSON; their smaller/newer encoded sizes must
not be attributed to the call exporter. The earlier compact-writer change is a
separate experiment. Unsupported old schema graphs remain unavailable with valid
C retained, rather than acquiring guessed call records.
The first paired question runs at `01e5c5955` both explain the primary state flow
correctly but omit the oracle's separate return-1/return-18 distinctions. Because
the initial question did not explicitly enumerate those codes, the next round
clarifies that requirement in `sqlite-call-question.txt` before either lane
starts. The required facts and independent oracle stay unchanged. Earlier
questions and costs remain pinned to their original commits; neither incomplete
sample is counted as a full-oracle success.
+20
View File
@@ -0,0 +1,20 @@
import { readFile } from "node:fs/promises";
import { AtomicSession } from "./api.mjs";
import { executeJavaScript } from "./executor.mjs";
const [binary, script] = process.argv.slice(2);
if (!binary || !script)
throw new Error("Usage: node cli.mjs BINARY JAVASCRIPT_FILE");
const session = new AtomicSession();
try {
await session.open(binary);
const result = await executeJavaScript(
session,
await readFile(script, "utf8"),
);
console.log(JSON.stringify(result));
process.exitCode = result.ok ? 0 : 1;
} finally {
session.abort();
await session.close();
}
@@ -0,0 +1,51 @@
# Client waiting and whole-task cost
The [selected-case pair](switch-study.md) exposes one-second client polls after
the snapshot execution yields. Direct makes six waits (three one-second then
three ten-second requests); Code Mode makes three one-second requests. Five
direct and two Code Mode wait responses still report a running cell. Initial
execution yields after about 31 seconds in both lanes. This is observed client
behavior, separate from MCP attempts and native source-capture time.
Native latency remains variable, including in sequential scripted MCP runs
without models. The [overlap canary](overlap-study.md) did not establish a cause.
No analysis acceleration or cache invalidation change is proposed here.
## Frozen diagnostic
The opt-in runner flag `--guide-long-waits` appends exactly the same client wait
policy in both lanes: resume an already-running cell with a 30,000 ms wait,
respect the advertised schema, and do not sleep or duplicate analysis merely
to check progress. It changes neither the initial execution yield policy nor
the provider/Code Mode deadlines. It supplies no analysis facts. The policy,
flag and instruction digest are recorded in each trial's settings. Older runs
without this flag remain a separate stratum.
Run Code Mode first, then direct, with the unchanged
[question](sqlite-switch-question.txt) and [oracle](switch-oracle.md), final
`a9ca8a8a...` engine, complete four/eighteen-tool catalogs and the same exact
discovery/envelope guidance as the first pair. Use fresh leases and isolated
model workspaces, requested `gpt-6.1-sol/high`, Codex 0.161.0 and the same one-CPU,
4 GiB hard / 3 GiB early-stop resource policy. No runtime edits during the pair.
Local records use `sqlite-switch-{codemode,direct}-v2*`.
Review complete answers against the frozen independent machine oracle and
retained source, including selector widths, all requested cases, signed target
arithmetic, case/edge identities, native default versus machine out-of-range
guard, provenance and unknown runtime behavior. Require one VM capture and
focused reads; a lower token count without a correct answer is not a win.
Record actual wait arguments and running/completed responses, model responses,
all input/cached/uncached/output/reasoning tokens, MCP/API attempts, wall time,
family CPU, kernel family peak, native source-capture time, identity/freshness
and cleanup/OOM status. Count discovery, guidance and unsuccessful waits in
whole-task usage. Fewer polls would establish adherence in this client, not
causally lower tokens or native latency. A before/after pair remains subject to
model choices, prompt caches, host contention and run order; do not subtract
native or polling intervals to manufacture an adapter speedup.
The [official Codex CLI reference](https://learn.chatgpt.com/docs/cli/reference)
documents client configuration but does not establish the `wait` contract used
here. The actual captured client calls and subsequent run establish behavior;
no configuration feature flag is treated as proof of the available surface.
The native/API contracts are unchanged. Results are pending.
@@ -0,0 +1,119 @@
// Immediate postdominators on the reversed recovered CFG, using linear retained
// state. Control dependence is derived for finite paths to recovered exits.
import { FactFailure } from "./fact-error.mjs";
export function controlDependencies(
blocks,
{ workLimit = 2_000_000, dependents } = {},
) {
const exit = Symbol("recovered-exit");
const successors = new Map(
blocks.map((block) => [
block.id,
block.successors.map((edge) => edge.target),
]),
);
const reverse = new Map([
[exit, []],
...blocks.map((block) => [block.id, []]),
]);
const tick = (() => {
let work = 0;
return () => {
if (++work > workLimit)
throw new FactFailure(
"capacity",
"Control-dependence work budget exceeded",
);
};
})();
for (const [from, targets] of successors) {
if (!targets.length) targets.push(exit);
for (const to of targets) reverse.get(to).push(from);
}
const { parent, visited } = postdominatorParents(
exit,
successors,
reverse,
tick,
);
const relations = [];
for (const block of blocks) {
if (!parent.has(block.id)) continue;
for (const edge of block.successors) {
let runner = edge.target;
while (
parent.has(runner) &&
runner !== parent.get(block.id) &&
runner !== exit
) {
tick();
if (!dependents || dependents.has(runner))
relations.push({ controller: block.id, dependent: runner, edge });
runner = parent.get(runner);
}
}
}
return {
relations,
coverage: {
scope:
"derived control dependence in the recovered graph; zero-successor blocks treated as exits; finite paths only",
status:
visited.size === blocks.length + 1
? "complete-for-recovered-finite-exit-paths"
: "partial",
exit_unreachable_blocks: blocks
.filter((block) => !visited.has(block.id))
.map((block) => block.id),
nontermination: "unknown",
runtime_reachability: "unknown",
},
};
}
function postdominatorParents(exit, successors, reverse, tick) {
const visited = new Set([exit]);
const stack = [[exit, 0]],
postorder = [];
while (stack.length) {
tick();
const frame = stack.at(-1),
children = reverse.get(frame[0]);
if (frame[1] < children.length) {
const child = children[frame[1]++];
if (!visited.has(child)) {
visited.add(child);
stack.push([child, 0]);
}
} else {
postorder.push(frame[0]);
stack.pop();
}
}
const order = postorder.reverse();
const position = new Map(order.map((id, i) => [id, i]));
const parent = new Map([[exit, exit]]);
const intersect = (a, b) => {
while (a !== b) {
tick();
if (position.get(a) > position.get(b)) a = parent.get(a);
else b = parent.get(b);
}
return a;
};
let changed = true;
while (changed) {
changed = false;
for (const node of order.slice(1)) {
tick();
const known = successors.get(node).filter((id) => parent.has(id));
if (!known.length) continue;
const next = known.slice(1).reduce(intersect, known[0]);
if (parent.get(node) !== next) {
parent.set(node, next);
changed = true;
}
}
}
return { parent, visited };
}
+261
View File
@@ -0,0 +1,261 @@
# Task coverage before cost comparison
The callable projection now also has a completed [PHI-format model pair](trials.md).
Both answers reconstruct all four alternatives with exact retained identities,
independent machine/string checks and partial-scope limits. An observed reference
lookup mismatch adds work in the direct lane; its costs remain recorded and that
discovery boundary needs correction before another same-question comparison.
Code Mode can only compose facts the underlying API actually supplies. A task
cost is comparable after reviewing the completed answer and its evidence. A
successful tool call, fewer response bytes or more graph records is insufficient.
No claim of complete reverse-engineering capability follows from these tasks.
The [boundary-facet question/oracle](boundary-oracle.md) now also has a completed
real-model pair. Both answers correctly use calls and boundary values despite
full-graph capacity failure, reuse one VM analysis and preserve definition/use
and runtime uncertainty. Code Mode is faster in this sample and uses slightly
less cumulative input, while uncached input and maximum response context are
higher. See the full costs and independent source checks in [trials.md](trials.md).
This verifies the narrower recovered-call capability; it does not supply selected
definition bodies when the full graph is unavailable.
The next completed pair reuses that exact question and oracle after adding compact
call counts to the shared snapshot facet. Both adapters receive the same new
metadata, derived after native validation from the complete retained call set;
the engine, fact methods and selection semantics are unchanged. The hypothesis
is that avoiding full-set paging for these counts reduces task costs. Count
coverage remains recovered block-member calls, not complete machine calls or
runtime binding. Both answers satisfy the same oracle and avoid call-set paging.
Cumulative input is lower in both new samples, but elapsed/uncached costs improve
only for direct; direct now beats Code Mode within the pair. These observations
support useful shared aggregates, without establishing a general efficiency
effect. Selected rows/full source Evidence remain available for composition and
audit; all native VM function records equal the earlier captures.
The atomic direct and Code Mode lanes share the same method registry, provider
owner, selectors, failures and retained sources. The traditional lane selects
relevant production tools, including the discovery route advertised by its
errors; its selection and extra discovery guidance are recorded. Use the same
target, engine, authority and question. Freeze runtime files for each model batch.
Keep full original errors, retries and partial answers in the record. Never show
this oracle document or repository sources to a measured model.
Review task facts separately from answer length. A confident false physical
branch target fails the value-origin task even if its reconstructed IR equation
is right. An answer stopping at an import stub does not complete a task whose
other lane traces the formatter implementation. A bounded missing graph is not
an empty graph. Mark unsupported facts unknown and report incompletion when the
question actually needs them.
Cache regimes are distinct: fresh analysis captures, a reused capture in the
same lease, a validated persistent project in a new session, OS file cache and
the model's reported prompt-cache tokens. These experiments do not clear the OS
cache. The prototype has no cross-session result cache. Whole-model cold and
reused-project trials therefore remain separate from the API-only same-lease
cache benchmark. Report cumulative input, included cached input, uncached input,
output and included reasoning; do not invent server compute or price savings.
All resource trials are serial with one CPU, 4 GiB hard memory, early stop at
3 GiB, and an observed cleanup receipt. Kernel family memory includes charged
file cache. A single stochastic trial is an observation, not a reliable ranking;
API-only replay cannot replace real model trials or establish their token cost.
The CLI transcripts establish that all lanes have a client JavaScript `exec`
surface, even with `code_mode=false` requested. Count actual model responses,
client dispatch/wait calls and MCP requests separately. This comparison is not
an ablation against a client incapable of code composition. The optional
`agent-trial.py --normalize-mcp-guidance` adds identical envelope/stopping guidance
to any lane: select `structuredContent` or parsed text once, avoid printing both
encodings, stop when supported and close the target. It is recorded explicitly;
the initial SQLite and VM rounds do not have that later guidance. Their native
analysis commands/profiles also differ (selected functions versus production
project and facet workflows); engine equality alone does not isolate Code Mode.
The pinned SQLite artifact is `/usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6`,
SHA-256 `2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
The malformed-MATCH oracle and earlier unequal-depth results are retained in
[trials.md](trials.md). The next VM task is fixed in `sqlite-vm-question.txt`
before model trials. Its oracle is checked with GNU `objdump -d -M intel` over
the shipped executable, independently of both model answers and Kuna IR. Dynamic
symbol inventory (`nm -D --defined-only`) identifies `sqlite3VdbeExec` at
`0xea6d0`. Reconstructed C and printer line mappings help locate checks, but do
not replace the physical instruction evidence:
| Required local fact | Independently checked anchors |
| --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Open and release strings are distinct | `0x12c500` and `0x12c538`; loads at `0xf2381` and `0xf2437` |
| Operation selector zero takes the open path | Load `[op+4]` at `0xedbe5`, test `0xedbf0`, `JNE 0xf0340` at `0xedbf2` |
| Open is rejected for a positive signed counter | Load `[db+0xd8]` at `0xedbf8`, test `0xedbff`, `JG 0xf2372` at `0xedc01` |
| Release first resolves the named savepoint, then tests counter and selector | Chain `[db+0x2f0]` at `0xf0340`, links `+0x18` at `0xf0358`, name comparison `0xf036f`; signed `JLE` at `0xf0382` bypasses rejection, selector `==1` at `0xf0384` / `0xf0389` selects `0xf242d` |
| Both use the same emission/error tail | Open falls to `sqlite3VdbeError@plt` call `0xf238b`; release jumps there at `0xf2446`; local status becomes `5` at `0xf239d`, then jumps to `0xea828` at `0xf23b8` |
| Missing name differs | Null chain tests `0xf0347` / `0xf035f` reach `0xf1139`; separate format `"no such savepoint: %s"` and call `0xf115c` |
| Static authority and coverage remain explicit | Counter/selector names are interpretations of offsets; runtime feasibility, actual dynamic binding and the final result after shared error processing require additional evidence |
The VM's real native-IR exporter exceeds its operation capacity. It retains
3,654 lines of reconstructed C and complete source Evidence. This task tests
whether a model can use those available facts plus selected physical instruction
windows without inventing a recovered graph. Its local status `5` must not be
silently promoted to an unconditional final function return: shared error
processing can still change behavior. No target execution is authorized here.
The post-serialization-fix Pragma task is fixed in `sqlite-pragma-question.txt`.
It tests a previously inaccessible code facet rather than treating a byte-limit
failure as a fast completed answer. The two experimental calling modes use the
same twelve methods and new engine digest `2290e8b1…`, with common normalization
guidance enabled in both. There is no new traditional cost cell for this task.
For the same pinned SQLite artifact, independent GNU instruction checks establish:
- `sqlite3Pragma` entry `0xb90e0`; value-pointer test `0xbae56` / branch `0xbae5c`
distinguishes the read path from change handling at `0xb9b8c`.
- `0xb9b90` tests connection byte `+0x65`; zero branches at `0xb9b94` to diagnostic
selection at `0xbc58a`. This precedes the additional selector-one bypass at
`0xb9b9a` / `0xb9b9f`, so their ordering must be preserved.
- Allowed local update computes `(value+1)&7`, substitutes one for zero, and
stores the selected record's byte `+0x10` at `0xb9bdc`; byte `+0x11` becomes one
at `0xb9be3`. The input interpretation and source member names are derivations.
- The actual diagnostic call is `0xbc596` through `sqlite3ErrorMsg@plt` at
`0x2a1e0`; `0xbc59b` jumps to common cleanup `0xb9293`.
- Cleanup releases both temporary strings, checks the stack canary, and returns
at `0xb92d9`. Do not invent a fixed numeric return from this reconstructed
void routine. No SQL-level runtime trigger is demonstrated.
Pragma's complete retained code and mappings are checked equal before/after the
serialization fix. Its recovered graph remains explicitly unavailable at the
encoded-byte capacity; that is a coverage limitation in both experimental lanes.
The next seeded table-composition task is fixed in `sqlite-table-question.txt`.
Both experimental modes have the same thirteen methods including `bytes`; a
traditional comparison is not claimed because its selected catalog/available
Kuna byte capability has not been established for this task. GNU disassembly of
the pinned artifact shows `sqlite3GetToken` at `0xe1480`: unsigned input byte load
at `0xe1480`, class-table base `0x12bcc0` at `0xe1484`, bound `0x1e` at `0xe1492`,
class reload `0xe1498`, jump-table base `0x12b600` at `0xe149c`, sign-extending
32-bit table load at `0xe14a3`, base addition `0xe14a7`, indirect jump `0xe14aa`.
The out-of-range path begins `0xe14f8`.
Independent direct file reads are valid here because `readelf -lW` binds this
artifact's selected addresses to the same file offsets in its read-only LOAD
segment (`0x11a000..0x154bfc`). A Python `struct.unpack_from('<i', ...)` check of
the actual table bytes gives these initial dispatch facts, before model trials:
| Input byte | Class | Signed displacement | Initial machine destination |
| ---------- | ----: | ------------------: | --------------------------- |
| `0x2d` | 11 | -302928 | `0xe16b0` |
| `0x2f` | 16 | -302144 | `0xe19c0` |
| `0x27` | 8 | -302480 | `0xe1870` |
| `0x22` | 8 | -302480 | `0xe1870` |
| `0x3f` | 6 | -302272 | `0xe1940` |
| `0x80` | 27 | -303440 | `0xe14b0` |
This oracle establishes static table/dispatch arithmetic, not final token
semantics, future runtime table contents or a feasible SQL input. Models must
retain those boundaries. This document is not supplied to measured models.
## Cross-function facts work; adapter gains remain mixed
The structured-call experiment closes an observed native export gap: the call's
fspec annotation does not identify its target code address, and an argument's
SSA storage is not its passing location. Reusing the native call-spec owner now
allows a model to connect a caller value to a callee input without parsing a
display string. Real SQLite answers support this connection with machine
instructions, trace a shared parenthesis counter across two functions, and
preserve the limits of recovered slots, aliases and runtime feasibility.
The first question pair omitted oracle-required alternative return paths in both
answers. Its wording was less explicit than its oracle, so that result stays a
partial-coverage record. A new question clarification was committed before the
next pair; both then satisfy the task. The failed engine-pin setup launch also
remains recorded. No sample is silently converted into a completed task.
In the correctness-equal pair, Code Mode uses 596,978 cumulative input tokens and
156.821 seconds versus direct's 611,321 and 147.424 seconds. Uncached input,
output, largest observed response input and family CPU are higher for Code Mode;
family memory peak is lower. Both use the same fourteen fact methods and a client
with JavaScript dispatch. The evidence supports usable composable call/input
facts, without proving that this server adapter is faster or cheaper. Broader
complex-software superiority and provider parity remain unproven.
Two remaining boundaries deserve independent experiments. First, the initial call view lived inside full recovered IR, so graph capacity
could hide a small otherwise useful call view. The next exporter revision now
captures calls and their boundary values independently; real VM verification
retains both while the graph exceeds capacity. Second, shallow outgoing references should serve a callee/reference list before
requesting deeper argument/value recovery. The subsequent implementation now
exposes Kuna's existing `xrefs --from` through the shared parser, with native kind
filters, explicit scope and paging. Real VM known-target references match an
independent disassembly pair set, and remain available when the full IR facet
exceeds capacity. The next pre-pinned model question measures this use directly. Keeping those costs
and coverage distinctions explicit is more useful than treating a smaller tool
catalog as proof of better analysis.
The shallow-reference question now completes through both real-model adapters.
Each composes 396 VM reference rows into 171 exact destinations and 16 calls to
the selected PLT address, then distinguishes a real stack-check call absent from
optimized IR. Both keep unresolved indirect/discovery/runtime limits. Native
reference pairs from both model sessions match the complete independent set of
396 direct immediate machine calls; the separate 21 indirect instructions are
not silently included in those known-target totals.
In this one pair, Code Mode takes 93.837 seconds and 264,680 cumulative input
versus direct's 106.183 and 285,086. Their maximum observed response input
contexts are almost equal. This is useful positive evidence for selected facts
plus local aggregation, while prior completed tasks show mixed adapter costs.
It supports continuing the experiment, not a universal speed/token/provider
superiority claim. Tool coverage and representation semantics were prerequisites
for this comparison: a recovered-call-only view would have missed a real call,
and VM full-IR capacity cannot establish that its shallow references are absent.
An additional shared CLI/API run with the unpatched stock Kuna 1.833 executable
also returns the same 396/171/16 VM aggregates, with fresh owned HOME/cache. The
outgoing-reference capability uses native support, independently of the opt-in
IR exporter. It takes 3.044 family wall / 2.695 CPU seconds and 182.6 MiB peak,
with cleanup verified and OOM 0; these scripted costs are not model-token data or
an adapter comparison. Stock external-spec freshness is still outside the engine
binary digest. Full repository typechecking remains resource-limited as recorded
in `native-engine.md`; source build, focused tests, changed-test typing and full
lint/module checks pass.
The bounded call-rooted definition projection now supports a local VM PHI task
that full-graph capacity previously prevented. Four real-model answers over two
revisions correctly recover both PHIs, all four alternatives and machine
transfers, with source identities and global partial coverage retained. This is
selected static-task coverage, not general complex-software superiority.
The discovery correction removes an observed name-list miss and fixes defaulted
input schemas, but the second pair does not show stable cost improvement. Direct
has a 120-second native timeout before a successful retry; Code Mode changes its
query strategy and costs more than its first run. The complete costs and failure
remain in [the ledger](trials.md#phi-alternatives-after-named-discovery-correction).
Fairness requires sufficient facts, correct discovery and honest retry reporting.
The wide-PHI experiment provides a more precise candidate atom: an input slot is
an individually selectable relationship, not just an array inside an operation.
Its identity includes the operation and slot index; several relationships can
share one value while preserving different predecessor edges. A value definition
can then be composed with selected operation/value reads. Both measured models
recover those distinctions and separate retained-node absence from unretained
membership without fetching all 156 slots.
This validates the shared query boundary, not adapter superiority. Code Mode
groups the same seven fact requests, but four discovery lookups erase the
MCP-request reduction at the observed model-response/client-dispatch level. Its
95.699 seconds versus direct's 91.817, nearly equal cumulative input and lower
uncached input remain a mixed single sample. Discovery is itself part of the
analyst workflow and deserves a separate test. Broader behavior reconstruction,
complete large-function CFG/use coverage, aliases and a universal RE basis are
still unproven. See [the complete scoped results](trials.md#selected-wide-phi-input-relationships).
The same question after adding a static method index again completes correctly
through both adapters. Code Mode now uses two discovery calls instead of four,
without changing the seven fact reads. Observed model responses decrease by one,
but total time increases slightly; direct also takes longer and uses fewer input
tokens without changing query counts. The second pair is approximately 98.8
seconds each, with more cumulative and less uncached input for Code Mode. This
supports the index's discoverability, while leaving whole-task cost mixed.
There is a further distinction to test: an atomic fact API does not itself make
native computation atomic. A fresh selected-slot question still incurs an entire
function decompilation and C rendering. Phase costs are unmeasured, so neither
removing C nor introducing persistent state is yet justified as a speed fix.
Measure that boundary before claiming analysis-time improvement from narrower
fact responses.
+241
View File
@@ -0,0 +1,241 @@
import { getQuickJS } from "quickjs-emscripten";
import { methods } from "./api.mjs";
import { MAX_BRIDGE_BYTES as MAX_BRIDGE } from "./limits.mjs";
const MAX_OUTPUT = 32768;
const failure = (code, message) => ({ ok: false, error: { code, message } });
const projectGuestError = (code, error) => {
const message =
typeof error?.message === "string" ? error.message : String(error);
// Check raw size first: JSON escaping expands each code unit at most sixfold,
// so the second check has a 6×32 KiB string bound plus a fixed envelope.
if (
Buffer.byteLength(message) > MAX_OUTPUT ||
Buffer.byteLength(JSON.stringify({ code, message })) > MAX_OUTPUT
)
return {
code: "output_budget",
message:
"JavaScript error exceeds the 32 KiB output budget; its message is omitted. Native Evidence remains retained.",
details: {
original_error_code: code,
original_message_code_units: message.length,
original_message_omitted: true,
},
};
return { code, message };
};
export const executeJavaScript = async (
session,
code,
{
signal = new AbortController().signal,
timeoutMs = 120000,
cpuBudgetMs = 2000,
} = {},
) => {
if (Buffer.byteLength(code) > 32768)
return failure("code_budget", "JavaScript source exceeds 32 KiB");
const started = performance.now();
const context = session.context;
const lease = context?.lease;
const stop = new AbortController();
const combined = AbortSignal.any([signal, stop.signal]);
const deadline = Date.now() + timeoutMs;
const timer = setTimeout(() => stop.abort(), timeoutMs);
const QuickJS = await getQuickJS();
const runtime = QuickJS.newRuntime();
runtime.setMemoryLimit(64 * 1024 * 1024);
runtime.setMaxStackSize(512 * 1024);
const budget = { used: 0, deadline: 0, exhausted: false };
const runGuest = (work) => {
budget.deadline = Date.now() + Math.max(1, cpuBudgetMs - budget.used);
const before = performance.now();
try {
return work();
} finally {
budget.used += performance.now() - before;
}
};
runtime.setInterruptHandler(() => {
budget.exhausted ||= Date.now() >= budget.deadline;
return budget.exhausted || combined.aborted || Date.now() >= deadline;
});
const vm = runtime.newContext();
const pending = new Set();
const calls = [];
const state = { disposed: false, evidence: new Set() };
const pump = () => {
if (!state.disposed) {
const result = runGuest(() => runtime.executePendingJobs());
if (result.error) {
result.error.dispose();
stop.abort();
}
}
};
installBridge(vm, session, { combined, lease, calls, pending, pump, state });
let outcome;
try {
const text = await evaluateGuest(vm, code, { runGuest, pump, combined });
if (Buffer.byteLength(text) > MAX_OUTPUT)
outcome = {
ok: false,
error: {
code: "output_budget",
message:
"Return a smaller projection (32 KiB UTF-8 output budget); native Evidence remains retained",
},
};
else
outcome = {
ok: true,
value: text === "undefined" ? null : JSON.parse(text),
};
} catch (error) {
outcome = {
ok: false,
error: projectGuestError(
budget.exhausted
? "cpu_budget"
: combined.aborted
? "cancelled"
: "javascript_error",
budget.exhausted
? "Guest synchronous execution budget exceeded"
: error,
),
};
} finally {
clearTimeout(timer);
stop.abort();
await Promise.allSettled(pending);
state.disposed = true;
vm.dispose();
runtime.dispose();
}
return {
...outcome,
context,
evidence_ids: [...state.evidence],
calls,
elapsed_ms: performance.now() - started,
};
};
const retainEvidence = (response, evidence) => {
const observation =
response.data ?? response.error?.details?.partial_observation;
for (const row of [observation, ...(observation?.results ?? [])]) {
const id =
row?.source?.evidence_id ??
row?.evidence_id ??
row?.error?.details?.evidence_id;
if (id) evidence.add(id);
}
if (response.error?.details?.evidence_id)
evidence.add(response.error.details.evidence_id);
};
const installBridge = (
vm,
session,
{ combined, lease, calls, pending, pump, state },
) => {
const bridge = vm.newFunction("analysis", (methodHandle, inputHandle) => {
const name = vm.getString(methodHandle);
const inputText = vm.getString(inputHandle);
if (calls.length >= 64 || Buffer.byteLength(inputText) > MAX_BRIDGE)
throw new Error("Host call or argument budget exceeded");
const promise = vm.newPromise();
const index = calls.length;
calls.push({
method: name,
input: JSON.parse(inputText),
status: "pending",
});
const work = Promise.resolve()
.then(async () => {
const before = performance.now();
const response = await session.call(
name,
JSON.parse(inputText),
combined,
lease,
);
retainEvidence(response, state.evidence);
calls[index].status = response.ok ? "ok" : "error";
if (!response.ok) calls[index].error = response.error;
calls[index].elapsed_ms = performance.now() - before;
let serialized = JSON.stringify(response);
if (Buffer.byteLength(serialized) > MAX_BRIDGE)
serialized = JSON.stringify({
ok: false,
error: {
code: "bridge_budget",
message:
"Select fewer fields or a smaller page; full source remains available via evidence",
},
});
if (!state.disposed) {
const value = vm.newString(serialized);
promise.resolve(value);
value.dispose();
pump();
}
})
.catch((error) => {
if (!state.disposed) {
const value = vm.newString(
JSON.stringify({
ok: false,
error: { code: "bridge_failed", message: error.message },
}),
);
promise.resolve(value);
value.dispose();
pump();
}
})
.finally(() => pending.delete(work));
pending.add(work);
return promise.handle;
});
vm.setProp(vm.global, "__analysis", bridge);
bridge.dispose();
};
const evaluateGuest = async (vm, code, { runGuest, pump, combined }) => {
const bootstrap = `globalThis.rea=Object.freeze(Object.fromEntries(${JSON.stringify(Object.keys(methods))}.map(name=>[name,async(input={})=>JSON.parse(await __analysis(name,JSON.stringify(input)))])));`;
vm.unwrapResult(runGuest(() => vm.evalCode(bootstrap))).dispose();
const promiseHandle = vm.unwrapResult(
runGuest(() =>
vm.evalCode(
`(async()=>JSON.stringify(await(async()=>{\n${code}\n})()))()`,
),
),
);
try {
const resolved = vm.resolvePromise(promiseHandle);
pump();
// A never-settling guest Promise must not hang the MCP request forever.
const aborted = new Promise((_, reject) => {
if (combined.aborted) {
reject(new Error("Execution cancelled"));
return;
}
combined.addEventListener(
"abort",
() => reject(new Error("Execution cancelled or timed out")),
{ once: true },
);
});
const result = await Promise.race([resolved, aborted]);
const value = vm.unwrapResult(result);
const text = vm.getString(value);
value.dispose();
return text;
} finally {
promiseHandle.dispose();
}
};
@@ -0,0 +1,120 @@
import test from "node:test";
import assert from "node:assert/strict";
import { executeJavaScript } from "./executor.mjs";
const fake = {
context: { lease: "test" },
call: async (name, input) => ({ ok: true, data: { name, input } }),
};
test("real QuickJS composes async API calls and does not expose host globals", async () => {
const result = await executeJavaScript(
fake,
'const a=await rea.strings({query:"x"}); const b=await rea.operations({ids:[a.data.name]}); return {a,b,host:[typeof process,typeof require,typeof fetch]};',
);
assert.equal(result.ok, true, JSON.stringify(result));
assert.deepEqual(result.value.host, ["undefined", "undefined", "undefined"]);
assert.equal(result.calls.length, 2);
assert.equal(result.value.b.data.input.ids[0], "strings");
});
test("infinite guest execution and never-resolving promises terminate", async () => {
const loop = await executeJavaScript(fake, "while(true){}", {
timeoutMs: 30,
});
assert.equal(loop.ok, false);
const stalled = await executeJavaScript(
fake,
"await new Promise(()=>{});return 1",
{ timeoutMs: 30 },
);
assert.equal(stalled.ok, false);
});
test("output budgets and failures survive without silently clipping evidence", async () => {
const result = await executeJavaScript(fake, 'return "x".repeat(40000)');
assert.equal(result.error.code, "output_budget");
const failure = await executeJavaScript(
fake,
'throw new Error("explicit failure")',
);
assert.equal(failure.ok, false);
});
test("guest CPU budget is independent from the native-analysis wall deadline", async () => {
const before = performance.now();
const result = await executeJavaScript(fake, "while(true){}", {
timeoutMs: 5000,
cpuBudgetMs: 25,
});
assert.equal(result.error.code, "cpu_budget");
assert.ok(performance.now() - before < 2000);
});
test("cancelled reads keep partial-observation Evidence in the execution receipt", async () => {
const result = await executeJavaScript(
{
context: { lease: "partial" },
call: async () => ({
ok: false,
error: {
code: "cancelled",
details: {
partial_observation: { source: { evidence_id: "ev_partial" } },
},
},
}),
},
"return await rea.strings({query:'x'})",
);
assert.deepEqual(result.evidence_ids, ["ev_partial"]);
assert.equal(result.value.error.code, "cancelled");
});
test("failed compositions keep completed facts and their provenance after draining", async () => {
const owner = {
context: { lease: "retained" },
call: async () => ({ ok: true, data: { evidence_id: "ev_retained" } }),
};
for (const ending of [
'throw new Error("after analysis")',
'return "x".repeat(40000)',
]) {
const result = await executeJavaScript(
owner,
`await rea.strings({}); ${ending}`,
);
assert.equal(result.ok, false);
assert.deepEqual(result.evidence_ids, ["ev_retained"]);
assert.deepEqual(result.context, owner.context);
assert.equal(result.calls[0].status, "ok");
}
});
test("guest exceptions obey output capacity, including JSON expansion, and preserve observations", async () => {
for (const code of [
'throw new Error("x".repeat(40000))',
'throw new Error("\\0".repeat(6000))',
]) {
const result = await executeJavaScript(
{
context: { lease: "error-output" },
call: async () => ({
ok: true,
data: { evidence_id: "ev_before_error" },
}),
},
`await rea.strings({}); ${code}`,
);
assert.equal(result.ok, false);
assert.equal(result.error.code, "output_budget");
assert.equal(result.error.details.original_error_code, "javascript_error");
assert.equal(result.error.details.original_message_omitted, true);
assert.ok(Buffer.byteLength(JSON.stringify(result.error)) <= 32768);
assert.deepEqual(result.evidence_ids, ["ev_before_error"]);
assert.equal(result.context.lease, "error-output");
}
const ordinary = await executeJavaScript(
fake,
'throw new Error("x".repeat(10000))',
);
assert.equal(ordinary.error.code, "javascript_error");
assert.ok(ordinary.error.message.includes("x".repeat(10000)));
});
@@ -0,0 +1,24 @@
import { projectAnalysisError } from "../../dist/domain/analysisErrorProjection.js";
// Shared expected-failure algebra; execution adapters do not depend on native protocols.
export class FactFailure extends Error {
constructor(code, message, details = {}) {
super(message);
this.code = code;
this.details = details;
}
}
export const requireResult = (result, code = "provider_failure") => {
if (!result.ok) {
const failure = result.error?._tag
? projectAnalysisError(result.error)
: result.error;
throw new FactFailure(
failure?.code ?? code,
String(failure?.message ?? failure?.reason ?? failure),
{ failure },
);
}
return result.value;
};
@@ -0,0 +1,69 @@
// Shared read projections and partial batch semantics; no provider protocols.
// Parsed JSON facts are owned by their cache. SDK callers may retain views,
// but must not be able to change later reads under the same Evidence identity.
export function freezeFacts(value) {
if (value && typeof value === "object") {
for (const child of Object.values(value)) freezeFacts(child);
Object.freeze(value);
}
return value;
}
export const sourceSummary = ({ evidence_id, sha256, bytes, elapsed_ms }) => ({
evidence_id,
sha256,
bytes,
elapsed_ms,
});
export const projectFact = (row, fields, identity) =>
fields
? Object.fromEntries(
[...new Set([...identity, ...fields])]
.filter((key) => Object.hasOwn(row, key))
.map((key) => [key, row[key]]),
)
: row;
export const pageFacts = (rows, input, identity = []) => ({
items: rows
.slice(input.offset, input.offset + input.limit)
.map((row) => projectFact(row, input.fields, identity)),
total: rows.length,
next_offset:
input.offset + input.limit < rows.length
? input.offset + input.limit
: null,
});
export async function readFactBatch(requests, capture, read) {
const results = [];
for (const request of requests) {
let source;
try {
const captured = await capture(request);
source = captured.source;
const facts = await read(captured, request);
results.push({
...facts,
ok: true,
address: request.address,
source: sourceSummary(source),
});
} catch (error) {
results.push({
ok: false,
address: request.address,
error: {
code: error.code ?? "provider_failure",
message: error.message,
details: {
...error.details,
...(source ? { evidence_id: source.evidence_id } : {}),
},
},
});
}
}
return { results };
}
@@ -0,0 +1,21 @@
// Read-only kernel counters for the currently bounded diagnostic payload.
import assert from "node:assert/strict";
import { readFile } from "node:fs/promises";
export async function familyCounters() {
const group = (await readFile("/proc/self/cgroup", "utf8"))
.split("\n")
.find((row) => row.startsWith("0::"))
?.slice(3);
assert.ok(group?.includes("rea-efficiency-"), "Run under resource-run.py");
const cpu = (await readFile(`/sys/fs/cgroup${group}/cpu.stat`, "utf8")).match(
/^usage_usec (\d+)$/mu,
);
assert.ok(cpu, "Missing cgroup CPU counter");
return {
cpu: Number(cpu[1]),
peak: Number(
(await readFile(`/sys/fs/cgroup${group}/memory.peak`, "utf8")).trim(),
),
};
}
@@ -0,0 +1,33 @@
import { parseKunaInventory } from "../../dist/kuna/KunaProjectFiles.js";
import { requireResult } from "./fact-error.mjs";
import { freezeFacts, pageFacts } from "./fact-reads.mjs";
export const parseFunctionFacts = (text) =>
freezeFacts(
requireResult(parseKunaInventory(text), "invalid_provider_output"),
);
export function selectFunctionFacts(functions, input) {
const addresses = input.addresses && new Set(input.addresses);
const normalize = (name) =>
input.case_sensitive ? name : name.toLowerCase();
const query = normalize(input.query);
const matches = functions.filter(
(entry) =>
(!addresses || addresses.has(entry.address)) &&
[entry.name, ...entry.aliases].some((name) =>
input.exact
? normalize(name) === query
: normalize(name).includes(query),
),
);
return {
...pageFacts(matches, input, ["address"]),
coverage: {
scope: "native callable-entry inventory, including import stubs",
discovery: "unknown",
extents: "upper bounds clipped at the next entry; zero means unmeasured",
symbol_binding_at_runtime: "unknown",
},
};
}
@@ -0,0 +1,93 @@
import test from "node:test";
import assert from "node:assert/strict";
import { parseFunctionFacts, selectFunctionFacts } from "./function-facts.mjs";
import { readFactBatch } from "./fact-reads.mjs";
import { FactFailure } from "./fact-error.mjs";
test("function names retain multiple entries and aliases instead of choosing a PLT or implementation", () => {
const facts = parseFunctionFacts(
JSON.stringify({
functions: [
{
name: "format_error",
address_hex: "0x1000",
aliases: ["format_error@plt"],
size: 16,
},
{
name: "format_error",
address_hex: "0x2000",
aliases: ["FormatError"],
size: 0,
},
],
}),
);
const query = {
query: "format_error",
exact: true,
case_sensitive: true,
offset: 0,
limit: 20,
};
assert.deepEqual(
selectFunctionFacts(facts, query).items.map((item) => item.address),
["0x1000", "0x2000"],
);
assert.equal(
selectFunctionFacts(facts, { ...query, query: "format_error@plt" }).items[0]
.address,
"0x1000",
);
assert.equal(
selectFunctionFacts(facts, {
...query,
query: "FORMAT_ERROR",
case_sensitive: false,
}).total,
2,
);
assert.deepEqual(
selectFunctionFacts(facts, {
...query,
addresses: ["0x2000"],
fields: ["size"],
}).items,
[{ address: "0x2000", size: 0 }],
);
assert.throws(() => {
facts[0].aliases.push("foreign");
});
});
test("batch transport and decoder failures preserve successful items and captured source identity", async () => {
const attempts = [];
const result = await readFactBatch(
[{ address: "0x1" }, { address: "0x2" }, { address: "0x3" }],
async (request) => {
attempts.push(request.address);
if (request.address === "0x2")
throw new FactFailure("capacity", "transport bound");
return {
source: {
evidence_id: `ev_${request.address}`,
sha256: "digest",
bytes: 3,
elapsed_ms: 1,
},
};
},
async (_, request) => {
if (request.address === "0x3")
throw new FactFailure(
"invalid_provider_output",
"decoder rejected source",
);
return { items: [{ value: 1 }] };
},
);
assert.deepEqual(attempts, ["0x1", "0x2", "0x3"]);
assert.equal(result.results[0].items[0].value, 1);
assert.equal(result.results[1].error.code, "capacity");
assert.equal(result.results[2].error.details.evidence_id, "ev_0x3");
});
@@ -0,0 +1,109 @@
// The machine-decoding boundary is separate from optimized recovered IR.
import { z } from "zod";
import { FactFailure } from "./fact-error.mjs";
import { NATIVE_OUTPUT_BYTES } from "./limits.mjs";
const hex = z.string().regex(/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/u);
const rowSchema = z.object({
address_hex: hex,
size: z.number().int().positive().max(65536),
bytes: z.string().regex(/^(?:[0-9a-f]{2})+$/u),
mnemonic: z.string(),
operands: z.string(),
text: z.string(),
});
const envelopeSchema = z.object({
binary: z.string(),
kind: z.literal("code"),
start_hex: hex,
end_hex: hex,
count: z.number().int().nonnegative().max(4096),
bytes: z.number().int().nonnegative(),
truncated: z.boolean(),
notes: z.array(z.string()),
});
const requireFact = (condition, message) => {
if (!condition) throw new FactFailure("invalid_provider_output", message);
};
export function parseInstructions(
text,
inputPath,
request,
fields = ["bytes", "text"],
) {
try {
requireFact(
Buffer.byteLength(text) <= NATIVE_OUTPUT_BYTES,
"Instruction source exceeds transport capacity",
);
const raw = JSON.parse(text);
requireFact(
Array.isArray(raw?.instructions) && raw.instructions.length <= 4096,
"Instruction window exceeds decoded-row capacity",
);
const doc = envelopeSchema.parse(raw);
requireFact(
doc.binary === inputPath && doc.start_hex === request.address,
"Instruction source does not match the selected input/window",
);
requireFact(
doc.count === raw.instructions.length && doc.count <= request.count,
"Instruction count does not match the requested window",
);
let next = BigInt(request.address);
const items = raw.instructions.map((value) => {
const row = rowSchema.parse(value);
requireFact(
BigInt(row.address_hex) === next && row.bytes.length === row.size * 2,
"Instruction bytes/size or sequential address mismatch",
);
next += BigInt(row.size);
const fact = {
address: row.address_hex,
size: row.size,
status: row.mnemonic === ".byte" ? "undecoded" : "decoded",
bytes: row.bytes,
mnemonic: row.mnemonic,
operands: row.operands,
text: row.text,
};
return Object.fromEntries(
[...new Set(["address", "size", "status", ...fields])].map((key) => [
key,
fact[key],
]),
);
});
requireFact(
next === BigInt(doc.end_hex) &&
next - BigInt(request.address) === BigInt(doc.bytes),
"Instruction window end/byte coverage mismatch",
);
return {
items,
resume_address: doc.end_hex,
coverage: {
authority: "machine bytes and native linear decoding; not recovered IR",
request,
listed_count: items.length,
truncated: doc.truncated,
notes: doc.notes,
alignment:
"caller-selected start; runtime execution and instruction alignment are not established",
},
};
} catch (error) {
if (error instanceof FactFailure) throw error;
throw new FactFailure(
"invalid_provider_output",
"Native instruction window does not match the declared format",
{
reason:
error instanceof z.ZodError
? error.issues[0]?.message.slice(0, 256)
: String(error.message).slice(0, 256),
},
);
}
}
@@ -0,0 +1,92 @@
import test from "node:test";
import assert from "node:assert/strict";
import { parseInstructions } from "./instruction-facts.mjs";
const document = (address = "0xd97e") => ({
binary: "/private/input",
kind: "code",
start_hex: address,
end_hex: `0x${(BigInt(address) + 6n).toString(16)}`,
count: 1,
bytes: 6,
truncated: true,
notes: ["selected window"],
instructions: [
{
address_hex: address,
size: 6,
bytes: "0f87e9080000",
mnemonic: "JA",
operands: "0xe26d",
text: "JA 0xe26d",
},
],
});
const parse = (doc, address = doc.start_hex) =>
parseInstructions(JSON.stringify(doc), "/private/input", {
address,
count: 1,
});
test("instruction facts preserve exact bytes/addresses and do not substitute an optimized CFG target", () => {
const result = parse(document());
assert.equal(result.items[0].address, "0xd97e");
assert.equal(result.items[0].text, "JA 0xe26d");
assert.equal(result.items[0].bytes, "0f87e9080000");
assert.equal(result.items[0].status, "decoded");
assert.deepEqual(result.coverage.notes, ["selected window"]);
const high = document("0x20000000000001");
high.instructions[0].address = Number(BigInt(high.start_hex));
assert.equal(parse(high).items[0].address, "0x20000000000001");
});
test("unknown bytes remain undecoded and projection retains byte-range identity", () => {
const doc = document();
doc.instructions[0].mnemonic = ".byte";
const result = parseInstructions(
JSON.stringify(doc),
"/private/input",
{ address: "0xd97e", count: 1 },
["mnemonic"],
);
assert.deepEqual(result.items[0], {
address: "0xd97e",
size: 6,
status: "undecoded",
mnemonic: ".byte",
});
});
test("wrong target/window, gaps, extent and byte-size contradictions fail at decoding boundary", () => {
for (const mutate of [
(doc) => {
doc.binary = "/other/input";
},
(doc) => {
doc.instructions[0].address_hex = "0xd97f";
},
(doc) => {
doc.instructions[0].size = 5;
},
(doc) => {
doc.end_hex = "0xd985";
},
(doc) => {
doc.count = 2;
},
]) {
const doc = document();
mutate(doc);
assert.throws(
() => parse(doc),
(error) => error.code === "invalid_provider_output",
);
}
assert.throws(() => parse(document(), "0xd970"), /window/u);
});
test("decoded-row capacity rejects oversized malformed arrays before expanding errors", () => {
const doc = document();
doc.instructions = Array.from({ length: 4097 }, () => ({}));
assert.throws(() => parse(doc), /capacity/u);
});
@@ -0,0 +1,86 @@
--- a/decompiler/crates/kuna-console/src/project.rs
+++ b/decompiler/crates/kuna-console/src/project.rs
@@ -693,6 +693,17 @@
flow_ovr.extend(seed.flow_overrides.iter().cloned());
let replay = crate::assertions::has_symbol_scoped(prog, &name, single_target);
prog.arch_mut().kuna_record_names = replay;
+ let mut profile_clock = (want_atomic_facts
+ && std::env::var_os("KUNA_EXPERIMENT_ATOMIC_TIMING").as_deref()
+ == Some(std::ffi::OsStr::new("1")))
+ .then(std::time::Instant::now);
+ let mut profile_phase = |phase: &str| {
+ if let Some(clock) = profile_clock.as_mut() {
+ let now = std::time::Instant::now();
+ eprintln!("{{\"kind\":\"rea_atomic_phase\",\"address\":\"0x{byte_address:x}\",\"phase\":\"{phase}\",\"wall_ns\":{}}}", now.duration_since(*clock).as_nanos());
+ *clock = std::time::Instant::now();
+ }
+ };
let step = crate::decompile_step::decompile_one(
prog.arch_mut(),
&name,
@@ -751,6 +762,7 @@
}
other => other,
};
+ profile_phase("decompile_drive");
match result {
Ok(mut fd) => {
// (kuna `--assert`) The flow overrides the follower REFUSED are only
@@ -813,6 +825,7 @@
kuna_decomp::kuna_elemptr::record(prog.arch_mut(), &fd);
prog.arch_mut().print_mut().set_record_pointer_arguments(want_pointer_arguments);
let rendered = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
+ profile_phase("pre_render_metadata");
// Trim the surrounding newlines the same way `kuna decompile`
// does (`decompile.rs::trim_newlines`), so the per-function
// `code` is byte-identical to the single-shot path.
@@ -826,6 +839,7 @@
};
let pointerargs = prog.arch_mut().print_mut().take_pointer_arguments(&untrimmed);
let code = untrimmed.trim_matches('\n').to_string();
+ profile_phase("render_c");
let globals = kuna_decomp::decompile_drive::extract_global_objects(prog.arch());
let field_calls = kuna_decomp::kuna_calltargettype::field_declarations(prog.arch(), &fd);
let mut variables =
@@ -848,15 +862,20 @@
prog, &fd, &var_refs, &raw_tokens, &untrimmed, &code, &variables,
))
});
+ profile_phase("metadata_extract");
if want_atomic_facts {
let detail = detail.get_or_insert_with(Default::default);
let membership = kuna_decomp::atomic_membership::inspect(&fd);
+ profile_phase("membership");
detail.atomic_calls = Some(membership.as_ref().map_err(Clone::clone)
.and_then(|m| kuna_decomp::atomic_calls::collect_all(&fd, m)));
+ profile_phase("calls");
detail.atomic_facts = Some(membership.as_ref().map_err(Clone::clone)
.and_then(|m| kuna_decomp::atomic_facts::collect(prog.arch(), &fd, m)));
+ profile_phase("full_graph");
detail.atomic_projection = Some(membership.as_ref().map_err(Clone::clone)
.and_then(|m| kuna_decomp::atomic_projection::collect(prog.arch(), &fd, m)));
+ profile_phase("projection");
}
for variable in &mut variables {
for address in &mut variable.addresses {
@@ -881,6 +900,7 @@
.as_ref()
.map(|ctx| ctx.scan(&fd, byte_address))
.unwrap_or_default();
+ profile_phase("metadata_tail");
(code, variables, types, (globals, field_calls), proto, line_mappings, callee_hints, detail, pointerargs, stack_objects)
}));
match rendered {
--- a/docs/spec/00-overview.md
+++ b/docs/spec/00-overview.md
@@ -3496,3 +3496,10 @@
runtime reachability. Collection uses the same live function after C rendering;
no second decompilation is performed. Complete source Evidence keeps the native
frontier while callers may select a compact summary and individual facts.
+
+An experiment-only timing overlay recognizes `KUNA_EXPERIMENT_ATOMIC_TIMING=1`
+only with atomic capture enabled. It records sequential per-function wall times
+to stderr for the decompile drive, pre-render metadata, C rendering, metadata
+extraction, membership, calls, full graph, projection and metadata tail. It does
+not change analysis parameters or JSON facts. Loading, setup and final JSON
+encoding are outside these phase intervals; phase CPU costs are not measured.
@@ -0,0 +1,28 @@
--- a/decompiler/crates/kuna-decomp/src/substrate/atomic_facts.rs
+++ b/decompiler/crates/kuna-decomp/src/substrate/atomic_facts.rs
@@ -64,10 +64,10 @@
pub fn collect(arch: &Architecture, fd: &Funcdata) -> Result<FunctionFacts, String> {
// Bounds cover allocations before JSON serialization, independently of the
// transport byte budget. Overflow is an unavailable facet, not clipped IR.
- const MAX_OPERATIONS: usize = 12_000;
- const MAX_VALUES: usize = 36_000;
- const MAX_INPUTS: usize = 50_000;
- const MAX_DISPLAY_BYTES: usize = 2 * 1024 * 1024;
+ const MAX_OPERATIONS: usize = 32_000;
+ const MAX_VALUES: usize = 72_000;
+ const MAX_INPUTS: usize = 128_000;
+ const MAX_DISPLAY_BYTES: usize = 16 * 1024 * 1024;
if fd.bblocks_get_size() <= 0 {
return Err("recovered basic-block graph is unavailable".into());
}
--- a/decompiler/crates/kuna-decomp/src/substrate/atomic_values.rs
+++ b/decompiler/crates/kuna-decomp/src/substrate/atomic_values.rs
@@ -7,7 +7,7 @@
use slotmap::Key;
use std::collections::BTreeSet;
-pub const MAX_TEXT_BYTES: usize = 2 * 1024 * 1024;
+pub const MAX_TEXT_BYTES: usize = 16 * 1024 * 1024;
#[derive(Debug, Clone, Serialize)]
pub struct ValueFact {
pub id: String,
File diff suppressed because it is too large. Load diff
+5
View File
@@ -0,0 +1,5 @@
// Shared application resource policy, independent of native wire formats.
export const NATIVE_OUTPUT_BYTES = 8 * 1024 * 1024;
export const MAX_BRIDGE_BYTES = 2 * 1024 * 1024;
// Aggregate selected bytes keep hex at 128 KiB, with headroom for provenance.
export const MAX_BYTE_BATCH_BYTES = 64 * 1024;
@@ -0,0 +1,70 @@
"""Lossless encoding feasibility on retained native JSON; no provider implementation."""
import argparse
import base64
import gzip
import hashlib
import json
from pathlib import Path
import time
def measure(source, output):
with source.open('rb') as stream:
raw = stream.read(32 * 1024 * 1024 + 1)
if len(raw) > 32 * 1024 * 1024:
raise ValueError('Input exceeds the diagnostic 32 MiB source bound')
document = json.loads(raw)
if len(document['functions']) != 1:
raise ValueError('Select one retained native function')
function = document['functions'][0]
graph = function['atomic_facts']
graph_json = json.dumps(graph, separators=(',', ':'), ensure_ascii=False).encode()
wall, cpu = time.perf_counter(), time.process_time()
compressed = gzip.compress(graph_json, compresslevel=1, mtime=0)
base64_data = base64.b64encode(compressed)
encode_cost = {'wall_ms': (time.perf_counter() - wall) * 1000,
'cpu_ms': (time.process_time() - cpu) * 1000}
wall, cpu = time.perf_counter(), time.process_time()
restored = gzip.decompress(base64.b64decode(base64_data, validate=True))
if restored != graph_json:
raise ValueError('Lossless byte roundtrip failed')
decode_cost = {'wall_ms': (time.perf_counter() - wall) * 1000,
'cpu_ms': (time.process_time() - cpu) * 1000}
results = {}
for name, data in [('gzip-base64', base64_data.decode()),
('gzip-hex', compressed.hex())]:
function['atomic_facts'] = {
'schema_version': 4, 'encoding': name,
'raw_bytes': len(graph_json), 'data': data,
}
results[name] = len(json.dumps(document, separators=(',', ':'),
ensure_ascii=False).encode())
report = {
'source': str(source), 'source_sha256': hashlib.sha256(raw).hexdigest(),
'original_document_bytes': len(raw), 'graph_json_bytes': len(graph_json),
'graph_statement_bytes': sum(len(op['statement'].encode()) for op in graph['operations']),
'graph_operations': len(graph['operations']),
'graph_values': len(graph['values']), 'graph_blocks': len(graph['blocks']),
'gzip_level': 1, 'gzip_bytes': len(compressed),
'candidate_document_bytes': results,
'public_transport_budget_bytes': 8 * 1024 * 1024,
'lossless_graph_byte_roundtrip': True,
'python_gzip_and_base64_encoding': encode_cost,
'python_base64_and_gzip_decoding': decode_cost,
'scope': 'Offline retained-source format feasibility, not a native/provider/model '
'implementation. Timings are Python only, not Rust/Node or agent savings. '
'Whole-run CPU/peak family memory are separate resource-run receipts.',
}
with output.open('x') as stream:
json.dump(report, stream, indent=2)
stream.write('\n')
print(json.dumps(report))
if __name__ == '__main__':
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('source', type=Path)
parser.add_argument('output', type=Path)
args = parser.parse_args()
measure(args.source, args.output)
@@ -0,0 +1,186 @@
"""Size diagnostic SQLite SSA projections; does not implement a provider API."""
import argparse
from collections import deque
import hashlib
import json
from pathlib import Path
COLUMNS = {
'blocks': ['id', 'start', 'operations', 'predecessors', 'successors'],
'operations': ['id', 'address', 'block', 'opcode', 'statement', 'is_dead', 'inputs', 'output'],
'inputs': ['input_index', 'value_id', 'phi_predecessor', 'phi_predecessor_edge'],
'successors': ['id', 'target', 'index', 'polarity', 'condition_value', 'condition_test'],
'values': ['id', 'space', 'offset', 'size_bytes', 'is_constant', 'is_input',
'is_annotation', 'definition', 'type_name'],
}
def encoded(value):
return json.dumps(value, ensure_ascii=True, separators=(',', ':')).encode()
def bounded_source(path, budget):
with path.open('rb') as source:
raw = source.read(budget + 1)
assert len(raw) <= budget, 'Use a bounded native capture'
return raw
def verify_capture(golden, path):
raw = bounded_source(path, 8 * 1024 * 1024)
document = json.loads(raw)
assert len(document['functions']) == 1
candidate = document['functions'][0]
ordinary = lambda fn: {key: row for key, row in fn.items()
if not key.startswith('atomic_')}
assert ordinary(candidate) == ordinary(golden)
assert candidate['atomic_calls'] == golden['atomic_calls']
projection = candidate['atomic_projection']
assert projection['schema_version'] == 1
assert projection['coverage']['operation_budget'] == 4096
operations = {row['id']: row for row in golden['atomic_facts']['operations']}
values = {row['id']: row for row in golden['atomic_facts']['values']}
assert all(row == operations[row['id']] for row in projection['operations'])
assert all(row == values[row['id']] for row in projection['values'])
roots = [row['id'] for row in golden['atomic_calls']['items']]
assert projection['roots'] == roots
queue, queued, expected = deque(roots), set(roots), []
while queue and len(expected) < 4096:
key = queue.popleft()
if key not in operations:
continue
expected.append(key)
for operand in operations[key]['inputs']:
value = values.get(operand['value_id'])
definition = value['definition'] if value else None
if definition is not None and definition not in queued:
queued.add(definition)
queue.append(definition)
actual = [row['id'] for row in projection['operations']]
assert actual == expected
referenced = {row['output'] for row in projection['operations'] if row['output'] is not None} | {
i['value_id'] for row in projection['operations'] for i in row['inputs']
if i['value_id'] is not None}
assert len(projection['values']) == len(referenced)
assert {row['id'] for row in projection['values']} == referenced
missing = {row['definition'] for row in projection['values']
if row['definition'] is not None} - set(actual)
expected_frontier = {key: 'operation-budget' if key in operations
else 'outside-recovered-membership' for key in missing}
actual_frontier = {row['definition']: row['reason']
for row in projection['coverage']['frontier']}
assert len(actual_frontier) == len(projection['coverage']['frontier'])
assert actual_frontier == expected_frontier
return {'scope': 'Retained-source comparison, not independent machine/runtime equivalence',
'source': str(path), 'source_sha256': hashlib.sha256(raw).hexdigest(),
'ordinary_record_and_call_facet_equal': True,
'all_operation_and_value_records_equal': True,
'breadth_first_budget_and_frontier_match': True,
'operations': len(actual), 'values': len(referenced), 'frontier': len(missing)}
def pack(table, rows):
result = []
for row in rows:
assert set(row) == set(COLUMNS[table]), 'Do not silently omit unknown fields'
result.append([pack(key, row[key]) if key in ('inputs', 'successors') else row[key]
for key in COLUMNS[table]])
return result
def unpack(table, rows):
return [{key: unpack(key, value) if key in ('inputs', 'successors') else value
for key, value in zip(COLUMNS[table], row, strict=True)} for row in rows]
def projection(operations, values, roots, depth, cap):
retained, wave = set(roots), set(roots)
exhausted = False
for _ in range(depth):
following = {values[i['value_id']]['definition'] for key in wave
for i in operations[key]['inputs'] if i['value_id'] is not None
and values[i['value_id']]['definition'] in operations} - retained
if len(retained | following) > cap:
exhausted = True
break
retained |= following
wave = following
ids = {operations[key]['output'] for key in retained
if operations[key]['output'] is not None} | {
i['value_id'] for key in retained for i in operations[key]['inputs']
if i['value_id'] is not None}
frontier = {values[i['value_id']]['definition'] for key in retained
for i in operations[key]['inputs'] if i['value_id'] is not None
and values[i['value_id']]['definition'] is not None} - retained
payload = {'operations': [operations[key] for key in sorted(retained)],
'values': [values[key] for key in sorted(ids)]}
return {'requested_definition_depth': depth, 'operation_budget': cap,
'operation_budget_exhausted': exhausted, 'operations': len(retained),
'values': len(ids), 'encoded_bytes': len(encoded(payload)),
'unexpanded_definition_frontier': len(frontier)}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('native_json', type=Path)
parser.add_argument('report_json', type=Path)
parser.add_argument('--verify-capture', type=Path,
help='Also compare a public projection capture with this full diagnostic')
args = parser.parse_args()
budget = 32 * 1024 * 1024
raw = bounded_source(args.native_json, budget)
document = json.loads(raw)
assert len(document['functions']) == 1
function = document['functions'][0]
assert function['address_hex'] == '0xea6d0', 'This is the seeded SQLite VM experiment'
graph = function['atomic_facts']
assert graph['schema_version'] == 3
assert len(graph['operations']) <= 32000 and len(graph['values']) <= 72000
tables = ['blocks', 'operations', 'values']
assert set(graph) == {'schema_version', 'coverage', *tables}
packed = {'encoding': 'diagnostic-tuples-proposal-not-public',
'schema_version': graph['schema_version'], 'columns': COLUMNS,
'coverage': graph['coverage'],
**{key: pack(key, graph[key]) for key in tables}}
restored = {'schema_version': packed['schema_version'], 'coverage': packed['coverage'],
**{key: unpack(key, packed[key]) for key in tables}}
assert restored == graph
operations = {row['id']: row for row in graph['operations']}
values = {row['id']: row for row in graph['values']}
assert len(operations) == len(graph['operations'])
assert len(values) == len(graph['values'])
calls = function['atomic_calls']['items']
selected = [row['id'] for row in calls
if row['origin_address'] in ('0xf115c', '0xf238b')]
assert len(selected) == 2
report = {
'scope': 'Offline diagnostic sizes, not public-provider or model results. '
'Tuple roundtrip is lossless, but no native codec is implemented. '
'Projections follow retained SSA operands only; no CFG, aliases, '
'source equivalence or runtime proof. Bytes are not token/time savings.',
'source': str(args.native_json), 'source_sha256': hashlib.sha256(raw).hexdigest(),
'whole_native_document_bytes': len(raw), 'full_graph_bytes': len(encoded(graph)),
'lossless_tuple_graph_bytes': len(encoded(packed)), 'tuple_roundtrip_equal': True,
'estimated_whole_tuple_document_bytes': len(raw) - len(encoded(graph)) + len(encoded(packed)),
'operations': len(operations), 'values': len(values), 'blocks': len(graph['blocks']),
'inputs': sum(len(row['inputs']) for row in operations.values()),
'cfg_edges': sum(len(row['successors']) for row in graph['blocks']),
'statement_utf8_bytes': sum(len(row['statement'].encode()) for row in operations.values()),
'value_type_utf8_bytes': sum(len(row['type_name'].encode()) for row in values.values()),
'selected_call_projections': [projection(operations, values, selected, depth, 512)
for depth in range(1, 6)],
'all_call_projections': [projection(operations, values, [r['id'] for r in calls], depth, 4096)
for depth in range(1, 5)],
}
if args.verify_capture:
report['verified_projection'] = verify_capture(function, args.verify_capture)
with args.report_json.open('x') as output:
json.dump(report, output, indent=2)
output.write('\n')
print(json.dumps(report, indent=2))
if __name__ == '__main__':
main()
@@ -0,0 +1,57 @@
import { z } from "zod";
import { nativeCall } from "./native-calls.mjs";
import {
ordinal,
value,
rowParser,
readNativeFacet,
requireFact,
} from "./native-contract.mjs";
const callFacet = z.strictObject({
schema_version: z.literal(1),
items: z.array(nativeCall).max(12000),
values: z.array(value).max(36000),
coverage: z.strictObject({
scope: z.string(),
blocks_examined: ordinal.min(1),
operations_examined: z
.number()
.int()
.nonnegative()
.max(Number.MAX_SAFE_INTEGER),
value_scope: z.string(),
machine_call_equivalence: z.literal("unknown"),
}),
});
const parseCall = rowParser(nativeCall, {
arguments: (row) => nativeCall.shape.arguments.element.parse(row),
});
const parseFacet = rowParser(callFacet, {
items: parseCall,
values: (row) => value.parse(row),
});
export const readCallFacet = (fn) =>
readNativeFacet(fn, "atomic_calls", "recovered calls", (raw) => {
requireFact(
raw && typeof raw === "object" && raw.schema_version === 1,
"Unsupported native call facet schema; expected 1",
);
for (const [name, limit] of [
["items", 12000],
["values", 36000],
])
requireFact(
Array.isArray(raw[name]) && raw[name].length <= limit,
`Native call ${name} exceeds capacity or has invalid shape`,
);
requireFact(
raw.items.reduce(
(sum, row) =>
sum + (Array.isArray(row?.arguments) ? row.arguments.length : 0),
0,
) <= 50000,
"Native call inputs exceed capacity",
);
return parseFacet(raw);
});
@@ -0,0 +1,162 @@
import { callCoverage } from "./native-calls.mjs";
import { requireFact, requireSameNativeValue } from "./native-contract.mjs";
import { uniqueNative, readNativeIdentities } from "./native-reads.mjs";
import { freezeFacts } from "./fact-reads.mjs";
export function indexCallFacet(facts) {
const calls = uniqueNative(facts.items),
values = uniqueNative(facts.values),
used = new Set();
requireFact(
calls.size <= facts.coverage.operations_examined,
"Native call count exceeds examined operation coverage",
);
const reference = (id) => {
requireFact(
id === null || values.has(id),
"Dangling native call-boundary value",
);
if (id !== null) used.add(id);
};
for (const row of calls.values()) {
requireFact(
row.id.split(":")[0] === row.origin_address,
"Native call origin/identity mismatch",
);
reference(row.target_value_id);
reference(row.output);
requireFact(
row.output === null || values.get(row.output).definition === row.id,
"Native call output/definition mismatch",
);
for (const [i, a] of row.arguments.entries()) {
reference(a.value_id);
requireFact(
a.input_index === i + 1 &&
(a.storage === null) === (a.storage_authority === null),
"Native call input/storage binding mismatch",
);
}
requireFact(
row.spec_available
? row.prototype !== null
: row.target === null &&
row.target_name === null &&
row.prototype === null &&
row.stack_offset === null &&
row.arguments.every((a) => a.storage === null),
"Native missing call specification has invented facts",
);
}
requireFact(
used.size === values.size,
"Native call boundary contains unrelated values",
);
for (const value of values.values())
if (value.definition !== null && calls.has(value.definition))
requireFact(
calls.get(value.definition).output === value.id,
"Native boundary value/known-call definition mismatch",
);
return { calls, values };
}
export function validateCallGraph(callIndex, graphIndex) {
const { calls, values } = callIndex,
{ operations } = graphIndex;
for (const row of calls.values()) {
const op = operations.get(row.id);
requireFact(
op &&
op.address === row.origin_address &&
op.block === row.block &&
op.opcode === row.opcode &&
op.is_dead === row.is_dead &&
op.output === row.output,
"Native call/graph ownership mismatch",
);
requireFact(
row.target_value_id === (op.inputs[0]?.value_id ?? null) &&
row.arguments.length === Math.max(0, op.inputs.length - 1) &&
row.arguments.every((a, i) => a.value_id === op.inputs[i + 1].value_id),
"Native call/graph operand mismatch",
);
}
for (const value of values.values())
requireSameNativeValue(
value,
graphIndex.values.get(value.id),
"Native call/graph value mismatch",
);
for (const op of operations.values())
requireFact(
!["CPUI_CALL", "CPUI_CALLIND"].includes(op.opcode) || calls.has(op.id),
"Native call operation omitted from call coverage",
);
}
const summarizeCalls = (calls, coverage) => {
const counts = {
CPUI_CALL: { known_target_count: 0, unknown_target_count: 0 },
CPUI_CALLIND: { known_target_count: 0, unknown_target_count: 0 },
};
for (const row of calls.values())
counts[row.opcode][
row.target === null ? "unknown_target_count" : "known_target_count"
]++;
return freezeFacts({
count: calls.size,
by_opcode: Object.entries(counts).map(([opcode, row]) => ({
opcode,
...row,
})),
coverage,
});
};
export function createCallViews(facts, indexes) {
const { calls, values } = indexes;
freezeFacts(facts);
const coverage = freezeFacts({
...callCoverage,
...facts.coverage,
definition_operations: "not collected in call-boundary facet",
});
const valueCoverage = freezeFacts({
...coverage,
scope: facts.coverage.value_scope,
});
return {
calls: (input) => ({
...readNativeIdentities(
calls,
{
fields: [
"spec_available",
"opcode",
"is_dead",
"target",
"target_name",
"arguments",
"output",
],
...input,
...(input.fields && {
fields: ["spec_available", "opcode", "is_dead", ...input.fields],
}),
},
(r) =>
!input.origin_addresses ||
input.origin_addresses.includes(r.origin_address),
),
coverage,
}),
values: (input) => ({
...readNativeIdentities(
values,
input,
(r) => input.is_input === undefined || r.is_input === input.is_input,
),
coverage: valueCoverage,
}),
summary: summarizeCalls(calls, coverage),
value_count: values.size,
};
}
@@ -0,0 +1,51 @@
// The native owner supplies targets and passing locations; never parse display.
import { z } from "zod";
import { hex, ordinal, valueId, operationId } from "./native-contract.mjs";
const location = z.strictObject({ space: z.string().min(1), offset: hex });
const storage = location.extend({
size_bytes: z.number().int().positive().max(0x7fffffff),
});
export const nativeCall = z.strictObject({
id: operationId,
origin_address: hex,
block: ordinal,
opcode: z.enum(["CPUI_CALL", "CPUI_CALLIND"]),
is_dead: z.boolean(),
target_value_id: valueId.nullable(),
spec_available: z.boolean(),
target: location.nullable(),
target_name: z.string().min(1).nullable(),
arguments: z.array(
z.strictObject({
input_index: z.number().int().positive().max(0x7fffffff),
value_id: valueId.nullable(),
storage: storage.nullable(),
storage_authority: z
.enum(["final-input-storage", "locked-prototype"])
.nullable(),
}),
),
output: valueId.nullable(),
prototype: z
.strictObject({
model: z.string().nullable(),
model_unknown: z.boolean().nullable(),
inputs_locked: z.boolean(),
variadic: z.boolean(),
parameter_shift: z.number().int().min(-0x80000000).max(0x7fffffff),
})
.nullable(),
stack_offset: hex.nullable(),
});
export const callCoverage = Object.freeze({
scope: "all retained CALL/CALLIND operations in this recovered function",
target_authority: "native recovered call specification; not runtime binding",
argument_authority:
"native finalized passing storage or locked prototype; null remains unknown",
source_parameter_ordinals: "unknown",
source_signature: "unknown",
machine_call_equivalence: "unknown",
memory_alias_dependencies: "unknown",
});
@@ -0,0 +1,351 @@
import test from "node:test";
import assert from "node:assert/strict";
import { parseNativeIR } from "./native-ir.mjs";
test("malformed compressed graph leaves independent call facts and code readable", () => {
const doc = fixture();
const originalCode = doc.functions[0].code;
doc.functions[0].atomic_facts = {
schema_version: 4,
encoding: "gzip-base64",
raw_bytes: 64,
data: "AAAA",
};
const model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.code, originalCode);
assert.equal(model.facets.recovered_ir.availability, "unavailable");
assert.equal(model.facets.calls.availability, "available");
assert.equal(model.calls({}).total, 2);
});
const input = (input_index, value_id) => ({
input_index,
value_id,
phi_predecessor: null,
phi_predecessor_edge: null,
});
const value = (id, space, offset, flags = {}) => ({
id,
space,
offset,
size_bytes: 8,
is_constant: false,
is_annotation: false,
is_input: false,
definition: null,
type_name: "",
...flags,
});
const callOperations = () => [
{
id: "0x1100:0",
address: "0x1100",
block: 0,
opcode: "CPUI_CALL",
statement: "deliberately misleading target 0xbad",
is_dead: false,
inputs: [input(0, "v0"), input(1, "v1"), input(2, "v2")],
output: null,
},
{
id: "0x1110:0",
address: "0x1110",
block: 0,
opcode: "CPUI_CALLIND",
statement: "not a signature",
is_dead: false,
inputs: [input(0, "v2")],
output: null,
},
];
const fixtureValues = () => [
value("v0", "fspec", "0x10000", { is_annotation: true }),
value("v1", "const", "0x2a", { is_constant: true }),
value("v2", "register", "0x30", { is_input: true }),
];
function fixture() {
const operations = callOperations(),
values = fixtureValues();
const rows = [
{
id: "0x1100:0",
spec_available: true,
target: { space: "ram", offset: "0x2000" },
target_name: "callee",
arguments: [
{
input_index: 1,
value_id: "v1",
storage: { space: "register", offset: "0x38", size_bytes: 8 },
storage_authority: "final-input-storage",
},
{
input_index: 2,
value_id: "v2",
storage: null,
storage_authority: null,
},
],
output: null,
prototype: {
model: "__cdecl",
model_unknown: false,
inputs_locked: false,
variadic: false,
parameter_shift: 0,
},
stack_offset: null,
},
{
id: "0x1110:0",
spec_available: false,
target: null,
target_name: null,
arguments: [],
output: null,
prototype: null,
stack_offset: null,
},
];
return {
binary: "/input",
count: 1,
language: "c",
error: null,
functions: [
{
address_hex: "0x1000",
code: "retained code",
error: null,
atomic_facts: {
schema_version: 3,
blocks: [
{
id: 0,
start: "0x1000",
operations: operations.map((o) => o.id),
predecessors: [],
successors: [],
},
],
operations,
values,
coverage: {
scope: "native graph",
machine_semantic_equivalence: "unknown",
indirect_branch_operations: [],
omitted_definition_ids: [],
issues: [],
},
},
atomic_calls: {
schema_version: 1,
items: rows.map((row, i) => ({
...row,
origin_address: operations[i].address,
block: 0,
opcode: operations[i].opcode,
is_dead: false,
target_value_id: operations[i].inputs[0].value_id,
})),
values: structuredClone(values),
coverage: {
scope: "all recovered block-member CALL/CALLIND operations",
blocks_examined: 1,
operations_examined: 2,
value_scope:
"call inputs including target and outputs only; definitions may be outside this facet",
machine_call_equivalence: "unknown",
},
},
},
],
};
}
test("structured calls separate fspec identity, SSA value and passing location; missing targets stay unknown", () => {
const model = parseNativeIR(JSON.stringify(fixture()), "0x1000");
const result = model.calls({ limit: 1 });
assert.equal(result.total, 2);
assert.equal(result.next_offset, 1);
const row = result.items[0];
assert.deepEqual(row.target, { space: "ram", offset: "0x2000" });
assert.equal(row.arguments[0].value_id, "v1");
assert.equal(row.arguments[0].storage.offset, "0x38");
assert.equal(row.arguments[1].storage, null);
assert.equal(model.values({ ids: ["v1"] }).items[0].offset, "0x2a");
assert.equal(Object.hasOwn(row, "prototype"), false);
assert.throws(() => {
row.arguments[0].storage.offset = "0xbad";
}, TypeError);
const unknown = model.calls({ offset: 1 }).items[0];
assert.equal(unknown.target, null);
assert.equal(unknown.spec_available, false);
assert.equal(result.coverage.machine_call_equivalence, "unknown");
assert.equal(result.coverage.source_parameter_ordinals, "unknown");
});
test("selected call pages and function input values compose without full reports", () => {
const model = parseNativeIR(JSON.stringify(fixture()), "0x1000");
const result = model.calls({
ids: ["0x1100:0", "missing"],
origin_addresses: ["0x1100"],
fields: ["prototype"],
});
assert.deepEqual(result.missing, ["missing"]);
assert.deepEqual(
Object.keys(result.items[0]).sort(),
[
"id",
"origin_address",
"block",
"spec_available",
"opcode",
"is_dead",
"prototype",
].sort(),
);
assert.equal(
model.calls({ ids: ["0x1100:0"], origin_addresses: [] }).total,
0,
);
assert.deepEqual(
model.values({ is_input: true }).items.map((v) => v.id),
["v2"],
);
assert.equal(model.values({ is_input: true, ids: ["v1"] }).total, 0);
});
test("bad call facets preserve valid graph and code; cross-facet conflicts disable call composition", () => {
const changes = [
(f) => {
f.items.pop();
},
(f) => {
f.items.push(f.items[0]);
},
(f) => {
f.items[0].arguments[0].value_id = "v2";
},
(f) => {
f.items[0].arguments[0].input_index = 0;
},
(f) => {
f.items[0].arguments[0].storage_authority = null;
},
(f) => {
f.items[1].target = { space: "ram", offset: "0x2000" };
},
(f) => {
f.values[1].offset = "0x2b";
},
(f) => {
f.schema_version = 2;
},
];
for (const change of changes) {
const doc = fixture();
change(doc.functions[0].atomic_calls);
const model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.code, "retained code");
assert.equal(model.facets.recovered_ir.availability, "available");
assert.equal(model.facets.calls.availability, "unavailable");
assert.throws(
() => model.calls({}),
(e) => e.code === "invalid_provider_output",
);
assert.equal(model.values({ ids: ["v1"] }).items[0].offset, "0x2a");
}
});
test("graph capacity failure retains calls, boundary values and explicit unreadable definitions", () => {
const doc = fixture(),
fn = doc.functions[0];
delete fn.atomic_facts;
fn.atomic_facts_error = "atomic operation capacity exceeded";
const model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.facets.recovered_ir.availability, "unavailable");
assert.equal(model.facets.calls.availability, "available");
assert.equal(model.facets.values.scope, "call-boundary-only");
assert.equal(model.calls({}).total, 2);
const values = model.values({ ids: ["v0", "v1", "missing"] });
assert.equal(values.items[0].space, "fspec");
assert.equal(values.items[1].offset, "0x2a");
assert.deepEqual(values.missing, ["missing"]);
assert.match(values.coverage.value_scope, /only/);
assert.equal(values.coverage.scope, values.coverage.value_scope);
assert.throws(
() => model.select({}),
(e) => e.code === "capability_unavailable",
);
assert.equal(
model.calls({ ids: ["0x1110:0"], fields: ["target_value_id"] }).items[0]
.target_value_id,
"v2",
);
});
test("malformed graph is isolated from valid calls; call-only malformed identities still fail", () => {
const doc = fixture();
doc.functions[0].atomic_facts.operations[0].output = "vbad";
let model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.facets.recovered_ir.availability, "unavailable");
assert.equal(model.facets.calls.availability, "available");
assert.equal(model.values({ is_input: true }).items[0].id, "v2");
doc.functions[0].atomic_calls.items[0].target_value_id = "vdead";
model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.facets.calls.availability, "unavailable");
assert.throws(
() => model.values({}),
(e) => e.code === "invalid_provider_output",
);
assert.equal(model.code, "retained code");
});
test("capture call counts distinguish target knowledge from opcode and include dead rows", () => {
const doc = fixture(),
fn = doc.functions[0];
delete fn.atomic_facts;
fn.atomic_facts_error = "capacity";
const [direct, indirect] = fn.atomic_calls.items;
direct.target = null;
direct.is_dead = true;
indirect.target = { space: "ram", offset: "0x3000" };
indirect.spec_available = true;
indirect.prototype = direct.prototype;
const model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.deepEqual(model.facets.calls.by_opcode, [
{ opcode: "CPUI_CALL", known_target_count: 0, unknown_target_count: 1 },
{ opcode: "CPUI_CALLIND", known_target_count: 1, unknown_target_count: 0 },
]);
assert.equal(model.facets.calls.count, 2);
assert.equal(model.facets.calls.coverage.machine_call_equivalence, "unknown");
assert.equal(model.calls({ ids: [direct.id], fields: [] }).total, 1);
assert.equal(model.facets.calls.count, 2);
assert.throws(() => {
model.facets.calls.by_opcode[0].known_target_count = 99;
}, TypeError);
});
test("an empty available call facet has zero counts; unavailable counts remain absent", () => {
const doc = fixture(),
fn = doc.functions[0];
delete fn.atomic_facts;
fn.atomic_facts_error = "capacity";
fn.atomic_calls.items = [];
fn.atomic_calls.values = [];
let model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.facets.calls.availability, "available");
assert.equal(model.facets.calls.count, 0);
assert.ok(
model.facets.calls.by_opcode.every(
(r) => r.known_target_count === 0 && r.unknown_target_count === 0,
),
);
delete fn.atomic_calls;
fn.atomic_calls_error = "capacity";
model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.facets.calls.availability, "unavailable");
assert.equal(Object.hasOwn(model.facets.calls, "count"), false);
assert.equal(Object.hasOwn(model.facets.calls, "by_opcode"), false);
});
@@ -0,0 +1,122 @@
# Native computation and retained fact coverage
This diagnostic separates native computation from fact selection and Code Mode
dispatch after the wide-operand model pairs. It does not change those adapters
or establish model-token savings.
## Phase instrument and actual measurements
Use experiment commit `7d0da075e` and its versions of both patches to reproduce
these historical measurements. Apply `kuna-native-ir.patch`, then
`kuna-atomic-timing-diagnostic.patch`, to Kuna
`fe01d6294eb0d1053ce343c8943646df88e143fc`. Reuse existing compiled specs and the
one-job opt-level-1 embed-specs build in `native-engine.md`. Preserved engine:
`c755184455b8fe8950a5194788d954bc0476602779b6dd614cb893a230f6fde7`;
overlay: `b2ddfd5b6d3fc4a386cfbb07d97fbd0fcbfd8279837ce68327e62baa3e0f3018`.
The preceding `a0685486...` public executable remains independently preserved.
The instrument requires `--atomic-facts` and the exact environment value
`KUNA_EXPERIMENT_ATOMIC_TIMING=1`. It emits nine sequential per-function wall
intervals to stderr, excluding logging time from the next interval. Disabled
instrumentation emits none. Analysis parameters and JSON facts are unchanged.
These are phase wall times, not phase CPU. Loading, preparation before the drive
and final output encoding are outside the intervals; the residual cannot all be
attributed to one phase.
`profile-native.mjs` uses production target/installation/process owners and a
private runtime/cache root. Its diagnostic launcher adds only the timing flag;
ambient environment is not persisted. It measures two new native processes:
fresh private cache, then the same cache/staged input. This second process is
distinct from the API owner's warm result-cache hit. Complete stdout/stderr and
digests are retained before validation. OS cache remains uncontrolled.
```sh
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
node experiments/atomic-codemode/profile-native.mjs \
/absolute/diagnostic-kuna /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 \
0xea6d0 /absolute/new-profile.json /absolute/prior-public-capture.json
```
Add `--without-timing` for the disabled control. Target: SQLite Debian
`3.40.1-2+deb12u2` amd64, SHA-256
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
VM entry is `0xea6d0`; parser-helper entry is `0x64060`.
| Wall milliseconds | VM fresh | VM same cache/new process | Helper fresh | Helper same cache/new process |
| --------------------- | ---------: | ------------------------: | -----------: | ----------------------------: |
| Entire owned request | 36,336.306 | 36,353.240 | 2,339.393 | 2,193.311 |
| Decompile drive | 33,681.369 | 33,855.870 | 256.200 | 266.629 |
| C rendering | 220.052 | 215.235 | 9.399 | 7.416 |
| Metadata extraction | 64.434 | 128.247 | 1.576 | 1.564 |
| Membership scan | 3.259 | 3.496 | 0.137 | 0.093 |
| Call facts | 2.889 | 3.328 | 0.245 | 0.133 |
| Full graph collection | 0.006* | 0.007* | 6.993 | 5.473 |
| Projection collection | 31.383 | 31.711 | 1.955 | 1.770 |
Pre-render/tail metadata also appear in the nine-interval raw receipts. `*` VM
full-graph collection fails its early operation-capacity check; these numbers
do not time a complete graph export. Whole-function analysis nevertheless
produces the live IR. Each function record deeply equals its prior public golden
and its repeat. The disabled helper control also agrees, with zero timing rows.
| Whole-run resource scope | Family wall seconds | CPU seconds | Kernel peak MiB |
| ------------------------------------- | ------------------: | ----------: | --------------: |
| Instrument build | 115.279 | 92.082 | 760.7 |
| VM, two processes plus validation | 74.218 | 73.867 | 572.2 |
| Helper, two processes plus validation | 5.554 | 5.328 | 191.9 |
| Disabled helper, two processes | 5.556 | 5.409 | 191.8 |
| Final cached build | 0.322 | 0.155 | 26.8 |
All use one CPU, 4 GiB hard / 3 GiB early-stop bounds, verified cleanup and OOM 0.
Kernel peak includes charged file cache, not RSS. Final build keeps the same
digest. Patch forward/reverse and lenient `check-spec` pass; full native corpus,
stages and workspace gates remain unexecuted. No upstream commit/PR or integration
readiness is claimed. Receipts/raw sources are local
`phase-timing-{vm,helper,helper-off}.json*` and resource logs under
`.cache/atomic-codemode/`.
C rendering is below 1% of this VM request; skipping it cannot remove the
approximately 34-second drive. The helper instead spends most time outside the
instrumented phases, requiring further profiling before choosing persistent
state. Same-cache new processes reanalyze. Previously measured same-lease API
source reuse remains a separate approximately 40 ms path.
## Lossless full-graph encoding feasibility
The earlier diagnostic source has 21,349 operations, 35,755 values and 1,229
blocks, with no omitted definition IDs. Its whole document is 17,801,972 bytes;
the graph is 17,008,223 bytes, including 2,255,616 bytes of operation display and
90,013 bytes of value type names. Plain tuple packing was still too large.
```sh
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
python3 experiments/atomic-codemode/measure-native-encoding.py \
/absolute/diagnostic-native.json /absolute/new-encoding-report.json
```
Gzip level 1 makes the graph 2,020,464 bytes. Hypothetical documents retaining
ordinary C/metadata/calls and replacing only the graph with a typed envelope are:
| Encoding | Entire JSON bytes | Existing transport budget |
| ------------- | ----------------: | ------------------------: |
| gzip + base64 | 3,487,776 | 8,388,608 |
| gzip + hex | 4,834,749 | 8,388,608 |
The graph byte roundtrip is exact. Python gzip/base64 encode costs 69.037 ms
wall / 69.037 ms CPU; decode 34.632 / 34.571 ms. Whole sizing task: 0.826 wall /
0.654 CPU seconds, 169.8 MiB peak, cleanup verified and OOM 0. These are offline
Python feasibility measurements, not Rust/Node timings, native output, public
support or agent savings. Source SHA-256:
`0a31513350be440983094710197b155a2ec8c49d28852bb20a524292091637b7`;
reproducible receipt: `full-graph-encoding-repro.json` beside its resource log.
This supports testing bounded lossless transport before introducing live-function
ownership or repeating analysis for missing definitions. Required checks include
allocation/encoding growth, bounded inflation, exact format/length/row admission,
malformed-graph isolation from calls/C, complete Evidence access, actual CPU and
memory, fallback behavior and real-model workflows. The shared owner must return
selected facts rather than send an encoded graph to the agent. The current
public engine at this sizing checkpoint still has a bounded call-rooted
projection. The [subsequent implementation](native-graph-transport-study.md)
now verifies full recovered-graph access through bounded native/Node encoding
and both public MCP adapters; real-model verification remains separate.
@@ -0,0 +1,105 @@
// Shared native identity/value contracts and bounded row/facet decoding.
import { z } from "zod";
import { FactFailure } from "./fact-error.mjs";
import { ok, err } from "../../dist/domain/result.js";
export const requireFact = (condition, reason) => {
if (!condition) throw new FactFailure("invalid_provider_output", reason);
};
export const hex = z.string().regex(/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/u);
export const edgeId = z.string().regex(/^\d+:\d+$/u);
export const ordinal = z.number().int().nonnegative().max(0x7fffffff);
export const valueId = z.string().regex(/^v[0-9a-f]{1,16}$/u);
export const operationId = z
.string()
.regex(/^0x[0-9a-f]{1,16}:[0-9a-f]{1,8}$/u);
export const value = z.strictObject({
id: valueId,
space: z.string().nullable(),
offset: hex,
size_bytes: z.number().int().positive().max(0x7fffffff),
is_constant: z.boolean(),
is_input: z.boolean(),
is_annotation: z.boolean(),
definition: operationId.nullable(),
type_name: z.string(),
});
export const requireSameNativeValue = (value, reference, message) =>
requireFact(
reference &&
Object.entries(value).every(([key, v]) => reference[key] === v),
message,
);
export const rowParser = (schema, arrays, fields = {}) => {
const metadata = schema.omit(
Object.fromEntries(
[...Object.keys(arrays), ...Object.keys(fields)].map((key) => [
key,
true,
]),
),
);
return (raw) => {
const metadataFields = { ...raw };
for (const key of [...Object.keys(arrays), ...Object.keys(fields)])
delete metadataFields[key];
const result = metadata.parse(metadataFields);
for (const [key, parse] of Object.entries(arrays)) {
requireFact(Array.isArray(raw[key]), `Native ${key} must be an array`);
result[key] = raw[key].map(parse);
}
for (const [key, parse] of Object.entries(fields))
result[key] = parse(raw[key]);
return result;
};
};
export const formatFailure = (error) => {
if (error instanceof FactFailure) return error;
const details =
error instanceof z.ZodError
? {
issues: error.issues.slice(0, 8).map(({ path, code, message }) => ({
path,
code,
message: message.slice(0, 160),
})),
}
: { reason: String(error.message).slice(0, 256) };
return new FactFailure(
"invalid_provider_output",
"Native IR source does not match the declared format",
details,
);
};
export const readNativeFacet = (fn, name, label, parse) => {
try {
const reason = fn[`${name}_error`];
if (reason !== undefined) {
requireFact(
fn[name] === undefined,
`Native ${label} is both present and unavailable`,
);
return err(
new FactFailure(
"capability_unavailable",
`Native ${label} facet is unavailable`,
{
native_error: reason.slice(0, 1024),
native_error_truncated: reason.length > 1024,
},
),
);
}
if (fn[name] === undefined)
return err(
new FactFailure(
"capability_unavailable",
`Native ${label} facet was not captured`,
{ availability: "not-captured" },
),
);
return ok(parse(fn[name]));
} catch (error) {
return err(formatFailure(error));
}
};
@@ -0,0 +1,682 @@
# Native recovered IR experiment
Stock Kuna's diagnostic text omits CFG successors and does not encode operand or
phi edge identities. `kuna-native-ir.patch` is a read-only exploratory exporter
against [Kuna v1.833](https://github.com/Noelo-Lab/kuna/tree/fe01d6294eb0d1053ce343c8943646df88e143fc).
It is not an upstream release or an integrated REA capability. No upstream commit,
PR or merge is being made.
`decompile --json --atomic-facts` captures the live function that produced C.
The default path does not collect the facet. Graph records include ordered CFG
edges, native value identities, indexed operands, outputs and phi input bindings
to particular incoming edges. Duplicate instruction origins or predecessor blocks
do not collapse identities. Values are local to one snapshot. Constant offsets
remain hexadecimal u64 address-space offsets, with width and annotation status
separate; they are not arbitrary-width computed bitvectors.
`native-ir.mjs` owns validation, immutable indexes and selected reads.
`control-dependence.mjs` derives control dependence using immediate
postdominators of the reversed graph, with linear retained graph state and a
bounded work count. It treats zero-successor recovered blocks as exits and
considers finite paths only. Exit-unreachable blocks, indirect-branch operations,
missing definitions, runtime successors and memory alias limitations stay
explicit. This is not a proof of feasible runtime paths.
The shared API adds `values`, `blocks` and `relations(kind:"controllers")`.
Diagnostic SSA sessions report these missing capabilities explicitly. Native
definition/use edges include operand and phi bindings; diagnostic printed
occurrences retain their original, weaker semantics. CLI, SDK and both MCP
adapters use the same owner. Ordinary source summaries are compact; `evidence`
retains the exact command argv and separate display argv, complete output and
digest. Earlier trials substituted display paths in `arguments`; they must not
be retroactively claimed to have captured exact argv.
To reproduce, check out the pinned upstream revision in a disposable directory
and apply this patch with `git apply`. Reuse already compiled `.sla` files from
the corresponding stock release under the same relative `specs/` paths; do not
recompile specs or replace existing unrelated files. Build from the REA root:
```sh
env CARGO_INCREMENTAL=0 CARGO_PROFILE_RELEASE_DEBUG=0 \
CARGO_PROFILE_RELEASE_OPT_LEVEL=1 CARGO_PROFILE_RELEASE_LTO=false \
CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 \
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
cargo build --manifest-path /absolute/kuna/decompiler/Cargo.toml \
--locked --release -j1 -p kuna-cli --features embed-specs
```
Select that executable through `REA_KUNA_COMMAND` and explicitly pin its actual
SHA-256 through `REA_ATOMIC_NATIVE_IR_ENGINE_SHA256`. Native IR requires embedded
specs and rejects a caller-supplied `--sleighpath`; the provider creates a fresh
owned engine cache per lease. The executable digest therefore covers its embedded
assets, without reusing an unvalidated external spec cache. The current prototype
uses optimization level 1, not the stock release's assumed optimization profile;
comparisons must retain this difference or use the same engine in all lanes.
Native extraction bounds blocks/operations at 12,000, values at 36,000, input and
CFG edges at 50,000 and display text at 2 MiB, before the 8 MiB transport bound.
Capacity overflow makes this facet unavailable, preserving C and a reason. The
provider retains at most two 8 MiB native indexes (two 2 MiB diagnostic indexes)
and 64 MiB of complete source captures per lease. The outer observer still
enforces one CPU, 4 GiB hard memory and early termination at 3 GiB.
Real Linux zlib verification produced 3,860 operations, 5,898 values, 472 blocks,
779 CFG edges and 790 phi operations. Its 5,533,649-byte native JSON has no omitted
definitions or reported producer issues. C is byte-identical to the previously
captured stock 1.833 JSON (52,431 characters), and line mappings are identical.
The CLI composition independently discovers the string, owning function, store
and its two local controllers. Its observed cold session takes 9.191 seconds,
8.887 process-family CPU seconds and 198.9 MiB kernel memory peak, with verified
cleanup and zero OOM kills. This scripted lane is not a real-model cost result.
Targeted JS checks cover malformed/foreign identities, dangling relations, phi
order, exact hex offsets, immutable facts, loop/exit coverage, resource failure,
and all 64 four-node DAGs against an independent path-enumeration oracle.
Upstream `check-spec` passes. The full upstream corpus, stages and workspace gates
have not run; this patch is not declared upstream-ready. The model comparisons
and subsequent boundary findings below do not establish wider provider parity.
The value-origin model comparison exposed a representation boundary: optimized
recovered blocks can absorb a machine predecessor, so an IR edge is not an actual
branch instruction target. The experimental API now names operation locations
`origin_address`, their selector `origin_addresses`, and block locations
`origin_start`; printed CFG targets use `target_origin`. These replace the old
ambiguous fields without aliases. Historical captures retain their pinned old
contracts. Raw upstream JSON is preserved unchanged in Evidence.
The added `instructions` method batches selected machine-address windows, using
Kuna's existing [disassemble CLI](https://github.com/Noelo-Lab/kuna/blob/fe01d6294eb0d1053ce343c8943646df88e143fc/decompiler/crates/kuna-cli/src/disassemble.rs),
with exact bytes, decoded text, extent, coverage and per-window errors. Kuna
already has focused agent-oriented read surfaces; its port ancestry does not
establish which primitives are missing. No assembly decoding or branch-target
parser is duplicated in the MCP adapter.
Windows are capped at 4,096 decoded rows and 16 windows per call, sharing the
8 MiB capture and 64 MiB retained-source budgets. A real maximum-row CLI test
returns 4,096 rows in an 842,106-byte source, taking 1.227 family wall seconds,
1.042 CPU seconds and 112.0 MiB kernel memory peak. Both real stock and patched
MCP tests preserve good windows alongside an unmapped-address failure and report
the actual `JA 0xe26d` / `JMP 0xd9c9` bytes. The representation revision had 33
targeted tests; its subsequent real-model trial is recorded below.
That fresh trial now verifies the missing machine path correctly; see
[actual task costs](trials.md). The SQLite task then exposes a separate missing
primitive: name/alias lookup across distinct callable entries. `functions` now
reuses the production inventory parser, preserving every matching entry, import
stub ambiguity, original aliases, clipped extents and unknown runtime binding.
The inventory is read/validated once per lease, and query/field/page selection
returns only the requested view. Batch xref and instruction reads share one
partial-outcome owner, retaining good observations and source IDs on failures.
A real `sqlite3VdbeExec` capture at `0xea6d0` also exceeds the graph export's
operation capacity while producing 126,519 characters of valid C. The old adapter
incorrectly rejects the entire capture. The revised owner validates the function
envelope and input identity independently, then models graph availability as a
facet result. Code and its language remain available under the same handle;
graph counts stay unknown and graph reads return their reason and Evidence ID.
The real MCP regression reads 3,654 code lines after the graph failure and verifies
that `sqlite3Fts3ErrMsg` has distinct entries `0x268f0` and `0x5efc0`. Native graph
capacity remains a real limitation, not a resolved scalability claim.
`verify-cache.mjs` measures a real cold/warm pair within one open lease. Cold zlib
snapshot reading takes 6,934.4 ms / 6.928 family CPU seconds; warm reading takes
67.7 ms / 0.068 CPU seconds and preserves the exact snapshot/Evidence ID. The
cumulative kernel memory peak is 275.1 MiB for both phases; the warm number is an
upper bound on its isolated phase peak. This is an API-only measurement, with no
model token result or cross-session cache claim. Targeted tests now total 38,
including late cancellation retaining partial facts and their provenance.
A subsequent real boundary sweep found a different loss of usable facts:
`sqlite3Pragma` (`0xb90e0`) and `sqlite3Parser` (`0xb3d00`) stay below the row
limits but overflow the provider's 8 MiB stdout budget. Both public snapshots
fail, losing C as well as IR. A separate diagnostic capture with a stated 32 MiB
observer limit establishes the exact producer sizes; it is not relabeled as a
successful public-provider workflow. The old engine is preserved locally by its
`7d357e75…` digest for historical reproduction.
| Function | Previous pretty JSON bytes | Revised source bytes | C characters | Revised graph |
| --------------- | -------------------------: | -------------------: | -----------: | ----------------------------------------------- |
| `sqlite3Parser` | 8,492,810 | 4,526,811 | 50,508 | All 5,164 operations retained |
| `sqlite3Pragma` | 16,728,407 | 136,501 | 67,300 | Explicitly unavailable at encoded-byte capacity |
The structural fix is at the producing serialization boundary. Atomic output
uses the existing record producer and shared compact ASCII-safe writer.
`--atomic-output-bytes` takes an explicit encoded-document budget (default 8 MiB);
the provider passes the same bound it supervises. The writer checks each encoded
write before growing its bounded buffer, accounting for escaping and newline.
If necessary, the document drops only the graph and records an exact facet
failure. C, line mappings and metadata survive under the same Evidence. If even
the base document cannot fit, the command fails rather than clipping it or
repeating the analysis. Ordinary non-atomic JSON keeps its previous formatting.
Row limits alone never promise that the whole encoded graph fits.
The rebuilt local engine is SHA-256
`2290e8b13caabe30aab6832ab163d92925b2e652eebf7fc0cdbe65b889cb251b`
(same pinned source/release profile plus the revised patch). Real Code Mode MCP
verification succeeds for both functions. The preserved C and line mappings are
identical to the earlier full diagnostic captures; the Parser graph is also
deep-equal, and retained source hashes verify. Pragma's unknown graph counts and
failed graph reads retain the correct source ID. No limit was increased.
The first successful revised MCP check took 13.721 family wall seconds, 13.461
CPU seconds and 326.9 MiB kernel family peak, with verified cleanup and no OOM.
This is a two-function API regression, not a model-cost comparison or a claimed
analysis speedup. Native writer/escaping checks (14) and graph-only fallback (1)
pass in the release test binary; the shared JS tests (38), SDK ownership probe,
lint and patch apply/reverse checks pass. Full upstream corpus/stages/workspace
gates still have not run. The first MCP launch used a stale pre-format-build hash
and was correctly refused; the retained successful retry uses the digest above.
The `bytes` primitive reuses Kuna's existing `read --as data --json` protocol.
Unlike instruction decoding, it asks for an exact byte range. It disables the
global `listing` / `fast_funcdisc` passes for this data-only query; instructions
retain their existing profile. A native probe returns exactly equal JSON with
and without those flags. Its 484 / 290 ms timings share a workspace and run in
that order, so they do not establish an isolated speedup. Native/source ownership,
freshness, decoded-window admission and partial batches remain shared.
Real public MCP checks pass in native Code Mode, native direct and stock 1.833
Code Mode: a known 53-byte string, a 4-of-16-byte read at a mapped-image boundary,
an unmapped address, same-source cache reuse and the 64 KiB maximum. Both adapter
catalogs validate after SDK conversion. The byte parser rejects wrong subjects,
contradictory duplicated hex, invalid extents and rows, and preserves exact u64
addresses. Completeness follows examined byte coverage rather than the native
truncation flag. It never claims these loaded-image bytes are live memory or
proves an original-file mapping. Targeted JS tests now total 41.
## Structured call relations (wire version 2)
The next observed gap was input zero of CALL: a native `fspec` annotation is an
internal reference, not a code address. The read-only `atomic_calls.rs` exporter
now uses the existing `FuncCallSpecs` owner for recovered entry/name, model flags,
argument passing storage and stack offset. A single operation-to-spec index
avoids a linear lookup for every call. Input slots and SSA IDs remain separate
from passing locations. Finalized storage is admitted only at matching arity;
locked prototype storage is the alternative. Shifted inputs, placeholders,
missing specs, unavailable storage and unresolved indirect entries remain
explicit. There is no display-text parser or original-source ordinal claim.
`calls` provides ID/origin filtering, field projection and paging through the
shared SDK/CLI/MCP owner. Callee `values(is_input:true)` exposes retained input
values for exact storage candidate matching. Source schemas, unique call
ownership, call coverage, operand indexes/values and output identities are
validated before exposing an index; a bad or old-version graph preserves valid
C. Machine CALL instructions remain the independent check of a recovered edge.
Engine `64c6b3fe5737056fc29c10b0aad6c7cbd489ad7d17bd1726141b77bdcf5e15b7`
uses the same upstream revision, embedded specs and one-job build profile.
The rebuild took 317.090 family wall seconds, 315.808 CPU seconds and a 2,145.695
MiB kernel process-family memory peak (including charged file cache), OOM 0.
The previous compact exporter engine is preserved by its SHA-256 locally.
Real SQLite MCP verification passes both adapters, including a composed Code
Mode join: caller `0x65211 -> 0x64060`, argument passing storage
`register:0x38/8`, one matching callee input, and a bounded definition slice from
the context updates to that input. Formatter calls `0x5f031`/`0x5f062` point to
`0x2adc0`/`0x27e50`; their fspec offsets remain `0x10000`/`0x10001`.
All three functions' C, ordinary metadata/line mappings, blocks, operations,
values and existing coverage equal retained earlier captures. Those earlier
sources were pretty-printed: encoded-size changes are not a call-exporter
performance result. The structured call counts are 8, 33 and 2 respectively.
The Code Mode verifier took 7.872 family wall / 7.573 CPU seconds, 297.1 MiB;
the direct verifier took 7.771 / 7.499 seconds, 278.7 MiB. These differ in assertion
calls and warm file-cache state, so they are verification receipts, not a
controlled adapter speed comparison. Both verified cleanup and OOM 0. The
existing real zlib controller/phi/instruction verifier also passes with this
engine (10.992 wall / 10.795 CPU seconds, 285.7 MiB, cleanup verified, OOM 0).
`check-spec` passes in its default lenient mode; full upstream gates still have
not run. The independent cross-function question and acceptance criteria are in
[call-oracle.md](call-oracle.md), pinned before model evaluation.
The strengthened verifier additionally reads all five retained indirect calls in
this parser and checks their unknown targets. Machine bytes `ff5328` at
`0x644eb` independently describe a register-indirect CALL. The Code Mode rerun
passes (16.628 wall / 8.239 CPU seconds, 300.5 MiB); native test compilation was
also active on the same CPU, so its wall time is not an isolated performance
sample. The source checks pass 45 JS tests across both prototype directories and
two Python ledger tests; changed source lint and formatting pass.
A current coverage limit remains: call facts are part of the recovered IR
facet. A function that exceeds graph capacity also loses these call records,
even if a selected call-target view could fit. This is a reason to explore
independent fact facets; it must not be mistaken for proof of no calls or full
coverage on large functions. No full provider parity claim is made.
The current producer also passes fourteen native JSON-format tests. The initial
command compiled unrelated integration targets and a non-embedded CLI build,
which replaced Cargo's mutable `target/release/kuna`. Its 181.103 wall / 172.430
CPU seconds, 651.6 MiB peak and OOM 0 describe that broader compilation, not just
JSON test execution. Future focused commands should select `--bin kuna` and
`--features embed-specs`. The original embedded build restores from Cargo cache
and has the same `64c6b3f...` digest.
The first model launch after those tests correctly fails at `open_binary` with
`engine_profile_mismatch`; it produces no target Evidence and is not a completed
answer. Its rejected-trial costs are retained in `trials.md`. The producing
boundary is now fixed by preserving an independent executable copy in a directory
named by the full SHA-256, outside Cargo outputs. The model harness checks the
explicit experimental engine pin before copying credentials or launching a model,
and verifies the same bytes after the task. Changed/missing engines reject the
comparison while retaining costs. Other provider trials keep this field unknown.
Three regression tests cover missing/malformed pins, replacement and removal;
a real driver invocation with a wrong pin rejects before output/profile creation.
## Shallow outgoing references without decompilation
Kuna already supplies native `xrefs --from`, including exact-instruction and
owning-function queries. The shared `KunaReferenceFiles.ts` boundary now owns
both directions, validates direction-dependent peer fields and query ownership,
and preserves exact u64 endpoints and source-row occurrences. The old
incoming-facts-only parser is removed; both experimental consumers use the same
named query contract. Production incoming address-only queries keep their
existing projection and bounded cache semantics.
The experimental `references` method adds direction, explicit address/function
scope, native kind filtering and fields. Incoming function scope is rejected;
unknown outgoing function applicability, unattributed/excluded rows and unresolved
indirect/discovery/runtime coverage remain explicit. Native source cache keys
cover result-affecting query choices while scope, fields and pages reuse the same
source. `reference-reads.mjs` owns protocol selection/projection; the resource
owner and CLI/MCP adapters do not reinterpret native endpoint fields. Empty
field selection now requests the existing always-retained identities, rather
than being rejected without a format/resource reason.
Real outgoing CLI probes return 396 VM call-reference rows in 205,432 bytes,
versus one row/737 bytes for selected call site `0xf238b`. Four sequential probes
(VM function/instruction and wrapper function/instruction) take 6.772 family wall
/ 6.613 CPU seconds and 120.0 MiB peak. Each command takes about 1.6–1.7 seconds;
these are native metadata-query measurements with warm uncontrolled file caches,
not real-model speed/token results.
Independent GNU objdump decoding over the exported VM symbol (`0xea6d0`, size
`0x7e7e`) confirms all 396 `(origin,target)` pairs, 171 distinct exact targets,
and 16 calls to `0x296c0`. It also finds 21 indirect calls, outside these
known-target counts. In the wrapper, nine lifted machine call references compare
with eight retained IR calls: machine `0x6534b -> 0x29370` is an omitted
`__stack_chk_fail@plt` call. Its absence from optimized IR/C is not absence from
the executable. No native analysis or C behavior is changed to repair that
representation difference.
The real Code Mode MCP verifier passes paging, identity-only fields, scope-cache
reuse, exact sites, unknown applicability and local aggregation. It also captures
the VM anew: C is available, IR fails with `atomic operation capacity exceeded`,
and shallow references still work. This verifier takes 48.142 family wall /
47.803 CPU seconds, 610.1 MiB peak; VM decompilation/source capture itself takes
35.945 seconds. These are different requested capabilities, not an equivalent
workflow speedup claim. The direct verifier passes the same reference contract
without that VM decompilation (12.001 / 11.759 seconds, 256.8 MiB). Both verify
cleanup and OOM 0. An earlier direct-verifier attempt incorrectly expected target
metadata on an SDK input-validation error; the corrected verifier respects the
SDK rejection boundary. That failed assertion receipt is retained.
The exact focused boundary suites pass 38 tests, with a fresh strict production
build; changed-test strict typechecking and 45 experimental JS tests also pass.
Full lint/module boundaries and changed-file formatting pass. Repository lint
warnings remain outside this change. The focused build/test scope peaks at
3,030.6 MiB, below the 3 GiB early-stop line, with OOM 0. Two full `check:fast`
attempts (including serial execution and Go runtime memory controls) reach that
early-stop budget and are terminated before OOM; they are not reported as passed.
The changed-test typecheck uses 70.1 MiB and standalone full lint 919.0 MiB.
Full-repository typechecking and full upstream Kuna gates remain unverified.
The [next question/oracle](reference-oracle.md) are committed before model trials.
## Independent call and boundary-value facets
Graph schema 3 removes embedded call records. An independent `atomic_calls`
version-1 facet now carries every retained block-member CALL/CALLIND, target-input
SSA identity, origin/block/opcode/dead status, call specs and boundary values.
`atomic_values.rs` owns common native value interpretation. The call scan walks
native block membership without allocating an operation vector for every block;
its own call/input/value/text budgets do not depend on the 12,000-op graph cap.
The default native path still does not collect either facet and C analysis phases
are unchanged. All captures occur after the same C rendering.
The bounded serializer retains fitting facets independently and preserves ordinary
C/metadata. It tries the full document, graph without calls, calls without graph,
then the base record with both reasons. This keeps existing graph availability
when both facets individually fit but their combination does not. Facet contents
are moved for these bounded alternatives rather than cloned; no reanalysis or
clipped collection labeled complete is introduced.
The JS owner validates each facet independently with shared ID/value contracts
and row-wise admission. When both parse, call identities, origins, operands,
outputs and boundary values must agree with the graph owner. Bad call facts do
not erase a valid graph; a bad/oversized graph does not erase valid calls. Code
survives either failure. `values` uses the complete retained graph when present,
otherwise the explicitly scoped call-boundary set; absent IDs in that set do not
prove absence from the native function. Defining operation identities may remain
unreadable, and source signatures, source parameter ordinals, aliases and runtime
binding stay unknown. `is_dead` is always retained on call rows, like an
instruction decode status. CLI, SDK and both MCP adapters share these views.
The new preserved engine is
`e7d3f0cc8d8aae814bd6a2625a567d6aaad96c3fe2405a4f08dc8e065b851588`.
Its full rebuild took 315.815 family wall / 315.305 CPU seconds, 2,119.4 MiB
kernel process-family peak. Four focused native atomic-JSON tests pass, including
both overflow directions, both-overflow C preservation and graph preference when
the combined document is too large (95.981 / 95.631 seconds including test-target
compilation, 626.8 MiB). The final CLI rebuild after tests is 26.203 / 25.952
seconds, 492.7 MiB. It is copied outside Cargo outputs and pinned by full digest.
All these runs verify cleanup and OOM 0; full upstream gates remain unexecuted.
Real public MCP verification keeps VM C available, reports graph operation
capacity failure, and now exposes 416 calls plus 1,294 boundary values. The call
scan examines 21,349 operations over 1,229 blocks. It preserves 395 direct known
targets and 21 unresolved indirect calls; all direct pairs match independent
machine pairs except the omitted `0xf2493 -> 0x29370` stack-check call. All 21
indirect origin addresses also match independent disassembly. Selected
`0xf115c`/`0xf238b` calls target `0x296c0`; their fspec inputs remain separate
annotation values. Repeated snapshots reuse the same handle and source, and an
explicit unavailable graph read fails without hiding these retained facts.
The VM source grows from 273,703 to 793,791 bytes to retain this new data; the
wrapper grows from 202,206 to 208,083 bytes. Their ordinary records, including C,
variables and line mappings, are identical to the prior capture. The wrapper's
blocks/operations/values/coverage also remain identical. These are fidelity and
source-volume observations, not model-token or speed results.
The real reference/VM verifier passes in 49.471 wall / 49.130 CPU seconds,
615.8 MiB. The existing cross-function call/input verifier passes in 8.377 /
8.094 seconds, 292.9 MiB; encoded Pragma/parser C/graph checks pass in 12.403 /
12.195 seconds, 335.0 MiB. All verify cleanup and OOM 0. The updated contract
suite passes 47 JS tests, including independent facet failure, cross-facet
conflicts, input-zero identity and boundary-only value reads. `check-spec` passes
in default lenient mode, lint and changed-file formatting pass, and the patch
applies forward to the pinned upstream revision and reverses from the clone.
Full repository typechecking retains its previously recorded resource limitation.
The [boundary question/oracle](boundary-oracle.md) are fixed before model runs.
The subsequent real-model pair passes that oracle in both calling modes. Each
uses one VM analysis and the same retained function record; neither invents a
complete graph from the available call-boundary values. Source digests, target
and preserved-engine pins, selected machine checks, cleanup and OOM-zero receipts
verify. See [whole-task costs](trials.md): the capability survives real model use,
but a single mixed-context-cost pair does not establish general efficiency.
## Compact counts from the validated call owner
The following API-only change keeps the preserved engine and native source
format unchanged. Snapshot call metadata includes counts by recovered opcode
and known/unknown native target, plus call coverage. The owner computes these
once after validation, before the shared CLI/SDK/MCP views. Counts include dead
rows and do not infer target knowledge from CALL versus CALLIND. Available empty
sets have zeros; unavailable sets have no counts. Selected call reads do not
change this whole-facet summary or require a new analysis.
The focused suite passes 49 tests, including known indirect/unknown direct
targets, dead rows, immutable summaries and empty/unavailable distinction.
Both real MCP adapters verify the wrapper's eight direct calls and parser's
28 direct/five unresolved indirect calls while preserving the existing
cross-function checks and validating advertised schemas after SDK conversion.
Code Mode verification takes 29.263 family wall / 21.548 CPU seconds, 283.9 MiB
kernel family peak; direct takes 32.893 / 22.781 seconds, 280.6 MiB. Both have
verified cleanup and OOM 0. These are validation runs, not a controlled speed
comparison with previous verifier timings. An owner replay of the prior
digest-verified VM source yields the expected 395/21 distribution; it is not
relabeled as a fresh provider or model run. The identical boundary question and
oracle are retained for subsequent real-model comparison.
That comparison now passes in both modes. Both consume the compact counts and
read just the two selected calls, reusing one VM analysis. All native VM function
records match the preceding pair and all retained sources/digests verify. Direct
has lower observed task costs than Code Mode in this sample; the before/after
results remain mixed. [trials.md](trials.md) records cumulative/cached/uncached
tokens, model responses, elapsed time, CPU and process-family memory, including
the slower Code Mode result. No native engine or analysis-speed improvement is
claimed for this shared owner metadata change.
## Tagged cleanup ownership regression
Source inspection finds that the experimental owner awaits but ignores
`KunaTargetCommands.close()`'s tagged `Result`. On error it would clear its state
and let `AtomicSession` drop the owner while claiming a successful close. The
SDK verifier reproduces this before the fix: an injected `ProviderCleanupError`
does not reject close. Its fallback cleanup still removes the real owned state;
the original failing result is retained as `cleanup-before.json`.
The owning boundary now uses the existing `requireResult` projection before
closing its cache root or clearing state. The existing session queue therefore
retains the owner on failure. Both close and target replacement report
`cleanup_incomplete`, preserve the staged file identity and original lease, and
succeed on retry; the file is then absent. MCP's open/close error adapter now
preserves context and resource details rather than dropping that evidence. The
CLI already propagates the thrown error as a failed command. No alternate
cleanup owner or downstream retry workaround is added.
`verify-cleanup.mjs` runs against real provider reads and resources, injecting
one tagged failure per close/replacement case. `verify-cleanup-mcp.mjs` injects
one explicit test-only failure in an isolated server, then verifies public
Code Mode and direct error metadata and successful retry. These are controlled
boundary failures, not a claim that the real engine failed to stop. The SDK
check takes 1.730 family wall / 1.567 CPU seconds, 130.4 MiB kernel peak; MCP Code
Mode takes 1.931 / 1.639 seconds, 200.0 MiB; direct takes 1.730 / 1.489 seconds,
192.8 MiB. All verify process cleanup and OOM 0. The original failing verifier
takes 1.226 / 0.985 seconds, 122.7 MiB and also verifies fallback cleanup/OOM 0.
The 49 focused JS tests, lint and formatting pass. This lifecycle-only change
follows the recorded model pairs; no new model token or speed claim is made.
## Callable bounded definition projection
The next native exporter retains a version-1 `atomic_projection` independently
of graph version 3 and call version 1. One checked native membership scan owns
call roots and an optional index of at most 65,536 operation keys; exceeding the
index budget disables the projection without discarding calls. Full-graph row
capacity is checked before rendering its rows. Full and selected facts share
one operation/value normalizer, including exact identities, indexed operands,
outputs, phi predecessor edges, flags and display. Default non-atomic analysis
and C generation are unchanged.
All recovered CALL/CALLIND operations are retained as projection roots, followed
by backward operand definitions in breadth-first order. The projection bounds
4,096 operations, 12,288 values, 16,384 inputs and 2 MiB text. Too many roots or
another admission failure makes the facet explicitly unavailable. Omitted
definitions retain exact IDs and operation-budget/outside-membership reasons.
A fitting full graph omits its redundant projection; an encoded-overflow graph
restores the fitting projection. Facets are moved through bounded alternatives,
and a valid C/call record survives projection failure. There is no repeated
decompilation or clipped collection labeled complete.
The JS boundary shares operand/value validation and query indexes between full
and sparse views. It checks root reachability, exact frontier membership,
call/value agreement and phi metadata without inventing a CFG. In the absence
of a full graph, conflicting projection facts do not erase independently valid
calls. `snapshot.facets.operations` and `facets.values` expose their scopes;
selected definitions/uses and slices work over the retained set. Uses remain
selected-only, CFG/controller queries retain their graph error and Evidence ID,
and slice missing definitions keep their native reasons. A missing projection
preserves the original graph failure instead of hiding it behind a generic
not-captured error. All adapters use this owner.
Coverage returns counts instead of global omitted-ID inventories, at most eight
160-character issue summaries, full issue counts and omission metadata. Complete
native coverage/frontier details remain in source Evidence. Native operation
rows retain `is_dead` with their identities; constant summaries add `value_id`
and `input_index`, preserving exact offset/width semantics. No original-signature,
arbitrary-width computation, memory-alias or runtime proof is introduced.
Preserved engine SHA-256:
`a0685486518e09d3c65f756325a7e9fed8df123a2a988aca2972a72335ce8edc`.
The same one-job/embedded-spec build takes 309.805 family wall / 309.523 CPU
seconds, 2,099.5 MiB kernel family peak. Six focused native JSON tests pass,
including projection restoration and independent overflow (99.752 / 98.662
seconds with test compilation, 626.9 MiB). The final CLI build reuses artifacts
(0.320 / 0.146 seconds, 28.8 MiB), leaving the preserved digest unchanged.
Real SQLite VM public reads expose 4,096 operations, 7,469 values and 326
unexpanded definitions, while the complete graph remains unavailable. Source is
4,008,032 bytes, inside the unchanged 8 MiB public budget. Every selected operation
and value equals the earlier 17.8 MB diagnostic capture; ordinary C/metadata and
the complete call facet also equal it. An independent BFS over that retained
full graph produces exactly the same operation order and frontier. The extended
`measure-native-projection.py --verify-capture` reproduces these comparisons.
This is producer-fidelity evidence, not independent machine/runtime equivalence.
Both public MCP adapters read the selected PHI definitions and slices from one
VM source. Code Mode verifies unavailable controllers preserve the source ID.
The corrected Code Mode verifier takes 52.706 wall / 47.550 CPU seconds,
597.4 MiB peak; direct takes 39.696 / 39.418 seconds, 592.5 MiB. The first
Code Mode verifier incorrectly expected INDIRECT definitions; independent source
inspection shows MULTIEQUAL. Its 37.070 / 36.781 seconds, 593.3 MiB failed
assertion is retained, then the verifier is corrected without changing native
facts. These scripted runs are validation costs, not an adapter speed comparison.
Small-function cross-call checks pass (8.476 / 8.204 seconds, 293.6 MiB); all three
complete function records equal the prior exporter. Encoded Pragma/Parser checks
pass (13.712 / 13.507 seconds, 361.5 MiB): Pragma retains 3,627 projected operations
after graph encoding overflow, while Parser keeps its full 5,164-operation graph
without a duplicate projection. Zlib control/phi regression passes (10.893 /
10.700 seconds, 286.3 MiB). Real CLI reads verify both selected PHIs and repeated
snapshot/source identity (37.367 / 37.174 seconds, 527.1 MiB).
The real VM MCP cache benchmark observes cold snapshot wall 36,448.8 ms / family
CPU 36.425 seconds, then warm 40.2 ms / 0.041 seconds with the identical handle
and source. Cumulative family peak is 590.1 MiB, an upper bound on isolated warm
phase memory. Its complete run is 37.667 / 37.492 seconds. This is same-lease
capture reuse, not cross-session persistence or a model-token result; OS caches
remain uncontrolled. All resource runs above verify cleanup and OOM 0.
The focused JS suite passes 53 tests; lint/format, default lenient `check-spec`,
Python compilation and patch forward/reverse checks pass. Full upstream corpus,
stages and workspace gates remain unexecuted; the patch is not upstream-ready.
No whole-repository typecheck pass is claimed beyond the previously recorded
resource limitation. The [PHI-format task/oracle](projection-oracle.md) are fixed
before real-model launches. No model benefit is assumed from these smaller facts.
## Named reference discovery and input schemas
The first PHI model pair completes correctly but exposes a documentation boundary:
direct help displays `facts_*` names while lookup only recognizes raw method
substrings. A multi-name query returns no matches and causes a full-reference
fallback. Registration and help now share one explicit name map, with canonical
identity separate from presentation. Exact lists preserve caller order,
deduplicate names and retain unmatched terms. Topic/all-contract lookup remain;
ambiguous labels fail instead of choosing by catalog position.
The real client verifier also catches that plain `z.toJSONSchema` defaults to
output semantics, making defaulted `offset`/`limit` falsely required in advertised
input schemas. Reference generation now explicitly selects `io:input`. Converted
schemas are frozen and weakly cached by schema identity; a replacement schema
invalidates the conversion, while description/presentation is read separately.
The documentation module removes the adapter rename pass. No target analysis,
live-result caching or native-engine change is involved.
The shared suite passes 55 tests. `verify-discovery.mjs` checks both real SDK
catalogs and reference responses without opening a target, compiles every catalog
and all fourteen reference schemas with Ajv2020, and validates the same
omitted-default requests against named reference/direct contracts. It proves the
six-name query matches all requested contracts without a miss. The all-schema
verifier takes 1.936 family wall / 1.760 CPU seconds, 138.8 MiB kernel peak,
cleanup verified and OOM 0. The earlier six-schema probe takes 1.628 / 1.449
seconds, 128.9 MiB. Its original 1.730 / 1.487 seconds, 128.1 MiB failed
omitted-default check remains recorded. Six contracts occupy 10,835 Code Mode /
10,360 direct bytes, versus 21,746 / 20,699 for all fourteen. Bytes do not prove
token or time savings. The same PHI question/oracle are retained for the next
actual model pair.
## Wide native operand relations
A retained-source replay exposes a whole-row cost: VM PHI
`0xea998:14649` has 156 inputs and 156 definition edges. Its normalized old
operation is 44,447 JSON bytes, and a default one-node slice returns 44,980 data
bytes. Real QuickJS returns `output_budget` while preserving the source ID.
This replays real producer facts through the shared parser/executor, without
a fresh native analysis or model call (`wide-phi-before.json`). Explicitly
requesting only `inputs` already fits; the gap is default size and lack of
independently selected/paged operands, not inability to obtain the data at all.
`relations kind:operands` now owns slot-level paging and optional input-index
selection. Snapshot + operation + input_index identifies the relationship;
value identity, native definition and exact phi edge are separate fields. It
retains constants, function inputs and nulls that definition edges cannot cover.
Requested out-of-range slots of retained seeds are absent, while unretained
seeds stay explicit/unknown. Seeds and selectors deduplicate in caller order;
total counts filtered rows before paging. Full graph and projection share this
owner; diagnostic SSA rejects the unsupported relation with source context.
Paging counts existing relation groups before materializing selected rows,
avoiding a complete flattened relation copy. Native operation defaults are
compact summaries with exact input/definition/constant counts; detailed fields
remain selectable. Repeated value/block identity lookups deduplicate at their
shared read boundary. A discriminated contract replaces the prior runtime-only
relation refinement, so advertised schemas also enforce seed types and prohibit
input_indices on other relationship kinds. No native rebuild, higher native
capacity, new method or additional authority is involved.
The shared suite passes 57 tests; changed JS lint has zero warnings and formatting
passes. Both real SDK catalogs and all reference schemas compile; selected
operand inputs and invalid selector applicability agree with runtime validation.
That discovery run takes 7.443 family wall / 4.928 CPU seconds, 141.8 MiB peak
including charged file cache. The real Code Mode VM verifier passes ten calls
in 54.327 / 47.069 seconds, 597.6 MiB peak. Its default slice fits, selected
slots preserve exact native facts, and explicit all-input reads still work.
Both runs verify cleanup and OOM 0. This first public run precedes the final
identity-deduplication adjustment; subsequent direct/model reads cover it.
Scripted bytes and costs are not model-token savings or an adapter speed claim.
The subsequent direct verifier passes all ten calls, including duplicate-value
deduplication, in 39.385 wall / 39.163 CPU seconds, 591.3 MiB peak, with cleanup
verified and OOM 0. The different scripted read paths/host state do not isolate
adapter costs. The [wide-operand oracle](operands-oracle.md) fixes the next seeded
real-model task before launch; it is not a before/after of the PHI-format task.
## Static method index
The wide-operand pair exposes a discovery cost: Code Mode asks four times for
help before using the same seven fact requests as direct. `reference view:index`
now returns the same fourteen canonical/display names and descriptions without
constructing schemas, return contracts or examples. `view:contracts` preserves
the existing default and exact-name/topic behavior. Both presentations share one
named documentation contract/owner and the API's registry; no analysis change or
engine rebuild is involved. An index is static declared support, never live
provider capability. Runtime facets remain authoritative.
The shared suite passes 58 tests, including a getter that proves an index does
not access an unrequested input contract. Changed JS lint/format and both real
SDK catalogs/reference schemas pass. The first discovery probe takes 5.284 family
wall / 4.028 CPU seconds, 137.6 MiB peak, cleanup verified and OOM 0. Code Mode's
index is 7,212 JSON bytes versus 24,010 for all contracts; direct is 7,240 versus
22,939. These bytes are static discovery output, not demonstrated token/time
savings. A subsequent focused check and real probe cover moving the input schema
into the documentation owner and unifying presentation construction. The same
wide-operand question/oracle are retained for a new real-model pair.
The final real SDK probe passes in 4.380 wall / 3.807 CPU seconds, 141.4 MiB peak,
cleanup verified and OOM 0. Both subsequent real-model answers satisfy the
unchanged oracle. Code Mode uses index plus one exact contract query, reducing
its discovery lookups from four to two. Whole-task costs remain mixed; all
attempts and limits are recorded in [trials.md](trials.md).
## Fact granularity versus computation granularity
The eight successful real-model VM captures using the preserved `a0685486...`
engine each retain 4,008,032 bytes, including 126,519 bytes of reconstructed C
that the selected-operand models never read. Fresh-lease native request wall
ranges from 36.702 to 53.124 seconds. A separate failed first attempt hits the
120-second timeout before one successful capture; its cause remains unknown.
These are fresh private native caches, not controlled OS-cold samples. Same-lease
warm source reuse remains separately measured above.
Source inspection explains the current computation boundary: the project core
drives whole-function decompilation, renders C and extracts metadata, then
collects independent call/graph/projection facets from the live function. The
new slot atom reduces selected read/serialization scope, but does not implement
selected-slot native analysis. The complete VM CFG is still unavailable and
arbitrary definitions outside the retained call-rooted projection cannot be
expanded by a fact read. Provider parity and general complex-software superiority
remain unestablished.
At that checkpoint, pipeline/render/metadata/export costs were not separately
profiled. The [subsequent native computation study](native-compute-study.md) now
measures them: VM drive about 34 seconds, C rendering 0.22 seconds and projection
0.03 seconds. Skipping C is a small opportunity for this workload. An offline
lossless encoding probe also puts the earlier full 17.8 MB diagnostic record
inside the unchanged 8 MiB transport. The [subsequent codec study](native-graph-transport-study.md)
now implements and verifies this through both public experimental MCP adapters,
including complete decoded Evidence and shared text-budget ownership. Actual
guard/control model behavior is now checked in two real-model pairs. Model
comparisons share the same computation/fact owners, avoiding a capability
difference between adapters. The [switch extension](switch-study.md) adds an
independent native table facet with exact labels and row/edge mapping, selected
queries and a verified 2 MiB fallback that retains cases without a full graph.
Its final engine is `a9ca8a8ac224936afb08a9d4be9c1225feb69473ad9f35764e8d1c4f04875b5b`;
the fixed case/default model question is separate from the earlier guard pairs.
@@ -0,0 +1,49 @@
// One bounded decoded source view; the source owner revalidates bytes per read.
import { createHash } from "node:crypto";
import { readNativeGraphEvidence } from "./native-ir-contract.mjs";
export class NativeEvidenceReads {
#graph;
constructor(readSource) {
this.readSource = readSource;
}
clear() {
this.#graph = undefined;
}
async read(input) {
const { text: original, source } = await this.readSource(input.evidence_id);
let text = original,
view;
if (input.component === "recovered_ir") {
const key = `${input.evidence_id}:${source.sha256}`;
if (this.#graph?.key !== key) {
const decoded = readNativeGraphEvidence(text);
this.#graph = {
key,
...decoded,
sha256: createHash("sha256").update(decoded.text).digest("hex"),
bytes: Buffer.byteLength(decoded.text),
};
}
const decoded = this.#graph;
text = decoded.text;
view = {
component: "recovered_ir",
encoding: decoded.encoding,
sha256: decoded.sha256,
bytes: decoded.bytes,
authority:
"uninterpreted retained graph source; fact validity is separate",
};
}
const offset = input.offset ?? 0,
limit = input.limit ?? 4096;
return {
source,
...(view ? { view } : {}),
text: text.slice(offset, offset + limit),
total_code_units: text.length,
next_offset: offset + limit < text.length ? offset + limit : null,
};
}
}
@@ -0,0 +1,86 @@
import assert from "node:assert/strict";
import test from "node:test";
import { createHash } from "node:crypto";
import { gzipSync } from "node:zlib";
import { NativeEvidenceReads } from "./native-evidence-reads.mjs";
const hash = (s) => createHash("sha256").update(s).digest("hex");
function captured(text) {
const bytes = Buffer.from(text);
const wire = JSON.stringify({
binary: "/input",
count: 1,
language: "c",
error: null,
functions: [
{
address_hex: "0x1000",
code: "retained C",
error: null,
atomic_facts: {
schema_version: 4,
encoding: "gzip-base64",
raw_bytes: bytes.length,
data: gzipSync(bytes).toString("base64"),
},
},
],
});
return {
text: wire,
source: {
evidence_id: "ev_owned",
sha256: hash(wire),
bytes: Buffer.byteLength(wire),
},
};
}
test("decoded Evidence pages preserve their own byte identity and raw source provenance", async () => {
const logical = "{malformed JSON with 中文 and 😀",
source = captured(logical);
const reader = new NativeEvidenceReads(async () => source);
const raw = await reader.read({ evidence_id: "ev_owned", limit: 1e5 });
assert.equal(raw.text, source.text);
assert.equal(raw.view, undefined);
const chunks = [];
for (let offset = 0; offset < logical.length; offset += 3) {
const page = await reader.read({
evidence_id: "ev_owned",
component: "recovered_ir",
offset,
limit: 3,
});
chunks.push(page.text);
assert.equal(page.source.sha256, source.source.sha256);
assert.equal(page.view.sha256, hash(logical));
assert.equal(page.view.bytes, Buffer.byteLength(logical));
assert.equal(page.total_code_units, logical.length);
}
assert.equal(chunks.join(""), logical);
});
test("decoded view reuse still requires source-owner validation and follows content identity", async () => {
let source = captured('{"schema_version":3}'),
failure = false,
reads = 0;
const reader = new NativeEvidenceReads(async () => {
reads++;
if (failure) throw new Error("source_changed");
return source;
});
const query = { evidence_id: "ev_owned", component: "recovered_ir" };
await reader.read(query);
failure = true;
await assert.rejects(reader.read(query), /source_changed/);
failure = false;
source = captured('{"schema_version":3,"new":true}');
assert.equal(
(await reader.read(query)).text,
'{"schema_version":3,"new":true}',
);
reader.clear();
assert.equal(
(await reader.read(query)).view.sha256,
hash('{"schema_version":3,"new":true}'),
);
assert.equal(reads, 4);
});
@@ -0,0 +1,36 @@
// The wire owner validates the archive before normal graph-row admission.
import { gunzipSync } from "node:zlib";
import { z } from "zod";
import { requireFact } from "./native-contract.mjs";
import { NATIVE_OUTPUT_BYTES } from "./limits.mjs";
export const MAX_GRAPH_JSON_BYTES = 32 * 1024 * 1024;
const envelope = z.strictObject({
schema_version: z.literal(4),
encoding: z.literal("gzip-base64"),
raw_bytes: z.number().int().min(1).max(MAX_GRAPH_JSON_BYTES),
data: z.string().max(NATIVE_OUTPUT_BYTES),
});
export function decodeGraphArchive(raw) {
if (raw?.schema_version !== 4) return undefined;
const encoded = envelope.parse(raw);
requireFact(
encoded.data.length > 0 &&
encoded.data.length % 4 === 0 &&
/^[A-Za-z0-9+/]*={0,2}$/u.test(encoded.data),
"Native compressed graph has invalid base64",
);
const bytes = Buffer.from(encoded.data, "base64");
requireFact(
bytes.toString("base64") === encoded.data,
"Native compressed graph has noncanonical base64",
);
// The native-declared length is admitted before inflation, not checked only
// after an unbounded allocation. Node also verifies the gzip framing/CRC.
const decoded = gunzipSync(bytes, { maxOutputLength: encoded.raw_bytes });
requireFact(
decoded.length === encoded.raw_bytes,
"Native compressed graph length differs from its declaration",
);
return new TextDecoder("utf-8", { fatal: true }).decode(decoded);
}
@@ -0,0 +1,66 @@
import assert from "node:assert/strict";
import test from "node:test";
import { gzipSync } from "node:zlib";
import {
decodeGraphArchive,
MAX_GRAPH_JSON_BYTES,
} from "./native-graph-encoding.mjs";
const encodeBytes = (bytes) => ({
schema_version: 4,
encoding: "gzip-base64",
raw_bytes: bytes.length,
data: gzipSync(bytes, { level: 1 }).toString("base64"),
});
const encode = (value) => encodeBytes(Buffer.from(JSON.stringify(value)));
test("bounded gzip preserves logical JSON fields and legacy graphs without a second interpretation", () => {
const graph = {
schema_version: 3,
statement: "中文\u0000😀",
offset: "0xffffffffffffffff",
};
assert.equal(decodeGraphArchive(encode(graph)), JSON.stringify(graph));
assert.equal(decodeGraphArchive(graph), undefined);
});
test("admission rejects unsupported envelopes and malformed base64 before inflation", () => {
const good = encode({ schema_version: 3 });
for (const change of [
{ encoding: "deflate" },
{ raw_bytes: 0 },
{ raw_bytes: MAX_GRAPH_JSON_BYTES + 1 },
{ unexpected: true },
{ data: "AAAA\n" },
{ data: "A===" },
{ data: "?AAA" },
{ data: "A".repeat(8 * 1024 * 1024 + 1) },
])
assert.throws(() => decodeGraphArchive({ ...good, ...change }));
assert.throws(
() => decodeGraphArchive({ ...good, data: "AB==" }),
/noncanonical/,
);
});
test("inflation respects declared size and verifies framing without interpreting logical source", () => {
const graph = { schema_version: 3, statement: "x".repeat(1024 * 1024) };
const good = encode(graph);
assert.throws(() => decodeGraphArchive({ ...good, raw_bytes: 64 }), {
code: "ERR_BUFFER_TOO_LARGE",
});
assert.throws(
() => decodeGraphArchive({ ...good, raw_bytes: good.raw_bytes + 1 }),
/differs/,
);
const corrupt = Buffer.from(good.data, "base64");
corrupt[corrupt.length - 8] ^= 1;
assert.throws(() =>
decodeGraphArchive({ ...good, data: corrupt.toString("base64") }),
);
assert.throws(() =>
decodeGraphArchive({
...good,
data: Buffer.from(good.data, "base64").subarray(0, -5).toString("base64"),
}),
);
assert.throws(() => decodeGraphArchive(encodeBytes(Buffer.from([0xff]))));
assert.equal(decodeGraphArchive(encodeBytes(Buffer.from("{"))), "{");
});
@@ -0,0 +1,124 @@
# Bounded full recovered graphs
This implements the preceding native encoding probe as an opt-in experiment,
not stock Kuna support, an integrated release or provider parity.
## Format and owners
Logical graph schema 3 stays unchanged. A fitting ordinary record keeps its
existing representation. Before dropping facts, the writer can encode only the
graph in a schema-4 `{encoding:"gzip-base64",raw_bytes,data}` envelope. Ordinary
C/metadata and independent calls keep their fields. Existing graph/call/
projection/code fallback and explicit failure still apply if encodings do not
fit; no graph is clipped as complete.
Raw serialization stops at 32 MiB. Envelope overhead and base64 expansion are
accounted for before allocation; gzip output and encoded alternatives share the
unchanged caller budget of 8 MiB. Alternatives move ownership instead of cloning
graphs. Construction separately admits 32,000 operations, 36,000 values, 100,000
inputs, 12,000 blocks, 50,000 CFG edges and 4 MiB display/type text. Calls and
projections keep their earlier limits.
Each collector now owns one `TextBudget` for combined display/type charges. The
initial larger graph exposed a hidden second 2 MiB limit in value collection;
the first real VM probe rejected it with `atomic text capacity exceeded` while
preserving calls/C/projection. Passing the collector-owned budget through value
extraction removes this competing policy. Final graph text is 2,345,629 bytes.
`native-graph-encoding.mjs` owns envelope admission, canonical base64, gzip
integrity, exact lengths and fatal UTF-8 decoding. Inflation uses Node's
[`maxOutputLength`](https://github.com/nodejs/node/blob/main/doc/api/zlib.md#class-options)
with an admitted 32 MiB maximum. Logical schema/row/relationship/text admission
and existing native identity/CFG/phi checks then operate on the same graph.
Nested encodings fail logical admission; bad graphs preserve independent calls/C.
`evidence` defaults to the exact source wire record and original hash.
`component:"recovered_ir"` pages decoded graph source with a separate view hash
and byte count. Well-framed malformed logical facts remain inspectable without
being accepted as facts. `NativeEvidenceReads` owns one decoded view up to 32 MiB
per lease; the source owner revalidates bytes on every read. Content identity
keys the view; successful close clears it. All adapters share these owners.
## Real verification
These graph-only engine/patch measurements use commit `e2205313a` and its
version of `kuna-native-ir.patch`. The subsequent [switch extension](switch-study.md)
has its own pinned engine/patch and retains the same logical graph fields.
Kuna upstream: `fe01d6294eb0d1053ce343c8943646df88e143fc`.
Final preserved engine:
`677b9795d49a65e459054ecbd3c0df37d603b541d72df9f74b46778947ed3936`.
Patch SHA-256:
`9f5754a15bd971326d14085f4c515348735d140be3bf36a371e7cf336fc291f3`.
Binaries are copied outside mutable Cargo outputs, not hardlinked or committed.
On the pinned Debian SQLite target, both actual MCP adapters under Node 24.18.0
verify 21,349 operations, 35,755 values, 1,229 blocks and zero omitted definitions.
The complete graph deeply equals the earlier diagnostic golden, and ordinary
C/metadata plus all 416 recovered calls are unchanged. Selected old projection
frontiers now resolve, selected CFG/PHI reads work, both complete Evidence views
verify and repeated snapshots reuse one source. This establishes producer
fidelity, not independent machine/IR equivalence or runtime feasibility.
The actual wire document is **3,788,881 bytes**; decoded graph is **17,008,223
bytes**. The public byte budget remains 8,388,608. Raw receipts/source copies are
`full-graph-codec-*` under `.cache/atomic-codemode/`.
| Executed scope | Family wall seconds | CPU seconds | Kernel peak MiB |
| ---------------------------------------- | ------------------: | ----------: | --------------: |
| Initial build | 441.295 | 391.223 | 2,194.8 |
| Focused native tests, including compile | 98.589 | 98.034 | 632.9 |
| Final budget-version native tests | 176.333 | 175.748 | 644.1 |
| Formatted cached rebuild | 26.999 | 26.749 | 499.3 |
| Collector-budget rebuild | 361.041 | 342.220 | 2,192.9 |
| First probe: hidden text-cap failure | 39.279 | 39.010 | 650.9 |
| Exploratory success under system Node 20 | 44.858 | 44.480 | 576.6 |
| Final Code Mode MCP, pinned Node 24 | 42.422 | 42.111 | 651.0 |
| Final direct MCP, pinned Node 24 | 41.496 | 41.252 | 644.2 |
| CLI: incorrect unique-name assumption | 1.229 | 1.037 | 132.5 |
| CLI: corrected string/xref composition | 43.512 | 43.217 | 584.2 |
| Full-graph cache benchmark | 41.912 | 39.955 | 588.4 |
| Both catalogs/reference schemas | 3.644 | 1.716 | 142.7 |
| Three small functions / call composition | 8.579 | 8.358 | 300.4 |
| Legacy engine / current reader | 39.813 | 39.425 | 642.5 |
All resource runs use one CPU, 4 GiB hard / 3 GiB early-stop bounds, verified
cleanup and OOM 0. Kernel peak includes charged file cache, not RSS. The system
Node probe is outside pinned-toolchain validation; the verifier now rejects a
wrong Node before native analysis. The first failed probe used preserved
pre-fix `fb717dde...`. Native focused tests pass ten cases; shared JS passes 65.
Changed JS lint/format, lenient native `check-spec`, applied-source whitespace
and patch forward/reverse checks pass. Patch-context spaces are format markers.
The three small-function records retain their prior ordinary representations;
the current reader also accepts the old engine's explicit full-graph refusal,
independent calls and 4,096-operation projection through real MCP reads.
The final native test rerun also passes the same ten selected cases after the
collector-budget change. Other CLI integration tests were compiled but filtered
out; they are not additional executed coverage.
Native full corpus/stages/workspace and whole-REA gates remain unexecuted or
previously resource-limited; no upstream/PR/integration readiness is claimed.
Fresh-lease snapshot: 39,677.7 ms / 38.726 family CPU seconds. Same-lease warm:
67.1 ms / 0.068 seconds, identical handle/source. Cumulative peak is 588.4 MiB,
an upper bound on isolated warm memory. OS cache is uncontrolled; no cross-session
cache or model-token benefit is inferred from this scripted benchmark.
## From names/messages to conditions
The inventory has two identical VDBE names (`0x288f0`, `0xea6d0`) without aliases.
The initial CLI assumed uniqueness. Its corrected workflow composes strings,
incoming references and owning-function addresses before one VM snapshot; then
constant producers, uses, derived controllers and selected values/definitions.
It obtains both savepoint guards in sixteen host reads without the C report.
The final example also checks pagination before treating selected facts as a
complete set. A retained-source replay derives the two controller relations in
about 8.9 ms wall; isolated replay CPU/peak memory were not measured.
The [question/oracle](savepoint-control-oracle.md) are fixed before model trials,
with function name/messages but no supplied code addresses. Selected independent
machine evidence checks local guards, load/spill/setup and shared call. Recovered
CFG/use coverage still does not establish full machine flow, memory aliases,
nontermination or feasible execution. The first [real-model pair](savepoint-control-study.md)
now passes the frozen question through both adapters; its whole-task costs are
mixed and include discovery noise and a Code Mode parameter retry. Public
scripted checks alone do not establish those model outcomes.
@@ -0,0 +1,258 @@
// Native JSON decoding and resource admission; row-wise validation bounds errors.
import { z } from "zod";
import { FactFailure } from "./fact-error.mjs";
import { NATIVE_OUTPUT_BYTES } from "./limits.mjs";
import { readCallFacet } from "./native-call-contract.mjs";
import { readProjectionFacet } from "./native-projection-contract.mjs";
import { readSwitchFacet } from "./native-switch-contract.mjs";
import { operation, parseOperation } from "./native-operation-contract.mjs";
import {
decodeGraphArchive,
MAX_GRAPH_JSON_BYTES,
} from "./native-graph-encoding.mjs";
import {
hex,
ordinal,
valueId,
operationId,
edgeId,
value,
rowParser,
requireFact,
formatFailure,
readNativeFacet,
} from "./native-contract.mjs";
const edge = z.strictObject({
id: edgeId,
target: ordinal,
index: ordinal,
polarity: z.enum(["true", "false"]).nullable(),
condition_value: valueId.nullable(),
condition_test: z.enum(["eq-zero", "ne-zero"]).nullable(),
});
const factsSchema = z.strictObject({
schema_version: z.literal(3),
blocks: z
.array(
z.strictObject({
id: ordinal,
start: hex,
operations: z.array(operationId),
predecessors: z.array(ordinal),
successors: z.array(edge),
}),
)
.min(1)
.max(12000),
operations: z.array(operation).max(32000),
values: z.array(value).max(36000),
coverage: z.strictObject({
scope: z.string(),
machine_semantic_equivalence: z.literal("unknown"),
indirect_branch_operations: z.array(operationId),
omitted_definition_ids: z.array(operationId),
issues: z.array(z.string()),
}),
});
const documentSchema = z.object({
binary: z.string(),
count: z.literal(1),
language: z.string(),
error: z.null(),
functions: z
.array(
z.object({
address_hex: hex,
code: z.string(),
error: z.null(),
atomic_facts: z.unknown().optional(),
atomic_facts_error: z.string().optional(),
atomic_calls: z.unknown().optional(),
atomic_calls_error: z.string().optional(),
atomic_switches: z.unknown().optional(),
atomic_switches_error: z.string().optional(),
atomic_projection: z.unknown().optional(),
atomic_projection_error: z.string().optional(),
}),
)
.length(1),
});
// Parse one row at a time: a bad provider must not expand thousands of malformed
// records into a giant Zod issue list before the caller can reject the source.
const nativeBlock = factsSchema.shape.blocks.element;
const parseBlock = rowParser(nativeBlock, {
operations: (id) => operationId.parse(id),
predecessors: (id) => ordinal.parse(id),
successors: (row) => edge.parse(row),
});
const graphCoverage = factsSchema.shape.coverage;
const parseCoverage = rowParser(graphCoverage, {
indirect_branch_operations: (row) => operationId.parse(row),
omitted_definition_ids: (row) => operationId.parse(row),
issues: (row) => z.string().parse(row),
});
const parseFacts = rowParser(
factsSchema,
{
blocks: parseBlock,
operations: parseOperation,
values: (row) => value.parse(row),
},
{ coverage: parseCoverage },
);
function preflight(facts) {
requireFact(
facts && typeof facts === "object",
"Native atomic facts facet is unavailable; inspect the retained source for its reason and C",
);
requireFact(
facts.schema_version === 3,
"Unsupported native atomic facts schema version; expected 3",
);
requireFact(
facts.blocks?.length > 0,
"Native recovered graph is unavailable",
);
for (const [key, limit] of [
["blocks", 12000],
["operations", 32000],
["values", 36000],
])
requireFact(
Array.isArray(facts[key]) && facts[key].length <= limit,
`Native ${key} exceeds resource capacity or has invalid shape`,
);
for (const [rows, limit] of [
[facts.coverage?.indirect_branch_operations, 32000],
[facts.coverage?.omitted_definition_ids, 36000],
[facts.coverage?.issues, 12000],
])
requireFact(
Array.isArray(rows) && rows.length <= limit,
"Native graph coverage exceeds capacity or has invalid shape",
);
const inputs = facts.operations.reduce(
(sum, op) => sum + (Array.isArray(op?.inputs) ? op.inputs.length : 0),
0,
);
const members = facts.blocks.reduce(
(sum, block) =>
sum + (Array.isArray(block?.operations) ? block.operations.length : 0),
0,
);
const edges = facts.blocks.reduce(
(sum, block) =>
sum +
(Array.isArray(block?.successors) ? block.successors.length : 0) +
(Array.isArray(block?.predecessors) ? block.predecessors.length : 0),
0,
);
requireFact(
inputs <= 100000 && members <= 32000 && edges <= 100000,
"Native relationships exceed resource capacity",
);
const displayBytes =
facts.operations.reduce(
(sum, op) =>
sum +
(typeof op?.statement === "string"
? Buffer.byteLength(op.statement)
: 0),
0,
) +
facts.values.reduce(
(sum, value) =>
sum +
(typeof value?.type_name === "string"
? Buffer.byteLength(value.type_name)
: 0),
0,
);
requireFact(
displayBytes <= 4 * 1024 * 1024,
"Native graph display exceeds resource capacity",
);
}
const readGraph = (fn) =>
readNativeFacet(fn, "atomic_facts", "recovered IR", (raw) => {
const decoded = decodeGraphArchive(raw);
const graph = decoded === undefined ? raw : JSON.parse(decoded);
preflight(graph);
return parseFacts(graph);
});
function readSource(text) {
requireFact(
Buffer.byteLength(text) <= NATIVE_OUTPUT_BYTES,
"Native JSON source exceeds transport budget",
);
return JSON.parse(text);
}
function readFunction(raw, entry, expectedInput) {
requireFact(
Array.isArray(raw?.functions) && raw.functions.length === 1,
"Native source must describe exactly one function",
);
const doc = documentSchema.parse(raw),
fn = doc.functions[0];
if (expectedInput !== undefined && doc.binary !== expectedInput)
throw new FactFailure(
"subject_mismatch",
"Native source names a different command input",
{ expected: expectedInput, observed: doc.binary },
);
if (entry !== undefined && fn.address_hex !== entry)
throw new FactFailure(
"subject_mismatch",
"Native function entry differs from requested subject",
);
return { doc, fn };
}
export function parseNativeFunction(text, entry, expectedInput) {
try {
const { doc, fn } = readFunction(readSource(text), entry, expectedInput);
return {
code: fn.code,
language: doc.language,
facts: readGraph(fn),
calls: readCallFacet(fn),
switches: readSwitchFacet(fn),
projection: readProjectionFacet(fn),
};
} catch (error) {
throw formatFailure(error);
}
}
export function readNativeGraphEvidence(text) {
try {
const raw = readSource(text);
if (raw?.functions === undefined)
throw new FactFailure(
"capability_unavailable",
"This retained source is not a native function record",
);
const { fn } = readFunction(raw);
if (fn.atomic_facts === undefined)
throw new FactFailure(
"capability_unavailable",
"Recovered graph source was not captured",
{ native_error: fn.atomic_facts_error?.slice(0, 1024) },
);
const decoded = decodeGraphArchive(fn.atomic_facts);
const view = decoded ?? JSON.stringify(fn.atomic_facts);
requireFact(
Buffer.byteLength(view) <= MAX_GRAPH_JSON_BYTES,
"Native graph Evidence view exceeds decoded capacity",
);
return {
text: view,
encoding: decoded === undefined ? "json" : "gzip-base64",
};
} catch (error) {
throw formatFailure(error);
}
}
+355
View File
@@ -0,0 +1,355 @@
// The native-format boundary owns validation and indexes; consumers never parse
// diagnostic statements to recover value identity, operand indexes or CFG edges.
import { FactFailure } from "./fact-error.mjs";
import {
indexSwitchFacet,
validateSwitchGraph,
validateSwitchValues,
createSwitchViews,
} from "./native-switch-views.mjs";
import { parseNativeFunction } from "./native-ir-contract.mjs";
import { controlDependencies } from "./control-dependence.mjs";
import { freezeFacts as freeze } from "./fact-reads.mjs";
import {
indexCallFacet,
validateCallGraph,
createCallViews,
} from "./native-call-views.mjs";
import { requireFact } from "./native-contract.mjs";
import {
indexOperationFacts,
createOperationViews,
coverageIssues,
} from "./native-operation-views.mjs";
import {
indexProjection,
createProjectionViews,
} from "./native-projection-views.mjs";
import {
uniqueNative as unique,
nativePage as page,
readNativeIdentities as readIdentities,
} from "./native-reads.mjs";
function validateBlocks(operations, values, blocks) {
const membership = new Set();
for (const block of blocks.values()) {
for (const id of block.operations) {
requireFact(
operations.get(id)?.block === block.id && !membership.has(id),
"Native operation/block ownership mismatch",
);
membership.add(id);
}
for (const [index, edge] of block.successors.entries()) {
requireFact(
edge.index === index &&
edge.id === `${block.id}:${index}` &&
blocks.get(edge.target)?.predecessors.includes(block.id),
"Native CFG successor/predecessor mismatch",
);
requireFact(
edge.condition_value === null || values.has(edge.condition_value),
"Dangling native branch condition",
);
}
for (const from of block.predecessors)
requireFact(
blocks.get(from)?.successors.some((edge) => edge.target === block.id),
"Dangling native predecessor",
);
}
requireFact(
membership.size === operations.size,
"Native operation omitted from block membership",
);
}
function validatePhiBindings(operations, blocks) {
for (const op of operations.values())
for (const [position, input] of op.inputs.entries()) {
if (input.phi_predecessor !== null)
requireFact(
blocks.get(op.block).predecessors[position] === input.phi_predecessor,
"Native phi/predecessor mismatch",
);
if (input.phi_predecessor_edge !== null)
requireFact(
blocks
.get(input.phi_predecessor)
?.successors.some(
(edge) =>
edge.id === input.phi_predecessor_edge &&
edge.target === op.block,
),
"Native phi incoming-edge identity mismatch",
);
}
}
function indexNative(facts) {
const core = indexOperationFacts(
facts.operations,
facts.values,
facts.coverage.omitted_definition_ids,
);
const blocks = unique(
facts.blocks.map(({ start, ...block }) =>
freeze({ ...block, origin_start: start }),
),
);
validateBlocks(core.operations, core.values, blocks);
validatePhiBindings(core.operations, blocks);
const incoming = new Map(),
outgoing = new Map();
for (const block of blocks.values())
for (const edge of block.successors) {
const row = freeze({
...edge,
from: block.id,
to: edge.target,
operation: block.operations.at(-1) ?? null,
kind: edge.polarity ? "conditional" : "recovered",
});
for (const [map, key] of [
[incoming, edge.target],
[outgoing, block.id],
]) {
if (!map.has(key)) map.set(key, []);
map.get(key).push(row);
}
}
return { ...core, blocks, incoming, outgoing };
}
function createViews(facts, code, indexes) {
const { blocks, incoming, outgoing, normalized } = indexes;
const {
issues,
indirect_branch_operations,
omitted_definition_ids,
...metadata
} = facts.coverage;
const coverage = freeze({
...metadata,
...coverageIssues(issues),
indirect_branch_operation_count: indirect_branch_operations.length,
omitted_definition_count: omitted_definition_ids.length,
operations: normalized.size,
values: indexes.values.size,
blocks: blocks.size,
control_flow: "complete-for-recovered-graph",
representation:
"optimized recovered IR; origins do not establish machine branch targets or block extents",
runtime_successors: "unknown",
memory_alias_dependencies: "unknown",
});
const core = createOperationViews(indexes, coverage);
return Object.freeze({
...core,
code,
block: (id) => blocks.get(id) ?? null,
incoming: (id) => incoming.get(id) ?? [],
blocks: (input) => readIdentities(blocks, input),
edges: (input) => {
if (["definitions", "uses", "operands"].includes(input.kind))
return core.operationRelations(input);
if (input.kind === "controllers") {
const controls = controlDependencies(facts.blocks, {
dependents: new Set(input.seeds),
});
return {
...page(controls.relations, input),
relation_coverage: controls.coverage,
};
}
return page(
input.seeds.flatMap(
(seed) =>
(input.kind === "incoming" ? incoming : outgoing).get(seed) ?? [],
),
input,
);
},
});
}
const projectedFailure = (error) => ({
code: error.code ?? "invalid_provider_output",
message: error.message,
details: error.details ?? {},
});
const operationFailure = (projectionError, graphError) =>
projectionError?.details?.availability === "not-captured"
? graphError
: (projectionError ?? graphError);
const captureFacets = ({
graph,
projection,
calls,
graphError,
operationError,
callError,
switches,
switchError,
}) =>
freeze({
code: { availability: "available" },
recovered_ir: graph
? { availability: "available" }
: { availability: "unavailable", error: graphError },
operations: graph
? {
availability: "available",
scope: "complete-for-recovered-graph",
count: graph.coverage.operations,
}
: projection
? {
availability: "available",
scope: "bounded-call-rooted-definitions",
count: projection.coverage.operations,
root_count: projection.coverage.root_count,
definition_frontier_count:
projection.coverage.definition_frontier_count,
}
: {
availability: "unavailable",
error: operationError,
},
calls: calls
? { availability: "available", ...calls.summary }
: { availability: "unavailable", error: callError },
switches: switches
? { availability: "available", ...switches.summary }
: { availability: "unavailable", error: switchError },
values: graph
? {
availability: "available",
scope: "complete-for-recovered-graph",
count: graph.coverage.values,
}
: projection
? {
availability: "available",
scope: "call-rooted-definitions",
count: projection.coverage.values,
}
: calls
? {
availability: "available",
scope: "call-boundary-only",
count: calls.value_count,
}
: { availability: "unavailable", error: callError },
});
const loadFacet = (result, build) => {
if (!result.ok) return { error: projectedFailure(result.error) };
try {
return build(result.value);
} catch (error) {
return { error: projectedFailure(error) };
}
};
const unavailable =
(facet, error = {}) =>
() => {
throw new FactFailure(error.code, error.message, {
...error.details,
facet,
});
};
export function parseNativeIR(text, entry, expectedInput) {
const capture = parseNativeFunction(text, entry, expectedInput);
const graph = loadFacet(capture.facts, (facts) => {
freeze(facts);
const index = indexNative(facts);
return { index, view: createViews(facts, capture.code, index) };
});
const calls = loadFacet(capture.calls, (facts) => {
const index = indexCallFacet(facts);
if (graph.view) validateCallGraph(index, graph.index);
return { index, view: createCallViews(facts, index) };
});
const unavailableGraph = unavailable("recovered_ir", graph.error);
const projection = graph.view
? {}
: loadFacet(capture.projection, (facts) => {
const index = indexProjection(facts);
// Without a full graph, the independent call owner validates the projection.
if (calls.view) validateCallGraph(calls.index, index);
return {
index,
view: createProjectionViews(
facts,
capture.code,
index,
unavailableGraph,
),
};
});
const unavailableCalls = unavailable("calls", calls.error);
const switches = loadFacet(capture.switches, (facts) => {
const index = indexSwitchFacet(facts);
if (graph.view) validateSwitchGraph(index, graph.index);
else {
validateSwitchValues(index, calls.index?.values);
validateSwitchValues(index, projection.index?.values);
}
return { view: createSwitchViews(facts, index) };
});
const unavailableSwitches = unavailable("switches", switches.error);
const operationError = operationFailure(projection.error, graph.error);
const unavailableOperations = unavailable("operations", operationError);
const fallback = {
code: capture.code,
coverage: freeze({
scope: "native reconstructed code; recovered IR facet unavailable",
operations: null,
blocks: null,
control_flow: "unknown",
machine_semantic_equivalence: "unknown",
issues: [graph.error?.message],
}),
...Object.fromEntries(
["operation", "findConstant", "uses", "select", "slice"].map((name) => [
name,
unavailableOperations,
]),
),
...Object.fromEntries(
["block", "incoming", "blocks"].map((name) => [name, unavailableGraph]),
),
edges: (input) =>
["definitions", "uses"].includes(input.kind)
? unavailableOperations()
: unavailableGraph(),
};
return Object.freeze({
...(graph.view ?? projection.view ?? fallback),
edges: (input) =>
input.kind === "cases"
? (switches.view?.cases ?? unavailableSwitches)(input)
: (graph.view ?? projection.view ?? fallback).edges(input),
language: capture.language,
calls: calls.view?.calls ?? unavailableCalls,
switches: switches.view?.switches ?? unavailableSwitches,
values:
graph.view?.values ??
projection.view?.values ??
calls.view?.values ??
unavailableCalls,
facets: captureFacets({
graph: graph.view,
projection: projection.view,
calls: calls.view,
graphError: graph.error,
operationError,
callError: calls.error,
switches: switches.view,
switchError: switches.error,
}),
});
}
@@ -0,0 +1,401 @@
import test from "node:test";
import assert from "node:assert/strict";
import { gzipSync } from "node:zlib";
import { parseNativeIR } from "./native-ir.mjs";
import { readNativeGraphEvidence } from "./native-ir-contract.mjs";
import { controlDependencies } from "./control-dependence.mjs";
const id = (n) => `0x${n}:0`;
const val = (n, definition = null) => ({
id: `v${n}`,
space: "register",
offset: "0x0",
size_bytes: 8,
is_constant: false,
is_input: definition === null,
is_annotation: false,
definition,
type_name: "uint64",
});
const input = (n, value_id, phi_predecessor = null) => ({
input_index: n,
value_id,
phi_predecessor,
phi_predecessor_edge:
phi_predecessor === null ? null : `${phi_predecessor}:0`,
});
const edge = (from, target, index, polarity = null) => ({
id: `${from}:${index}`,
target,
index,
polarity,
condition_value: polarity ? "v0" : null,
condition_test: polarity ? "ne-zero" : null,
});
const op = (block, opcode, inputs, output = null) => ({
id: id(block),
address: `0x${block}`,
block,
opcode,
inputs,
output,
statement: "display is deliberately unparseable",
is_dead: false,
});
function diamond() {
return {
binary: "/private/input",
count: 1,
language: "c",
error: null,
functions: [
{
address_hex: "0x1000",
error: null,
code: "reconstructed C",
atomic_facts: {
schema_version: 3,
blocks: [
{
id: 0,
start: "0x1000",
operations: [id(0)],
predecessors: [],
successors: [edge(0, 1, 0, "false"), edge(0, 2, 1, "true")],
},
{
id: 1,
start: "0x1001",
operations: [id(1)],
predecessors: [0],
successors: [edge(1, 3, 0)],
},
{
id: 2,
start: "0x1002",
operations: [id(2)],
predecessors: [0],
successors: [edge(2, 3, 0)],
},
{
id: 3,
start: "0x1003",
operations: [id(3)],
predecessors: [1, 2],
successors: [],
},
],
operations: [
op(0, "CPUI_CBRANCH", [input(0, "v0")]),
op(1, "CPUI_COPY", [input(0, "v4")], "v1"),
op(2, "CPUI_COPY", [input(0, "v4")], "v2"),
op(
3,
"CPUI_MULTIEQUAL",
[input(0, "v1", 1), input(1, "v2", 2)],
"v3",
),
],
values: [
val(0),
val(1, id(1)),
val(2, id(2)),
val(3, id(3)),
{
...val(4),
is_constant: true,
is_input: false,
space: "const",
offset: "0xffffffffffffffff",
},
],
coverage: {
scope: "native recovered graph",
machine_semantic_equivalence: "unknown",
indirect_branch_operations: [],
omitted_definition_ids: [],
issues: [],
},
},
},
],
};
}
test("compressed and ordinary graphs have identical parsed facts and malformed archives preserve code", () => {
const doc = diamond(),
raw = Buffer.from(JSON.stringify(doc.functions[0].atomic_facts));
const ordinary = parseNativeIR(JSON.stringify(doc), "0x1000");
doc.functions[0].atomic_facts = {
schema_version: 4,
encoding: "gzip-base64",
raw_bytes: raw.length,
data: gzipSync(raw).toString("base64"),
};
const encoded = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.deepEqual(encoded.coverage, ordinary.coverage);
assert.deepEqual(
encoded.select({ fields: ["inputs", "output", "definitions"] }),
ordinary.select({ fields: ["inputs", "output", "definitions"] }),
);
assert.deepEqual(encoded.blocks({}), ordinary.blocks({}));
assert.deepEqual(readNativeGraphEvidence(JSON.stringify(doc)), {
text: raw.toString(),
encoding: "gzip-base64",
});
for (const change of [
{ raw_bytes: 2 },
{ data: "broken" },
{ raw_bytes: 32 * 1024 * 1024 + 1 },
]) {
const bad = structuredClone(doc);
Object.assign(bad.functions[0].atomic_facts, change);
const model = parseNativeIR(JSON.stringify(bad), "0x1000");
assert.equal(model.code, "reconstructed C");
assert.equal(model.facets.recovered_ir.availability, "unavailable");
assert.throws(() => model.select({}));
}
for (const text of [
"{malformed logical JSON",
JSON.stringify({ schema_version: 4, data: "nested" }),
]) {
const bad = structuredClone(doc),
bytes = Buffer.from(text);
bad.functions[0].atomic_facts = {
schema_version: 4,
encoding: "gzip-base64",
raw_bytes: bytes.length,
data: gzipSync(bytes).toString("base64"),
};
const source = JSON.stringify(bad);
assert.equal(
parseNativeIR(source, "0x1000").facets.recovered_ir.availability,
"unavailable",
);
assert.equal(
readNativeGraphEvidence(source).text,
text,
"Malformed logical facts remain inspectable through owned decoded source",
);
}
assert.throws(
() => readNativeGraphEvidence('{"hex":"0102"}'),
(error) => error.code === "capability_unavailable",
);
});
test("native identity, implicit CFG and operand/phi bindings do not depend on display text", () => {
const model = parseNativeIR(JSON.stringify(diamond()), "0x1000");
assert.equal(model.operation("0x3:0").origin_address, "0x3");
assert.equal(Object.hasOwn(model.operation("0x3:0"), "address"), false);
assert.equal(model.block(1).origin_start, "0x1001");
const operand = model.edges({
kind: "operands",
seeds: ["0x3:0"],
input_indices: [1],
}).items[0];
assert.deepEqual(operand, {
operation: "0x3:0",
input_index: 1,
value_id: "v2",
definition: "0x2:0",
phi_predecessor: 2,
phi_predecessor_edge: "2:0",
});
assert.equal(Object.hasOwn(model.block(1), "start"), false);
assert.equal(model.select({ origin_addresses: ["0x3"] }).total, 1);
assert.deepEqual(model.findConstant("0xffffffffffffffff"), [
"0x1:0",
"0x2:0",
]);
assert.equal(model.edges({ kind: "outgoing", seeds: [1] }).items[0].to, 3);
assert.deepEqual(
model
.operation("0x3:0")
.definitions.map(({ input_index, phi_predecessor }) => [
input_index,
phi_predecessor,
]),
[
[0, 1],
[1, 2],
],
);
assert.equal(
model.values({ ids: ["v4"] }).items[0].offset,
"0xffffffffffffffff",
);
const batch = model.values({ ids: ["v4", "vdead"] });
assert.equal(batch.items.length, 1);
assert.deepEqual(batch.missing, ["vdead"]);
assert.throws(() => {
model.operation("0x3:0").inputs[0].value_id = "foreign";
});
assert.deepEqual(
model
.edges({ kind: "controllers", seeds: [1] })
.items.map((row) => [row.controller, row.edge.polarity]),
[[0, "false"]],
);
assert.equal(model.edges({ kind: "controllers", seeds: [3] }).total, 0);
});
test("malformed identities, dangling relationships, foreign entry and phi reorder fail at the producer boundary", () => {
for (const mutate of [
(facts) => {
facts.values.push(facts.values[0]);
},
(facts) => {
facts.operations[0].inputs[0].value_id = "vdead";
},
(facts) => {
facts.blocks[1].predecessors = [];
},
(facts) => {
facts.operations[3].inputs[0].phi_predecessor = 2;
},
(facts) => {
facts.values[1].definition = "0xf:0";
},
]) {
const doc = diamond();
mutate(doc.functions[0].atomic_facts);
const model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.facets.recovered_ir.availability, "unavailable");
assert.equal(model.code, "reconstructed C");
assert.throws(() => model.select({}));
}
assert.throws(
() => parseNativeIR(JSON.stringify(diamond()), "0x2000"),
/subject/u,
);
});
test("typed slices retain symbolic input frontiers and precise definition bindings", () => {
const model = parseNativeIR(JSON.stringify(diamond()), "0x1000");
const result = model.slice({
seeds: ["0x3:0", "0x0:0"],
max_operations: 10,
stop_at: [],
stop_kinds: [],
});
assert.equal(result.operations.length, 4);
assert.deepEqual(result.value_frontier, [
{ value_id: "v0", reason: "function-input" },
]);
assert.equal(result.coverage.runtime_reachability, "unknown");
});
test("native facet failure preserves C without fabricating an empty graph; source identity still gates both", () => {
const doc = diamond();
delete doc.functions[0].atomic_facts;
doc.functions[0].atomic_facts_error = "atomic operation capacity exceeded";
const model = parseNativeIR(JSON.stringify(doc), "0x1000", "/private/input");
assert.equal(model.code, "reconstructed C");
assert.equal(model.language, "c");
assert.equal(model.coverage.operations, null);
assert.equal(model.facets.code.availability, "available");
assert.equal(model.facets.recovered_ir.availability, "unavailable");
assert.throws(
() => model.select({}),
(error) =>
error.code === "capability_unavailable" &&
error.details.native_error === doc.functions[0].atomic_facts_error,
);
assert.throws(
() => parseNativeIR(JSON.stringify(doc), "0x1000", "/other/input"),
(error) => error.code === "subject_mismatch",
);
});
test("oversized or malformed native arrays fail before diagnostic expansion", () => {
const capacity = diamond();
capacity.functions[0].atomic_facts.operations = Array.from(
{ length: 32001 },
() => ({}),
);
assert.throws(
() => parseNativeIR(JSON.stringify(capacity), "0x1000").select({}),
/capacity/u,
);
const malformed = diamond();
malformed.functions[0].atomic_facts.operations = Array.from(
{ length: 12000 },
() => ({}),
);
assert.throws(
() => parseNativeIR(JSON.stringify(malformed), "0x1000").select({}),
(error) =>
error.code === "invalid_provider_output" &&
error.details.issues.length <= 8 &&
JSON.stringify(error.details).length < 2048,
);
});
test("control dependence handles multiple exits, loops, unreachable exits and bounded work", () => {
const blocks = [
{ id: 0, successors: [{ target: 1 }, { target: 2 }] },
{ id: 1, successors: [] },
{ id: 2, successors: [] },
{ id: 3, successors: [{ target: 3 }] },
];
const result = controlDependencies(blocks);
assert.deepEqual(
result.relations.map((r) => [r.controller, r.dependent]),
[
[0, 1],
[0, 2],
],
);
assert.deepEqual(result.coverage.exit_unreachable_blocks, [3]);
assert.throws(() => controlDependencies(blocks, { workLimit: 1 }), /budget/u);
const linear = Array.from({ length: 12000 }, (_, id) => ({
id,
successors: id < 11999 ? [{ target: id + 1 }] : [],
}));
assert.equal(controlDependencies(linear).relations.length, 0);
});
test("postdominator control dependence agrees with independently enumerated paths for every four-node DAG", () => {
const possible = [
[0, 1],
[0, 2],
[0, 3],
[1, 2],
[1, 3],
[2, 3],
];
for (let mask = 0; mask < 64; mask++) {
const blocks = Array.from({ length: 4 }, (_, id) => ({
id,
successors: possible
.filter(([from], i) => from === id && mask & (1 << i))
.map(([, target]) => ({ target })),
}));
const paths = (node) =>
blocks[node].successors.length
? blocks[node].successors.flatMap(({ target }) =>
paths(target).map((path) => [node, ...path]),
)
: [[node]];
const postdominators = blocks.map(({ id }) =>
paths(id).reduce(
(shared, path) =>
new Set([...shared].filter((node) => path.includes(node))),
new Set([0, 1, 2, 3]),
),
);
const expected = blocks.flatMap((block) =>
block.successors.flatMap((edge) =>
[...postdominators[edge.target]]
.filter((dependent) => !postdominators[block.id].has(dependent))
.map((dependent) => `${block.id}:${edge.target}:${dependent}`),
),
);
const actual = controlDependencies(blocks).relations.map(
(row) => `${row.controller}:${row.edge.target}:${row.dependent}`,
);
assert.deepEqual(actual.sort(), expected.sort(), `DAG mask ${mask}`);
}
});
@@ -0,0 +1,31 @@
// One native operand/operation schema shared by complete and projected facts.
import { z } from "zod";
import {
hex,
ordinal,
valueId,
operationId,
edgeId,
rowParser,
} from "./native-contract.mjs";
export const operation = z.strictObject({
id: operationId,
address: hex,
block: ordinal,
opcode: z.string(),
statement: z.string(),
is_dead: z.boolean(),
inputs: z.array(
z.strictObject({
input_index: ordinal,
value_id: valueId.nullable(),
phi_predecessor: ordinal.nullable(),
phi_predecessor_edge: edgeId.nullable(),
}),
),
output: valueId.nullable(),
});
export const parseOperation = rowParser(operation, {
inputs: (row) => operation.shape.inputs.element.parse(row),
});
@@ -0,0 +1,312 @@
// Shared immutable SSA operands and queries; no CFG completeness is inferred.
import { backwardSlice } from "../atomic-re-tools/kuna-ssa.mjs";
import { freezeFacts as freeze } from "./fact-reads.mjs";
import { requireFact } from "./native-contract.mjs";
import {
uniqueNative as unique,
nativePage as page,
readNativeIdentities as readIdentities,
} from "./native-reads.mjs";
export const DEFAULT_NATIVE_OPERATION_FIELDS = Object.freeze([
"kind",
"opcode",
"output",
"input_count",
"definition_count",
"constant_count",
]);
export const coverageIssues = (issues) => ({
issues: issues.slice(0, 8).map((message) => message.slice(0, 160)),
issue_count: issues.length,
issues_omitted: Math.max(0, issues.length - 8),
issue_messages_truncated: issues.slice(0, 8).some((s) => s.length > 160),
});
const index = (map, key, row) => {
if (!map.has(key)) map.set(key, []);
map.get(key).push(row);
};
const kind = (opcode) =>
({
CPUI_LOAD: "load",
CPUI_STORE: "store",
CPUI_MULTIEQUAL: "phi",
CPUI_CALL: "call",
CPUI_CALLIND: "call",
CPUI_BRANCH: "branch",
CPUI_CBRANCH: "branch",
CPUI_BRANCHIND: "switch",
CPUI_RETURN: "return",
})[opcode] ?? "value";
function validateOperands(operations, values) {
const referenced = new Set();
for (const op of operations.values()) {
requireFact(
op.id.split(":")[0] === op.address,
"Native operation origin/identity mismatch",
);
for (const [position, input] of op.inputs.entries()) {
requireFact(
input.input_index === position &&
(input.value_id === null || values.has(input.value_id)),
"Dangling or misindexed native operand",
);
requireFact(
(input.phi_predecessor === null) ===
(input.phi_predecessor_edge === null) &&
(input.phi_predecessor === null ||
(op.opcode === "CPUI_MULTIEQUAL" &&
input.phi_predecessor_edge.startsWith(
`${input.phi_predecessor}:`,
))),
"Native phi binding is inconsistent",
);
if (input.value_id !== null) referenced.add(input.value_id);
}
requireFact(
op.output === null || values.get(op.output)?.definition === op.id,
"Native output/definition mismatch",
);
if (op.output !== null) referenced.add(op.output);
}
requireFact(
referenced.size === values.size,
"Native operation set contains unrelated values",
);
}
function validateDefinitions(operations, values, omittedIDs) {
const omitted = new Set(omittedIDs),
missing = new Set();
requireFact(
omitted.size === omittedIDs.length,
"Duplicate native definition frontier",
);
for (const value of values.values())
if (value.definition !== null) {
if (!operations.has(value.definition)) missing.add(value.definition);
requireFact(
operations.get(value.definition)?.output === value.id ||
(!operations.has(value.definition) && omitted.has(value.definition)),
"Native value/definition mismatch",
);
}
requireFact(
missing.size === omitted.size,
"Native definition frontier contains unrelated identities",
);
}
export function indexOperationFacts(rows, valueRows, omittedIDs) {
const operations = unique(rows),
values = unique(valueRows);
validateOperands(operations, values);
validateDefinitions(operations, values, omittedIDs);
const users = new Map(),
constants = new Map();
const normalized = new Map();
for (const op of operations.values()) {
const definitions = [],
literals = [];
for (const input of op.inputs) {
const value = values.get(input.value_id);
if (value?.definition) {
const relation = freeze({
definition: value.definition,
use: op.id,
input_index: input.input_index,
value_id: value.id,
phi_predecessor: input.phi_predecessor,
phi_predecessor_edge: input.phi_predecessor_edge,
});
definitions.push(relation);
index(users, value.definition, relation);
}
if (value?.is_constant) {
literals.push({
value_id: value.id,
input_index: input.input_index,
value: value.offset,
printed_size_bytes: value.size_bytes,
is_annotation: value.is_annotation,
});
index(constants, value.offset, op.id);
}
}
const { address: origin_address, ...payload } = op;
normalized.set(
op.id,
freeze({
...payload,
origin_address,
kind: kind(op.opcode),
source_line: null,
definitions,
constants: literals,
input_count: op.inputs.length,
definition_count: definitions.length,
constant_count: literals.length,
}),
);
}
return { operations, values, users, constants, normalized };
}
export function createOperationViews(
indexes,
coverage,
frontierReasons = new Map(),
) {
const { values, users, constants, normalized } = indexes;
const model = {
coverage,
operation: (id) => normalized.get(id) ?? null,
findConstant: (value) => [...new Set(constants.get(value) ?? [])],
uses: (id) => users.get(id) ?? [],
select: (input = {}) =>
selectOperations(normalized, constants, {
...input,
fields: input.fields ?? DEFAULT_NATIVE_OPERATION_FIELDS,
}),
values: (input) => ({
...readIdentities(
values,
input,
(value) =>
input.is_input === undefined || value.is_input === input.is_input,
),
coverage,
}),
operationRelations: (input) =>
readOperationRelations(indexes, coverage, input),
slice: (input) => sliceValues(model, values, input, frontierReasons),
};
return Object.freeze(model);
}
function readOperationRelations(indexes, coverage, input) {
const { normalized, values, users } = indexes;
const seeds = [...new Set(input.seeds)];
const operands = input.kind === "operands";
const indices = input.input_indices && [...new Set(input.input_indices)];
const groups = seeds.map((seed) => {
const op = normalized.get(seed);
const rows = operands
? indices
? indices.flatMap((index) => op?.inputs[index] ?? [])
: (op?.inputs ?? [])
: input.kind === "definitions"
? (op?.definitions ?? [])
: (users.get(seed) ?? []);
return { seed, rows };
});
const offset = input.offset ?? 0,
limit = input.limit ?? 20;
let total = 0;
const items = [];
// Count groups without flattening every relation or copying unselected rows.
for (const { seed, rows } of groups) {
const start = Math.max(0, offset - total);
const end = Math.min(rows.length, offset + limit - total);
for (let i = start; i < end; i++) {
const row = rows[i];
items.push(
operands
? {
operation: seed,
...row,
definition: values.get(row.value_id)?.definition ?? null,
}
: row,
);
}
total += rows.length;
}
return {
items,
total,
next_offset: offset + limit < total ? offset + limit : null,
unretained_seeds: seeds.filter((id) => !normalized.has(id)),
relation_coverage: {
scope: operands
? "all selected native input slots of retained operations; null values remain explicit"
: input.kind === "uses"
? (coverage.uses_scope ??
"complete-for-retained-recovered-operations")
: "definitions of retained operands; missing defining bodies remain unknown",
...(operands
? {
slot_presence:
"out-of-range slots of retained seeds are absent; unretained seeds remain unknown",
order:
"distinct caller seed order, then distinct requested input-index order or native slot order",
}
: {}),
},
};
}
function selectOperations(normalized, constants, input = {}) {
const ids = input.ids && new Set(input.ids),
addresses = input.origin_addresses && new Set(input.origin_addresses),
blockIds = input.block_ids && new Set(input.block_ids);
const candidates = ids
? [...ids].flatMap((id) => normalized.get(id) ?? [])
: input.constant
? [...new Set(constants.get(input.constant) ?? [])].map((id) =>
normalized.get(id),
)
: normalized.values();
return page(
[...candidates].filter(
(op) =>
(!addresses || addresses.has(op.origin_address)) &&
(!blockIds || blockIds.has(op.block)) &&
(!input.constant ||
op.constants.some((literal) => literal.value === input.constant)) &&
(!input.kind || op.kind === input.kind),
),
input,
["id", "origin_address", "block", "is_dead"],
);
}
function sliceValues(model, values, input, frontierReasons) {
const result = backwardSlice(model, input.seeds, {
maxOperations: input.max_operations,
stopAt: input.stop_at,
stopKinds: input.stop_kinds,
});
const frontier = new Map();
for (const op of result.operations)
for (const input of op.inputs) {
const value = values.get(input.value_id);
if (!value)
frontier.set(`${op.id}:${input.input_index}`, {
operation: op.id,
input_index: input.input_index,
reason: "native-null-input",
});
else if (!value.definition && !value.is_constant && !value.is_annotation)
frontier.set(value.id, {
value_id: value.id,
reason: value.is_input ? "function-input" : "definition-unavailable",
});
}
return {
...result,
value_frontier: [...frontier.values()],
coverage: {
...result.coverage,
status:
result.coverage.status === "complete-for-printed-definitions"
? "complete-for-retained-value-definitions"
: result.coverage.status,
source_scope: model.coverage.scope,
missing_reasons: result.coverage.missing.map((definition) => ({
definition,
reason: frontierReasons.get(definition) ?? "not-retained",
})),
relation_authority: "native SSA value identities and operand indexes",
},
};
}
@@ -0,0 +1,68 @@
// Sparse facts retain their own resource, root and frontier contract.
import { z } from "zod";
import {
operationId,
value,
rowParser,
requireFact,
readNativeFacet,
} from "./native-contract.mjs";
import { operation, parseOperation } from "./native-operation-contract.mjs";
const frontier = z.strictObject({
definition: operationId,
reason: z.enum(["operation-budget", "outside-recovered-membership"]),
});
const coverage = z.strictObject({
scope: z.string(),
operation_budget: z.literal(4096),
frontier: z.array(frontier).max(12288),
issues: z.array(z.string()).max(4096),
machine_semantic_equivalence: z.literal("unknown"),
});
const schema = z.strictObject({
schema_version: z.literal(1),
roots: z.array(operationId).max(4096),
operations: z.array(operation).max(4096),
values: z.array(value).max(12288),
coverage,
});
const parseCoverage = rowParser(coverage, {
frontier: (row) => frontier.parse(row),
issues: (row) => z.string().parse(row),
});
const parseProjection = rowParser(
schema,
{
roots: (row) => operationId.parse(row),
operations: parseOperation,
values: (row) => value.parse(row),
},
{ coverage: parseCoverage },
);
export const readProjectionFacet = (fn) =>
readNativeFacet(fn, "atomic_projection", "operation projection", (raw) => {
requireFact(
raw?.schema_version === 1,
"Unsupported native projection schema",
);
for (const [rows, limit] of [
[raw.roots, 4096],
[raw.operations, 4096],
[raw.values, 12288],
[raw.coverage?.frontier, 12288],
[raw.coverage?.issues, 4096],
])
requireFact(
Array.isArray(rows) && rows.length <= limit,
"Native projection exceeds capacity or has invalid shape",
);
requireFact(
raw.operations.reduce(
(n, op) => n + (Array.isArray(op?.inputs) ? op.inputs.length : 0),
0,
) <= 16384,
"Native projection input capacity exceeded",
);
return parseProjection(raw);
});
@@ -0,0 +1,85 @@
import { requireFact } from "./native-contract.mjs";
import { freezeFacts as freeze } from "./fact-reads.mjs";
import {
indexOperationFacts,
createOperationViews,
coverageIssues,
} from "./native-operation-views.mjs";
export function indexProjection(facts) {
const reasons = new Map();
for (const row of facts.coverage.frontier) {
requireFact(!reasons.has(row.definition), "Duplicate projection frontier");
reasons.set(row.definition, row.reason);
requireFact(
row.reason !== "operation-budget" ||
facts.operations.length === facts.coverage.operation_budget,
"Projection claims an unexhausted operation budget",
);
}
const index = indexOperationFacts(facts.operations, facts.values, [
...reasons.keys(),
]);
const roots = new Set(facts.roots);
requireFact(roots.size === facts.roots.length, "Duplicate projection root");
const callIDs = [...index.operations.values()]
.filter((op) => ["CPUI_CALL", "CPUI_CALLIND"].includes(op.opcode))
.map((op) => op.id);
requireFact(
roots.size === callIDs.length && callIDs.every((id) => roots.has(id)),
"Projection roots differ from retained call operations",
);
const reached = new Set(roots),
queue = [...roots];
for (const id of queue)
for (const edge of index.normalized.get(id).definitions)
if (
index.operations.has(edge.definition) &&
!reached.has(edge.definition)
) {
reached.add(edge.definition);
queue.push(edge.definition);
}
requireFact(
reached.size === index.operations.size,
"Projection contains operations unrelated to its roots",
);
return { ...index, reasons };
}
export function createProjectionViews(facts, code, indexes, unavailableGraph) {
freeze(facts);
const coverage = freeze({
scope: facts.coverage.scope,
operations: indexes.operations.size,
values: indexes.values.size,
blocks: null,
root_count: facts.roots.length,
operation_budget: facts.coverage.operation_budget,
definition_frontier_count: facts.coverage.frontier.length,
...coverageIssues(facts.coverage.issues),
control_flow: "not-collected",
definition_scope:
"retained operand definitions; unexpanded IDs stay unknown",
uses_scope: "selected-operations-only; no complete-use claim",
value_scope:
"operands and outputs of the retained call-rooted operation set",
machine_semantic_equivalence: "unknown",
memory_alias_dependencies: "unknown",
runtime_successors: "unknown",
representation:
"optimized recovered IR; origins and block IDs do not prove machine extents or branch targets",
});
const core = createOperationViews(indexes, coverage, indexes.reasons);
return Object.freeze({
...core,
code,
block: unavailableGraph,
incoming: unavailableGraph,
blocks: unavailableGraph,
edges: (input) =>
["definitions", "uses", "operands"].includes(input.kind)
? core.operationRelations(input)
: unavailableGraph(),
});
}
@@ -0,0 +1,318 @@
import test from "node:test";
import assert from "node:assert/strict";
import { parseNativeIR } from "./native-ir.mjs";
const value = (id, definition, extra = {}) => ({
id,
definition,
space: "register",
offset: "0x30",
size_bytes: 8,
is_constant: false,
is_input: false,
is_annotation: false,
type_name: "",
...extra,
});
const input = (input_index, value_id, predecessor = null, edge = null) => ({
input_index,
value_id,
phi_predecessor: predecessor,
phi_predecessor_edge: edge,
});
const op = (id, opcode, inputs, output = null) => ({
id,
address: id.split(":")[0],
block: 9,
opcode,
inputs,
output,
statement: "unparseable display",
is_dead: false,
});
function fixture() {
const operations = [
op("0x1100:1", "CPUI_CALL", [
input(0, "v0"),
input(1, "v3"),
input(2, "v7"),
input(3, "v9"),
]),
op(
"0x1080:2",
"CPUI_MULTIEQUAL",
[input(0, "v1", 7, "7:1"), input(1, "v2", 8, "8:0")],
"v3",
),
op("0x1050:3", "CPUI_PTRSUB", [input(0, "v4"), input(1, "v5")], "v1"),
op("0x1060:4", "CPUI_PTRSUB", [input(0, "v4"), input(1, "v6")], "v2"),
];
const values = [
value("v0", null, {
space: "fspec",
offset: "0x10000",
is_annotation: true,
}),
value("v1", "0x1050:3"),
value("v2", "0x1060:4"),
value("v3", "0x1080:2"),
value("v4", null, { space: "const", offset: "0x0", is_constant: true }),
value("v5", null, { space: "const", offset: "0x2100", is_constant: true }),
value("v6", null, { space: "const", offset: "0x2200", is_constant: true }),
value("v7", null, { is_input: true }),
value("v9", "0x9000:0"),
];
return {
binary: "/input",
count: 1,
language: "c",
error: null,
functions: [
{
address_hex: "0x1000",
code: "retained C",
error: null,
atomic_facts_error: "atomic operation capacity exceeded",
atomic_projection: {
schema_version: 1,
roots: ["0x1100:1"],
operations,
values,
coverage: {
scope: "call-rooted bounded backward SSA definitions",
operation_budget: 4096,
frontier: [
{
definition: "0x9000:0",
reason: "outside-recovered-membership",
},
],
issues: [],
machine_semantic_equivalence: "unknown",
},
},
atomic_calls: {
schema_version: 1,
items: [
{
id: "0x1100:1",
origin_address: "0x1100",
block: 9,
opcode: "CPUI_CALL",
is_dead: false,
target_value_id: "v0",
spec_available: false,
target: null,
target_name: null,
arguments: ["v3", "v7", "v9"].map((value_id, i) => ({
input_index: i + 1,
value_id,
storage: null,
storage_authority: null,
})),
output: null,
prototype: null,
stack_offset: null,
},
],
values: structuredClone(
values.filter((v) => ["v0", "v3", "v7", "v9"].includes(v.id)),
),
coverage: {
scope: "all recovered block-member CALL/CALLIND operations",
blocks_examined: 10,
operations_examined: 100,
value_scope: "call inputs including target and outputs only",
machine_call_equivalence: "unknown",
},
},
},
],
};
}
test("operand relations preserve every slot, page across seeds and select exact caller indices", () => {
const source = fixture();
source.functions[0].atomic_projection.operations[1].inputs.push(
input(2, "v5", 2, "2:0"),
input(3, "v7", 3, "3:0"),
input(4, null, 4, "4:0"),
);
const model = parseNativeIR(JSON.stringify(source), "0x1000");
const read = (extra = {}) =>
model.edges({
kind: "operands",
seeds: ["0x1080:2"],
offset: 0,
limit: 2,
...extra,
});
const first = read();
assert.equal(first.total, 5);
assert.equal(first.next_offset, 2);
assert.deepEqual(
first.items.map((r) => r.input_index),
[0, 1],
);
assert.deepEqual(
read({ offset: 2, limit: 10 }).items.map((r) => r.definition),
[null, null, null],
);
const selected = read({ input_indices: [4, 2, 4, 500], limit: 10 });
assert.equal(selected.total, 2);
assert.equal(selected.items[0].value_id, null);
assert.deepEqual(
selected.items.map((r) => [r.input_index, r.phi_predecessor_edge]),
[
[4, "4:0"],
[2, "2:0"],
],
);
assert.equal(read({ input_indices: [] }).total, 0);
const batch = read({
seeds: ["0x1080:2", "0x1050:3", "0x1050:3", "0x9000:0"],
offset: 4,
limit: 2,
});
assert.equal(batch.total, 7);
assert.equal(batch.next_offset, 6);
assert.deepEqual(
batch.items.map((r) => [r.operation, r.input_index]),
[
["0x1080:2", 4],
["0x1050:3", 0],
],
);
assert.deepEqual(batch.unretained_seeds, ["0x9000:0"]);
assert.equal(
read({ seeds: ["0x1100:1"], input_indices: [3] }).items[0].definition,
"0x9000:0",
);
const summary = model.select({ ids: ["0x1080:2"] }).items[0];
assert.equal(summary.input_count, 5);
assert.equal(summary.definition_count, 2);
assert.equal(summary.constant_count, 1);
assert.equal(Object.hasOwn(summary, "inputs"), false);
const repeatedValues = model.values({ ids: ["v3", "v3", "vbad", "vbad"] });
assert.deepEqual(
repeatedValues.items.map((v) => v.id),
["v3"],
);
assert.deepEqual(repeatedValues.missing, ["vbad"]);
});
test("partial definition facts compose through phi operands without implying a CFG or full uses", () => {
const model = parseNativeIR(JSON.stringify(fixture()), "0x1000");
assert.equal(model.facets.recovered_ir.availability, "unavailable");
assert.equal(
model.facets.operations.scope,
"bounded-call-rooted-definitions",
);
assert.equal(model.facets.values.scope, "call-rooted-definitions");
assert.equal(model.coverage.definition_frontier_count, 1);
assert.equal(JSON.stringify(model.coverage).includes("0x9000:0"), false);
const phi = model.select({
ids: ["0x1080:2"],
fields: ["inputs", "output"],
limit: 1,
}).items[0];
assert.equal(phi.output, "v3");
assert.equal(phi.inputs[0].phi_predecessor_edge, "7:1");
assert.throws(() => {
phi.inputs[0].value_id = "vdead";
}, TypeError);
const sliced = model.slice({
seeds: ["0x1100:1"],
max_operations: 10,
stop_kinds: [],
});
assert.equal(sliced.operations.length, 4);
assert.equal(sliced.coverage.status, "partial");
assert.deepEqual(sliced.coverage.missing_reasons, [
{ definition: "0x9000:0", reason: "outside-recovered-membership" },
]);
assert.deepEqual(sliced.value_frontier, [
{ value_id: "v7", reason: "function-input" },
]);
assert.deepEqual(model.findConstant("0x2100"), ["0x1050:3"]);
const uses = model.edges({
kind: "uses",
seeds: ["0x9000:0"],
offset: 0,
limit: 10,
});
assert.equal(uses.items.length, 1);
assert.deepEqual(uses.unretained_seeds, ["0x9000:0"]);
assert.match(uses.relation_coverage.scope, /selected-operations-only/);
for (const read of [
() => model.blocks({}),
() => model.edges({ kind: "controllers", seeds: [9] }),
])
assert.throws(read, (e) => e.code === "capability_unavailable");
});
test("bad or contradictory projections preserve independently valid call facts and code", () => {
const changes = [
(p) => {
p.operations[1].inputs[0].value_id = "vdead";
},
(p) => {
p.coverage.frontier = [];
},
(p) => {
p.coverage.frontier[0].reason = "operation-budget";
},
(p) => {
p.roots = ["0x1050:3"];
},
(p) => {
p.roots.push("0x1100:1");
},
(p) => {
p.operations.push(op("0x1070:5", "CPUI_COPY", [input(0, "v7")]));
},
(p) => {
p.values[3].offset = "0xbad";
},
(p) => {
p.coverage.frontier = Array(12289).fill(null);
},
];
for (const change of changes) {
const doc = fixture();
change(doc.functions[0].atomic_projection);
const model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.code, "retained C");
assert.equal(model.facets.operations.availability, "unavailable");
assert.equal(model.facets.calls.availability, "available");
assert.equal(model.facets.values.scope, "call-boundary-only");
assert.ok(JSON.stringify(model.facets.operations.error).length < 4096);
}
});
test("bad call facets do not erase valid projection definitions", () => {
const doc = fixture();
doc.functions[0].atomic_calls.items[0].arguments[0].value_id = "vdead";
const model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.facets.calls.availability, "unavailable");
assert.equal(model.facets.operations.availability, "available");
assert.equal(model.values({ ids: ["v5"] }).items[0].offset, "0x2100");
});
test("compact coverage bounds issue text while slices retain the actual issue count", () => {
const doc = fixture();
doc.functions[0].atomic_projection.coverage.issues = Array(10).fill(
"x".repeat(10000),
);
const model = parseNativeIR(JSON.stringify(doc), "0x1000");
assert.equal(model.coverage.issue_count, 10);
assert.equal(model.coverage.issues.length, 8);
assert.equal(model.coverage.issues_omitted, 2);
assert.equal(model.coverage.issue_messages_truncated, true);
assert.ok(JSON.stringify(model.coverage).length < 4096);
assert.equal(
model.slice({ seeds: ["0x1100:1"], max_operations: 10 }).coverage
.source_issues,
10,
);
});
@@ -0,0 +1,28 @@
import { pageFacts } from "./fact-reads.mjs";
import { requireFact } from "./native-contract.mjs";
export const nativePage = (
rows,
input = {},
identity = ["id", "origin_address", "block"],
) => pageFacts(rows, { offset: 0, limit: 20, ...input }, identity);
export const uniqueNative = (rows) => {
const index = new Map();
for (const row of rows) {
requireFact(!index.has(row.id), `Duplicate native identity ${row.id}`);
index.set(row.id, row);
}
return index;
};
export const readNativeIdentities = (index, input, filter = () => true) => ({
...nativePage(
(input.ids
? [...new Set(input.ids)].flatMap((id) => index.get(id) ?? [])
: [...index.values()]
).filter(filter),
input,
),
missing: input.ids
? [...new Set(input.ids)].filter((id) => !index.has(id))
: [],
});
@@ -0,0 +1,50 @@
const read = async (name,input) => {
const result=await rea[name](input);
if(!result.ok)throw new Error(JSON.stringify(result.error));
return result.data;
};
const complete=(page)=>{
if(page.next_offset!==null)throw new Error("Continue the selected fact page before drawing a complete conclusion");
return page;
};
const found=await read("functions",{query:"sqlite3VdbeExec",exact:true});
complete(found);
const formats=[];
for(const query of ["cannot open savepoint - SQL statements in progress","cannot release savepoint - SQL statements in progress"]) {
const strings=await read("strings",{query});
complete(strings);
const addresses=[...new Set(strings.items.filter((s)=>s.value===query).map((s)=>s.address))];
if(addresses.length!==1)throw new Error("Ambiguous format: "+JSON.stringify(strings));
const references=await read("references",{addresses,direction:"incoming",fields:["fromFunction","instruction"]});
for(const result of references.results) {
if(!result.ok)throw new Error(JSON.stringify(result.error));
complete(result);
}
formats.push({query,address:addresses[0],string_evidence_id:strings.source.evidence_id,references});
}
const owners=formats.map((f)=>new Set(f.references.results.flatMap((r)=>r.ok?r.items.map((ref)=>ref.fromFunction?.address).filter(Boolean):[])));
const candidates=found.items.filter((f)=>owners.every((set)=>set.has(f.address)));
if(candidates.length!==1)throw new Error("Unresolved format-owning function: "+JSON.stringify({found,formats}));
const capture=await read("snapshot",{entry:candidates[0].address});
const snapshot=capture.snapshot;
for(const format of formats) {
const producers=await read("operations",{snapshot,constant:format.address,fields:["opcode","output"]});
complete(producers);
const setup=producers.items.filter((op)=>op.opcode==="CPUI_PTRSUB");
if(setup.length!==1)throw new Error("Ambiguous format producer: "+JSON.stringify(producers));
const uses=await read("relations",{snapshot,kind:"uses",seeds:[setup[0].id]});
complete(uses);
Object.assign(format,{producer:setup[0],uses});
}
const controls=await read("relations",{snapshot,kind:"controllers",seeds:formats.map((f)=>f.producer.block)});
complete(controls);
const branchIDs=[...new Set(controls.items.flatMap((relation)=>relation.edge.condition_value?[relation.edge.condition_value]:[]))];
const conditions=await read("values",{snapshot,ids:branchIDs});
complete(conditions);
const definitions=await read("operations",{snapshot,ids:conditions.items.map((v)=>v.definition).filter(Boolean),fields:["opcode","inputs","output","constants"]});
complete(definitions);
const values=await read("values",{snapshot,ids:[...new Set(definitions.items.flatMap((op)=>op.inputs.map((i)=>i.value_id).filter(Boolean)))]});
complete(values);
const evidence=await read("evidence",{evidence_id:capture.source.evidence_id,component:"recovered_ir",limit:128});
if((await read("snapshot",{entry:candidates[0].address})).snapshot!==snapshot)throw new Error("Capture was not reused");
return {capture,formats,controls,conditions,definitions,values,evidence};
@@ -0,0 +1,104 @@
// Native switch framing/rows are admitted independently of graph and calls.
import { z } from "zod";
import {
hex,
ordinal,
edgeId,
operationId,
value,
rowParser,
readNativeFacet,
requireFact,
} from "./native-contract.mjs";
const edge = z.strictObject({ id: edgeId, index: ordinal, target: ordinal });
const location = z.strictObject({ space: z.string(), offset: hex });
const row = z.strictObject({
index: ordinal,
label: z
.strictObject({ status: z.enum(["value", "no-label"]), raw_bits: hex })
.nullable(),
recovered_target: location.nullable(),
edges: z.array(edge),
});
const table = z.strictObject({
partial: z.boolean(),
labelled: z.boolean(),
consumed_mask: hex,
signed_render_hint: z.boolean(),
default_raw_index: z.number().int().min(-1).max(0x7fffffff),
default_edge: edge.nullable(),
default_is_folded: z.boolean(),
rows: z.array(row),
});
const item = z.strictObject({
id: operationId,
origin_address: hex,
block: ordinal,
selector: value.nullable(),
table: table.nullable(),
});
const facet = z.strictObject({
schema_version: z.literal(1),
items: z.array(item).max(12000),
coverage: z.strictObject({
scope: z.string(),
blocks_examined: ordinal.min(1),
operations_examined: z
.number()
.int()
.nonnegative()
.max(Number.MAX_SAFE_INTEGER),
machine_semantic_equivalence: z.literal("unknown"),
}),
});
const parseTable = rowParser(table, {
rows: rowParser(row, { edges: (v) => edge.parse(v) }),
});
const parseItem = rowParser(
item,
{},
{ table: (v) => (v === null ? null : parseTable(v)) },
);
const parseFacet = rowParser(facet, { items: parseItem });
export const readSwitchFacet = (fn) =>
readNativeFacet(fn, "atomic_switches", "recovered switches", (raw) => {
requireFact(
raw?.schema_version === 1,
"Unsupported native switch facet schema; expected 1",
);
requireFact(
Array.isArray(raw.items) && raw.items.length <= 12000,
"Native switch count exceeds capacity or has invalid shape",
);
let rows = 0,
mappings = 0,
text = 0;
for (const item of raw.items) {
for (const s of [item?.selector?.space, item?.selector?.type_name])
if (typeof s === "string") text += Buffer.byteLength(s);
if (item?.table === null) continue;
requireFact(
Array.isArray(item?.table?.rows),
"Native switch rows must be an array",
);
rows += item.table.rows.length;
requireFact(rows <= 100000, "Native switch rows exceed capacity");
for (const row of item.table.rows) {
requireFact(
Array.isArray(row?.edges),
"Native case edges must be an array",
);
mappings += row.edges.length;
requireFact(
mappings <= 100000,
"Native switch mappings exceed capacity",
);
if (typeof row?.recovered_target?.space === "string")
text += Buffer.byteLength(row.recovered_target.space);
}
}
requireFact(text <= 2 * 1024 * 1024, "Native switch text exceeds capacity");
return parseFacet(raw);
});
@@ -0,0 +1,320 @@
// Native table ownership is separate from optimized CFG and machine transfers.
import { requireFact, requireSameNativeValue } from "./native-contract.mjs";
import { uniqueNative, readNativeIdentities } from "./native-reads.mjs";
import { freezeFacts } from "./fact-reads.mjs";
function validateEdge(edge, block) {
requireFact(
edge.id === `${block}:${edge.index}`,
"Native switch edge identity mismatch",
);
}
function indexTable(sw) {
const { rows, ...metadata } = sw.table;
requireFact(
(metadata.default_raw_index === -1) === (metadata.default_edge === null),
"Native switch default availability mismatch",
);
if (metadata.default_edge) {
validateEdge(metadata.default_edge, sw.block);
requireFact(
metadata.default_edge.index === metadata.default_raw_index,
"Native switch default index mismatch",
);
}
const counts = {
case_count: rows.length,
mapping_count: 0,
known_label_count: 0,
missing_label_count: 0,
no_label_count: 0,
unmapped_row_count: 0,
unknown_target_count: 0,
};
const byLabel = new Map();
for (const [i, row] of rows.entries()) {
requireFact(row.index === i, "Native case row ordinal mismatch");
const edges = new Set();
for (const edge of row.edges) {
validateEdge(edge, sw.block);
requireFact(
!edges.has(edge.id),
"Duplicate native case/out-edge binding",
);
edges.add(edge.id);
}
if (row.label === null) counts.missing_label_count++;
else if (row.label.status === "no-label") counts.no_label_count++;
else {
counts.known_label_count++;
if (!byLabel.has(row.label.raw_bits)) byLabel.set(row.label.raw_bits, []);
byLabel.get(row.label.raw_bits).push(row);
}
counts.mapping_count += row.edges.length;
if (row.edges.length === 0) counts.unmapped_row_count++;
if (row.recovered_target === null) counts.unknown_target_count++;
}
const hasRawLabels = counts.missing_label_count < rows.length;
requireFact(
metadata.labelled === hasRawLabels,
"Native switch label availability mismatch",
);
const prefix = caseOffsets(rows);
const table = { ...metadata, ...counts, relation_count: prefix.at(-1) };
// Retain source rows and label references, not a second full row projection.
// A numeric row lookup is O(1); selected label reads use this index.
return { table, lookup: { rows, prefix, byLabel, labelOffsets: new Map() } };
}
export function indexSwitchFacet(facts) {
freezeFacts(facts);
const switches = uniqueNative(facts.items),
summaries = new Map(),
cases = new Map();
requireFact(
switches.size <= facts.coverage.operations_examined,
"Native switch count exceeds examined operations",
);
for (const sw of switches.values()) {
requireFact(
sw.id.split(":")[0] === sw.origin_address,
"Native switch origin/identity mismatch",
);
let table = null;
if (sw.table) {
const indexed = indexTable(sw);
table = indexed.table;
cases.set(sw.id, indexed.lookup);
}
summaries.set(
sw.id,
freezeFacts({
id: sw.id,
origin_address: sw.origin_address,
block: sw.block,
selector: sw.selector,
table,
}),
);
}
return { switches, summaries, cases };
}
export function validateSwitchGraph(index, graph) {
for (const sw of index.switches.values()) {
const op = graph.operations.get(sw.id),
block = graph.blocks.get(sw.block);
requireFact(
op?.opcode === "CPUI_BRANCHIND" &&
op.block === sw.block &&
op.address === sw.origin_address &&
block,
"Native switch/graph ownership mismatch",
);
requireFact(
(op.inputs[0]?.value_id ?? null) === (sw.selector?.id ?? null),
"Native switch/graph selector mismatch",
);
if (sw.selector)
requireSameNativeValue(
sw.selector,
graph.values.get(sw.selector.id),
"Native switch/graph selector value mismatch",
);
const checkEdge = (edge) =>
requireFact(
block.successors[edge.index]?.id === edge.id &&
block.successors[edge.index]?.target === edge.target,
"Native switch/graph edge mapping mismatch",
);
if (sw.table?.default_edge) checkEdge(sw.table.default_edge);
for (const row of sw.table?.rows ?? [])
for (const edge of row.edges) checkEdge(edge);
}
for (const op of graph.operations.values())
requireFact(
op.opcode !== "CPUI_BRANCHIND" || index.switches.has(op.id),
"Native indirect branch omitted from switch coverage",
);
}
export function validateSwitchValues(index, values) {
if (!values) return;
for (const sw of index.switches.values())
if (sw.selector && values.has(sw.selector.id))
requireSameNativeValue(
sw.selector,
values.get(sw.selector.id),
"Native switch/shared value mismatch",
);
}
function caseOffsets(rows) {
const prefix = new Uint32Array(rows.length + 1);
for (let i = 0; i < rows.length; i++)
prefix[i + 1] = prefix[i] + Math.max(1, rows[i].edges.length);
return prefix;
}
function selectCaseRows(lookup, labels, indices) {
if (!labels && !indices) return lookup;
if (!indices && labels?.size === 1) {
const label = labels.values().next().value;
const rows = lookup.byLabel.get(label) ?? [];
if (!lookup.labelOffsets.has(label) && lookup.byLabel.has(label))
lookup.labelOffsets.set(label, caseOffsets(rows));
return {
rows,
prefix: lookup.labelOffsets.get(label) ?? new Uint32Array(1),
};
}
const rows = indices
? indices
.flatMap((i) => lookup.rows[i] ?? [])
.filter(
(r) =>
!labels ||
(r.label?.status === "value" && labels.has(r.label.raw_bits)),
)
: [...labels]
.flatMap((label) => lookup.byLabel.get(label) ?? [])
.sort((a, b) => a.index - b.index);
return { rows, prefix: caseOffsets(rows) };
}
function firstCaseAt(prefix, offset) {
let low = 0,
high = prefix.length - 1;
while (low < high) {
const mid = (low + high) >>> 1;
if (prefix[mid + 1] <= offset) low = mid + 1;
else high = mid;
}
return low;
}
function casePage(sw, selected, start, end) {
const { id, block } = sw;
const items = [];
for (
let i = firstCaseAt(selected.prefix, start);
i < selected.rows.length && selected.prefix[i] < end;
i++
) {
const r = selected.rows[i],
case_id = `${id}:case:${r.index}`;
const begin = Math.max(0, start - selected.prefix[i]);
const finish = Math.min(
Math.max(1, r.edges.length),
end - selected.prefix[i],
);
for (let j = begin; j < finish; j++) {
const edge = r.edges[j] ?? null;
items.push({
id: `${case_id}:edge:${edge?.index ?? "unknown"}`,
case_id,
switch: id,
block,
case_index: r.index,
label: r.label,
recovered_target: r.recovered_target,
edge,
});
}
}
return items;
}
function selectedCaseCoverage(seeds, summaries) {
return seeds.flatMap((id) => {
const sw = summaries.get(id),
t = sw?.table;
return sw
? [
{
id,
table_available: t !== null,
partial: t?.partial ?? null,
case_count: t?.case_count ?? null,
relation_count: t?.relation_count ?? null,
mapping_count: t?.mapping_count ?? null,
known_label_count: t?.known_label_count ?? null,
missing_label_count: t?.missing_label_count ?? null,
no_label_count: t?.no_label_count ?? null,
unmapped_row_count: t?.unmapped_row_count ?? null,
},
]
: [];
});
}
function readCases(index, coverage, input) {
const seeds = [...new Set(input.seeds)],
labels = input.case_values && new Set(input.case_values),
indices =
input.case_indices &&
[...new Set(input.case_indices)].sort((a, b) => a - b),
offset = input.offset ?? 0,
limit = input.limit ?? 20,
items = [];
let total = 0;
for (const id of seeds) {
const lookup = index.cases.get(id);
if (!lookup) continue;
const selected = selectCaseRows(lookup, labels, indices),
count = selected.prefix.at(-1);
const start = Math.max(0, offset - total),
end = Math.min(count, offset + limit - total);
if (start < end)
items.push(...casePage(index.summaries.get(id), selected, start, end));
total += count;
}
return {
items,
total,
next_offset: offset + limit < total ? offset + limit : null,
unretained_seeds: seeds.filter((id) => !index.switches.has(id)),
coverage,
relation_coverage: {
scope:
"one relation per captured case/out-edge; an unmapped case has one explicit unknown-edge row; not all feasible runtime cases",
switches: selectedCaseCoverage(seeds, index.summaries),
},
};
}
export function createSwitchViews(facts, index) {
const coverage = freezeFacts({
...facts.coverage,
case_identity:
"snapshot + switch operation + native address-table row index",
relation_identity:
"case_id + native out-edge index, or one explicit unknown-edge row",
label_authority:
"native recovered label and explicit reserved-no-label status; row/edge indexes are not label values",
selector_authority:
"native current BRANCHIND input; may be a normalized selector rather than a machine jump address",
target_authority:
"native address-table target and separate recovered out-edge mapping; optimized block origins may differ",
default_authority:
"native model default out-edge, not necessarily a machine out-of-range guard destination",
signedness: "signed_render_hint is only a native rendering hint",
runtime_feasibility: "unknown",
memory_alias_dependencies: "unknown",
});
const tables = [...index.summaries.values()].map((s) => s.table);
const summary = freezeFacts({
count: index.switches.size,
table_count: tables.filter(Boolean).length,
partial_table_count: tables.filter((t) => t?.partial).length,
case_count: tables.reduce((n, t) => n + (t?.case_count ?? 0), 0),
relation_count: tables.reduce((n, t) => n + (t?.relation_count ?? 0), 0),
coverage,
});
return {
summary,
switches: (input) => ({
...readNativeIdentities(
index.summaries,
input,
(sw) =>
!input.origin_addresses ||
input.origin_addresses.includes(sw.origin_address),
),
coverage,
}),
cases: (input) => readCases(index, coverage, input),
};
}
@@ -0,0 +1,351 @@
import assert from "node:assert/strict";
import test from "node:test";
import { parseNativeIR } from "./native-ir.mjs";
const switchID = "0x1000:1";
const edge = (index, target) => ({ id: `0:${index}`, index, target });
function tableFixture() {
return {
partial: true,
labelled: true,
consumed_mask: "0xff",
signed_render_hint: true,
default_raw_index: 1,
default_edge: edge(1, 2),
default_is_folded: true,
rows: [
{
index: 0,
label: { status: "value", raw_bits: "0x42" },
recovered_target: { space: "ram", offset: "0xabc" },
edges: [edge(0, 1)],
},
{
index: 1,
label: { status: "value", raw_bits: "0x7" },
recovered_target: { space: "ram", offset: "0xdef" },
edges: [edge(0, 1)],
},
{
index: 2,
label: { status: "no-label", raw_bits: "0xbad1abe1bad1abe1" },
recovered_target: { space: "ram", offset: "0xf00" },
edges: [edge(1, 2)],
},
{ index: 3, label: null, recovered_target: null, edges: [] },
],
};
}
function graphFixture(selector) {
const operation = (address, block, opcode, inputs = []) => ({
id: `${address}:1`,
address,
block,
opcode,
inputs,
output: null,
statement: "uninterpreted display",
is_dead: false,
});
return {
schema_version: 3,
blocks: [
{
id: 0,
start: "0x1000",
operations: [switchID],
predecessors: [],
successors: [edge(0, 1), edge(1, 2)].map((e) => ({
...e,
polarity: null,
condition_value: null,
condition_test: null,
})),
},
{
id: 1,
start: "0x1010",
operations: ["0x1010:1"],
predecessors: [0],
successors: [],
},
{
id: 2,
start: "0x1020",
operations: ["0x1020:1"],
predecessors: [0],
successors: [],
},
],
operations: [
operation("0x1000", 0, "CPUI_BRANCHIND", [
{
input_index: 0,
value_id: "v1",
phi_predecessor: null,
phi_predecessor_edge: null,
},
]),
operation("0x1010", 1, "CPUI_RETURN"),
operation("0x1020", 2, "CPUI_RETURN"),
],
values: [{ ...selector }],
coverage: {
scope: "fixture recovered graph",
machine_semantic_equivalence: "unknown",
indirect_branch_operations: [switchID],
omitted_definition_ids: [],
issues: [],
},
};
}
function fixture() {
const selector = {
id: "v1",
space: "register",
offset: "0x10",
size_bytes: 8,
is_constant: false,
is_input: true,
is_annotation: false,
definition: null,
type_name: "uint64",
};
return {
binary: "/input",
count: 1,
language: "c",
error: null,
functions: [
{
address_hex: "0x1000",
code: "retained C",
error: null,
atomic_facts: graphFixture(selector),
atomic_switches: {
schema_version: 1,
items: [
{
id: switchID,
origin_address: "0x1000",
block: 0,
selector,
table: tableFixture(),
},
],
coverage: {
scope: "captured native tables",
blocks_examined: 3,
operations_examined: 3,
machine_semantic_equivalence: "unknown",
},
},
},
],
};
}
const load = (doc) => parseNativeIR(JSON.stringify(doc), "0x1000");
const cases = (model, extra = {}) =>
model.edges({ kind: "cases", seeds: [switchID], ...extra });
test("case-to-edge paging bounds a single case with several mappings and keeps a stable case identity", () => {
const doc = fixture();
doc.functions[0].atomic_switches.items[0].table.rows[0].edges.push(
edge(1, 2),
);
const model = load(doc),
first = cases(model, { case_values: ["0x42"], limit: 1 }),
second = cases(model, { case_values: ["0x42"], offset: 1, limit: 1 });
assert.equal(model.switches({}).items[0].table.case_count, 4);
assert.equal(model.switches({}).items[0].table.relation_count, 5);
assert.equal(first.total, 2);
assert.equal(first.items.length, 1);
assert.equal(first.next_offset, 1);
assert.equal(first.items[0].case_id, second.items[0].case_id);
assert.notEqual(first.items[0].id, second.items[0].id);
assert.equal(first.items[0].edge.index, 0);
assert.equal(second.items[0].edge.index, 1);
assert.equal(second.next_offset, null);
});
test("a late two-edge page does not materialize thousands of unrequested mappings", () => {
const doc = fixture(),
fn = doc.functions[0];
delete fn.atomic_facts;
const table = fn.atomic_switches.items[0].table;
table.rows = table.rows.slice(0, 1);
table.rows[0].edges = Array.from({ length: 4096 }, (_, i) => edge(i, i + 1));
fn.atomic_switches.coverage.blocks_examined = 4097;
fn.atomic_switches.coverage.operations_examined = 4097;
const model = load(doc);
const page = cases(model, { case_values: ["0x42"], offset: 4090, limit: 2 });
assert.equal(page.total, 4096);
assert.equal(page.next_offset, 4092);
assert.deepEqual(
page.items.map((r) => r.edge.index),
[4090, 4091],
);
assert.equal(page.items[0].case_id, page.items[1].case_id);
assert.ok(Buffer.byteLength(JSON.stringify(page.items)) < 1024);
});
test("case ordinals, labels, original targets and optimized edges remain distinct and paged", () => {
const model = load(fixture());
assert.equal(model.facets.switches.availability, "available");
const sw = model.switches({ fields: ["table"] }).items[0];
assert.equal(sw.selector, undefined);
assert.equal(sw.table.case_count, 4);
assert.equal(sw.table.known_label_count, 2);
assert.equal(sw.table.missing_label_count, 1);
assert.equal(sw.table.no_label_count, 1);
assert.equal(sw.table.unmapped_row_count, 1);
assert.equal(sw.table.default_edge.id, "0:1");
const page = cases(model, {
seeds: [switchID, switchID, "unknown"],
case_values: ["0x7", "0x42", "0xbad1abe1bad1abe1"],
limit: 1,
});
assert.equal(page.total, 2);
assert.equal(page.next_offset, 1);
assert.deepEqual(page.unretained_seeds, ["unknown"]);
assert.equal(page.items[0].case_index, 0);
assert.equal(page.items[0].label.raw_bits, "0x42");
assert.equal(page.items[0].recovered_target.offset, "0xabc");
assert.equal(page.items[0].edge.target, 1);
assert.throws(() => {
page.items[0].edge.target = 2;
}, TypeError);
assert.equal(cases(model, { case_indices: [0] }).items[0].edge.target, 1);
assert.equal(model.block(1).origin_start, "0x1010");
const second = cases(model, {
case_values: ["0x42", "0x7"],
offset: 1,
limit: 1,
});
assert.equal(second.items[0].case_index, 1);
assert.equal(second.next_offset, null);
assert.deepEqual(
cases(model, { case_indices: [2, 0, 1], case_values: ["0x7"] }).items.map(
(r) => r.case_index,
),
[1],
);
assert.equal(cases(model, { case_values: [] }).total, 0);
});
test("unlabelled/unmapped and partial or unavailable tables do not become absent runtime cases", () => {
const doc = fixture(),
model = load(doc);
assert.equal(cases(model, { case_values: ["0xbad1abe1bad1abe1"] }).total, 0);
const source = cases(model, { case_indices: [2, 3] });
assert.equal(source.items[0].label.status, "no-label");
assert.equal(source.items[1].label, null);
assert.deepEqual(source.items[1].edge, null);
assert.equal(source.relation_coverage.switches[0].partial, true);
assert.equal(source.coverage.runtime_feasibility, "unknown");
doc.functions[0].atomic_switches.items[0].table = null;
const unknown = cases(load(doc));
assert.equal(unknown.relation_coverage.switches[0].table_available, false);
assert.equal(unknown.relation_coverage.switches[0].case_count, null);
});
test("switch rows survive graph failure and missing old-engine capture remains unavailable", () => {
const doc = fixture();
delete doc.functions[0].atomic_facts;
doc.functions[0].atomic_facts_error = "graph capacity";
const model = load(doc);
assert.equal(model.facets.recovered_ir.availability, "unavailable");
assert.equal(model.switches({}).items[0].selector.size_bytes, 8);
assert.equal(cases(model).total, 4);
delete doc.functions[0].atomic_switches;
const legacy = load(doc);
assert.equal(legacy.facets.switches.availability, "unavailable");
assert.throws(() => cases(legacy));
assert.equal(legacy.code, "retained C");
doc.functions[0].atomic_switches = {
schema_version: 1,
items: [],
coverage: {
scope: "all recovered switches",
blocks_examined: 3,
operations_examined: 3,
machine_semantic_equivalence: "unknown",
},
};
const empty = load(doc);
assert.equal(empty.facets.switches.availability, "available");
assert.equal(empty.facets.switches.count, 0);
});
test("contradictory switch ownership or values fail only that independent facet", () => {
for (const change of [
(sw) => (sw.table.rows[0].edges[0].target = 2),
(sw) => (sw.table.rows[0].index = 1),
(sw) => (sw.selector.size_bytes = 4),
(sw) => (sw.table.default_raw_index = 0),
(sw) => sw.table.rows[0].edges.push(edge(0, 1)),
]) {
const doc = fixture();
change(doc.functions[0].atomic_switches.items[0]);
const model = load(doc);
assert.equal(model.facets.switches.availability, "unavailable");
assert.equal(model.facets.recovered_ir.availability, "available");
assert.equal(model.code, "retained C");
}
});
test("partial facets cannot disagree about a shared value identity when the graph is unavailable", () => {
const doc = fixture(),
fn = doc.functions[0];
delete fn.atomic_facts;
fn.atomic_calls = {
schema_version: 1,
items: [
{
id: "0x1100:0",
origin_address: "0x1100",
block: 0,
opcode: "CPUI_CALL",
is_dead: false,
target_value_id: null,
spec_available: false,
target: null,
target_name: null,
arguments: [
{
input_index: 1,
value_id: "v1",
storage: null,
storage_authority: null,
},
],
output: null,
prototype: null,
stack_offset: null,
},
],
values: [{ ...fn.atomic_switches.items[0].selector }],
coverage: {
scope: "all recovered calls",
blocks_examined: 3,
operations_examined: 3,
value_scope: "call inputs and outputs",
machine_call_equivalence: "unknown",
},
};
assert.equal(load(doc).facets.switches.availability, "available");
fn.atomic_switches.items[0].selector.size_bytes = 4;
const invalid = load(doc);
assert.equal(invalid.facets.switches.availability, "unavailable");
assert.equal(invalid.facets.calls.availability, "available");
assert.equal(invalid.calls({}).total, 1);
});
test("switch capacity rejection precedes row validation and preserves the valid graph", () => {
for (const oversized of [
(table) => (table.rows = Array(100001).fill({})),
(table) => (table.rows[0].edges = Array(100001).fill({})),
]) {
const doc = fixture();
oversized(doc.functions[0].atomic_switches.items[0].table);
const model = load(doc);
assert.equal(model.facets.switches.availability, "unavailable");
assert.match(model.facets.switches.error.message, /capacity/);
assert.equal(model.facets.recovered_ir.availability, "available");
}
});
@@ -0,0 +1,23 @@
const read = async (promise) => {
const result = await promise;
if (!result.ok) throw new Error(JSON.stringify(result));
return result.data;
};
const found = await read(rea.strings({ query: "incorrect header check", limit: 1 }));
const literal = found.items[0].address;
const refs = await read(rea.references({ addresses: [literal] }));
const entry = refs.results[0].items[0].fromFunction.address;
const snapshot = await read(rea.snapshot({ entry }));
const s = snapshot.snapshot;
const seed = await read(rea.operations({ snapshot: s, constant: literal, fields: ["kind", "statement", "inputs", "output"] }));
const uses = await read(rea.relations({ snapshot: s, kind: "uses", seeds: seed.items.map((row) => row.id) }));
const consumers = await read(rea.operations({ snapshot: s, ids: uses.items.map((row) => row.use) }));
const blockIds = [...new Set(consumers.items.map((row) => row.block))];
const blocks = await read(rea.blocks({ snapshot: s, ids: blockIds }));
const controls = await read(rea.relations({ snapshot: s, kind: "controllers", seeds: blockIds, limit: 128 }));
const guards = await read(rea.operations({ snapshot: s, block_ids: [...new Set(controls.items.map((row) => row.controller))], kind: "branch", fields: ["kind", "statement", "inputs"] }));
const phi = await read(rea.operations({ snapshot: s, kind: "phi", fields: ["inputs", "output"], limit: 2 }));
const valueIds = [...new Set(phi.items.flatMap((op) => op.inputs.map((input) => input.value_id)).filter(Boolean))];
const values = await read(rea.values({ snapshot: s, ids: valueIds }));
const slice = await read(rea.slice({ snapshot: s, seeds: guards.items.map((row) => row.id), max_operations: 64 }));
return { literal, entry, source: snapshot.source, coverage: snapshot.coverage, blocks, controls, guards, phi, values, slice: { coverage: slice.coverage, value_frontier: slice.value_frontier, operations: slice.operations.map(({ id, statement }) => ({ id, statement })) } };
@@ -0,0 +1,44 @@
# Wide native operand retrieval oracle
Pinned before any model launch on this question; not supplied to measured models.
This is a seeded relationship-retrieval task in a complex function, not an
unseeded behavior reconstruction, source-level explanation or runtime proof.
Target: Debian SQLite 3.40.1-2+deb12u2 amd64,
`/usr/lib/x86_64-linux-gnu/libsqlite3.so.0` -> `.0.8.6`, SHA-256
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
Engine: preserved `a0685486518e09d3c65f756325a7e9fed8df123a2a988aca2972a72335ce8edc`.
From the retained public-budget native projection: operation `0xea998:14649` is
`CPUI_MULTIEQUAL`, block 1203, output `v30000f159`, 156 native input slots.
It is not dead. Input relationships must retain their own identities even when
four selected slots refer to one value:
| Input index | Value | Predecessor | Predecessor edge |
| ----------- | ---------- | ----------: | ---------------- |
| 0 | v300034de6 | 7 | 7:1 |
| 1 | v300034de6 | 8 | 8:72 |
| 77 | v300034de6 | 633 | 633:0 |
| 105 | v10000cb23 | 738 | 738:0 |
| 155 | v300034de6 | 668 | 668:0 |
Distinct values and retained immediate definitions:
| Value | Space | Offset | Width | Definition | Opcode/output |
| ---------- | ------ | ------------------ | ----: | ------------- | ---------------------------- |
| v300034de6 | stack | 0xfffffffffffffec0 | 8 | 0xea7d8:146d5 | CPUI_MULTIEQUAL / v300034de6 |
| v10000cb23 | unique | 0x23e00 | 8 | 0xebd6c:1a344 | CPUI_CAST / v10000cb23 |
Both values are nonconstant, nonannotation and not native function inputs.
Their offsets are storage offsets, not computed values or machine destinations.
Slot 156 is absent for this retained 156-slot operation. `0xffff:0` is not
retained; its existence outside the projection remains unknown, not an empty
operation. Selected operands plus definition identities suffice; no complete
input array, C report, exhaustive graph or transitive slice is required.
Public projection: 4,096 operations, 7,469 values and 326 definition-frontier
entries. Complete recovered CFG is unavailable; uses concern the retained set,
aliases and runtime feasibility remain unknown. Selected immediate definitions
are retained, without closing the global frontier. Evidence references and
target digest must be real. These producer-gold identities were also compared
to the earlier full diagnostic capture; neither is independent machine/IR proof.
@@ -0,0 +1,52 @@
# Sequential and overlapping public requests
This follows the unexplained 107.804-second source capture in the
[guided model pair](savepoint-control-study.md). The same preserved `677b9795...`
engine, target digest, VM entry and three instruction windows are used through
real stdio MCP. Each case starts a fresh lease/private native cache; cases run
serially in code/sequential, direct/overlap, direct/sequential, code/overlap order.
No model is involved and no token savings are inferred.
`verify-overlap.mjs` first requests one full VM capture and the windows, either
sequentially or with the second request starting 100 ms after the first. The
client records starts/finishes to prove request overlap; it does not assume
provider concurrency. `AtomicSession.call` serializes provider admission for
both adapters. Thus an instruction request can wait behind the snapshot. Its
API wall time is not its native command execution time.
Every full recovered graph and ordinary function field matches the complete
golden. Selected instruction items/coverage agree across all four cases. Both
snapshot and instruction reads repeat under the same retained Evidence and
handle. Target/frame/decoding selection and all limits are preserved.
| Case | Fresh task seconds | Fresh family CPU s | VM capture s | Warm task ms | Warm family CPU s |
| ------------------ | -----------------: | -----------------: | -----------: | -----------: | ----------------: |
| Code, sequential | 39.488 | 39.432 | 37.112 | 43.737 | 0.044 |
| Direct, overlap | 38.931 | 38.865 | 36.635 | 6.561 | 0.007 |
| Direct, sequential | 39.171 | 39.098 | 36.987 | 7.759 | 0.008 |
| Code, overlap | 40.616 | 40.564 | 38.262 | 32.288 | 0.033 |
CPU is a delta over each complete phase in the owned cgroup; no overlapping
per-request CPU attribution is attempted. Kernel peaks are cumulative and
include charged file cache, not RSS; the warm peak is only an upper bound.
Complete family totals/peaks and cleanup counters are in the resource receipts
under `.cache/atomic-codemode/overlap-v1-*`. One CPU, 4 GiB hard / 3 GiB early-stop,
192-task bounds apply. All four cases have verified cleanup, no abort and OOM 0.
| Case | Whole family wall s | Whole family CPU s | Kernel peak MiB |
| ------------------ | ------------------: | -----------------: | --------------: |
| Code, sequential | 41.299 | 41.026 | 623.1 |
| Direct, overlap | 40.695 | 40.427 | 616.2 |
| Direct, sequential | 40.893 | 40.660 | 617.6 |
| Code, overlap | 42.506 | 42.239 | 630.0 |
The 108-second capture does not reproduce in this matched sample. Request
queueing is confirmed by the owner and explains waiting outside native source
times; it does not explain the original long capture itself. There is no basis
to change provider concurrency or claim a native speed fix. Host contention,
OS caches and unrecorded native phases remain uncontrolled/unmeasured. This is
a four-case diagnostic, not a distribution or proof that the anomaly is fixed.
Both repeated tasks are small; Code Mode's additional guest setup is visible
here. This is not evidence against composition over a larger dependency graph,
nor a comparison with a client that lacks its own JavaScript executor.
+80
View File
@@ -0,0 +1,80 @@
{
"name": "rea-atomic-codemode-experiment",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "rea-atomic-codemode-experiment",
"dependencies": {
"quickjs-emscripten": "0.32.0"
}
},
"node_modules/@jitl/quickjs-ffi-types": {
"version": "0.32.0",
"resolved": "https://registry.npmjs.org/@jitl/quickjs-ffi-types/-/quickjs-ffi-types-0.32.0.tgz",
"integrity": "sha512-v9T+GQpmk43VDJ7d72sf0Nexhk+ArvtUihW27dy7lqAl0zBObFKtSBBIm5RBjwIhE8VwsPPm9PNuvPvNqLWUEg==",
"license": "MIT"
},
"node_modules/@jitl/quickjs-wasmfile-debug-asyncify": {
"version": "0.32.0",
"resolved": "https://registry.npmjs.org/@jitl/quickjs-wasmfile-debug-asyncify/-/quickjs-wasmfile-debug-asyncify-0.32.0.tgz",
"integrity": "sha512-EX8zbXwGqCgAE764M+qvkHtyXDi/FUoMBea0JnES7vCM3P7a2+EOZOjGv85wtZ2sJhI1oJ+nekmqpOODFDY+hw==",
"license": "MIT",
"dependencies": {
"@jitl/quickjs-ffi-types": "0.32.0"
}
},
"node_modules/@jitl/quickjs-wasmfile-debug-sync": {
"version": "0.32.0",
"resolved": "https://registry.npmjs.org/@jitl/quickjs-wasmfile-debug-sync/-/quickjs-wasmfile-debug-sync-0.32.0.tgz",
"integrity": "sha512-LeYWrPGC1uNCTBWvibo3ZLJj0CSVNYUXvJpXMCmuQ5Sap2cCACc3uvGvYV4homHHBAzfw5akoTqMMS4YFRtw+Q==",
"license": "MIT",
"dependencies": {
"@jitl/quickjs-ffi-types": "0.32.0"
}
},
"node_modules/@jitl/quickjs-wasmfile-release-asyncify": {
"version": "0.32.0",
"resolved": "https://registry.npmjs.org/@jitl/quickjs-wasmfile-release-asyncify/-/quickjs-wasmfile-release-asyncify-0.32.0.tgz",
"integrity": "sha512-3oSwPfja12ICz4aIblB58cuY8JlEq5Txt8Cut4VLo+LH47QN+mzCnSgnbB03hWzg1LBcc+VyyI9UOag7a1NF+Q==",
"license": "MIT",
"dependencies": {
"@jitl/quickjs-ffi-types": "0.32.0"
}
},
"node_modules/@jitl/quickjs-wasmfile-release-sync": {
"version": "0.32.0",
"resolved": "https://registry.npmjs.org/@jitl/quickjs-wasmfile-release-sync/-/quickjs-wasmfile-release-sync-0.32.0.tgz",
"integrity": "sha512-BKNDI/TPBfGlLNGYpLrhcDGXmIk4xHm4MRAisOBnOzpXVn9HZWsfmMAc9WMBrAHjvvds6HOikKeaOBKdPdpVrg==",
"license": "MIT",
"dependencies": {
"@jitl/quickjs-ffi-types": "0.32.0"
}
},
"node_modules/quickjs-emscripten": {
"version": "0.32.0",
"resolved": "https://registry.npmjs.org/quickjs-emscripten/-/quickjs-emscripten-0.32.0.tgz",
"integrity": "sha512-So0Sqw869y/S2oE3Nuc0uT3Dhqgvsj8FSrwBdsuTosVsG8ME5/OcudU1GxsrIFdFABgy17GHnTVO9TYV/bLQcA==",
"license": "MIT",
"dependencies": {
"@jitl/quickjs-wasmfile-debug-asyncify": "0.32.0",
"@jitl/quickjs-wasmfile-debug-sync": "0.32.0",
"@jitl/quickjs-wasmfile-release-asyncify": "0.32.0",
"@jitl/quickjs-wasmfile-release-sync": "0.32.0",
"quickjs-emscripten-core": "0.32.0"
},
"engines": {
"node": ">=16.0.0"
}
},
"node_modules/quickjs-emscripten-core": {
"version": "0.32.0",
"resolved": "https://registry.npmjs.org/quickjs-emscripten-core/-/quickjs-emscripten-core-0.32.0.tgz",
"integrity": "sha512-QFnPfjFey8EqknSrSxe1hZrf1/8z7/6s1QzGOmKo6++02r7QRRX7ZoyNaZh7JuVjWsVW87KnQrbZqnHkOAzUyg==",
"license": "MIT",
"dependencies": {
"@jitl/quickjs-ffi-types": "0.32.0"
}
}
}
}
+12
View File
@@ -0,0 +1,12 @@
{
"dependencies": {
"quickjs-emscripten": "0.32.0"
},
"name": "rea-atomic-codemode-experiment",
"private": true,
"type": "module",
"packageManager": "npm@11.16.0",
"scripts": {
"test": "node --test *.test.mjs"
}
}
@@ -0,0 +1,221 @@
// Diagnostic native timings, using REA's target, environment and process owners.
import assert from "node:assert/strict";
import { createHash } from "node:crypto";
import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { parseArgs } from "node:util";
import { parseBinaryTarget } from "../../dist/application/BinaryTargetResolver.js";
import { inspectKunaInstallation } from "../../dist/kuna/KunaInstallation.js";
import { KunaTargetCommands } from "../../dist/kuna/KunaTargetCommands.js";
import { PrivateRuntimeRoot } from "../../dist/process/PrivateRuntimeRoot.js";
import { spawnOwnedProviderProcess } from "../../dist/process/ProviderProcess.js";
import { requireResult } from "./fact-error.mjs";
import { parseNativeIR } from "./native-ir.mjs";
import { NATIVE_OUTPUT_BYTES } from "./limits.mjs";
import {
createVerifierRun,
completeVerifierRun,
assertVerifierCleanupComplete,
} from "../../scripts/lib/verifier-run.mjs";
const { values: options, positionals } = parseArgs({
allowPositionals: true,
options: {
"without-timing": { type: "boolean", default: false },
"output-bytes": { type: "string", default: String(NATIVE_OUTPUT_BYTES) },
},
});
const [engine, binary, entry, output, goldenPath] = positionals;
const timingEnabled = !options["without-timing"];
assert.ok(engine && binary && entry && output);
const outputBytes = Number(options["output-bytes"]);
assert.ok(
Number.isSafeInteger(outputBytes) &&
outputBytes > 0 &&
outputBytes <= NATIVE_OUTPUT_BYTES,
"Choose a positive native output budget within the supervisor's 8 MiB capture capacity",
);
const sha = (s) => createHash("sha256").update(s).digest("hex");
const expectedPhases = [
"decompile_drive",
"pre_render_metadata",
"render_c",
"metadata_extract",
"membership",
"calls",
"full_graph",
"projection",
"metadata_tail",
];
const verifier = createVerifierRun(),
runs = [],
failures = [];
const root = await PrivateRuntimeRoot.create({ prefix: "rea-atomic-profile-" });
let commands, installation, target;
try {
target = requireResult(await parseBinaryTarget(binary));
installation = requireResult(
await inspectKunaInstallation({
...process.env,
REA_KUNA_COMMAND: engine,
REA_KUNA_CACHE_DIR: join(root.path, "engine-cache"),
}),
);
commands = new KunaTargetCommands(target, {
signal: new AbortController().signal,
launcher: (options) =>
spawnOwnedProviderProcess({
...options,
env: {
...options.env,
...(timingEnabled ? { KUNA_EXPERIMENT_ATOMIC_TIMING: "1" } : {}),
},
}),
});
let firstFunction;
for (const phase of [
"fresh-private-native-cache",
"same-private-native-cache-new-process",
]) {
const before = performance.now();
const captured = requireResult(
await commands.run({
installation,
operation: "experiment:atomic-phase-profile",
arguments: (input) => [
"decompile",
input,
entry,
"--addr",
"--json",
"--atomic-facts",
"--atomic-output-bytes",
String(outputBytes),
"--max-fn-memory",
"1024",
],
outputBytes: NATIVE_OUTPUT_BYTES,
timeoutMs: 120000,
signal: new AbortController().signal,
}),
);
const wallMs = performance.now() - before;
const stdout = captured.stdout.text,
stderr = captured.stderr.text;
const stdoutPath = `${output}.run-${runs.length}.stdout.json`,
stderrPath = `${output}.run-${runs.length}.stderr.jsonl`;
await writeFile(stdoutPath, stdout, { flag: "wx" });
await writeFile(stderrPath, stderr, { flag: "wx" });
const record = {
phase,
wall_ms: wallMs,
status: "unverified",
source: {
stdout_path: stdoutPath,
stdout_sha256: sha(stdout),
stdout_bytes: Buffer.byteLength(stdout),
stderr_path: stderrPath,
stderr_sha256: sha(stderr),
},
};
runs.push(record);
assert.notEqual(captured.diagnosticTruncated, true);
const timings = stderr
.split("\n")
.filter((line) => line.startsWith('{"kind":"rea_atomic_phase",'))
.map((line) => JSON.parse(line));
assert.deepEqual(
timings.map((row) => row.phase),
timingEnabled ? expectedPhases : [],
);
assert.ok(
timings.every(
(row) =>
row.address === entry &&
Number.isSafeInteger(row.wall_ns) &&
row.wall_ns >= 0,
),
);
const model = parseNativeIR(stdout, entry, captured.input);
const functionRecord = JSON.parse(stdout).functions[0];
if (firstFunction) assert.deepEqual(functionRecord, firstFunction);
firstFunction ??= functionRecord;
if (goldenPath)
assert.deepEqual(
functionRecord,
JSON.parse(await readFile(goldenPath, "utf8")).functions[0],
);
const accountedMs = timingEnabled
? timings.reduce((n, row) => n + row.wall_ns / 1e6, 0)
: null;
if (accountedMs !== null) assert.ok(accountedMs <= wallMs);
Object.assign(record, {
status: "verified",
accounted_wall_ms: accountedMs,
unprofiled_wall_ms: accountedMs === null ? null : wallMs - accountedMs,
timings,
code_bytes: Buffer.byteLength(functionRecord.code),
facets: model.facets,
function_record_matches_prior_run: runs.length > 1,
function_record_matches_golden: goldenPath ? true : null,
});
}
} catch (error) {
failures.push({
code: error.code ?? "verification_failed",
message: error.message,
details: error.details ?? {},
stack: error.stack,
});
} finally {
let closed = true;
if (commands) {
const result = await commands.close();
closed = result.ok;
if (!closed)
failures.push({
code: result.error.code,
message: result.error.message,
details: result.error.details,
});
}
if (closed) {
try {
await root.close();
} catch (error) {
failures.push({
code: "cleanup_incomplete",
message: error.message,
resources: [root.path],
});
}
}
}
const lifecycle = await completeVerifierRun(verifier);
assertVerifierCleanupComplete(lifecycle.process_lineage);
await writeFile(
output,
JSON.stringify(
{
engine: installation && {
path: installation.executable,
sha256: installation.sha256,
},
target: target && { path: target.path, sha256: target.sha256 },
entry,
timing_enabled: timingEnabled,
output_budget_bytes: outputBytes,
supervisor_capture_budget_bytes: NATIVE_OUTPUT_BYTES,
runs,
failures,
lifecycle,
scope:
"Diagnostic native processes through production owners with an explicit timing-only launcher environment. Phase walls only, not phase CPU. Whole-run CPU/peak memory are separate resource-run receipts; OS cache uncontrolled. Not a model or public MCP measurement.",
},
null,
2,
),
{ flag: "wx" },
);
console.log(JSON.stringify({ output, runs: runs.length, failures }));
if (failures.length) process.exitCode = 1;
@@ -0,0 +1,136 @@
# Selected definitions after whole-graph capacity failure
Historical probe at REA experiment revision `4d7fccbce`. Use that revision's
patches to reproduce its preserved diagnostic engine and results below. The
subsequent callable projection is documented in [native-engine.md](native-engine.md);
its real-model question/oracle are [pinned separately](projection-oracle.md).
This is a diagnostic feasibility probe following the completed boundary model
pairs, not a new public provider capability. The analyst question is whether
selected call-argument definitions can remain readable without materializing an
entire recovered graph in the public transport. The alternatives are a smaller
whole-graph encoding and a bounded selection of operations/values. Neither
fewer bytes nor an offline graph traversal establishes model-token or time gains.
## Actual graph and capacity boundaries
The artifact remains Debian SQLite `3.40.1-2+deb12u2`, SHA-256
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
The selected implementation is `sqlite3VdbeExec`, entry `0xea6d0`.
The disposable diagnostic patch temporarily admits 32,000 operations, 72,000
values, 128,000 inputs and 16 MiB text. It is applied after
`kuna-native-ir.patch`, against the same pinned Kuna upstream revision. Only
this diagnostic capture uses a 32 MiB encoded/stdout budget. The committed
public patch, preserved public engine and 8 MiB public budget are unchanged.
| Observed quantity | Value |
| ----------------- | ----: |
| Recovered operations | 21,349 |
| Recovered values | 35,755 |
| Recovered blocks | 1,229 |
| Operand inputs | 51,951 |
| CFG successor edges | 2,061 |
| Operation statement UTF-8 bytes | 2,255,616 |
| Value type-name UTF-8 bytes | 90,013 |
| Compact ordinary graph JSON bytes | 17,008,223 |
| Entire native document bytes | 17,801,972 |
Raising only the current 12,000-operation bound would not suffice: inputs also
exceed 50,000, display text exceeds 2 MiB, and the entire document exceeds 8 MiB.
Values happen to remain below 36,000, with little headroom. Native coverage
reports no producer issues or omitted defining-operation IDs. This is the
retained recovered graph, not proof of complete/correct machine or runtime
semantics. The ordinary function record, including C and mappings, and the
independent call facet deeply equal the preserved public-engine capture.
`measure-native-projection.py` constructs a lossless tuple encoding of every
graph field and verifies decoding equals the original graph. Its graph occupies
8,225,047 bytes; estimated whole-document size is 9,018,796 bytes after retaining
C/calls/metadata. Thus this simple encoding still does not fit the unchanged
8,388,608-byte public budget. This is an exact synthetic serialization size and
roundtrip check, not an implemented native codec or measured codec speed.
## Focused definition facts
Using the two existing call identities at `0xf115c` and `0xf238b` as roots, the
script follows native operand values to their defining operations. It retains
exact IDs and complete selected operation/value records. A 512-operation budget
is not reached in these depth samples; unresolved definitions remain explicit.
| Requested backward depth | Operations | Values | Encoded payload bytes | Unexpanded definition frontier |
| ------------------------ | ---------: | -----: | --------------------: | -----------------------------: |
| 1 | 5 | 12 | 4,335 | 6 |
| 2 | 11 | 24 | 8,589 | 3 |
| 3 | 14 | 29 | 10,447 | 1 |
| 4 | 15 | 32 | 11,457 | 1 |
| 5 | 16 | 33 | 11,934 | 1 |
The same experiment over all 416 recovered call roots gives:
| Requested backward depth | Operations | Values | Encoded payload bytes |
| ------------------------ | ---------: | -----: | --------------------: |
| 1 | 965 | 2,269 | 830,233 |
| 2 | 1,494 | 3,076 | 1,188,208 |
| 3 | 2,007 | 3,920 | 1,541,735 |
| 4 | 2,401 | 4,670 | 1,849,015 |
The 4,096-operation probe budget is not reached in these samples. Depth three
still has 394 unexpanded definitions. These payload sizes cover selected
operations/values alone, before source/provenance/transport framing. Depths are
probe parameters, not proposed product limits. This is backward SSA dependence,
without CFG, control dependence, complete use lists, memory aliases or feasible
runtime paths. No model receives these offline projections in the completed
trials. They do not become accessible through an existing snapshot by this probe.
The signal favors trying a bounded call-rooted operation/value facet when the
full graph is unavailable. The native producer already has the live function at
the capture boundary, so this could reuse that analysis rather than reconstruct
definitions from C or force repeated decompilation. A focused initial capture
is another option; later requests would need a new analysis or a separately
owned live native model. That lifecycle/time tradeoff remains unresolved.
Any prototype must share native identity/value/operand interpretation, retain
frontier reasons and source Evidence, expose partial definition/use scope, and
keep CFG-dependent reads unavailable. A native block/origin ID does not establish
a machine instruction extent or branch target. Both direct and Code Mode must
receive the same facts through the shared API before another model comparison.
## Reproduction and measured scope
Apply `kuna-native-ir.patch`, then `kuna-graph-scale-diagnostic.patch`, to pinned
upstream `fe01d6294eb0d1053ce343c8943646df88e143fc`. Reuse its stock SLA assets
and the one-job embedded-spec build profile in [native-engine.md](native-engine.md).
The preserved diagnostic executable has SHA-256
`6d0d1cef9de13180e19af77a6f62075c417178831740bccc7174bbac0af764cc`.
It is copied outside Cargo outputs, separately from the public `e7d3f0cc...` engine.
The initial build launch failed before Cargo because its PATH omitted the existing
Cargo installation; that setup failure remains recorded, followed by a corrected
build. No toolchain was installed or upgraded.
The capture reuses `KunaTargetCommands.run`, a fresh private installation
`configuration.cacheRoot`, owned target staging and normal supervision. Its argv
is `decompile INPUT 0xea6d0 --addr --json --atomic-facts
--atomic-output-bytes 33554432 --max-fn-memory 1024`, with matching 32 MiB
supervised stdout capacity. It does not execute the target. The temporary native
source changes are restored after capture; diagnostic patch reverse/forward and
public patch reverse checks pass.
The diagnostic build takes 316.764 family wall / 316.136 CPU seconds, 2,113.5 MiB
kernel family peak. Native capture takes 37.078 / 36.869 seconds, 551.4 MiB.
Offline sizing/roundtrip takes 1.428 / 1.242 seconds, 170.0 MiB. All use one CPU,
4 GiB hard memory, 3 GiB early stop and verify cleanup/OOM 0; peaks include charged
file cache. OS caches are uncontrolled. There is no equivalent completed-task
speed comparison, public-provider success claim or model-token measurement here.
Run the bounded sizing script against the retained diagnostic native JSON:
```sh
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
python3 experiments/atomic-codemode/measure-native-projection.py \
/absolute/native.json /absolute/new-sizing-report.json
```
Raw receipts are `graph-scale-diagnostic-engine.json`,
`graph-scale-diagnostic-receipt.json`, `graph-scale-repro-sizing.json` and their
resource logs under `.cache/atomic-codemode/`. Binaries and the 17.8 MB native
capture remain local; they are not committed.
@@ -0,0 +1,66 @@
# Phi alternatives through bounded native facts
Fix `sqlite-projection-question.txt` and this oracle before either model launch.
Both lanes receive the same fourteen methods, native engine, model/effort request,
common guidance and resource bounds. This is a seeded follow-up investigation,
not discovery from an unknown entry. The task differs from counting boundary
calls, so its costs are not a before/after speedup comparison with that task.
Artifact: Debian SQLite `3.40.1-2+deb12u2`, `libsqlite3.so.0.8.6`, SHA-256
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
Implementation entry is `0xea6d0`, with a distinct same-name PLT entry.
Real public projection verification and comparison with the earlier full native
diagnostic graph establish these retained identities. The complete graph remains
unavailable; bounded operations/values are available. It is not necessary for the
answer to enumerate every global budget/counter or all unselected frontier IDs.
| Call origin / identity | Recovered RSI value | Immediate PHI definition |
| ---------------------- | ------------------- | ------------------------ |
| `0xf115c` / `0xf115c:6a84` | `v1000340ac` | `0xf1152:cf0f` |
| `0xf238b` / `0xf238b:7130` | `v100033f25` | `0xf238b:ceb7` |
Both calls resolve through native call specifications to `ram:0x296c0`.
RSI is recovered input slot two, with passing storage `register:0x30/8`;
that slot is not an established source parameter ordinal. The immediate
definitions are `CPUI_MULTIEQUAL`, not a guessed linear register assignment.
| PHI | Operand index | Value ID | Native predecessor / edge | Defining producer |
| --- | ------------: | -------- | ------------------------- | ----------------- |
| `0xf1152:cf0f` | 0 | `v1000118a8` | `1087` / `1087:0` | `0xf2086:6a72` |
| `0xf1152:cf0f` | 1 | `v10001358b` | `1153` / `1153:0` | `0xf1146:7560` |
| `0xf238b:ceb7` | 0 | `v100012a75` | `1142` / `1142:0` | `0xf2381:712c` |
| `0xf238b:ceb7` | 1 | `v10001352e` | `1158` / `1158:0` | `0xf2437:753c` |
Each producer is native `CPUI_PTRSUB` with two nonannotation constant-space
operands, zero and the following exact u64 offset, both width eight. The addresses
are independently checked through machine LEA instructions; do not treat PTRSUB
as arbitrary integer subtraction or a native origin as a physical branch target.
| Producer origin | Verified address | NUL-terminated format text |
| --------------- | ---------------- | -------------------------- |
| `0xf2086` | `0x12c600` | `cannot change %s wal mode from within a transaction` |
| `0xf1146` | `0x12c698` | `no such savepoint: %s` |
| `0xf2381` | `0x12c500` | `cannot open savepoint - SQL statements in progress` |
| `0xf2437` | `0x12c538` | `cannot release savepoint - SQL statements in progress` |
GNU objdump decoding from the exported VM entry independently verifies all four
LEA origins and addresses. `0xf208d` jumps to `0xf1152`, whose local setup reaches
the `0xf115c CALL 0x296c0` (`e85f85f3ff`). The alternate release setup jumps at
`0xf2446` to `0xf238b`; that CALL targets `0x296c0` (`e83073f3ff`). Open setup
locally reaches the latter call. These selected static transfers do not establish
runtime path feasibility, memory stability or full function control dependence.
Independent direct file reads check these string bytes only after `readelf -lW`
binds addresses to identical file offsets in the read-only LOAD segment
`0x11a000..0x154bfc`. Native loaded-image byte queries have their own source IDs.
The byte sequences, including terminators, occupy 52, 22, 51 and 54 bytes in the
table's order. Adjacent bytes must not be merged into the reported format.
Required final facts: correct entry/target digest and facet scope; both call/PHI
links, all four operand-to-producer alternatives with predecessor edge IDs; the
four correct addresses and strings; verified LEA/call/jump facts; real Evidence;
one VM analysis with subsequent reads reusing its snapshot; clear limits for
unexpanded definitions, complete uses/CFG, aliases and runtime behavior. Judge
these facts and source arguments separately from answer verbosity or RPC count.
The oracle and repository are absent from the isolated model workspace.
+492
View File
@@ -0,0 +1,492 @@
// Native protocols stay here. API and execution adapters consume parsed facts.
import { createHash, randomUUID } from "node:crypto";
import { writeFile, readFile, mkdir } from "node:fs/promises";
import { join } from "node:path";
import { parseBinaryTarget } from "../../dist/application/BinaryTargetResolver.js";
import { inspectKunaInstallation } from "../../dist/kuna/KunaInstallation.js";
import { KunaTargetCommands } from "../../dist/kuna/KunaTargetCommands.js";
import { referenceReader } from "./reference-reads.mjs";
import { PrivateRuntimeRoot } from "../../dist/process/PrivateRuntimeRoot.js";
import { parseNativeIR } from "./native-ir.mjs";
import { NativeEvidenceReads } from "./native-evidence-reads.mjs";
import { DEFAULT_NATIVE_OPERATION_FIELDS } from "./native-operation-views.mjs";
import { FactFailure, requireResult } from "./fact-error.mjs";
import { windowReaders } from "./window-readers.mjs";
import { NATIVE_OUTPUT_BYTES } from "./limits.mjs";
import { sourceSummary, projectFact, readFactBatch } from "./fact-reads.mjs";
import { parseFunctionFacts, selectFunctionFacts } from "./function-facts.mjs";
import { parseStringFacts, selectStringFacts } from "./string-facts.mjs";
import {
parseKunaSsaDump,
backwardSlice,
} from "../atomic-re-tools/kuna-ssa.mjs";
const sha256 = (value) => createHash("sha256").update(value).digest("hex");
const missingNativeFacet = () => ({
availability: "unavailable",
error: {
code: "capability_unavailable",
message: "Requires the pinned native exporter",
},
});
export class KunaFactOwner {
#lifetime = new AbortController();
#sources = new Map();
#snapshots = new Map();
#work = new Map();
#loaded = new Map();
#root;
#commands;
#target;
#installation;
#retainedBytes = 0;
#strings;
#functions;
#captureDirectory;
#evidence = new NativeEvidenceReads(async (id) => ({
text: await this.#sourceText(id),
source: this.#sources.get(id).source,
}));
#nativeIR = false;
context;
static async open(path, environment = process.env) {
const owner = new KunaFactOwner();
owner.#target = requireResult(
await parseBinaryTarget(path),
"invalid_target",
);
owner.#installation = requireResult(
await inspectKunaInstallation(environment),
"provider_unavailable",
);
const nativeIRHash = environment.REA_ATOMIC_NATIVE_IR_ENGINE_SHA256;
if (
nativeIRHash !== undefined &&
(!/^[0-9a-f]{64}$/u.test(nativeIRHash) ||
nativeIRHash !== owner.#installation.sha256)
)
throw new FactFailure(
"engine_profile_mismatch",
"Native IR requires the exact caller-pinned experimental engine SHA-256",
);
owner.#nativeIR = nativeIRHash !== undefined;
if (!owner.#nativeIR && owner.#installation.version !== "1.833")
throw new FactFailure(
"unsupported_version",
"SSA diagnostic decoder is verified only for Kuna 1.833",
);
owner.#root = await PrivateRuntimeRoot.create({
prefix: "rea-atomic-facts-",
});
if (owner.#nativeIR)
owner.#installation = {
...owner.#installation,
configuration: {
...owner.#installation.configuration,
cacheRoot: join(owner.#root.path, "engine-cache"),
},
};
owner.#commands = new KunaTargetCommands(owner.#target, {
signal: owner.#lifetime.signal,
});
owner.#captureDirectory = environment.REA_ATOMIC_EVIDENCE_DIR;
owner.context = Object.freeze({
lease: randomUUID(),
artifact_sha256: owner.#target.sha256,
path: owner.#target.path,
selected_path: path,
provider: "kuna",
engine_sha256: owner.#installation.sha256,
engine_version: owner.#installation.version,
address_space: "native-default",
analysis_model: owner.#nativeIR
? "experimental-native-ir-v3"
: "diagnostic-ssa-1.833",
profile: owner.#nativeIR
? "pinned engine with embedded specs; fresh private engine cache per lease; byte reads suppress global discovery; other native CLI defaults; no annotations"
: "byte reads suppress global discovery; other native CLI defaults, no annotations; external spec cache freshness is not fingerprinted",
});
return owner;
}
async #native(key, arguments_, signal) {
if (this.#work.has(key)) return this.#work.get(key);
if (this.#sources.size >= 128)
throw new FactFailure(
"capacity",
"Session source capacity reached; close and reopen the target",
);
const work = this.#capture(arguments_, signal);
this.#work.set(key, work);
try {
return await work;
} catch (error) {
this.#work.delete(key);
throw error;
}
}
async #capture(arguments_, signal) {
const start = performance.now();
const snapshot = requireResult(
await this.#commands.run({
installation: this.#installation,
operation: "experiment:atomic-facts",
arguments: arguments_,
outputBytes: NATIVE_OUTPUT_BYTES,
timeoutMs: 120000,
signal: AbortSignal.any([signal, this.#lifetime.signal]),
}),
);
const actualArguments = arguments_(snapshot.input);
const id = `ev_${sha256(JSON.stringify([this.context.lease, actualArguments, sha256(snapshot.stdout.text)]))}`;
const bytes = Buffer.byteLength(snapshot.stdout.text);
if (this.#retainedBytes + bytes > 64 * 1024 * 1024)
throw new FactFailure(
"capacity",
"64 MiB retained-source budget reached; close and reopen",
);
const file = join(this.#root.path, id);
await writeFile(file, snapshot.stdout.text, { flag: "wx" });
const source = Object.freeze({
evidence_id: id,
sha256: sha256(snapshot.stdout.text),
bytes: Buffer.byteLength(snapshot.stdout.text),
arguments: Object.freeze(actualArguments),
display_arguments: Object.freeze(
actualArguments.map((s) =>
s === snapshot.input ? this.#target.path : s,
),
),
elapsed_ms: performance.now() - start,
});
this.#sources.set(id, { source, file });
this.#retainedBytes += bytes;
if (this.#captureDirectory) {
await mkdir(this.#captureDirectory, { recursive: true, mode: 0o700 });
await writeFile(
join(this.#captureDirectory, `${id}.txt`),
snapshot.stdout.text,
{ flag: "wx", mode: 0o600 },
);
await writeFile(
join(this.#captureDirectory, `${id}.json`),
JSON.stringify({ context: this.context, source }),
{ flag: "wx", mode: 0o600 },
);
}
return { source, input: snapshot.input };
}
async #sourceText(id) {
const record = this.#sources.get(id);
if (!record)
throw new FactFailure(
"unknown_evidence",
"Evidence is not owned by this target lease",
);
const text = await readFile(record.file, "utf8");
if (sha256(text) !== record.source.sha256)
throw new FactFailure("evidence_changed", "Retained source changed");
return text;
}
async strings(input, signal) {
const capture = await this.#native(
"strings",
(path) => [
"strings",
path,
"--json",
"--encoding",
"all",
"--min-length",
"5",
"--termination",
"any",
"--no-xrefs",
],
signal,
);
this.#strings ??= parseStringFacts(
await this.#sourceText(capture.source.evidence_id),
capture.input,
);
return {
source: sourceSummary(capture.source),
...selectStringFacts(this.#strings, input),
};
}
async functions(input, signal) {
const capture = await this.#native(
"functions",
(path) => ["functions", path, "--json"],
signal,
);
this.#functions ??= parseFunctionFacts(
await this.#sourceText(capture.source.evidence_id),
);
return {
source: sourceSummary(capture.source),
...selectFunctionFacts(this.#functions, input),
};
}
async references(input, signal) {
return readFactBatch(
input.addresses.map((address) => referenceReader.query(input, address)),
(query) =>
this.#native(
referenceReader.key(query),
(path) => referenceReader.arguments(path, query),
signal,
),
async (capture, query) =>
referenceReader.read(
await this.#sourceText(capture.source.evidence_id),
capture.input,
query,
input,
),
);
}
async snapshot(input, signal) {
const capture = await this.#native(
`ssa:${input.entry}`,
(path) => [
"decompile",
path,
input.entry,
"--addr",
...(this.#nativeIR
? [
"--json",
"--atomic-facts",
"--atomic-output-bytes",
String(NATIVE_OUTPUT_BYTES),
]
: ["--raw"]),
"--max-fn-memory",
"1024",
],
signal,
);
const id = `snap_${sha256(this.context.lease + input.entry + capture.source.sha256)}`;
if (!this.#snapshots.has(id))
this.#snapshots.set(id, {
entry: input.entry,
input: capture.input,
evidence_id: capture.source.evidence_id,
});
const parsed = await this.#load(id);
if (!this.#nativeIR && parsed.block(0).origin_start !== input.entry)
throw new FactFailure(
"subject_mismatch",
"Native SSA entry does not match the requested function entry",
);
return {
snapshot: id,
source: sourceSummary(capture.source),
entry: input.entry,
coverage: parsed.coverage,
facets: parsed.facets ?? {
code: { availability: "available" },
recovered_ir: { availability: "available", coverage: "partial" },
operations: {
availability: "available",
scope: "diagnostic-printed-SSA",
},
calls: missingNativeFacet(),
values: missingNativeFacet(),
},
};
}
async instructions(input, signal) {
return this.#readWindows("instructions", input, signal);
}
async bytes(input, signal) {
return this.#readWindows("bytes", input, signal);
}
async #readWindows(name, input, signal) {
const reader = windowReaders[name];
return readFactBatch(
input.windows,
(request) =>
this.#native(
`${name}:${reader.key(request)}`,
(path) => reader.arguments(path, request),
signal,
),
async (capture, request) =>
reader.parse(
await this.#sourceText(capture.source.evidence_id),
capture.input,
request,
input.fields,
),
);
}
async #load(id) {
const descriptor = this.#snapshots.get(id);
if (!descriptor)
throw new FactFailure(
"unknown_snapshot",
"Snapshot is not owned by the current target lease",
);
if (!this.#loaded.has(id)) {
const { source } = this.#sources.get(descriptor.evidence_id);
if (
source.bytes > (this.#nativeIR ? NATIVE_OUTPUT_BYTES : 2 * 1024 * 1024)
)
throw new FactFailure(
"capacity",
"Source exceeds this format's indexing budget; at most two resident models are retained; full source remains readable",
{ evidence_id: descriptor.evidence_id },
);
const text = await this.#sourceText(descriptor.evidence_id);
let parsed;
try {
parsed = this.#nativeIR
? parseNativeIR(text, descriptor.entry, descriptor.input)
: parseKunaSsaDump(text);
} catch (error) {
throw new FactFailure(
error.code ?? "invalid_provider_output",
error.message.slice(0, 1024),
{
...error.details,
evidence_id: descriptor.evidence_id,
source_retained: true,
},
);
}
if (this.#loaded.size >= 2)
this.#loaded.delete(this.#loaded.keys().next().value);
this.#loaded.set(id, parsed);
}
return this.#loaded.get(id);
}
async #query(input, read) {
const model = await this.#load(input.snapshot);
const evidence_id = this.#snapshots.get(input.snapshot).evidence_id;
try {
return { ...read(model), evidence_id };
} catch (error) {
throw new FactFailure(error.code ?? "analysis_failed", error.message, {
...error.details,
evidence_id,
source_retained: true,
});
}
}
async operations(input) {
return this.#query(input, (model) => ({
...model.select(input),
coverage: model.coverage,
}));
}
async relations(input) {
if (["controllers", "operands", "cases"].includes(input.kind))
this.#requireNativeIR(input.snapshot);
return this.#query(input, (model) => ({
coverage: model.coverage,
...model.edges(input),
}));
}
async slice(input) {
return this.#query(input, (model) => {
const result = model.slice
? model.slice(input)
: backwardSlice(model, input.seeds, {
maxOperations: input.max_operations,
stopAt: input.stop_at,
stopKinds: input.stop_kinds,
});
return {
...result,
operations: result.operations.map((row) =>
projectFact(
row,
input.fields ??
(this.#nativeIR ? DEFAULT_NATIVE_OPERATION_FIELDS : undefined),
["id", "origin_address", "block", "source_line", "is_dead"],
),
),
};
});
}
#requireNativeIR(snapshot) {
if (!this.#nativeIR)
throw new FactFailure(
"capability_unavailable",
"This relation requires the pinned experimental native IR engine; the diagnostic SSA format does not contain it",
this.#snapshots.has(snapshot)
? { evidence_id: this.#snapshots.get(snapshot).evidence_id }
: {},
);
}
async values(input) {
this.#requireNativeIR(input.snapshot);
return this.#query(input, (model) => model.values(input));
}
async calls(input) {
this.#requireNativeIR(input.snapshot);
return this.#query(input, (model) => model.calls(input));
}
async switches(input) {
this.#requireNativeIR(input.snapshot);
return this.#query(input, (model) => model.switches(input));
}
async blocks(input) {
this.#requireNativeIR(input.snapshot);
return this.#query(input, (model) => ({
...model.blocks(input),
coverage: model.coverage,
}));
}
async code(input) {
const descriptor = this.#snapshots.get(input.snapshot);
if (!descriptor)
throw new FactFailure(
"unknown_snapshot",
"Snapshot is not owned by the current lease",
);
const model = this.#nativeIR ? await this.#load(input.snapshot) : null;
const code =
model?.code ??
(await this.#sourceText(descriptor.evidence_id)).split(
"\n0\nBasic Block",
1,
)[0];
const lines = code.split("\n");
return {
evidence_id: descriptor.evidence_id,
start_line: input.offset + 1,
language: model?.language ?? null,
text: lines.slice(input.offset, input.offset + input.limit).join("\n"),
total_lines: lines.length,
next_offset:
input.offset + input.limit < lines.length
? input.offset + input.limit
: null,
coverage: {
authority: "native reconstructed code",
scope: "selected display lines",
},
};
}
async evidence(input) {
return this.#evidence.read(input);
}
async close() {
this.#lifetime.abort();
// A failed Result retains native cleanup owners; leave this lease retryable.
requireResult(await this.#commands.close());
await this.#root.close();
this.#loaded.clear();
this.#sources.clear();
this.#work.clear();
this.#snapshots.clear();
this.#strings = undefined;
this.#functions = undefined;
this.#evidence.clear();
}
cancel() {
this.#lifetime.abort();
}
}
@@ -0,0 +1,45 @@
# Known-target references versus recovered call operations
This question and acceptance oracle are fixed before either real-model lane.
The question is `sqlite-reference-question.txt`. Both lanes expose the same
fourteen fact methods; the server adapters have four/seventeen MCP tools. Use the
same preserved engine `64c6b3fe5737056fc29c10b0aad6c7cbd489ad7d17bd1726141b77bdcf5e15b7`,
artifact, model/effort request, common envelope guidance and resource limits.
The task explicitly requests reference aggregates for the VM rather than
decompilation. Its cost comparison is limited to this selected task and these
adapters, with a client that already supports JavaScript dispatch in both lanes.
The artifact is Debian ELF `libsqlite3.so.0.8.6`, SHA-256
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
Independent GNU objdump decoding from exported function entries establishes:
- Wrapper `sqlite3Fts3ExprParse`, entry `0x65180`, has nine known-target machine
CALL references. Its optimized recovered graph retains eight call operations.
The omitted machine call is `0x6534b -> 0x29370` (`__stack_chk_fail@plt`), bytes
`e8 20 40 fc ff`. Do not patch or reinterpret the graph as complete machine
semantics; retain the distinction between representations.
- VM `sqlite3VdbeExec`, entry `0xea6d0`, has ELF symbol size `0x7e7e`. GNU objdump
identifies 396 direct immediate CALL instructions, 171 exact destinations and
16 calls to `0x296c0`. The set of all 396 `(origin,target)` pairs exactly equals
the native outgoing call-reference set; this is stronger than a row-count-only
comparison.
- The selected VM instruction is `0xf238b -> 0x296c0` (`sqlite3VdbeError@plt`).
A name or PLT entry does not establish runtime symbol binding.
- Independent disassembly also finds 21 indirect CALL instructions in the VM.
The requested native known-target totals do not include them. The model must
preserve unknown indirect/discovery/runtime coverage; it need not discover the
number 21 using an additional full instruction read. Reporting 396 as every
possible runtime or machine call would be wrong.
The question's required final facts are the nine/eight wrapper counts, the
verified differing machine call, all three VM aggregates, the selected VM call,
real source Evidence and the stated coverage limits. Exact instruction versus
function scope and source-row identity are also checked by the public MCP
verifier. An empty or unattributed function query must remain unknown, not proof
of no references.
Raw independent disassembly and the full pair comparison are retained locally
in `.cache/atomic-codemode/outgoing-oracle-vm.asm` and
`outgoing-vm-independent-oracle.json`. The model workspace contains neither.
Native CLI probes separately show source sizes and costs; fewer source bytes are
not a model-token or completed-task speed result.
@@ -0,0 +1,49 @@
// Shared native reference selection; adapters never reinterpret wire endpoints.
import { parseKunaReferenceFacts } from "../../dist/kuna/KunaReferenceFiles.js";
import { pageFacts } from "./fact-reads.mjs";
import { requireResult } from "./fact-error.mjs";
export const referenceReader = {
query: (input, address) => ({
address,
direction: input.direction === "incoming" ? "to" : "from",
scope: input.scope,
kinds: input.kinds,
}),
// Scope/fields/pages select the same native source; do not re-run its analysis.
key: (query) =>
`xref:${JSON.stringify([query.direction, query.address, query.kinds])}`,
arguments: (path, query) => [
"xrefs",
path,
`--${query.direction}`,
query.address,
...(query.kinds ? ["--kind", query.kinds.join(",")] : []),
"--json",
],
read: (text, path, query, input) => {
const parsed = requireResult(
parseKunaReferenceFacts(text, path, query),
"invalid_provider_output",
);
return {
...pageFacts(parsed.facts, input, ["source_row", "from", "to", "kind"]),
coverage: {
direction: query.direction === "to" ? "incoming" : "outgoing",
scope:
query.scope === "function"
? "native owning-function references"
: "exact address; not an object extent",
function_scope: parsed.functionScope,
examined_rows: parsed.rows,
excluded_alias_rows: parsed.excludedAliasRows,
excluded_scope_rows: parsed.excludedScopeRows,
unattributed_scope_rows: parsed.unattributedScopeRows,
discovery: "unknown",
indirect_targets: "unknown",
machine_semantic_equivalence: "unknown",
runtime_binding: "unknown",
},
};
},
};
+151
View File
@@ -0,0 +1,151 @@
// Named documentation selection; adapters supply their exact registered labels.
import { z } from "zod";
import { FactFailure } from "./fact-error.mjs";
import { freezeFacts } from "./fact-reads.mjs";
const referenceViews = ["index", "contracts"];
export const referenceInput = z.strictObject({
query: z.string().default(""),
view: z.enum(referenceViews).default("contracts"),
});
// Compact return contracts let code compose before first observing a response.
const returns = {
switches:
"data:{items:[{id,origin_address,block,selector:{native value fields}|null,table:{partial,labelled,consumed_mask,signed_render_hint,default_raw_index,default_edge:{id,index,target}|null,default_is_folded,case_count,mapping_count,relation_count,known_label_count,missing_label_count,no_label_count,unmapped_row_count,unknown_target_count}|null}],total,next_offset,missing,evidence_id,coverage}. No case inventory is embedded. selector is the native current BRANCHIND input and can be normalized; default_raw_index=-1/default_edge:null means undefined, not proof of no default. signed_render_hint does not establish general signedness. default_edge is the native model default, not necessarily the machine out-of-range guard destination. relations kind:cases reads selected original rows and mappings, even without a full graph if the independent switch facet exists.",
bytes:
"data:{results:[{ok,address,source:{evidence_id,sha256},hex,returned_bytes,end_address,coverage:{status,requested_bytes,provider_truncated,notes,original_file_mapping,runtime_memory}}|{ok:false,address,error}]}; ranges are [address,end_address). Completeness concerns requested bytes, not the provider truncation flag or adjacent bytes. Complete native hexdump and metadata remain in Evidence.",
functions:
"data:{source:{evidence_id,sha256},items:[{address,name,aliases,size}],total,next_offset,coverage}; size is an upper bound, or zero if unmeasured. Multiple matching addresses are retained, never selected by catalog order or display name alone.",
instructions:
"data:{results:[{ok,address,source:{evidence_id,sha256},items:[{address,size,status,bytes?,text?,mnemonic?,operands?}],resume_address,coverage}|{ok:false,address,error}]}; decoded rows describe selected machine bytes, not optimized IR branches. Partial batches retain successful windows.",
strings:
"data:{source:{evidence_id,sha256},items:[{address,value,string:{encoding,byte_length}}],total,next_offset,coverage}",
references:
"data:{results:[{ok,address,source:{evidence_id},items:[{source_row,from,to,kind,fromFunction:{address,name}|null,toFunction:{address,name}|null,instruction}],total,next_offset,coverage}|{ok:false,address,error}]}. source_row is the zero-based row in retained native xrefs; Evidence identity plus source_row identifies an occurrence. Coverage preserves examined/excluded/unattributed counts and unknown function scope/discovery. fields projects optional columns; endpoints/kind/source_row stay.",
snapshot:
"data:{snapshot:string,entry,source:{evidence_id,sha256},facets:{code:{availability},recovered_ir:{availability,error?},operations:{availability,scope?,count?,root_count?,definition_frontier_count?,error?},calls:{availability,count?,by_opcode?:[{opcode,known_target_count,unknown_target_count}],coverage?,error?},values:{availability,scope?,count?,error?}},coverage}. Call counts cover all retained recovered calls, including is_dead rows, not complete machine/runtime calls; target knowledge comes from a non-null native target, independent of opcode. An unavailable facet omits counts; an available empty call facet reports zeros. The complete graph can be unavailable while operations reports a bounded projection count; an unavailable operation facet has unknown/null counts, never a fabricated empty set; code remains readable through the same handle.",
operations:
"data:{items:[{id,origin_address,block,is_dead,kind,opcode,output,input_count,definition_count,constant_count}],total,next_offset,evidence_id,coverage}. Native defaults are compact summaries; fields optionally selects statement,inputs,definitions,constants or summary columns; identity/is_dead stay. Native inputs:[{input_index,value_id,phi_predecessor,phi_predecessor_edge}], output:value_id|null; constants:[{value_id,input_index,value,printed_size_bytes,is_annotation}] retain exact u64 offsets, not arbitrary-width computed values; definitions:[{definition,use,input_index,value_id,phi_predecessor,phi_predecessor_edge}]. Page large input lists with relations kind:operands. Diagnostic rows retain legacy printed statements/occurrences and omit unavailable native counts. Projection scope is partial, with unknown CFG and definition frontier coverage.",
relations:
"data:{items:operands{operation,input_index,value_id,definition,phi_predecessor,phi_predecessor_edge} OR definition/use edges OR CFG edges{from,to,operation,polarity?,condition_value?,condition_test?} OR controllers{controller,dependent,edge} OR cases{id,case_id,switch,block,case_index,label:{status:value|no-label,raw_bits}|null,recovered_target:{space,offset}|null,edge:{id,index,target}|null},total,next_offset,evidence_id,coverage,relation_coverage?,unretained_seeds?}. Cases page individual case/out-edge relations; unmapped cases contribute one unknown-edge row. case_id stays the same across multiple edges and total counts relations, not native table rows. Cases use switch operation-ID seeds; case_values selects native labels tagged value, case_indices selects original row ordinals. Filters intersect; seeds are deduplicated in caller order and rows keep native table order. Missing/no-label/unmapped and partial/unavailable tables stay explicit in relation_coverage, not proof of absent runtime cases. operands/definitions/uses seeds are operation-ID strings; CFG/controller seeds are numeric block IDs. operands accepts optional input_indices; deduplicates seeds/indices in caller order, otherwise keeps native slot order. All selected slots include constants/function inputs/nulls; values provides their metadata. For retained seeds, a requested out-of-range slot is absent; unretained seeds remain unknown. total is the filtered relation count before paging. Native def/use edges retain exact input/value/phi bindings; diagnostic edges include printed occurrence only.",
values:
"data:{items:[{id,space,offset,size_bytes,is_constant,is_input,is_annotation,definition,type_name}],total,next_offset,evidence_id,coverage}; definitions are operation IDs, offsets are exact hex strings. When the graph is unavailable, values may come from a bounded definition projection or only the call boundary; coverage states that scope and operations/definitions may be unreadable.",
calls:
"data:{items:[{id,origin_address,block,opcode,spec_available,is_dead,target:{space,offset}|null,target_name,arguments:[{input_index,value_id,storage:{space,offset,size_bytes}|null,storage_authority}],output}],total,next_offset,missing,evidence_id,coverage}. fields selects target/target_name/arguments/output/prototype/stack_offset/target_value_id/is_dead. Values may come from a bounded call-rooted definition projection or call-boundary set when the graph is unavailable; coverage states that scope. Calls include all retained call operations; null facts are unknown, not proof of no arguments or runtime binding. Candidate cross-function links require matching target identity and exact passing/input storage; join/subregister/alias interpretation remains unknown.",
blocks:
"data:{items:[{id,origin_start,operations,predecessors,successors:[{target,index,polarity,condition_value,condition_test}]}],total,next_offset,evidence_id,coverage}; IDs are local to this snapshot.",
slice:
"data:{operations:[operation rows as above],coverage:{status,boundaries:[{id,reason}],missing,missing_reasons?,source_scope?,source_issues,runtime_reachability,memory_alias_dependencies},value_frontier?:[{value_id?|operation?,input_index?,reason}],evidence_id}",
code: "data:{evidence_id,start_line,language:string|null,text,total_lines,next_offset,coverage}; language is native-reported or unknown. Read/filter full reconstructed text within Code Mode without returning it all.",
evidence:
"data:{source:{evidence_id,sha256,bytes,arguments},view?:{component,encoding,sha256,bytes,authority},text,total_code_units,next_offset}. Default component:source preserves exact native wire output. component:recovered_ir exposes complete decoded graph JSON with a separate view hash, including compressed graphs; source hash still identifies the original wire capture. This source view preserves malformed logical facts without asserting their validity. One decoded Evidence view up to 32 MiB is cached per lease and source freshness is checked on reads.",
};
const referenceSchemas = new WeakMap();
const schemaFor = (input) => {
if (!referenceSchemas.has(input))
referenceSchemas.set(
input,
freezeFacts(z.toJSONSchema(input, { io: "input" })),
);
return referenceSchemas.get(input);
};
function referenceNames(methods, names) {
const keys = Object.keys(methods),
lookup = new Map(keys.map((key) => [key, key])),
labels = {};
if (Object.keys(names).some((key) => !Object.hasOwn(methods, key)))
throw new FactFailure(
"invalid_input",
"Unknown method in reference presentation",
);
for (const key of keys) {
const label = names[key] ?? `rea.${key}`;
if (
typeof label !== "string" ||
!label ||
(lookup.has(label) && lookup.get(label) !== key)
)
throw new FactFailure(
"invalid_input",
"Ambiguous or invalid reference method name",
);
lookup.set(label, key);
labels[key] = label;
}
return { lookup, labels };
}
function selectReference(methods, query, lookup) {
const normalized = query.trim();
if (!normalized)
return { keys: Object.keys(methods), selection: "all", unmatched: [] };
const terms = normalized.split(/[\s,]+/u),
named = terms.map((term) => lookup.get(term));
if (named.some(Boolean))
return {
keys: [...new Set(named.filter(Boolean))],
selection: "named",
unmatched: terms.filter((term) => !lookup.has(term)),
};
const topic = normalized.toLowerCase();
const keys = Object.keys(methods).filter(
(key) =>
key.includes(topic) ||
methods[key].description.toLowerCase().includes(topic),
);
return { keys, selection: "topic", unmatched: keys.length ? [] : [query] };
}
export const createReference =
(methods) =>
(
query = "",
{ names = {}, language, includeExamples = true, view = "contracts" } = {},
) => {
if (!referenceViews.includes(view))
throw new FactFailure("invalid_input", "Unknown reference view");
const { lookup, labels } = referenceNames(methods, names),
selected = selectReference(methods, query, lookup);
const catalog = selected.keys.map((method) => ({
method,
name: labels[method],
description: methods[method].description,
}));
const coverage = {
declared_methods: Object.keys(methods).length,
matched_methods: selected.keys.length,
selection: selected.selection,
view,
};
if (view === "index")
return {
methods: catalog,
coverage,
unmatched_queries: selected.unmatched,
notes: [
"Static declared methods, not live provider availability. Request exact displayed names with view:contracts for schemas; inspect snapshot facets for runtime coverage.",
],
};
return {
language:
language ??
"JavaScript async function body; return a JSON value. All rea methods return {ok,context,data} or {ok:false,error}.",
methods: catalog.map((entry) => ({
...entry,
input_schema: schemaFor(methods[entry.method].input),
returns: returns[entry.method],
...(includeExamples ? { example: methods[entry.method].example } : {}),
})),
coverage,
unmatched_queries: selected.unmatched,
limits: {
execution_heap_mib: 64,
max_host_calls: 64,
output_utf8_bytes: 32768,
},
notes: [
"Select exact displayed method names or canonical method IDs, separated by spaces/commas; a topic query searches descriptions. Empty query returns all contracts.",
"No process, filesystem, network, imports or Node globals in the execution environment.",
"Select facts inside code, return only what the question needs, and keep real source IDs and unknowns.",
"A fresh JavaScript environment is used per execute_javascript call; analysis snapshots live until close_binary.",
],
};
};
@@ -0,0 +1,118 @@
import test from "node:test";
import assert from "node:assert/strict";
import Ajv2020 from "ajv/dist/2020.js";
import { methods, reference } from "./api.mjs";
import { createReference } from "./reference.mjs";
test("reference index discovers named methods without constructing unrequested schemas", () => {
const registry = {
sample: {
description: "Declared capability; live coverage is unknown.",
get input() {
throw new Error("Schema was not requested");
},
},
};
const lookup = createReference(registry);
const index = lookup("", {
names: { sample: "facts_sample" },
view: "index",
});
assert.deepEqual(index.methods, [
{
method: "sample",
name: "facts_sample",
description: registry.sample.description,
},
]);
assert.equal(index.coverage.view, "index");
assert.equal(index.coverage.declared_methods, 1);
assert.throws(() => lookup("sample"), /Schema was not requested/);
assert.throws(
() => lookup("", { view: "unknown" }),
/Unknown reference view/,
);
const all = reference("", { view: "index" });
assert.equal(all.methods.length, 15);
assert.ok(
all.methods.some(
(m) => m.method === "relations" && m.description.includes("operands"),
),
);
assert.ok(
all.methods.every(
(m) => !Object.hasOwn(m, "input_schema") && !Object.hasOwn(m, "returns"),
),
);
assert.ok(
Buffer.byteLength(JSON.stringify(all)) <
Buffer.byteLength(JSON.stringify(reference())),
);
});
test("named reference lists preserve caller order and exact schemas across presentations", () => {
const names = Object.fromEntries(
Object.keys(methods).map((name) => [name, `facts_${name}`]),
);
const result = reference(
"facts_values, facts_snapshot facts_values unknown",
{ names, includeExamples: false },
);
assert.deepEqual(
result.methods.map((m) => m.method),
["values", "snapshot"],
);
assert.deepEqual(
result.methods.map((m) => m.name),
["facts_values", "facts_snapshot"],
);
assert.deepEqual(result.unmatched_queries, ["unknown"]);
assert.equal(result.coverage.declared_methods, 15);
assert.equal(result.coverage.matched_methods, 2);
assert.equal(result.coverage.selection, "named");
const javascript = reference("rea.values snapshot");
assert.deepEqual(
javascript.methods.map((m) => m.method),
["values", "snapshot"],
);
assert.deepEqual(
result.methods.map((m) => m.input_schema),
javascript.methods.map((m) => m.input_schema),
);
const validate = new Ajv2020({ strict: false }).compile(
result.methods[1].input_schema,
);
assert.equal(validate({ entry: "0xea6d0" }), true);
assert.equal(validate({ snapshot: "wrong contract" }), false);
const validateValues = new Ajv2020({ strict: false }).compile(
result.methods[0].input_schema,
);
assert.equal(validateValues({ snapshot: "snap_example", ids: ["v1"] }), true);
assert.equal(
methods.values.input.parse({ snapshot: "snap_example" }).limit,
20,
);
assert.equal(Object.hasOwn(result.methods[0], "example"), false);
});
test("topic/all reference selection and shared schema ownership remain explicit", () => {
assert.equal(reference().methods.length, 15);
assert.equal(reference("no-such-topic").coverage.matched_methods, 0);
assert.ok(
reference("memory aliases").methods.some((m) => m.method === "slice"),
);
const before = reference("snapshot").methods[0];
assert.throws(() => {
before.input_schema.properties.entry.type = "number";
}, TypeError);
const after = reference("snapshot").methods[0];
assert.equal(after.input_schema.properties.entry.type, "string");
assert.throws(
() => reference("values", { names: { snapshot: "values" } }),
/Ambiguous/,
);
assert.throws(
() => reference("values", { names: { unknown: "facts_values" } }),
/Unknown/,
);
});
@@ -0,0 +1,95 @@
# Savepoint format/control oracle
Pinned before model trials; not provided to measured models. This question gives
a function name and messages, not entry, instruction, graph or value addresses.
It is still a scoped local investigation, not general unseeded reconstruction.
Both atomic adapters share the same native facts and authority. No traditional
provider cell exists for this question, whose required graph identities are not
a fair provider-independent outcome for such a cell.
Target: Debian SQLite `3.40.1-2+deb12u2` amd64,
`/usr/lib/x86_64-linux-gnu/libsqlite3.so.0` -> `.0.8.6`, SHA-256
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
Preserved engine: `677b9795d49a65e459054ecbd3c0df37d603b541d72df9f74b46778947ed3936`.
The exact-name inventory has entries `0x288f0` (16 bytes) and `0xea6d0` (32,384
bytes), with identical names and no aliases. The inventory alone does not label
PLT versus implementation. Incoming data references from the two strings identify
the owning `sqlite3VdbeExec` entry `0xea6d0`; the CLI reproducer verifies this
composition. Do not select by catalog order or claim the name establishes binding.
| Format | Address / NUL-inclusive bytes | Producer / block | Merge input |
| ----------------------------------------------------- | ----------------------------- | ------------------- | -------------------------- |
| cannot open savepoint - SQL statements in progress | 0x12c500 / 51 | 0xf2381:712c / 1142 | 0, v100012a75, edge 1142:0 |
| cannot release savepoint - SQL statements in progress | 0x12c538 / 54 | 0xf2437:753c / 1158 | 1, v10001352e, edge 1158:0 |
Both `CPUI_PTRSUB` format producers use constant-zero input 0 and the eight-byte
address constant at input 1. Both feed PHI `0xf238b:ceb7` in block 1159, output
`v100033f25`, passed in RSI (eight-byte register storage at `0x30`) at recovered
input index 2 of shared CALL `0xf238b:7130`. The native target is `ram:0x296c0`,
label `sqlite3VdbeError`, independently verified as the machine direct CALL target;
runtime binding and complete source signatures remain unknown.
## Local recovered conditions
| Setup | Controller / CBRANCH | True edge / destination | False edge / destination | Condition |
| ------------------- | ------------------------ | ----------------------- | ------------------------ | ------------------- |
| open, block 1142 | block 1141, 0xedc01:7065 | 1141:1 -> 1142 | 1141:0 -> 1143 | v100012864, ne-zero |
| release, block 1158 | block 1157, 0xf0389:71b8 | 1157:1 -> 1158 | 1157:0 -> 1161 | v100012bd9, ne-zero |
The open condition is one-byte `v100012864`, defined by
`0xedc01:7064 CPUI_INT_SLESS`: input 0 is four-byte constant zero
`v30001285f`; input 1 is four-byte `v100012840`, defined by
`0xedbf8:7056 CPUI_LOAD`. Thus the local recovered guard is a **signed** positive
loaded 32-bit value, not an unsigned nonzero test.
The release condition is one-byte `v100012bd9`, defined by
`0xf0384:71b3 CPUI_INT_EQUAL`: input 0 is four-byte `v10001280a`, defined by
`0xedbe5:7041 CPUI_LOAD`; input 1 is four-byte constant one `v3000393a3`.
Its local guard is equality to one. Both load contents and aliases remain unknown.
Models need not invent source member names for either memory object.
Derived controllers return precisely the two relations above for dependents
1142/1158 in the actual retained-source replay. Coverage is complete for finite
paths to recovered zero-successor exits; nontermination and runtime reachability
remain unknown. These are direct/local control relations, not all transitive
earlier guards or proof of feasible execution.
## Independent selected machine evidence
GNU objdump over the same ELF independently supplies these bytes/transfers;
ELF file/virtual mappings also verify their raw bytes. The complete recovered
graph, including these identities, deeply equals the earlier diagnostic native
golden. That producer comparison is not independent machine/IR equivalence.
| Address | Bytes | Machine fact |
| ------- | -------------- | ------------------------------- |
| 0xedbe5 | 8b7004 | MOV ESI,[RAX+4] |
| 0xedbec | 89742440 | MOV [RSP+0x40],ESI |
| 0xedbf8 | 418b9dd8000000 | MOV EBX,[R13+0xd8] |
| 0xedbff | 85db | TEST EBX,EBX |
| 0xedc01 | 0f8f6b470000 | JG 0xf2372; fallthrough 0xedc07 |
| 0xf0384 | 837c244001 | CMP dword [RSP+0x40],1 |
| 0xf0389 | 0f849e200000 | JE 0xf242d; fallthrough 0xf038f |
| 0xf2381 | 488d3578a10300 | LEA RSI,[0x12c500] |
| 0xf238b | e83073f3ff | CALL 0x296c0 |
| 0xf2437 | 488d35faa00300 | LEA RSI,[0x12c538] |
| 0xf2446 | e940ffffff | JMP 0xf238b |
The earlier load at `0xedbe5` and spill at `0xedbec` correspond to the native
value later compared at `0xf0384`; selected bytes do not prove all intervening
memory/runtime behavior. Machine setup/branch windows and native optimized CFG
edges are separate evidence representations. PHI is an IR operation at a shared
origin, not a machine instruction.
Expected recovered coverage is now a full block-member graph: 21,349 operations,
35,755 values, 1,229 blocks and zero omitted definition IDs. It includes 416
recovered calls; 395 known CALL targets and 21 unresolved CALLIND targets. This
does not establish full machine-flow/call coverage or memory alias resolution.
The previous call-rooted projection lacked both CBRANCH/Boolean definitions and
the open-path LOAD, as well as all CFG access; those absences remain recorded.
Grade the complete answer and actual trace, not keywords alone: both formats and
shared call/PHI alternatives, both edges for each local condition, precise Boolean
and memory widths/signedness, focused machine checks, real source IDs/digest,
scope/earlier-guard/alias/nontermination/runtime limits, and retained-capture reuse.
@@ -0,0 +1,169 @@
# Real-agent savepoint control investigation
The [question and oracle](savepoint-control-oracle.md) were committed at
`e2205313a` before either model launch. Both lanes have the same atomic facts,
target, engine, question and common envelope/stopping guidance. They start with
the function name and two messages, without supplied code addresses. This tests
the newly available full recovered graph, not provider parity or superiority
over a traditional RE interface.
## Initial pair: natural discovery
Both complete answers pass the frozen source/answer review:
- Incoming string references select `0xea6d0` from two same-name entries before
one retained VM capture. Neither lane fetches the complete C report or graph.
- Both format pointers merge through `0xf238b:ceb7` and reach CALL
`0xf238b:7130`, with the correct predecessor/edge bindings and RSI storage.
- Each answer supplies both successor edges for each nearest local conditional
predecessor. The open guard is signed four-byte `0 < loaded32`; release is
four-byte equality to one. Both trace one-byte Booleans to the right LOADs,
preserve operand indexes/widths and stop before guessing loaded contents.
- Focused instruction windows check loads, the spill, signed JG/equality JZ,
format setups and shared machine CALL. Independent GNU/ELF verification checks
eleven instruction sites against the same target bytes. PHI/edge identities
remain separate from machine instructions and transfers.
- Real source IDs and the target digest are present. Each answer limits itself
to recovered graph/local guards and states machine-flow, alias,
nontermination, earlier-path and runtime-feasibility limits. Neither uses
derived controllers or claims a SQL trigger. Both close the lease.
Each answer cites ten observed source IDs. Both have one successful VM capture,
whose recovered graph equals the complete producer golden; facts come from the
preserved `677b9795...` engine. Full answers, events, tool context, traces and
resource receipts are retained locally under
`.cache/atomic-codemode/sqlite-savepoint-control-{direct,codemode}-v1*`.
| Whole-task measure | Direct atomic | Code Mode |
| --------------------------------- | ------------: | --------: |
| Cumulative input tokens | 515,024 | 610,715 |
| Included cached input | 466,304 | 563,328 |
| Uncached input | 48,720 | 47,387 |
| Output tokens | 5,537 | 6,345 |
| Included reasoning output | 1,344 | 1,502 |
| Observed model responses | 17 | 18 |
| MCP attempts, including rejection | 24 | 15 |
| Successful atomic fact reads | 20 | 21 |
| Model wall seconds | 206.269 | 223.636 |
| Family wall seconds | 207.427 | 224.133 |
| Family CPU seconds | 53.794 | 52.777 |
| Kernel process-family peak MiB | 874.9 | 883.2 |
Code Mode's fifteen attempts include a rejected `timeout_ms:200000` request;
the actual schema maximum is 120,000. It retries within that bound. The client
also initially tries to JSON-parse the validation error's text and gets a
SyntaxError. Both costs remain in the ledger. There are no recorded inner-API
failures or resource aborts. Direct has no failed MCP attempt. MCP attempts,
successful host reads, client execution calls and model responses are distinct
counts; the trace has no handler entry for the schema rejection.
In this one sample, Code Mode reduces uncached input by about 2.7% but increases
cumulative input by 18.6%, output by 14.6% and model wall time by 8.4%. Fewer MCP
requests do not establish a whole-task advantage. The lanes also choose slightly
different fact selections and output detail. This is an observational pair,
not a controlled estimate of a single dispatch mechanism's causal effect.
The first client discovery call in both lanes applies a broad `rea.*...` regular
expression to all tool names/descriptions. It matches unrelated tools, including
creation tools, and prints large, truncated metadata. This is a real observed
agent cost, not an analysis-provider failure; it also adds noise to comparing
the two interfaces. A repeat should give both lanes the same exact-name discovery
policy while retaining every advertised tool and charging that guidance to the
whole task. No analysis steps, addresses or answers should be added.
Both runs request `gpt-6.1-sol/high` through Codex 0.161.0; the resolved service
build is unknown. Per-response usage sums match the runner ledger. Cached input
is already included in input, and reasoning is included in output. One CPU,
4 GiB hard / 3 GiB early-stop limits apply. Kernel peak includes charged file
cache; it is not RSS. Both runs have OOM 0, verified payload/profile cleanup,
unchanged runtime fingerprints, engine and target digests. Each lease is fresh;
OS/provider-service caches and remote compute costs are uncontrolled/unmeasured.
## Discovery-controlled repeat
Executed at `7a7c4a6d7`, Code Mode first, then direct. The repeat keeps the same
question, oracle, native engine and all atomic methods, and uses the runner's existing
`--enabled-tools` / `--guide-enabled-tools` options with server name `rea`.
Each manifest lists that lane's complete catalog: seventeen direct tools or four
Code Mode tools. No analysis method is removed. Captured first client calls
select these exact function names; neither repeats the broad metadata spill.
This changes discovery guidance and namespace together for both lanes; it is a
separate stratum from the natural-discovery pair, not a retroactive replacement.
Both complete answers again pass the same oracle: correct format ownership,
PHI/call and native edges, signed-positive versus equality guards, Boolean/load
widths and operand indexes, independent selected machine bytes, real provenance
and explicit static/alias/runtime limits. Both use one retained VM capture and
close the lease. Direct cites ten observed source IDs; Code Mode cites nine,
combining some instruction windows. All recovered graph and ordinary function
fields equal the earlier full golden in both lanes. Neither fetches a whole C
report or graph. Neither uses derived controllers; both keep local guards
separate from all earlier conditions and runtime feasibility.
| Whole-task measure | Direct atomic | Code Mode |
| ------------------------------ | ------------: | --------: |
| Cumulative input tokens | 538,013 | 608,622 |
| Included cached input | 457,088 | 530,688 |
| Uncached input | 80,925 | 77,934 |
| Output tokens | 5,150 | 5,989 |
| Included reasoning output | 697 | 1,462 |
| Observed model responses | 19 | 25 |
| MCP attempts | 24 | 13 |
| Successful atomic fact reads | 20 | 18 |
| Model wall seconds | 187.544 | 266.141 |
| Family wall seconds | 188.086 | 266.650 |
| Family CPU seconds | 52.113 | 102.739 |
| Kernel process-family peak MiB | 859.1 | 883.1 |
| VM source-capture wall seconds | 37.026 | 107.804 |
There are no failed MCP attempts, recorded inner-API failures or resource aborts.
Per-response usage sums agree; OOM is zero, cleanup is verified, and target,
engine and runtime fingerprints remain unchanged. Full records use suffix `-v2`
under the same local artifact directory. Request/model settings and resource
bounds are unchanged from the first pair.
Code Mode again has slightly lower uncached input (3.7%) and fewer MCP requests,
but more cumulative input, output, model responses, wall time and family CPU in
this sample. It has twelve client waits versus three for direct. Discovery
guidance alone has not established an overall improvement across strata.
The VM capture is a substantial confound: its 107.804 seconds differs from
37.026 in direct and 38.013/38.871 in the first pair. Source elapsed time includes
command supervision and retention; it is not an isolated native drive interval.
No per-command CPU or phase diagnostic was captured during these model trials.
The additional instruction-window API call overlapped the pending Code Mode
snapshot; its own wall time also includes waiting beyond the retained native
source times. Whether concurrency, host contention or another lower boundary
caused the variation is unresolved. CPU affinity/quota do not give an exclusive
core. Do not subtract these intervals from whole-task time to manufacture a
Code Mode speedup or attribute the difference to dispatch alone.
Both lanes actually use the client's `exec`/`wait` JavaScript surface, despite
the recorded client Code Mode feature flags being false. Direct atomic already
composes MCP calls through that client executor. These results compare a server
Code Mode boundary against directly called atomic MCP tools in that environment;
they do not compare JavaScript composition against its absence. Tool counts and
configured feature flags cannot establish the client's actual available surface.
## Remaining questions
Four real runs establish this scoped control/data-flow task on the full recovered
graph, without a stable overall efficiency win. There is no traditional-provider
cell for this graph-identity question and no general complex-software superiority
claim. Whole-task cumulative input across all four runs is 2,272,374 tokens,
including 2,017,408 cached and 254,966 uncached; output is 23,021 including 5,005
reasoning tokens. Total model wall is 883.590 seconds. These totals retain the
initial failed request and discovery costs, not a pooled treatment estimate.
The retained graph also exposes the next coverage question: `0xea801:fe` is one
`CPUI_BRANCHIND` in block 8 with 149 recovered successors. Current edges expose
identities/destinations but no case values. The native jump-table owner should
be examined before exporting case-to-edge/target relations; a successor ordinal
is not a case value. This is a fact-export limitation, not evidence that Kuna
itself cannot recover the labels. The source-capture/concurrent-read cost
variation also needs a bounded, matched diagnostic before further speed claims.
The [subsequent four-case public MCP diagnostic](overlap-study.md) confirms
provider admission is serialized and does not reproduce the long capture;
the original cause remains unknown rather than being declared fixed.
The [native switch extension](switch-study.md) now exports the case labels and
checks all 182 original targets independently; its new model question is separate.
Loaded 100 of 619 files, more files were not shown because too many files have changed in this diff. Show more