test(experiment): freeze client wait guidance diagnostic

Keep identical analysis contracts, question, engine and complete catalogs. Test common opt-in wait guidance while retaining polling and native latency in whole-task costs.

Co-authored-by: gensokyo-dev <342580529+gensokyo-dev@users.noreply.github.com>

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

experiments/analysis-efficiency/agent-trial.py

@@ -99,6 +99,8 @@ def main():
                         help='Opt-in exact-name discovery guidance for selected REA tools')
     parser.add_argument('--normalize-mcp-guidance', action='store_true',
                         help='Opt-in common envelope/stopping guidance, identical across protocol lanes')
+    parser.add_argument('--guide-long-waits', action='store_true',
+                        help='Opt-in common client wait guidance; does not change analysis contracts or deadlines')
     args = parser.parse_args()
     if args.guide_enabled_tools and args.enabled_tools is None:
         parser.error('--guide-enabled-tools requires --enabled-tools')
@@ -235,6 +237,15 @@ def main():
             'return only their metadata. Preserve the original analyst question, '
             'dependency order and source Evidence.'
         )
+    if args.guide_long_waits:
+        instruction += (
+            ' When the client executor reports a running cell during a long analysis, '
+            'resume that same cell with wait and yield_time_ms:30000, within the '
+            'advertised tool schema, rather than repeated one-second polls. '
+            'The wait may return early when the result is ready. Do not sleep or '
+            'start duplicate analysis calls just to check completion. Keep the '
+            'original analysis dependencies, evidence and stopping requirements.'
+        )
     quote = json.dumps
     config = '\n'.join([
         'approval_policy = "never"', 'sandbox_mode = "read-only"',
@@ -290,6 +301,7 @@ def main():
             'capture_tool_context': args.capture_tool_context,
             'guide_enabled_tools': args.guide_enabled_tools,
             'normalize_mcp_guidance': args.normalize_mcp_guidance,
+            'guide_long_waits': args.guide_long_waits,
             'driver_commit': subprocess.check_output(
                 ['git', 'rev-parse', 'HEAD'], cwd=driver_repository, text=True).strip(),
             'runtime_revision': args.revision,

experiments/atomic-codemode/README.md

@@ -294,4 +294,8 @@ orchestration. Source fidelity is not machine equivalence, provider parity or a
 general speedup. The [native switch experiment](switch-study.md) now exports and independently checks
 182 labels over those 149 targets, exposes selected case-to-edge relations and
 keeps them readable when a 2 MiB native output budget omits the full graph.
-New real-model case/default verification is pending.
+Both real models now answer the selected-case/default question correctly using
+one retained capture and focused case/byte reads. Code Mode uses fewer tokens
+in this pair, but unequal native capture time and extra client polling confound
+the comparison. Whole-task costs and limits are in the same study; a general
+efficiency advantage remains unproved.

experiments/atomic-codemode/client-wait-study.md

@@ -0,0 +1,51 @@
+# Client waiting and whole-task cost
+
+The [selected-case pair](switch-study.md) exposes one-second client polls after
+the snapshot execution yields. Direct makes six waits (three one-second then
+three ten-second requests); Code Mode makes three one-second requests. Five
+direct and two Code Mode wait responses still report a running cell. Initial
+execution yields after about 31 seconds in both lanes. This is observed client
+behavior, separate from MCP attempts and native source-capture time.
+
+Native latency remains variable, including in sequential scripted MCP runs
+without models. The [overlap canary](overlap-study.md) did not establish a cause.
+No analysis acceleration or cache invalidation change is proposed here.
+
+## Frozen diagnostic
+
+The opt-in runner flag `--guide-long-waits` appends exactly the same client wait
+policy in both lanes: resume an already-running cell with a 30,000 ms wait,
+respect the advertised schema, and do not sleep or duplicate analysis merely
+to check progress. It changes neither the initial execution yield policy nor
+the provider/Code Mode deadlines. It supplies no analysis facts. The policy,
+flag and instruction digest are recorded in each trial's settings. Older runs
+without this flag remain a separate stratum.
+
+Run Code Mode first, then direct, with the unchanged
+[question](sqlite-switch-question.txt) and [oracle](switch-oracle.md), final
+`a9ca8a8a...` engine, complete four/eighteen-tool catalogs and the same exact
+discovery/envelope guidance as the first pair. Use fresh leases and isolated
+model workspaces, requested `gpt-6.1-sol/high`, Codex 0.161.0 and the same one-CPU,
+4 GiB hard / 3 GiB early-stop resource policy. No runtime edits during the pair.
+Local records use `sqlite-switch-{codemode,direct}-v2*`.
+
+Review complete answers against the frozen independent machine oracle and
+retained source, including selector widths, all requested cases, signed target
+arithmetic, case/edge identities, native default versus machine out-of-range
+guard, provenance and unknown runtime behavior. Require one VM capture and
+focused reads; a lower token count without a correct answer is not a win.
+
+Record actual wait arguments and running/completed responses, model responses,
+all input/cached/uncached/output/reasoning tokens, MCP/API attempts, wall time,
+family CPU, kernel family peak, native source-capture time, identity/freshness
+and cleanup/OOM status. Count discovery, guidance and unsuccessful waits in
+whole-task usage. Fewer polls would establish adherence in this client, not
+causally lower tokens or native latency. A before/after pair remains subject to
+model choices, prompt caches, host contention and run order; do not subtract
+native or polling intervals to manufacture an adapter speedup.
+
+The [official Codex CLI reference](https://learn.chatgpt.com/docs/cli/reference)
+documents client configuration but does not establish the `wait` contract used
+here. The actual captured client calls and subsequent run establish behavior;
+no configuration feature flag is treated as proof of the available surface.
+The native/API contracts are unchanged. Results are pending.

experiments/atomic-codemode/switch-study.md

@@ -128,4 +128,79 @@ source-byte/normalized-selector widths, signed relative target arithmetic and
 the native-default versus machine-guard distinction, starting from function
 name/messages. Both lanes get the full catalog and the same exact discovery,
 envelope and stopping guidance; reverse engineering facts are not precomputed
-for the measured models. Real-model results are pending.
+for the measured models.
+
+## Real-model selected-case investigation
+
+The question and oracle were frozen at `f44dca9c3` before both launches. Direct
+ran first, then Code Mode, with the preserved final engine above, fresh leases,
+and isolated model workspaces outside the repository. Both receive their full
+catalog: eighteen direct tools or four Code Mode tools. Exact-name discovery
+guidance is identical in policy, with no precomputed analysis facts.
+
+Both complete answers pass the frozen oracle/source review:
+
+- Two format references select implementation `0xea6d0` over the other callable
+  entry before a single retained VM capture. Neither fetches a full case
+  inventory, C report or recovered graph.
+- The one-byte loaded selector, eight-byte normalized ZEXT and computed machine
+  jump address remain distinct. Both verify unsigned `JA`, the table base and
+  signed little-endian four-byte displacement arithmetic with selected bytes.
+- All five requested in-range labels have the correct original row, native
+  edge/block and recovered target. Labels `0x34`/`0x39` retain distinct case IDs
+  despite sharing a destination/edge. Querying six labels returns five rows;
+  `0xb6` has no captured explicit case in this nonpartial table.
+- Both distinguish native default `8:42` / `0xeacf0` from the machine guard
+  destination `0xea998`, and do not equate default index 42 with label `0x2a`.
+- Real source IDs and the target digest support the answers. Both preserve
+  machine-flow, memory-alias, nontermination and runtime-feasibility limits and
+  close the lease. Direct cites twelve observed source IDs; Code Mode cites nine.
+
+Both retained VM sources decode to the full prior graph golden, with ordinary
+C/metadata/calls unchanged. This establishes producer fidelity separately from
+the independent selected machine checks; it does not prove global equivalence.
+The 3,818,818-byte native document stays outside model context. Full answers,
+events, source Evidence, client tool context and resource receipts are retained
+locally under `.cache/atomic-codemode/sqlite-switch-{direct,codemode}-v1*`.
+
+| Whole-task measure             | Direct atomic | Code Mode |
+| ------------------------------ | ------------: | --------: |
+| Cumulative input tokens        |       423,153 |   335,309 |
+| Included cached input          |       367,104 |   308,352 |
+| Uncached input                 |        56,049 |    26,957 |
+| Output tokens                  |         5,135 |     4,947 |
+| Included reasoning output      |         1,311 |       964 |
+| Observed model responses       |            18 |        15 |
+| Client execution / wait calls  |        11 / 6 |    11 / 3 |
+| MCP attempts                   |            17 |        13 |
+| Successful atomic fact reads   |            13 |        14 |
+| Model wall seconds             |       220.581 |   180.656 |
+| Family wall seconds            |       222.275 |   181.173 |
+| Family CPU seconds             |        93.003 |    57.957 |
+| Kernel process-family peak MiB |         722.4 |     782.3 |
+| VM source-capture wall seconds |        78.026 |    41.741 |
+
+There are no failed MCP/API attempts or resource aborts. Per-response usage
+sums match the ledger. Both request `gpt-6.1-sol/high` through Codex 0.161.0;
+the resolved service build and remote compute costs are unknown. Cached input
+is included in input; reasoning is included in output. Runtime, engine and
+target fingerprints remain unchanged. One CPU, 4 GiB hard / 3 GiB early-stop
+bounds apply, with OOM 0 and verified payload/profile cleanup. Peaks include
+charged file cache, not RSS. Leases/provider caches start fresh; OS caches,
+model prompt caches and host contention are uncontrolled.
+
+Code Mode uses 20.8% less cumulative input, 51.9% less uncached input and 18.1%
+less model wall time in this one sample. Native capture takes 78.0 versus 41.7
+seconds, a substantial confound already observed without models or overlap.
+The lanes also select different details and discovery contracts. Both clients
+already compose calls through their JavaScript executor. This is not evidence
+of a general Code Mode speedup or traditional-provider superiority/parity.
+
+Client waiting is another observed whole-task cost: each initial snapshot
+execution yields after about 31 seconds. Both then request one-second waits;
+direct tries three before switching to longer waits, while Code Mode completes
+on its third. A slower capture can therefore add model responses and token
+usage as well as wall time. Those costs remain in the table; subtracting them
+would not produce a controlled comparison. The next diagnostic should test
+the same longer-wait guidance in both lanes without changing the question,
+analysis facts or completeness requirements.