Compare commits

..
Author SHA1 Message Date
Kaoru0822 98b87e7fd9 Merge remote-tracking branch 'origin/main' into sweep/2644
# Conflicts:
#	tests/process-global/cleanLinuxHopperRuntime.test.ts
2026-10-15 07:30:06 +00:00
morluto 4c99912e8d chore(main): merge pinned CI checkpoint 2895a18 2026-10-14 03:12:47 +08:00
morluto b3a60b6b03 test(ci): provide proc metadata for Linux hopper fixtures 2026-10-14 03:02:01 +08:00
morluto d5b5441a3a fix(hopper): refresh runtime cleanup against main 2026-10-14 02:27:23 +08:00
morluto 4f83c64da8 fix(hopper): merge current main into runtime cleanup 2026-10-14 02:12:58 +08:00
Morluto 780191226c test(hopper): run Python-named fixture through Node eval 2026-10-14 02:03:25 +08:00
morluto 1738989590 test: isolate hopper scanner process fixtures 2026-10-14 01:26:30 +08:00
morluto 3f18439acd fix(types): declare hopper inventory proc root 2026-10-14 01:24:48 +08:00
morluto dd401a2804 test(process): isolate hopper inventory proc root 2026-10-14 01:06:02 +08:00
morluto 2bc898fae2 chore: merge captured main into Hopper runtime cleanup 2026-10-13 21:45:06 +08:00
N0zoM1z0andKaoru0822 b2e743ef24 test(hopper): reap orphan fixtures independently of container init
Keep a test-owned Linux subreaper until token-verified process-group cleanup completes, then wait for each adopted child. Observe from a separate verifier so the orphan remains outside its lineage. Assert disappearance from procfs after reaping.

Reproduced two failures and eight retained zombies without an init reaper. The corrected fourteen process cases pass with no residual children; typecheck and lint pass.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 12:25:38 +00:00
morluto 51b3418f2b chore: merge upstream CI baseline fixes 2026-10-13 20:13:27 +08:00
N0zoM1z0andKaoru0822 3bb09b8d2d fix(verification): retain command-less runtime observations
Use the kernel command name for processes without argv or an executable link. Preserve uncertainty for command-less Python interpreters and detect Hopper/Xvfb zombies without signaling. Real unreaped-child regressions keep cleanup with their waiting parent.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
N0zoM1z0andKaoru0822 3deda4781d fix(hopper): parse Python helper interpreter options
Resolve the script operand using CPython option boundaries, including combined flags, separate or attached option values, and --. Keep command/module arguments distinct and defer unknown invocation failures rather than declaring them clean.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
N0zoM1z0andKaoru0822 2ac4795df5 test(hopper): cover a reparented Python helper command
Launch a real Python child with the helper filename, wait for readiness before the Node parent exits, bind its procfs arguments, require non-signaling detection, and clean it through the existing run-token/group identity primitive.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
N0zoM1z0andKaoru0822 a588dc6376 fix(hopper): distinguish runtime processes from argument text
Parse procfs argument boundaries rather than searching rendered ps command text. Keep executable identity checks and deferred permission failures, eliminate the extra ps process, and put shared process observations in the serial process-boundary lane.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-13 10:26:57 +00:00
663 changed files with 7052 additions and 51618 deletions

No files matched your search

@@ -59,7 +59,7 @@ availability have different remedies; avoid repeating setup for all of them.
absent, inspect the client's MCP launch error rather than repeating setup.
REA setup never installs or upgrades Node.js, npm, Homebrew, Java, Ghidra, IDA,
JADX, Kuna, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
JADX, Binwalk, or Unblob. Use existing prerequisites; do not install unrelated
software to repair
MCP registration. For an unsupported client, use manual stdio registration
with a version-pinned `rea-agents` package or continue through the CLI.
@@ -79,7 +79,7 @@ is within the authorized scope, use `npx -y rea-agents@latest` in its place.
Configuration repair starts with a read-only setup plan; show paths, backups and
skill replacement before applying authorized changes. Preserve existing approval
and target-execution boundaries. REA setup never installs or upgrades unrelated
software, including Node.js, npm, Homebrew, Java, Ghidra, IDA, JADX, Kuna, Binwalk or Unblob.
software, including Node.js, npm, Homebrew, Java, Ghidra, IDA, JADX, Binwalk or Unblob.
## Route the target first
@@ -215,20 +215,6 @@ after opening; IDA does not supply `binary_overview`. Consult session availabili
when composing broader workflows. Modern direct callers and unsupported dossier
facets remain unknown; live IDA results are not replayed from snapshots.
With `REA_KUNA_COMMAND` configured, `provider_id: "kuna"` answers procedure
lists, lookups, pseudocode, assembly and `analyze_function` from Kuna's cached
whole-program export. Callers/callees use a separate body-free graph shared by
the session; `xrefs` returns exact-address incoming sites and filters forwarding
aliases. These queries and inventory lookups do not trigger a C export. Indirect
targets remain unknown and can make relationship coverage partial. Reference
caches expire on close; a fresh CLI process repeats the native reference walk.
Typed references, CFG, raw-memory tools and annotations remain unsupported. A cold first
answer can come from a single-function run (its limitations say so); assembly
waits for the export. For pseudocode and assembly, `raw_result.source` names the
cached `.c`/`.asm` file and byte span, which you may search directly instead of
calling per function. See the
[Kuna provider guide](https://github.com/morluto/rea/blob/main/docs/kuna-provider.md).
If the user says only "my app" without a name, path, or target already selected
for this task, ask which app before searching for artifacts. Paths in unrelated
saved reports and nearby test fixtures do not select the user's target. Resolve
@@ -94,36 +94,6 @@ recorded observations, identity, hashes, exports and source ranges; it does not
include every child syntax observation, related edge or original source file.
An absent item fact is not evidence of absence from the whole application.
For canonical identities and retained source-range counts, use a typed reference
page instead of reading every full item:
```json
{
"source": {
"kind": "retained-evidence",
"evidence_id": "ev_ACTUAL_PARENT_ID"
},
"view": {
"kind": "javascript",
"collection": "modules",
"projection": "references",
"where": { "path": "main.js", "kind": "javascript-module" },
"offset": 0,
"limit": 16
}
}
```
Filters match exact retained strings; omit a filter to include all values, use
`path: null` for unavailable paths, and `path: ""` for a recorded empty path.
Rows preserve canonical `identity` and distinct observation support states in
`basis`. `retained_source_range_count` counts retained source-range observations,
not runtime imports or unique reference sites. Use a row's `node_id` with the item
view above for exact locations and full supporting observations. Follow
`coverage.next_offset`; page exhaustion describes matching retained rows only.
Check `source_coverage` for source selection, partial/unknown graph coverage and
unknown counts before drawing a negative conclusion.
## Trace only the remaining question
Use `trace_application_feature` on the parent application Evidence for one
-13
View File
@@ -60,7 +60,6 @@ jobs:
web_runtime: ${{ steps.plan.outputs.web_runtime }}
web_source_map: ${{ steps.plan.outputs.web_source_map }}
nativeaot: ${{ steps.plan.outputs.nativeaot }}
ghidra_bridge: ${{ steps.plan.outputs.ghidra_bridge }}
website: ${{ steps.plan.outputs.website }}
matrix: ${{ steps.plan.outputs.matrix }}
test_selection: ${{ steps.plan.outputs.test_selection }}
@@ -140,11 +139,6 @@ jobs:
- name: Validate workflow syntax and expressions
if: needs.changes.outputs.workflows == 'true'
run: go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 -shellcheck= -pyflakes=
# Code tiers run this source-only policy test in their existing shards.
# Workflow-only changes need the guard, without preparing runtime artifacts.
- name: Check action pins for workflow-only changes
if: needs.changes.outputs.workflows == 'true' && needs.changes.outputs.code != 'true'
run: node scripts/run-exact-tests.mjs run tests/boundary/filesystem/workflowActionPinning.test.ts
- name: Record static task source and execution identity
continue-on-error: true
if: needs.changes.outputs.code == 'true'
@@ -747,7 +741,6 @@ jobs:
tests/boundary/filesystem/javascriptApplicationWorkflows.test.ts
tests/boundary/filesystem/webScriptManifestPaths.test.ts
tests/boundary/process/ProcessEvidence.host.test.ts
tests/boundary/process/packageNpmInvocation.test.ts
tests/boundary/process/windowsOutputCapture.test.ts
tests/acceptance/webModuleCli.test.ts
tests/acceptance/webSourceLocationCli.test.ts
@@ -905,11 +898,6 @@ jobs:
if: needs.changes.outputs.nativeaot == 'true'
uses: ./.github/workflows/nativeaot-fixtures.yml
ghidra_bridge:
needs: [changes, static]
if: needs.changes.outputs.ghidra_bridge == 'true'
uses: ./.github/workflows/ghidra-bridge-sources.yml
website:
needs: [changes, static]
if: needs.changes.outputs.website == 'true'
@@ -945,7 +933,6 @@ jobs:
web_runtime,
web_source_map,
nativeaot,
ghidra_bridge,
website,
]
runs-on: ubuntu-24.04
@@ -1,48 +0,0 @@
name: Ghidra bridge source compilation
on:
workflow_dispatch:
workflow_call:
permissions:
contents: read
env:
HUSKY: "0"
jobs:
ghidra-bridge-sources:
name: Compile packaged Ghidra bridge sources
runs-on: ubuntu-24.04
timeout-minutes: 15
env:
GHIDRA_RELEASE_URL: https://github.com/NationalSecurityAgency/ghidra/releases/download/Ghidra_12.1.4_build/ghidra_12.1.4_PUBLIC_20260921.zip
GHIDRA_RELEASE_SHA256: ddac49f903da9d5bac833e5cc79395098b9c33cfd3279be5f31bd00387d2d4db
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version-file: .nvmrc
- name: Validate the SDK pin against the provider's verified release
run: node scripts/verify-ghidra-bridge-sources.mjs --check-sdk-pin >> "$GITHUB_ENV"
# Only the jars and application.properties are extracted: the check
# compiles against them and never launches Ghidra or imports a target.
- name: Acquire checksum-verified Ghidra jars in isolated runner storage
shell: bash
run: |
ghidra_root="${RUNNER_TEMP}/rea-ghidra"
mkdir -p "${ghidra_root}"
curl -fsSL --retry 3 --write-out 'ghidra_download_bytes=%{size_download} ghidra_download_seconds=%{time_total}\n' -o "${ghidra_root}/ghidra.zip" "${GHIDRA_RELEASE_URL}"
echo "${GHIDRA_RELEASE_SHA256} ${ghidra_root}/ghidra.zip" | sha256sum -c -
unzip -q "${ghidra_root}/ghidra.zip" '*/Ghidra/*.jar' '*/Ghidra/application.properties' -d "${ghidra_root}"
rm "${ghidra_root}/ghidra.zip"
properties="$(find "${ghidra_root}" -path '*/Ghidra/application.properties' -print -quit)"
test -n "${properties}"
echo "GHIDRA_INSTALL_DIR=$(dirname "$(dirname "${properties}")")" >> "${GITHUB_ENV}"
# JAVA_HOME_<major>_X64 comes from the runner image, not the env context.
- name: Compile bridge sources with the verified runner JDK
shell: bash
run: |
java_home_variable="JAVA_HOME_${GHIDRA_JAVA_MAJOR}_X64"
test -n "${!java_home_variable}"
JAVA_HOME="${!java_home_variable}" npm run verify:ghidra:bridge-sources -- --verified-sdk
+3 -3
View File
@@ -22,7 +22,7 @@ jobs:
sudo apt-get update
sudo apt-get install --no-install-recommends -y g++-mingw-w64-x86-64
- run: npm run build:windows-native
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
- uses: actions/upload-artifact@v4
with:
name: cutter-windows-native
path: native/windows/build/
@@ -53,7 +53,7 @@ jobs:
uses: ./.github/actions/npm-cache
- run: npm ci
- if: matrix.platform == 'Windows'
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
uses: actions/download-artifact@v4
with:
name: cutter-windows-native
path: native/windows/build/
@@ -67,7 +67,7 @@ jobs:
env:
EXPECTED_PLATFORM: ${{ matrix.platform }}
run: npm run verify:cutter
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
- uses: actions/upload-artifact@v4
if: always()
with:
name: cutter-upstream-${{ matrix.platform }}
-2
View File
@@ -20,8 +20,6 @@ Give each interpretation and resource a clear owner. Consumers must use that own
Keep proof of ownership separate from validity, editability, and readiness. Malformed owned data may still be safely removable. Unresolved identity or applicability must remain unknown rather than be treated as absent, foreign, or aligned.
For lifecycle changes, follow [ownership and completion invariants](docs/tool-design.md#own-the-lifecycle) and select the applicable [lifecycle regressions](docs/testing.md#lifecycle-regressions). Include late handshakes, cancellation during final cleanup, mixed child outcomes, and cleanup retry where those stages exist; active-execution cancellation alone does not cover finalization.
Assess brittleness in touched code during implementation, debugging, review, and maintenance. Fix the producing representation or ownership rule before adding downstream repairs. Normalize once at the owning boundary using format-aware parsers. Remove superseded representations, aliases, wrappers, and redundant tests after verifying affected consumers. Introduce abstractions only when they remove observed complexity.
When fixing a bug, inspect the producing boundary and affected consumers for duplicated interpretation or unclear ownership. Complete any directly related refactor in the same change and remove superseded code. Passing regression tests alone does not establish that the brittleness is resolved. If related cleanup remains, explain what remains and why.
+1 -2
View File
@@ -205,8 +205,7 @@ or update either provider.
Native formats and host support vary by provider. See
[Hopper and Ghidra setup](docs/installation.md#hopper), the
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), the
[Kuna decompiler guide](docs/kuna-provider.md), and
[IDA guide](docs/ida-provider.md) (including an optional native MCP bridge), and
[experimental Windows Ghidra support](docs/windows-ghidra-p0.md).
Ghidra also supports [16-bit DOS analysis](docs/ghidra-dos.md).
For large binaries, raise the provider's startup deadline with
-1
View File
@@ -58,7 +58,6 @@ pin inspection sources; they are not included in the npm package.
| ghidra-nativeaot | [MIT](https://github.com/Washi1337/ghidra-nativeaot/blob/effeb734fc570c32650f88b159608979dc7b423e/LICENSE.md) | Pinned inspection source; derived parser attribution above. |
| JADX | [Apache-2.0](https://github.com/skylot/jadx/blob/v1.5.6/LICENSE) | The audited jadx-headless-mcp source uses JADX 1.5.6 for Android decompilation; the engine JAR is separately supplied. |
| jadx-headless-mcp | [Apache-2.0](https://github.com/1013503897/jadx-headless-mcp/blob/5844800a486d2248fa949b74c9896285f5b54de2/LICENSE) | v0.7.1; pinned submodule and user-supplied JAR. |
| Kuna | [Apache-2.0](https://github.com/Noelo-Lab/kuna/blob/v1.833/LICENSE), with its [NOTICE](https://github.com/Noelo-Lab/kuna/blob/v1.833/NOTICE) | Bring your own `kuna`; REA verified 1.833 and never downloads, installs or bundles it. REA invokes it as a separate process. |
| Apktool | [Apache-2.0](https://github.com/iBotPeaches/Apktool/blob/master/LICENSE.md) | User-supplied Android resource decoder. |
| Binwalk | [MIT](https://github.com/ReFirmLabs/binwalk/blob/4fdab3d464d97b68e0af9088df3f9e2e1545b21c/LICENSE) | v3.1.0; pinned submodule and user-supplied firmware engine. |
| Unblob | [MIT](https://github.com/onekey-sec/unblob/blob/1fcc7a0a584a70a96c31f5a276c20944d199a089/LICENSE) | 26.6.4; pinned submodule and user-supplied firmware engine. |
+1 -9
View File
@@ -1057,10 +1057,6 @@ def _serve_connection(connection):
):
raise PermissionError("Invalid bridge capability")
authenticated = True
# The connection grace period only bounds unauthenticated idle
# clients. Native analysis and a connected owner's idle time
# remain unbounded after capability validation.
connection.settimeout(None)
_write_message(file, {"id": request_id, "event": {
"type": "progress",
"phase": "hopper_bridge",
@@ -1152,17 +1148,13 @@ def _run():
server.bind(REA_SOCKET)
os.chmod(REA_SOCKET, 0o600)
server.listen(1)
# A bridge can start after its caller's startup/cleanup deadline. Bound
# the first accept as well as reconnects so an orphan cannot pin Hopper's
# dedicated Python thread and prevent later documents from starting REA.
server.settimeout(REA_RECONNECT_TIMEOUT_SECONDS)
try:
while True:
try:
connection, _ = server.accept()
except socket.timeout:
break
connection.settimeout(REA_RECONNECT_TIMEOUT_SECONDS)
connection.settimeout(None)
if _serve_connection(connection):
break
# Match REA's cleanup handshake budget without pinning Hopper's
+4 -48
View File
@@ -555,38 +555,6 @@ def _failed_launch_description(process):
return None
def _retire_failed_launch(process, config, listener, stdout, stderr):
"""Reconcile an SBProcess returned alongside a failed Launch error."""
if not process.IsValid():
stdout.close()
stderr.close()
return False
pid = process.GetProcessID()
if pid > 0:
# Even an already-exited process needs its PID published so the host
# can distinguish a completed launch failure from an unknown target.
with open(config["pid_path"], "w") as handle:
handle.write(str(pid))
if process.GetState() != lldb.eStateExited:
stdout.mark_launched()
stderr.mark_launched()
if pid > 0:
# Hold a surviving target until the host has captured its start
# identity. If that proof is unavailable, Kill still addresses
# this exact SBProcess created by the current launch request.
_wait_for_identity_ack(config)
process.Kill()
_wait_for_exit(listener, process, 5)
exit_observed = process.GetState() == lldb.eStateExited
expected_exit = exit_observed
stdout.close(expected_exit=expected_exit)
stderr.close(expected_exit=expected_exit)
return exit_observed
def trace(debugger, config):
global _TRACE_CONTEXT
debugger.SetAsync(True)
@@ -662,9 +630,8 @@ def trace(debugger, config):
)
process = target.Launch(_launch_info(target, config, listener), error)
if not error.Success() or not process.IsValid():
exit_observed = _retire_failed_launch(
process, config, listener, stdout, stderr
)
stdout.close()
stderr.close()
return {
"status": "launch-error",
"error": error.GetCString(),
@@ -673,26 +640,15 @@ def trace(debugger, config):
if process.IsValid()
else {}
),
**(
{"target_process_exit_observed": True}
if exit_observed
else {}
),
}
launch_failure = _failed_launch_description(process)
if launch_failure is not None:
exit_observed = _retire_failed_launch(
process, config, listener, stdout, stderr
)
stdout.close(expected_exit=True)
stderr.close(expected_exit=True)
return {
"status": "launch-error",
"error": launch_failure,
"target_process_created": True,
**(
{"target_process_exit_observed": True}
if exit_observed
else {}
),
}
stdout.mark_launched()
stderr.mark_launched()
-132
View File
@@ -15,8 +15,6 @@ fake_lldb = sys.modules.setdefault("lldb", types.ModuleType("lldb"))
fake_lldb.LLDB_INVALID_ADDRESS = -1
fake_lldb.eLaunchFlagDebug = 2
fake_lldb.eLaunchFlagStopAtEntry = 4
fake_lldb.eStateExited = 10
fake_lldb.SBEvent = lambda: object()
spec = importlib.util.spec_from_file_location(
"rea_lldb_tracer", os.path.join(os.path.dirname(__file__), "rea_lldb_tracer.py")
)
@@ -25,136 +23,6 @@ spec.loader.exec_module(tracer)
class NativeTraceBudgetTests(unittest.TestCase):
def test_failed_launch_does_not_infer_exit_from_a_failed_kill(self):
class Process:
def IsValid(self):
return True
def GetProcessID(self):
return 12345
def GetState(self):
return 6
def Kill(self):
return None
class Sink:
close_exit = None
def mark_launched(self):
pass
def close(self, expected_exit=False):
self.close_exit = expected_exit
with tempfile.TemporaryDirectory() as directory:
stdout, stderr = Sink(), Sink()
with patch.object(tracer, "_wait_for_identity_ack", return_value="unavailable"), patch.object(
tracer, "_wait_for_exit", return_value=None
):
exited = tracer._retire_failed_launch(
Process(), {"pid_path": os.path.join(directory, "pid")}, object(), stdout, stderr
)
self.assertFalse(exited)
self.assertFalse(stdout.close_exit)
self.assertFalse(stderr.close_exit)
def test_failed_launch_publishes_pid_and_retires_live_process(self):
class Process:
state = 6
killed = False
def IsValid(self):
return True
def GetProcessID(self):
return 12345
def GetState(self):
return self.state
def Kill(self):
self.killed = True
self.state = tracer.lldb.eStateExited
class Sink:
launched = False
close_exit = None
def mark_launched(self):
self.launched = True
def close(self, expected_exit=False):
self.close_exit = expected_exit
with tempfile.TemporaryDirectory() as directory:
pid_path = os.path.join(directory, "pid")
process = Process()
stdout, stderr = Sink(), Sink()
def acknowledge(config):
with open(config["pid_path"], "r") as pid_file:
self.assertEqual(pid_file.read(), "12345")
return "readable"
with patch.object(tracer, "_wait_for_identity_ack", acknowledge):
exited = tracer._retire_failed_launch(
process,
{"pid_path": pid_path},
object(),
stdout,
stderr,
)
self.assertTrue(exited)
self.assertTrue(process.killed)
self.assertTrue(stdout.launched)
self.assertTrue(stderr.launched)
self.assertTrue(stdout.close_exit)
self.assertTrue(stderr.close_exit)
def test_failed_launch_records_when_valid_process_is_already_exited(self):
class Process:
def IsValid(self):
return True
def GetProcessID(self):
return 12345
def GetState(self):
return tracer.lldb.eStateExited
def Kill(self):
raise AssertionError("an exited process must not be killed")
class Sink:
close_exit = None
def mark_launched(self):
raise AssertionError("an exited process has no output to drain")
def close(self, expected_exit=False):
self.close_exit = expected_exit
with tempfile.TemporaryDirectory() as directory:
pid_path = os.path.join(directory, "pid")
stdout, stderr = Sink(), Sink()
with patch.object(
tracer,
"_wait_for_identity_ack",
side_effect=AssertionError("exit is already observed"),
):
exited = tracer._retire_failed_launch(
Process(), {"pid_path": pid_path}, object(), stdout, stderr
)
with open(pid_path, "r") as pid_file:
self.assertEqual(pid_file.read(), "12345")
self.assertTrue(exited)
self.assertTrue(stdout.close_exit)
self.assertTrue(stderr.close_exit)
def test_pre_target_integrity_failure_reports_no_process_created(self):
original_debugger = getattr(tracer.lldb, "SBDebugger", None)
-1
View File
@@ -60,7 +60,6 @@ export default defineConfig({
collapsed: true,
items: [
{ text: "IDA", link: "/ida-provider" },
{ text: "Kuna", link: "/kuna-provider" },
{ text: "Ghidra on Windows", link: "/windows-ghidra-p0" },
{ text: "Ghidra DOS analysis", link: "/ghidra-dos" },
{ text: "Ghidra raw binary", link: "/raw-binary" },
-10
View File
@@ -108,13 +108,6 @@ The `replaced` result is `null`: the successful install response does not say
whether the package was installed fresh or replaced, and REA does not query the
device first. The install verdict and host APK digest remain available.
Push and install consume a private snapshot of the selected local input. The
reported size and SHA-256 identify those consumed bytes; `local_path` and
`apk_path` retain the original input path, whose contents may change later.
Snapshot creation needs temporary disk space proportional to the input size.
Failed snapshot cleanup remains owned and is retried before later operations
and during shutdown.
## Interpret the result
- **Directory names retain whitespace and newlines.** Symlink targets do too,
@@ -175,9 +168,6 @@ and during shutdown.
- Cancellation during a package pull preserves caller-selected output. The
error includes partial observations identifying completed artifacts and the
interrupted local path; remaining APKs were not attempted.
- Screen capture checks cancellation before publishing its exclusive output.
If cancellation arrives after publication, the error retains the committed
file's path, digest and dimensions; the file remains available as evidence.
- Bugreports collect into isolated staging and publish with exclusive creation;
filename collisions select the next numeric suffix. If publication fails, the
completed archive remains in staging. The error's partial observation gives
-18
View File
@@ -147,24 +147,6 @@ received signal alone does not establish its exact cause. Dynamic tags require
a complete DT_NULL inside PT_DYNAMIC; interpreter names and ranges end at the
first NUL, with any padding still represented by the original segment range.
## Ghidra process failures
When output from the failing Ghidra startup or operation reports
`java.lang.OutOfMemoryError`, REA returns `resource_constraint`. Java heap or GC
overhead exhaustion includes heap-specific recovery advice. Metaspace and array
size failures do not recommend increasing the heap; native-thread exhaustion is
reported as a capacity constraint with unknown limits.
`configured_jvm_max_heap` records the explicit `-Xmx` value selected by REA's
launch builder. It is a configuration fact, not measured memory use or proof of
the effective JVM limit; JVM environment options may affect that limit.
When a Windows script wrapper selects the JVM, this value remains unknown.
Historical stdout and stderr remain available, but classification uses only the
failing operation's `failure_output`. Output draining waits at most one second,
subject to earlier cancellation or the startup deadline. `output_incomplete`
marks a snapshot taken before stream closure; cleanup ownership remains with
the session until release succeeds.
## Portable PE resources
`inspect_pe_resources` / `inspect-pe-resources` inspects an explicit PE32 or
+8 -19
View File
@@ -66,25 +66,14 @@ Electron `file://` pages use a separate provider and target boundary; see [elect
## Provider authorities
CDP page capture and V8 Inspector observation attach to an already-running
target without launching it, driving actions, or evaluating target code.
Playwright scenarios drive actions and capture step snapshots in either a
provider-owned browser or a caller-selected external browser. Separate providers
preserve these authority and lifecycle differences. They share one exclusion
vocabulary (`BrowserExclusionReason`): every denial reason maps onto it, even
where a wire schema keeps a narrower historical bucket.
CDP attachment is not a guarantee that the browser preserves all state.
Chromium can clear another client's `navigator.platform` emulation when REA's
page session disconnects, even if that session sent no emulation commands.
This affects page inspection, bundle capture, session observation, screenshots,
WebMCP discovery, and Electron page inspection; target listing alone does not
attach a page session. REA cannot read the other client's complete
user-agent/client-hint configuration and does not replay guessed settings.
Use a disposable browser when preserving externally configured emulation is
required, or have the owning client reapply its complete configuration afterward.
See [the reproduction and Chromium ownership limitation](https://github.com/morluto/rea/issues/4089).
CDP page capture, V8 Inspector observation, and Playwright scenarios stay
separate providers because their authorities differ: CDP and V8 attach to an
already-running target and never launch, drive, evaluate, or mutate it, while
Playwright providers own and drive the runtime (launch browsers, run actions,
capture step snapshots). Merging them would mix attach-only and owned-process
lifecycles in one contract. What they share instead is one exclusion
vocabulary (`BrowserExclusionReason`): every denial reason in every stack maps
onto it, even where a wire schema keeps a narrower historical bucket.
Existing static application Evidence and passive web/Electron captures can be
combined later through
[JavaScript static/runtime reconciliation](javascript-runtime-reconciliation.md).
-18
View File
@@ -144,24 +144,6 @@ distinction: it closes and deletes provider-owned profiles, but only disconnects
from an external CDP browser. Real-browser verification checks both outcomes and
confirms that an attached external browser remains alive.
Keeping that browser alive does not guarantee its emulation state survives.
Chromium can clear caller-owned `navigator.platform` emulation when a page CDP
session disconnects, including sessions that sent no emulation commands. REA
cannot snapshot the other client's complete user-agent/client-hint settings and
does not replay a partial reconstruction. Use launch mode to keep this effect
inside a provider-owned browser, or have the external owning client reapply its
complete configuration after capture. Connected captures report this limitation;
see [issue #4089](https://github.com/morluto/rea/issues/4089).
`environment.locale` sets Intl formatting in both modes. Launch also sets
`navigator.language`, `navigator.languages`, and the request `Accept-Language`.
Connect preserves the caller's page and request languages; every attached
capture identifies this locale limitation, including retained partial
observations. Use launch mode when the application must select the requested
language through navigator values or request headers. Artifact completeness
covers the selected observations, not equivalence of launch and connect
language environments.
HTTP(S) URLs may include ordinary query values and fragments directly. Use
ordered structured query entries when a value needs to reference a declared
environment-backed secret. Form, storage, and cookie values are either literal
+2 -9
View File
@@ -95,9 +95,8 @@ dependency edges, and the section's file range. A missing `.dep-v0` section is
null metadata, not proof that the binary has no dependencies. See
[Rust binaries](rust-binaries.md).
Configure [Hopper or Ghidra](installation.md#hopper), the
[IDA adapter](ida-provider.md), or [Kuna](kuna-provider.md) before analyzing a
native target. Substitute
Configure [Hopper or Ghidra](installation.md#hopper), or the
[IDA adapter](ida-provider.md), before analyzing a native target. Substitute
your target, search text and function name or address in these examples:
```bash
@@ -168,12 +167,6 @@ be verified. See [MCP contracts](mcp-contracts.md) for session lifecycle and
[Ghidra first-query deadlines](mcp-contracts.md#ghidra-first-query-deadlines-and-recovery)
for import, client timeouts and recovery.
If one-shot analysis fails and closing its session also fails, the original
error details remain inline and `details.cleanup_error` describes the later
close failure. Each diagnostic retains its own reported resources; the later
failure does not establish whether an earlier resource was released. Captured
output and partial observations remain in the original details.
## Save and reuse analysis snapshots
A snapshot retains successful analysis results for later queries. REA reuses
-6
View File
@@ -2,12 +2,6 @@
REA can attach to an Electron/Chromium CDP endpoint and inspect existing `file://` renderer pages without evaluating JavaScript or invoking Electron APIs. The request supplies the endpoint directly; REA accepts only literal-port loopback HTTP endpoints. Selecting an endpoint exposes every Electron target it serves, including local file paths and page metadata.
Disconnecting the inspection's page session can clear caller-owned
`navigator.platform` emulation inside Chromium. REA cannot restore another
client's complete user-agent/client-hint configuration. Use a disposable target
or let the owning client reapply its configuration; see the shared
[CDP state limitation](browser-observation.md#provider-authorities).
This passive runtime surface is distinct from the target-free static
[`analyze_javascript_application`](javascript-artifact-reconstruction.md)
workflow. Static analysis reads a local ASAR or extracted directory without
-6
View File
@@ -69,12 +69,6 @@ Pass a returned MethodTable address to `inspect_native_data_type`. A direct PE
query also derives the snapshot metadata from the authenticated Ghidra session
handshake, so it does not require a preceding load-image call. In a persistent
MCP session, load-image followed by type inspection reuses that same derivation.
The bridge resolves the caller's address spelling once; metadata lookup uses
its canonical default-space identity. Bare hexadecimal, prefixed hexadecimal,
and encoded default-space selectors therefore select the same MethodTable.
Other address spaces and unresolved identities are not matched by offset alone.
Metadata already supplied by the engine is retained without reading or deriving
an additional source snapshot.
The database result remains `unavailable` when Ghidra has no DataType at that
address; its `metadata_recovery.source` is `read-only-derived-overlay` and
carries related type, interface, slot, and target observations. Generated
+1 -11
View File
@@ -139,12 +139,6 @@ overview merely because its source artifact is unchanged.
## Admitted function-analysis semantics
Concurrent callers share one startup while retaining independent cancellation.
Only the final departing caller abandons startup and waits for its cleanup.
New attempts wait for that cleanup; failed cleanup retains its original error
and owned directories until a successful close. Closing the provider reports a
provider-state failure to live callers rather than cancelling their signals.
| Concern | Ghidra contract |
| ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Decompiler lifetime | One persistent `DecompInterface` is opened for the imported Program and disposed during bridge shutdown. Native decompilation has no fixed per-function deadline. External functions or functions without bodies return `null`; cancellation and native failure remain distinct. |
@@ -199,11 +193,7 @@ these tools before compilation. Keeping this matrix separate lets Linux
host/provider acceptance run with only the host compiler.
`npm run verify:ghidra:windows` checks source-owned native x86-64 PE applications;
`-- --x86`, `-- --arm`, or `-- --arm64` selects another target architecture;
`-- --dll` selects DLLs. Choose one architecture and optionally combine it with
`--dll`. These static fixtures contain source-owned encoded instructions; they
are not compiler-produced programs. See the [Windows verification scope](windows-ghidra-p0.md#native-authority-and-target-identity)
for real-provider coverage and exclusions.
`-- --x86` selects x86 and `-- --dll` selects DLLs, and the flags can be combined.
These lanes check all 26 read-only operations, target/snapshot/import SHA-256 linkage,
and project, endpoint, process, and runtime cleanup. The independent native
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
+2 -9
View File
@@ -39,8 +39,8 @@ container format, architecture, word size and byte order. A recognized
- `module.dependencies`, with the same fields and their original order;
- `module.settings`, retaining ordered key/value entries and duplicate keys;
- `module.unparsed_lines`, `module.unparsed_line_bytes_base64` and
`module.complete`, which says whether every present module-text line was
parsed. It does not establish that the dependency inventory is complete.
`module.complete`, so unfamiliar, malformed, conflicting or undecodable records
do not disappear behind a complete result.
Missing checksums are `null`; empty fields and development/local replacement
versions remain unchanged. Build settings such as `GOOS`, `GOARCH` or `vcs.revision`
@@ -50,13 +50,6 @@ base64 while other decodable module records are still reported; the reader does
not insert replacement characters or reject the entire artifact for those
strings.
When no main-module (`mod`) record was decoded, `module.main` is `null` and
dependency coverage is unknown. Successfully decoded dependency records remain
visible. GOPATH-mode builds may contain only a package `path` and build settings;
their empty dependency array does not establish the absence of dependencies.
The result's limitations state this explicitly. A module-mode build with a main
module record and no dependency records has an observed empty dependency list.
## Supported boundary
The built-in TypeScript reader supports ELF, PE and thin Mach-O images with Go's
+1 -1
View File
@@ -50,7 +50,7 @@ For runtime instrumentation, see the [Frida guide](./frida-instrumentation.md).
| Task | Guides |
| --------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Inspect native functions, references, or types | [Native binaries](native-investigation.md), [Go metadata](go-binaries.md), [Rust dependencies](rust-binaries.md), [offline ELF layout](binary-diagnostics.md), [recorded crashes](recorded-crashes.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [Kuna](kuna-provider.md), [JEB](jeb-analysis.md) |
| Select a native provider or import format | [Ghidra semantics](ghidra-provider.md), [Windows](windows-ghidra-p0.md), [DOS](ghidra-dos.md), [raw bytes](raw-binary.md), [MIPS](ghidra-mips.md), [PSP](ghidra-psp.md), [NativeAOT](ghidra-nativeaot.md), [IDA](ida-provider.md), [JEB](jeb-analysis.md) |
| Analyze JavaScript or Electron artifacts | [Static analysis](javascript-artifact-reconstruction.md), [tracing and comparisons](javascript-application-workflows.md), [application graph](javascript-application-graph.md), [source recovery](javascript-recovery.md) |
| Compare static JavaScript with runtime observations | [Runtime attribution](javascript-runtime-observation.md), [reconciliation](javascript-runtime-reconciliation.md), [Electron observation](electron-observation.md), [Electron fuses](electron-fuses.md) |
| Inspect assemblies, packages, or firmware | [.NET](managed-code-analysis.md), [Android](android-analysis.md), [Apktool resources](apktool-resource-analysis.md), [ADB acquisition](adb-device-analysis.md), [Apple](apple-application-analysis.md), [Harmony](harmony-analysis.md), [firmware](firmware-analysis.md), [WASM](wasm-artifacts.md), [EVM](evm-bytecode.md) |
+14 -32
View File
@@ -3,13 +3,13 @@
Run REA, connect your selected agents, and check the tools needed for your task.
Setup previews configuration changes and asks for approval before applying them.
| Task | Section |
| --------------------------------- | --------------------------------------------------------------------------------------------------- |
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), the [IDA guide](ida-provider.md), or [Kuna](kuna-provider.md) |
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
| Task | Section |
| --------------------------------- | ------------------------------------------------------------------------- |
| Connect an agent | [Start setup](#start-setup), then [review changes](#review-setup-changes) |
| Diagnose one task | [Scoped readiness checks](#check-readiness-for-your-task) |
| Configure an analysis engine | [Hopper](#hopper), [Ghidra](#ghidra), or the [IDA guide](ida-provider.md) |
| Update or remove REA | [Maintenance](#diagnose-update-and-remove) |
| Resolve a client-specific problem | [Client configuration](client-configuration.md) |
## Start setup
@@ -179,8 +179,6 @@ as well, so it is broader than this Claude-only migration.
history under `CLAUDE_CONFIG_DIR`. Setup follows the same
variable: it writes the registration to `$CLAUDE_CONFIG_DIR/.claude.json` and
the skill to `$CLAUDE_CONFIG_DIR/skills`.
For REA setup, an empty `CLAUDE_CONFIG_DIR` is treated as unset: registration
goes to `~/.claude.json` and the skill goes to `~/.claude/skills`.
```bash
CLAUDE_CONFIG_DIR=~/.claude-rea npx rea-agents setup --client claude_code
@@ -637,11 +635,6 @@ bridge readiness. For large targets, set `REA_HOPPER_STARTUP_TIMEOUT_MS` to a
decimal integer from 1 to 2,147,483,647 milliseconds in the environment that
starts REA. CLI and MCP use the same setting; invalid values fail configuration
validation. An MCP client's own request timeout must also allow enough time.
`open_binary` validates and selects a target without waiting for native provider
readiness. During Hopper startup, `binary_session.provider_operation_health`
reports `busy` at stage `launch`; `analysis_activity` describes bridge requests
and can remain idle until the socket is ready. Cleanup work reports `busy` at
stage `connection`.
If macOS denies Apple Events, allow the application that starts REA under
**System Settings → Privacy & Security → Automation**, then retry. REA releases
@@ -651,12 +644,9 @@ Arguments containing quotes, backslashes, or line breaks remain unchanged but
cannot establish that a failed handoff opened no document. Other failed handoffs
retain ownership until document closure is confirmed. If a timed-out bridge
becomes ready later, `close_binary` connects to that run, authenticates its
identity, and requests document shutdown. Once its listener is ready, the bridge
allows 30 seconds for the first connection and capability validation. After a
connection drops, it allows the same grace period for an authenticated cleanup
reconnect, then releases Hopper's Python execution thread. Initial analysis is
outside this listener grace period, and authenticated native work is not bounded
by it. Expiry does not establish document closure, so REA
identity, and requests document shutdown. After a connection drops, the bridge
allows 30 seconds for an authenticated cleanup reconnect, then releases Hopper's
Python execution thread. Expiry does not establish document closure, so REA
retains unconfirmed ownership. Retry close while the bridge remains available.
If a Linux launcher exists but cannot start, inspect its shared libraries:
@@ -746,17 +736,9 @@ Once document and process closure are confirmed, the lease can be released even
if temporary-file cleanup fails. Another client can then launch, but the owning
client must finish its retained cleanup before starting again.
An unconfirmed external document can be retried while its authenticated bridge
remains available. The same authentication and process-identity capture apply
when the first successful handshake occurs during cleanup after a startup
timeout. REA can confirm closure if that exact authenticated process exits,
including while a shutdown request is in flight. A helper exit alone is
insufficient, and an application that never authenticated has no verified
process identity. If no matching bridge becomes available, inspect and close
the reported document in Hopper before ending the owning REA connection.
Export the Evidence bundle before `close_binary` clears session records.
`binary_session.cleanup_pending` remains true while provider cleanup is retained,
and provider health remains visible even with `open: false`; that flag alone
does not establish successful cleanup.
remains connected. If that bridge has disconnected, another close cannot confirm
the document: inspect and close the reported document in Hopper before ending
the owning REA connection.
### Hopper in CI
@@ -815,7 +797,7 @@ and `set_addresses_names` name a function entry or any other mapped address
session metadata edits leave executable bytes unchanged and are discarded on close. GUI controls
require Hopper; Windows P0 remains read-only.
Windows P0 admits native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE applications and DLLs, plus explicitly
Windows P0 admits native x86 and x86-64 PE applications and DLLs, plus explicitly
selected [raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied
entry at file offset 0, on fixed local NTFS volumes. Other raw CPU families remain
unsupported on Windows.
-11
View File
@@ -68,17 +68,6 @@ The node vocabulary covers:
- runtime script instance;
- an explicit `unknown` kind when classification is not justified.
Static network endpoints can have `resolution: "partial"` when the analyzer
recognizes a request call but cannot resolve its target to one exact value.
The observation's `value` then preserves the source expression, and its
evidence points to the call site. Partial and exact endpoints have distinct
identities even when their displayed values match. Partial identities also retain
the source site: matching expression text at different sites does not establish
the same runtime target. This records observed source syntax; it does not
establish that the call ran or contacted that endpoint.
Graph coverage describes the analyzed artifact scope and resource limits, not
runtime execution. A partial target alone does not imply omitted source.
The directed relation vocabulary is `contains`, `loads`, `imports`, `maps_to`,
`exposes`, `sends`, `invokes`, `handles`, `calls`, `persists_to`, `observed_as`,
and `changed_from`. Provider names are deliberately absent from both
+3 -74
View File
@@ -6,7 +6,7 @@ are `inspect_analysis_view`, `trace_application_feature`,
`trace_javascript_semantics`, `compare_application_versions`,
`compare_source_to_bundle`, and `compare_javascript_export_shapes`; their CLI
equivalents use the same names with hyphens. `inspect_analysis_view` projects a
summary, module page, typed module references, or one module from already completed application
summary, module page, or one module from already completed application
Evidence; it does not walk relationships.
`inspect_analysis_view` also projects retained `inspect_binary_layout`
@@ -40,46 +40,6 @@ combine its fields with the fact's location. Semantic trace results include the
canonical context subset referenced by their returned facts, so those facts
remain self-contained in the response.
## Retained module facts
For canonical identities and retained source-range counts, query the existing
`inspect_analysis_view` tool with a typed module-reference projection:
```json
{
"source": {
"kind": "retained-evidence",
"evidence_id": "ev_ACTUAL_PARENT_ID"
},
"view": {
"kind": "javascript",
"collection": "modules",
"projection": "references",
"where": { "path": "main.js", "kind": "javascript-module" },
"offset": 0,
"limit": 16
}
}
```
Replace the parent ID with the actual analysis summary's `parent_evidence_id`.
Filters match exact retained paths and kinds. Omission includes all values;
`path: null` selects an unavailable path and `path: ""` selects a recorded empty
path. Multiple identities can share a path. The default offset is 0 and limit
is 64; follow `coverage.next_offset` for later matching rows in retained order.
Each row contains its `node_id`, kind, path, canonical identity, retained
source-range observation count, and distinct supporting authority/state/confidence
combinations in `basis`. These counts do not prove unique reference sites or
runtime loading. Use the existing module item view with the row's `node_id` for
exact source ranges and complete observations. The result retains parent/query
Evidence identity, artifact identity, limitations and unknowns. `source_coverage`
keeps selected/unselected sources, graph coverage and unknown counts separate
from page exhaustion; an exhausted or empty page cannot prove absence from an
incompletely analyzed application. CLI `inspect-analysis-view` reads the same
request from a JSON file; portable input uses `source.kind: "inline"` with the
complete parent Evidence.
## Feature tracing
Select one literal seed kind: node ID, route, string, API, IPC channel, module,
@@ -95,14 +55,6 @@ matches exactly. Otherwise the result recommends provider-neutral follow-up
tools and reports `requires-provider-analysis`; it never starts or switches a
provider implicitly.
A handoff requires the add-on's exact content identity. Unavailable unpacked
companions remain visible in the trace graph with their canonical path,
availability and container-backed Evidence. They yield no actionable handoff
until their bytes are available and the application is reanalyzed; the archive
digest does not identify the missing add-on. An empty handoff list does not prove
that the trace contains no native components. Source partial coverage remains
visible independently of handoff eligibility.
## Semantic relation tracing
`trace_javascript_semantics` queries the authenticated companion graph without
@@ -125,17 +77,6 @@ registrations/removals and dispatch, Node timers and cancellation handles, async
defaults, request construction and response consumers, parse/coercion/
validation boundaries, and built-in resource acquisition/release.
Local callback candidates do not establish a complete target set when a
conditional write may leave a parameter, catch binding, or uninitialized
variable unchanged. These calls retain candidate edges and dynamic uncertainty;
default traces remain ambiguous with partial coverage. Opt into
`include_ambiguous_dynamic_edges` to traverse the candidates. Proven preceding
unconditional assignments, including writes dominating a read in the same
conditional block, can resolve a target; later writes do not replace an
earlier read or captured alias. Unproved ordering across branches or function
activations remains conservative. Returned-reference effects use the same
read-position facts as call resolution.
Mutation tracking covers explicit member assignments, updates, deletion, and
loop assignment targets, including the supported local aliases and shared
nested references. Calls conservatively invalidate exposed mutable references;
@@ -185,13 +126,8 @@ graph and read literal values from `properties.value`, rather than decoding
role keys or labels. Previously stored graphs and their original IDs remain
valid inputs.
Endpoint observations keep their value in `properties.value`. When
`properties.resolution` is `partial`, that value is the unresolved source
expression, with an identity distinct from an exact endpoint of the same text.
Semantic request and response nodes keep their endpoint in `properties.endpoint`. Endpoint
lookup identities are preserved independently of the primitive-value lattice's
byte budget. An oversized normalized literal can become unknown while the same
string remains an exact endpoint for tracing and comparison. Endpoint
Endpoint nodes keep the exact endpoint in each observation's `properties.value`;
semantic request and response nodes keep it in `properties.endpoint`. Endpoint
identity keys use `value-sha256:<digest>` over the canonical JSON string when
that is shorter, or when the original value starts with the reserved digest
prefix. Other short endpoint keys remain unchanged. Endpoint labels use the
@@ -272,13 +208,6 @@ Direct return expressions, including expression-bodied arrows, are evaluated
through an execution-free value lattice. Literal object fields and direct
return sites are represented in the result. Calls, dynamic spreads, computed
keys, and parser recovery remain partial or unknown.
Rebound values are evaluated at their read position: saving an incoming parameter
before a later assignment preserves caller-dependent uncertainty, while a proven
preceding assignment can supply a constant. Unresolved ordering remains unknown.
Supported reads within object and array literals follow left-to-right evaluation;
an unproved relative order remains unknown. A later write in a separate conditional
block can still leave an earlier capture unknown because ordering across those
blocks is not proven.
Objects and arrays passed to calls, constructors, or tagged templates, used
as method receivers, or stored through property targets are not assumed
unchanged afterward. Property stores conservatively retain reference escape
+14 -57
View File
@@ -182,18 +182,6 @@ partial Evidence. Cancellation after source analysis begins can also retain
completed facts, including when interrupted during cleanup, graph construction,
or Evidence sealing. Unexamined relationships remain unknown.
Per-file gaps identify the applicable processing phase: JavaScript source analysis
uses AST authority, while package/JSON, HTML-reference and source-map processing
use artifact-byte authority. File-analysis interruption gaps carry `analysis_status`:
`interrupted` means the phase was entered (including source-worker startup),
and `not-started` means interruption prevented that phase from beginning.
Neither state establishes that parsing completed. Remaining parsers are not run
after interruption just to populate a result.
Native add-ons remain metadata-only and acquire no JavaScript parse or analysis
gap. This also applies when their paths are outside `source_paths`; their inventory
and caller selection remain available without implying applicable AST work.
MCP returns a `partial_observation` retained reference after recording succeeds;
open it with `inspect_analysis_view` on the same connection. If the SDK stopped
waiting before receiving the reply, follow [MCP cancellation recovery](mcp-contracts.md#progress-and-cancellation)
@@ -288,13 +276,8 @@ RPC client syntax also contributes network endpoint candidates, in both
named operation found at the start of a string or cooked template literal —
the name must be followed by variable definitions (`($`), a directive, or a
selection set starting with a field name — or in a precompiled
`OperationDefinition` document object. A standalone Source object's `body`
remains an operation candidate even when it has `locationOffset` metadata.
A graphql-tag `loc.source.body` copy is suppressed only when that source
belongs to a recovered document with the same operation and its containing
document, location and source have proven private metadata ownership. Writes,
escapes and classic-script global bindings retain the source observation.
Anonymous operations are not reported.
`OperationDefinition` document object. A graphql-tag `loc.source.body` copy is
not reported twice. Anonymous operations are not reported.
Literal link options and precompiled document fields follow object property
order: the final definition owns the key. A later spread, dynamic computed key,
@@ -311,25 +294,13 @@ suffix. Utility calls such as `useUtils().invoice.list.fetch()`, `useQueries`,
and links whose callee was renamed by a minifier are not recognized.
Each recovered bundle module retains the exact factory-source digest. A complete
bounded AST also receives a `babel-ast-v3` structural fingerprint. It ignores
ordinary identifier spellings while retaining literal and directive values,
operators, fixed property names, and selected syntax flags such as computed
keys, declaration kind, function async/generator state, update-expression
prefix, method kind, and class static state. Local names used in computed keys
remain rename-resistant; private member names are also ignored while their
method kind and static state remain represented.
This fingerprint is structural evidence for candidate matching, not proof that
two modules behave equivalently. Its scope follows the AST facts listed above;
unrepresented syntax can still collide. The exact source digest remains
available to show byte-level change, and comparison reports content changes even
when structural matching succeeds.
Version comparison includes the algorithm in its structural matching key.
Retained `babel-ast-v1` results can still match other v1 results, and v2 results
can still match v2 results, but v1/v2 results cannot structurally match v3
results. Other matching tiers remain available across algorithm versions.
Re-analyze both snapshots to compare using the v3 rules.
bounded AST also receives a `babel-ast-v2` structural fingerprint that ignores
ordinary identifier names while retaining syntax, literals, operators, object
keys, and member properties. Computed member identifiers retain their access
syntax without treating a local variable's spelling as a fixed property; template
literal contents remain part of the fingerprint. Version comparison keeps the
algorithm in its matching key, so retained `babel-ast-v1` fingerprints cannot match
these new fingerprints. Re-analyze both snapshots to compare their new structure.
If fingerprint construction reaches its structural
bound, the fingerprint is unavailable and its status is `truncated`; a bounded
prefix is never presented as a complete stable fingerprint.
@@ -426,13 +397,10 @@ For `.node` bindings, member names mean “requested by JavaScript syntax.” A
resolved add-on path does not convert them into verified native exports. Native
symbol verification remains a separate deep-analysis claim.
Exact endpoint values preserve useful local diagnostics, including query values
and fragments, while removing URL username/password credentials when the value
parses as a URL. When a recognized request target cannot be resolved exactly,
the endpoint's partial resolution retains the original source expression with
its call-site provenance. This does not establish that the request executed.
Artifact paths, digests, parse locations, and analysis metadata remain
actionable because REA is local-only.
Endpoint observations preserve useful local diagnostics, including query values
and fragments, while removing only URL username/password credentials. Artifact
paths, digests, parse locations, and analysis metadata remain actionable
because REA is local-only.
## Safety boundary
@@ -464,18 +432,7 @@ retained candidate strings and temporary canonical-key serialization. This is
an in-memory semantic allocation bound independent of CLI or MCP transport
budgets. Source literal bytes remain in the parsed source and artifact evidence;
when a normalized semantic value exceeds the bound, its value is unknown and
coverage is partial. Direct string literals, cooked templates without
substitutions, and derived strings all use this normalization boundary. A
single exact primitive is validated without allocating deduplication or sort
state or serializing a canonical key. Configuration defaults use the same
normalizer; limited defaults retain `value_resource_limit` and an anchored
resource-limit unknown. Known null, false, zero, and empty-string defaults stay
exact. This primitive-value bound does not truncate lookup identities such as
source paths, field names or request endpoints. Full endpoint properties remain
available for endpoint queries, tracing and comparison. Each analysis records encountered limit classifications independently of
later projections: a selected healthy sibling can remain exact while whole-file
coverage reports the encountered bound. Projection preserves limiting values
from the shared property-slot owner. Expression
coverage is partial. Expression
evaluation and provenance walks stop after 256 nested levels. Union products
and string lengths are checked before allocating combinations or concatenated
strings. When a boundary is reached, the value stays unknown, semantic coverage
+4 -7
View File
@@ -91,20 +91,17 @@ nodes. REA evaluates compatible static candidates conservatively:
1. A captured-source SHA-256 plus one matching mapped location yields
a content-and-location match.
2. When no static candidate exists at a mapped location, one unique exact
content digest can match. Whole-asset digests and reconstructed module-factory source
2. One unique exact content digest can match even when no location mapping is
available. Whole-asset digests and reconstructed module-factory source
digests remain distinguishable in the reported basis.
3. Without captured bytes, one unique mapped artifact path can produce a
medium-confidence location match.
4. Multiple candidates remain `ambiguous`; no `observed_as` edge is emitted.
With captured source and mapped candidates, only mapped content matches
qualify; unrelated content matches elsewhere are excluded.
`candidate_static_count` reports the full qualified layer/node count, and
`candidate_static_nodes` includes every unique candidate reference.
5. If captured bytes disagree with the static candidate at the mapped path,
the result is an explicit mismatch. An equal digest elsewhere cannot hide
that disagreement. If a mapped candidate's digest is unavailable and no
mapped content match is known, the comparison remains `unknown`.
the result is an explicit mismatch. A path never overrides a digest
disagreement.
6. Missing authorization mappings and incomplete or truncated input coverage
remain unknown or truncated rather than becoming absence claims.
-113
View File
@@ -1,113 +0,0 @@
# Kuna provider
[Kuna](https://github.com/Noelo-Lab/kuna) is an Apache-2.0 decompiler distributed as one `kuna` executable. REA uses a `kuna` you supply. It does not download, install, update or bundle Kuna. REA verified this provider with Kuna 1.833; it accepts other releases and reports them as unverified in `rea doctor` and in each analysis profile.
Kuna is static: it reads the selected image and never executes it.
## Configure
```bash
export REA_KUNA_COMMAND=/absolute/path/to/kuna
rea doctor --provider kuna
```
Select it explicitly with `--provider kuna`, MCP's `open_binary.provider_id: "kuna"`, or `REA_ANALYSIS_PROVIDER=kuna`. With Kuna and another provider configured for the same target, `auto` reports `ambiguous` instead of choosing one.
| Setting | Default | Meaning |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `REA_KUNA_COMMAND` | none (required) | Absolute path of the `kuna` executable. |
| `REA_KUNA_CACHE_DIR` | `~/Library/Caches/rea/kuna` (macOS), `$XDG_CACHE_HOME/rea/kuna` (Linux; `~/.cache/rea/kuna` when `XDG_CACHE_HOME` is unset or relative) | Absolute directory REA owns for project exports. Kuna's own spec cache also lives here, not in your Kuna cache. |
| `REA_KUNA_CACHE_MAX_BYTES` | 4 GiB | Size cap; least recently used exports are evicted above it. |
| `REA_KUNA_PROJECT_TIMEOUT_MS` | 30 minutes | Deadline for whole-program inventory/export, reference and string scans. |
| `REA_KUNA_TIMEOUT_MS` | 120 s | Deadline for one single-function decompile, and for the inventory run REA makes when the export ends without one. |
| `REA_KUNA_JOBS` | 1 | `decompile-project --jobs`. 1 is Kuna's reference answer; more jobs finish sooner but lose callee-first prototypes, and change about 40–60% of function bodies on the measured binaries. |
Windows is not supported yet. Supported targets are ELF and thin Mach-O executables for x86-64, AArch64 and ARM; PE and universal Mach-O are rejected with a typed reason.
## What Kuna answers
| Tool | Source |
| ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `list_procedures`, `search_procedures`, `procedure_address` | Kuna's callable inventory (`kuna functions --json`), including import slots it does not decompile |
| `address_name` | Function entries, then the data labels Kuna's export assigns |
| `procedure_pseudo_code`, `batch_decompile` | The project export, or a single-function run while the export builds |
| `procedure_assembly` | The project export's labeled listing |
| `procedure_callers`, `procedure_callees` | Call-kind edges from a lazy, body-free Kuna graph; synthetic import nodes are accepted by address |
| `list_strings`, `search_strings` | Typed string candidates from one cached, reference-free scan; exact-address, literal and ECMAScript regex selection |
| `xrefs` | Exact-address incoming reference sites, with forwarding aliases filtered explicitly |
| `analyze_function` | Pseudocode and assembly from the export, plus call relationships; typed references, comments, strings and CFG remain unavailable |
Typed procedure references, basic blocks, raw-memory tools, annotations and type recovery are not available from this provider. Use another provider for those operations.
## How answers are produced
The first text or assembly request in a session looks for this target's export in the cache. `list_documents` reads the selected path without initializing analysis. Inventory, function lookup, string queries, call relationships and xrefs do not start a C export. A data-name lookup starts it only when the address is absent from the function inventory. The export is keyed by the target's SHA-256, the Kuna executable's SHA-256, its version and `--jobs`.
- **Warm cache:** analysis reads reuse the verified export. A fresh server verifies its file digests once, even for concurrent first requests; that cost scales with export size.
- **Cold cache:**
- REA starts `kuna decompile-project` in the background on a private copy of the target.
- Meanwhile, pseudocode comes from one `kuna decompile --addr` run. Name and address lookups, and listings, use the inventory the export takes with `kuna functions --json` before it decompiles. The session runs `kuna functions` itself, under `REA_KUNA_TIMEOUT_MS`, only when no export is building (an inventory-only request does not start one) or the export ends without an inventory.
- Requests for the same function share one run; it stops when its last waiting request is cancelled. Each run analyzes the whole image again in its own process, so one REA process runs at most as many at once as the host has cores beyond `REA_KUNA_JOBS` (at least one). Further requests wait in order, and a waiting request is answered from the export if it finishes first. A run whose stop REA cannot prove keeps its slot until `close_binary` proves it stopped.
- The session keeps each completed single-function text, up to 16 MiB in all with the least recently used dropped first, and answers repeats from it until the export can. After a failed export that lasts for the session.
- Assembly and data names wait for the export; progress notifications report the wait every ten seconds. A session does not rebuild a failed export: requests that need it fail with `retryable: false`, the export's failure, its captured Kuna output and what to change before reopening the target. A setting such as `REA_KUNA_PROJECT_TIMEOUT_MS` or `REA_KUNA_JOBS` is read when REA starts, so changing it also needs a restart. Pseudocode keeps using single-function runs.
- A single-function answer says so in its limitations. It lacks the export's whole-program prototypes, so a later request can return different text for the same function.
Pseudocode, assembly, `analyze_function` and data-name answers record where their text came from in `raw_result.source`: `single-function`, or `project-export` with the cache directory. `procedure_pseudo_code` and `procedure_assembly` answers from the export also name the file, byte span and the file's SHA-256. Combined `analyze_function` answers retain both facet records in `raw_result.source.files`, each with its file, digest and byte span. Both texts come from one export; losing that project retries the complete report. Listings and procedure lookups carry no `raw_result`. Agents can open or search the export's `.c` and `.asm` there directly when that is cheaper than calling a tool per function. Text read that way is target-derived, like any decompiler output, but it bypasses the untrusted-content marker that REA's tool results carry.
The background export belongs to the session. `close_binary`, or the end of a CLI command, cancels it unless another session still waits for it. A one-shot CLI decompile therefore answers from a single-function run; any request that needs the export, such as `procedure_assembly` or `analyze_function`, keeps it.
The assembly label index is scanned once per export reader. Cancelling a request stops its wait without interrupting other readers; closing the session aborts and drains the shared scan before releasing its files.
Regex procedure and string search uses ECMAScript Unicode matching in a cancellable worker with a five-second matching deadline. Literal matching remains the default. Cancellation or a matching deadline leaves the session available.
Single-function subprocess output and each cached source facet share a 16 MiB buffered-text capacity. Cached spans are checked before allocation; larger text remains in the local export and the error identifies the file and range. `analyze_function` initially returns the complete dossier, which can still be large; later retained views do not remove that first transfer. Prefer the individual text tools for a focused question.
## Strings
`list_strings` and `search_strings` share one lazy `kuna strings --json --encoding all --min-length 5 --termination any --no-xrefs` scan of the digest-checked private input. Skipping references avoids program lifting, a call graph and a C export. REA validates the input, scan policy, every row and the total before caching any facts. Encoding, character count, byte length and terminator facts must agree; supplementary UTF-8 characters count as Unicode scalars. Malformed or oversized output is an error, not an empty search. JSON admits at most 16 MiB before parsing. String scans use the whole-program `REA_KUNA_PROJECT_TIMEOUT_MS` deadline; `REA_KUNA_TIMEOUT_MS` applies to single-function decompilation. The parsed inventory belongs to one selected target/executable profile and expires on session close; deterministic output-buffer failures are retained until close, while other failures are not cached.
Results keep exact u64 virtual addresses, decoded text, native encoding, terminator status and byte length including any terminator. Kuna scans mapped sections or segments for runs of at least five characters; unterminated fragments can include instruction bytes. `all` includes ASCII, UTF-8 and the engine's Latin-range UTF-16LE reading. Other text, including short strings, non-Latin wide text and UTF-16BE, can remain undiscovered. An empty result does not prove absence.
Use `search_strings` for the matching rows or `list_strings` with an exact address for that location. Literal and regex matching run in REA, preserving the existing case and ECMAScript Unicode semantics instead of substituting Kuna's different regex dialect. Procedure and string regex queries share one worker slot per session. Input projections are materialized only after admission, so queued queries retain no per-query inventory arrays. Close cancels and drains matching. Native metadata queries also share one admission slot, including references; a failed stop retains admission until cleanup is proven.
`raw_result.strings` reports the source JSON digest/size, scanned scope, extraction policy, complete inventory count and selected count without copying unrequested text into the answer. Omit the address from `list_strings` to retrieve the complete inventory as Evidence; normal session retention and bundle export apply, including transport-capacity errors for oversized inline replies. These operations preserve their complete-result contracts and do not add paging. Function-referenced string facets in `analyze_function` remain unavailable; call `xrefs` explicitly when references to one returned string are needed.
## Calls and references
The first call query runs `kuna decompile-graph --functions ''`, admitting no C or assembly bodies. One validated graph and forward/reverse indexes serve later call queries and existing `get_call_graph` / `trace_call_path` workflows. Numeric u64 addresses are parsed from their original JSON tokens. Only `call` edges become callers/callees; jumps and data references retain their distinct meaning. Graph counts, edge endpoints and ordering are validated before any relationship is returned. A forwarding address may be an instruction inside a function, so it is retained separately and never promoted to a function node or call edge. Reported indirect calls make affected relationship coverage partial; no target is invented for them.
`xrefs` runs native `xrefs --to ADDRESS --json` on demand. Kuna unifies forwarding aliases and suppresses edges internal to their class; REA filters to the requested address and records the excluded-row count. Duplicate source sites appear once. An empty result cannot establish the absence of references. Addresses are explicit u64 values or exact, unambiguous procedure names; Kuna has no cursor.
Graph JSON admits at most 32 MiB before parsing; xref JSON at most 16 MiB. A large image can exceed the graph budget: an arm64 Firefox `XUL` (404k functions) does, while a 14 MB Sublime Text binary produced 31.8 MB. Output beyond a buffer is reported as a `memory` resource constraint with `retryable: false`, because the same target and executable produce the same output. The session remembers a graph that overflowed, so later call queries fail at once instead of repeating the whole-program walk. The graph is cached for the selected target and executable profile within one session. Xref reuse is an LRU charged by source bytes plus metadata, limited to 16 MiB and 128 entries; at most eight distinct native xref queries may be pending. Reference commands run one at a time per session and share the provider's foreground budget with single-function decompiles. An unproven process stop retains its admission slots until cleanup succeeds. Concurrent callers share work, individual cancellation releases only that waiter, and the last departing waiter cancels its producer and receives any unproven cleanup. Session close aborts and drains producers, releases parsed caches and retries any retained process/file cleanup.
Evidence includes only selected addresses or dossier neighbors, the producing JSON's SHA-256/byte count, coverage and relevant function facts; unrequested program-wide JSON and instruction text are omitted. Reference caches are in memory and expire on close: a fresh CLI process or MCP server rebuilds them. A first call query therefore costs a whole-program reference walk even when a C export is warm. These operations do not establish full Ghidra/IDA feature parity.
## Provenance and limitations
- Addresses and bytes are Kuna's observations of the image. Procedure entries and extents are its derivations: extents are upper bounds clipped at the next entry, and Kuna can split one function at an entry its own analysis infers. Pseudocode is its reconstruction, never original source.
- Pseudocode and assembly are exact Kuna text; text from Kuna and from another provider is not comparable as equal or different.
- Results are re-read rather than replayed from REA's analysis snapshots, because a cold answer can differ from the export's.
## Cache ownership
REA writes Kuna's exports only inside `REA_KUNA_CACHE_DIR`:
- `projects/<key>/` holds published, immutable exports with a manifest of file digests.
- `staging/` holds builds whose processes may still be live.
- `trash/` holds retired builds and evicted exports awaiting removal.
- `home/` is Kuna's HOME.
Builds publish by one rename, so a reader sees a complete export or none. A session keeps the export it uses from eviction only within its own REA process. If another REA process sharing the cache evicts it, the session rebuilds it; meanwhile pseudocode falls back to single-function runs. A damaged or edited export is removed and rebuilt. Staging moves to `trash/` for removal only after REA proves its Kuna process stopped. Staging of a process it cannot prove stopped is never swept; REA keeps it owned, retries the stop on `close_binary`, and reports it until the stop is proven. Failed file removal also stays bound to its export and is retried by `close_binary`; closing does not report success while that cleanup remains. `rea doctor --provider kuna` reports the cache directory without creating it.
## Verify with a real Kuna
```bash
REA_KUNA_COMMAND=/absolute/path/to/kuna npm run verify:kuna
# Compare with Ghidra on caller-built binaries
REA_KUNA_COMMAND=/absolute/path/to/kuna GHIDRA_INSTALL_DIR=/absolute/path/to/ghidra \
npm run verify:kuna -- --targets DIR --functions inflate,crc32 --compare-ghidra
```
The lane compiles a small source-owned fixture with the host C compiler and verifies the MCP workflow end to end. `--targets DIR --functions a,b` adds caller-built binaries. `--compare-ghidra` (with `GHIDRA_INSTALL_DIR`) runs those same functions through Ghidra; it requires `--targets` and `--functions` and refuses empty matrices or requested function names missing from every target. See [testing](testing.md#kuna-lane) for coverage.
Failed runs also report process-lineage evidence and exit unsuccessfully. If provider or transport cleanup is unconfirmed, the report says `cleanup: "incomplete"` and records `retained_workspace`; that directory is preserved for investigation and cleanup retries.
+1 -1
View File
@@ -352,7 +352,7 @@ install it or its toolchain. The [NativeAOT guide](ghidra-nativeaot.md) defines
these routes, their layout/host restrictions, derived-memory provenance, and
verification coverage.
REA's experimental Windows Ghidra boundary admits native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE
REA's experimental Windows Ghidra boundary admits native x86 and x86-64 PE
applications and DLLs for static analysis. For x64 PE NativeAOT, the read-only
parser also runs there; the [NativeAOT guide](ghidra-nativeaot.md) lists what a
Windows host covers.
+4 -36
View File
@@ -393,26 +393,15 @@ concrete version, selection source, and complete analysis profile through
choices return typed selection details. A selected provider is never replaced
automatically after a runtime failure.
Each provider client receives `binary_session.analysis_run.run_id` before any
provider startup, including clients whose startup fails. `process_lineage` is
`not_observed` until a dynamic provider starts, then becomes `snapshots` with
every started provider's identity
Every successful target transition allocates `binary_session.analysis_run.run_id`
before any provider startup. `process_lineage` is `not_observed` until a dynamic
provider starts, then becomes `snapshots` with every started provider's identity
and retained ownership observation. Each observation records `observed_at` and
is `unavailable` with a reason when ownership could not be revalidated, or
`verified` with launcher PID, parent PID, process group, and descendants observed
at that bounded check. These are historical snapshots, not live process
inventories, and do not claim that no short-lived descendant existed.
`binary_session.cleanup_pending` reports whether the session retains a provider
client, session file resource, or provider-candidate resource whose cleanup has
not succeeded. It can be true with `open: false`; target retirement does not
establish resource release. Health, activity, and lineage remain available
under a retained client's original `analysis_run.run_id` where the provider
supplies them. Successful client cleanup clears the retained run; a replacement
client receives a fresh run ID. Follow the cleanup error's reported resources
and recovery action before reopening; a missing health observation does not
establish an idle provider.
## Progress and cancellation
REA accepts ordinary `tools/call` progress tokens. The numeric `progress` counts
@@ -439,10 +428,6 @@ Native call tracing and process capture retain available observations in
`details.partial_observation` on failure, including when cleanup succeeds.
The observation reports its partial coverage; cleanup details describe host
state separately from the execution failure.
For these tools, cancellation observed during result finalization or owned
cleanup also returns typed cancellation with the available observations.
A normal target exit does not establish request completion. Incomplete cleanup
still takes precedence and retains its resource diagnostics.
A process capture that fails during a finalization interval keeps every signal
attempt and its delivery result in `details.partial_observation`, with
`elapsed_ms` and `exit_code` null when no exit was observed.
@@ -512,10 +497,7 @@ declared by their tools. Artifact extraction materializes the selected regular
files into a fresh REA-chosen temporary directory and reports its path.
`analyze_javascript_application` reads the supplied local directory or ASAR path
and returns its result and Evidence inline. Its owned analysis worker and private
temporary source files are declared as `launchesProcess` and `writesFilesystem`,
including for a single JavaScript source. This internal analysis does not execute
the supplied application code.
and returns its result and Evidence inline.
## Ghidra first-query deadlines and recovery
@@ -709,20 +691,6 @@ after SDK conversion and check advertised validation and actual calls.
Individual model APIs can impose additional nesting limits; complete producer
captures can exceed ten structural levels.
Boolean literal constraints in input schemas are relaxed only in this
advertisement. For example, a boolean `const` is presented as `type: boolean`;
an affected `not` is omitted, conditional predicates and their branches are
omitted together, pure boolean-literal `oneOf` branches collapse to a boolean,
and other affected `oneOf` branches are presented as `anyOf` to retain branch
guidance without making overlapping widened branches reject inputs.
This can make the advertised schema accept inputs the canonical contract
rejects. The MCP handler still validates every call with the canonical
contract. The Gemini CLI verifier checks the outgoing
`parametersJsonSchema` and canonical contract examples against that declaration
using a loopback model fixture; it does not establish live Google API or
OpenCode client behavior. Set `REA_VERIFY_GEMINI_SCHEMA_PROFILE=compact` to
verify the compact advertisement through the same client boundary.
### Compact input schema profile
Some function-calling providers reject a request while any single tool's
+2 -6
View File
@@ -111,12 +111,8 @@ list does not establish that the target has no procedures.
Suggestions are Unicode-normalized, case-insensitive prefix matches. The server
deduplicates them and sorts by code point for deterministic results. Procedure
completion derives candidates from the complete inventory, so a name is offered
only when it identifies one address in that inventory. A completion reply
carries at most 100 values, the MCP limit. `total` counts every match and
`hasMore` is `true` when values were omitted. Completion has no paging
controls. A longer prefix reaches an omitted value only when it excludes other
matches; matching uses normalized, case-insensitive forms, so a value can stay
unreachable when more than 100 matches share its normalized prefix.
only when it identifies one address in that inventory. Completion returns all
matching values; it does not expose paging controls.
## Lifecycle and safety
+2 -26
View File
@@ -270,9 +270,7 @@ Mach-O image. Architecture
`file_offset` values are container-relative. Entrypoint, segment and section
`file_offset` values retain their source meaning and are relative to the
selected Mach-O slice. Raw load-command fields retain their original values.
A field otool prints more than once, such as each `LC_BUILD_VERSION` tool, is
keyed `name #n` in print order. Each `LC_LINKER_OPTION` string is keyed
`string #n` and checked against the declared `count`. Evidence locations map the LC_MAIN entrypoint and nonempty file-backed segment
Evidence locations map the LC_MAIN entrypoint and nonempty file-backed segment
ranges to container offsets by adding the observed slice offset. Zero-fill
segments do not produce file ranges. When the selected slice's container offset
is unknown, those mapped locations are omitted with an explicit limitation;
@@ -451,11 +449,6 @@ The run ends when the process exits, `max_events` entries are recorded,
Bounded termination reports partial coverage. REA checks process termination
using a launch-time OS start identity; if ownership or cleanup cannot be
verified, the result or error preserves that uncertainty. The result reports:
Launch errors keep the fact that LLDB created a process separate from whether
it remains alive. An exited `SBProcess` is reported as exited only after LLDB
confirms that state; unresolved ownership remains retained for cleanup retry.
Cancellation observed while output and Evidence are finalized returns partial
cancellation Evidence, even if the target has already exited.
- the outcome, exit status and captured stdout/stderr (1 MiB kept per stream
while excess bytes are drained and counted without growing capture files).
@@ -486,11 +479,6 @@ with partial coverage. Available output prefixes remain inline; their byte
counts are observed lower bounds when final drain counters are unavailable.
The original failure classification and cleanup uncertainty remain separate.
An interrupted journal row is ignored with an explicit limitation.
If the trace completed before cancellation or runtime-directory cleanup failed,
the retained observation also keeps its exit status, termination proof, launch
identity, breakpoint results and final stream counters. Those fields remain
absent when the interrupted producer did not report them; partial coverage alone
does not mean that every retained fact is incomplete.
Only entries are observed: return values, floating-point and stack arguments,
and inlined or `objc_direct` calls are not. The target runs with the current
@@ -507,7 +495,7 @@ is present. A denied trace reports the `debugger-attach-denied` reason.
Install a Ghidra 12.1.x release and the 64-bit full JDK it declares, then configure REA to
use them. Ghidra analysis supports Linux x64/arm64 and macOS x64/arm64; each installation requires
the matching native decompiler. Experimental Windows x64 P0 admits native
x86, x86-64, ARMNT (Thumb-2) and ARM64 PE applications and DLLs, plus explicitly selected
x86 and x86-64 PE applications and DLLs, plus explicitly selected
[raw NMOS 6502 images](raw-binary.md#nmos-6502) with a caller-supplied load address,
on fixed local NTFS volumes. Other raw CPU families remain unsupported on Windows.
The boundary uses bundled Job Object ownership, protected runtime DACLs, and
@@ -587,18 +575,6 @@ Exact and ambiguous value counts describe emitted root/object value instances;
a root and an object may contain the same value. Archive digests identify the
original bytes.
Interface Builder's XML and binary plist archives use the same number metadata
and exact-decimal representation for emitted node attributes. Projection follows
object and connection classification, so archive IDs, UID links and graph counts
keep their meaning. Precision limitations name the source document. Unsafe
integer-literal counts cover the archive, including fields not emitted as node
attributes; exact and ambiguous projection counts cover emitted attribute values.
Integral reals stay numbers, and ambiguous or incompletely observed values retain
explicit precision limitations. Before plist decoding, the existing aggregate
decode budget reserves estimated room for number metadata and attribute
projection; these estimates are not a hard process-memory bound. Native
NIBArchive decoding is unchanged by this plist-specific representation.
Dates are reported as `{ "$plist_type": "date", "iso8601": "<timestamp>" }`. A
date the decoder cannot represent, such as an XML date with a year beyond 9999
or a negative year, is reported as `{ "$plist_type": "date", "iso8601": null }`
+1 -3
View File
@@ -425,9 +425,7 @@ results for unaffected dimensions and mark affected dimensions unknown; an
incomplete capture cannot locate the first divergence across all dimensions.
Trace assertions use coverage for the sources they select, so an assertion
about complete raw terminal output need not fail because rendered snapshots
were omitted. A trace difference applies to the dimensions identified by its
diagnostic; unrelated dimensions retain their own comparison, including unknown
coverage. Captures missing `truncation_details` are rejected on import;
were omitted. Captures missing `truncation_details` are rejected on import;
preserve them as historical files and recapture for current comparisons.
`limits.file_bytes` is already the cumulative whole-file hashing budget,
independent of `limits.output_bytes`, which bounds retained terminal content.
+1 -1
View File
@@ -166,7 +166,7 @@ npx rea-agents@latest setup
<a id="choosing-a-deep-analysis-provider"></a>
تختلف صيغ ملفات الشيفرة الأصلية والأنظمة المضيفة المدعومة باختلاف الموفّر. راجع [إعداد Hopper وGhidra](../installation.md#hopper)، و[دليل IDA](../ida-provider.md)، [Kuna](../kuna-provider.md)، و[دعم Ghidra التجريبي على Windows](../windows-ghidra-p0.md). يدعم Ghidra أيضًا [تحليل DOS ذي 16 بت](../ghidra-dos.md). لاختيار الموفّر، راجع [دليل CLI](../cli.md#choose-a-provider). تحقّق من [توفر الميزات في الإصدارات](../installation.md#released-package-and-main) للميزات المضافة بعد أحدث إصدار على npm.
تختلف صيغ ملفات الشيفرة الأصلية والأنظمة المضيفة المدعومة باختلاف الموفّر. راجع [إعداد Hopper وGhidra](../installation.md#hopper)، و[دليل IDA](../ida-provider.md)، و[دعم Ghidra التجريبي على Windows](../windows-ghidra-p0.md). يدعم Ghidra أيضًا [تحليل DOS ذي 16 بت](../ghidra-dos.md). لاختيار الموفّر، راجع [دليل CLI](../cli.md#choose-a-provider). تحقّق من [توفر الميزات في الإصدارات](../installation.md#released-package-and-main) للميزات المضافة بعد أحدث إصدار على npm.
## دراسات الحالة
+1 -1
View File
@@ -166,7 +166,7 @@ Statische JavaScript- und .NET-Untersuchungen lesen die bereitgestellten Dateien
<a id="choosing-a-deep-analysis-provider"></a>
Native Formate und unterstützte Hostsysteme unterscheiden sich je nach Provider. Siehe [Hopper- und Ghidra-Einrichtung](../installation.md#hopper), den [IDA-Leitfaden](../ida-provider.md), [Kuna](../kuna-provider.md) und die [experimentelle Ghidra-Unterstützung unter Windows](../windows-ghidra-p0.md). Ghidra unterstützt auch [16-Bit-DOS-Analyse](../ghidra-dos.md). Die Provider-Auswahl beschreibt der [CLI-Leitfaden](../cli.md#choose-a-provider). Prüfe die [Verfügbarkeit in Releases](../installation.md#released-package-and-main) für Funktionen, die seit der neuesten npm-Version hinzugekommen sind.
Native Formate und unterstützte Hostsysteme unterscheiden sich je nach Provider. Siehe [Hopper- und Ghidra-Einrichtung](../installation.md#hopper), den [IDA-Leitfaden](../ida-provider.md) und die [experimentelle Ghidra-Unterstützung unter Windows](../windows-ghidra-p0.md). Ghidra unterstützt auch [16-Bit-DOS-Analyse](../ghidra-dos.md). Die Provider-Auswahl beschreibt der [CLI-Leitfaden](../cli.md#choose-a-provider). Prüfe die [Verfügbarkeit in Releases](../installation.md#released-package-and-main) für Funktionen, die seit der neuesten npm-Version hinzugekommen sind.
## Fallstudien
+1 -1
View File
@@ -179,7 +179,7 @@ npx rea-agents@latest setup
<a id="choosing-a-deep-analysis-provider"></a>
Οι μορφές κώδικα μηχανής και τα υποστηριζόμενα συστήματα διαφέρουν ανά πάροχο ανάλυσης. Δες τη [ρύθμιση Hopper και Ghidra](../installation.md#hopper), τον [οδηγό IDA](../ida-provider.md) (που περιλαμβάνει προαιρετική γέφυρα MCP για κώδικα μηχανής), [Kuna](../kuna-provider.md) και την [πειραματική υποστήριξη Ghidra στα Windows](../windows-ghidra-p0.md). Το Ghidra υποστηρίζει επίσης [ανάλυση DOS 16 bit](../ghidra-dos.md). Για μεγάλα δυαδικά αρχεία, αύξησε το χρονικό όριο εκκίνησης του παρόχου με `REA_GHIDRA_STARTUP_TIMEOUT_MS` ή `REA_HOPPER_STARTUP_TIMEOUT_MS`. Για την επιλογή παρόχου, δες τον [οδηγό CLI](../cli.md#choose-a-provider). Έλεγξε τη [διαθεσιμότητα ανά έκδοση](../installation.md#released-package-and-main) για λειτουργίες που προστέθηκαν μετά την τελευταία έκδοση στο npm.
Οι μορφές κώδικα μηχανής και τα υποστηριζόμενα συστήματα διαφέρουν ανά πάροχο ανάλυσης. Δες τη [ρύθμιση Hopper και Ghidra](../installation.md#hopper), τον [οδηγό IDA](../ida-provider.md) (που περιλαμβάνει προαιρετική γέφυρα MCP για κώδικα μηχανής) και την [πειραματική υποστήριξη Ghidra στα Windows](../windows-ghidra-p0.md). Το Ghidra υποστηρίζει επίσης [ανάλυση DOS 16 bit](../ghidra-dos.md). Για μεγάλα δυαδικά αρχεία, αύξησε το χρονικό όριο εκκίνησης του παρόχου με `REA_GHIDRA_STARTUP_TIMEOUT_MS` ή `REA_HOPPER_STARTUP_TIMEOUT_MS`. Για την επιλογή παρόχου, δες τον [οδηγό CLI](../cli.md#choose-a-provider). Έλεγξε τη [διαθεσιμότητα ανά έκδοση](../installation.md#released-package-and-main) για λειτουργίες που προστέθηκαν μετά την τελευταία έκδοση στο npm.
## Μελέτες περίπτωσης
+1 -1
View File
@@ -166,7 +166,7 @@ La inspección estática de JavaScript y .NET lee los archivos suministrados sin
<a id="choosing-a-deep-analysis-provider"></a>
Los formatos nativos y los sistemas anfitriones compatibles varían según el proveedor. Consulta [configuración de Hopper y Ghidra](../installation.md#hopper), la [guía de IDA](../ida-provider.md), [Kuna](../kuna-provider.md) y la [compatibilidad experimental de Ghidra en Windows](../windows-ghidra-p0.md). Ghidra también admite [análisis de DOS de 16 bits](../ghidra-dos.md). Para elegir un proveedor, consulta la [guía de CLI](../cli.md#choose-a-provider). Comprueba la [disponibilidad por versión](../installation.md#released-package-and-main) de las funciones añadidas después de la última versión publicada en npm.
Los formatos nativos y los sistemas anfitriones compatibles varían según el proveedor. Consulta [configuración de Hopper y Ghidra](../installation.md#hopper), la [guía de IDA](../ida-provider.md) y la [compatibilidad experimental de Ghidra en Windows](../windows-ghidra-p0.md). Ghidra también admite [análisis de DOS de 16 bits](../ghidra-dos.md). Para elegir un proveedor, consulta la [guía de CLI](../cli.md#choose-a-provider). Comprueba la [disponibilidad por versión](../installation.md#released-package-and-main) de las funciones añadidas después de la última versión publicada en npm.
## Casos prácticos
+1 -1
View File
@@ -158,7 +158,7 @@ REA به Node.js 22.x (>=22.19)، 24.x (>=24.11) یا 26+، به‌همراه np
<a id="choosing-a-deep-analysis-provider"></a>
فرمت‌های بومی و پشتیبانی سیستم میزبان بر اساس ارائه‌دهنده متفاوت‌اند. [راه‌اندازی Hopper و Ghidra](../installation.md#hopper)، [راهنمای IDA](../ida-provider.md)، [Kuna](../kuna-provider.md) و [پشتیبانی آزمایشی Ghidra در ویندوز](../windows-ghidra-p0.md) را ببینید. Ghidra همچنین از [تحلیل DOS شانزده‌بیتی](../ghidra-dos.md) پشتیبانی می‌کند. برای انتخاب ارائه‌دهنده، به [راهنمای CLI](../cli.md#choose-a-provider) مراجعه کنید. برای قابلیت‌هایی که پس از آخرین انتشار npm اضافه شده‌اند، [وضعیت انتشار](../installation.md#released-package-and-main) را بررسی کنید.
فرمت‌های بومی و پشتیبانی سیستم میزبان بر اساس ارائه‌دهنده متفاوت‌اند. [راه‌اندازی Hopper و Ghidra](../installation.md#hopper)، [راهنمای IDA](../ida-provider.md) و [پشتیبانی آزمایشی Ghidra در ویندوز](../windows-ghidra-p0.md) را ببینید. Ghidra همچنین از [تحلیل DOS شانزده‌بیتی](../ghidra-dos.md) پشتیبانی می‌کند. برای انتخاب ارائه‌دهنده، به [راهنمای CLI](../cli.md#choose-a-provider) مراجعه کنید. برای قابلیت‌هایی که پس از آخرین انتشار npm اضافه شده‌اند، [وضعیت انتشار](../installation.md#released-package-and-main) را بررسی کنید.
## نمونه‌های عملی
+1 -1
View File
@@ -170,7 +170,7 @@ L'inspection statique JavaScript et .NET lit les fichiers fournis sans exécuter
<a id="choosing-a-deep-analysis-provider"></a>
Les formats natifs et les hôtes pris en charge varient selon le fournisseur. Consultez [la configuration de Hopper et Ghidra](../installation.md#hopper), le [guide IDA](../ida-provider.md), [Kuna](../kuna-provider.md) et [la prise en charge expérimentale de Ghidra sous Windows](../windows-ghidra-p0.md). Ghidra prend aussi en charge [l'analyse DOS 16 bits](../ghidra-dos.md). Pour les gros binaires, augmentez son délai de démarrage avec `REA_GHIDRA_STARTUP_TIMEOUT_MS`. Pour le choix du fournisseur, consultez le [guide CLI](../cli.md#choose-a-provider). Vérifiez la [disponibilité dans les versions publiées](../installation.md#released-package-and-main) pour les fonctionnalités ajoutées depuis la dernière version npm.
Les formats natifs et les hôtes pris en charge varient selon le fournisseur. Consultez [la configuration de Hopper et Ghidra](../installation.md#hopper), le [guide IDA](../ida-provider.md) et [la prise en charge expérimentale de Ghidra sous Windows](../windows-ghidra-p0.md). Ghidra prend aussi en charge [l'analyse DOS 16 bits](../ghidra-dos.md). Pour les gros binaires, augmentez son délai de démarrage avec `REA_GHIDRA_STARTUP_TIMEOUT_MS`. Pour le choix du fournisseur, consultez le [guide CLI](../cli.md#choose-a-provider). Vérifiez la [disponibilité dans les versions publiées](../installation.md#released-package-and-main) pour les fonctionnalités ajoutées depuis la dernière version npm.
## Études de cas
+1 -1
View File
@@ -173,7 +173,7 @@ Pemeriksaan JavaScript dan .NET statis membaca berkas yang disediakan tanpa menj
<a id="choosing-a-deep-analysis-provider"></a>
Format native dan dukungan sistem host berbeda-beda menurut penyedia. Lihat [penyiapan Hopper dan Ghidra](../installation.md#hopper), [panduan IDA](../ida-provider.md), [Kuna](../kuna-provider.md), dan [dukungan eksperimental Ghidra di Windows](../windows-ghidra-p0.md).
Format native dan dukungan sistem host berbeda-beda menurut penyedia. Lihat [penyiapan Hopper dan Ghidra](../installation.md#hopper), [panduan IDA](../ida-provider.md), dan [dukungan eksperimental Ghidra di Windows](../windows-ghidra-p0.md).
Ghidra juga mendukung [analisis DOS 16-bit](../ghidra-dos.md).
+1 -1
View File
@@ -170,7 +170,7 @@ L'ispezione statica di JavaScript e .NET legge i file forniti senza eseguire l'a
<a id="choosing-a-deep-analysis-provider"></a>
I formati nativi e il supporto host variano in base al provider. Vedi la [configurazione di Hopper e Ghidra](../installation.md#hopper), la [guida IDA](../ida-provider.md), [Kuna](../kuna-provider.md) e il [supporto sperimentale Windows Ghidra](../windows-ghidra-p0.md). Ghidra supporta anche l'[analisi DOS a 16 bit](../ghidra-dos.md). Per binari di grandi dimensioni, aumenta il timeout di avvio con `REA_GHIDRA_STARTUP_TIMEOUT_MS`. Per la selezione del provider, vedi la [guida CLI](../cli.md#choose-a-provider). Controlla la [disponibilità delle release](../installation.md#released-package-and-main) per le funzionalità aggiunte dall'ultima release npm.
I formati nativi e il supporto host variano in base al provider. Vedi la [configurazione di Hopper e Ghidra](../installation.md#hopper), la [guida IDA](../ida-provider.md) e il [supporto sperimentale Windows Ghidra](../windows-ghidra-p0.md). Ghidra supporta anche l'[analisi DOS a 16 bit](../ghidra-dos.md). Per binari di grandi dimensioni, aumenta il timeout di avvio con `REA_GHIDRA_STARTUP_TIMEOUT_MS`. Per la selezione del provider, vedi la [guida CLI](../cli.md#choose-a-provider). Controlla la [disponibilità delle release](../installation.md#released-package-and-main) per le funzionalità aggiunte dall'ultima release npm.
## Showcase
+1 -1
View File
@@ -166,7 +166,7 @@ REA には Node.js 22.x(>=22.19)、24.x(>=24.11)、または 26+ と npm
<a id="choosing-a-deep-analysis-provider"></a>
対応するネイティブ形式とホストは、プロバイダーによって異なります。[Hopper と Ghidra のセットアップ](../installation.md#hopper)、[IDA ガイド](../ida-provider.md)、[Kuna](../kuna-provider.md)、[Windows 向け Ghidra の実験的対応](../windows-ghidra-p0.md)を参照してください。Ghidra は [16 ビット DOS 分析](../ghidra-dos.md)にも対応しています。プロバイダーの選択は [CLI ガイド](../cli.md#choose-a-provider)を参照してください。最新の npm リリース以降に追加された機能については、[リリースでの利用可否](../installation.md#released-package-and-main)を確認してください。
対応するネイティブ形式とホストは、プロバイダーによって異なります。[Hopper と Ghidra のセットアップ](../installation.md#hopper)、[IDA ガイド](../ida-provider.md)、[Windows 向け Ghidra の実験的対応](../windows-ghidra-p0.md)を参照してください。Ghidra は [16 ビット DOS 分析](../ghidra-dos.md)にも対応しています。プロバイダーの選択は [CLI ガイド](../cli.md#choose-a-provider)を参照してください。最新の npm リリース以降に追加された機能については、[リリースでの利用可否](../installation.md#released-package-and-main)を確認してください。
## 事例
+1 -1
View File
@@ -175,7 +175,7 @@ REA에는 Node.js 22.x(>=22.19), 24.x(>=24.11) 또는 26+와 npm이 필요합니
<a id="choosing-a-deep-analysis-provider"></a>
지원하는 네이티브 형식과 호스트는 제공자마다 다릅니다. [Hopper 및 Ghidra 설정](../installation.md#hopper), [IDA 가이드](../ida-provider.md)(선택 사항인 네이티브 MCP 브리지 포함), [Kuna](../kuna-provider.md), [실험적 Windows Ghidra 지원](../windows-ghidra-p0.md)을 참고하세요. Ghidra는 [16비트 DOS 분석](../ghidra-dos.md)도 지원합니다. 큰 바이너리를 분석할 때는 `REA_GHIDRA_STARTUP_TIMEOUT_MS` 또는 `REA_HOPPER_STARTUP_TIMEOUT_MS`로 제공자의 시작 제한 시간을 늘리세요. 제공자 선택은 [CLI 가이드](../cli.md#choose-a-provider)를 참고하세요. 최신 npm 릴리스 이후 추가된 기능은 [릴리스에서의 사용 가능 여부](../installation.md#released-package-and-main)를 확인하세요.
지원하는 네이티브 형식과 호스트는 제공자마다 다릅니다. [Hopper 및 Ghidra 설정](../installation.md#hopper), [IDA 가이드](../ida-provider.md)(선택 사항인 네이티브 MCP 브리지 포함), [실험적 Windows Ghidra 지원](../windows-ghidra-p0.md)을 참고하세요. Ghidra는 [16비트 DOS 분석](../ghidra-dos.md)도 지원합니다. 큰 바이너리를 분석할 때는 `REA_GHIDRA_STARTUP_TIMEOUT_MS` 또는 `REA_HOPPER_STARTUP_TIMEOUT_MS`로 제공자의 시작 제한 시간을 늘리세요. 제공자 선택은 [CLI 가이드](../cli.md#choose-a-provider)를 참고하세요. 최신 npm 릴리스 이후 추가된 기능은 [릴리스에서의 사용 가능 여부](../installation.md#released-package-and-main)를 확인하세요.
## 사례
+1 -1
View File
@@ -166,7 +166,7 @@ Statyczna analiza JavaScript i .NET odczytuje dostarczone pliki bez uruchamiania
<a id="choosing-a-deep-analysis-provider"></a>
Obsługiwane formaty natywne i platformy zależą od dostawcy analizy. Zobacz [konfigurację Hoppera i Ghidry](../installation.md#hopper), [przewodnik IDA](../ida-provider.md), [Kuna](../kuna-provider.md) oraz [eksperymentalną obsługę Ghidry w systemie Windows](../windows-ghidra-p0.md). Ghidra obsługuje też [analizę 16-bitowego DOS-a](../ghidra-dos.md). Wybór dostawcy opisuje [przewodnik CLI](../cli.md#choose-a-provider). Sprawdź [dostępność w wydaniach](../installation.md#released-package-and-main) dla funkcji dodanych po najnowszym wydaniu npm.
Obsługiwane formaty natywne i platformy zależą od dostawcy analizy. Zobacz [konfigurację Hoppera i Ghidry](../installation.md#hopper), [przewodnik IDA](../ida-provider.md) oraz [eksperymentalną obsługę Ghidry w systemie Windows](../windows-ghidra-p0.md). Ghidra obsługuje też [analizę 16-bitowego DOS-a](../ghidra-dos.md). Wybór dostawcy opisuje [przewodnik CLI](../cli.md#choose-a-provider). Sprawdź [dostępność w wydaniach](../installation.md#released-package-and-main) dla funkcji dodanych po najnowszym wydaniu npm.
## Przykłady zastosowań
+1 -1
View File
@@ -166,7 +166,7 @@ A inspeção estática de JavaScript e .NET lê os arquivos fornecidos sem execu
<a id="choosing-a-deep-analysis-provider"></a>
Os formatos nativos e os sistemas anfitriões compatíveis variam por provedor. Consulte [configuração do Hopper e Ghidra](../installation.md#hopper), o [guia do IDA](../ida-provider.md), [Kuna](../kuna-provider.md) e o [suporte experimental ao Ghidra no Windows](../windows-ghidra-p0.md). O Ghidra também oferece [análise de DOS de 16 bits](../ghidra-dos.md). Para selecionar um provedor, consulte o [guia da CLI](../cli.md#choose-a-provider). Verifique a [disponibilidade nas versões](../installation.md#released-package-and-main) das funcionalidades adicionadas após a versão mais recente no npm.
Os formatos nativos e os sistemas anfitriões compatíveis variam por provedor. Consulte [configuração do Hopper e Ghidra](../installation.md#hopper), o [guia do IDA](../ida-provider.md) e o [suporte experimental ao Ghidra no Windows](../windows-ghidra-p0.md). O Ghidra também oferece [análise de DOS de 16 bits](../ghidra-dos.md). Para selecionar um provedor, consulte o [guia da CLI](../cli.md#choose-a-provider). Verifique a [disponibilidade nas versões](../installation.md#released-package-and-main) das funcionalidades adicionadas após a versão mais recente no npm.
## Casos de uso
+1 -1
View File
@@ -166,7 +166,7 @@ REA требует Node.js 22.x (>=22.19), 24.x (>=24.11) или 26+, а так
<a id="choosing-a-deep-analysis-provider"></a>
Нативные форматы и поддерживаемые системы различаются у разных провайдеров. См. [настройку Hopper и Ghidra](../installation.md#hopper), [руководство по IDA](../ida-provider.md), [Kuna](../kuna-provider.md) и [экспериментальную поддержку Ghidra в Windows](../windows-ghidra-p0.md). Ghidra также поддерживает [анализ 16-битного DOS](../ghidra-dos.md). Выбор провайдера описан в [руководстве по CLI](../cli.md#choose-a-provider). Для функций, добавленных после последней версии в npm, проверьте [доступность в релизах](../installation.md#released-package-and-main).
Нативные форматы и поддерживаемые системы различаются у разных провайдеров. См. [настройку Hopper и Ghidra](../installation.md#hopper), [руководство по IDA](../ida-provider.md) и [экспериментальную поддержку Ghidra в Windows](../windows-ghidra-p0.md). Ghidra также поддерживает [анализ 16-битного DOS](../ghidra-dos.md). Выбор провайдера описан в [руководстве по CLI](../cli.md#choose-a-provider). Для функций, добавленных после последней версии в npm, проверьте [доступность в релизах](../installation.md#released-package-and-main).
## Примеры
+1 -1
View File
@@ -166,7 +166,7 @@ REA ต้องใช้ Node.js 22.x (>=22.19), 24.x (>=24.11) หรือ 26
<a id="choosing-a-deep-analysis-provider"></a>
รูปแบบไบนารีเนทีฟและแพลตฟอร์มที่รองรับแตกต่างกันตามผู้ให้บริการวิเคราะห์ ดู[การตั้งค่า Hopper และ Ghidra](../installation.md#hopper), [คู่มือ IDA](../ida-provider.md) [Kuna](../kuna-provider.md) และ[การรองรับ Ghidra บน Windows แบบทดลอง](../windows-ghidra-p0.md) Ghidra ยังรองรับ[การวิเคราะห์ DOS แบบ 16 บิต](../ghidra-dos.md) สำหรับการเลือกผู้ให้บริการ ดู[คู่มือ CLI](../cli.md#choose-a-provider) ตรวจสอบ[ความพร้อมใช้งานในแต่ละรุ่น](../installation.md#released-package-and-main) สำหรับฟีเจอร์ที่เพิ่มหลังรุ่นล่าสุดบน npm
รูปแบบไบนารีเนทีฟและแพลตฟอร์มที่รองรับแตกต่างกันตามผู้ให้บริการวิเคราะห์ ดู[การตั้งค่า Hopper และ Ghidra](../installation.md#hopper), [คู่มือ IDA](../ida-provider.md) และ[การรองรับ Ghidra บน Windows แบบทดลอง](../windows-ghidra-p0.md) Ghidra ยังรองรับ[การวิเคราะห์ DOS แบบ 16 บิต](../ghidra-dos.md) สำหรับการเลือกผู้ให้บริการ ดู[คู่มือ CLI](../cli.md#choose-a-provider) ตรวจสอบ[ความพร้อมใช้งานในแต่ละรุ่น](../installation.md#released-package-and-main) สำหรับฟีเจอร์ที่เพิ่มหลังรุ่นล่าสุดบน npm
## ตัวอย่างการใช้งาน
+1 -1
View File
@@ -171,7 +171,7 @@ Statik JavaScript ve .NET incelemesi, uygulamayı çalıştırmadan sağlanan do
<a id="choosing-a-deep-analysis-provider"></a>
Yerel makine kodu içeren ikili dosya biçimleri ve ana bilgisayar desteği sağlayıcıya göre değişir. [Hopper ve Ghidra kurulumu](../installation.md#hopper), isteğe bağlı native MCP bridge'i de açıklayan [IDA rehberi](../ida-provider.md), [Kuna](../kuna-provider.md) ve [Windows için deneysel Ghidra desteği](../windows-ghidra-p0.md) belgelerine bakın. Ghidra ayrıca [16 bit DOS analizini](../ghidra-dos.md) destekler. Büyük ikili dosyalar için sağlayıcının başlatma zaman aşımını `REA_GHIDRA_STARTUP_TIMEOUT_MS` veya `REA_HOPPER_STARTUP_TIMEOUT_MS` ile artırın. Sağlayıcı seçimi için [CLI rehberine](../cli.md#choose-a-provider) bakın. En son npm sürümünden sonra eklenen özellikler için [sürümlerde kullanılabilirliği](../installation.md#released-package-and-main) kontrol edin.
Yerel makine kodu içeren ikili dosya biçimleri ve ana bilgisayar desteği sağlayıcıya göre değişir. [Hopper ve Ghidra kurulumu](../installation.md#hopper), isteğe bağlı native MCP bridge'i de açıklayan [IDA rehberi](../ida-provider.md) ve [Windows için deneysel Ghidra desteği](../windows-ghidra-p0.md) belgelerine bakın. Ghidra ayrıca [16 bit DOS analizini](../ghidra-dos.md) destekler. Büyük ikili dosyalar için sağlayıcının başlatma zaman aşımını `REA_GHIDRA_STARTUP_TIMEOUT_MS` veya `REA_HOPPER_STARTUP_TIMEOUT_MS` ile artırın. Sağlayıcı seçimi için [CLI rehberine](../cli.md#choose-a-provider) bakın. En son npm sürümünden sonra eklenen özellikler için [sürümlerde kullanılabilirliği](../installation.md#released-package-and-main) kontrol edin.
## Örnek çalışmalar
+1 -1
View File
@@ -166,7 +166,7 @@ REA потребує Node.js 22.x (>=22.19), 24.x (>=24.11) або 26+, а та
<a id="choosing-a-deep-analysis-provider"></a>
Нативні формати й підтримка систем відрізняються залежно від провайдера. Дивіться [налаштування Hopper і Ghidra](../installation.md#hopper), [посібник IDA](../ida-provider.md), [Kuna](../kuna-provider.md) та [експериментальну підтримку Ghidra у Windows](../windows-ghidra-p0.md). Ghidra також підтримує [аналіз 16-бітного DOS](../ghidra-dos.md). Для вибору провайдера дивіться [посібник CLI](../cli.md#choose-a-provider). Для функцій, доданих після найновішої версії в npm, перевірте [доступність у випусках](../installation.md#released-package-and-main).
Нативні формати й підтримка систем відрізняються залежно від провайдера. Дивіться [налаштування Hopper і Ghidra](../installation.md#hopper), [посібник IDA](../ida-provider.md) та [експериментальну підтримку Ghidra у Windows](../windows-ghidra-p0.md). Ghidra також підтримує [аналіз 16-бітного DOS](../ghidra-dos.md). Для вибору провайдера дивіться [посібник CLI](../cli.md#choose-a-provider). Для функцій, доданих після найновішої версії в npm, перевірте [доступність у випусках](../installation.md#released-package-and-main).
## Приклади
+1 -1
View File
@@ -166,7 +166,7 @@ Kiểm tra tĩnh JavaScript và .NET đọc các tệp được cung cấp mà k
<a id="choosing-a-deep-analysis-provider"></a>
Định dạng mã máy và nền tảng chạy REA được hỗ trợ khác nhau tùy trình phân tích. Xem [thiết lập Hopper và Ghidra](../installation.md#hopper), [hướng dẫn IDA](../ida-provider.md), [Kuna](../kuna-provider.md) và [hỗ trợ Ghidra thử nghiệm trên Windows](../windows-ghidra-p0.md). Ghidra cũng hỗ trợ [phân tích DOS 16 bit](../ghidra-dos.md). Để chọn trình phân tích, xem [hướng dẫn CLI](../cli.md#choose-a-provider). Kiểm tra [khả năng sử dụng theo bản phát hành](../installation.md#released-package-and-main) đối với tính năng được bổ sung sau phiên bản npm mới nhất.
Định dạng mã máy và nền tảng chạy REA được hỗ trợ khác nhau tùy trình phân tích. Xem [thiết lập Hopper và Ghidra](../installation.md#hopper), [hướng dẫn IDA](../ida-provider.md) và [hỗ trợ Ghidra thử nghiệm trên Windows](../windows-ghidra-p0.md). Ghidra cũng hỗ trợ [phân tích DOS 16 bit](../ghidra-dos.md). Để chọn trình phân tích, xem [hướng dẫn CLI](../cli.md#choose-a-provider). Kiểm tra [khả năng sử dụng theo bản phát hành](../installation.md#released-package-and-main) đối với tính năng được bổ sung sau phiên bản npm mới nhất.
## Các nghiên cứu điển hình
+1 -1
View File
@@ -170,7 +170,7 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
<a id="choosing-a-deep-analysis-provider"></a>
不同提供者支援的原生格式和主機系統有所不同。請參閱 [Hopper 與 Ghidra 設定](../installation.md#hopper)、[IDA 指南](../ida-provider.md)、[Kuna](../kuna-provider.md)和[實驗性 Windows Ghidra 支援](../windows-ghidra-p0.md)。Ghidra 也支援 [16 位 DOS 分析](../ghidra-dos.md)。提供者選擇見 [CLI 指南](../cli.md#choose-a-provider)。對於最新 npm 版本發佈後新增的功能,請查看[發佈可用性](../installation.md#released-package-and-main)。
不同提供者支援的原生格式和主機系統有所不同。請參閱 [Hopper 與 Ghidra 設定](../installation.md#hopper)、[IDA 指南](../ida-provider.md)和[實驗性 Windows Ghidra 支援](../windows-ghidra-p0.md)。Ghidra 也支援 [16 位 DOS 分析](../ghidra-dos.md)。提供者選擇見 [CLI 指南](../cli.md#choose-a-provider)。對於最新 npm 版本發佈後新增的功能,請查看[發佈可用性](../installation.md#released-package-and-main)。
## 案例
+1 -1
View File
@@ -170,7 +170,7 @@ REA 需要 Node.js 22.x(>=22.19)、24.x(>=24.11)或 26+,以及 npm。
<a id="choosing-a-deep-analysis-provider"></a>
不同提供方支持的原生格式和主机系统有所不同。请参阅 [Hopper 与 Ghidra 设置](../installation.md#hopper)、[IDA 指南](../ida-provider.md)、[Kuna](../kuna-provider.md)和[实验性 Windows Ghidra 支持](../windows-ghidra-p0.md)。Ghidra 也支持 [16 位 DOS 分析](../ghidra-dos.md)。提供方选择见 [CLI 指南](../cli.md#choose-a-provider)。对于最新 npm 版本发布后新增的功能,请查看[发布可用性](../installation.md#released-package-and-main)。
不同提供方支持的原生格式和主机系统有所不同。请参阅 [Hopper 与 Ghidra 设置](../installation.md#hopper)、[IDA 指南](../ida-provider.md)和[实验性 Windows Ghidra 支持](../windows-ghidra-p0.md)。Ghidra 也支持 [16 位 DOS 分析](../ghidra-dos.md)。提供方选择见 [CLI 指南](../cli.md#choose-a-provider)。对于最新 npm 版本发布后新增的功能,请查看[发布可用性](../installation.md#released-package-and-main)。
## 案例
-43
View File
@@ -1,43 +0,0 @@
# Reconstruction coverage
`rea evaluate-reconstruction-coverage coverage.json --json` evaluates an inline
coverage workspace and its selected `boundary_id`. MCP exposes the same input
through `evaluate_reconstruction_coverage`. Results contain a status, reasons,
recommended probes and Evidence references; the supplied bundle retains the
complete records.
Evidence IDs and bundle manifests must authenticate before evaluation. A valid
record can still be insufficient proof: a passing verifier summary cannot
promote inferred Evidence into a replay observation. That case returns
`unknown`, while a missing or tampered record remains an input error.
## Comparable verifier Evidence
At least one record explicitly cited by the latest verifier observation must
support the declared contract authority:
| Contract authority | Evidence source | Confidence |
| ------------------- | ----------------------------------------- | ----------------------- |
| `shipped-artifact` | `shipped-artifact` | `observed` or `derived` |
| `controlled-replay` | `controlled-replay` | `observed` |
| `live-observation` | `controlled-replay` or `external-service` | `observed` |
| `external` | `external-service` | `observed` |
Contextual references can accompany qualifying proof. An unrelated observed
record elsewhere in the bundle cannot repair a result that cites only
inference. Older observed results cannot replace the newest weak result.
Equivalent observations at the same instant can share qualifying proof from
one of those observations; conflicting tied assessments remain unknown.
## Package and integration proof
Build, clean-install and runtime-routing claims require observed runtime
Evidence (`controlled-replay` or `external-service`). Package-content and
authority-independence audits can also use observed or derived
`shipped-artifact` Evidence. Each proof must cite its qualifying records and
cover the selected boundary's artifact digests. An inference alone establishes
neither a passing nor a failed package workflow.
The evaluator preserves Evidence provenance and returns an actionable unknown
when qualification is unavailable. It does not change the cited records to
make their authority match a caller's declaration.
@@ -323,6 +323,3 @@ disposition but does not silently count it as verified.
The tool returns every obligation inline. Its `ledger_id` and `closure_digest`
identify the complete result, including summaries, reports, ownership and
dependency graphs, Evidence links, and limitations.
For named boundaries with explicit verifier contracts and package-proof kinds,
see [reconstruction coverage](reconstruction-coverage.md).
+2 -3
View File
@@ -16,12 +16,11 @@ can run.
`inspect_with_objdump` and `rea inspect-with-objdump` capture stdout and stderr as inline Evidence. Artifact inspection Evidence records SHA-256 digests before and after the provider call and whether those observed digests differ; this cannot prove the exact bytes consumed if the artifact changes during execution. Choose one artifact operation: `file_headers`, `section_headers`, `symbols`, `relocations`, `dwarf`, `disassemble`, or `disassemble_all`. `disassemble` maps to `-d`; `disassemble_all` maps to `-D` and has different symbol-boundary semantics. The pathless `inspect_with_objdump {"operation":"architectures"}` MCP call or `rea list-objdump-architectures` CLI command maps to `-i` and produces subjectless provider-capability Evidence describing BFD targets compiled into the installed binary, not a guarantee that a target is supported.
For DWARF, REA does not follow local debug links by default. Set `follow_debug_links` to `true` to pass `-WK`; external debug-file identities are not bound to Evidence when following links is enabled. The CLI flag is `--follow-debug-links`; `--no-follow-debug-links` or `--follow-debug-links=false` keeps the default. A separate `true` or `false` word after the flag is not its value. REA clears `DEBUGINFOD_URLS` in the objdump child environment to disable debuginfod network access on GNU binutils versions that do not support `-WE`. The complete raw output is authoritative; REA does not normalize it into a version-independent disassembly schema.
For DWARF, REA does not follow local debug links by default. Set `follow_debug_links` to `true` to pass `-WK`; external debug-file identities are not bound to Evidence when following links is enabled. The CLI option is `--follow-debug-links true`. REA clears `DEBUGINFOD_URLS` in the objdump child environment to disable debuginfod network access on GNU binutils versions that do not support `-WE`. The complete raw output is authoritative; REA does not normalize it into a version-independent disassembly schema.
```sh
rea inspect-with-objdump ./program disassemble
rea inspect-with-objdump ./program dwarf
rea inspect-with-objdump ./program dwarf --follow-debug-links
rea inspect-with-objdump ./program dwarf --follow-debug-links false
```
Captured stdout and stderr are retained up to a 16 MiB combined limit; `output_truncated` marks output beyond that limit.
-14
View File
@@ -105,20 +105,6 @@ overall native heap and total resident memory are not capped. Row limits do not
override these byte and resource limits. Resource-limit failures do not return
an incomplete schema as successful complete evidence.
If the combined database and WAL exceed the snapshot budget, the refusal is a
`resource_constraint` and reports both the configured byte limit and the exact
observed combined size as a decimal string. The remediation directs callers to
create a smaller consistent offline snapshot containing only relevant data
through the application's supported export workflow. A generic backup or copy
does not guarantee a smaller result; REA does not truncate or modify the
selected source.
On hosts that cannot address the derived WAL or rollback-journal path for a
long selected database name, REA reports that the SQLite capability is
unavailable. It does not assume an inaccessible sidecar is absent. Use the
application's supported consistent offline export or copy workflow to place
the complete known file set at a shorter path.
MCP also applies its configured response budget to the complete Evidence
envelope. If delivery exceeds that budget, it reports a transport constraint
with guidance for exporting retained Evidence, requesting a matching larger
+1 -83
View File
@@ -199,44 +199,6 @@ Measure slow cases before pruning capacity coverage. Retain inputs beyond the
formerly failing size/depth and distinguish cold runs, warm builds, coverage,
and toolchain conditions when comparing timings.
## Lifecycle regressions
For changes to resource acquisition, readiness, cancellation, or cleanup, select
the affected owners and applicable scenarios below. Include temporary snapshots
and one-shot commands. Use public CLI/MCP workflows where practical and a
controlled production seam for interleavings that cannot be reproduced reliably.
| Scenario | Assertion and control |
| --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Target admitted; startup still pending | Status preserves the readiness distinction and observed startup activity; readiness requires the provider's documented proof. |
| First handshake is late, absent, invalid, or recovered during close | Initial and recovery paths retain the same authenticated identity facts. Foreign run/token controls cannot authorize cleanup; authenticated idle/native work survives the unauthenticated listener deadline. |
| Launch returns an error with a valid process handle | Cover already-exited and still-live owned processes. Verify observed exit separately from creation; failed termination or unavailable identity remains unresolved. |
| Cancellation arrives after workload exit, during result reads or temporary-root removal | Return typed cancellation with available partial observations and actual cleanup status. The non-aborted control succeeds; existing primary/cleanup failures keep their precedence. |
| Children have mixed health or cleanup outcomes | Missing peer health does not erase an observed failure or activity. Idle plus unknown stays unknown. Retry a failed child without closing a successful peer again, including after a close-option change. |
| Cleanup fails after success or primary failure | Public output retains the completed observation or primary diagnostic, remaining resources, and owner recovery action. Restore the denied resource and verify the owning cleanup retry. |
| Target exits without an acknowledgement, or transport disappears while it stays alive | Positive owned exit proof resolves target lifetime; other resources still need cleanup. Lost transport alone does not prove exit. Verify unrelated/attached targets survive and same-connection recovery works when supported. |
Synchronize cancellation with the finalization boundary being tested; a fixed
delay after dispatch does not establish that phase. Use markers, barriers, or
owned cleanup seams while preserving real cleanup where feasible. Check server
settlement and retained resources even if the MCP SDK rejects the client promise
before the handler finishes. A cancellation during active execution does not
cover cancellation during final cleanup.
For atomic writes, exercise cancellation on both sides of the documented commit
point and preserve any publication receipt. A late interruption does not prove
that no output was published.
Representative regressions include
[process cleanup-time SIGINT](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/process/processCaptureCancellationAdapters.test.ts),
[late authenticated cleanup](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/providers/hopper/hopperClient.handshakeRecovery.test.ts),
[LLDB launch and finalization](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/providers/native/lldbCallTracerFailures.test.ts),
[composed child ownership](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/src/application/binary/CompositeProvider.test.ts),
and [public CLI cleanup denial](https://github.com/morluto/rea/blob/e78455338fe2d06c4cd03d59cbf33473d754998b/tests/boundary/cli/wasmCleanupCli.test.ts).
These controlled probes do not replace the matching real-provider lane. Record
the base failure, passing control, exact host/provider, verified cleanup, and
unavailable coverage; a missing engine is not a pass.
## Behavioral depths
| Depth | Location | What it proves |
@@ -855,38 +817,6 @@ Native IDA MCP verification uses `"protocol": "native"` with an already-running
The initial Linux x64 IDA Free 9.4 run used native MCP 1.1.2 with a local navigation correction. Windows x64 verification uses **IDA Pro 9.3**, the two-function public fixture, and the shipped Windows `whoami.exe` application. The native plugin was built from upstream `c9d15d98e8a143fc74f6e14037c48f63889b6d6c`, including the merged navigation fix, with a local loopback-listener change. Production CLI/MCP parity, switching, identity and cleanup passed. Correctness and performance measurements are recorded in the [Windows native verification notes](https://github.com/morluto/rea/blob/main/experiments/analysis-efficiency/ida-native-windows.md). Stock 1.1.2 switching, native macOS, and Windows IDA Free remain unverified. See the [native bridge prerequisites](ida-provider.md#optional-native-ida-mcp-bridge).
## Kuna lane
`npm run verify:kuna` requires a caller-supplied `kuna` in `REA_KUNA_COMMAND`
and a host C compiler (`REA_CC`, default `cc`) with `strip` and `nm`. It builds a
source-owned fixture with non-inlined calls, with symbols and stripped, then verifies through MCP that
callers, callees and exact incoming xrefs agree with those calls, graph/path
composition reaches the callee, reference-only queries leave project/staging
entries unchanged, repeated relationships reuse the graph, and
the first cold pseudocode answer comes from a single-function run or an export
that already finished, assembly waits for and serves from that export, and
pseudocode and both dossier facets match their recorded file spans and digests,
a fresh server reuses the export without rebuilding it, staging is empty after
a completed export, and no owned process survives. Pass
`--targets DIR --functions a,b` to add caller-built binaries laid out as
`DIR/<flavor>/<name>` beside `<name>.stripped`; addresses come from the
symbol-bearing build. `--compare-ghidra` with `GHIDRA_INSTALL_DIR` runs those
caller-built functions through Ghidra (it refuses to run without `--targets` and
`--functions`, an empty matrix, or names missing from every supplied target) and
reports latency, sampled process-tree CPU and peak memory, UTF-8 response size,
and goto and warning-marker counts. Those counts are not a correctness judgement,
and model tokens are not measured. Reference timings and response sizes are
reported separately; reference caches expire when the server closes. `--report PATH` writes the JSON report.
The first matrix run used Kuna 1.833 and Ghidra 12.1.4 with JDK 21 on macOS
arm64. It covered zlib 1.3.2 `minigzip`, coreutils 9.9 `sort` and the sqlite
3.54.0 shell, built at `-O2` for ELF x86-64, ELF AArch64, Mach-O arm64 and
Mach-O x86-64, each with symbols and stripped. A separate Linux x86-64 run verified
the fixture and system zlib 1.2.13 / SQLite 3.40.1 libraries through MCP, including
cached byte spans and a Ghidra comparison. See the [Linux report](https://github.com/morluto/rea/pull/3854#issuecomment-6185870391)
for measured scope and limitations. Windows, PE and universal Mach-O remain
unverified by this lane.
## DOS Ghidra analysis
`npm run verify:ghidra:dos` requires the supported Ghidra and JDK installation
@@ -1027,19 +957,7 @@ the Apple or Windows curated lanes invoke explicitly, which also select that
lane. Edits to verifier entrypoints run only by manual, licensed or self-hosted
workflows run static checks and the Linux suite. Website changes run website
checks. Provider workflow changes validate the workflow and run that provider's
verifier. Changes under `bridge/ghidra/` also select the `ghidra_bridge` lane,
which compiles the packaged bridge sources against the pinned Ghidra 12.1.4
jars with `npm run verify:ghidra:bridge-sources`. It never launches Ghidra or
imports a target, so a missing import fails the PR instead of every Ghidra
session. The launcher, compiler and package checks share named required source
roles. The manifest must match the complete Java cohort, so deleting a helper
or adding an unregistered dependency cannot silently change the shipped set.
The preflight reuses the provider's Java Properties parser with the pinned Node
toolchain's native type stripping, without an npm install or runtime build.
The shared release policy owns the verified version and JDK major. CI checks
the archive pin before downloading and verifies the extracted SDK version;
changes to these source owners also select the compile lane.
Manual-provider/Android workflow changes run workflow validation only.
verifier. Manual-provider/Android workflow changes run workflow validation only.
The four-version SQLite compatibility matrix has its own `sqlite` scope for
SQLite implementation, contracts, CLI/MCP registration, fixtures, and shared
runtime boundaries. SQLite and server-lifecycle test edits and the SQLite npm
-48
View File
@@ -68,54 +68,6 @@ satisfy. A plan without execution does not establish runtime behavior.
constraints. Report pagination, truncation, cancellation, and partial failure.
An assumed agent budget is insufficient justification for a limit.
## Own the lifecycle
For tools that acquire resources, identify the owner and the observations that
establish each transition. Include temporary files and one-shot commands, as
well as persistent engine sessions. Use the existing provider contracts;
providers need not expose identical state machines.
- Keep target admission, provider readiness, active work, observed process exit,
and resource release separate. A valid process handle, successful launcher,
or selected target establishes only its own fact. Publish available identity
and exit observations even when launch fails; retain unknown ownership.
- Authenticate and capture identity through the same boundary on initial and
recovery handshakes. Bound abandoned unauthenticated listeners where the
protocol permits it; that deadline must not expire authenticated idle sessions
or long-running native work. Listener expiry does not prove target closure.
- Shared startup has one producer and independently cancellable caller waits.
Keep it running while any caller still needs it. The final departing caller
waits for owned cleanup and retains its failure. Provider closure is distinct
from caller cancellation. Retained cleanup capabilities block replacement;
derive readiness from the current startup's completion and live connection.
- For cancellable observations, carry cancellation through result consumption
and awaited cleanup. Before returning success, check the signal after the
last awaited finalization step.
Retain collected observations under typed cancellation. Preserve an existing
primary failure and let incomplete cleanup retain its own classification.
Client cancellation alone does not prove native work stopped.
For atomic publication, document the cancellation cutoff and preserve any
committed output's publication receipt.
- Release only owned resources. Attaching owns a connection or session unless
the provider contract explicitly establishes target ownership. A helper exit,
missing reply, or missing PID does not establish target exit.
- Retain successful child cleanup and retry only unresolved children, including
when close options change. Preserve observed child failure or activity when
another child's health is unavailable; idle plus missing coverage stays
unknown.
Bind cleanup failures to their exact resource owner. Resolve current cleanup
uncertainty only after that owner confirms release, while preserving the
original failure and completed observations. Later failure composition must
report only resources whose cleanup remains unresolved.
- Await one-shot cleanup without discarding the completed result or primary
diagnostic. Preserve partial observations, remaining resources, and the cleanup
owner's recovery instructions. CLI adapters should reuse
[`finishAnalysisCommand`](https://github.com/morluto/rea/blob/main/src/cli/finishAnalysisCommand.ts) where its contract
applies rather than duplicate close/finally handling.
Exercise the applicable [lifecycle regression scenarios](testing.md#lifecycle-regressions)
through the producing owner and affected CLI/MCP consumers.
## Implement and evaluate
Put provider protocols/parsing in adapters and shared CLI/MCP workflows in the
@@ -1,249 +0,0 @@
{
"base_revision": "e598b4656ad90d59ede2d10454a0ac34f3813558",
"scope": "Canonical effect metadata only; real static Obsidian reads, no target execution",
"baseline_source_sha256": {
"src/contracts/toolEffects.ts": "39505b5f1501768515229a37ff67e8a1a512faedfb6669aee7a690a2ac3ac633",
"src/javascript/analysis/JavaScriptAnalysisWorker.ts": "49ea4e6344060fae3529ffe6e14bfadc534132067cfa5c370263dc7476375be0",
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1"
},
"final_source_sha256": {
"src/contracts/toolEffects.ts": "feb66f1b71c4a6849d1730227c9c74e4efbb67b4537d75fd41ce9ff615c25753",
"src/javascript/analysis/JavaScriptAnalysisWorker.ts": "49ea4e6344060fae3529ffe6e14bfadc534132067cfa5c370263dc7476375be0",
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"tests/boundary/mcp/electronMcp.test.ts": "e483bec8512cfeafa7f951c8faf25e1aecd6e6987ea74aead14230501912928d",
"tests/boundary/mcp/toolSchemaValidity.test.ts": "ab6d9ba37035a6ff9279b820aa8660c50a3ef9f31d25f838c0ba40929148e89c"
},
"cohorts": {
"baseline": {
"metadata": {
"tool": "analyze_javascript_application",
"advertised_annotations": {
"readOnlyHint": false,
"destructiveHint": false,
"idempotentHint": true,
"openWorldHint": false
},
"session_effects": ["mutatesSession", "idempotent"],
"session_annotations": ["idempotent"],
"catalog_tool_count": 207
},
"metadata_probe_resources": {
"elapsed_seconds": 10.711,
"cpu_seconds": 10.503415,
"peak_cgroup_bytes": 842891264,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
},
"real_workflow": {
"artifact": {
"path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f"
},
"parent_evidence_id": "ev_73ef06e3b9b944f54c5257f7960279127d8518d2a73cd8618bd045dbe44e6b5a",
"samples": [
{
"round": 0,
"mode": "references",
"elapsed_ms": 109.19401700000162,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
}
],
"token_costs": {
"tokenizer": "cl100k_base",
"tiktoken_version": "0.12.0",
"costs": {
"references": {
"selected_catalog": 7977,
"full_catalog": 195522,
"common_request": 53,
"common_summary": 2675,
"requests": 102,
"structured_replies": 5741,
"selected_catalog_task_total": 16548,
"full_catalog_task_total": 204093
}
}
},
"cli_parity": {
"evidence_id": "ev_b3586dfdf7d4f01157f8f2978326aaa7103987110f2df4fa5bf3331ff2898749",
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"exact_cli_mcp_view_parity": true
}
},
"real_workflow_resources": {
"elapsed_seconds": 30.47,
"cpu_seconds": 29.939807,
"peak_cgroup_bytes": 1551282176,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"final": {
"metadata": {
"tool": "analyze_javascript_application",
"advertised_annotations": {
"readOnlyHint": false,
"destructiveHint": false,
"idempotentHint": true,
"openWorldHint": true
},
"session_effects": [
"mutatesSession",
"writesFilesystem",
"launchesProcess",
"idempotent"
],
"session_annotations": ["idempotent", "open_world"],
"catalog_tool_count": 207
},
"metadata_probe_resources": {
"elapsed_seconds": 10.909,
"cpu_seconds": 10.664997,
"peak_cgroup_bytes": 853958656,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
},
"real_workflow": {
"artifact": {
"path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f"
},
"parent_evidence_id": "ev_73ef06e3b9b944f54c5257f7960279127d8518d2a73cd8618bd045dbe44e6b5a",
"samples": [
{
"round": 0,
"mode": "references",
"elapsed_ms": 110.17946600000141,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
}
],
"token_costs": {
"tokenizer": "cl100k_base",
"tiktoken_version": "0.12.0",
"costs": {
"references": {
"selected_catalog": 7977,
"full_catalog": 195522,
"common_request": 53,
"common_summary": 2675,
"requests": 102,
"structured_replies": 5741,
"selected_catalog_task_total": 16548,
"full_catalog_task_total": 204093
}
}
},
"cli_parity": {
"evidence_id": "ev_b3586dfdf7d4f01157f8f2978326aaa7103987110f2df4fa5bf3331ff2898749",
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"exact_cli_mcp_view_parity": true
}
},
"real_workflow_resources": {
"elapsed_seconds": 31.059,
"cpu_seconds": 30.663163,
"peak_cgroup_bytes": 1584574464,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
}
},
"conditions": "Separate fresh MCP sessions, uncontrolled warm filesystem/build and shared-host scheduling. One CPU, 8 GiB hard cap, 6 GiB early stop, no swap. Resource counters are at payload exit with the observer alive, not final whole-scope counters; memory includes charged file cache, not summed RSS. Whole real-verifier resources include analysis, export, CLI parity and tokenizer; metadata probe resources are separate. Static BPE uses name/description/inputSchema catalog projections plus requests/structured replies, excludes annotations, metadata-probe calls, rendering/replay/reasoning/final-answer/model usage. No performance improvement or equivalence bound established by one before/after pair.",
"regressions": {
"baseline": "20 executed: 2 expected metadata failures, 18 passed",
"final": "27 passed after the test ownership cleanup across canonical effects and MCP/catalog boundaries",
"interim_correction": "26 passed before metadata assertions were split into their own target-free case to satisfy function-size lint"
},
"selected_input": {
"input_path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
"detail": "summary",
"source_paths": ["package.json", "main.js"]
},
"selected_view": {
"kind": "javascript",
"collection": "modules",
"projection": "references",
"where": {
"path": "main.js",
"kind": "javascript-module"
}
},
"observer_sources_sha256": {
"resource-run.py": "7728862171415e7dee2c11af2ead2b9a54265fccaf82a7113956d36b29065e33",
"resource_payload.py": "6f089e645459eba80c5d73e39c21606c78173a44daf49fabc3b88c4631ce51c6"
},
"runtime": {
"platform": "Linux x64",
"node": "24.18.0",
"npm": "11.16.0",
"tiktoken": "0.12.0"
}
}
@@ -1,121 +0,0 @@
# JavaScript analysis effect metadata verification
Tracks #3867 under #4027. The canonical effect record omitted existing internal
worker launches and source staging. `CapabilityInventory` and SDK registration
already consume that owner directly; no adapter override is required. The
[worker](../../src/javascript/analysis/JavaScriptAnalysisWorker.ts) creates a
private runtime root, stages `.source` files and launches an owned REA worker.
These are source-derived facts; the analyzed application code is not executed.
## Executed results
Public stdio MCP before/after:
| Surface | Before | After |
| ---------------------------- | ------------------------------ | ------------------------------------------ |
| `tools/list` `openWorldHint` | `false` | `true` |
| `binary_session` effects | `mutatesSession`, `idempotent` | also `writesFilesystem`, `launchesProcess` |
| Session annotation flags | `idempotent` | also `open_world` |
| Advertised tools | 207 | 207 |
Regression assertions failed on the original contract: **2 failed,
18 passed**. After correction and separating the target-free metadata case, **27 passed** across canonical effects, public
Electron/JavaScript MCP and SDK-converted catalog validity. The tests retain the
non-destructive, idempotent hints and exact session effects; no foreign target
execution is declared.
The existing real verifier analyzed official Obsidian 1.14.4 with `package.json`
and `main.js` selected, then read the typed module references and verified exact
public CLI/MCP parity. Both revisions returned the same parent/view Evidence
identities and 23-row fact digest. Query replies remained 22,158 bytes and the
selected-schema task remained 16,548 `cl100k_base` BPE tokens (204,093 with the full
name/description/inputSchema catalog projection). These are static counts, not
model billing; annotations, the separate metadata probe, host rendering/replay,
reasoning and final answers are outside this projection. No hosted agent was run.
| Complete real verifier | Before | After |
| ---------------------- | ------------: | ------------: |
| Elapsed seconds | 30.470 | 31.059 |
| CPU seconds | 29.939807 | 30.663163 |
| Peak cgroup bytes | 1,551,282,176 | 1,584,574,464 |
The scope includes shared analysis, temporary export, CLI parity and tokenizer.
The metadata-only probe separately used 10.711/10.909 s wall,
10.503415/10.664997 s CPU and 842,891,264/853,958,656 B peak cgroup memory.
All scopes used one CPU, an 8 GiB hard cap, 6 GiB early stop and no swap; cleanup
was verified with no OOM. Memory includes charged file cache, not summed RSS.
Counters were observed at payload exit while the observer remained alive.
Fresh MCP sessions do not establish an OS-cold cache. Shared-host scheduling and
filesystem/build warmness were uncontrolled; one pair establishes neither a
performance improvement nor an equivalence bound. Linux x64 static ASAR analysis
was executed; live Electron and other platforms were not exercised locally.
[Compact records and source hashes](javascript-analysis-effects-2026-10-15.json)
preserve these distinctions without archiving the downloadable application.
## Reproduce
Use pinned Node/npm, `npm ci`, then `npm run build:cached`. Run the following as
an ES module from the checkout root (`node --input-type=module < probe.mjs`):
```javascript
import assert from "node:assert/strict";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { toolAvailabilityInventory } from "./dist/contracts/toolOutputSchemaPrimitives.js";
import { closeVerifierResources } from "./scripts/lib/verifier-cleanup.mjs";
const client = new Client({
name: "javascript-effect-verification",
version: "1",
});
const transport = new StdioClientTransport({
command: process.execPath,
args: ["scripts/rea.mjs", "mcp"],
stderr: "pipe",
});
let primaryError;
try {
await client.connect(transport);
transport.stderr?.resume();
const { tools } = await client.listTools();
const tool = tools.find(
({ name }) => name === "analyze_javascript_application",
);
assert.ok(tool);
const response = await client.callTool({
name: "binary_session",
arguments: {},
});
assert.notEqual(response.isError, true);
const inventory = toolAvailabilityInventory.parse(
response.structuredContent?.result?.tool_availability,
);
const group = inventory.groups.find(({ names }) => names.includes(tool.name));
assert.ok(group);
console.log(
JSON.stringify({
tool: tool.name,
advertised_annotations: tool.annotations,
session_effects: group.effects,
session_annotations: group.annotations,
catalog_tool_count: tools.length,
}),
);
} catch (cause) {
primaryError = cause;
throw cause;
} finally {
await closeVerifierResources(
[() => client.close(), () => transport.close()],
primaryError,
);
}
```
For the real workflow, follow the artifact acquisition/digest instructions in
[module-reference verification](javascript-module-references-2026-10-15.md).
Run its existing command with `--mode references --rounds 1 --cli-parity` and the
same selected source paths and tokenizer. That verifier is unchanged by this
fix. The preserved 8 GiB cgroup observer and its source hashes are documented in
the same guide. The machine-readable records separate metadata-probe resources
from the complete real-workflow resources.
File diff suppressed because it is too large. Load diff
@@ -1,245 +0,0 @@
# Interrupted application processing: applicability and Evidence
Tracks #3948 under #4027. Measured baseline:
`c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb`.
The question is which processing phases remain unavailable after a source
interruption, while keeping native inventory and already collected facts.
Native identity and provider linkage must stay independent of application-wide
partial coverage. This is static artifact analysis; no target application or
native provider runs.
## Observed failure and correction
The interruption producer previously attached one generic failure to every
remaining relevant file. Graph projection interpreted it as JavaScript AST work.
Real Joplin reproduced an `analyze-javascript-source` / `ast-static-analysis`
unavailable scope on keytar's native binary and on unvisited package, HTML and
source-map files. Public CLI/MCP controls reproduced the same applicability
problem. Deterministic cancellation before native metadata additionally produced
a phantom JavaScript semantic module for the binary.
The producer now records the applicable operation/authority and whether the
phase was entered. `analysis_status: interrupted` includes worker startup and
does not imply parsing completed; `not-started` identifies unvisited phases.
Native add-ons need no post-inventory source-processing phase, so neither interrupted
nor caller-excluded native files acquire AST scopes or semantic modules.
Actual JavaScript failures, structured-file unknowns, caller selection and byte
identity remain available. The graph consumes these produced facts.
The AST-only unavailable Evidence helper is replaced by an explicitly typed
static helper. Normal package/JSON/map gaps also retain their artifact-byte
authority. Per-file progress uses the same applicability owner: JavaScript keeps
its source phase, and metadata uses `process_javascript_artifact_metadata`.
Remaining parsers do not run after interruption. Failure sharing is limited to
five applicable phases during one interruption; no persistent cache is added.
A second review found the alternate full-analysis semantic builder still
created modules for every excluded file kind. An actual filesystem-producer
regression on revision `323fae8cbf3e36677bcb7b22ba31fa1a7e3c5cd4` reproduced
phantom native/JSON/HTML/package/source-map modules, while selected and excluded
JavaScript were positive controls. Semantic applicability now belongs to the
shared `recordUnavailableFile` projection entry point, before retaining limits,
nodes, roots or unknowns. Three duplicate caller checks are removed. Both full
and incremental construction use this boundary; actual excluded JavaScript
continues to carry source-bound unknowns and partial semantic coverage.
## Real input and reproduction
Use the [official Joplin v3.7.21 acquisition and extraction instructions](https://github.com/morluto/rea/blob/c7f6b8bad1a34d5efb30ff2a017672dea1fcdabb/docs/verification/javascript-native-handoff-identity-2026-10-15.md).
The AppImage SHA-256 is
`5d202dcaca31e14206826aa3f04125e7c218dbc3dc3d98479d811efbd6b856c1`;
ASAR SHA-256 is
`767df409b18effe60421fbefcca873de237c7e813eac744ecdeebbc5e825a33a`.
The 6,324-file inventory includes 46 unpacked entries. Both cohorts stage all
three native companions; the other 43 unpacked entries remain unavailable.
Build with pinned Node 24.18.0/npm 11.16.0 and locked dependencies. The extended
existing verifier also runs against the baseline runtime; copy the verifier and
`scripts/lib/javascript-file-scope-oracle.mjs` from the fixed revision into a
built baseline checkout when reproducing the failure.
```bash
artifact_path="/absolute/path/to/cases/available/app.asar"
common=(
--artifact "$artifact_path"
--addon-path node_modules/keytar/build/Release/keytar.node
--native-sha256 0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a
--source-path ElectronAppWrapper.js
--source-path ElectronAppWrapper.js.map
--source-path index.html
--source-path package.json
--source-path node_modules/keytar/build/Release/keytar.node
--structured-path ElectronAppWrapper.js.map
--structured-path index.html
--structured-path package.json
--structured-expectations '{"ElectronAppWrapper.js.map":{"operation":"parse-local-source-map","analysis_status":"not-started"},"index.html":{"operation":"parse-html-script-references","analysis_status":"not-started"},"package.json":{"operation":"parse-package-json","analysis_status":"not-started"}}'
--rounds 3 --cli-parity
--tokenizer-python /absolute/path/to/python-with-tiktoken-0.12.0
)
node scripts/verify/javascript/native-handoffs.mjs "${common[@]}" \
--analysis-timeout-ms 1 --expect-timeout --require-safe
node scripts/verify/javascript/native-handoffs.mjs "${common[@]}" \
--validate-structured-from-export --require-safe
```
Omit `--require-safe` on the baseline timeout run to record its incorrect
applicability. Its diagnostic exit 0 is not a passing analyst task. A supported
1 ms deadline includes worker startup; the verifier requires a typed timeout
and retained partial reference for that cohort. Each structured path supplies an
independent expected operation and phase-entry status; the safety oracle matches
all gaps exactly, including unavailable state and unknown confidence. Missing or
conflicting expectations do not pass.
The native path is explicitly selected. An earlier structured-only run excluded
it and therefore did not reproduce the native interruption. Input selections
and that preliminary result remain recorded. Both final cohorts preserve
partial coverage for unselected/unavailable scopes.
The supplied native Evidence is simulated solely to check exact subject linkage.
The verifier checks physical companion bytes and the exported parent's root
digest, validates public CLI/MCP native-trace parity, and reports only after
owned transport/command/private-file cleanup. Applications and large Evidence
are regenerable and are not archived.
## Two distinct task projections
The **timeout task** includes the expected typed-timeout preparation, one inline
native trace and three inline structured-path traces. It checks that native has
no AST failure and that each unvisited structured phase remains unavailable with
the appropriate artifact-byte authority and `not-started` status.
The **default-deadline control** models the summary plus one inline native trace.
Its named structured paths are additionally validated from the already exported
complete parent, explicitly labelled `exported-parent-validation`. Those checks
are diagnostic overhead outside the modeled native-task tokens and inside total
verifier resources. They are not presented as inline metadata answers.
A control that also requested the complete outgoing `package.json` feature trace
hit stdio admission twice: at least 10,587,373 bytes against a 10,484,736-byte
result budget, with identical retained parent/trace identities. It is not a
completed equivalent task or an efficiency baseline. [#4190](https://github.com/morluto/rea/issues/4190)
records the separate need for exact retained metadata/native fact views. Complete
export remains available. Do not compare costs across these different tasks.
## Measurements
| Cohort | Revision | Wall s | CPU s | Peak cgroup bytes | Applicability |
| ------------------------------ | -------- | ------: | ---------: | ----------------: | ------------- |
| Timeout, four inline traces | baseline | 60.738 | 60.289464 | 3,092,045,824 | wrong scope |
| Timeout, four inline traces | final | 57.515 | 57.089883 | 2,763,276,288 | pass |
| Default control, native inline | baseline | 61.542 | 61.135522 | 2,808,336,384 | pass |
| Default control, native inline | final | 186.085 | 142.371100 | 2,959,585,280 | pass |
| Defined task | Selected catalog + parsed/structured replies BPE | Selected catalog + complete tool replies BPE |
| -------------- | -----------------------------------------------: | -------------------------------------------: |
| Timeout | 21,136 → 20,161 | 36,468 → 34,471 |
| Native control | 11,776 → 11,720 | 17,738 → 17,632 |
The full-catalog alternatives and per-query samples are preserved in JSON.
Native content identity, observed byte provenance, source selection and handoff
provider Evidence remain equal. Corrected global provenance changes derived
parent/trace identities; null partial-Evidence subjects remain null. Both final
runs have exact CLI/MCP native-trace parity and match the earlier post-fix
controls in every recorded non-timing field.
The whole verifier includes input hashing, analysis, three native trace rounds,
applicable structured queries, export, CLI parity and tokenization. Every heavy
run uses the preserved [resource observer](https://github.com/morluto/rea/blob/b3bdabab694af1f9801eff7caf75001b92314a58/experiments/analysis-efficiency/resource-run.py):
one CPU, 8 GiB hard cap, 6 GiB early stop and no swap. Cgroup peaks include charged
file cache, not summed RSS. Counters are observed at payload exit while the
observer remains alive, not final whole-scope counters. Completed runs verify
owned cleanup and report no OOM.
Each MCP process is fresh, but input hashing warms its file cache. Build/OS caches
and shared-host scheduling are uncontrolled. First and repeated retained-query
wall samples remain separate; isolated query CPU/memory are unmeasured. An
earlier fixed timeout took 150.173 s wall / 119.468013 s CPU; its exact unchanged
runtime repeat took 58.827 s / 58.482996 s, with identical non-timing facts,
Evidence and BPE. Both receipts are retained. The final default control also took 186.085 s wall / 142.371100 s CPU; its
unchanged-runtime repeat took 62.651 s / 62.240964 s with identical non-timing
results. The slow final row remains reported. Attribution remains unresolved;
these runs establish no speedup or performance-equivalence bound.
The stricter post-review verifier reruns also passed without changing production
code: timeout 162.645 s wall / 125.876114 s CPU / 2,809,643,008 B peak; default
control 61.367 s / 60.970077 s / 2,526,949,376 B. Both preserve the original final
analytical fields, identities, coverage and BPE exactly; only timings and the new
explicit oracle-input field differ. These additional receipts do not replace the
slower rows above.
After centralizing semantic applicability, both real public controls passed again:
timeout 60.443 s wall / 59.970344 s CPU / 3,078,295,552 B peak; default control
63.574 s / 63.177884 s / 3,189,719,040 B. Analytical results, identities, coverage,
BPE and exact CLI/MCP parity remain unchanged. JSON identifies the runtime and
verifier source manifests for each cohort and keeps all earlier receipts.
Static `cl100k_base`/tiktoken 0.12.0 projections include the selected or complete
name/description/inputSchema catalog, requests and structured Evidence or parsed
text errors. Complete-tool-reply alternatives include MCP text/structured
duplication. The selected two-tool catalog is 5,570 BPE; the complete projection
is 195,522. Output schemas, host rendering/replay, notifications, model reasoning,
final answers and billing are unmeasured. Repeated rounds, export, CLI parity,
tokenization and exported-parent diagnostics are validation overhead outside the
defined task tokens but inside resource receipts. Removing unsupported claims
from the incorrect timeout baseline earns no efficiency credit.
## Verification and limitations
The initial production fix had 49 distinct passing cases across eight files. The initial
pre-progress-cleanup seven-file selection passed 44/44. After progress cleanup,
the same selection passed 43 and hit the existing 30-second test deadline in one
native identity case; an unchanged two-file rerun passed both native cases and
five public CLI cancellation cases. The original deadline was not increased.
The runtime verifier collects interruption-bearing scopes; separate regressions
cover native caller exclusion and malformed structured data. Source selection,
malformed structures, actual worker resource constraints,
retained large partial observations, cancellation/publication/cleanup, native
identity and both public adapters remain covered.
The first review found that the real verifier previously accepted any non-JavaScript
operation with artifact-byte authority and ignored phase-entry status. The
scenario now supplies exact operation/status expectations independently of the
production phase map. Six additional oracle cases use the actual retained
Joplin snapshots and reject wrong metadata operations, entered or missing
statuses, and absent or conflicting gaps. They passed, bringing the distinct
coverage to 55 cases across nine files. The production implementation did not
change during this review correction.
After the shared semantic-boundary correction from the second review, 78
distinct cases passed across ten files: the nine-file selection passed 65/65,
and the original root reconstruction file passed 13/13. This includes the
alternate-builder owning test file's 23 cases. The new producer-boundary
regression checks both construction paths, all five excluded metadata/native
kinds, and selected/excluded JavaScript controls. Real Joplin controls exercise
the public incremental workflow; the alternate full-builder path has filesystem
boundary verification rather than a separately measured real-app workflow.
Recorded attempts also include the verifier's initial non-null-subject assumption
and an invalid top-level semantic-module-path absence check. Both were corrected
before their assertions were used as proof. Corrected cancellation reproduction
shows the actual native AST scope and phantom module on the baseline. Final
source and execution-driver versions, source pins, input digests, prior attempts
and resource receipts are in the [compact JSON record](javascript-interruption-applicability-2026-10-15.json).
Before review, `npm run check && npm run docs:check` passed, including five consistency
tests: 165.553 s wall / 150.247257 s CPU /
4,232,491,008 B peak cgroup memory, clean owned exit and no OOM.
The first full check stopped on a 172-line test callback exceeding the 150-line
rule. A named assertion helper now shares scope lookup across all four public
results; the strengthened six-case file passed again before the final gates.
After the review correction, the full static/docs gate passed again, including
the five consistency tests: 111.395 s wall / 110.979710 s CPU /
4,347,125,760 B peak, with clean owned exit and no OOM.
After centralizing semantic applicability, the final full static/docs gate also
passed, including five consistency tests: 117.636 s wall / 117.194118 s CPU /
4,263,153,664 B peak, clean owned exit and no OOM.
At the initial PR revision, 36 CI checks passed; macOS `hdiutil create` failed
while creating its DMG fixture before REA verification, also failing the aggregate
check. No Apple fixture/workflow code changed. That attempt provides no passing
Apple runtime evidence.
Live Electron/application execution, real native-provider behavior, other-platform
runtime coverage and hosted-agent costs are **NOT_ENOUGH_INFO**. The verification
does not claim them.
@@ -1,547 +0,0 @@
{
"base_revision": "e9a71efa31d6c3675b67938b8a79fa4ddf01604c",
"scope": "Semantic per-value string-byte normalization; real static shipped-application control, no hosted-model performance claim",
"runtime": {
"platform": "Linux x64",
"node": "24.18.0",
"npm": "11.16.0",
"tiktoken": "0.12.0"
},
"baseline_source_sha256": {
"src/domain/javascript/javascriptSemanticValues.ts": "df3fc30b5ba91901a85fe56f200355930da32d054bb026a747e3414af233b6b5",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "c12b29f0faf4b6511777cba7ed8ac007c6d6170d657f01e43e194feacb3e0601",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0203d207e0d6bb8aaa17746090f008876a11653afd660fce3dc44f3fda646a6c",
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1"
},
"final_source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "844d4639401a9f050ce42c80192572a6d0d74fe4f81b37e93a2c20aaca9fb9a6",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a3f802ca3446c2b5376cc58b19d818eff318e6775bd47752793c3f649994f8a2",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "2e78717d30e637b5fc5bab6d0eff063f7a62e0199d33697c1cdfaf8317baa698",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "8f9cf1a36f05b03dd816eaf9a6cdba0160d34ad7d17a961206e1bc32444d2ccc",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "e69408f2f5910a0107fd5d0cae0ff60ed706448857bc8e535c37cac1a0cd4d37",
"src/domain/javascript/javascriptSemanticIr.ts": "68dad360d0a9e6bd72108b2717cae756b84ca1efb48455248ded2ff699d2c113",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0993a397f2a2a48c2b6c35494e940b3fce5c2c9032cfe33c7cf0401128c79dde",
"src/domain/javascript/javascriptSemanticState.ts": "bfcface5dcda22594f47ed390efa15f9140fe41cb2d5995896e52dd0ee31b978",
"src/domain/javascript/javascriptSemanticValues.ts": "a4ba51cc60fff38ef11fd5af35a2e1ad753cdb2dea9d44895f91d89320c5ddeb",
"tests/acceptance/applications/javascriptEndpointPayload.test.ts": "b3cfcde1c5b5e0698f84afd51f4b53e42e56101267707c20c0d038320f5cfbd2",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "070106115b482269794799e448bd20b8e02df1f511f4ceb2b8fa75612c7f78e4",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"artifact": {
"path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f"
},
"selected_source_paths": ["package.json", "main.js"],
"where": {
"path": "main.js",
"kind": "javascript-module"
},
"identical_parent_evidence_id": "ev_73ef06e3b9b944f54c5257f7960279127d8518d2a73cd8618bd045dbe44e6b5a",
"cli_parity": {
"evidence_id": "ev_b3586dfdf7d4f01157f8f2978326aaa7103987110f2df4fa5bf3331ff2898749",
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"exact_cli_mcp_view_parity": true
},
"unchanged_query_shapes": {
"legacy": {
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
"references": {
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
}
},
"unchanged_token_costs": {
"tokenizer": "cl100k_base",
"tiktoken_version": "0.12.0",
"costs": {
"legacy": {
"selected_catalog": 7977,
"full_catalog": 195522,
"common_request": 53,
"common_summary": 2675,
"requests": 2836,
"structured_replies": 71761,
"selected_catalog_task_total": 85302,
"full_catalog_task_total": 272847
},
"references": {
"selected_catalog": 7977,
"full_catalog": 195522,
"common_request": 53,
"common_summary": 2675,
"requests": 102,
"structured_replies": 5741,
"selected_catalog_task_total": 16548,
"full_catalog_task_total": 204093
}
}
},
"cohorts": {
"baseline": {
"preparation_ms": 19089.119622,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 383.8260569999984
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 15.778886999996757
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 13.15173700000014
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 240.44363500000327
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 346.04796499999793
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 13.727306999997381
}
],
"resources": {
"elapsed_seconds": 33.394,
"cpu_seconds": 33.016802,
"peak_cgroup_bytes": 1979863040,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"final": {
"preparation_ms": 19061.292544,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 344.6316079999997
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 14.438351000000694
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 11.38763100000142
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 298.41579600000114
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 211.45682099999976
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.275370999999723
}
],
"resources": {
"elapsed_seconds": 32.949,
"cpu_seconds": 32.559187,
"peak_cgroup_bytes": 1880096768,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
}
},
"conditions": "First query is after shared artifact analysis; subsequent queries share one MCP session. Legacy item reads are serial; a concurrent legacy baseline is unmeasured. Filesystem/build warmness uncontrolled. Token costs count one post-change catalog, preparation request/summary and each request/structured reply once; client rendering, replay, reasoning, final answer and model latency are unmeasured. Preparation and parity/tokenization overhead are outside per-query elapsed times; external family-resource receipts include the complete verifier and children. Before/after used separate fresh MCP sessions. CPU/memory includes analysis, query arms, export, CLI parity and tokenization; cgroup memory includes file cache, not summed RSS. Counters are at payload exit while observer lives, not final scope counters. One before/after control establishes no speedup or equivalence bound. Catalog BPE projection excludes annotations/output schemas.",
"resource_policy": {
"memory_max_bytes": 8589934592,
"early_stop_bytes": 6442450944,
"cpu_quota_percent": 100,
"swap": false
},
"observer_sources_sha256": {
"resource-run.py": "7728862171415e7dee2c11af2ead2b9a54265fccaf82a7113956d36b29065e33",
"resource_payload.py": "6f089e645459eba80c5d73e39c21606c78173a44daf49fabc3b88c4631ce51c6"
},
"regressions": {
"original_main": "Quoted/template oversized literals failed through public CLI/MCP; addition/small controls passed. Corrected domain boundary reproduced 2 failures/16 passes.",
"review_head": "8 failed/25 passed before fixes for projection tags, encountered-but-discarded limits and bare quoted/template defaults; includes exact null default control.",
"interim_review_fix": "65/66 focused passed; old doubling-chain expectation omitted a genuine expression-depth limit now observed. Adjusted that assertion to preserve the byte-limit requirement while allowing other observed classes.",
"final_focused": "66 passed across five files on the restored 38d263a runtime. Final corrected public selection passed 7/7 in two files: the six literal/default/member cases plus the augmented original large-endpoint acceptance. 67 distinct focused cases across invocations; not a single 67-case invocation.",
"final_semantic_suite": "932 passed in 36 files on 38d263a. Final production src tree (including these domain tests) verified byte-identical to that revision; no repeated semantic run claimed.",
"arithmetic": "Computed 174762*6+2=1048574 and 174763*6+2=1048580. Corrected the erroneous published 1048578 total.",
"final_grouped_domain": "27 passed after grouping observations/defaults/allocation within function-size rules; no scenarios removed.",
"endpoint_contract_correction": "Restored exact endpoint lookup identities rather than shrinking the existing large-endpoint fixture. Added the identical oversized string as an exported primitive in that acceptance workflow: literal normalization must report its bound while full endpoint lookup/feature trace/semantic trace/version comparison continue to work. Production src tree restored byte-identical to 38d263a; 932 semantic tests from that state remain applicable. Final corrected public tests/gates recorded after execution.",
"final_corrected_gates": "Full static gate and docs gate (5 consistency tests) passed after endpoint identity restoration and strengthened unchanged-large-input acceptance. Production src remains byte-identical to 38d263a; final public 7/7 and fresh real control passed."
},
"prior_attempts": [
{
"stage": "Initial literal normalization, incomplete for member/default paths identified in review",
"revision": "40b9d3c5b9cbeae7661d8e351e1b5ef6e5338931",
"source_sha256": {
"src/domain/javascript/javascriptSemanticValues.ts": "2debbb4f32d7d189819f2413f0f7d05b9bdcf497451a1c578475e3f2ec6c8554",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0203d207e0d6bb8aaa17746090f008876a11653afd660fce3dc44f3fda646a6c",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "747d893d6ddd91cf24d3b62b7eb50e0af9b1715ab355bbfc2e10ed0dc171ec17",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "46ceb5a169de3eec3a335b05ef4aa7b1b341a5ac06ad325bd191e862c5b6be70",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50",
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1"
},
"real_control_cohort": {
"preparation_ms": 18876.764036,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 346.7870870000006
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 18.34765800000241
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 11.56626999999935
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 227.17703500000061
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 251.5148330000011
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.888227999999799
}
],
"resources": {
"elapsed_seconds": 31.347,
"cpu_seconds": 31.05517,
"peak_cgroup_bytes": 1589915648,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"validation": {
"baseline_public": "Quoted and cooked-template cases retained 200000-unit normalized literals and failed; addition and small exact controls passed. Aggregate-capacity test passed.",
"baseline_domain_corrected": "2 over-bound failures, 16 passed. Initial combined-run domain assertions had incorrectly required an empty optional resourceLimits array; corrected before using the domain reproduction as evidence.",
"fixed_focused": "55 passed across five files.",
"semantic_suite": "923 passed across 36 files.",
"capacity_after_fixture_cleanup": "1 passed; all 16 bounded values, aggregate size >= original, typed identities and retained tracing preserved.",
"interim_static_failure": "Typecheck rejected resume() on the new test SDK stderr Stream declaration. Switched to its declared data event, matching existing typed tests; analyzer source unchanged.",
"public_boundary_after_stream_type_fix": "4 passed across actual CLI and stdio MCP; analyzer source unchanged."
}
},
{
"stage": "Projection/default correction; final review subsequently found unnormalized request endpoints",
"revision": "38d263a369e7853ccfae1c8e0a3bd387f920c7cf",
"source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "844d4639401a9f050ce42c80192572a6d0d74fe4f81b37e93a2c20aaca9fb9a6",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a3f802ca3446c2b5376cc58b19d818eff318e6775bd47752793c3f649994f8a2",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "2e78717d30e637b5fc5bab6d0eff063f7a62e0199d33697c1cdfaf8317baa698",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "8f9cf1a36f05b03dd816eaf9a6cdba0160d34ad7d17a961206e1bc32444d2ccc",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "e69408f2f5910a0107fd5d0cae0ff60ed706448857bc8e535c37cac1a0cd4d37",
"src/domain/javascript/javascriptSemanticIr.ts": "68dad360d0a9e6bd72108b2717cae756b84ca1efb48455248ded2ff699d2c113",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0993a397f2a2a48c2b6c35494e940b3fce5c2c9032cfe33c7cf0401128c79dde",
"src/domain/javascript/javascriptSemanticState.ts": "bfcface5dcda22594f47ed390efa15f9140fe41cb2d5995896e52dd0ee31b978",
"src/domain/javascript/javascriptSemanticValues.ts": "a4ba51cc60fff38ef11fd5af35a2e1ad753cdb2dea9d44895f91d89320c5ddeb",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "070106115b482269794799e448bd20b8e02df1f511f4ceb2b8fa75612c7f78e4",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"real_control_cohort": {
"preparation_ms": 19599.643274,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 396.9265450000021
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 15.658120000000054
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 12.24905100000251
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 237.03392900000108
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 357.78818299999693
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 14.37622100000226
}
],
"resources": {
"elapsed_seconds": 33.969,
"cpu_seconds": 33.499223,
"peak_cgroup_bytes": 1893015552,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"validation": {
"original_main": "Quoted/template oversized literals failed through public CLI/MCP; addition/small controls passed. Corrected domain boundary reproduced 2 failures/16 passes.",
"review_head": "8 failed/25 passed before fixes for projection tags, encountered-but-discarded limits and bare quoted/template defaults; includes exact null default control.",
"interim_review_fix": "65/66 focused passed; old doubling-chain expectation omitted a genuine expression-depth limit now observed. Adjusted that assertion to preserve the byte-limit requirement while allowing other observed classes.",
"final_focused": "66 passed in five files.",
"final_semantic_suite": "932 passed in 36 files.",
"arithmetic": "Computed 174762*6+2=1048574 and 174763*6+2=1048580. Corrected the erroneous published 1048578 total.",
"final_grouped_domain": "27 passed after grouping observations/defaults/allocation within function-size rules; no scenarios removed.",
"endpoint_review_baseline": "Production at 38d263a: 7 failed/14 passed across 13 new domain and 8 CLI/MCP cases before endpoint normalization. The later public linked-response case was not in this baseline run. Regression tests were uncommitted; final formatted test hashes are preserved, not claimed as the baseline test bytes.",
"endpoint_selection_attempt": "First six-file selection exited before execution because a signed-zero test filename was mistyped; corrected the exact path. No test result claimed for that attempt."
}
},
{
"stage": "Rejected endpoint normalization: violated existing full-endpoint lookup/trace contract; reverted after CI acceptance failure",
"revision": "580887209c4fceda432eac4083fac82a0dd78f15",
"source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "a648f97e631985edcb5259ff4f1fdf0332dca18b8c10b57e050e36a785c1c524",
"src/domain/javascript/javascriptSemanticAnalysis.requestBudget.test.ts": "8e84ffa70d834f5589d04118f0aaf33b0383b15ea97545ff69c66a74473aca73",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a3f802ca3446c2b5376cc58b19d818eff318e6775bd47752793c3f649994f8a2",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "2e78717d30e637b5fc5bab6d0eff063f7a62e0199d33697c1cdfaf8317baa698",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "8f9cf1a36f05b03dd816eaf9a6cdba0160d34ad7d17a961206e1bc32444d2ccc",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "eed961e0b1d8a320e76bbba933408e950b4a4a446110401e8cf470f1e74d8315",
"src/domain/javascript/javascriptSemanticIr.ts": "3b2ede73dbcfc74c4014e4d1affcbf00ecccc29a422ed495a7a3f2f517056c9b",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0993a397f2a2a48c2b6c35494e940b3fce5c2c9032cfe33c7cf0401128c79dde",
"src/domain/javascript/javascriptSemanticState.ts": "bfcface5dcda22594f47ed390efa15f9140fe41cb2d5995896e52dd0ee31b978",
"src/domain/javascript/javascriptSemanticValues.ts": "a4ba51cc60fff38ef11fd5af35a2e1ad753cdb2dea9d44895f91d89320c5ddeb",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "61319ea954de7f256841f43ca9355a1706ce8e93d30f2a8ba851d5e717bd19e7",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"real_control_cohort": {
"preparation_ms": 19939.160272,
"query_samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 348.3974309999976
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 13.97052899999835
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 12.060379000002285
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 212.01217800000086
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 223.73954800000138
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.450989999997546
}
],
"resources": {
"elapsed_seconds": 33.062,
"cpu_seconds": 32.71624,
"peak_cgroup_bytes": 1897406464,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
},
"validation": {
"original_main": "Quoted/template oversized literals failed through public CLI/MCP; addition/small controls passed. Corrected domain boundary reproduced 2 failures/16 passes.",
"review_head": "8 failed/25 passed before fixes for projection tags, encountered-but-discarded limits and bare quoted/template defaults; includes exact null default control.",
"interim_review_fix": "65/66 focused passed; old doubling-chain expectation omitted a genuine expression-depth limit now observed. Adjusted that assertion to preserve the byte-limit requirement while allowing other observed classes.",
"final_focused": "81 passed in six files; the same CLI/MCP file then passed 9/9 after adding a linked-response case. That follow-up replaces its earlier 8-case coverage; 82 distinct cases across the final six-file selection, not one 82-case invocation.",
"final_semantic_suite": "945 passed in 37 files, overlapping focused domain coverage.",
"arithmetic": "Computed 174762*6+2=1048574 and 174763*6+2=1048580. Corrected the erroneous published 1048578 total.",
"final_grouped_domain": "27 passed after grouping observations/defaults/allocation within function-size rules; no scenarios removed.",
"endpoint_review_baseline": "Production at 38d263a: 7 failed/14 passed across 13 new domain and 8 CLI/MCP cases before endpoint normalization. The later public linked-response case was not in this baseline run. Regression tests were uncommitted; final formatted test hashes are preserved, not claimed as the baseline test bytes.",
"endpoint_selection_attempt": "First six-file selection exited before execution because a signed-zero test filename was mistyped; corrected the exact path. No test result claimed for that attempt.",
"endpoint_typecheck_attempt": "Full static gate found the inferred optional dataField in the public test computed key. Replaced the in-property check with an explicit bare === true discriminant; scenario inputs and runtime implementation unchanged. Corrected static/public checks recorded separately.",
"final_gates": "Full static gate passed after the explicit test discriminant; final public file passed 9/9 again, then docs gate passed including 5 consistency tests. Runtime implementation unchanged after the 945-test semantic suite and real control."
},
"ci_failure": {
"run_id": 38758262643,
"job_id": 116503076738,
"test": "tests/acceptance/applications/javascriptEndpointPayload.test.ts",
"result": "1 failed/1573 passed/19 skipped; request endpoint no longer matched its complete source value. Other local tests and real Obsidian control passed but did not establish this compatibility contract.",
"aggregate_result": "Merged source/built report: 1 failed/15625 passed/138 skipped. Portable tests and CI required failed as aggregates of the same endpoint assertion; no separate additional failed test."
}
}
],
"review_baseline_revision": "40b9d3c5b9cbeae7661d8e351e1b5ef6e5338931",
"review_baseline_source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/application/javascript/JavaScriptSemanticGraphDataProjection.ts": "ee541e634264363822bf4ce05c42c72272029c7e8da72bbeed643ad1341d0756",
"src/domain/javascript/javascriptSemanticAnalysis.ts": "a1cf791a61ee4742c351c006b69b64eba6bef3068f7a8450ed8f9366bfe4ff88",
"src/domain/javascript/javascriptSemanticAnalysis.unions.test.ts": "747d893d6ddd91cf24d3b62b7eb50e0af9b1715ab355bbfc2e10ed0dc171ec17",
"src/domain/javascript/javascriptSemanticDataEffectHelpers.ts": "5ee7af18dd847726d8fe1828ea25f386ff9ef317d996a3f7969253bf73973cfa",
"src/domain/javascript/javascriptSemanticDataEffects.ts": "4c62462016246bc7198294a74a60e87598fc009c53f1601abc2d4662d41b5bd1",
"src/domain/javascript/javascriptSemanticIr.ts": "14955cb4b7912ed6a4f26f804a2480f841dcb6084dbddd15856375fa62cdcfa1",
"src/domain/javascript/javascriptSemanticPrimitives.ts": "899eeb6ccdf666f504f8b722327599997c924c8e4fb13a1cbd13edaf08c96640",
"src/domain/javascript/javascriptSemanticResourceLimits.ts": "0203d207e0d6bb8aaa17746090f008876a11653afd660fce3dc44f3fda646a6c",
"src/domain/javascript/javascriptSemanticState.ts": "d94c5a88bb105284fab192406a8c88860d59f80c93394ed3ee7264969df254a6",
"src/domain/javascript/javascriptSemanticValues.ts": "2debbb4f32d7d189819f2413f0f7d05b9bdcf497451a1c578475e3f2ec6c8554",
"tests/boundary/mcp/javascriptLiteralBudget.test.ts": "46ceb5a169de3eec3a335b05ef4aa7b1b341a5ac06ad325bd191e862c5b6be70",
"tests/boundary/mcp/javascriptLiteralPayload.test.ts": "03618272c605f8877ed171ab39e037e4d80eac72b7478421640076f739adfb50"
},
"implementation": "Per-analysis state records encountered resource classifications at expression/default normalization; final coverage consumes those facts. Slot projection preserves tagged unknown values. Removed retained-value recursive coverage scan and duplicate raw default reader. Defaults retain optional valueResourceLimit and graph value_resource_limit plus an anchored resource-limit unknown; known null/false/0/empty remain exact. Endpoint identities intentionally remain exact lookup values, outside the normalized primitive lattice; rejected endpoint normalization and its CI failure are preserved separately.",
"restored_production_revision": "38d263a369e7853ccfae1c8e0a3bd387f920c7cf",
"final_corrected_resource_receipts": {
"public_7": {
"elapsed_seconds": 43.621,
"cpu_seconds": 43.097453,
"peak_cgroup_bytes": 1520324608,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
},
"static_and_docs": {
"elapsed_seconds": 111.403,
"cpu_seconds": 110.830839,
"peak_cgroup_bytes": 4268244992,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
}
}
}
}
@@ -1,86 +0,0 @@
# Semantic literal byte-budget verification
Tracks #3933 under #4027. All normalized semantic literals and configuration
defaults use the primitive owner. Its singleton path validates strings/nonfinite
numbers/negative zero without a Set, sorting or canonical-key serialization.
Each source analysis owns a bounded set of encountered limit classifications;
coverage consumes these observations after evaluation. This replaces a recursive
scan that only saw retained binding/return values. No persistent cache is added.
## Correctness and coverage
Public CLI and actual stdio MCP cases cover quoted strings, cooked templates,
equivalent concatenation, member receivers, a configuration-only expression
statement and a small exact control. Direct/default oversized values become
`resource-limit` unknowns with partial application coverage and
`javascript_semantic_primitive_bytes = 1048576`. Defaults carry
`value_resource_limit` inline and an unknown anchored to their config node.
Known null/false/zero/empty defaults remain exact. Original source/digests and
CLI/MCP identity parity are preserved.
Member and destructuring projections use the shared slot owner's tagged value.
An encountered limit stays visible even when a subsequent projection selects a
healthy sibling: the selected value remains exactly `7`, while file coverage is
partial. This avoids inferring completeness from the final retained values alone.
The duplicate raw default reader and superseded retained-value scan are removed.
Boundary controls admit 174,762 UTF-16 units (1,048,574 estimated JSON bytes)
and reject 174,763 (1,048,580 bytes). Actual arithmetic corrected the earlier
published 1,048,578 total; the unit-count boundary assertions were already correct.
The estimate counts six bytes per unit plus two quotes. It does not establish
actual ASCII JSON size above 1 MiB or an observed OOM.
The initial main baseline reproduced both public literal failures, with addition
and small controls passing. A mistaken optional-limit-array control was corrected
before the separate domain reproduction (2 failed/16 passed). Review head
`40b9d3c5` then reproduced the additional projection/default gaps (8 failed,
25 passed). The first review correction passed 65/66 focused tests; a doubling
chain now truthfully retains an additional encountered expression-depth limit,
so its byte-growth assertion was updated to allow other recorded limits.
Prior validation on the restored runtime: **66 focused tests** in five files and **932 semantic tests**
in 36 files, with overlapping domain coverage. The large-wire fixture retains
16 individually admitted values and at least the former aggregate UTF-16 payload,
plus complete CLI/MCP delivery, typed identities, empty strings and retained
tracing. Inputs regenerate from the tests.
## Endpoint identity compatibility
An attempted endpoint normalization at `5808872` passed its new local cases and real Obsidian control, but failed the existing CI endpoint-payload acceptance workflow (1 failed/1,573 passed/19 skipped in that shard). It violated the established full endpoint lookup/trace contract. This extension was reverted; endpoint values are lookup identities, outside the normalized primitive lattice. The original large endpoint is preserved. Corrected public validation passed **7/7** across two files; this adds one acceptance workflow to the previous 66-case focused selection (**67 distinct cases**, not one 67-case invocation). Its acceptance workflow now also normalizes that identical string as an exported primitive, requiring the typed value bound while endpoint lookup, both trace workflows and version comparison remain usable. Prior measurements and failed attempts remain in the JSON record. Production source is restored byte-identical to `38d263a`; the 932-test semantic run belongs to that state. Final full static and documentation gates (five consistency tests) passed after the compatibility correction.
## Real application control and costs
Official Obsidian 1.14.4 ASAR, selected `package.json` and `main.js`. Follow the
acquisition/digest and existing verifier command in
[module-reference verification](javascript-module-references-2026-10-15.md).
Use three alternating-order query pairs with CLI parity on each revision.
Application code is not executed.
The final correction retains the exact baseline full parent/view Evidence IDs,
23-row fact digest, source coverage and unknowns. Static `cl100k_base` task costs
remain 16,548 with selected name/description/inputSchema contracts, or 204,093
with the full catalog projection, for the reference workflow. Serial legacy
reads also keep their original costs. The query projection improvement belongs
to #4097; these are controls for semantic normalization and coverage.
| Complete real verifier | Before | Final correction |
| ---------------------- | ------------: | ---------------: |
| Elapsed seconds | 33.394 | 32.949 |
| CPU seconds | 33.016802 | 32.559187 |
| Peak cgroup bytes | 1,979,863,040 | 1,880,096,768 |
Includes initial analysis, both query arms, export, CLI parity and tokenizer.
One CPU, 8 GiB hard cap, 6 GiB early stop, no swap; cleanup verified and no OOM.
Memory includes charged file cache, not summed RSS. Counters are at payload exit
while the observer remains alive. Fresh MCP sessions do not establish OS-cold
cache conditions. Shared-host scheduling and filesystem/build warmness were
uncontrolled. This one before/after control establishes neither a speedup nor
an equivalence bound. Static BPE excludes annotations/output schemas, client
rendering/replay, reasoning and final answers; no hosted agent was run. Linux
static ASAR analysis was exercised, not live Electron or other platforms.
[Compact records, samples, source hashes and prior attempts](javascript-literal-byte-budget-2026-10-15.json)
preserve the initial incomplete correction separately from the final measurements.
No downloadable application or complete bundle is archived. Reproduce boundary
outcomes through the listed public tests and reuse the existing real verifier
and preserved 8 GiB observer linked above. Observer hashes and result-affecting
touched source hashes are pinned in the records.
@@ -1,691 +0,0 @@
{
"base_revision": "c9f9b83c6f5878f1d5b102ab206a2ce214749258",
"source_sha256": {
"scripts/verify/javascript/module-references.mjs": "9d5f0b8b35ed2ebb4591f9dc3baea0fc116a0c3fd7d8e5ce30ec476c84671ed1",
"src/contracts/analysisViewToolContracts.ts": "f7c64b3472e7ec1090f9d47ede864b8c9630d506d896454a800747f2c90a8960",
"src/domain/analysisView/analysisView.ts": "854b03988791c19642b01d17813b379b51d8edc10d2190a3c236c9a1b2feb365",
"src/domain/analysisView/javascriptApplicationView.ts": "f2c41ad15fdc9f891412c9161a832b60baf3dae55e3c4d60159098d07f7d1ef1",
"src/domain/analysisView/javascriptModuleReferences.ts": "34c7a7ff466e9c49d43f1b11b72f878041de61a1bc09086ec190a5833b454806",
"scripts/fixtures/tokenizer-family-wrapper.py": "2e9b7092ee539ae1c6a477492f05bb6007d4612dadfff9b90051d621de43049b"
},
"verification": {
"scope": "Completed module-fact reads from identical retained Evidence; no hosted-model performance claim",
"artifact": {
"path": "/tmp/rea-codemode-experiment-20261015/artifacts/obsidian-1.14.4.asar",
"sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f"
},
"parent_evidence_id": "ev_73ef06e3b9b944f54c5257f7960279127d8518d2a73cd8618bd045dbe44e6b5a",
"source_selection": {
"requested_paths": ["package.json", "main.js"],
"unselected_files": 36
},
"source_coverage": {
"application": {
"status": "partial",
"truncated": false,
"omitted_count": null,
"limits": []
},
"semantic": {
"status": "partial",
"truncated": false,
"omitted_nodes": null,
"omitted_relations": null,
"limits": [],
"families": [
{
"family": "boundary",
"status": "partial",
"retained_relations": 14,
"omitted_relations": null,
"unknown_count": 31
},
{
"family": "call-flow",
"status": "partial",
"retained_relations": 845,
"omitted_relations": null,
"unknown_count": 1278
},
{
"family": "child-process",
"status": "partial",
"retained_relations": 0,
"omitted_relations": null,
"unknown_count": 31
},
{
"family": "closure",
"status": "partial",
"retained_relations": 1187,
"omitted_relations": null,
"unknown_count": 31
},
{
"family": "configuration",
"status": "partial",
"retained_relations": 0,
"omitted_relations": null,
"unknown_count": 31
},
{
"family": "data-flow",
"status": "partial",
"retained_relations": 9388,
"omitted_relations": null,
"unknown_count": 1135
},
{
"family": "event",
"status": "partial",
"retained_relations": 127,
"omitted_relations": null,
"unknown_count": 89
},
{
"family": "object-flow",
"status": "partial",
"retained_relations": 1462,
"omitted_relations": null,
"unknown_count": 2292
},
{
"family": "promise-ownership",
"status": "partial",
"retained_relations": 80,
"omitted_relations": null,
"unknown_count": 98
},
{
"family": "request",
"status": "partial",
"retained_relations": 0,
"omitted_relations": null,
"unknown_count": 31
},
{
"family": "resource-lifecycle",
"status": "partial",
"retained_relations": 4,
"omitted_relations": null,
"unknown_count": 31
},
{
"family": "timer",
"status": "partial",
"retained_relations": 8,
"omitted_relations": null,
"unknown_count": 31
}
]
},
"application_unknowns": 36,
"semantic_unknowns": 5109
},
"where": {
"path": "main.js",
"kind": "javascript-module"
},
"preparation_ms": 20978.249922,
"samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 363.02500200000213,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 14.895039999999426,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 12.951290000000881,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 232.7533779999976,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 235.16672899999685,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 10.566158999998152,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
}
],
"token_costs": {
"tokenizer": "cl100k_base",
"tiktoken_version": "0.12.0",
"costs": {
"legacy": {
"selected_catalog": 7977,
"full_catalog": 195446,
"common_request": 53,
"common_summary": 2675,
"requests": 2836,
"structured_replies": 71761,
"selected_catalog_task_total": 85302,
"full_catalog_task_total": 272771
},
"references": {
"selected_catalog": 7977,
"full_catalog": 195446,
"common_request": 53,
"common_summary": 2675,
"requests": 102,
"structured_replies": 5741,
"selected_catalog_task_total": 16548,
"full_catalog_task_total": 204017
}
}
},
"cli_parity": {
"evidence_id": "ev_b3586dfdf7d4f01157f8f2978326aaa7103987110f2df4fa5bf3331ff2898749",
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"exact_cli_mcp_view_parity": true
},
"conditions": "First query is after shared artifact analysis; subsequent queries share one MCP session. Legacy item reads are serial; a concurrent legacy baseline is unmeasured. Filesystem/build warmness uncontrolled. Token costs count one post-change catalog, preparation request/summary and each request/structured reply once; client rendering, replay, reasoning, final answer and model latency are unmeasured. Preparation and parity/tokenization overhead are outside per-query elapsed times; external family-resource receipts include the complete verifier and children."
},
"resources": {
"scope": "Complete verifier, MCP/analysis descendants, temporary export, CLI parity and tokenizer; cgroup memory includes charged file cache and is not summed RSS",
"elapsed_seconds": 35.797,
"cpu_seconds": 35.197074,
"peak_cgroup_bytes": 1938313216,
"memory_max_bytes": 8589934592,
"memory_early_stop_bytes": 6442450944,
"cpu_quota_percent": 100,
"payload_exit_code": 0,
"payload_cleanup_verified": true,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"observer_sources_sha256": {
"resource-run.py": "7728862171415e7dee2c11af2ead2b9a54265fccaf82a7113956d36b29065e33",
"resource_payload.py": "6f089e645459eba80c5d73e39c21606c78173a44daf49fabc3b88c4631ce51c6"
}
},
"prior_attempts": [
{
"scope": "Initial verifier incorrectly requested full MCP delivery; completed analysis exceeded existing response admission",
"elapsed_seconds": 21.971,
"cpu_seconds": 21.78546,
"peak_cgroup_bytes": 1080860672,
"outcome": "resource_constraint transport; verifier corrected to summary-first; no performance credit"
},
{
"scope": "Interim successful parity run before eliminating verifier assertion serialization and counting the bootstrap request",
"elapsed_seconds": 31.335,
"cpu_seconds": 31.07044,
"peak_cgroup_bytes": 1652293632,
"outcome": "parity passed; superseded measurement, not pooled with final samples"
},
{
"scope": "Successful three-pair probe before adding the shared analyze_javascript_application schema to selected-catalog accounting",
"preparation_ms": 18317.110148,
"samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 313.59318599999824,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 13.595478999999614,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 11.230690000000322,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 206.19418099999893,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 242.2037199999977,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 13.734700000000885,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
}
],
"elapsed_seconds": 31.037,
"cpu_seconds": 30.786998,
"peak_cgroup_bytes": 1886543872,
"outcome": "Facts and CLI/MCP parity passed. Selected-catalog totals omitted the common analysis-tool schema; superseded, not pooled. Shared-host timing differed substantially from the final run."
},
{
"scope": "Completed three-pair probe with full token accounting before the review-added tokenizer deadline",
"preparation_ms": 66696.731143,
"samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 1114.080965000001,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 46.572758999987855,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 37.09324900001229,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 675.7124039999908,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 777.2660329999926,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 36.31178899999941,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
}
],
"elapsed_seconds": 108.372,
"cpu_seconds": 81.551676,
"peak_cgroup_bytes": 1884962816,
"outcome": "All facts/costs and CLI/MCP parity passed. Superseded execution source after adding a tokenizer deadline; retained separately, not pooled."
},
{
"scope": "Successful data comparison with direct-child tokenizer timeout before adopting shared owned-process supervision",
"preparation_ms": 19051.368892000002,
"samples": [
{
"round": 0,
"mode": "legacy",
"elapsed_ms": 330.7313439999998,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 0,
"mode": "references",
"elapsed_ms": 13.62184999999954,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
},
{
"round": 1,
"mode": "references",
"elapsed_ms": 12.475999999998749,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
},
{
"round": 1,
"mode": "legacy",
"elapsed_ms": 187.18667300000016,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 2,
"mode": "legacy",
"elapsed_ms": 293.4882039999975,
"calls": 24,
"structured_reply_bytes": 325201,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 55,
"total": 55,
"next_offset": null,
"exhausted": true
}
},
{
"round": 2,
"mode": "references",
"elapsed_ms": 8.146389999998064,
"calls": 1,
"structured_reply_bytes": 22158,
"facts_sha256": "d3be5e91d2ec3fd9a95851cfbbdef9c168bb4c7153f78810b8b235e005401299",
"matching_modules": 23,
"coverage": {
"status": "page",
"examined": 23,
"total": 23,
"next_offset": null,
"exhausted": true
}
}
],
"elapsed_seconds": 30.742,
"cpu_seconds": 30.425218,
"peak_cgroup_bytes": 1592938496,
"outcome": "Fact and cost parity passed. That harness only killed its immediate tokenizer child; superseded by process-family supervision and detached-helper verification."
}
],
"prior_tokenizer_timeout_verifications": [
{
"artifact_sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f",
"scenario": "Real Obsidian references plus CLI parity, then a deliberately stalled tokenizer",
"injected_tokenizer_source": "#!/bin/sh\nexec /bin/sleep 30\n",
"timeout_ms": 20,
"observed_error_code": "ETIMEDOUT",
"expected_payload_exit_code": 1,
"payload_cleanup_verified": true,
"remaining_owned_temporary_files": 0,
"elapsed_seconds": 31.258,
"cpu_seconds": 31.00025,
"peak_cgroup_bytes": 1644077056,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"scope": "Robustness check; no efficiency credit",
"limitation": "Immediate child only; did not test detached descendants and is superseded by the owned-family check."
}
],
"tokenizer_timeout_verification": {
"artifact_sha256": "146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f",
"scenario": "Real Obsidian references and CLI parity, then an owned tokenizer wrapper with a detached SIGTERM-ignoring helper",
"fixture": "scripts/fixtures/tokenizer-family-wrapper.py",
"fixture_sha256": "2e9b7092ee539ae1c6a477492f05bb6007d4612dadfff9b90051d621de43049b",
"timeout_ms": 2000,
"observed_failure_type": "OwnedCommandFailure",
"observed_reason": "timeout",
"cleanup_failure": null,
"expected_payload_exit_code": 1,
"payload_cleanup_verified": true,
"remaining_owned_temporary_files": 0,
"helper_observation": {
"pid": 2871108,
"group": 2871108,
"parent": 2871107,
"run_id": "rea-module-token-f139b74a-dd83-498b-b663-e765104159ec",
"detached": true,
"ignores_sigterm": true
},
"elapsed_seconds": 55.08,
"cpu_seconds": 45.245603,
"peak_cgroup_bytes": 1563475968,
"events_at_payload_exit": {
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0,
"oom_group_kill": 0
},
"pids.events": {
"max": 0
}
},
"scope": "Robustness check; no efficiency credit. PIDs are observations; signaling authority comes from the shared process owner."
}
}
@@ -1,123 +0,0 @@
# Retained JavaScript module-reference verification
Tracks #4092 under #4027 and the decision in #4091. This verifies an existing
tool's data projection, without a code-mode host or SDK. [Machine-readable
results](javascript-module-references-2026-10-15.json) pin the base revision,
executed source hashes, samples, tokenizer and resource observations.
## Real application and question
Official Obsidian 1.14.4 ASAR, 27,477,966 bytes, SHA-256
`146ef8470d8cbb4a6db209bdc19bf2bad13350444186c0fd93c31087b142b92f`.
The [official gzip](https://github.com/obsidianmd/obsidian-releases/releases/download/v1.14.4/obsidian-1.14.4.asar.gz)
has SHA-256 `d1ed428c363968774f0f3906e67d8a53a058ece0b6cdd924b07865977b2ada21`.
Select `package.json` and `main.js`; the other 36 inventoried files remain
explicitly unselected. No target code is executed.
Question: retrieve every retained `javascript-module` identity at exact path
`main.js`, its retained source-range observation count, and distinct support
states. Compare the existing module page plus full item reads with one filtered
typed reference view. All six completed query samples use one identical retained
parent. All 23 selected rows, identities, counts and support states match. Public
CLI inline input and MCP retained input produce the exact same view and Evidence
ID. This establishes producer-fact consistency, not independent extractor
correctness or proof of runtime loading.
## Completed read costs
| Measurement | Existing page + items | Typed reference view |
| ------------------------------------------------------------- | --------------------: | -------------------: |
| Calls per query task | 24 | 1 |
| Structured reply bytes | 325,201 | 22,158 |
| First query in its arm, after preparation | 363.03 ms | 14.90 ms |
| Subsequent same-session query times | 232.75 / 235.17 ms | 12.95 / 10.57 ms |
| Request + structured-reply BPE tokens | 74,597 | 5,843 |
| With selected tool catalog + shared bootstrap request/summary | 85,302 | 16,548 |
| With complete tool catalog + shared bootstrap request/summary | 272,771 | 204,017 |
Order alternates: existing/reference, reference/existing, existing/reference.
Times include verifier-side fact normalization and equality checks; byte/token
accounting follows the timed reads. The initial artifact analysis and summary
took 20.978 seconds and is shared preparation, not a query-speed improvement.
The retained source remains partial; unknown counts and source selection are
preserved independently of exhausted pages. No query cache was introduced.
Tokens are `cl100k_base` counts with tiktoken 0.12.0 over serialized values, not
hosted-model usage. Catalog projections contain name/description/inputSchema;
both workflows use the same post-change catalog. Each request/reply is counted
once. Output schemas, host-specific rendering/replay, reasoning, final answers,
agent success rates and model latency are unmeasured. This is a completed
data-retrieval comparison; it does not establish a whole-agent speedup or a
before/after catalog-size improvement. Legacy item reads are serial; a concurrent
legacy baseline was not measured. Selected discovery includes both the
analysis and view tool schemas. Builds/filesystem are warm or uncontrolled,
so the first query is not an OS-cold analysis measurement. Shared-host scheduling
was uncontrolled: an earlier successful probe had substantially higher wall/CPU
times. Its samples are retained separately; neither run establishes a portable
latency guarantee.
The complete final verifier scope used 35.797 seconds elapsed, 35.197 seconds
CPU and 1,938,313,216 bytes peak cgroup memory. This includes shared analysis,
both query arms, temporary export, CLI parity and tokenization; CPU/memory were
not isolated by arm. Cgroup memory includes charged file cache and is not summed
process RSS. The whole scope had one CPU, an 8 GiB hard memory cap, 6 GiB early
stop and no swap. Payload cleanup was verified; no OOM/capacity event occurred.
An earlier full-response transport failure and interim probes are recorded
separately in the JSON and earn no performance credit.
A review follow-up exercised a stalled tokenizer after real Obsidian reference
reads and CLI parity. The committed POSIX fault fixture forks a helper into a
separate process group, ignores SIGTERM and stalls. With a caller-selected
2,000 ms deadline, the shared owned-command supervisor reported
`OwnedCommandFailure` with reason `timeout` and no cleanup failure. The verifier
failed as expected, its complete owned scope was cleaned, and its selected
temporary root was empty. The separate robustness run used 55.080 s wall /
45.246 s CPU / 1,563,475,968 B peak cgroup memory, with cleanup verified and no
OOM. It earns no efficiency credit. An earlier 20 ms immediate-child timeout
probe did not establish descendant cleanup; it remains a separate prior attempt.
The detached-helper scenario was executed on Linux, not Windows or macOS.
A deterministic CI regression also asserts two retained
observations at the same source range, excludes an unavailable location, and
checks agreement with the full item view; the count deliberately measures
observations rather than unique source sites.
## Reproduce
Use the pinned Node/npm toolchain and `npm ci`, then `npm run build:cached`.
Download/decompress the versioned application outside the checkout and check its
digest. Run with an operator-selected absolute artifact path:
```bash
node scripts/verify/javascript/module-references.mjs \
--artifact /absolute/path/obsidian-1.14.4.asar \
--source-path package.json --source-path main.js \
--path main.js --kind javascript-module --rounds 3 --cli-parity \
--tokenizer-python /absolute/path/to/tiktoken-venv/bin/python
```
The optional Python interpreter needs tiktoken 0.12.0; omit that flag to skip
token accounting. It has a 60-second wall-time deadline, configurable with the
positive integer `--tokenizer-timeout-ms`. The tokenizer uses the existing
owned-command supervisor, with native identity checks and inherited run-token
sweeping for descendants. Inputs and temporary Evidence use the existing private
runtime root. Cleanup attempts preserve the primary error; incomplete command
cleanup retains its owner and private files for recovery. `--mode legacy` or
`--mode references` runs one arm. Fresh runs
reanalyze the artifact and can produce different parent identities/resource
coverage; the verifier requires equal facts within its paired run. The measured
run additionally used the preserved [8 GiB cgroup observer](https://github.com/morluto/rea/blob/b3bdabab694af1f9801eff7caf75001b92314a58/experiments/analysis-efficiency/resource-run.py)
and its adjacent `resource_payload.py`; observer hashes are in the JSON.
No application, complete bundle, raw observations or ambient environment is
archived. Temporary exported/inline Evidence files are removed after successful
owned-command cleanup.
For the Linux descendant-timeout check, create an operator-selected private
directory and set `REA_TOKENIZER_PROBE_RECORD` to a file within it. Run the same
real-artifact command with `--mode references --rounds 1`, replace the tokenizer
with the absolute path to `scripts/fixtures/tokenizer-family-wrapper.py`, and add
`--tokenizer-timeout-ms 2000`. Expect a timeout failure. The explicit marker
confirms the detached helper started; the resource observer must separately
verify the complete owned scope is empty, and the selected temporary root must
contain no verifier files. The recorded PID is an observation, not authority to
signal that process.
File diff suppressed because it is too large. Load diff
@@ -1,179 +0,0 @@
# Native handoff byte-identity verification
Tracks #3926 under #4027. Baseline: `9f8659e0536e4e50a438f5b9d5661f3c9624f6ac`.
Final integration base: `62bf09809ef7bb224028916424f8f07652a20f4f`.
The analyst already knows an ASAR-relative native path and supplies prior native
Evidence. The question is which Evidence can be linked to that component's exact
bytes. Discovery and native-provider execution are outside this task.
## Observed behavior and correction
The native-node producer distinguishes a content digest from a canonical path
whose bytes are unavailable. Its unavailable observation legitimately cites the
containing archive. The handoff consumer previously treated that supporting
artifact digest as the native component's digest.
Public CLI/stdio MCP regression: the missing-companion case failed and the
available-byte control passed. Real Joplin reproduced the same result: missing
keytar bytes produced an `evidence-linked` handoff with the ASAR digest and the
simulated container observation. CLI/MCP agreed on that incorrect answer.
The consumer now uses only the node owner's content-digest identity. The fallback
observation scan is removed. Unavailable native nodes retain their canonical
path, availability, container-backed provenance and partial coverage in the
inline graph; they yield no false actionable handoff or native Evidence link.
An empty handoff list describes eligibility, not absence of native components.
The per-query node index is allocated only after an eligible target exists; no
persistent cache, schema, tool or target authority is added.
Final real checks passed both cases with exact CLI/MCP trace parity. The available
control preserved the complete parent/trace Evidence IDs, native facts, links,
coverage and all token projections. The missing control preserved its parent,
native node, source provenance and coverage, while removing the wrong handoff
and container Evidence link.
## Real input and scope
[Official Joplin v3.7.21 Linux AppImage](https://github.com/laurent22/joplin/releases/download/v3.7.21/Joplin-3.7.21.AppImage),
published 2026-09-25, 201,462,481 bytes, SHA-256
`5d202dcaca31e14206826aa3f04125e7c218dbc3dc3d98479d811efbd6b856c1`.
Read-only 7z 26.02 inspection found 6,324 ASAR files, 46 unpacked entries and
three `.node` components. Extract only the ASAR and those three binaries:
```sh
input_root="$(mktemp -d -t rea-native-handoff-XXXXXX)"
curl --fail --location --output "$input_root/Joplin-3.7.21.AppImage" \
https://github.com/laurent22/joplin/releases/download/v3.7.21/Joplin-3.7.21.AppImage
sha256sum "$input_root/Joplin-3.7.21.AppImage"
7z x -mmt=1 "-o$input_root/extracted" "$input_root/Joplin-3.7.21.AppImage" \
resources/app.asar \
resources/app.asar.unpacked/node_modules/keytar/build/Release/keytar.node \
resources/app.asar.unpacked/node_modules/sqlite3/lib/binding/napi-v6-linux-glibc-x64/node_sqlite3.node \
resources/app.asar.unpacked/node_modules/onnxruntime-node/bin/napi-v6/linux/x64/onnxruntime_binding.node
python3 - "$input_root" <<'PY'
import os, sys
from pathlib import Path
root = Path(sys.argv[1])
source = root / 'extracted/resources'
for mode in ('available', 'missing'):
case = root / 'cases' / mode
case.mkdir(parents=True)
os.link(source / 'app.asar', case / 'app.asar')
if mode == 'available':
for native in (source / 'app.asar.unpacked').rglob('*.node'):
target = case / native.relative_to(source)
target.parent.mkdir(parents=True, exist_ok=True)
os.link(native, target)
PY
```
Verify the asset digest above before extraction. The ASAR is 233,575,014 bytes,
SHA-256 `767df409b18effe60421fbefcca873de237c7e813eac744ecdeebbc5e825a33a`.
Keytar is 76,960 bytes, SHA-256
`0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a`.
All extracted component hashes are in the JSON record. 7z reported one
trailing-data warning for the AppImage's embedded SquashFS and exited 0.
Both cases select only `package.json` for static source analysis while retaining
the inventory. The available case stages all three native companions; the other
43 unpacked entries are not staged. The missing case stages none. Whole-application
coverage is therefore partial in both cases, independently of the selected
native component's known or unknown byte identity. Neither the application nor
a native analysis provider is executed. Native Evidence is explicitly simulated
to test exact subject linkage. Live Electron, real Ghidra/Hopper/IDA behavior and
Windows/macOS verification are **NOT_ENOUGH_INFO**.
## Reproduce the public workflow
Use pinned Node 24.18.0/npm 11.16.0, `npm ci`, then `npm run build:cached`.
The verifier returns compact records and deletes its private exported parent and
CLI input after owned cleanup. No downloadable application or large Evidence is
archived.
```sh
for mode in available missing; do
node scripts/verify/javascript/native-handoffs.mjs \
--artifact "$input_root/cases/$mode/app.asar" \
--addon-path node_modules/keytar/build/Release/keytar.node \
--native-sha256 0700016bcd761cbe02d275f2046c7cc42f39f730a50d66e3d3307aa9d413373a \
--source-path package.json --rounds 3 --cli-parity --require-safe
done
```
Add `--tokenizer-python /absolute/path/to/python-with-tiktoken-0.12.0` for the
reported BPE projections. The npm alias `verify:javascript:native-handoffs`
builds first; recorded costs use the direct Node invocation above after building.
For a pre-fix observation, run the saved verifier on the baseline build without
`--require-safe`; `safe_identity: false` is an incorrect answer, not a passing
analyst task. Paths affect serialized requests and some Evidence identities.
The exact resource runs additionally used the preserved
[8 GiB observer](https://github.com/morluto/rea/blob/b3bdabab694af1f9801eff7caf75001b92314a58/experiments/analysis-efficiency/resource-run.py)
and adjacent `resource_payload.py`; both hashes are pinned in the JSON.
## Completed workflow costs and limits
| Case | Revision | Wall s | CPU s | Peak cgroup bytes | Identity oracle |
| ---------------------- | -------- | ------: | ---------: | ----------------: | -------------------------- |
| Native bytes available | Before | 60.369 | 59.806670 | 2,662,629,376 | pass |
| Native bytes available | After | 258.885 | 106.221003 | 2,678,456,320 | pass, exact baseline facts |
| Native bytes missing | Before | 57.930 | 57.479109 | 2,719,457,280 | wrong identity/linkage |
| Native bytes missing | After | 59.738 | 59.254776 | 2,571,235,328 | pass, unknown retained |
Whole-verifier resources include input hashing, analysis, three retained trace
queries, export, CLI parity and tokenization. One CPU, 8 GiB hard cap, 6 GiB early
stop, no swap; cleanup verified and no OOM. Memory includes charged file cache,
not summed RSS. Counters are at payload exit while the observer remains alive.
The available control's final wall and CPU times are substantially higher than
baseline. Earlier post-fix controls on the original base took 61.657/74.956 s wall
and 61.135962/60.352428 s CPU for available/missing cases; their complete receipts
remain separately recorded. The final missing case took 59.738 s wall. This
variability is unresolved and cannot be attributed to the native consumer change.
These runs establish neither speedup nor performance equivalence. Input hashing warms the ASAR
cache, build/OS caches and shared-host scheduling are uncontrolled. Fresh MCP
sessions do not establish OS-cold conditions. First post-analysis and repeated
warm-query samples are preserved separately; query CPU/memory are unmeasured.
One defined task is one analysis summary plus one retained trace, with the path
and prior Evidence already supplied. Static `cl100k_base`/tiktoken 0.12.0 counts
include both requests, replies and name/description/inputSchema contracts:
| Available-byte task, unchanged before/after | BPE tokens |
| ----------------------------------------------- | ---------: |
| Selected catalog + structured replies | 12,340 |
| Selected catalog + complete tool replies | 18,906 |
| Full catalog projection + structured replies | 202,292 |
| Full catalog projection + complete tool replies | 208,858 |
These are alternative reply/catalog projections. The selected catalog alone is
5,570 BPE; the full projection is 195,522. Output schemas, annotations, host
rendering/replay, reasoning, final answers and model billing are unmeasured.
Repeated queries/export/CLI/tokenization are validation overhead outside this
one-task token count, but inside resource receipts. Missing-case structured-task
counts change 11,372 → 11,103 by removing unsupported claims; the incorrect
baseline earns no efficiency credit.
## Regression and verification record
The initial four-file focused selection passed 24 tests. The two public cases
were rerun after adding named summary-result parsing and correcting an optional
architecture factory input. Omitting that field instead of passing null produced
byte-identical normalized simulated Evidence in all four real-case inputs.
The initial CLI-selector harness error and first typecheck failure are retained
as harness/validation attempts, not product reproductions.
After syncing the reviewed upstream changes, the updated application workflow
CLI acceptance file passed 34 tests, bringing the focused selection to 58 distinct
cases across five files. Both Joplin controls were then repeated on the final
integration base. Canonical byte identity is also used by the related version
matching/content comparison consumers; no equivalent observation-digest fallback
was found there.
Final `npm run check && npm run docs:check` passed on the integration base,
including five documentation consistency tests: 128.732 s wall, 115.518920 s CPU,
4,301,684,736 B peak cgroup memory, clean owned exit and no OOM.
Final public and static/docs outcomes are recorded in the
[compact measurements and source pins](javascript-native-handoff-identity-2026-10-15.json).
+3 -51
View File
@@ -17,7 +17,7 @@ additional permission flag or degraded mode is required.
- Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+;
- an operator-installed Ghidra 12.1.x distribution (verified with 12.1.4);
- a 64-bit full JDK inside that installation's declared Java range (JDK 21 or newer, with no maximum, for current 12.1 releases);
- an explicit native, non-managed x86, x86-64, ARMNT (Thumb-2) or ARM64 PE application/DLL, or a
- an explicit native, non-managed x86 or x86-64 PE application/DLL, or a
[raw NMOS 6502 image](raw-binary.md#nmos-6502) with a caller-supplied entry; and
- the 26 read-only Ghidra inventory, memory, and function-analysis operations.
@@ -135,55 +135,9 @@ and artifact SHA-256 before loading the package-owned addon. Native failure
reasons remain distinct from a valid installation and an unsupported target.
PE header classification and SHA-256 come from the same open file. The Windows
provider admits native x86, x86-64, ARMNT (Thumb-2) and ARM64 PE applications and DLLs and rejects unsupported roles,
provider admits native x86 and x86-64 PE applications and DLLs and rejects unsupported roles,
architectures, managed images, and malformed headers. The original selected
source coordinate is preserved for native admission. The original COFF machine is
retained separately from the normalized CPU family. ARM admission requires the
observed ARMNT machine 0x01C4; ARM64 requires 0xAA64. Legacy ARM (0x01C0), Thumb
(0x01C2), and unavailable machine facts do not select this Windows lane.
Without a caller-selected processor override, Ghidra's PE loader
selects the language from the PE machine field: ARM (`IMAGE_FILE_MACHINE_ARMNT`)
imports as `ARM:LE:32:v8` (Thumb), ARM64 (`IMAGE_FILE_MACHINE_ARM64`) as
`AARCH64:LE:64:v8A`. Explicit language/compiler selections remain caller choices
in the analysis profile and are recorded as explicit imports. The source
fixture verifier compares the complete observed language ID, not just its CPU
family prefix. ARM/ARM64 admission is a bounded extension of the x86/x64
lane. `npm run verify:ghidra:windows -- --arm` or `-- --arm64`, optionally
combined with `--dll`, selects encoded Thumb or ARM64 PE fixtures on a Windows
x64 host. The packaged verifier accepts the same flags. These fixtures are not
compiler-produced programs. ARM64EC (machine 0xA641) and ARM64X are not admitted;
this target support does not establish Windows-on-Arm host support.
Real Windows x64 verification with Ghidra 12.1.4 and JDK 21 completed the ARM64
application and ARMNT/Thumb DLL fixtures' 26 read-only operations, constant-return
decompilation, byte searches, CLI/MCP file-offset parity, source identity, and
owned cleanup. The DLL fixture also verifies the exported caller/callee
relationship. The x86/x64 compact-reference value-flow battery does not cover
ARM or ARM64 instructions. ARM64 EXE file-offset verification includes ambiguous
mappings; the single-section ARM32 DLL cannot exercise that case.
The runs used two logical CPUs, 512 MiB Node/JVM heaps, a 900-second startup
budget, and an explicit `REA_GHIDRA_DISABLED_ANALYZERS=WindowsResourceReference`
test configuration. Default-analyzer runs exceeded the selected startup budget;
resource-reference analysis and default-configuration completion are not
established by these results. A Microsoft-signed Coreinfo ARM64 application also
completed focused MCP decompilation, repeated readback, source-mapped memory
reads, unchanged-input and cleanup checks. On this constrained host the first
function query, including cold analysis, took about 434 seconds; its identical
same-session read took about 23 ms and returned about 4.6 KiB of Evidence. These
are individual observations, not general latency guarantees. Public ARM32
DLL verification also completed against Microsoft Visual C++ 2013 ARM
`msvcr120.dll`, including decompilation of its exported `abs`, repeated readback,
source-mapped bytes, unchanged-input and cleanup checks. Its first function
query took about 601 seconds; the same-session repeat took about 11 ms and
returned about 4.8 KiB of Evidence. After preserving original COFF machines in the
admission/profile, a further real Windows check rejected legacy ARM/Thumb and
retained ARMNT/ARM64 admission. A rebuilt-candidate MSVCR120 query completed
in about 407 seconds with a 15 ms repeat (4,830 bytes of Evidence), unchanged
input and complete cleanup. That retry selected an 1,800-second startup budget
after an additional 900-second attempt timed out. The resource bounds were
unchanged; these observations do not establish a speedup. Configured processor
selection is covered by profile tests, not these real-provider runs.
source coordinate is preserved for native admission.
Admission opens and checks every path component without following a reparse
point. It retains handles, verifies local NTFS semantics, and observes volume
@@ -246,8 +200,6 @@ npm run verify:ghidra:windows
npm run verify:ghidra:windows -- --x86
npm run verify:ghidra:windows -- --dll
npm run verify:ghidra:windows -- --x86 --dll
npm run verify:ghidra:windows -- --arm64
npm run verify:ghidra:windows -- --arm64 --dll
npm run verify:ghidra:windows:package -- --x86
npm run verify:ghidra:windows:package -- --dll
npm run verify:ghidra:windows:package -- --x86 --dll
+1
View File
@@ -10,6 +10,7 @@
"scripts/verify/fridaBindingsCompatibility.ts",
"scripts/verify/termux/browser.mjs",
"scripts/verify-real-ghidra-noreturn.mjs",
"scripts/run-exact-tests.mjs",
"tests/processOwnershipRunner.ts"
],
"project": ["src/**/*.ts", "tests/**/*.ts", "scripts/**/*.mjs"],
+4 -11
View File
@@ -111,14 +111,9 @@ automatically. Alternatively, build it using the Linux/MinGW lane above.
- Cleanup walks locked directory objects, removes files by handle, and unlinks
reparse entries without traversing their targets. Failed removal retains the
root and ancestor leases for retry; a partially removed runtime accepts only
cleanup, and its registry entry remains until removal succeeds. The runtime
owner retries only native close failures with Win32 sharing/lock codes 32/33,
for at most ten attempts with a 100 ms delay between attempts. Other native
errors are returned immediately; exhausting the retry budget preserves the
runtime owner for a later explicit close. These cleanup retries do not use an
analysis cancellation signal and schedule up to 900 ms of backoff, in
addition to native cleanup time. The owned root handle includes attribute-write access so cleanup can clear a
read-only marker, matching the child cleanup handles. Normal close, cancellation,
cleanup, and its registry entry remains until removal succeeds. The owned root
handle includes attribute-write access so cleanup can clear a read-only marker,
matching the child cleanup handles. Normal close, cancellation,
startup failure, and timeout release provider resources. Abrupt owner death
kills the owned job. The Android provider additionally uses a detached helper
that owns its private runtime directory and cleans it when the owner pipe
@@ -138,9 +133,7 @@ text do not establish a safe retry. Analysis errors propagate the producer's
retry disposition through JavaScript, source-map and WABT workflows; transient
failures report `category: unavailable` and retain the original reason.
Incomplete cleanup blocks an operation retry until its owned resources settle.
The close-stage retry is separate from allocation and analysis retries: it
repeats only the existing owned cleanup operation, and does not retry provider
work or create another runtime.
REA adds no automatic retry loop or new analysis cache for this behavior.
Ordinary drive-absolute paths accept backslashes, forward slashes, or mixed
separators. File identity retains the original requested spelling. Separator
+2 -6
View File
@@ -194,7 +194,6 @@
"verify:ghidra:swift": "npm run build:cached && node scripts/verify-real-ghidra-swift.mjs",
"verify:ida": "npm run build:cached && node scripts/verify-real-ida.mjs",
"verify:ghidra:switch": "npm run build:cached && node scripts/verify-real-ghidra-switch.mjs",
"verify:kuna": "npm run build:cached && node scripts/verify-real-kuna.mjs",
"verify:ghidra:dos": "npm run build:cached && node scripts/verify-real-ghidra-dos.mjs",
"verify:ghidra:com": "npm run build:cached && node scripts/verify-real-ghidra-com.mjs",
"verify:ghidra:raw": "npm run build:cached && node scripts/verify-real-ghidra-raw.mjs",
@@ -213,8 +212,6 @@
"verify:javascript-runtime": "npm run build:cached && node scripts/verify/javascript/runtime-observation.mjs",
"verify:javascript:digests": "npm run build:cached && node scripts/verify/javascript/digests.mjs",
"verify:javascript:output": "npm run build:cached && node scripts/verify/javascript/output.mjs",
"verify:javascript:module-references": "npm run build:cached && node scripts/verify/javascript/module-references.mjs",
"verify:javascript:native-handoffs": "npm run build:cached && node scripts/verify/javascript/native-handoffs.mjs",
"verify:application-workflows": "npm run build:cached && node scripts/verify/javascript/application-workflows.mjs",
"verify:managed": "npm run build:cached && node scripts/verify/managed/conformance.mjs",
"verify:firmware": "npm run build:cached && node scripts/verify/firmware/analysis.mjs",
@@ -273,7 +270,6 @@
"verify:windows-native": "node scripts/verify-windows-native.mjs",
"verify:ghidra:windows:package": "node scripts/verify-packaged-ghidra-windows.mjs",
"build:ghidra:nativeaot": "node scripts/build-ghidra-nativeaot.mjs",
"verify:ghidra:bridge-sources": "node scripts/verify-ghidra-bridge-sources.mjs",
"build:fixtures:nativeaot": "node scripts/generate-nativeaot-fixtures.mjs",
"verify:ghidra:nativeaot": "npm run build:cached && node scripts/verify-real-ghidra-nativeaot.mjs",
"verify:browser:scripts": "npm run build:cached && node scripts/verify-browser-scripts.mjs",
@@ -294,8 +290,8 @@
"verify:solidity:metadata": "npm run build:cached && node scripts/verify-solidity-metadata.mjs",
"verify:recorded:crash": "npm run build:cached && node scripts/verify-recorded-crash.mjs",
"mcp-catalog:check:built": "node scripts/generate-mcp-tool-catalog.mjs --check",
"compile": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/bundle-runtime.mjs",
"compile:termux": "node scripts/clean-build-output.mjs && node --max-old-space-size=3072 node_modules/typescript-termux/lib/tsc.js -p tsconfig.build.json && node scripts/record-catalog-digests.mjs && node scripts/bundle-runtime.mjs",
"compile": "node scripts/clean-build-output.mjs && tsc -p tsconfig.build.json && node scripts/bundle-runtime.mjs",
"compile:termux": "node scripts/clean-build-output.mjs && node --max-old-space-size=3072 node_modules/typescript-termux/lib/tsc.js -p tsconfig.build.json && node scripts/bundle-runtime.mjs",
"artifacts:mcp-catalog": "node scripts/generate-mcp-tool-catalog.mjs",
"artifacts:skills": "node scripts/generate-skill-metadata.mjs",
"artifacts:product-catalog": "node scripts/generate-product-catalog.mjs",
+1 -1
View File
@@ -157,7 +157,7 @@ function propertyValue(content, name) {
return null;
}
/** Release line supported by the separately pinned NativeAOT integration. */
/** Same 12.1 line as GhidraInstallationPolicy, including a distro suffix. */
function ghidraReleaseLineAccepted(raw) {
const match = /^(\d+)\.(\d+)(?:\.(\d+))?([-+].*)?$/u.exec(raw ?? "");
return match !== null && Number(match[1]) === 12 && Number(match[2]) === 1;
-10
View File
@@ -104,7 +104,6 @@ export const SCOPES = {
"tests/boundary/filesystem/npmRelease.test.ts",
"tests/boundary/process/releaseArtifactCanary.test.ts",
"src/windows/**",
"src/ghidra/GhidraBridgeSources.ts",
"native/windows/**",
"src/process/capture/FilesystemSnapshot.ts",
"src/process/capture/FilesystemSnapshot.test.ts",
@@ -139,7 +138,6 @@ export const SCOPES = {
"src/domain/javascript/javascriptRuntimeStaticCandidates.ts",
"tests/boundary/filesystem/webScriptManifestPaths.test.ts",
"tests/boundary/process/ProcessEvidence.host.test.ts",
"tests/boundary/process/packageNpmInvocation.test.ts",
"tests/boundary/process/windowsOutputCapture.test.ts",
"tests/acceptance/webModuleCli.test.ts",
"tests/acceptance/webSourceLocationCli.test.ts",
@@ -437,14 +435,6 @@ export const SCOPES = {
"tests/fixtures/nativeaot/**",
".github/workflows/nativeaot-fixtures.yml",
],
ghidra_bridge: [
"bridge/ghidra/**",
"src/ghidra/GhidraBridgeSources.ts",
"src/ghidra/GhidraJavaProperties.ts",
"src/ghidra/GhidraReleasePolicy.ts",
"scripts/verify-ghidra-bridge-sources.mjs",
".github/workflows/ghidra-bridge-sources.yml",
],
website: [
"website/**",
"scripts/*website*.py",
+45 -101
View File
@@ -3,52 +3,42 @@ import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { writeWindowsGhidraByteSearchFixture } from "./lib/ghidra-windows-byte-search.mjs";
import {
WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES,
windowsGhidraFixtureOptions,
} from "./lib/ghidra-windows-fixture-options.mjs";
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
// Source-owned static fixtures: no compiler, target loading or execution.
for (const architecture of WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES) {
for (const architecture of ["x86_64", "x86"]) {
for (const dll of [false, true]) {
const x86 = architecture === "x86";
const arm = architecture === "arm";
const arm64 = architecture === "arm64";
const fixture = windowsGhidraFixtureOptions([
...(architecture === "x86_64" ? [] : [`--${architecture}`]),
...(dll ? ["--dll"] : []),
]);
const pe32 = fixture.pe32;
const output = resolve(root, "build", "fixtures", fixture.filename);
const output = resolve(
root,
"build",
"fixtures",
`rea-ghidra-windows${x86 ? "-x86" : ""}.${dll ? "dll" : "exe"}`,
);
const bytes = Buffer.alloc(dll ? 1024 : 1536);
const peOffset = 0x80;
const optionalHeader = peOffset + 24;
const optionalHeaderSize = pe32 ? 0xe0 : 0xf0;
const optionalHeaderSize = x86 ? 0xe0 : 0xf0;
const sectionTable = optionalHeader + optionalHeaderSize;
const directories = optionalHeader + (pe32 ? 96 : 112);
const directories = optionalHeader + (x86 ? 96 : 112);
bytes.write("MZ", 0, "ascii");
bytes.writeUInt32LE(peOffset, 0x3c);
bytes.write("PE\0\0", peOffset, "binary");
// The fixture descriptor owns COFF machine, width and function coordinates.
bytes.writeUInt16LE(fixture.machine, peOffset + 4);
bytes.writeUInt16LE(x86 ? 0x14c : 0x8664, peOffset + 4);
bytes.writeUInt16LE(dll ? 1 : 2, peOffset + 6);
bytes.writeUInt16LE(optionalHeaderSize, peOffset + 20);
bytes.writeUInt16LE(
(pe32 ? 0x102 : 0x22) | (dll ? 0x2000 : 0),
(x86 ? 0x102 : 0x22) | (dll ? 0x2000 : 0),
peOffset + 22,
);
// ARM64 PE uses the PE32+ optional header (magic 0x20b), like x86-64.
bytes.writeUInt16LE(pe32 ? 0x10b : 0x20b, optionalHeader);
bytes.writeUInt16LE(x86 ? 0x10b : 0x20b, optionalHeader);
bytes.writeUInt32LE(0x200, optionalHeader + 4);
if (!dll) bytes.writeUInt32LE(0x200, optionalHeader + 8);
bytes.writeUInt32LE(dll ? 0 : arm ? 0x1001 : 0x1000, optionalHeader + 16);
bytes.writeUInt32LE(dll ? 0 : 0x1000, optionalHeader + 16);
bytes.writeUInt32LE(0x1000, optionalHeader + 20);
if (pe32)
bytes.writeUInt32LE(Number(fixture.imageBase), optionalHeader + 28);
else bytes.writeBigUInt64LE(fixture.imageBase, optionalHeader + 24);
if (x86) bytes.writeUInt32LE(0x400000, optionalHeader + 28);
else bytes.writeBigUInt64LE(0x1_4000_0000n, optionalHeader + 24);
bytes.writeUInt32LE(0x1000, optionalHeader + 32);
bytes.writeUInt32LE(0x200, optionalHeader + 36);
bytes.writeUInt16LE(6, optionalHeader + 40);
@@ -56,8 +46,8 @@ for (const architecture of WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES) {
bytes.writeUInt32LE(dll ? 0x2000 : 0x3000, optionalHeader + 56);
bytes.writeUInt32LE(0x200, optionalHeader + 60);
bytes.writeUInt16LE(3, optionalHeader + 68);
bytes.writeUInt16LE(pe32 ? 0x8140 : 0x8160, optionalHeader + 70);
if (pe32) {
bytes.writeUInt16LE(x86 ? 0x8140 : 0x8160, optionalHeader + 70);
if (x86) {
bytes.writeUInt32LE(0x10_0000, optionalHeader + 72);
bytes.writeUInt32LE(0x1000, optionalHeader + 76);
bytes.writeUInt32LE(0x10_0000, optionalHeader + 80);
@@ -79,52 +69,18 @@ for (const architecture of WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES) {
bytes.writeUInt32LE(0x6000_0020, sectionTable + 36);
// EXE entry returns zero. DLL caller adds one to the callee's 42.
if (arm) {
// ARMNT uses Thumb instructions and low-bit-tagged PE entry/export RVAs.
// PUSH keeps SP aligned and preserves LR across BL from 0x1002 to 0x1020.
bytes.set(
[
0x10,
0xb5,
0x00,
0xf0,
0x0d,
0xf8,
dll ? 0x01 : 0x00,
dll ? 0x30 : 0x20,
0x10,
0xbd,
],
0x200,
);
bytes.set([0x2a, 0x20, 0x70, 0x47], 0x220);
} else if (arm64) {
// Preserve the caller's frame/link registers across BL. The callee
// starts at RVA 0x1020, outside the five-instruction caller.
const instructions = [
0xa9bf7bfd, // STP X29, X30, [SP, #-16]!
0x94000007, // BL +0x1c from RVA 0x1004 to RVA 0x1020
dll ? 0x11000400 : 0x52800000, // ADD W0,W0,#1 / MOV W0,#0
0xa8c17bfd, // LDP X29, X30, [SP], #16
0xd65f03c0, // RET
];
for (const [index, instruction] of instructions.entries())
bytes.writeUInt32LE(instruction, 0x200 + index * 4);
bytes.set([0x40, 0x05, 0x80, 0x52, 0xc0, 0x03, 0x5f, 0xd6], 0x220);
} else {
bytes.set(
dll
? [0xe8, 0x0b, 0, 0, 0, 0x83, 0xc0, 1, 0xc3]
: [0xe8, 0x0b, 0, 0, 0, 0xe8, 0x16, 0, 0, 0, 0x31, 0xc0, 0xc3],
0x200,
);
bytes.set(
x86 && !dll
? [0xb8, 0x2a, 0, 0, 0, 0xe8, 0x66, 0, 0, 0, 0xc3]
: [0xb8, 0x2a, 0, 0, 0, 0xc3],
0x210,
);
}
bytes.set(
dll
? [0xe8, 0x0b, 0, 0, 0, 0x83, 0xc0, 1, 0xc3]
: [0xe8, 0x0b, 0, 0, 0, 0xe8, 0x16, 0, 0, 0, 0x31, 0xc0, 0xc3],
0x200,
);
bytes.set(
x86 && !dll
? [0xb8, 0x2a, 0, 0, 0, 0xe8, 0x66, 0, 0, 0, 0xc3]
: [0xb8, 0x2a, 0, 0, 0, 0xc3],
0x210,
);
if (x86 && !dll) {
// A statically analyzed Borland-style SEH registration whose handler
@@ -193,31 +149,19 @@ for (const architecture of WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES) {
"ffd7", // CALL EDI: unknown target.
"c3",
]
: arm
? ["7047"] // Thumb return; x86 reference-flow controls do not apply.
: arm64
? [
"010000b0", // ADRP X1, .data page (RVA 0x2000): DATA.
"200040b9", // LDR W0, [X1]: READ.
"200000b9", // STR W0, [X1]: WRITE.
"c0035fd6", // RET.
]
: [
"488d15d90f0000", // LEA RDX, [RIP + displacement]: DATA.
"8b05d30f0000", // MOV EAX, [RIP + displacement]: READ.
"8905d10f0000", // MOV [RIP + displacement], EAX: WRITE.
"488d15d60f0000", // Indexed base, unknown RCX.
"85db", // TEST EBX, EBX: two candidate bases.
"7407", // JZ over the alternate base.
"488d15d30f0000", // LEA RDX, alternate base.
"c6040a01", // MOV byte [RDX + RCX], 1.
"41ffd0", // CALL R8: unknown target.
"c3",
];
bytes.set(
Buffer.from(referenceInstructions.join(""), "hex"),
arm || arm64 ? 0x230 : 0x220,
);
: [
"488d15d90f0000", // LEA RDX, [RIP + displacement]: DATA.
"8b05d30f0000", // MOV EAX, [RIP + displacement]: READ.
"8905d10f0000", // MOV [RIP + displacement], EAX: WRITE.
"488d15d60f0000", // Indexed base, unknown RCX.
"85db", // TEST EBX, EBX: two candidate bases.
"7407", // JZ over the alternate base.
"488d15d30f0000", // LEA RDX, alternate base.
"c6040a01", // MOV byte [RDX + RCX], 1.
"41ffd0", // CALL R8: unknown target.
"c3",
];
bytes.set(Buffer.from(referenceInstructions.join(""), "hex"), 0x220);
}
if (dll) {
@@ -232,8 +176,8 @@ for (const architecture of WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES) {
bytes.writeUInt32LE(0x1080, exports + 28);
bytes.writeUInt32LE(0x1088, exports + 32);
bytes.writeUInt32LE(0x1090, exports + 36);
bytes.writeUInt32LE(arm ? 0x1001 : 0x1000, 0x280);
bytes.writeUInt32LE(fixture.calleeRva + (arm ? 1 : 0), 0x284);
bytes.writeUInt32LE(0x1000, 0x280);
bytes.writeUInt32LE(0x1010, 0x284);
bytes.writeUInt32LE(0x1120, 0x288);
bytes.writeUInt32LE(0x1140, 0x28c);
bytes.writeUInt16LE(0, 0x290);
@@ -247,6 +191,6 @@ for (const architecture of WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES) {
await mkdir(dirname(output), { recursive: true });
await writeFile(output, bytes);
if (architecture === "x86_64" && !dll) process.stdout.write(`${output}\n`);
if (!x86 && !dll) process.stdout.write(`${output}\n`);
}
}
+1 -14
View File
@@ -12,22 +12,13 @@ export async function startBrowserVerifierSite() {
return;
}
const requestUrl = new URL(request.url ?? "/", "http://127.0.0.1");
if (requestUrl.pathname === "/scenario-request-language") {
response.setHeader("content-type", "application/json");
response.setHeader("cache-control", "no-store");
response.end(JSON.stringify(request.headers["accept-language"] ?? null));
return;
}
if (requestUrl.pathname === "/scenario-environment") {
const requestedRun = requestUrl.searchParams.get("run");
const run = ["A", "B", "C"].includes(requestedRun)
? requestedRun
: "default";
response.setHeader("content-type", "text/html");
const requestLanguage = encodeURIComponent(
JSON.stringify(request.headers["accept-language"] ?? null),
);
response.end(`<!doctype html><html><body data-run="${run}" data-request-language="${requestLanguage}"><pre id="environment"></pre><pre id="language"></pre>
response.end(`<!doctype html><html><body data-run="${run}"><pre id="environment"></pre>
<button id="refresh-environment">Refresh environment</button><script>
const refresh = () => document.querySelector('#environment').textContent = JSON.stringify({
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
@@ -35,10 +26,6 @@ export async function startBrowserVerifierSite() {
dpr: devicePixelRatio, width: innerWidth, height: innerHeight
});
document.querySelector('#refresh-environment').onclick = refresh;
document.querySelector('#language').textContent = JSON.stringify({
language: navigator.language, languages: [...navigator.languages],
request_language: JSON.parse(decodeURIComponent(document.body.dataset.requestLanguage))
});
window.releaseScenario = () => {
const release = document.createElement('span');
release.id = 'release-' + document.body.dataset.run;
@@ -1,36 +0,0 @@
#!/usr/bin/env python3
"""POSIX fault fixture for the optional module-reference token counter.
Use as --tokenizer-python with an explicit REA_TOKENIZER_PROBE_RECORD path.
It deliberately ignores the counter arguments, forks a detached helper that
ignores SIGTERM, and stalls. The production owned-command supervisor must
terminate the verified family. Recorded PIDs are observations, not authority
to signal a process; verification also checks the owned scope after exit.
"""
import json
import os
from pathlib import Path
import signal
import time
def main():
record = Path(os.environ["REA_TOKENIZER_PROBE_RECORD"])
child = os.fork()
if child == 0:
os.setsid()
signal.signal(signal.SIGTERM, signal.SIG_IGN)
record.write_text(json.dumps({
"pid": os.getpid(),
"group": os.getpgrp(),
"parent": os.getppid(),
"run_id": os.environ.get("REA_PROCESS_RUN_ID"),
"detached": True,
"ignores_sigterm": True,
}) + "\n", encoding="utf-8")
time.sleep(30)
if __name__ == "__main__":
main()
@@ -11,7 +11,6 @@ import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
import { AnalysisCancelledError } from "../../dist/domain/analysisErrorCore.js";
import { runScenarioCli } from "./browser-scenario-verifier.mjs";
import { requireMcpEvidenceResult } from "./mcp-verifier-results.mjs";
import { closeVerifierResources } from "./verifier-cleanup.mjs";
const expected = {
timezone: "Pacific/Honolulu",
@@ -97,32 +96,8 @@ const assertCapture = (capture, values = expected) => {
}
};
const assertCallerLanguage = (capture, baseline) => {
assert.ok(
capture.limitations.some((value) =>
value.includes("Attached locale fr-FR applies to Intl formatting only"),
),
"Attached capture must identify its page/request language limitation",
);
for (const step of capture.steps) {
const text = step.artifacts.dom.value.text;
const observed = /<pre id="language">([^<]+)<\/pre>/u.exec(text);
assert.ok(
observed,
"DOM omitted the navigator/request language observations",
);
assert.deepEqual(JSON.parse(observed[1]), baseline);
}
};
/** Verify actual attached-page emulation and restoration across scenario outcomes. */
export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
const languageExpression = `(async () => JSON.stringify({
language: navigator.language, languages: [...navigator.languages],
request_language: await (await fetch(
${JSON.stringify(`${origin}/scenario-request-language`)}, { cache: "no-store" }
)).json()
}))()`;
const targets = await (await fetch(`${endpoint}/json/list`)).json();
const target = targets.find(({ id }) => id === targetId);
assert.ok(target, "External fixture target is missing");
@@ -130,38 +105,28 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
target.webSocketDebuggerUrl,
"capture_browser_scenario",
);
const readEnvironment = async (expressionToRead = expression) => {
const readEnvironment = async () => {
const value = await connection.send("Runtime.evaluate", {
expression: expressionToRead,
expression,
returnByValue: true,
awaitPromise: true,
});
assert.equal(value.result.type, "string");
return JSON.parse(value.result.value);
};
const provider = createBrowserScenarioProvider(process.env);
let verificationError;
const baseline = await readEnvironment();
const scenario = environmentScenario({
endpoint,
targetId,
origin,
values: expected,
});
const assertRestored = async () => {
assert.deepEqual(await readEnvironment(), baseline);
const remaining = await (await fetch(`${endpoint}/json/list`)).json();
assert.ok(remaining.some(({ id }) => id === targetId));
};
try {
const baseline = await readEnvironment();
const callerLanguage = await readEnvironment(languageExpression);
const scenario = environmentScenario({
endpoint,
targetId,
origin,
values: expected,
});
const assertRestored = async () => {
assert.deepEqual(await readEnvironment(), baseline);
assert.deepEqual(
await readEnvironment(languageExpression),
callerLanguage,
);
const remaining = await (await fetch(`${endpoint}/json/list`)).json();
assert.ok(remaining.some(({ id }) => id === targetId));
};
const cliCapture = (await runScenarioCli(scenario)).normalized_result;
assertCapture(cliCapture);
assertCallerLanguage(cliCapture, callerLanguage);
assertCapture((await runScenarioCli(scenario)).normalized_result);
await assertRestored();
const transport = new StdioClientTransport({
command: process.execPath,
@@ -179,15 +144,15 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
({ name }) => name === "capture_browser_scenario",
);
assert.ok(definition, "Scenario tool is absent from the current catalog");
const mcpCapture = requireMcpEvidenceResult(
await client.callTool(
{ name: "capture_browser_scenario", arguments: scenario },
{ toolDefinition: definition },
assertCapture(
requireMcpEvidenceResult(
await client.callTool(
{ name: "capture_browser_scenario", arguments: scenario },
{ toolDefinition: definition },
),
"capture_browser_scenario",
),
"capture_browser_scenario",
);
assertCapture(mcpCapture);
assertCallerLanguage(mcpCapture, callerLanguage);
await verifyConcurrentMcpCapture({
client,
definition,
@@ -202,6 +167,7 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
await transport.close();
}
await assertRestored();
const provider = createBrowserScenarioProvider(process.env);
const initializationFailure = await provider.captureScenario({
...scenario,
environment: { ...scenario.environment, timezone: "REA/Invalid" },
@@ -233,21 +199,13 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
stdio_mcp: true,
observed: expected,
restored: baseline,
caller_language_preserved: callerLanguage,
attached_locale_limitation: true,
initialization_failure_cleanup: true,
action_failure_cleanup: true,
cancellation_cleanup: true,
external_target_preserved: true,
};
} catch (cause) {
verificationError = cause;
throw cause;
} finally {
await closeVerifierResources(
[() => provider.close(), () => connection.close()],
verificationError,
);
await connection.close();
}
}
@@ -398,11 +356,7 @@ const releaseScenario = async (connection) => {
});
};
export async function verifyCancelledEnvironment(
provider,
connection,
scenario,
) {
async function verifyCancelledEnvironment(provider, connection, scenario) {
const controller = new AbortController();
const priorSnapshot = await readScenarioSnapshot(connection);
const pending = provider.captureScenario(
@@ -424,15 +378,9 @@ export async function verifyCancelledEnvironment(
await settlement;
}
const cancelled = await pending;
if (!cancelled.ok) {
if (!(cancelled.error instanceof AnalysisCancelledError))
throw cancelled.error;
assert.equal(cancelled.error.cleanupIncomplete, false);
} else {
assert.ok(
assert.ok(
!cancelled.ok ||
cancelled.value.steps.some(({ status }) => status === "cancelled"),
"Cancellation unexpectedly completed every action",
);
assert.equal(cancelled.value.browser.cleanup, "disconnected-external");
}
"Cancellation unexpectedly completed every action",
);
}
+48 -88
View File
@@ -10,7 +10,6 @@ import {
runScenarioCli,
scenarioProfiles,
} from "./browser-scenario-verifier.mjs";
import { closeVerifierResources } from "./verifier-cleanup.mjs";
const scenarioFor = (browser, origin) =>
browserScenarioSchema.parse({
@@ -300,7 +299,6 @@ const verifyConnectedStorage = async (endpoint, targetId, origin) => {
target.webSocketDebuggerUrl,
"capture_browser_scenario",
);
let failure;
try {
await assertStorageKeyPartition(observer, origin);
const result = await runScenarioCli(scenario);
@@ -315,14 +313,11 @@ const verifyConnectedStorage = async (endpoint, targetId, origin) => {
assert.equal(capture.browser.cleanup, "disconnected-external");
await verifyCallerDebuggerPause(observer, origin, scenario);
await verifyCancelledStorage(observer, endpoint, targetId, scenario);
} catch (cause) {
failure = cause;
throw cause;
} finally {
await closeVerifierResources(
[() => restoreTarget(observer, target.url), () => observer.close()],
failure,
await raceWithTimeout(restoreTarget(observer, target.url), 1_000).catch(
() => undefined,
);
await raceWithTimeout(observer.close(), 1_000).catch(() => undefined);
}
};
@@ -508,8 +503,7 @@ const verifyCallerDebuggerPause = async (caller, origin, scenario) => {
],
});
let settled = false;
const provider = createBrowserScenarioProvider(process.env);
const pending = provider
const pending = createBrowserScenarioProvider(process.env)
.captureScenario(debuggerScenario, { signal: controller.signal })
.finally(() => {
settled = true;
@@ -592,10 +586,9 @@ const verifyCallerDebuggerPause = async (caller, origin, scenario) => {
});
} finally {
unsubscribe();
await closeVerifierResources([
() => caller.send("Debugger.disable"),
() => provider.close(),
]);
await raceWithTimeout(caller.send("Debugger.disable"), 1_000).catch(
() => undefined,
);
}
};
@@ -622,48 +615,38 @@ const verifyCancelledStorage = async (
expression: "performance.timeOrigin",
returnByValue: true,
});
const provider = createBrowserScenarioProvider(process.env);
const pending = provider.captureScenario(cancelledScenario, {
signal: controller.signal,
const pending = createBrowserScenarioProvider(process.env).captureScenario(
cancelledScenario,
{ signal: controller.signal },
);
await waitForNewDocument(
observer,
"/storage-main",
priorDocument.result.value,
);
const state = await observer.send("Runtime.evaluate", {
expression:
"JSON.parse(document.querySelector('#main-report').textContent)",
returnByValue: true,
});
let failure;
try {
await waitForNewDocument(
observer,
"/storage-main",
priorDocument.result.value,
);
const state = await observer.send("Runtime.evaluate", {
expression:
"JSON.parse(document.querySelector('#main-report').textContent)",
returnByValue: true,
});
assert.deepEqual(state.result.value, {
local: "main-initial",
session: "main-session-initial",
removed: "main-remove-seed",
empty: "main-empty-seed",
});
controller.abort(new Error("cancel storage seed scenario"));
const result = await pending;
if (result.ok)
assert.ok(
result.value.steps.some(({ status }) => status === "cancelled"),
"Cancellation unexpectedly completed every scenario step",
);
else assert.equal(result.error._tag, "AnalysisCancelledError");
const targets = await (await fetch(`${endpoint}/json/list`)).json();
assert.ok(
targets.some(({ id }) => id === targetId),
"Cancelled connect-mode storage cleanup closed the external target",
);
} catch (cause) {
failure = cause;
throw cause;
} finally {
controller.abort(new Error("cancel failed storage verification"));
await closeVerifierResources([() => provider.close()], failure);
}
assert.deepEqual(state.result.value, {
local: "main-initial",
session: "main-session-initial",
removed: "main-remove-seed",
empty: "main-empty-seed",
});
controller.abort(new Error("cancel storage seed scenario"));
const result = await pending;
assert.ok(
!result.ok ||
result.value.steps.some(({ status }) => status === "cancelled"),
"Cancellation unexpectedly completed every scenario step",
);
const targets = await (await fetch(`${endpoint}/json/list`)).json();
assert.ok(
targets.some(({ id }) => id === targetId),
"Cancelled connect-mode storage cleanup closed the external target",
);
};
const waitForNewDocument = async (connection, path, priorTimeOrigin) => {
@@ -686,38 +669,15 @@ const waitForNewDocument = async (connection, path, priorTimeOrigin) => {
};
const restoreTarget = async (connection, url) => {
const loaded = new Set();
let loaderId;
let resolveLoaded;
const load = new Promise((resolve) => {
resolveLoaded = resolve;
});
const unsubscribe = connection.onEvent((event) => {
if (
event.method !== "Page.lifecycleEvent" ||
event.params?.name !== "load" ||
typeof event.params.loaderId !== "string"
)
return;
loaded.add(event.params.loaderId);
if (event.params.loaderId === loaderId) resolveLoaded();
});
try {
await connection.send("Page.setLifecycleEventsEnabled", { enabled: true });
const navigation = await connection.send("Page.navigate", { url });
assert.equal(navigation.errorText, undefined);
assert.equal(typeof navigation.loaderId, "string");
loaderId = navigation.loaderId;
// A URL changes at commit, before old child frames have been retired.
// Do not hand the shared page to the next verifier until this load ends.
if (!loaded.has(loaderId))
await raceWithTimeout(load, 10_000, () => {
throw new Error(
"External browser did not finish restoring its document",
);
});
} finally {
unsubscribe();
await connection.send("Page.setLifecycleEventsEnabled", { enabled: false });
await connection.send("Page.navigate", { url });
const deadline = Date.now() + 10_000;
while (Date.now() < deadline) {
const result = await connection.send("Runtime.evaluate", {
expression: "location.href",
returnByValue: true,
});
if (result.result.value === url) return;
await new Promise((resolve) => setTimeout(resolve, 25));
}
throw new Error("External browser did not return to its original URL");
};
-7
View File
@@ -218,13 +218,6 @@ export const providerCatalog = (sources) => {
identity: sources.idaProvider.IDA_PROVIDER_IDENTITY,
contracts: declaredContracts("ida", sources.idaProvider.IDA_OPERATIONS),
},
{
identity: sources.kunaProvider.KUNA_PROVIDER_IDENTITY,
contracts: declaredContracts(
"kuna",
sources.kunaProvider.KUNA_OPERATIONS,
),
},
{
identity: sources.nativeProvider.NATIVE_MACOS_PROVIDER_IDENTITY,
contracts: sources.nativeContracts.NATIVE_TOOL_CONTRACTS,
-1
View File
@@ -109,7 +109,6 @@ const SOURCE_PATHS = {
hopperProvider: "dist/hopper/HopperProviderCapabilities.js",
ghidraProvider: "dist/ghidra/GhidraProviderCapabilities.js",
idaProvider: "dist/ida/IdaProviderCapabilities.js",
kunaProvider: "dist/kuna/KunaProviderCapabilities.js",
nativeProvider: "dist/native/NativeMacOSProviderMetadata.js",
artifactProviders: "dist/application/InvestigationProviders.js",
browserIdentities: "dist/browser/providerIdentities.js",
+2 -1
View File
@@ -18,12 +18,13 @@ export function writeWindowsGhidraByteSearchFixture(bytes) {
export async function verifyWindowsGhidraByteSearch({
invoke,
sourceBytes,
imageBase,
architecture,
sourceOwnedFixture = true,
fallbackAddress,
fallbackFileOffset,
runCliPage,
}) {
const imageBase = architecture === "x86" ? 0x400000n : 0x140000000n;
const query = sourceOwnedFixture
? {
pattern: WINDOWS_GHIDRA_BYTE_SEARCH_FIXTURE.pattern,
@@ -1,21 +0,0 @@
export const WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES: readonly [
"x86_64",
"x86",
"arm",
"arm64",
];
export const WINDOWS_GHIDRA_FIXTURE_FLAGS: readonly string[];
/** Select one architecture without interpreting positional paths as flags. */
export function windowsGhidraFixtureOptions(arguments_: readonly string[]): {
architecture: (typeof WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES)[number];
dll: boolean;
pe32: boolean;
imageBase: bigint;
calleeRva: number;
machine: number;
filename: string;
languageId: string;
callerAddress: string;
calleeAddress: string;
};
@@ -1,50 +0,0 @@
/** Source-owned PE fixtures; these are encoded bytes, not compiled programs. */
/** @type {readonly ["x86_64", "x86", "arm", "arm64"]} */
export const WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES = Object.freeze([
"x86_64",
"x86",
"arm",
"arm64",
]);
export const WINDOWS_GHIDRA_FIXTURE_FLAGS = Object.freeze([
"--x86",
"--arm",
"--arm64",
"--dll",
]);
/** Select one architecture without interpreting positional paths as flags. */
/** @param {readonly string[]} arguments_ */
export const windowsGhidraFixtureOptions = (arguments_) => {
const selected = WINDOWS_GHIDRA_FIXTURE_ARCHITECTURES.filter(
(architecture) =>
architecture !== "x86_64" && arguments_.includes(`--${architecture}`),
);
if (selected.length > 1)
throw new Error("Choose one Windows PE fixture architecture.");
const architecture = selected[0] ?? "x86_64";
const dll = arguments_.includes("--dll");
const pe32 = architecture === "x86" || architecture === "arm";
const imageBase = pe32 ? 0x400000n : 0x140000000n;
const calleeRva = architecture.startsWith("arm") ? 0x1020 : 0x1010;
return {
architecture,
dll,
pe32,
imageBase,
calleeRva,
machine: { x86: 0x14c, x86_64: 0x8664, arm: 0x1c4, arm64: 0xaa64 }[
architecture
],
filename: `rea-ghidra-windows${architecture === "x86_64" ? "" : `-${architecture}`}.${dll ? "dll" : "exe"}`,
languageId:
architecture === "arm"
? "ARM:LE:32:v8"
: architecture === "arm64"
? "AARCH64:LE:64:v8A"
: `x86:LE:${pe32 ? 32 : 64}:default`,
callerAddress: `0x${(imageBase + 0x1000n).toString(16)}`,
calleeAddress: `0x${(imageBase + BigInt(calleeRva)).toString(16)}`,
};
};
@@ -1,28 +0,0 @@
import type { z } from "zod";
export interface StructuredScopeExpectation {
readonly operation:
| "parse-package-json"
| "parse-json-module"
| "parse-html-script-references"
| "parse-local-source-map";
readonly analysis_status: "interrupted" | "not-started";
}
export interface StructuredProcessingScope {
readonly properties: Readonly<Record<string, unknown>>;
readonly evidence: {
readonly authority: string;
readonly state: string;
readonly confidence: string;
};
}
export const structuredScopeExpectationsSchema: z.ZodType<
Record<string, StructuredScopeExpectation>
>;
export function matchesStructuredScopeExpectation(
scopes: readonly StructuredProcessingScope[],
expected: StructuredScopeExpectation | undefined,
): boolean;
@@ -1,28 +0,0 @@
import { z } from "zod";
/** Independent expected outcomes supplied by the verification scenario. */
export const structuredScopeExpectationsSchema = z.record(
z.string().min(1),
z.strictObject({
operation: z.enum([
"parse-package-json",
"parse-json-module",
"parse-html-script-references",
"parse-local-source-map",
]),
analysis_status: z.enum(["interrupted", "not-started"]),
}),
);
/** Every retained gap must match the named phase and its unavailable status. */
export const matchesStructuredScopeExpectation = (scopes, expected) =>
expected !== undefined &&
scopes.length > 0 &&
scopes.every(
({ evidence, properties }) =>
evidence.authority === "artifact-bytes" &&
evidence.state === "unavailable" &&
evidence.confidence === "unknown" &&
properties.operation === expected.operation &&
properties.analysis_status === expected.analysis_status,
);
@@ -1,46 +0,0 @@
import assert from "node:assert/strict";
import { realpath } from "node:fs/promises";
import { runKunaCommand } from "../../dist/kuna/KunaCommand.js";
import { inspectKunaInstallation } from "../../dist/kuna/KunaInstallation.js";
import { KUNA_XREF_OUTPUT_BYTES } from "../../dist/kuna/KunaReferenceFiles.js";
/** Independent projection of native exact-target rows, bypassing REA's parser. */
export const nativeKunaIncomingSites = async ({
executable,
cacheRoot,
workspace,
binary,
address,
retain,
}) => {
const installed = await inspectKunaInstallation({
REA_KUNA_COMMAND: executable,
REA_KUNA_CACHE_DIR: cacheRoot,
});
assert.ok(installed.ok, "Native reference oracle could not inspect Kuna");
const input = await realpath(binary);
const timeoutMs = installed.value.configuration.functionTimeoutMs;
const outcome = await runKunaCommand({
installation: installed.value,
arguments: ["xrefs", input, "--to", address, "--json"],
workspace,
operation: "verify:kuna-native-xrefs",
timeoutMs,
reportedTimeoutMs: timeoutMs,
outputBytes: KUNA_XREF_OUTPUT_BYTES,
signal: new AbortController().signal,
});
if (outcome.kind === "retained") retain(outcome);
if (outcome.kind !== "ok") throw outcome.error;
const document = JSON.parse(outcome.snapshot.stdout.text);
assert.equal(document.binary, input);
assert.equal(document.direction, "to");
assert.equal(document.target.address_hex, address);
assert.equal(document.count, document.xrefs.length);
const sites = document.xrefs
.filter((row) => row.to_address_hex === address)
.map((row) => row.from_address_hex);
assert.ok(sites.every((site) => /^0x[0-9a-f]+$/u.test(site)));
return [...new Set(sites)].sort();
};
-109
View File
@@ -1,109 +0,0 @@
import assert from "node:assert/strict";
import { readdir } from "node:fs/promises";
import { join } from "node:path";
import { combineFunctionRelationshipCoverage } from "../../dist/domain/functionCallCoverage.js";
const entries = async (path) => {
try {
return (await readdir(path)).sort();
} catch (cause) {
if (cause.code === "ENOENT") return [];
throw cause;
}
};
/** Verify selected public reference answers without requesting source text. */
export const verifyKunaReferences = async ({
call,
cacheRoot,
name,
address,
expectedCaller,
nativeIncoming,
}) => {
const before = await Promise.all([
entries(join(cacheRoot, "projects")),
entries(join(cacheRoot, "staging")),
]);
const callers = await call("procedure_callers", { procedure: address });
const callees = await call("procedure_callees", { procedure: address });
const incoming = await call("xrefs", { address });
for (const [label, result] of [
["callers", callers],
["callees", callees],
["xrefs", incoming],
]) {
assert.ok(
Array.isArray(result.value),
`${name} ${label} omitted its address list`,
);
assert.equal(
new Set(result.value).size,
result.value.length,
`${label} repeats an address`,
);
assert.ok(
result.value.every((value) =>
/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/u.test(value),
),
`${label} returned a noncanonical u64 address`,
);
assert.equal(result.structured.provider.id, "kuna");
const source = result.structured.raw_result.references;
assert.match(source.sha256, /^[0-9a-f]{64}$/u);
assert.ok(source.bytes > 0);
}
let composed;
if (expectedCaller !== undefined) {
assert.ok(
callers.value.includes(expectedCaller),
`${name} lost its source-owned caller`,
);
assert.ok(
incoming.value.length > 0,
`${name} lost its source-owned call site`,
);
assert.deepEqual(
[...incoming.value].sort(),
await nativeIncoming(address),
`${name} incoming references differ from exact-target native sites`,
);
const graph = await call("get_call_graph", {
address: expectedCaller,
direction: "forward",
});
assert.ok(graph.value.graph["0"][0].calls.includes(address));
const path = await call("trace_call_path", {
start: expectedCaller,
goal: address,
direction: "forward",
});
assert.equal(path.value.goal_status, "reached");
composed = { graph_ms: graph.ms, path_ms: path.ms };
}
assert.deepEqual(
await Promise.all([
entries(join(cacheRoot, "projects")),
entries(join(cacheRoot, "staging")),
]),
before,
"A reference-only workflow started a C project export",
);
const repeated = await call("procedure_callees", { procedure: address });
assert.deepEqual(repeated.value, callees.value);
return {
callers: callers.value.length,
callees: callees.value.length,
xrefs: incoming.value.length,
first_graph_ms: callers.ms,
warm_graph_ms: repeated.ms,
xrefs_ms: incoming.ms,
response_bytes: callers.bytes + callees.bytes + incoming.bytes,
relationship_coverage: combineFunctionRelationshipCoverage(
callers.structured.relationship_coverage,
callees.structured.relationship_coverage,
),
...composed,
};
};
-166
View File
@@ -1,166 +0,0 @@
import assert from "node:assert/strict";
import { readdir, realpath } from "node:fs/promises";
import { join } from "node:path";
import { runKunaCommand } from "../../dist/kuna/KunaCommand.js";
import { inspectKunaInstallation } from "../../dist/kuna/KunaInstallation.js";
const entries = (path) =>
readdir(path).catch((cause) => {
if (cause.code === "ENOENT") return [];
throw cause;
});
/** Independent native oracle; uses source hex fields rather than JS numeric addresses. */
export const nativeKunaStrings = async ({
executable,
cacheRoot,
workspace,
binary,
retain,
}) => {
const installed = await inspectKunaInstallation({
REA_KUNA_COMMAND: executable,
REA_KUNA_CACHE_DIR: cacheRoot,
});
assert.ok(installed.ok, "Native string oracle could not inspect Kuna");
const input = await realpath(binary);
const outcome = await runKunaCommand({
installation: installed.value,
arguments: [
"strings",
input,
"--json",
"--encoding",
"all",
"--min-length",
"5",
"--termination",
"any",
"--no-xrefs",
],
workspace,
operation: "verify:kuna-native-strings",
timeoutMs: installed.value.configuration.projectTimeoutMs,
reportedTimeoutMs: installed.value.configuration.projectTimeoutMs,
outputBytes: 16 * 1024 * 1024,
signal: new AbortController().signal,
});
if (outcome.kind === "retained") retain(outcome);
if (outcome.kind !== "ok") throw outcome.error;
const document = JSON.parse(outcome.snapshot.stdout.text);
assert.equal(document.binary, input);
assert.equal(document.count, document.strings.length);
assert.equal(document.xrefs, false);
return document.strings
.map((row) => ({
address: row.address_hex,
value: row.text,
string: {
encoding: row.encoding,
encoding_status: "observed",
termination: row.nul_terminated ? "present_or_not_required" : "missing",
byte_length: row.byte_length,
},
}))
.sort((left, right) => {
const a = BigInt(left.address),
b = BigInt(right.address);
return a < b
? -1
: a > b
? 1
: left.string.encoding.localeCompare(right.string.encoding, "en");
});
};
/** Verify focused/complete string operations before any procedure analysis starts. */
export const verifyKunaStrings = async ({
call,
cacheRoot,
nativeStrings,
expectedStrings = [],
}) => {
const expected = await nativeStrings();
if (expectedStrings.length > 0)
assert.ok(
expected.length > 0,
"Source-owned string fixture did not exercise a positive result",
);
const pattern =
expected.find(({ value }) => value.length >= 8 && value.length <= 100)
?.value ??
expected[0]?.value ??
"REA-empty-string-inventory";
const selected = expected.filter(({ value }) =>
value.toLowerCase().includes(pattern.toLowerCase()),
);
const before = await Promise.all([
entries(join(cacheRoot, "projects")),
entries(join(cacheRoot, "staging")),
]);
const cold = await call("search_strings", { pattern });
assert.deepEqual(
cold.value,
selected,
"String search differs from native scan",
);
const source = cold.structured.raw_result.strings;
assert.equal(source.inventory_count, expected.length);
assert.equal(source.selected_count, selected.length);
assert.equal(source.xrefs, false);
assert.match(source.sha256, /^[0-9a-f]{64}$/u);
assert.ok(source.bytes > 0);
assert.equal(cold.structured.provider.id, "kuna");
assert.ok(cold.structured.evidence_id);
const complete = await call("list_strings");
assert.deepEqual(complete.value, expected);
for (const value of expectedStrings)
assert.ok(
complete.value.some((row) => row.value === value),
`Required string ${JSON.stringify(value)} was not recovered`,
);
assert.equal(complete.structured.raw_result.strings.sha256, source.sha256);
const repeated = await call("search_strings", { pattern });
assert.deepEqual(repeated.value, selected);
let exact;
if (selected.length > 0) {
const address = selected[0].address;
exact = await call("list_strings", { address });
assert.deepEqual(
exact.value,
expected.filter((item) => item.address === address),
);
}
const regex = await call("search_strings", {
pattern: "^.{5,12}$",
mode: "regex",
case_sensitive: true,
});
assert.deepEqual(
regex.value,
expected.filter(({ value }) => /^.{5,12}$/u.test(value)),
);
const missing = await call("search_strings", {
pattern: "\u0000REA-negative-string-query\u0000",
});
assert.deepEqual(missing.value, []);
assert.deepEqual(
await Promise.all([
entries(join(cacheRoot, "projects")),
entries(join(cacheRoot, "staging")),
]),
before,
"String reads started a project export",
);
return {
inventory_count: expected.length,
selected_count: selected.length,
cold_scan_ms: cold.ms,
warm_search_ms: repeated.ms,
exact_address_ms: exact?.ms ?? null,
cold_response_bytes: cold.bytes,
complete_response_bytes: complete.bytes,
regex_response_bytes: regex.bytes,
};
};
@@ -146,9 +146,6 @@ async function verifySourceReadFailures({
...(permissionDenied ? ["permission"] : []),
];
for (const scenario of scenarios) {
// Same-target reopen may admit restored bytes through retained-client health.
// Each rejection needs a cold client, without an existing private snapshot.
await call("close_binary");
await call("open_binary", { path: source, provider_id: "ghidra" });
const before = await call("binary_session");
if (scenario === "permission") await chmod(source, 0);
+1 -1
View File
@@ -17,7 +17,7 @@ export const SOURCE_TEST_PROJECTS = [
group: "source",
include: [
"src/*.test.ts",
"src/{apktool,jeb,adb,artifacts,browser,dotnet,electron,evm,firmware,flutter,pcap,ghidra,go,hopper,ida,inspector,javascript,native,process,reference,rust,server,sqlite,wasm,windows}/**/*.test.ts",
"src/{apktool,jeb,adb,artifacts,browser,dotnet,electron,evm,firmware,flutter,pcap,ghidra,go,hopper,ida,inspector,javascript,native,process,reference,rust,server,wasm,windows}/**/*.test.ts",
],
},
{
-20
View File
@@ -1,6 +1,5 @@
import { execFile } from "node:child_process";
import { lstat } from "node:fs/promises";
import { dirname, join } from "node:path";
import { promisify } from "node:util";
import { Ajv2020 } from "ajv/dist/2020.js";
@@ -18,25 +17,6 @@ const loadMcpVerifierResults = () => import("./mcp-verifier-results.mjs");
/** Promisified child_process execution helper. */
export const exec = promisify(execFile);
/** Run npm through its JavaScript entry point so Windows command scripts are not required. */
export const execNpm = (arguments_, options = {}) => {
const environment = options.env ?? process.env;
const selectedEntry = environment.npm_execpath;
if (
selectedEntry !== undefined &&
(typeof selectedEntry !== "string" || selectedEntry.length === 0)
)
throw new Error("npm_execpath must name the selected npm CLI entry point");
const entry =
selectedEntry ??
(process.platform === "win32"
? join(dirname(process.execPath), "node_modules/npm/bin/npm-cli.js")
: undefined);
return entry === undefined
? exec("npm", arguments_, options)
: exec(process.execPath, [entry, ...arguments_], options);
};
/** Run a command and return its stdout. */
export const run = async (command, args, env) =>
(await exec(command, args, { env })).stdout;
-4
View File
@@ -1,4 +0,0 @@
// A single inspection can include native admission plus several WABT commands.
// The aggregate verifier covers seven complete fixtures and their failure paths.
export const WABT_OPERATION_TIMEOUT_MS = 4 * 60_000;
export const WABT_WORKFLOW_TIMEOUT_MS = 15 * 60_000;
-19
View File
@@ -1,19 +0,0 @@
import { writeFile } from "node:fs/promises";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath, pathToFileURL } from "node:url";
// Record the default catalog digests into the compiled runtime. Run after
// `tsc` and before bundling, so the digests come from the same compiled
// contracts and pinned schema converter that the runtime serves.
const compiledRoot = resolve(
dirname(fileURLToPath(import.meta.url)),
"../dist",
);
const { createCatalogDigests } = await import(
pathToFileURL(join(compiledRoot, "catalogIdentity.js")).href
);
await writeFile(
join(compiledRoot, "recordedCatalogDigests.js"),
`/** Recorded from the compiled default catalog by scripts/record-catalog-digests.mjs; do not edit. */\nexport const RECORDED_CATALOG_DIGESTS = Object.freeze(${JSON.stringify(createCatalogDigests())});\n`,
"utf8",
);
+4 -27
View File
@@ -76,30 +76,13 @@ function runCommand() {
env: process.env,
stdio: "inherit",
});
let spawned = false;
let childError;
let hasChildError = false;
child.once("spawn", () => {
spawned = true;
});
const rememberChildError = (cause) => {
if (!hasChildError) {
hasChildError = true;
childError = cause;
}
if (!spawned) {
reject(childError);
}
};
child.on("error", rememberChildError);
child.once("error", reject);
child.once("close", (code) => {
child.off("error", rememberChildError);
// During POSIX cancellation the group leader must stay alive, retaining
// both its group identity and SQLite ownership until the entire group is
// killed. A completed command can still have uncooperative descendants.
if (shutdownSignal !== undefined && process.platform !== "win32") return;
if (hasChildError) reject(childError);
else resolveResult({ code });
if (shutdownSignal === undefined || process.platform === "win32")
resolveResult({ code });
});
});
}
@@ -116,13 +99,7 @@ function handleDisconnect() {
function handleSignal(signal) {
if (shutdownSignal !== undefined) return;
shutdownSignal = signal;
if (child === undefined) return;
if (process.platform === "win32") {
// Windows has no POSIX process groups. Stop only the command owned by this
// supervisor, and keep the SQLite transaction until its close event.
child.kill(signal);
return;
}
if (child === undefined || process.platform === "win32") return;
// Establish cancellation before signalling the group. This supervisor is
// its live leader, so the group identifier cannot be reused during cleanup.
process.kill(-process.pid, signal);
-41
View File
@@ -322,47 +322,6 @@ class WebsiteMaintenanceChecks(unittest.TestCase):
self.assertTrue(any("missing Markdown fragment" in error for error in errors))
self.assertTrue(any("missing repository target" in error for error in errors))
def test_repository_links_accept_aliased_roots_and_reject_escaping_links(self):
with tempfile.TemporaryDirectory() as directory:
root = (Path(directory) / "repository").resolve()
(root / "website/public").mkdir(parents=True)
(root / "docs").mkdir()
(root / "docs/guide.md").write_text("# Guide\n")
page = root / "website/README.md"
page.write_text("[Guide](../docs/guide.md#guide)\n")
alias = Path(directory) / "repository-alias"
alias.symlink_to(root, target_is_directory=True)
self.assertEqual(VERIFY.check_repository_links(root), [])
self.assertEqual(VERIFY.check_repository_links(alias), [])
(root.parent / "outside.md").write_text("# Outside\n")
page.write_text("[Outside](../../outside.md)\n")
for boundary in (root, alias):
with self.subTest(boundary=boundary):
self.assertEqual(VERIFY.check_repository_links(boundary), [
"website/README.md: missing repository target: ../../outside.md"
])
def test_site_links_preserve_containment_through_aliased_roots(self):
with tempfile.TemporaryDirectory() as directory:
site = (Path(directory) / "site").resolve()
site.mkdir()
page = site / "index.html"
page.write_text('<html lang="en"><head><title>Fixture</title><meta name="robots" content="noindex"></head><body><h1>Fixture</h1><a href="guide.html#answer">Guide</a></body></html>')
(site / "guide.html").write_text('<html lang="en"><head><title>Guide</title><meta name="robots" content="noindex"></head><body><h1 id="answer">Answer</h1></body></html>')
(site / "sitemap.xml").write_text('<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"/>')
(site / "robots.txt").write_text("User-agent: *\nAllow: /\nSitemap: https://rea.tools/sitemap.xml\n")
alias = Path(directory) / "site-alias"
alias.symlink_to(site, target_is_directory=True)
# The tiny link fixture intentionally omits the social preview.
expected = ["Social preview PNG is missing; run scripts/prepare-website.py"]
self.assertEqual(VERIFY.check_site(site)[3], expected)
self.assertEqual(VERIFY.check_site(alias)[3], expected)
(site.parent / "outside.html").write_text("<p>Outside</p>")
page.write_text(page.read_text().replace("guide.html#answer", "../outside.html"))
for boundary in (site, alias):
with self.subTest(boundary=boundary):
self.assertIn("index.html: reference leaves the website: ../outside.html", VERIFY.check_site(boundary)[3])
def test_preview_preserves_pages_prefix_for_assets_and_directory_redirects(self):
with tempfile.TemporaryDirectory() as directory:
site = Path(directory)
+305 -328
View File
@@ -5,7 +5,6 @@ import { join } from "node:path";
import { AdbDeviceAnalysisService } from "../dist/application/adb/AdbDeviceAnalysisService.js";
import { createAdbDeviceAnalysisProvider } from "../dist/composition/adb.js";
import { closeVerifierResources } from "./lib/verifier-cleanup.mjs";
const options = parseArguments(process.argv.slice(2));
if (options.help) {
@@ -40,365 +39,343 @@ const run = async (operation, input) => {
return result.value.normalized_result;
};
let primaryFailure;
try {
const client = await run("inspect_adb_client", {});
if (client !== null) {
const client = await run("inspect_adb_client", {});
if (client !== null) {
check(
"adb client reports a version",
typeof client.client.version === "string" && client.client.version !== "",
`binary=${client.client.binary_path} version=${client.client.version}`,
);
}
const listing = await run("list_adb_devices", {});
let serial = options.serial ?? null;
if (listing !== null) {
check(
"device listing parsed",
Array.isArray(listing.devices),
`${listing.devices.length} device(s)`,
);
const usable = listing.devices.filter((device) => device.state === "device");
if (serial === null && usable.length > 0) serial = usable[0].serial;
if (serial === null) {
process.stdout.write(
"[FAIL] no usable device is attached; start an emulator or connect a phone\n",
);
failures += 1;
} else {
const selected = listing.devices.find((device) => device.serial === serial);
check(
"adb client reports a version",
typeof client.client.version === "string" && client.client.version !== "",
`binary=${client.client.binary_path} version=${client.client.version}`,
"selected device present with a kind inference",
selected !== undefined &&
["emulator", "physical", "unknown"].includes(selected.kind),
`serial=${serial} kind=${selected?.kind} basis=${selected?.kind_basis}`,
);
}
}
if (serial !== null) {
const device = await run("inspect_adb_device", { serial });
if (device !== null) {
check(
"device build identity projected",
device.properties.some(
(property) => property.name === "ro.build.version.sdk",
),
`sdk=${
device.properties.find((p) => p.name === "ro.build.version.sdk")?.value
} emulator_observed=${String(device.emulator_observed)}`,
);
}
const listing = await run("list_adb_devices", {});
let serial = options.serial ?? null;
if (listing !== null) {
// Observation tools: processes, window, display, settings, logcat.
const processes = await run("list_adb_processes", { serial });
if (processes !== null) {
check(
"device listing parsed",
Array.isArray(listing.devices),
`${listing.devices.length} device(s)`,
"process listing is non-empty with a header",
processes.processes.length > 0 && typeof processes.columns === "string",
`${processes.processes.length} process(es)`,
);
const usable = listing.devices.filter(
(device) => device.state === "device",
}
const window = await run("inspect_adb_window", { serial });
if (window !== null)
check(
"window focus observation completed",
typeof window.focused_window === "string" ||
window.focused_window === null,
`focused_window=${String(window.focused_window).slice(0, 60)} (null is legitimate on headless devices)`,
);
if (serial === null && usable.length > 0) serial = usable[0].serial;
if (serial === null) {
process.stdout.write(
"[FAIL] no usable device is attached; start an emulator or connect a phone\n",
);
failures += 1;
} else {
const selected = listing.devices.find(
(device) => device.serial === serial,
);
const display = await run("inspect_adb_display", { serial });
if (display !== null)
check(
"display reports a physical size",
display.physical_size !== null &&
display.physical_size.width > 0 &&
display.physical_size.height > 0,
`${display.physical_size?.width}x${display.physical_size?.height} @${String(display.physical_density)}`,
);
const setting = await run("read_adb_setting", {
serial,
namespace: "global",
key: "adb_enabled",
});
if (setting !== null)
check(
"settings read returns the device's own value",
setting.value === "1",
`adb_enabled=${String(setting.value)}`,
);
const logcat = await run("read_adb_logcat", {
serial,
count: 20,
buffer: "main",
});
if (logcat !== null)
check(
"logcat dump returns bounded lines",
logcat.lines.length <= 20,
`${logcat.lines.length} line(s), coverage=${logcat.coverage}`,
);
const features = await run("list_adb_features", { serial });
if (features !== null)
check(
"feature inventory is non-empty",
features.coverage === "complete" && features.features.length > 0,
`${features.features.length} feature(s)`,
);
const services = await run("list_adb_services", { serial });
if (services !== null)
check(
"service inventory is non-empty",
services.coverage === "complete" && services.services.length > 0,
`${services.services.length} service(s)`,
);
const directory = await run("list_adb_directory", {
serial,
device_path: "/data/local/tmp",
});
if (directory !== null)
check(
"directory listing completed",
directory.coverage === "complete",
`${directory.entries.length} entr(ies)`,
);
const packages = await run("list_adb_packages", {
serial,
scope: "third_party",
});
let pullCandidate = options.pull ?? null;
if (packages !== null) {
// A third-party scope may legitimately be empty (clean emulator images);
// completeness is about unparsed lines, not row count.
check(
"third-party package listing reports complete coverage",
packages.coverage === "complete",
`${packages.packages.length} package(s)`,
);
const allPackages = await run("list_adb_packages", {
serial,
scope: "all",
});
if (allPackages !== null) {
check(
"selected device present with a kind inference",
selected !== undefined &&
["emulator", "physical", "unknown"].includes(selected.kind),
`serial=${serial} kind=${selected?.kind} basis=${selected?.kind_basis}`,
"full package inventory is non-empty",
allPackages.coverage === "complete" && allPackages.packages.length > 0,
`${allPackages.packages.length} package(s)`,
);
}
if (pullCandidate === null && packages.packages.length > 0)
pullCandidate = packages.packages[0].package_name;
}
if (serial !== null) {
const device = await run("inspect_adb_device", { serial });
if (device !== null) {
check(
"device build identity projected",
device.properties.some(
(property) => property.name === "ro.build.version.sdk",
),
`sdk=${
device.properties.find((p) => p.name === "ro.build.version.sdk")
?.value
} emulator_observed=${String(device.emulator_observed)}`,
);
}
// Observation tools: processes, window, display, settings, logcat.
const processes = await run("list_adb_processes", { serial });
if (processes !== null) {
check(
"process listing is non-empty with a header",
processes.processes.length > 0 && typeof processes.columns === "string",
`${processes.processes.length} process(es)`,
);
}
const window = await run("inspect_adb_window", { serial });
if (window !== null)
check(
"window focus observation completed",
typeof window.focused_window === "string" ||
window.focused_window === null,
`focused_window=${String(window.focused_window).slice(0, 60)} (null is legitimate on headless devices)`,
);
const display = await run("inspect_adb_display", { serial });
if (display !== null)
check(
"display reports a physical size",
display.physical_size !== null &&
display.physical_size.width > 0 &&
display.physical_size.height > 0,
`${display.physical_size?.width}x${display.physical_size?.height} @${String(display.physical_density)}`,
);
const setting = await run("read_adb_setting", {
serial,
namespace: "global",
key: "adb_enabled",
});
if (setting !== null)
check(
"settings read returns the device's own value",
setting.value === "1",
`adb_enabled=${String(setting.value)}`,
);
const logcat = await run("read_adb_logcat", {
serial,
count: 20,
buffer: "main",
});
if (logcat !== null)
check(
"logcat dump returns bounded lines",
logcat.lines.length <= 20,
`${logcat.lines.length} line(s), coverage=${logcat.coverage}`,
);
const features = await run("list_adb_features", { serial });
if (features !== null)
check(
"feature inventory is non-empty",
features.coverage === "complete" && features.features.length > 0,
`${features.features.length} feature(s)`,
);
const services = await run("list_adb_services", { serial });
if (services !== null)
check(
"service inventory is non-empty",
services.coverage === "complete" && services.services.length > 0,
`${services.services.length} service(s)`,
);
const directory = await run("list_adb_directory", {
serial,
device_path: "/data/local/tmp",
});
if (directory !== null)
check(
"directory listing completed",
directory.coverage === "complete",
`${directory.entries.length} entr(ies)`,
);
const packages = await run("list_adb_packages", {
serial,
scope: "third_party",
});
let pullCandidate = options.pull ?? null;
if (packages !== null) {
// A third-party scope may legitimately be empty (clean emulator images);
// completeness is about unparsed lines, not row count.
check(
"third-party package listing reports complete coverage",
packages.coverage === "complete",
`${packages.packages.length} package(s)`,
);
const allPackages = await run("list_adb_packages", {
serial,
scope: "all",
});
if (allPackages !== null) {
check(
"full package inventory is non-empty",
allPackages.coverage === "complete" &&
allPackages.packages.length > 0,
`${allPackages.packages.length} package(s)`,
);
}
if (pullCandidate === null && packages.packages.length > 0)
pullCandidate = packages.packages[0].package_name;
}
if (pullCandidate !== null) {
const directory = await mkdtemp(join(tmpdir(), "rea-adb-verify-"));
try {
const pull = await run("pull_adb_package", {
serial,
package: pullCandidate,
output_directory: directory,
});
if (pull !== null) {
// Installed /data/app packages name their APKs base.apk and
// split_*.apk; system packages keep arbitrary names, so the lane
// checks a non-empty artifact set and verifies each file on disk.
check(
"pull returned a non-empty artifact set",
pull.artifacts.length > 0,
`${pull.artifacts.length} artifact(s), coverage=${pull.coverage}, roles=${pull.artifacts.map((a) => a.role).join(",")}`,
);
for (const artifact of pull.artifacts) {
const bytes = await stat(artifact.local_path).then(
(info) => info.size,
);
const digest = createHash("sha256")
.update(await readFile(artifact.local_path))
.digest("hex");
check(
`artifact digest matches on disk: ${artifact.file_name}`,
bytes === artifact.bytes && digest === artifact.sha256,
`${bytes} bytes`,
);
}
}
} finally {
await rm(directory, { recursive: true, force: true });
}
}
// File transfer: push one lane-owned file, verify both digests, pull it
// back, then clean up the device copy through the same fixed argv surface.
const transferRoot = await mkdtemp(join(tmpdir(), "rea-adb-transfer-"));
if (pullCandidate !== null) {
const directory = await mkdtemp(join(tmpdir(), "rea-adb-verify-"));
try {
const payload = join(transferRoot, "rea-verify.txt");
const payloadBytes = "rea adb transfer verification\n";
await import("node:fs/promises").then((fs) =>
fs.writeFile(payload, payloadBytes),
);
const deviceTarget = `/data/local/tmp/rea-verify-${Date.now()}.txt`;
const pushed = await run("push_adb_file", {
const pull = await run("pull_adb_package", {
serial,
local_path: payload,
device_path: deviceTarget,
overwrite: false,
package: pullCandidate,
output_directory: directory,
});
if (pushed !== null) {
if (pull !== null) {
// Installed /data/app packages name their APKs base.apk and
// split_*.apk; system packages keep arbitrary names, so the lane
// checks a non-empty artifact set and verifies each file on disk.
check(
"push reports matching device-side digest",
pushed.device_digest_source === "device_sha256sum" &&
pushed.device_sha256 === pushed.sha256,
`${pushed.bytes} bytes`,
"pull returned a non-empty artifact set",
pull.artifacts.length > 0,
`${pull.artifacts.length} artifact(s), coverage=${pull.coverage}, roles=${pull.artifacts.map((a) => a.role).join(",")}`,
);
const pulled = await run("pull_adb_file", {
serial,
device_path: deviceTarget,
output_directory: transferRoot,
});
if (pulled !== null)
check(
"pulled file digest matches the pushed digest",
pulled.sha256 === pushed.sha256,
`${pulled.bytes} bytes`,
for (const artifact of pull.artifacts) {
const bytes = await stat(artifact.local_path).then(
(info) => info.size,
);
}
const screenRoot = await mkdtemp(join(tmpdir(), "rea-adb-screen-"));
try {
const screen = await run("capture_adb_screen", {
serial,
output_directory: screenRoot,
});
if (screen !== null)
const digest = createHash("sha256")
.update(await readFile(artifact.local_path))
.digest("hex");
check(
"screen capture is a sized PNG",
screen.width !== null && screen.height !== null && screen.bytes > 0,
`${screen.width}x${screen.height}, ${screen.bytes} bytes`,
`artifact digest matches on disk: ${artifact.file_name}`,
bytes === artifact.bytes && digest === artifact.sha256,
`${bytes} bytes`,
);
} finally {
await rm(screenRoot, { recursive: true, force: true });
}
if (options.pull !== null) {
const details = await run("inspect_adb_package", {
serial,
package: options.pull,
});
if (details !== null)
check(
"dumpsys package projection completed",
details.coverage === "complete",
`versionCode=${String(details.version_code)}`,
);
}
if (options.installApk !== null && options.installApk !== undefined) {
const probePackage = "com.rea.adb.probe";
const installedProbe = await run("install_adb_package", {
serial,
apk_path: options.installApk,
replace: false,
});
if (installedProbe !== null) {
check(
"install reports the APK digest",
typeof installedProbe.sha256 === "string" &&
installedProbe.sha256.length === 64,
`${installedProbe.bytes} bytes`,
);
const resolved = await run("resolve_adb_packages", {
serial,
query: "rea.adb.probe",
});
if (resolved !== null)
check(
"resolve finds the installed probe uniquely",
resolved.matches.length === 1 &&
resolved.matches[0]?.package_name === probePackage,
`${resolved.matches.length} match(es)`,
);
const started = await run("start_adb_app", {
serial,
package: probePackage,
});
if (started !== null)
check(
"app start resolves and launches the launcher activity",
typeof started.component === "string" &&
started.component.startsWith(probePackage),
String(started.component),
);
const observedProcesses = await run("list_adb_processes", { serial });
if (observedProcesses !== null)
check(
"started app appears in the process listing",
observedProcesses.processes.some(
(process) => process.name === probePackage,
),
);
const stopped = await run("stop_adb_app", {
serial,
package: probePackage,
});
if (stopped !== null)
check(
"force-stop completes the dynamic-analysis run",
stopped.package_name === probePackage,
);
const removed = await run("uninstall_adb_package", {
serial,
package: probePackage,
});
if (removed !== null)
check(
"uninstall completes the lifecycle",
removed.package_name === probePackage,
);
const afterRemoval = await run("resolve_adb_packages", {
serial,
query: "rea.adb.probe",
});
if (afterRemoval !== null)
check(
"resolve reports zero matches after removal",
afterRemoval.matches.length === 0,
);
}
}
} finally {
await rm(transferRoot, { recursive: true, force: true });
await rm(directory, { recursive: true, force: true });
}
}
} catch (cause) {
primaryFailure = cause;
// File transfer: push one lane-owned file, verify both digests, pull it
// back, then clean up the device copy through the same fixed argv surface.
const transferRoot = await mkdtemp(join(tmpdir(), "rea-adb-transfer-"));
try {
const payload = join(transferRoot, "rea-verify.txt");
const payloadBytes = "rea adb transfer verification\n";
await import("node:fs/promises").then((fs) =>
fs.writeFile(payload, payloadBytes),
);
const deviceTarget = `/data/local/tmp/rea-verify-${Date.now()}.txt`;
const pushed = await run("push_adb_file", {
serial,
local_path: payload,
device_path: deviceTarget,
overwrite: false,
});
if (pushed !== null) {
check(
"push reports matching device-side digest",
pushed.device_digest_source === "device_sha256sum" &&
pushed.device_sha256 === pushed.sha256,
`${pushed.bytes} bytes`,
);
const pulled = await run("pull_adb_file", {
serial,
device_path: deviceTarget,
output_directory: transferRoot,
});
if (pulled !== null)
check(
"pulled file digest matches the pushed digest",
pulled.sha256 === pushed.sha256,
`${pulled.bytes} bytes`,
);
}
const screenRoot = await mkdtemp(join(tmpdir(), "rea-adb-screen-"));
try {
const screen = await run("capture_adb_screen", {
serial,
output_directory: screenRoot,
});
if (screen !== null)
check(
"screen capture is a sized PNG",
screen.width !== null && screen.height !== null && screen.bytes > 0,
`${screen.width}x${screen.height}, ${screen.bytes} bytes`,
);
} finally {
await rm(screenRoot, { recursive: true, force: true });
}
if (options.pull !== null) {
const details = await run("inspect_adb_package", {
serial,
package: options.pull,
});
if (details !== null)
check(
"dumpsys package projection completed",
details.coverage === "complete",
`versionCode=${String(details.version_code)}`,
);
}
if (options.installApk !== null && options.installApk !== undefined) {
const probePackage = "com.rea.adb.probe";
const installedProbe = await run("install_adb_package", {
serial,
apk_path: options.installApk,
replace: false,
});
if (installedProbe !== null) {
check(
"install reports the APK digest",
typeof installedProbe.sha256 === "string" &&
installedProbe.sha256.length === 64,
`${installedProbe.bytes} bytes`,
);
const resolved = await run("resolve_adb_packages", {
serial,
query: "rea.adb.probe",
});
if (resolved !== null)
check(
"resolve finds the installed probe uniquely",
resolved.matches.length === 1 &&
resolved.matches[0]?.package_name === probePackage,
`${resolved.matches.length} match(es)`,
);
const started = await run("start_adb_app", {
serial,
package: probePackage,
});
if (started !== null)
check(
"app start resolves and launches the launcher activity",
typeof started.component === "string" &&
started.component.startsWith(probePackage),
String(started.component),
);
const observedProcesses = await run("list_adb_processes", { serial });
if (observedProcesses !== null)
check(
"started app appears in the process listing",
observedProcesses.processes.some(
(process) => process.name === probePackage,
),
);
const stopped = await run("stop_adb_app", {
serial,
package: probePackage,
});
if (stopped !== null)
check(
"force-stop completes the dynamic-analysis run",
stopped.package_name === probePackage,
);
const removed = await run("uninstall_adb_package", {
serial,
package: probePackage,
});
if (removed !== null)
check(
"uninstall completes the lifecycle",
removed.package_name === probePackage,
);
const afterRemoval = await run("resolve_adb_packages", {
serial,
query: "rea.adb.probe",
});
if (afterRemoval !== null)
check(
"resolve reports zero matches after removal",
afterRemoval.matches.length === 0,
);
}
}
} finally {
await rm(transferRoot, { recursive: true, force: true });
}
}
let cleanupFailure;
try {
await closeVerifierResources([() => provider.close()], primaryFailure);
} catch (cause) {
cleanupFailure = cause;
await provider.close();
} catch {
// The provider owns nothing long-lived; closing is best-effort.
}
if (primaryFailure !== undefined || cleanupFailure !== undefined) {
const causes =
cleanupFailure instanceof AggregateError
? cleanupFailure.errors
: [cleanupFailure ?? primaryFailure];
process.stderr.write(
`ADB device verification failed: ${causes.map((cause) => (cause instanceof Error ? cause.message : String(cause))).join("; ")}\n`,
);
process.exitCode = 1;
} else if (failures > 0) {
if (failures > 0) {
process.stdout.write(`\n${failures} check(s) failed\n`);
process.exitCode = 1;
} else {
process.stdout.write("\nall adb device checks passed\n");
process.exit(1);
}
process.stdout.write("\nall adb device checks passed\n");
function parseArguments(argv) {
const parsed = { help: false, serial: null, pull: null, installApk: null };
+14 -41
View File
@@ -1,6 +1,6 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import { mkdtemp, readdir, readFile, rmdir } from "node:fs/promises";
import { mkdtemp, readdir, readFile, rm } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { promisify } from "node:util";
@@ -8,9 +8,7 @@ import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { toolContract } from "../dist/contracts/toolContracts.js";
import { hashStableFile } from "../dist/filesystem/StableFileHash.js";
import { OwnedRegularFileCleanupFailure } from "../dist/filesystem/RegularFile.js";
import { mcpTextValue } from "./lib/mcp-verifier-results.mjs";
import { closeVerifierResources } from "./lib/verifier-cleanup.mjs";
const options = parseArguments(process.argv.slice(2));
if (options.help) {
@@ -41,17 +39,6 @@ Object.assign(environment, {
TEMP: root,
});
const client = new Client({ name: "flutter-verifier", version: "1" });
let primaryError;
const hashOwners = new Map();
const hashOwned = async (...args) => {
try {
return await hashStableFile(...args);
} catch (cause) {
if (cause instanceof OwnedRegularFileCleanupFailure)
hashOwners.set(cause.owner, cause.path);
throw cause;
}
};
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
@@ -62,7 +49,7 @@ const execute = promisify(execFile);
try {
await client.connect(transport);
const apk = resolve(options.apk);
const identity = await hashOwned(apk);
const identity = await hashStableFile(apk);
const identified = await run("identify_flutter_build", { path: apk });
assert.deepEqual(identified.target, { path: apk, ...identity });
assert.equal(identified.flutter_detected, true);
@@ -189,46 +176,32 @@ try {
const negative = await run("identify_flutter_build", { path: plain });
assert.deepEqual(negative.target, {
path: plain,
...(await hashOwned(plain)),
...(await hashStableFile(plain)),
});
assert.equal(negative.flutter_detected, false);
assert.equal(negative.coverage, "complete");
}
assert.deepEqual(
await hashOwned(apk),
await hashStableFile(apk),
identity,
"Verification must preserve its input",
);
} catch (cause) {
primaryError = cause;
throw cause;
} finally {
const cleanup = [];
for (const [owner, path] of hashOwners) {
cleanup.push(async () => {
try {
await owner.close();
hashOwners.delete(owner);
} catch (cause) {
throw new Error(`Could not close hash file handle for ${path}`, {
cause,
});
}
});
}
cleanup.push(async () => {
await closeVerifierResources([
() => client.close(),
() => transport.close(),
]);
// Server shutdown retries retained provider cleanup and must settle before success.
try {
try {
await client.close();
} finally {
await transport.close();
}
assert.deepEqual(
await readdir(root),
[],
`Provider snapshots remain under ${root}`,
);
await rmdir(root);
});
await closeVerifierResources(cleanup, primaryError);
} finally {
await rm(root, { recursive: true, force: true });
}
}
process.stdout.write(
"all asserted Flutter CLI/MCP checks passed; unverified facets are reported above\n",
-62
View File
@@ -5,11 +5,6 @@ import { tmpdir } from "node:os";
import { join, resolve, sep } from "node:path";
import { fileURLToPath } from "node:url";
import Ajv2020 from "ajv/dist/2020.js";
import {
schemaArrays,
schemaChildren,
schemaMaps,
} from "../dist/contracts/inputSchemaPresentation.js";
import {
TOOL_CONTRACTS,
toolContract,
@@ -22,8 +17,6 @@ const repo = fileURLToPath(new URL("..", import.meta.url));
const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
const command = process.env.REA_VERIFY_GEMINI_COMMAND ?? "gemini";
const mode = process.argv[2] ?? "call";
const schemaProfile = process.env.REA_VERIFY_GEMINI_SCHEMA_PROFILE ?? "full";
assert(["full", "compact"].includes(schemaProfile));
// npm installs of Gemini CLI provide a JavaScript entry point; on Windows only a gemini.cmd shim reaches
// PATH, which Node does not spawn without a shell, so a selected .js entry runs under this Node. It is
// resolved once, because the preflight runs from the caller's directory and the client from the workspace.
@@ -67,7 +60,6 @@ const environment = {
GEMINI_CLI_NO_RELAUNCH: "1",
GEMINI_CLI_TRUST_WORKSPACE: "true",
REA_PROCESS_RUN_ID: verifier.run_id,
REA_MCP_INPUT_SCHEMA_PROFILE: schemaProfile,
};
let clientVersion;
try {
@@ -169,43 +161,8 @@ let skillRequested = false;
let finalSeen = false;
let catalogSize = 0;
let schemasChecked = 0;
let examplesChecked = 0;
let artifactDigest;
const schemaValidator = new Ajv2020({ strict: false, validateFormats: false });
const booleanLiteralPath = (schema, path = "$", seen = new Set()) => {
if (!schema || typeof schema !== "object" || seen.has(schema))
return undefined;
seen.add(schema);
for (const [key, child] of Object.entries(schema)) {
const childPath = `${path}.${key}`;
if (key === "const" && typeof child === "boolean") return childPath;
if (
key === "enum" &&
Array.isArray(child) &&
child.some((item) => typeof item === "boolean")
)
return childPath;
const schemaChildrenAtKey =
schemaMaps.has(key) &&
child &&
typeof child === "object" &&
!Array.isArray(child)
? Object.entries(child).map(([name, item]) => [
`${childPath}.${name}`,
item,
])
: schemaArrays.has(key) && Array.isArray(child)
? child.map((item, index) => [`${childPath}[${index}]`, item])
: schemaChildren.has(key)
? [[childPath, child]]
: [];
for (const [nestedPath, nested] of schemaChildrenAtKey) {
const found = booleanLiteralPath(nested, nestedPath, seen);
if (found !== undefined) return found;
}
}
return undefined;
};
const extractEvidence = (value) => {
if (typeof value === "string") {
if (
@@ -269,22 +226,6 @@ const fixture = await createGeminiModelFixture({
schemaValidator.validateSchema(tool.parametersJsonSchema),
`${tool.name}: invalid advertised JSON Schema: ${JSON.stringify(schemaValidator.errors)}`,
);
const booleanLiteral = booleanLiteralPath(tool.parametersJsonSchema);
assert.equal(
booleanLiteral,
undefined,
`${tool.name}: Gemini SDK declaration retains a boolean const/enum at ${booleanLiteral}`,
);
const name = tool.name.slice(prefix.length);
const contract = toolContract(name);
const validate = schemaValidator.compile(tool.parametersJsonSchema);
for (const example of contract.examples) {
assert(
validate(example.input),
`${tool.name}: Gemini SDK declaration rejects canonical example ${example.title}: ${JSON.stringify(schemaValidator.errors)}`,
);
examplesChecked++;
}
schemasChecked++;
}
}
@@ -415,9 +356,6 @@ try {
client: "Gemini CLI",
clientVersion,
mode,
schemaProfile,
schemasChecked,
examplesChecked,
passed: failure === undefined,
artifacts: lab,
runtimeRoot,
-182
View File
@@ -1,182 +0,0 @@
#!/usr/bin/env node
// Compile the packaged Ghidra bridge sources against an existing Ghidra
// installation's jars. Ghidra's script provider compiles the whole bridge
// directory when a session starts, so one unresolved symbol disables every
// operation and surfaces only after import and auto-analysis. This check
// reports the same javac diagnostics without launching Ghidra or selecting,
// importing or executing a target.
import { spawnSync } from "node:child_process";
import { mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join, relative, resolve } from "node:path";
import { fileURLToPath } from "node:url";
// The pinned Node toolchain strips types from these dependency-free owners;
// this preflight therefore uses runtime facts without needing npm ci or a build.
import { GHIDRA_BRIDGE_SOURCES } from "../src/ghidra/GhidraBridgeSources.ts";
import { parseGhidraJavaProperties } from "../src/ghidra/GhidraJavaProperties.ts";
import {
SUPPORTED_GHIDRA_VERSION,
SUPPORTED_GHIDRA_JAVA_MAJOR,
ghidraReleaseAccepted,
ghidraReleaseLine,
} from "../src/ghidra/GhidraReleasePolicy.ts";
const args = process.argv.slice(2);
if (
args.length > 1 ||
(args.length === 1 &&
!["--check-sdk-pin", "--verified-sdk"].includes(args[0]))
)
fail(
"Usage: verify-ghidra-bridge-sources.mjs [--check-sdk-pin|--verified-sdk]",
);
if (args[0] === "--check-sdk-pin") {
let url;
try {
url = new URL(process.env.GHIDRA_RELEASE_URL);
} catch {
fail("GHIDRA_RELEASE_URL must name the checksum-pinned official release");
}
const prefix = `/NationalSecurityAgency/ghidra/releases/download/Ghidra_${SUPPORTED_GHIDRA_VERSION}_build/ghidra_${SUPPORTED_GHIDRA_VERSION}_PUBLIC_`;
if (
url.origin !== "https://github.com" ||
url.username ||
url.password ||
url.search ||
url.hash ||
!url.pathname.startsWith(prefix) ||
!url.pathname.endsWith(".zip") ||
!/^[a-f0-9]{64}$/iu.test(process.env.GHIDRA_RELEASE_SHA256 ?? "")
)
fail(
`SDK URL and SHA256 must match verified Ghidra ${SUPPORTED_GHIDRA_VERSION}`,
);
console.log(`GHIDRA_JAVA_MAJOR=${SUPPORTED_GHIDRA_JAVA_MAJOR}`);
process.exit(0);
}
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const bridge = join(root, "bridge/ghidra");
const install = process.env.GHIDRA_INSTALL_DIR;
if (!install)
fail(
`Set GHIDRA_INSTALL_DIR to an existing Ghidra ${ghidraReleaseLine()} installation (verified with ${SUPPORTED_GHIDRA_VERSION}). REA does not install Ghidra.`,
);
const properties = await readFile(
join(install, "Ghidra/application.properties"),
"latin1",
).catch((cause) =>
fail(
`GHIDRA_INSTALL_DIR has no readable Ghidra/application.properties: ${cause.message}`,
),
);
const parsedProperties = parseGhidraJavaProperties(properties);
if (!parsedProperties.ok) fail(parsedProperties.error);
const version = parsedProperties.value.get("application.version");
if (!ghidraReleaseAccepted(version))
fail(
`Bridge sources require Ghidra ${ghidraReleaseLine()} (verified with ${SUPPORTED_GHIDRA_VERSION}). Observed application.version=${version === undefined ? "missing" : JSON.stringify(version)}.`,
);
if (args[0] === "--verified-sdk" && version !== SUPPORTED_GHIDRA_VERSION)
fail(
`CI SDK must be verified Ghidra ${SUPPORTED_GHIDRA_VERSION}; observed ${JSON.stringify(version)}`,
);
const sourceNames = (await readdir(bridge, { withFileTypes: true }))
.filter((entry) => entry.isFile() && entry.name.endsWith(".java"))
.map((entry) => entry.name);
for (const name of Object.values(GHIDRA_BRIDGE_SOURCES))
if (!sourceNames.includes(name))
fail(`Required Ghidra bridge source is missing: bridge/ghidra/${name}`);
const requiredNames = new Set(Object.values(GHIDRA_BRIDGE_SOURCES));
const unregistered = sourceNames.filter((name) => !requiredNames.has(name));
if (unregistered.length)
fail(
`Ghidra bridge sources absent from the manifest: ${unregistered.join(", ")}`,
);
const sources = sourceNames
.map((name) => relative(root, join(bridge, name)))
.sort();
const javac = process.env.JAVA_HOME
? join(process.env.JAVA_HOME, "bin", "javac")
: "javac";
const banner = run(javac, ["-version"]);
if (banner.error || banner.status !== 0)
fail(`javac is unavailable: ${banner.error?.message ?? output(banner)}`);
const major = Number(/^javac (\d+)/mu.exec(output(banner))?.[1]);
if (!Number.isInteger(major) || major < SUPPORTED_GHIDRA_JAVA_MAJOR)
fail(
`Ghidra ${ghidraReleaseLine()} compiles bridge scripts with Java ${SUPPORTED_GHIDRA_JAVA_MAJOR} or newer. Observed: ${output(banner) || "no version"}. REA does not install or upgrade Java.`,
);
const jars = (
await readdir(join(install, "Ghidra"), {
recursive: true,
withFileTypes: true,
})
)
.filter((entry) => entry.isFile() && entry.name.endsWith(".jar"))
.map((entry) => join(entry.parentPath, entry.name))
.sort();
if (jars.length === 0) fail(`No jars found under ${join(install, "Ghidra")}.`);
const scratch = await mkdtemp(join(tmpdir(), "rea-ghidra-bridge-"));
try {
// An argument file keeps the jar class path below Windows' command length.
const argumentFile = join(scratch, "javac.args");
await writeFile(
argumentFile,
[
"-proc:none",
"-encoding",
"UTF-8",
"--release",
String(SUPPORTED_GHIDRA_JAVA_MAJOR),
"-cp",
jars.join(process.platform === "win32" ? ";" : ":"),
"-d",
join(scratch, "classes"),
...sources,
]
.map((s) => `"${s.replaceAll("\\", "\\\\").replaceAll('"', '\\"')}"`)
.join("\n"),
);
// Sources are repository-relative, so diagnostics name bridge/ghidra/<file>:<line>.
const result = run(javac, ["-J-Xmx1g", `@${argumentFile}`]);
if (result.error || result.status !== 0) {
// Set the status instead of exiting so the scratch directory is removed.
process.stderr.write(
`${output(result) || result.error?.message || "javac failed"}\nGhidra ${version} could not compile the packaged bridge sources; the provider would fail at session start.\n`,
);
process.exitCode = 1;
} else {
console.log(
JSON.stringify({
ghidra_version: version,
javac: output(banner),
class_path_jars: jars.length,
sources,
}),
);
}
} finally {
await rm(scratch, { recursive: true, force: true });
}
function run(command, args) {
return spawnSync(command, args, {
cwd: root,
encoding: "utf8",
timeout: 300_000,
maxBuffer: 16 * 1024 * 1024,
});
}
function output(result) {
return `${result.stdout ?? ""}${result.stderr ?? ""}`.trim();
}
function fail(message) {
process.stderr.write(`${message}\n`);
process.exit(1);
}
Loaded 100 of 663 files, more files were not shown because too many files have changed in this diff. Show more