Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
279cca0828 | ||
|
|
bf86511477 | ||
|
|
500e5bb4be | ||
|
|
3e747586b4 | ||
|
|
b4bdd61ace | ||
|
|
fa8d6a7a23 | ||
|
|
ce4fe65170 |
No files matched your search
@@ -0,0 +1,154 @@
|
||||
# Atomic RE tools: an experiment
|
||||
|
||||
Experimental branch only: no PR, merge, production registration, or Code Mode.
|
||||
Started from REA main `5b16f2431062eea6ffd245a9e05512790007cb13`.
|
||||
|
||||
Read [measured results](results.md) and [mechanism / prior-art notes](research.md).
|
||||
The first round includes an object-reference counterexample and a failed
|
||||
code-mapping composition; it does not claim every hypothesis succeeded.
|
||||
The [control-flow / SSA follow-up](relations.md) repairs three zlib mapping
|
||||
paths, tests a SQLite call guard, and records where structured slices cost more.
|
||||
|
||||
The question is whether composable facts improve **completed reverse-engineering
|
||||
tasks**, especially on complex software. Three outcomes matter together:
|
||||
|
||||
1. Time, native work, CPU and process-family memory, including startup and retries.
|
||||
2. Whole-task estimated tokens: catalog, arguments, results and necessary follow-up.
|
||||
3. Correct answers with traceable evidence, meaningful coverage and explicit unknowns.
|
||||
|
||||
Smaller replies, fewer tool names and fewer calls are intermediate measurements,
|
||||
not proof of a better investigation. Negative results are useful experiment results.
|
||||
|
||||
## Working definition
|
||||
|
||||
An atom is a coherent fact or relationship about an identified object, with its
|
||||
producing evidence and uncertainty. It need not be a single field, MCP call,
|
||||
instruction, native command or engine pass. A string's encoding and the status of
|
||||
that encoding belong together; separating them can change the meaning.
|
||||
|
||||
Candidate reusable building blocks:
|
||||
|
||||
| Object / fact | Relationships and questions |
|
||||
| ------------------------------------ | --------------------------------------------------------------------------------------- |
|
||||
| Artifact and analysis context | Which bytes, architecture, address space and analysis profile produced this? |
|
||||
| Location and byte span | What is mapped here? Where are the source bytes? Is the span initialized? |
|
||||
| Instruction | What bytes decoded here, and with which mode and provenance? |
|
||||
| Function | Which entry is identified? Which locations does the engine actually assign to its body? |
|
||||
| String, symbol, type | What was observed or recovered, and what remains unknown? |
|
||||
| Reference / call / control-flow edge | What are both endpoints, the reference site, kind and coverage? |
|
||||
| Decompiled view | Which engine reconstruction and machine locations support this selected text? |
|
||||
|
||||
An inventory extent is an upper bound, not proof that an interior address belongs
|
||||
to the preceding function. A reference to a string's start address is not proof
|
||||
of a runtime string use, and misses references into its interior. Different
|
||||
providers' recovered functions and types are not automatically the same entities.
|
||||
|
||||
Identity needs artifact digest + analysis profile + address space + object kind
|
||||
and location/discriminator. A separate session lease controls whether a handle
|
||||
is still usable. Identity, existence, lookup success and readiness are distinct.
|
||||
|
||||
## Hypotheses to try to falsify
|
||||
|
||||
- Fine-grained getters can lose to coherent fact batches through repeated
|
||||
envelopes, schemas and round trips.
|
||||
- A small workflow can be built from the same facts without maintaining a second
|
||||
analysis implementation. Explicit selection must survive composition.
|
||||
- Batching at MCP may reduce agent turns without saving any native analysis.
|
||||
The engine's unit of work must be measured separately from the fact/read unit.
|
||||
- Richer structured relations can improve investigation completeness more than
|
||||
another wrapper around whole-function C text.
|
||||
- On complex targets, a narrow result can omit the decisive evidence. Compare
|
||||
coverage and final answers, counting all extra reads and failed paths.
|
||||
|
||||
## First prototype
|
||||
|
||||
`facts.mjs` consumes REA's existing normalized observations. It does not parse
|
||||
assembly/C or guess ownership. The same methods support individual reads,
|
||||
homogeneous batches, and one string-reference-function workflow. Native analysis,
|
||||
target admission, caches, cancellation and full Evidence ownership stay in REA.
|
||||
|
||||
`server.mjs` adds experimental tools to the **complete** production MCP catalog.
|
||||
It does not hide unrelated schemas to manufacture catalog savings. The additional
|
||||
schemas are included in costs. Its operations are static reads; it never executes
|
||||
the target or mutates an attached analysis database.
|
||||
|
||||
`benchmark.mjs` compares production calls, individual fact calls, batches and
|
||||
composition using actual MCP clients and actual Kuna/Ghidra. Verification runs
|
||||
are separate from timed equivalent-work runs. Each performance run repeats the
|
||||
task in one session to distinguish first queries and warm native caches. Cold
|
||||
Ghidra import/open cost must also be reported, not hidden in the query timer.
|
||||
|
||||
The prototype projects the current address-only `xrefs` contract. Reference kinds
|
||||
stay unknown. Unsupported containing-function resolution stays unknown without
|
||||
futile retries. This is intentionally a way to expose integration gaps rather
|
||||
than claim that the adapter can recover information already discarded upstream.
|
||||
Full Evidence is explicitly readable while its existing session owner retains
|
||||
it; closing the binary expires those records. Paging covers the provider-returned
|
||||
set and never establishes complete artifact discovery.
|
||||
|
||||
This is not a production contract proposal. Overlays, non-default address spaces,
|
||||
typed edges, native batching and live agent decision quality need further work.
|
||||
No measured claim about improved complex-program reasoning is established yet.
|
||||
|
||||
## Native experiments
|
||||
|
||||
After the build described below, with `REA_KUNA_COMMAND` configured:
|
||||
|
||||
```sh
|
||||
mkdir -p .cache/atomic-re-tools
|
||||
gcc -O2 -fno-pie -no-pie -o .cache/atomic-re-tools/interior-reference \
|
||||
experiments/atomic-re-tools/interior-reference.c
|
||||
|
||||
# Run once for each variant: per-address, then inventory.
|
||||
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
|
||||
node experiments/atomic-re-tools/native-relations.mjs \
|
||||
.cache/atomic-re-tools/interior-reference HEAD:REA per-address \
|
||||
.cache/atomic-re-tools/native-interior-per-address
|
||||
|
||||
python3 experiments/atomic-re-tools/verify-interior.py \
|
||||
.cache/atomic-re-tools/interior-reference \
|
||||
.cache/atomic-re-tools/native-interior-per-address.report.json \
|
||||
.cache/atomic-re-tools/native-interior-inventory.report.json
|
||||
```
|
||||
|
||||
Use the installed zlib/SQLite paths and `incorrect`/`malformed` to repeat the
|
||||
native plan comparison. Append `--code-slice` to the per-address experiment for
|
||||
an exact instruction-to-line probe. `mapped-code.mjs` rejects unsupported address
|
||||
precision and never substitutes a nearby line for a missing mapping.
|
||||
|
||||
The experiment adds an opt-in typed reference projection at the existing Kuna
|
||||
protocol owner. The existing public address-only result and its retained cache
|
||||
shape remain the same; ordinary queries do not materialize the richer facts.
|
||||
Experimental MCP tools have live-validated input schemas, but no canonical
|
||||
output-schema/product-catalog integration. They are not a production SDK/API.
|
||||
|
||||
The v2 facade labels reference scope explicitly and connects **locations** in its
|
||||
point-reference edges. A result's `target` preserves the caller's selector; it
|
||||
is not a separate observation proving that an object exists at that address.
|
||||
All experimental MCP results carry REA's canonical untrusted-target-content
|
||||
marker. The v1 measurement table is retained with its revision label; it is not
|
||||
relabelled as a measurement of this later response shape.
|
||||
|
||||
## Reproduce
|
||||
|
||||
Use the repository's pinned Node/npm and run `npm ci`, then
|
||||
`npm run build:cached`. Supply existing installations through `REA_KUNA_COMMAND`,
|
||||
`GHIDRA_INSTALL_DIR` and `JAVA_HOME`; this experiment installs no provider.
|
||||
|
||||
```sh
|
||||
node --test experiments/atomic-re-tools/facts.test.mjs
|
||||
|
||||
# Existing bounded runner: one CPU, 4 GiB hard memory cap, early stop at 3 GiB.
|
||||
# Its payload-exit counters include all processes in the owned scope.
|
||||
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
|
||||
node experiments/atomic-re-tools/benchmark.mjs \
|
||||
kuna batch /usr/lib/x86_64-linux-gnu/libz.so.1 incorrect \
|
||||
.cache/atomic-re-tools/kuna-batch
|
||||
```
|
||||
|
||||
Use `baseline`, `fine`, `batch`, or `workflow`, and `ghidra` or `kuna`.
|
||||
Append `--verify` for additional lifecycle, paging, negative and Evidence checks;
|
||||
those runs must not be used as equivalent-work performance samples. Binaries,
|
||||
provider projects, captured catalogs/traffic and local measurements stay ignored
|
||||
under `.cache/atomic-re-tools/`, never in commits. Summarize reproducible results
|
||||
and limitations here as experiments finish.
|
||||
@@ -0,0 +1,375 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { dirname, resolve } from "node:path";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import Ajv2020 from "ajv/dist/2020.js";
|
||||
import { requireMcpOperationResult } from "../../scripts/lib/mcp-verifier-results.mjs";
|
||||
import {
|
||||
createVerifierRun,
|
||||
completeVerifierRun,
|
||||
assertVerifierCleanupComplete,
|
||||
} from "../../scripts/lib/verifier-run.mjs";
|
||||
|
||||
const [provider, variant, binary, pattern, prefix, verification] =
|
||||
process.argv.slice(2);
|
||||
assert.ok(["ghidra", "kuna"].includes(provider));
|
||||
assert.ok(["baseline", "fine", "batch", "workflow"].includes(variant));
|
||||
assert.ok(binary && pattern && prefix);
|
||||
await mkdir(dirname(prefix), { recursive: true });
|
||||
const verifier = createVerifierRun();
|
||||
const client = new Client({ name: "rea-atomic-experiment", version: "0.1.0" });
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [
|
||||
resolve(
|
||||
variant === "baseline"
|
||||
? "dist/main.js"
|
||||
: "experiments/atomic-re-tools/server.mjs",
|
||||
),
|
||||
],
|
||||
env: {
|
||||
...process.env,
|
||||
REA_LOG_LEVEL: "error",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE:
|
||||
process.env.REA_MCP_OUTPUT_SCHEMA_PROFILE ?? "full",
|
||||
REA_KUNA_CACHE_DIR: resolve(`${prefix}.kuna-cache`),
|
||||
REA_KUNA_JOBS: "1",
|
||||
GHIDRA_HEADLESS_MAXMEM: "1G",
|
||||
REA_GHIDRA_STARTUP_TIMEOUT_MS: "120000",
|
||||
},
|
||||
stderr: "pipe",
|
||||
});
|
||||
let stderr = "";
|
||||
transport.stderr?.on("data", (part) => {
|
||||
stderr = (stderr + part).slice(-32_768);
|
||||
});
|
||||
const wire = [],
|
||||
calls = [],
|
||||
checks = [],
|
||||
failures = [];
|
||||
let phase = "lifecycle";
|
||||
const call = async (name, arguments_ = {}, expectError = false) => {
|
||||
const started = performance.now();
|
||||
const response = await client.callTool(
|
||||
{ name, arguments: arguments_ },
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
wire.push({ request: { name, arguments: arguments_ }, response, phase });
|
||||
if (name.startsWith("experiment_"))
|
||||
assert.equal(
|
||||
response._meta?.provenance?.untrusted_content,
|
||||
true,
|
||||
"Experimental results lost the canonical target-content marker",
|
||||
);
|
||||
calls.push({ name, phase, elapsed_ms: performance.now() - started });
|
||||
if (expectError) {
|
||||
assert.equal(response.isError, true);
|
||||
return JSON.parse(
|
||||
response.content.find((part) => part.type === "text").text,
|
||||
);
|
||||
}
|
||||
assert.notEqual(
|
||||
response.isError,
|
||||
true,
|
||||
JSON.stringify(response).slice(0, 2000),
|
||||
);
|
||||
return name.startsWith("experiment_")
|
||||
? JSON.parse(response.content.find((part) => part.type === "text").text)
|
||||
: requireMcpOperationResult(response, name);
|
||||
};
|
||||
const contextInput = (context) => ({ ref: context.ref, lease: context.lease });
|
||||
const stringLimit = 3,
|
||||
referenceLimit = 4;
|
||||
const query = { pattern, mode: "literal", case_sensitive: true };
|
||||
const identity = (row) => ({
|
||||
kind: "string",
|
||||
address: row.address,
|
||||
encoding: row.string?.encoding ?? null,
|
||||
});
|
||||
|
||||
const readBatch = async (name, common, { key, items, individually }) => {
|
||||
if (!individually) return call(name, { ...common, [key]: items });
|
||||
const results = [];
|
||||
for (const item of items)
|
||||
results.push(...(await call(name, { ...common, [key]: [item] })).results);
|
||||
return { results };
|
||||
};
|
||||
const readExperimentalTask = async (supportsOwners) => {
|
||||
const input = { query, limit: stringLimit, fields: ["value"] };
|
||||
if (variant === "workflow") {
|
||||
const traced = await call("experiment_trace_string_uses", {
|
||||
...input,
|
||||
reference_limit: referenceLimit,
|
||||
});
|
||||
return {
|
||||
found: traced.strings,
|
||||
refs: traced.references,
|
||||
owners: traced.functions,
|
||||
};
|
||||
}
|
||||
const found = await call("experiment_find_strings", input);
|
||||
const context = contextInput(found.context);
|
||||
const refs = await readBatch(
|
||||
"experiment_incoming_refs",
|
||||
{ context, limit: referenceLimit },
|
||||
{
|
||||
key: "targets",
|
||||
items: found.items.map((row) => row.id),
|
||||
individually: variant === "fine",
|
||||
},
|
||||
);
|
||||
const locations = [
|
||||
...new Map(
|
||||
refs.results
|
||||
.flatMap((row) => row.items ?? [])
|
||||
.map((row) => [row.from.address, row.from]),
|
||||
).values(),
|
||||
];
|
||||
const owners = await readBatch(
|
||||
"experiment_resolve_functions",
|
||||
{ context },
|
||||
{
|
||||
key: "locations",
|
||||
items: locations,
|
||||
individually: variant === "fine" && supportsOwners,
|
||||
},
|
||||
);
|
||||
return { found, refs, owners };
|
||||
};
|
||||
|
||||
let catalog,
|
||||
firstSourceId,
|
||||
firstContext,
|
||||
outcomes = [];
|
||||
try {
|
||||
await client.connect(transport);
|
||||
catalog = await client.listTools();
|
||||
const expectedOutputSchemas =
|
||||
process.env.REA_MCP_OUTPUT_SCHEMA_PROFILE === "off"
|
||||
? 0
|
||||
: catalog.tools.filter((tool) => !tool.name.startsWith("experiment_"))
|
||||
.length;
|
||||
assert.equal(
|
||||
catalog.tools.filter((tool) => Object.hasOwn(tool, "outputSchema")).length,
|
||||
expectedOutputSchemas,
|
||||
"Advertised output-schema profile differs from the requested profile",
|
||||
);
|
||||
const ajv = new Ajv2020({ strict: false, validateFormats: false });
|
||||
for (const tool of catalog.tools.filter((tool) =>
|
||||
tool.name.startsWith("experiment_"),
|
||||
)) {
|
||||
assert.equal(
|
||||
ajv.validateSchema(tool.inputSchema),
|
||||
true,
|
||||
JSON.stringify(ajv.errors),
|
||||
);
|
||||
ajv.compile(tool.inputSchema);
|
||||
}
|
||||
await call("open_binary", { path: resolve(binary), provider_id: provider });
|
||||
const status = await call("binary_session");
|
||||
const supportsOwners = status.capabilities.some(
|
||||
(row) => row.operation === "resolve_containing_procedure" && row.available,
|
||||
);
|
||||
for (phase of ["cold", "warm"]) {
|
||||
const started = performance.now();
|
||||
let strings, references, functions;
|
||||
if (variant === "baseline") {
|
||||
const all = await call("search_strings", query);
|
||||
strings = all
|
||||
.slice(0, stringLimit)
|
||||
.map((row) => ({ id: identity(row), value: row.value }));
|
||||
references = [];
|
||||
for (const string of strings)
|
||||
references.push({
|
||||
target: string.id,
|
||||
items: (await call("xrefs", { address: string.id.address })).slice(
|
||||
0,
|
||||
referenceLimit,
|
||||
),
|
||||
});
|
||||
const locations = [...new Set(references.flatMap((row) => row.items))];
|
||||
functions = [];
|
||||
for (const address of locations)
|
||||
functions.push(
|
||||
supportsOwners
|
||||
? {
|
||||
address,
|
||||
resolution: await call("resolve_containing_procedure", {
|
||||
address,
|
||||
}),
|
||||
}
|
||||
: { address, status: "unknown" },
|
||||
);
|
||||
} else {
|
||||
const { found, refs, owners } =
|
||||
await readExperimentalTask(supportsOwners);
|
||||
assert.equal(found.status, "ok");
|
||||
assert.ok(refs.results.every((row) => row.status === "ok"));
|
||||
firstSourceId ??= found.evidence_id;
|
||||
firstContext ??= found.context;
|
||||
strings = found.items.map((row) => ({ id: row.id, value: row.value }));
|
||||
references = refs.results.map((row) => ({
|
||||
target: row.target,
|
||||
items: row.items.map((edge) => edge.from.address),
|
||||
}));
|
||||
functions = owners.results.map((row) =>
|
||||
row.status === "ok"
|
||||
? { address: row.location.address, resolution: row.resolution }
|
||||
: { address: row.location.address, status: "unknown" },
|
||||
);
|
||||
}
|
||||
outcomes.push({
|
||||
phase,
|
||||
elapsed_ms: performance.now() - started,
|
||||
strings,
|
||||
references,
|
||||
functions,
|
||||
});
|
||||
}
|
||||
assert.deepEqual(outcomes[0].strings, outcomes[1].strings);
|
||||
assert.deepEqual(outcomes[0].references, outcomes[1].references);
|
||||
assert.deepEqual(outcomes[0].functions, outcomes[1].functions);
|
||||
assert.ok(outcomes[0].strings.length > 0);
|
||||
phase = "verification";
|
||||
if (variant !== "baseline" && verification === "--verify") {
|
||||
const raw = await call("experiment_read_evidence", {
|
||||
evidence_id: firstSourceId,
|
||||
});
|
||||
assert.equal(raw.evidence.operation, "search_strings");
|
||||
assert.equal(
|
||||
raw.evidence.analysis_profile.digest,
|
||||
firstContext.analysis_profile_digest,
|
||||
);
|
||||
assert.equal(
|
||||
raw.evidence.subject.digest.sha256,
|
||||
createHash("sha256")
|
||||
.update(await readFile(binary))
|
||||
.digest("hex"),
|
||||
);
|
||||
assert.deepEqual(
|
||||
raw.evidence.normalized_result
|
||||
.slice(0, stringLimit)
|
||||
.map((row) => ({ id: identity(row), value: row.value })),
|
||||
outcomes[0].strings,
|
||||
);
|
||||
const duplicate = await call("experiment_incoming_refs", {
|
||||
context: contextInput(firstContext),
|
||||
targets: [outcomes[0].strings[0].id, outcomes[0].strings[0].id],
|
||||
limit: 1,
|
||||
});
|
||||
assert.deepEqual(duplicate.results[0], duplicate.results[1]);
|
||||
const nextPage = await call("experiment_find_strings", {
|
||||
query,
|
||||
offset: stringLimit,
|
||||
limit: stringLimit,
|
||||
});
|
||||
assert.deepEqual(
|
||||
nextPage.items.map((row) => row.id),
|
||||
raw.evidence.normalized_result
|
||||
.slice(stringLimit, stringLimit * 2)
|
||||
.map(identity),
|
||||
);
|
||||
const negative = await call("experiment_find_strings", {
|
||||
query: { pattern: "\u0000REA-atomic-negative\u0000" },
|
||||
});
|
||||
assert.equal(negative.items.length, 0);
|
||||
assert.equal(negative.coverage.status, "unknown");
|
||||
const stale = await call(
|
||||
"experiment_incoming_refs",
|
||||
{
|
||||
context: {
|
||||
ref: "different-artifact-or-profile",
|
||||
lease: firstContext.lease,
|
||||
},
|
||||
targets: [outcomes[0].strings[0].id],
|
||||
},
|
||||
true,
|
||||
);
|
||||
assert.equal(stale.status, "error");
|
||||
await call("close_binary");
|
||||
assert.equal(
|
||||
(
|
||||
await call(
|
||||
"experiment_read_evidence",
|
||||
{ evidence_id: firstSourceId },
|
||||
true,
|
||||
)
|
||||
).error.code,
|
||||
"not_found",
|
||||
);
|
||||
await call("open_binary", { path: resolve(binary), provider_id: provider });
|
||||
const expired = await call(
|
||||
"experiment_incoming_refs",
|
||||
{
|
||||
context: contextInput(firstContext),
|
||||
targets: [outcomes[0].strings[0].id],
|
||||
},
|
||||
true,
|
||||
);
|
||||
assert.equal(expired.status, "error");
|
||||
checks.push(
|
||||
"retained-evidence",
|
||||
"paging",
|
||||
"duplicate-targets",
|
||||
"negative-unknown",
|
||||
"foreign-context",
|
||||
"expired-lease",
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
failures.push(String(error.stack ?? error));
|
||||
}
|
||||
phase = "lifecycle";
|
||||
try {
|
||||
await call("close_binary");
|
||||
} catch (error) {
|
||||
failures.push(String(error));
|
||||
}
|
||||
await client.close().catch((error) => failures.push(String(error)));
|
||||
await transport.close().catch((error) => failures.push(String(error)));
|
||||
const lifecycle = await completeVerifierRun(verifier);
|
||||
try {
|
||||
assertVerifierCleanupComplete(lifecycle.process_lineage);
|
||||
} catch (error) {
|
||||
failures.push(String(error));
|
||||
}
|
||||
const report = {
|
||||
provider,
|
||||
variant,
|
||||
binary: resolve(binary),
|
||||
pattern,
|
||||
node_version: process.versions.node,
|
||||
target_sha256: createHash("sha256")
|
||||
.update(await readFile(binary))
|
||||
.digest("hex"),
|
||||
string_limit: stringLimit,
|
||||
reference_limit: referenceLimit,
|
||||
outcomes,
|
||||
calls,
|
||||
checks,
|
||||
failures,
|
||||
verification_only: verification === "--verify",
|
||||
lifecycle,
|
||||
};
|
||||
await writeFile(`${prefix}.wire.json`, JSON.stringify(wire));
|
||||
await writeFile(`${prefix}.catalog.json`, JSON.stringify(catalog));
|
||||
await writeFile(`${prefix}.report.json`, JSON.stringify(report, null, 2));
|
||||
await writeFile(`${prefix}.stderr.txt`, stderr);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
provider,
|
||||
variant,
|
||||
calls: calls.length,
|
||||
outcomes: outcomes.map(({ strings, references, functions, ...rest }) => ({
|
||||
...rest,
|
||||
strings: strings.length,
|
||||
references: references.flatMap((row) => row.items).length,
|
||||
functions: functions.length,
|
||||
})),
|
||||
failures,
|
||||
}),
|
||||
);
|
||||
assert.deepEqual(failures, []);
|
||||
@@ -0,0 +1,140 @@
|
||||
// Capture the existing public MCP function dossier, without changing its contract.
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { dirname, resolve } from "node:path";
|
||||
import { Client } from "@modelcontextprotocol/client";
|
||||
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
|
||||
import {
|
||||
requireMcpOperationResult,
|
||||
requireMcpToolError,
|
||||
} from "../../scripts/lib/mcp-verifier-results.mjs";
|
||||
import {
|
||||
createVerifierRun,
|
||||
completeVerifierRun,
|
||||
assertVerifierCleanupComplete,
|
||||
} from "../../scripts/lib/verifier-run.mjs";
|
||||
|
||||
const [provider, binary, procedure, prefix] = process.argv.slice(2);
|
||||
assert.ok(
|
||||
["ghidra", "kuna"].includes(provider) && binary && procedure && prefix,
|
||||
);
|
||||
await mkdir(dirname(prefix), { recursive: true });
|
||||
const verifier = createVerifierRun();
|
||||
const client = new Client({
|
||||
name: "rea-relation-experiment",
|
||||
version: "0.1.0",
|
||||
});
|
||||
const transport = new StdioClientTransport({
|
||||
command: process.execPath,
|
||||
args: [resolve("dist/main.js")],
|
||||
env: {
|
||||
...process.env,
|
||||
REA_LOG_LEVEL: "error",
|
||||
REA_MCP_OUTPUT_SCHEMA_PROFILE: "off",
|
||||
GHIDRA_HEADLESS_MAXMEM: "1G",
|
||||
REA_GHIDRA_STARTUP_TIMEOUT_MS: "120000",
|
||||
REA_KUNA_CACHE_DIR: resolve(`${prefix}.kuna-cache`),
|
||||
REA_KUNA_JOBS: "1",
|
||||
},
|
||||
stderr: "pipe",
|
||||
});
|
||||
let stderr = "";
|
||||
transport.stderr?.on("data", (part) => {
|
||||
stderr = (stderr + part).slice(-32_768);
|
||||
});
|
||||
const wire = [],
|
||||
calls = [],
|
||||
failures = [];
|
||||
let catalog;
|
||||
const call = async (name, arguments_ = {}, acceptRetained = false) => {
|
||||
const start = performance.now();
|
||||
const response = await client.callTool(
|
||||
{ name, arguments: arguments_ },
|
||||
{ timeout: 180000 },
|
||||
);
|
||||
wire.push({ request: { name, arguments: arguments_ }, response });
|
||||
calls.push({ name, elapsed_ms: performance.now() - start });
|
||||
if (response.isError === true && acceptRetained) {
|
||||
const error = requireMcpToolError(response);
|
||||
const source = error.details?.reported_limits?.evidence_reference;
|
||||
assert.equal(error.code, "resource_constraint");
|
||||
assert.equal(source?.kind, "retained-evidence");
|
||||
await call("inspect_analysis_view", {
|
||||
source,
|
||||
view: { kind: "native", facet: "value_flow_summary" },
|
||||
});
|
||||
return { retained: source, initial_error: error };
|
||||
}
|
||||
assert.notEqual(
|
||||
response.isError,
|
||||
true,
|
||||
JSON.stringify(response).slice(0, 1600),
|
||||
);
|
||||
return requireMcpOperationResult(response, name);
|
||||
};
|
||||
try {
|
||||
await client.connect(transport);
|
||||
catalog = await client.listTools();
|
||||
assert.ok(
|
||||
catalog.tools.every((tool) => !Object.hasOwn(tool, "outputSchema")),
|
||||
);
|
||||
await call("open_binary", { path: resolve(binary), provider_id: provider });
|
||||
await call("binary_session");
|
||||
for (const phase of ["first", "warm"]) {
|
||||
const dossier = await call("analyze_function", { procedure }, true);
|
||||
await writeFile(`${prefix}.${phase}.json`, JSON.stringify(dossier), {
|
||||
flag: "wx",
|
||||
});
|
||||
}
|
||||
await call("export_evidence_bundle", {
|
||||
path: resolve(`${prefix}.bundle.json`),
|
||||
});
|
||||
} catch (error) {
|
||||
failures.push(String(error.stack ?? error));
|
||||
} finally {
|
||||
try {
|
||||
await call("close_binary");
|
||||
} catch (error) {
|
||||
failures.push(String(error));
|
||||
}
|
||||
await client.close().catch((error) => failures.push(String(error)));
|
||||
await transport.close().catch((error) => failures.push(String(error)));
|
||||
}
|
||||
const lifecycle = await completeVerifierRun(verifier);
|
||||
let cleanupVerified = false;
|
||||
try {
|
||||
assertVerifierCleanupComplete(lifecycle.process_lineage);
|
||||
cleanupVerified = true;
|
||||
} catch (error) {
|
||||
failures.push(String(error));
|
||||
}
|
||||
await writeFile(`${prefix}.wire.json`, JSON.stringify(wire), { flag: "wx" });
|
||||
await writeFile(`${prefix}.catalog.json`, JSON.stringify(catalog), {
|
||||
flag: "wx",
|
||||
});
|
||||
await writeFile(`${prefix}.stderr.txt`, stderr, { flag: "wx" });
|
||||
const report = {
|
||||
provider,
|
||||
binary: resolve(binary),
|
||||
procedure,
|
||||
target_sha256: createHash("sha256")
|
||||
.update(await readFile(binary))
|
||||
.digest("hex"),
|
||||
node_version: process.versions.node,
|
||||
calls,
|
||||
failures,
|
||||
lifecycle,
|
||||
};
|
||||
await writeFile(`${prefix}.report.json`, JSON.stringify(report, null, 2), {
|
||||
flag: "wx",
|
||||
});
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
provider,
|
||||
calls,
|
||||
failures,
|
||||
cleanup_verified: cleanupVerified,
|
||||
}),
|
||||
);
|
||||
if (failures.length) process.exitCode = 1;
|
||||
@@ -0,0 +1,109 @@
|
||||
// Native diagnostic probes; these text dumps are not a typed provider contract.
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, readFile, writeFile } from "node:fs/promises";
|
||||
import { dirname, resolve } from "node:path";
|
||||
import { parseBinaryTarget } from "../../dist/application/BinaryTargetResolver.js";
|
||||
import { inspectKunaInstallation } from "../../dist/kuna/KunaInstallation.js";
|
||||
import { KunaTargetCommands } from "../../dist/kuna/KunaTargetCommands.js";
|
||||
import { AnalysisError } from "../../dist/domain/analysisErrorBase.js";
|
||||
import { projectAnalysisError } from "../../dist/domain/analysisErrorProjection.js";
|
||||
import {
|
||||
createVerifierRun,
|
||||
completeVerifierRun,
|
||||
assertVerifierCleanupComplete,
|
||||
} from "../../scripts/lib/verifier-run.mjs";
|
||||
|
||||
const [binary, procedure, view, prefix] = process.argv.slice(2);
|
||||
assert.ok(binary && /^0x[0-9a-f]+$/u.test(procedure) && prefix);
|
||||
assert.ok(["raw", "regions", "assembly"].includes(view));
|
||||
await mkdir(dirname(prefix), { recursive: true });
|
||||
const verifier = createVerifierRun();
|
||||
const lifetime = new AbortController();
|
||||
const target = await parseBinaryTarget(binary);
|
||||
assert.ok(target.ok);
|
||||
const installed = await inspectKunaInstallation({
|
||||
...process.env,
|
||||
REA_KUNA_CACHE_DIR: resolve(`${prefix}.cache`),
|
||||
});
|
||||
assert.ok(installed.ok);
|
||||
const commands = new KunaTargetCommands(target.value, {
|
||||
signal: lifetime.signal,
|
||||
});
|
||||
const failures = [],
|
||||
calls = [];
|
||||
try {
|
||||
const arguments_ = (input) =>
|
||||
view === "assembly"
|
||||
? ["disassemble", input, procedure, "--addr", "--json"]
|
||||
: [
|
||||
"decompile",
|
||||
input,
|
||||
procedure,
|
||||
"--addr",
|
||||
`--${view}`,
|
||||
"--max-fn-memory",
|
||||
"1024",
|
||||
];
|
||||
const start = performance.now();
|
||||
const outcome = await commands.run({
|
||||
installation: installed.value,
|
||||
operation: "experiment:structure",
|
||||
arguments: arguments_,
|
||||
outputBytes: 16 * 1024 * 1024,
|
||||
timeoutMs: 120000,
|
||||
signal: lifetime.signal,
|
||||
});
|
||||
if (!outcome.ok) throw outcome.error;
|
||||
const snapshot = outcome.value;
|
||||
const file = `${prefix}.native.txt`;
|
||||
await writeFile(file, snapshot.stdout.text, { flag: "wx" });
|
||||
calls.push({
|
||||
arguments: arguments_(snapshot.input),
|
||||
elapsed_ms: performance.now() - start,
|
||||
file,
|
||||
bytes: Buffer.byteLength(snapshot.stdout.text),
|
||||
sha256: createHash("sha256").update(snapshot.stdout.text).digest("hex"),
|
||||
});
|
||||
} catch (error) {
|
||||
failures.push(
|
||||
error instanceof AnalysisError
|
||||
? projectAnalysisError(error)
|
||||
: String(error.stack ?? error),
|
||||
);
|
||||
} finally {
|
||||
lifetime.abort();
|
||||
await commands.close();
|
||||
}
|
||||
const lifecycle = await completeVerifierRun(verifier);
|
||||
let cleanupVerified = false;
|
||||
try {
|
||||
assertVerifierCleanupComplete(lifecycle.process_lineage);
|
||||
cleanupVerified = true;
|
||||
} catch (error) {
|
||||
failures.push(String(error));
|
||||
}
|
||||
const report = {
|
||||
binary: resolve(binary),
|
||||
procedure,
|
||||
view,
|
||||
calls,
|
||||
failures,
|
||||
lifecycle,
|
||||
engine: { version: installed.value.version, sha256: installed.value.sha256 },
|
||||
target_sha256: createHash("sha256")
|
||||
.update(await readFile(binary))
|
||||
.digest("hex"),
|
||||
};
|
||||
await writeFile(`${prefix}.report.json`, JSON.stringify(report, null, 2), {
|
||||
flag: "wx",
|
||||
});
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
view,
|
||||
calls,
|
||||
failures,
|
||||
cleanup_verified: cleanupVerified,
|
||||
}),
|
||||
);
|
||||
if (failures.length) process.exitCode = 1;
|
||||
@@ -0,0 +1,210 @@
|
||||
// Experimental application layer. Provider parsing and native caches remain in REA.
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
export const contextFor = (artifactSha, profile, lease) => {
|
||||
const identity = {
|
||||
artifact_sha256: artifactSha,
|
||||
analysis_profile_digest: profile.digest,
|
||||
address_space: "provider-default",
|
||||
};
|
||||
return {
|
||||
ref: createHash("sha256").update(JSON.stringify(identity)).digest("hex"),
|
||||
...identity,
|
||||
provider: profile.provider,
|
||||
lease,
|
||||
};
|
||||
};
|
||||
|
||||
const failure = (code, message) => ({
|
||||
status: "error",
|
||||
error: { code, message },
|
||||
});
|
||||
const source = (evidence) => ({
|
||||
evidence_id: evidence.evidence_id,
|
||||
confidence: evidence.confidence,
|
||||
limitations: evidence.limitations,
|
||||
});
|
||||
const pageOf = (items, offset, limit) => ({
|
||||
items: items.slice(offset, offset + limit),
|
||||
selection: {
|
||||
matched: items.length,
|
||||
offset,
|
||||
returned: Math.max(0, Math.min(limit, items.length - offset)),
|
||||
next_offset: offset + limit < items.length ? offset + limit : null,
|
||||
},
|
||||
});
|
||||
|
||||
/** One fact can be read alone or in a batch; a batch is not a new fact kind. */
|
||||
export class FactQueries {
|
||||
constructor(port) {
|
||||
this.port = port;
|
||||
}
|
||||
|
||||
async findStrings({ query, offset = 0, limit = 20, fields = ["value"] }) {
|
||||
const observed = await this.port.observe("search_strings", query);
|
||||
if (!observed.ok) return { status: "error", error: observed.error };
|
||||
const evidence = observed.value;
|
||||
const all = evidence.normalized_result;
|
||||
const page = pageOf(all, offset, limit);
|
||||
return {
|
||||
status: "ok",
|
||||
context: this.port.context(),
|
||||
...source(evidence),
|
||||
selection: page.selection,
|
||||
// Paging is complete over this provider's answer, not all artifact strings.
|
||||
coverage: {
|
||||
status: "unknown",
|
||||
reason:
|
||||
"Provider inventory is not proof of every string in the artifact.",
|
||||
},
|
||||
items: page.items.map((row) => ({
|
||||
id: {
|
||||
kind: "string",
|
||||
address: row.address,
|
||||
encoding: row.string?.encoding ?? null,
|
||||
},
|
||||
...Object.fromEntries(
|
||||
fields
|
||||
.filter((key) => Object.hasOwn(row, key))
|
||||
.map((key) => [key, row[key]]),
|
||||
),
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
validContext(context) {
|
||||
const current = this.port.context();
|
||||
return context.ref === current.ref && context.lease === current.lease;
|
||||
}
|
||||
|
||||
async incomingReferences({ context, targets, offset = 0, limit = 20 }) {
|
||||
if (!this.validContext(context))
|
||||
return failure(
|
||||
"invalid_request",
|
||||
"Fact context or session lease is stale; find facts again on the selected target.",
|
||||
);
|
||||
// Share duplicate targets within this batch. Native/session cache ownership
|
||||
// remains with the provider; this layer adds no long-lived result cache.
|
||||
const observations = new Map();
|
||||
const results = [];
|
||||
for (const target of targets) {
|
||||
let observed = observations.get(target.address);
|
||||
if (observed === undefined) {
|
||||
observed = await this.port.observe("xrefs", {
|
||||
address: target.address,
|
||||
});
|
||||
observations.set(target.address, observed);
|
||||
}
|
||||
if (!observed.ok) {
|
||||
results.push({ target, status: "error", error: observed.error });
|
||||
continue;
|
||||
}
|
||||
const evidence = observed.value;
|
||||
const page = pageOf(evidence.normalized_result, offset, limit);
|
||||
results.push({
|
||||
target,
|
||||
status: "ok",
|
||||
scope: { kind: "exact-address", address: target.address },
|
||||
...source(evidence),
|
||||
selection: page.selection,
|
||||
coverage: {
|
||||
status: "unknown",
|
||||
reason:
|
||||
"The existing xrefs contract does not carry reference completeness.",
|
||||
},
|
||||
items: page.items.map((address) => ({
|
||||
from: { kind: "location", address },
|
||||
to: { kind: "location", address: target.address },
|
||||
kind: "unknown",
|
||||
kind_reason: "The existing xrefs contract projects addresses only.",
|
||||
})),
|
||||
});
|
||||
}
|
||||
return {
|
||||
status: results.every((row) => row.status === "ok") ? "ok" : "partial",
|
||||
context: this.port.context(),
|
||||
results,
|
||||
};
|
||||
}
|
||||
|
||||
async resolveFunctions({ context, locations }) {
|
||||
if (!this.validContext(context))
|
||||
return failure(
|
||||
"invalid_request",
|
||||
"Fact context or session lease is stale; find facts again on the selected target.",
|
||||
);
|
||||
if (!this.port.supports("resolve_containing_procedure"))
|
||||
return {
|
||||
status: locations.length === 0 ? "ok" : "partial",
|
||||
context: this.port.context(),
|
||||
results: locations.map((location) => ({
|
||||
location,
|
||||
status: "unknown",
|
||||
error: {
|
||||
code: "unsupported",
|
||||
message:
|
||||
"The selected provider does not advertise containing-procedure resolution.",
|
||||
},
|
||||
})),
|
||||
};
|
||||
const observations = new Map();
|
||||
const results = [];
|
||||
for (const location of locations) {
|
||||
let observed = observations.get(location.address);
|
||||
if (observed === undefined) {
|
||||
observed = await this.port.observe("resolve_containing_procedure", {
|
||||
address: location.address,
|
||||
});
|
||||
observations.set(location.address, observed);
|
||||
}
|
||||
results.push(
|
||||
observed.ok
|
||||
? {
|
||||
location,
|
||||
status: "ok",
|
||||
...source(observed.value),
|
||||
// Preserve the owner's complete tagged found/negative outcome.
|
||||
resolution: observed.value.normalized_result,
|
||||
}
|
||||
: { location, status: "unknown", error: observed.error },
|
||||
);
|
||||
}
|
||||
return {
|
||||
status: results.every((row) => row.status === "ok") ? "ok" : "partial",
|
||||
context: this.port.context(),
|
||||
results,
|
||||
};
|
||||
}
|
||||
|
||||
async traceStringUses(input) {
|
||||
const strings = await this.findStrings(input);
|
||||
if (strings.status !== "ok") return strings;
|
||||
const references = await this.incomingReferences({
|
||||
context: strings.context,
|
||||
targets: strings.items.map((row) => row.id),
|
||||
offset: 0,
|
||||
limit: input.reference_limit ?? 20,
|
||||
});
|
||||
if (references.status === "error") return references;
|
||||
const locations = [
|
||||
...new Map(
|
||||
references.results
|
||||
.flatMap((row) => row.items ?? [])
|
||||
.map((row) => [row.from.address, row.from]),
|
||||
).values(),
|
||||
];
|
||||
const functions =
|
||||
input.include_functions === false
|
||||
? null
|
||||
: await this.resolveFunctions({ context: strings.context, locations });
|
||||
return {
|
||||
status:
|
||||
references.status === "partial" || functions?.status === "partial"
|
||||
? "partial"
|
||||
: "ok",
|
||||
strings,
|
||||
references,
|
||||
functions,
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,160 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { FactQueries, contextFor } from "./facts.mjs";
|
||||
|
||||
const profile = {
|
||||
digest: "p1",
|
||||
provider: { id: "test", name: "Test", version: "1" },
|
||||
};
|
||||
const context = contextFor("artifact-a", profile, "lease-1");
|
||||
const evidence = (normalized_result) => ({
|
||||
evidence_id: "ev:test",
|
||||
confidence: "derived",
|
||||
limitations: ["fixture coverage only"],
|
||||
normalized_result,
|
||||
});
|
||||
const port = (observe, extra = {}) => ({
|
||||
context: () => context,
|
||||
supports: () => true,
|
||||
observe,
|
||||
...extra,
|
||||
});
|
||||
const string = {
|
||||
kind: "string",
|
||||
address: "0x20000000000001",
|
||||
encoding: "utf8",
|
||||
};
|
||||
|
||||
test("artifact, profile and lease boundaries reject joins before invoking analysis", async () => {
|
||||
let calls = 0;
|
||||
const facts = new FactQueries(
|
||||
port(async () => {
|
||||
calls++;
|
||||
throw new Error("must not run");
|
||||
}),
|
||||
);
|
||||
for (const foreign of [
|
||||
contextFor("artifact-b", profile, "lease-1"),
|
||||
contextFor("artifact-a", { ...profile, digest: "p2" }, "lease-1"),
|
||||
contextFor("artifact-a", profile, "lease-2"),
|
||||
]) {
|
||||
assert.equal(
|
||||
(await facts.incomingReferences({ context: foreign, targets: [string] }))
|
||||
.status,
|
||||
"error",
|
||||
);
|
||||
assert.equal(
|
||||
(await facts.resolveFunctions({ context: foreign, locations: [] }))
|
||||
.status,
|
||||
"error",
|
||||
);
|
||||
}
|
||||
assert.equal(calls, 0);
|
||||
assert.equal(contextFor("artifact-a", profile, "lease-2").ref, context.ref);
|
||||
});
|
||||
|
||||
test("partial batches retain good observations and exact failure reasons", async () => {
|
||||
const cancelled = {
|
||||
code: "cancelled",
|
||||
message: "caller cancelled",
|
||||
details: { stage: "native" },
|
||||
};
|
||||
const facts = new FactQueries(
|
||||
port(async (_, parameters) =>
|
||||
parameters.address === "0xbad"
|
||||
? { ok: false, error: cancelled }
|
||||
: { ok: true, value: evidence(["0x20000000000002"]) },
|
||||
),
|
||||
);
|
||||
const result = await facts.incomingReferences({
|
||||
context,
|
||||
targets: [string, { ...string, address: "0xbad" }],
|
||||
});
|
||||
assert.equal(result.status, "partial");
|
||||
assert.equal(result.results[0].items[0].from.address, "0x20000000000002");
|
||||
assert.equal(result.results[0].items[0].kind, "unknown");
|
||||
assert.deepEqual(result.results[1].error, cancelled);
|
||||
assert.equal(result.results[0].coverage.status, "unknown");
|
||||
});
|
||||
|
||||
test("duplicate point lookups share work without turning selectors into observed objects", async () => {
|
||||
let calls = 0;
|
||||
const facts = new FactQueries(
|
||||
port(async () => {
|
||||
calls++;
|
||||
return { ok: true, value: evidence(["0x20"]) };
|
||||
}),
|
||||
);
|
||||
const otherEncoding = { ...string, encoding: "utf16le" };
|
||||
const result = await facts.incomingReferences({
|
||||
context,
|
||||
targets: [string, otherEncoding, string],
|
||||
});
|
||||
assert.equal(calls, 1);
|
||||
assert.deepEqual(
|
||||
result.results.map((row) => row.target),
|
||||
[string, otherEncoding, string],
|
||||
);
|
||||
assert.deepEqual(
|
||||
result.results.map((row) => row.items[0].to),
|
||||
Array(3).fill({ kind: "location", address: string.address }),
|
||||
);
|
||||
assert.equal(result.results[0].scope.kind, "exact-address");
|
||||
});
|
||||
|
||||
test("paging and field selection retain identities, source and unknown coverage", async () => {
|
||||
const rows = [
|
||||
{
|
||||
address: string.address,
|
||||
value: "😀 marker",
|
||||
string: {
|
||||
encoding: "utf8",
|
||||
encoding_status: "observed",
|
||||
byte_length: 12,
|
||||
},
|
||||
},
|
||||
{ address: "0x20000000000002", value: "other" },
|
||||
];
|
||||
const facts = new FactQueries(
|
||||
port(async () => ({ ok: true, value: evidence(rows) })),
|
||||
);
|
||||
const page = await facts.findStrings({
|
||||
query: { pattern: "" },
|
||||
limit: 1,
|
||||
fields: ["string"],
|
||||
});
|
||||
assert.deepEqual(page.items, [{ id: string, string: rows[0].string }]);
|
||||
assert.equal(page.selection.next_offset, 1);
|
||||
assert.equal(page.evidence_id, "ev:test");
|
||||
assert.equal(page.coverage.status, "unknown");
|
||||
const beyond = await facts.findStrings({
|
||||
query: { pattern: "" },
|
||||
offset: 99,
|
||||
limit: 1,
|
||||
});
|
||||
assert.equal(beyond.selection.matched, 2);
|
||||
assert.equal(beyond.selection.returned, 0);
|
||||
assert.equal(beyond.coverage.status, "unknown");
|
||||
});
|
||||
|
||||
test("unsupported function resolution remains unknown without futile native calls", async () => {
|
||||
const facts = new FactQueries(
|
||||
port(
|
||||
async () => {
|
||||
throw new Error("unsupported capability must not run");
|
||||
},
|
||||
{ supports: () => false },
|
||||
),
|
||||
);
|
||||
const result = await facts.resolveFunctions({
|
||||
context,
|
||||
locations: [{ kind: "location", address: "0x12" }],
|
||||
});
|
||||
assert.equal(result.status, "partial");
|
||||
assert.equal(result.results[0].status, "unknown");
|
||||
assert.equal(result.results[0].error.code, "unsupported");
|
||||
assert.equal(
|
||||
(await facts.resolveFunctions({ context, locations: [] })).status,
|
||||
"ok",
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,12 @@
|
||||
/* Static fixture: compile and inspect, never execute. The only reference into
|
||||
* this string lands after its first byte. Point xrefs and object uses differ. */
|
||||
static const char marker[] = "HEAD:REA atomic interior-only marker";
|
||||
|
||||
__attribute__((noinline)) const char *give_suffix(void) {
|
||||
return marker + 5;
|
||||
}
|
||||
|
||||
int main(int argc, char **argv) {
|
||||
(void)argv;
|
||||
return (unsigned char)*give_suffix() == (unsigned char)argc;
|
||||
}
|
||||
@@ -0,0 +1,300 @@
|
||||
// Decoder for the pinned 1.833 print_raw format, not a production Kuna API.
|
||||
// Relations come from native SSA annotations, never from reconstructed C text.
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
|
||||
const pc = "0x[0-9a-f]{1,16}";
|
||||
const seq = `${pc}:[0-9a-f]{1,16}`;
|
||||
const operationLine = new RegExp(`^(${seq}):\\s+(.+)$`, "u");
|
||||
const blockLine = new RegExp(`^Basic Block (\\d+) (${pc})-(${pc})$`, "u");
|
||||
const definitionToken = new RegExp(`\\((${seq})\\)`, "gu");
|
||||
const constantToken = /#(0x[0-9a-f]{1,16})(?::(\d+))?(?![0-9a-f])/gu;
|
||||
const destination = `Block_(\\d+):(${pc})`;
|
||||
const conditional = new RegExp(
|
||||
`^goto ${destination} if (.+) else ${destination}$`,
|
||||
"u",
|
||||
);
|
||||
const direct = new RegExp(`^goto ${destination}$`, "u");
|
||||
const implied = new RegExp(`^(${pc}):\\s+\\[ goto ${destination} \\]$`, "u");
|
||||
const address = (value) => `0x${BigInt(value).toString(16)}`;
|
||||
const operationId = (value) => {
|
||||
const [location, time] = value.split(":");
|
||||
return `${address(location)}:${BigInt(`0x${time}`).toString(16)}`;
|
||||
};
|
||||
const indexInto = (index, key, row) => {
|
||||
if (!index.has(key)) index.set(key, []);
|
||||
index.get(key).push(row);
|
||||
};
|
||||
const ordinal = (value) => {
|
||||
const n = Number(value);
|
||||
assert.ok(Number.isSafeInteger(n) && n >= 0);
|
||||
return n;
|
||||
};
|
||||
const freeze = (value) => {
|
||||
if (value && typeof value === "object" && !Object.isFrozen(value)) {
|
||||
Object.freeze(value);
|
||||
for (const child of Object.values(value)) freeze(child);
|
||||
}
|
||||
return value;
|
||||
};
|
||||
const validateDefinitions = (operations, issues) => {
|
||||
for (const row of operations.values()) {
|
||||
for (const relation of row.definitions) {
|
||||
if (!operations.has(relation.definition))
|
||||
issues.push({
|
||||
source_line: row.source_line,
|
||||
reason: `Missing definition ${relation.definition}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
const parseOperation = (match, blockId, sourceLine) => {
|
||||
const id = operationId(match[1]);
|
||||
const statement = match[2];
|
||||
const split = statement.indexOf(" = ");
|
||||
const lhs = split < 0 ? "" : statement.slice(0, split);
|
||||
const rhs = split < 0 ? statement : statement.slice(split + 3);
|
||||
const kind = lhs.startsWith("*(")
|
||||
? "store"
|
||||
: rhs.startsWith("*(")
|
||||
? "load"
|
||||
: rhs.startsWith("call ")
|
||||
? "call"
|
||||
: statement.startsWith("goto ")
|
||||
? "branch"
|
||||
: statement.startsWith("switch ")
|
||||
? "switch"
|
||||
: statement.startsWith("return(")
|
||||
? "return"
|
||||
: rhs.includes(" ? ")
|
||||
? "phi"
|
||||
: "value";
|
||||
// STORE's address is an input. An ordinary assignment's LHS is its output.
|
||||
const inputText = kind === "store" ? statement : rhs;
|
||||
const definitions = [...inputText.matchAll(definitionToken)].map(
|
||||
(token, occurrence) => ({
|
||||
definition: operationId(token[1]),
|
||||
use: id,
|
||||
occurrence,
|
||||
// This is printed-varnode order, NOT a p-code input_index or phi predecessor.
|
||||
}),
|
||||
);
|
||||
const literals = [...inputText.matchAll(constantToken)].map((token) => ({
|
||||
value: address(token[1]),
|
||||
printed_size_bytes: token[2] === undefined ? null : ordinal(token[2]),
|
||||
}));
|
||||
const row = {
|
||||
id,
|
||||
address: id.split(":")[0],
|
||||
block: blockId,
|
||||
kind,
|
||||
statement,
|
||||
definitions,
|
||||
constants: literals,
|
||||
source_line: sourceLine,
|
||||
};
|
||||
return row;
|
||||
};
|
||||
|
||||
const controlEdges = (row) => {
|
||||
const { statement } = row;
|
||||
const edges = [];
|
||||
const branch = conditional.exec(statement);
|
||||
const jump = direct.exec(statement);
|
||||
if (branch) {
|
||||
for (const [target, targetAddress, polarity] of [
|
||||
[branch[1], branch[2], "true"],
|
||||
[branch[4], branch[5], "false"],
|
||||
]) {
|
||||
edges.push({
|
||||
from: row.block,
|
||||
to: ordinal(target),
|
||||
target_address: address(targetAddress),
|
||||
kind: "conditional",
|
||||
polarity,
|
||||
predicate: branch[3],
|
||||
operation: row.id,
|
||||
source_line: row.source_line,
|
||||
});
|
||||
}
|
||||
} else if (jump) {
|
||||
edges.push({
|
||||
from: row.block,
|
||||
to: ordinal(jump[1]),
|
||||
target_address: address(jump[2]),
|
||||
kind: "direct",
|
||||
operation: row.id,
|
||||
source_line: row.source_line,
|
||||
});
|
||||
}
|
||||
return edges;
|
||||
};
|
||||
|
||||
const graphStart = (lines) => {
|
||||
const first = lines.findIndex(
|
||||
(line, i) => line.startsWith("Basic Block 0 ") && lines[i - 1] === "0",
|
||||
);
|
||||
assert.ok(first >= 0, "Missing native SSA block graph header");
|
||||
return first;
|
||||
};
|
||||
export const parseKunaSsaDump = (text) => {
|
||||
assert.ok(
|
||||
Buffer.byteLength(text) <= 16 * 1024 * 1024,
|
||||
"SSA dump exceeds capture budget",
|
||||
);
|
||||
const lines = text.split("\n");
|
||||
// The CLI places C before the root block graph header. Ignore that C entirely.
|
||||
const first = graphStart(lines);
|
||||
const operations = new Map(),
|
||||
blocks = new Map(),
|
||||
constants = new Map();
|
||||
const users = new Map(),
|
||||
incoming = new Map(),
|
||||
edges = [],
|
||||
issues = [];
|
||||
let block;
|
||||
for (let i = first; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
const header = blockLine.exec(line);
|
||||
if (header) {
|
||||
const id = ordinal(header[1]);
|
||||
assert.ok(!blocks.has(id), "Duplicate block identity");
|
||||
// Stop is the last instruction's address, NOT an exclusive byte bound.
|
||||
block = {
|
||||
id,
|
||||
start: address(header[2]),
|
||||
stop_instruction: address(header[3]),
|
||||
operations: [],
|
||||
source_line: i + 1,
|
||||
};
|
||||
blocks.set(id, block);
|
||||
continue;
|
||||
}
|
||||
if (!line.trim()) continue;
|
||||
const fallthrough = implied.exec(line);
|
||||
if (fallthrough) {
|
||||
edges.push({
|
||||
from: block.id,
|
||||
to: ordinal(fallthrough[2]),
|
||||
target_address: address(fallthrough[3]),
|
||||
kind: "printed-implied-goto",
|
||||
operation: null,
|
||||
source_line: i + 1,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
const match = operationLine.exec(line);
|
||||
if (!match) {
|
||||
issues.push({
|
||||
source_line: i + 1,
|
||||
reason: "Unrecognized native dump line",
|
||||
});
|
||||
continue;
|
||||
}
|
||||
const row = parseOperation(match, block.id, i + 1);
|
||||
assert.ok(!operations.has(row.id), "Duplicate operation identity");
|
||||
operations.set(row.id, row);
|
||||
block.operations.push(row.id);
|
||||
for (const literal of new Set(row.constants.map((c) => c.value)))
|
||||
indexInto(constants, literal, row.id);
|
||||
for (const edge of row.definitions) indexInto(users, edge.definition, edge);
|
||||
edges.push(...controlEdges(row));
|
||||
}
|
||||
validateDefinitions(operations, issues);
|
||||
for (const edge of edges) {
|
||||
assert.equal(
|
||||
blocks.get(edge.to)?.start,
|
||||
edge.target_address,
|
||||
"Branch target identity mismatch",
|
||||
);
|
||||
indexInto(incoming, edge.to, edge);
|
||||
}
|
||||
const outgoing = new Set(edges.map((edge) => edge.from));
|
||||
const unrepresented = [...blocks.values()]
|
||||
.filter((b) => {
|
||||
const tail = operations.get(b.operations.at(-1));
|
||||
return tail?.kind !== "return" && !outgoing.has(b.id);
|
||||
})
|
||||
.map((b) => b.id);
|
||||
const digest = createHash("sha256").update(text).digest("hex");
|
||||
for (const table of [operations, blocks, constants, users, incoming])
|
||||
for (const value of table.values()) freeze(value);
|
||||
return freeze({
|
||||
identity: { kind: "native-ssa-snapshot", sha256: digest },
|
||||
coverage: {
|
||||
scope: "printed-SSA",
|
||||
operations: operations.size,
|
||||
blocks: blocks.size,
|
||||
explicit_control_edges: edges.length,
|
||||
unrepresented_outgoing_blocks: unrepresented,
|
||||
issues,
|
||||
control_flow: "partial",
|
||||
machine_semantic_equivalence: "unknown",
|
||||
},
|
||||
// The owner parses/indexes once; all compositions reuse the same facts.
|
||||
operation: (id) => operations.get(id) ?? null,
|
||||
block: (id) => blocks.get(id) ?? null,
|
||||
findConstant: (value) => [...(constants.get(address(value)) ?? [])],
|
||||
uses: (id) => users.get(id) ?? [],
|
||||
incoming: (id) => incoming.get(id) ?? [],
|
||||
});
|
||||
};
|
||||
|
||||
// Dependency-closed only within the caller's selected budget and boundaries.
|
||||
export const backwardSlice = (
|
||||
snapshot,
|
||||
seeds,
|
||||
{ maxOperations = 48, stopAt = [], stopKinds = ["phi", "load"] } = {},
|
||||
) => {
|
||||
assert.ok(Number.isSafeInteger(maxOperations) && maxOperations > 0);
|
||||
const stopped = new Set(stopAt),
|
||||
kinds = new Set(stopKinds);
|
||||
const queue = [...new Set(seeds)],
|
||||
queued = new Set(queue),
|
||||
seen = new Set(),
|
||||
selected = [],
|
||||
boundaries = [],
|
||||
missing = [];
|
||||
for (let cursor = 0; cursor < queue.length; cursor++) {
|
||||
const id = queue[cursor];
|
||||
if (seen.has(id)) continue;
|
||||
seen.add(id);
|
||||
const row = snapshot.operation(id);
|
||||
if (!row) {
|
||||
missing.push(id);
|
||||
continue;
|
||||
}
|
||||
if (selected.length >= maxOperations) {
|
||||
boundaries.push({ id, reason: "operation-budget" });
|
||||
continue;
|
||||
}
|
||||
selected.push(row);
|
||||
if (stopped.has(id) || kinds.has(row.kind)) {
|
||||
boundaries.push({
|
||||
id,
|
||||
reason: stopped.has(id) ? "caller-selected" : row.kind,
|
||||
});
|
||||
} else
|
||||
for (const edge of row.definitions) {
|
||||
if (!queued.has(edge.definition)) {
|
||||
queue.push(edge.definition);
|
||||
queued.add(edge.definition);
|
||||
}
|
||||
}
|
||||
}
|
||||
return {
|
||||
operations: selected,
|
||||
coverage: {
|
||||
status:
|
||||
boundaries.length || missing.length || snapshot.coverage.issues.length
|
||||
? "partial"
|
||||
: "complete-for-printed-definitions",
|
||||
boundaries,
|
||||
missing,
|
||||
source_issues: snapshot.coverage.issues.length,
|
||||
runtime_reachability: "unknown",
|
||||
memory_alias_dependencies: "unknown",
|
||||
},
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,120 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { backwardSlice, parseKunaSsaDump } from "./kuna-ssa.mjs";
|
||||
import { queryLiteralUses } from "./query-ssa.mjs";
|
||||
|
||||
// Minimal native print_raw syntax, including a STORE pointer on its LHS.
|
||||
const dump = `ignored C text
|
||||
0
|
||||
Basic Block 0 0x100-0x106
|
||||
0x100:1: r0(0x100:1) = #0xffffffffffffffff
|
||||
0x102:2: u0(0x102:2) = r0(0x100:1) != #0x0
|
||||
0x106:3: goto Block_1:0x110 if (u0(0x102:2) != 0) else Block_2:0x120
|
||||
Basic Block 1 0x110-0x114
|
||||
0x110:4: u1(0x110:4) = ->(RDI(i),#0x30)
|
||||
0x112:5: *(ram,u1(0x110:4)) = r0(0x100:1)
|
||||
0x114:6: return(#0x0)
|
||||
Basic Block 2 0x120-0x120
|
||||
0x120:7: switch r0(0x100:1)
|
||||
`;
|
||||
|
||||
test("SSA definitions preserve STORE address dependencies, exact u64 and conditional polarity", () => {
|
||||
const snapshot = parseKunaSsaDump(dump);
|
||||
assert.deepEqual(snapshot.findConstant("0xffffffffffffffff"), ["0x100:1"]);
|
||||
assert.deepEqual(
|
||||
snapshot.operation("0x112:5").definitions.map((e) => e.definition),
|
||||
["0x110:4", "0x100:1"],
|
||||
);
|
||||
assert.equal(snapshot.operation("0x100:1").definitions.length, 0);
|
||||
assert.deepEqual(
|
||||
snapshot.incoming(1).map((e) => [e.from, e.polarity]),
|
||||
[[0, "true"]],
|
||||
);
|
||||
assert.deepEqual(
|
||||
snapshot.incoming(2).map((e) => [e.from, e.polarity]),
|
||||
[[0, "false"]],
|
||||
);
|
||||
assert.deepEqual(snapshot.coverage.unrepresented_outgoing_blocks, [2]);
|
||||
assert.equal(snapshot.coverage.control_flow, "partial");
|
||||
});
|
||||
|
||||
test("a bounded slice names budget frontiers; graph cycles terminate", () => {
|
||||
const snapshot = parseKunaSsaDump(dump);
|
||||
const result = backwardSlice(snapshot, ["0x112:5"], { maxOperations: 1 });
|
||||
assert.equal(result.operations.length, 1);
|
||||
assert.equal(result.coverage.boundaries.length, 2);
|
||||
assert.equal(result.coverage.status, "partial");
|
||||
const cyclic = parseKunaSsaDump(
|
||||
dump.replace("#0xffffffffffffffff", "r0(0x100:1)"),
|
||||
);
|
||||
assert.equal(backwardSlice(cyclic, ["0x100:1"]).operations.length, 1);
|
||||
});
|
||||
|
||||
test("snapshot identity changes with source; malformed and missing facts cannot look complete", () => {
|
||||
assert.notEqual(
|
||||
parseKunaSsaDump(dump).identity.sha256,
|
||||
parseKunaSsaDump(dump + "unknown native line\n").identity.sha256,
|
||||
);
|
||||
assert.equal(
|
||||
parseKunaSsaDump(dump + "unknown native line\n").coverage.issues.length,
|
||||
1,
|
||||
);
|
||||
assert.throws(
|
||||
() => parseKunaSsaDump(dump.replace("Block_1:0x110", "Block_1:0x111")),
|
||||
/identity mismatch/u,
|
||||
);
|
||||
assert.throws(
|
||||
() => parseKunaSsaDump(dump.replace("0x112:5:", "0x100:1:")),
|
||||
/Duplicate operation/u,
|
||||
);
|
||||
const missing = parseKunaSsaDump(
|
||||
dump.replace("r0(0x100:1) !=", "r0(0xff:9) !="),
|
||||
);
|
||||
assert.equal(missing.coverage.issues.length, 1);
|
||||
assert.deepEqual(backwardSlice(missing, ["0x102:2"]).coverage.missing, [
|
||||
"0xff:9",
|
||||
]);
|
||||
});
|
||||
|
||||
test("phi/load cutoffs preserve boundary rows and do not invent memory or predecessor bindings", () => {
|
||||
const snapshot = parseKunaSsaDump(
|
||||
dump.replace("r0(0x100:1) != #0x0", "r0(0x100:1) ? r0(0x100:1)"),
|
||||
);
|
||||
const result = backwardSlice(snapshot, ["0x102:2"]);
|
||||
assert.deepEqual(result.coverage.boundaries, [
|
||||
{ id: "0x102:2", reason: "phi" },
|
||||
]);
|
||||
assert.equal(result.operations.length, 1);
|
||||
assert.equal(result.operations[0].definitions.length, 2);
|
||||
assert.equal(result.coverage.memory_alias_dependencies, "unknown");
|
||||
});
|
||||
|
||||
test("cloned blocks share starts; operation identity includes sequence, and facts stay immutable", () => {
|
||||
const snapshot = parseKunaSsaDump(dump.replaceAll("0x120", "0x110"));
|
||||
assert.equal(snapshot.block(1).start, snapshot.block(2).start);
|
||||
assert.notEqual(snapshot.block(1).id, snapshot.block(2).id);
|
||||
assert.throws(() => {
|
||||
snapshot.operation("0x100:1").kind = "store";
|
||||
}, TypeError);
|
||||
assert.throws(() => snapshot.uses("0x100:1").pop(), TypeError);
|
||||
const query = queryLiteralUses(snapshot, "0xffffffffffffffff");
|
||||
assert.deepEqual(query.constant_operations, ["0x100:1"]);
|
||||
assert.ok(query.direct_uses.includes("0x112:5"));
|
||||
assert.equal(query.coverage.control_flow, "partial");
|
||||
});
|
||||
|
||||
test("a literal directly in STORE is itself an effect seed; shared block edges are returned once", () => {
|
||||
const snapshot = parseKunaSsaDump(
|
||||
dump.replace(
|
||||
"*(ram,u1(0x110:4)) = r0(0x100:1)",
|
||||
"*(ram,u1(0x110:4)) = #0x20",
|
||||
),
|
||||
);
|
||||
const query = queryLiteralUses(snapshot, "0x20");
|
||||
assert.deepEqual(query.effect_seed_operations, ["0x112:5"]);
|
||||
assert.deepEqual(
|
||||
query.control_edges.map((e) => e.operation),
|
||||
["0x106:3"],
|
||||
);
|
||||
assert.ok(query.operations.some((row) => row.id === "0x102:2"));
|
||||
});
|
||||
@@ -0,0 +1,121 @@
|
||||
// A view over native printer-to-instruction mappings, never a C-text parser.
|
||||
import assert from "node:assert/strict";
|
||||
import { z } from "zod";
|
||||
|
||||
const mapping = z.object({
|
||||
line_number: z.number().int().positive(),
|
||||
addresses: z.array(z.number().nonnegative()),
|
||||
});
|
||||
const documentSchema = z.object({
|
||||
binary: z.string(),
|
||||
count: z.number().int().nonnegative(),
|
||||
functions: z.array(
|
||||
z.object({
|
||||
address_hex: z.string(),
|
||||
address: z.number().nonnegative(),
|
||||
code: z.string().nullable(),
|
||||
error: z.string().nullable(),
|
||||
line_mappings: z.array(mapping).optional(),
|
||||
}),
|
||||
),
|
||||
});
|
||||
|
||||
export const mappedCode = async (native, reference, contextLines = 16) => {
|
||||
if (reference.functions.length !== 1 || !Array.isArray(reference.sites))
|
||||
return {
|
||||
status: "unknown",
|
||||
reason:
|
||||
"A unique native owning function and exact reference sites are required.",
|
||||
};
|
||||
const entry = reference.functions[0].address;
|
||||
const snapshot = await native((input) => [
|
||||
"decompile",
|
||||
input,
|
||||
entry,
|
||||
"--addr",
|
||||
"--json",
|
||||
"--max-fn-memory",
|
||||
"1024",
|
||||
]);
|
||||
const document = documentSchema.parse(JSON.parse(snapshot.stdout.text));
|
||||
assert.equal(document.binary, snapshot.input);
|
||||
assert.equal(document.count, document.functions.length);
|
||||
const candidates = document.functions.filter(
|
||||
(row) => row.address_hex === entry,
|
||||
);
|
||||
const fn = candidates.length === 1 ? candidates[0] : undefined;
|
||||
if (fn === undefined || fn.code === null || fn.line_mappings === undefined)
|
||||
return {
|
||||
status: "unknown",
|
||||
reason:
|
||||
"Native decompilation did not supply a body with instruction-to-line mappings.",
|
||||
native_error: fn?.error ?? null,
|
||||
};
|
||||
// This low-VMA experiment rejects unsafe numeric arrays, rather than rounding
|
||||
// u64 addresses. General u64 array support would need the owning JSON parser.
|
||||
if (
|
||||
!Number.isSafeInteger(fn.address) ||
|
||||
`0x${fn.address.toString(16)}` !== entry ||
|
||||
fn.line_mappings.some((row) =>
|
||||
row.addresses.some((value) => !Number.isSafeInteger(value)),
|
||||
)
|
||||
)
|
||||
return {
|
||||
status: "unknown",
|
||||
reason:
|
||||
"Instruction mapping contains an address outside this prototype's exact safe-integer coverage.",
|
||||
};
|
||||
const sites = new Set(reference.sites.map((row) => row.from));
|
||||
const hits = fn.line_mappings.filter((row) =>
|
||||
row.addresses.some((value) => sites.has(`0x${value.toString(16)}`)),
|
||||
);
|
||||
const mappedSites = new Set(
|
||||
hits.flatMap((row) =>
|
||||
row.addresses.map((value) => `0x${value.toString(16)}`),
|
||||
),
|
||||
);
|
||||
const unmappedSites = [...sites].filter((site) => !mappedSites.has(site));
|
||||
const lines = fn.code.split("\n");
|
||||
for (const hit of hits) assert.ok(hit.line_number <= lines.length);
|
||||
if (!hits.length)
|
||||
return {
|
||||
status: "unknown",
|
||||
reason:
|
||||
"No exact reference site maps to an emitted line; nearest-line guessing is disabled.",
|
||||
};
|
||||
const ranges = hits.map((hit) => ({
|
||||
start_line: Math.max(1, hit.line_number - contextLines),
|
||||
end_line: Math.min(lines.length, hit.line_number + contextLines),
|
||||
}));
|
||||
const slices = [];
|
||||
for (const range of ranges.sort((a, b) => a.start_line - b.start_line)) {
|
||||
const previous = slices.at(-1);
|
||||
if (previous !== undefined && range.start_line <= previous.end_line + 1)
|
||||
previous.end_line = Math.max(previous.end_line, range.end_line);
|
||||
else slices.push({ ...range });
|
||||
}
|
||||
return {
|
||||
status: unmappedSites.length || fn.error !== null ? "partial" : "ok",
|
||||
native_error: fn.error,
|
||||
entry,
|
||||
reference_sites: [...sites],
|
||||
unmapped_reference_sites: unmappedSites,
|
||||
exact_mapped_lines: hits.map((row) => row.line_number),
|
||||
total_lines: lines.length,
|
||||
total_characters: fn.code.length,
|
||||
slices: slices.map((range) => ({
|
||||
...range,
|
||||
text: lines.slice(range.start_line - 1, range.end_line).join("\n"),
|
||||
})),
|
||||
coverage: {
|
||||
status: "partial",
|
||||
reason:
|
||||
"Selected text around exact mapped sites; not a proof that every relevant guard or data dependency is included.",
|
||||
},
|
||||
limitations: [
|
||||
"Single-function engine reconstruction, not original source.",
|
||||
"Printer line mappings are derived and do not prove runtime reachability.",
|
||||
"Complete native body and mappings remain in the retained source document.",
|
||||
],
|
||||
};
|
||||
};
|
||||
@@ -0,0 +1,56 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { mappedCode } from "./mapped-code.mjs";
|
||||
|
||||
const reference = {
|
||||
functions: [{ address: "0x1000" }],
|
||||
sites: [{ from: "0x2000" }, { from: "0x2001" }],
|
||||
};
|
||||
const document = () => ({
|
||||
binary: "/private/input",
|
||||
count: 1,
|
||||
functions: [
|
||||
{
|
||||
address: 4096,
|
||||
address_hex: "0x1000",
|
||||
code: "one\ntwo\nthree\nfour\nfive",
|
||||
error: null,
|
||||
line_mappings: [{ line_number: 3, addresses: [8192] }],
|
||||
},
|
||||
],
|
||||
});
|
||||
const native = (value) => async () => ({
|
||||
input: "/private/input",
|
||||
stdout: { text: JSON.stringify(value) },
|
||||
});
|
||||
|
||||
test("a mapped subset names unmapped sites instead of implying complete mapping", async () => {
|
||||
const result = await mappedCode(native(document()), reference, 1);
|
||||
assert.equal(result.status, "partial");
|
||||
assert.deepEqual(result.unmapped_reference_sites, ["0x2001"]);
|
||||
assert.deepEqual(result.slices, [
|
||||
{ start_line: 2, end_line: 4, text: "two\nthree\nfour" },
|
||||
]);
|
||||
assert.equal(result.coverage.status, "partial");
|
||||
});
|
||||
|
||||
test("unmapped locations never select a nearby printed line", async () => {
|
||||
const result = await mappedCode(native(document()), {
|
||||
...reference,
|
||||
sites: [{ from: "0x1fff" }],
|
||||
});
|
||||
assert.equal(result.status, "unknown");
|
||||
});
|
||||
|
||||
test("unsafe numeric address arrays stay unknown without lossy conversion", async () => {
|
||||
const value = document();
|
||||
value.functions[0].line_mappings[0].addresses = [Number.MAX_SAFE_INTEGER + 1];
|
||||
const result = await mappedCode(native(value), reference);
|
||||
assert.equal(result.status, "unknown");
|
||||
});
|
||||
|
||||
test("malformed line locations reject the view before publishing fragments", async () => {
|
||||
const value = document();
|
||||
value.functions[0].line_mappings[0].line_number = 99;
|
||||
await assert.rejects(mappedCode(native(value), reference));
|
||||
});
|
||||
@@ -0,0 +1,274 @@
|
||||
// Native-boundary experiment, not MCP and not a production provider extension.
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { dirname, resolve } from "node:path";
|
||||
import { z } from "zod";
|
||||
import { AnalysisError } from "../../dist/domain/analysisErrorBase.js";
|
||||
import { projectAnalysisError } from "../../dist/domain/analysisErrorProjection.js";
|
||||
import { mappedCode } from "./mapped-code.mjs";
|
||||
import { parseBinaryTarget } from "../../dist/application/BinaryTargetResolver.js";
|
||||
import { inspectKunaInstallation } from "../../dist/kuna/KunaInstallation.js";
|
||||
import { KunaTargetCommands } from "../../dist/kuna/KunaTargetCommands.js";
|
||||
import { parseKunaJson } from "../../dist/kuna/KunaJson.js";
|
||||
import { parseKunaStringInventory } from "../../dist/kuna/KunaStringInventory.js";
|
||||
import { parseKunaIncomingReferenceFacts } from "../../dist/kuna/KunaReferenceFiles.js";
|
||||
import {
|
||||
createVerifierRun,
|
||||
completeVerifierRun,
|
||||
assertVerifierCleanupComplete,
|
||||
} from "../../scripts/lib/verifier-run.mjs";
|
||||
|
||||
const [binary, pattern, variant, prefix, codeOption] = process.argv.slice(2);
|
||||
assert.ok(binary && pattern && prefix);
|
||||
assert.ok(["per-address", "inventory"].includes(variant));
|
||||
await mkdir(dirname(prefix), { recursive: true });
|
||||
const verifier = createVerifierRun();
|
||||
const lifetime = new AbortController();
|
||||
const parsed = await parseBinaryTarget(binary);
|
||||
assert.ok(parsed.ok);
|
||||
const installed = await inspectKunaInstallation({
|
||||
...process.env,
|
||||
REA_KUNA_CACHE_DIR: resolve(`${prefix}.cache`),
|
||||
});
|
||||
assert.ok(installed.ok);
|
||||
const commands = new KunaTargetCommands(parsed.value, {
|
||||
signal: lifetime.signal,
|
||||
});
|
||||
const address = z.string().regex(/^0x(?:0|[1-9a-f][0-9a-f]{0,15})$/u);
|
||||
const functionSchema = z.object({
|
||||
name: z.string(),
|
||||
address,
|
||||
address_hex: address,
|
||||
});
|
||||
const relatedRow = z.object({
|
||||
address,
|
||||
address_hex: address,
|
||||
text: z.string(),
|
||||
length: z.number().int().nonnegative(),
|
||||
encoding: z.string(),
|
||||
byte_length: z.number().int().nonnegative(),
|
||||
nul_terminated: z.boolean(),
|
||||
xrefs_count: z.number().int().nonnegative(),
|
||||
functions: z.array(functionSchema),
|
||||
});
|
||||
const sourceKeys = new Set(["address", "from_address", "to_address"]);
|
||||
const limit = 16 * 1024 * 1024;
|
||||
const calls = [],
|
||||
failures = [];
|
||||
let facts;
|
||||
const native = async (arguments_) => {
|
||||
const start = performance.now();
|
||||
const outcome = await commands.run({
|
||||
installation: installed.value,
|
||||
operation: "experiment:native-relations",
|
||||
arguments: arguments_,
|
||||
outputBytes: limit,
|
||||
timeoutMs: 120000,
|
||||
signal: lifetime.signal,
|
||||
});
|
||||
if (!outcome.ok) throw outcome.error;
|
||||
const snapshot = outcome.value;
|
||||
const file = `${prefix}.native-${calls.length}.json`;
|
||||
await writeFile(file, snapshot.stdout.text);
|
||||
calls.push({
|
||||
elapsed_ms: performance.now() - start,
|
||||
file,
|
||||
arguments: arguments_(snapshot.input),
|
||||
sha256: createHash("sha256").update(snapshot.stdout.text).digest("hex"),
|
||||
bytes: Buffer.byteLength(snapshot.stdout.text),
|
||||
});
|
||||
return snapshot;
|
||||
};
|
||||
try {
|
||||
// Common producer: validate all string facts once through the existing owner.
|
||||
const scan = await native((input) => [
|
||||
"strings",
|
||||
input,
|
||||
"--json",
|
||||
"--encoding",
|
||||
"all",
|
||||
"--min-length",
|
||||
"5",
|
||||
"--termination",
|
||||
"any",
|
||||
"--no-xrefs",
|
||||
]);
|
||||
const inventory = parseKunaStringInventory(scan.stdout.text, scan.input);
|
||||
assert.ok(inventory.ok, inventory.error);
|
||||
const selected = inventory.value.items
|
||||
.filter((row) => row.value.includes(pattern))
|
||||
.slice(0, 3);
|
||||
assert.ok(selected.length > 0);
|
||||
const results = [];
|
||||
if (variant === "per-address") {
|
||||
for (const item of selected) {
|
||||
const result = await native((input) => [
|
||||
"xrefs",
|
||||
input,
|
||||
"--to",
|
||||
item.address,
|
||||
"--json",
|
||||
]);
|
||||
// Consume the owning parser's facts, without reinterpreting raw JSON.
|
||||
const validated = parseKunaIncomingReferenceFacts(
|
||||
result.stdout.text,
|
||||
result.input,
|
||||
item.address,
|
||||
);
|
||||
assert.ok(validated.ok, validated.error);
|
||||
const rows = validated.value.facts;
|
||||
const functions = [
|
||||
...new Map(
|
||||
rows
|
||||
.flatMap((row) =>
|
||||
row.fromFunction === null ? [] : [row.fromFunction],
|
||||
)
|
||||
.map((fn) => [fn.address, fn]),
|
||||
).values(),
|
||||
];
|
||||
results.push({
|
||||
string: item,
|
||||
scope: "exact-start-address",
|
||||
xrefs_count: rows.length,
|
||||
functions,
|
||||
sites: rows.map((row) => ({
|
||||
from: row.from,
|
||||
to: row.to,
|
||||
kind: row.kind,
|
||||
instruction: row.instruction,
|
||||
})),
|
||||
source: calls.at(-1),
|
||||
});
|
||||
}
|
||||
} else {
|
||||
const result = await native((input) => [
|
||||
"strings",
|
||||
input,
|
||||
"--json",
|
||||
"--encoding",
|
||||
"all",
|
||||
"--min-length",
|
||||
"5",
|
||||
"--termination",
|
||||
"any",
|
||||
]);
|
||||
const document = parseKunaJson(result.stdout.text, limit, sourceKeys);
|
||||
assert.equal(document.binary, result.input);
|
||||
assert.equal(document.xrefs, true);
|
||||
assert.equal(document.encoding, "all");
|
||||
assert.equal(document.min_length, 5);
|
||||
assert.equal(document.termination, "any");
|
||||
assert.equal(document.filter, null);
|
||||
assert.equal(document.section, null);
|
||||
assert.equal(document.scanned, inventory.value.scanned);
|
||||
assert.equal(document.count, document.strings.length);
|
||||
assert.equal(document.count, inventory.value.items.length);
|
||||
const known = new Map(
|
||||
inventory.value.items.map((row) => [
|
||||
`${row.address}|${row.string.encoding}`,
|
||||
row,
|
||||
]),
|
||||
);
|
||||
const examined = new Map();
|
||||
for (const value of document.strings) {
|
||||
const row = relatedRow.parse(value);
|
||||
assert.equal(row.address, row.address_hex);
|
||||
const key = `${row.address}|${row.encoding}`;
|
||||
assert.ok(!examined.has(key));
|
||||
const expected = known.get(key);
|
||||
assert.ok(expected);
|
||||
assert.equal(row.text, expected.value);
|
||||
assert.equal(row.length, [...expected.value].length);
|
||||
assert.equal(row.byte_length, expected.string.byte_length);
|
||||
assert.equal(
|
||||
row.nul_terminated,
|
||||
expected.string.termination === "present_or_not_required",
|
||||
);
|
||||
const entries = new Set();
|
||||
for (const fn of row.functions) {
|
||||
assert.equal(fn.address, fn.address_hex);
|
||||
assert.ok(!entries.has(fn.address));
|
||||
entries.add(fn.address);
|
||||
}
|
||||
assert.ok(row.functions.length <= row.xrefs_count);
|
||||
examined.set(key, row);
|
||||
}
|
||||
for (const item of selected) {
|
||||
const row = examined.get(`${item.address}|${item.string.encoding}`);
|
||||
results.push({
|
||||
string: item,
|
||||
scope: "any-byte-in-string-extent",
|
||||
xrefs_count: row.xrefs_count,
|
||||
functions: row.functions,
|
||||
sites: {
|
||||
status: "unknown",
|
||||
reason:
|
||||
"Native string attribution does not return reference sites or kinds.",
|
||||
},
|
||||
source: calls.at(-1),
|
||||
});
|
||||
}
|
||||
}
|
||||
const code =
|
||||
codeOption === "--code-slice" ? await mappedCode(native, results[0]) : null;
|
||||
facts = {
|
||||
artifact_sha256: parsed.value.sha256,
|
||||
engine: {
|
||||
version: installed.value.version,
|
||||
sha256: installed.value.sha256,
|
||||
},
|
||||
inventory_count: inventory.value.items.length,
|
||||
selected_count: selected.length,
|
||||
results,
|
||||
mapped_code: code === null ? null : { ...code, source: calls.at(-1) },
|
||||
limitations: [
|
||||
"These are native derived static references, not runtime uses.",
|
||||
"The per-address and inventory scopes differ; their timings do not establish equivalent-query speedup.",
|
||||
"An empty function set can mean unowned references, not that a string is unused.",
|
||||
"Complete native source documents are retained locally by file and SHA-256; this is not an MCP token measurement.",
|
||||
],
|
||||
};
|
||||
} catch (error) {
|
||||
failures.push(
|
||||
error instanceof AnalysisError
|
||||
? projectAnalysisError(error)
|
||||
: { message: String(error), stack: error.stack ?? null },
|
||||
);
|
||||
}
|
||||
lifetime.abort();
|
||||
const closed = await commands.close();
|
||||
if (!closed.ok) failures.push(String(closed.error));
|
||||
const lifecycle = await completeVerifierRun(verifier);
|
||||
try {
|
||||
assertVerifierCleanupComplete(lifecycle.process_lineage);
|
||||
} catch (error) {
|
||||
failures.push(String(error));
|
||||
}
|
||||
await writeFile(
|
||||
`${prefix}.report.json`,
|
||||
JSON.stringify(
|
||||
{
|
||||
variant,
|
||||
binary: resolve(binary),
|
||||
pattern,
|
||||
calls,
|
||||
facts,
|
||||
failures,
|
||||
lifecycle,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
variant,
|
||||
native_commands: calls.length,
|
||||
native_ms: calls.reduce((sum, row) => sum + row.elapsed_ms, 0),
|
||||
selected: facts?.selected_count,
|
||||
owners: facts?.results.map((row) => row.functions.length),
|
||||
failures,
|
||||
}),
|
||||
);
|
||||
assert.deepEqual(failures, []);
|
||||
@@ -0,0 +1,168 @@
|
||||
// A composition over one validated, immutable snapshot; no new engine work.
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { backwardSlice, parseKunaSsaDump } from "./kuna-ssa.mjs";
|
||||
import { assertVerifierCleanupComplete } from "../../scripts/lib/verifier-run.mjs";
|
||||
|
||||
export const queryLiteralUses = (snapshot, literal, maxOperations = 48) => {
|
||||
const allSeeds = snapshot.findConstant(literal);
|
||||
const seeds = allSeeds.slice(0, 16);
|
||||
const effectSeeds = seeds.filter((id) =>
|
||||
["store", "call"].includes(snapshot.operation(id).kind),
|
||||
);
|
||||
const allUses = [
|
||||
...new Set([
|
||||
...effectSeeds,
|
||||
...seeds.flatMap((id) => snapshot.uses(id).map((e) => e.use)),
|
||||
]),
|
||||
];
|
||||
const selectedUses = allUses.slice(0, 16);
|
||||
const consumers = selectedUses.map((id) => snapshot.operation(id));
|
||||
const controlEdges = [
|
||||
...new Map(
|
||||
consumers
|
||||
.flatMap((row) => snapshot.incoming(row.block))
|
||||
.map((edge) => [
|
||||
JSON.stringify([edge.from, edge.to, edge.operation, edge.polarity]),
|
||||
edge,
|
||||
]),
|
||||
).values(),
|
||||
];
|
||||
const branchIds = [
|
||||
...new Set(
|
||||
controlEdges.flatMap((edge) =>
|
||||
edge.operation === null ? [] : [edge.operation],
|
||||
),
|
||||
),
|
||||
];
|
||||
const guards = backwardSlice(snapshot, branchIds, { maxOperations });
|
||||
const facts = [
|
||||
...new Map(
|
||||
[
|
||||
...seeds.map((id) => snapshot.operation(id)),
|
||||
...consumers,
|
||||
...guards.operations,
|
||||
].map((row) => [row.id, row]),
|
||||
).values(),
|
||||
];
|
||||
const selected = new Set(facts.map((row) => row.id));
|
||||
return {
|
||||
literal,
|
||||
snapshot: snapshot.identity,
|
||||
constant_operations: seeds,
|
||||
effect_seed_operations: effectSeeds,
|
||||
direct_uses: selectedUses,
|
||||
control_edges: controlEdges,
|
||||
operations: facts,
|
||||
coverage: {
|
||||
status: "partial",
|
||||
seed_matches: allSeeds.length,
|
||||
omitted_seed_matches: allSeeds.length - seeds.length,
|
||||
omitted_direct_uses: allUses.length - selectedUses.length,
|
||||
dependencies: guards.coverage,
|
||||
unexpanded_definition_ids: [
|
||||
...new Set(
|
||||
facts.flatMap((row) =>
|
||||
row.definitions
|
||||
.filter((edge) => !selected.has(edge.definition))
|
||||
.map((edge) => edge.definition),
|
||||
),
|
||||
),
|
||||
],
|
||||
control_flow: snapshot.coverage.control_flow,
|
||||
unrepresented_outgoing_blocks:
|
||||
snapshot.coverage.unrepresented_outgoing_blocks.length,
|
||||
source_issues: snapshot.coverage.issues.length,
|
||||
scope:
|
||||
"literal equality, effect seeds and direct printed uses, their explicit incoming edges and bounded branch definitions",
|
||||
address_object_identity: "unknown",
|
||||
call_argument_bindings: "unknown",
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const sha256 = (value) => createHash("sha256").update(value).digest("hex");
|
||||
const main = async () => {
|
||||
const [prefix, values, output] = process.argv.slice(2);
|
||||
assert.ok(prefix && values && output);
|
||||
const literals = values.split(",");
|
||||
assert.ok(
|
||||
literals.length <= 16 &&
|
||||
literals.every((s) => /^0x[0-9a-f]{1,16}$/u.test(s)),
|
||||
);
|
||||
const report = JSON.parse(await readFile(`${prefix}.report.json`, "utf8"));
|
||||
assert.equal(report.view, "raw");
|
||||
assert.deepEqual(report.failures, []);
|
||||
assert.equal(report.calls.length, 1);
|
||||
assert.equal(
|
||||
report.engine.version,
|
||||
"1.833",
|
||||
"Decoder is pinned to the inspected printer format",
|
||||
);
|
||||
assertVerifierCleanupComplete(report.lifecycle.process_lineage);
|
||||
const source = report.calls[0];
|
||||
const raw = await readFile(source.file, "utf8");
|
||||
assert.equal(sha256(raw), source.sha256);
|
||||
assert.equal(Buffer.byteLength(raw), source.bytes);
|
||||
assert.equal(
|
||||
sha256(await readFile(report.binary)),
|
||||
report.target_sha256,
|
||||
"Artifact changed since capture",
|
||||
);
|
||||
const start = performance.now();
|
||||
const snapshot = parseKunaSsaDump(raw);
|
||||
const parseIndexMs = performance.now() - start;
|
||||
const runs = [];
|
||||
for (const phase of ["first-projection", "reused-index"]) {
|
||||
const before = performance.now();
|
||||
const answers = literals.map((literal) =>
|
||||
queryLiteralUses(snapshot, literal),
|
||||
);
|
||||
const serialized = JSON.stringify(answers);
|
||||
runs.push({
|
||||
phase,
|
||||
query_and_serialization_ms: performance.now() - before,
|
||||
answers,
|
||||
serialized_bytes: Buffer.byteLength(serialized),
|
||||
});
|
||||
}
|
||||
assert.deepEqual(runs[0].answers, runs[1].answers);
|
||||
await writeFile(
|
||||
output,
|
||||
JSON.stringify(
|
||||
{
|
||||
source: {
|
||||
artifact_sha256: report.target_sha256,
|
||||
engine: report.engine,
|
||||
procedure: report.procedure,
|
||||
capture_report: `${prefix}.report.json`,
|
||||
raw: source,
|
||||
profile:
|
||||
"one-function native decompile defaults + --raw; no annotations",
|
||||
},
|
||||
parse_index_ms: parseIndexMs,
|
||||
coverage: snapshot.coverage,
|
||||
runs,
|
||||
lifetime:
|
||||
"One local process; validated immutable snapshot reused without native analysis. No cross-session cache.",
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
{ flag: "wx" },
|
||||
);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
output,
|
||||
parse_index_ms: parseIndexMs,
|
||||
runs: runs.map(({ answers, ...timing }) => ({
|
||||
...timing,
|
||||
selected_operations: answers.map((a) => a.operations.length),
|
||||
})),
|
||||
}),
|
||||
);
|
||||
};
|
||||
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href)
|
||||
await main();
|
||||
@@ -0,0 +1,239 @@
|
||||
# Control-flow and definition/use experiment
|
||||
|
||||
This continues the first-round experiment on the same REA source baseline and
|
||||
Kuna 1.833. It is an exploratory native-boundary decoder, not a production
|
||||
provider capability or a new MCP catalog. No PR or merge is intended.
|
||||
|
||||
The analyst questions are which local conditions lead to zlib's
|
||||
`incorrect header check` assignment and SQLite's
|
||||
`malformed MATCH expression: [%s]` error-formatting call. Runtime reachability,
|
||||
memory aliasing, phi predecessor bindings and complete entry-state conditions
|
||||
remain unknown. These are not controlled agent trials.
|
||||
|
||||
`capture-function.mjs` records the existing public MCP function dossier. It
|
||||
preserves transport-budget failures and follows retained Evidence for a value-flow
|
||||
summary, then exports the complete local Evidence bundle before closing.
|
||||
`capture-kuna-structure.mjs` uses REA's existing target/process owners to capture
|
||||
the native `--raw`, `--regions` or disassembly output. All target inspection is
|
||||
static. The engine and artifact digests, command arguments, raw digest and cleanup
|
||||
proof are retained locally.
|
||||
|
||||
`kuna-ssa.mjs` owns the decoder for the inspected 1.833 SSA diagnostic format.
|
||||
It indexes explicit definition annotations, literal values, block membership and
|
||||
printed branch destinations once. Immutable rows carry native source lines;
|
||||
operation and block IDs are local to the content-digested snapshot. A block's
|
||||
printed stop is an instruction address, not an exclusive extent. Blocks can share
|
||||
start addresses. Unknown lines, omitted outgoing relationships, missing definitions
|
||||
and traversal boundaries remain explicit. Printed operand occurrences do not
|
||||
claim p-code input indexes or phi-to-predecessor bindings.
|
||||
|
||||
`query-ssa.mjs` composes literal equality, direct uses, incoming explicit edges
|
||||
and bounded backward definitions. It validates the retained capture against its
|
||||
digest and the current artifact before indexing. It retains complete native
|
||||
evidence; its local cache lives for one process. Default phi/load boundaries avoid
|
||||
expanding the whole program state. Selection limits report omissions.
|
||||
|
||||
Use the pinned Node from `.nvmrc`, an existing build and caller-supplied Kuna:
|
||||
|
||||
```sh
|
||||
node --test experiments/atomic-re-tools/kuna-ssa.test.mjs
|
||||
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
|
||||
node experiments/atomic-re-tools/capture-kuna-structure.mjs \
|
||||
/usr/lib/x86_64-linux-gnu/libz.so.1 0xc1e0 raw .cache/atomic-re-tools/relations-zlib
|
||||
python3 experiments/analysis-efficiency/resource-run.py --observe-payload-exit -- \
|
||||
node experiments/atomic-re-tools/query-ssa.mjs .cache/atomic-re-tools/relations-zlib \
|
||||
0x18fc4 .cache/atomic-re-tools/relations-zlib-query.json
|
||||
```
|
||||
|
||||
The addresses above apply only to the exact zlib artifact in [results](results.md).
|
||||
Do not reuse them across provider load spaces or binary versions. Complete
|
||||
measurement and validation notes follow below.
|
||||
|
||||
## Observed results
|
||||
|
||||
The same artifact and engine digests as [round one](results.md) apply. This is
|
||||
the existing experiment, not a newly rebased feature branch: newer REA main
|
||||
`42fc6c53ae188978deb3bedfe88287b6b316ba22` was fetched for inspection, but these
|
||||
runs retain baseline `5b16f2431062eea6ffd245a9e05512790007cb13` and Kuna 1.833.
|
||||
The source decoder was checked against Kuna's pinned
|
||||
[SSA renderer](https://github.com/Noelo-Lab/kuna/blob/fe01d6294eb0d1053ce343c8943646df88e143fc/decompiler/crates/kuna-decomp/src/substrate/funcdata_printraw.rs).
|
||||
No newer Kuna version was tested.
|
||||
|
||||
1. **SSA value origins repair all three failed zlib mappings.** The original
|
||||
string xrefs are LEAs whose addresses do not appear in printer mappings.
|
||||
Literal equality in optimized SSA finds a pointer-valued operation; its
|
||||
explicit def-use points to a STORE at a different instruction origin. Those
|
||||
origins do appear in the retained JSON line mappings:
|
||||
|
||||
| String address | Original LEA | STORE origin | Exact emitted line |
|
||||
| -------------- | ------------ | ------------ | -----------------: |
|
||||
| `0x18fc4` | `0xded7` | `0xdee6` | 164 |
|
||||
| `0x19003` | `0xdc6a` | `0xdc74` | 551 |
|
||||
| `0x19018` | `0xcd05` | `0xcd0f` | 653 |
|
||||
|
||||
The independently captured raw-mode C and JSON-mode C are byte-equal after
|
||||
removing outer whitespace. The verifier checks both retained source digests,
|
||||
artifact and engine equality before joining instruction origins. This is
|
||||
corroboration between two captures, not proof of a shared internal IR revision.
|
||||
A production interface should export these relationships together.
|
||||
|
||||
2. **Control and def-use answer a bounded local question.** For the header error,
|
||||
the result contains 21 operations: the literal origin, STORE and 19 branch
|
||||
dependencies. Two explicit incoming edges enter block 446. They correspond
|
||||
to a missing low wrap bit and a failed divisibility check on the constructed
|
||||
header value. The branch definitions stop at a memory load and a phi node.
|
||||
Original machine instructions independently show `test $1` / `je` and the
|
||||
reciprocal-division sequence / `jne` into the error block. The verifier
|
||||
exhaustively checks all 65,536 ordinary 16-bit header values against both
|
||||
the captured arithmetic sequence and modulo 31. No target code is executed.
|
||||
This agrees with [zlib 1.2.13 source](https://github.com/madler/zlib/blob/v1.2.13/inflate.c#L633)
|
||||
and [RFC 1950](https://www.rfc-editor.org/rfc/rfc1950.html#section-2.2).
|
||||
It does not establish all entry-state preconditions or primed bit-buffer states.
|
||||
|
||||
3. **The same composition finds a SQLite call guard.** For the first malformed
|
||||
MATCH string at `0x11dcd8`, five selected operations connect the literal to
|
||||
`sqlite3Fts3ErrMsg` and the observed condition `EBX == 1`. The original bytes
|
||||
have `cmp $1,%ebx` and `je` to the formatting path; the corresponding
|
||||
[3.40.1 source](https://github.com/sqlite/sqlite/blob/version-3.40.1/ext/fts3/fts3_expr.c#L1057)
|
||||
uses `rc == SQLITE_ERROR`, whose [code is 1](https://www.sqlite.org/rescode.html#error).
|
||||
The slice stops at the return-code phi: it does not explain why a particular
|
||||
input caused the parser error. Source-version agreement is corroboration,
|
||||
not proof of an exact Debian source/build match.
|
||||
|
||||
4. **Addresses do not identify IR blocks.** SQLite blocks 20 and 22 both start at
|
||||
`0x6523e`, yet have distinct identities and paths. Several operations also
|
||||
share instruction origins. Do not use a block's printed address span as its
|
||||
membership set, or pair phi inputs with CFG predecessors by printed order.
|
||||
|
||||
5. **Retained paging cannot recover producer omissions.** Ghidra `inflate` by
|
||||
name is ambiguous between PLT and body. Its exact body is `0x10c1e0`, in a
|
||||
different load space from Kuna's `0xc1e0`. The complete MCP dossier exceeded
|
||||
the 10 MiB transport budget. The server correctly retained Evidence and
|
||||
supplied continuation arguments; summary reads and local bundle export worked.
|
||||
However, high p-code was already truncated to 3,000 operations, with 239 known
|
||||
omitted edges and at least one omitted operation. The late error literal is
|
||||
outside that prefix. Its absence remains **unknown**. Raising the response
|
||||
limit or paging the retained prefix would not repair this missing coverage.
|
||||
|
||||
Kuna's raw dump contains 3,860 operations / 472 blocks for `inflate`, and
|
||||
253 operations / 23 blocks for the SQLite function. All printed definition IDs
|
||||
resolved in these captures. Nevertheless, 63 / 1 blocks respectively have
|
||||
unrepresented outgoing relationships. In particular, the diagnostic format does
|
||||
not enumerate switch successors or every layout fallthrough. Incoming edges are
|
||||
local observations, not a complete control-dependence or feasible-path proof.
|
||||
`--regions` exposes structuring output; it did not provide the missing typed CFG.
|
||||
|
||||
## Cost and limits of the comparison
|
||||
|
||||
The estimates below use tiktoken 0.14.0 / `o200k_base`, not bytes-to-token ratios.
|
||||
Full C is native text; the selected answer is JSON including source identity,
|
||||
raw-Evidence access, facts and unknown coverage. No hypothetical MCP wrapper is
|
||||
counted. These are **representation costs**, not equivalent completed agent tasks.
|
||||
|
||||
| Target / question | Full C tokens | Full raw C + SSA tokens | Selected answer tokens |
|
||||
| ------------------ | ------------: | ----------------------: | ---------------------: |
|
||||
| zlib header error | 17,726 | 254,265 | 3,270 |
|
||||
| SQLite MATCH error | 931 | 14,928 | 1,283 |
|
||||
|
||||
The three zlib answers together cost 5,485 tokens with one shared source envelope.
|
||||
The SQLite counterexample matters: a small function is cheaper as complete C than
|
||||
as this structured slice. A successful atomic API does not force tiny getters or
|
||||
graph output for every question. A C window around zlib line 164, on the other
|
||||
hand, still misses the earlier guard at lines 122–124; locating a line alone did
|
||||
not complete the local explanation.
|
||||
|
||||
| Stage | Elapsed s | Family CPU s | Peak family MiB |
|
||||
| ---------------------------------------------------------- | --------: | -----------: | --------------: |
|
||||
| zlib native raw, first capture | 6.965 | 6.608 | 208.1 |
|
||||
| zlib native raw, fresh repeat | 18.749 | 13.462 | 195.9 |
|
||||
| SQLite native raw | 2.738 | 2.557 | 172.5 |
|
||||
| zlib validation, index and two projections | 0.423 | 0.227 | 43.7 |
|
||||
| SQLite validation, index and two projections | 0.322 | 0.095 | 24.9 |
|
||||
| Ghidra exact selection, retained summary / bundle workflow | 49.170 | 48.536 | 1,466.6 |
|
||||
|
||||
The fresh zlib captures have identical raw digests, but different elapsed/CPU
|
||||
costs; the cause is unisolated. These samples establish neither native-analysis
|
||||
acceleration nor a cross-provider speed ranking. OS caches were not flushed.
|
||||
The Ghidra row includes first/warm requests and lifecycle, and is not equivalent
|
||||
to the one-command native rows. Its analyst question remained unfinished.
|
||||
|
||||
Within the final projection runs, parse/index took 130.20 ms / 7.96 ms for
|
||||
zlib / SQLite. First query plus serialization took 1.10 ms / 0.64 ms; repeat
|
||||
queries on the same immutable index took 0.29 ms / 0.09 ms. Native analysis did
|
||||
not run during either projection. This is a useful reuse mechanism, not a claim
|
||||
that the original native decompilation becomes that fast.
|
||||
|
||||
All resource observations use the existing one-CPU, 4 GiB hard limit, 3 GiB early
|
||||
stop and 192-task owner. Peak memory is the kernel cgroup `memory.peak` for the
|
||||
whole process family, **including charged file cache**, not RSS. Every captured
|
||||
run had verified payload-exit cleanup and zero OOM kills. The complete 12-run
|
||||
ledger includes the ambiguous-name failure, the transport failure, the region
|
||||
probe, superseded projections and automated verification: 178.051 s wall and
|
||||
168.094 s family CPU in total. Manual investigation, old discovery/xrefs, objdump,
|
||||
tokenization and development checks are outside that ledger.
|
||||
|
||||
The actual Ghidra retained workflow used the existing complete input-only catalog:
|
||||
206,954 estimated catalog tokens plus 39,620 captured request/response tokens,
|
||||
including failures, lifecycle and export. Exported bundle contents stayed local.
|
||||
These costs do not count as a successful solution or as comparable Kuna savings.
|
||||
No model reasoning tokens, answer-quality score, controlled agent A/B trial or
|
||||
end-to-end agent latency was measured.
|
||||
|
||||
## What to explore next
|
||||
|
||||
The useful candidate is a shared analysis snapshot with typed facts and relations,
|
||||
then bounded question-specific projections. Stable identity must include artifact,
|
||||
load space, profile and snapshot; value definitions and block membership need
|
||||
native identities in addition to instruction origins. Complete control dependence
|
||||
requires a CFG with coverage; alias effects and phi predecessor bindings need
|
||||
explicit producer support.
|
||||
|
||||
The nearest existing surfaces are `trace_native_values` for analysis and native
|
||||
facets of `inspect_analysis_view` for retained reads. Retained views should remain
|
||||
read-only: new analysis belongs to the analysis owner. Seeded selection must happen
|
||||
before Ghidra's producer limit. Kuna should ideally export a structured IR/CFG
|
||||
subset from the native owner instead of making this diagnostic-text decoder a
|
||||
production protocol. Do not add a separate MCP tool for each getter or a custom
|
||||
query language on the basis of these two functions.
|
||||
|
||||
Complex-software superiority is still unproven. The next discriminating question
|
||||
is why an input reaches a selected error: that needs phi predecessor bindings and
|
||||
control dependencies beyond these local slices, with a completed-task comparison
|
||||
against the ordinary C workflow. Keep the small-SQLite counterexample in that
|
||||
evaluation rather than optimizing only the large-function case.
|
||||
|
||||
## Verification and retained evidence
|
||||
|
||||
Six distinct SSA module tests plus the existing nine fact/mapping tests passed.
|
||||
Real static captures, exact mapping replay for three errors, bounded SQLite
|
||||
composition, 65,536 arithmetic cases, original instruction checks and Ghidra
|
||||
retained-Evidence recovery passed. The Ghidra late-literal query did not have
|
||||
sufficient producer coverage. Formatter, targeted lint, syntax and whitespace
|
||||
checks ran; no production runtime source was changed in this round.
|
||||
|
||||
Locally retained evidence is under `.cache/atomic-re-tools/`, including
|
||||
`relations-*.{report.json,log}`, `relations-*-raw*.native.txt`, the exported Ghidra
|
||||
bundle, `relations-*-query-v2.json`, `relations-verification-v2.json` and
|
||||
`relations-summary-v2.json`. Raw captures are not committed. Their SHA-256 values:
|
||||
|
||||
- zlib raw: `b1d6bc61755041d2934e4ab173aca4e5fdf40f5bda01ca404458a163b129681b`.
|
||||
- SQLite raw: `8a945a1549b855802958bd4d33f37b9aaf00781aa57059fe25cb740244a1cdde`.
|
||||
- zlib JSON body/mappings: `21631d31b4b93aa75c200be71a71ffe9f4b209982a9c329a2120d9f7c7509ea3`.
|
||||
|
||||
After producing captures with the prefixes used in the scripts, replay and count:
|
||||
|
||||
```sh
|
||||
node experiments/atomic-re-tools/verify-relations.mjs \
|
||||
.cache/atomic-re-tools .cache/atomic-re-tools/relations-verification-v2.json
|
||||
python experiments/atomic-re-tools/summarize-relations.py \
|
||||
.cache/atomic-re-tools .cache/atomic-re-tools/relations-summary-v2.json
|
||||
```
|
||||
|
||||
The second command requires tiktoken. The checked-in verifier intentionally binds
|
||||
to these exact artifacts and oracle addresses; it is not a generic correctness
|
||||
test for arbitrary zlib/SQLite versions. Capture Ghidra with
|
||||
`capture-function.mjs ghidra <zlib-path> 0x10c1e0 <prefix>`, and SQLite with
|
||||
`capture-kuna-structure.mjs <sqlite-path> 0x65180 raw <prefix>`. Caller-supplied
|
||||
engine paths and the existing resource wrapper are required as in round one.
|
||||
Capture prefixes and output paths must be fresh; exclusive writes preserve any
|
||||
existing evidence or caller-owned file.
|
||||
@@ -0,0 +1,112 @@
|
||||
# What might count as an RE atom?
|
||||
|
||||
This is an experiment record, not a claim that a new universal instruction set
|
||||
has been discovered. APIs already expose many relevant primitives. The open
|
||||
question here is how to make their evidence, cost and composition useful to an
|
||||
agent investigating a real program.
|
||||
|
||||
## Existing mechanisms and what they establish
|
||||
|
||||
- Ghidra has a program database, `ReferenceManager.getReferencesTo`, function-body
|
||||
ownership through `FunctionManager.getFunctionContaining`, listing objects and
|
||||
decompiler IR. Fine-grained **engine APIs already exist**. A poor MCP projection
|
||||
can still discard their metadata or require excessive round trips.
|
||||
Sources: [ReferenceManager](https://ghidra.re/ghidra_docs/api/ghidra/program/model/symbol/ReferenceManager.html),
|
||||
[FunctionManager](https://ghidra.re/ghidra_docs/api/ghidra/program/model/listing/FunctionManager.html).
|
||||
- Binary Ninja distinguishes code/data, type and variable references, including
|
||||
IL-level references. This shows why one undifferentiated `xref` is insufficient
|
||||
for some questions. It does not establish that an MCP interface using these
|
||||
APIs will be token-efficient or that another provider can reproduce the facts.
|
||||
Sources: [type/variable references](https://binary.ninja/2021/04/12/type-and-variable-xrefs.html),
|
||||
[BinaryView API](https://dev-api.binary.ninja/binaryninja.binaryview-module.html).
|
||||
- Kuna's Rust engine and SLEIGH compiler derive from Ghidra, while its standalone
|
||||
loader/analyzer tier and CLI are separate components. Port lineage does not
|
||||
imply identical observations or a shared live program database.
|
||||
Source: [NOTICE](https://github.com/Noelo-Lab/kuna/blob/fe01d6294eb0d1053ce343c8943646df88e143fc/NOTICE).
|
||||
- In **tested Kuna 1.833**, an `xrefs` command loads a program and builds an
|
||||
`XrefIndex` for its query. A different CLI invocation does that again.
|
||||
`strings` with attribution instead loads/indexes once and joins references over
|
||||
each string's byte extent. That is already internal composition, even though
|
||||
the CLI does not expose an arbitrary persistent query interface.
|
||||
Sources: [xref producer](https://github.com/Noelo-Lab/kuna/blob/fe01d6294eb0d1053ce343c8943646df88e143fc/decompiler/crates/kuna-cli/src/xrefs.rs),
|
||||
[string attribution](https://github.com/Noelo-Lab/kuna/blob/fe01d6294eb0d1053ce343c8943646df88e143fc/decompiler/crates/kuna-cli/src/strings.rs).
|
||||
|
||||
These are source-derived observations. They are not measurements of current
|
||||
upstream main, a guarantee of decompilation correctness, or a novelty claim.
|
||||
|
||||
## Granularity has at least three independent dimensions
|
||||
|
||||
| Dimension | Example | Consequence |
|
||||
| -------------- | ------------------------------------------------------ | --------------------------------------------------------------------- |
|
||||
| Fact unit | One typed reference with both endpoints and provenance | Can be retained and joined without reading unrelated C text |
|
||||
| Transport unit | A batch of references for several selectors | Amortizes headers and agent turns; need not change native work |
|
||||
| Compute unit | One image load and reference walk | Can serve many fact queries; may cost much more than one returned row |
|
||||
|
||||
Calling all three "atomic tools" hides the main optimization opportunity. A tiny
|
||||
getter can repeat a whole-program analysis. Conversely, one expensive shared
|
||||
analysis can provide precise, small, independently addressable facts.
|
||||
|
||||
## Candidate semantic core
|
||||
|
||||
1. **Identified objects and located observations**: artifact, image/address-space,
|
||||
byte span, decoded instruction, symbol, string candidate, recovered function,
|
||||
variable and type. Keep width, encoding, storage and confidence where needed
|
||||
to interpret a value. A recovered object is an analysis claim, not necessarily
|
||||
a source-language object that existed before compilation.
|
||||
2. **Relations and constraints**: maps-to, owns/contains, references, calls,
|
||||
flows-to, defines/uses, argument binding, type/storage association and branch
|
||||
guard. Keep source and destination, reference site, kind, supporting evidence
|
||||
and unknown/partial coverage. Do not collapse a call site into just a callee.
|
||||
3. **Evidence views**: retain the original observations and expose the requested
|
||||
fields, range or neighborhood. A readable decompilation is an important view;
|
||||
it does not replace a machine-grounded relation or prove original source.
|
||||
|
||||
Useful query operations resemble selection, projection, traversal, join and
|
||||
aggregation. This is a design vocabulary, **not a proposal to add a SQL/graph
|
||||
language or Code Mode**. Start with named, typed analyst questions; add more
|
||||
general execution only if repeated real workflows demonstrate the need.
|
||||
|
||||
## Composition rules that prevent false conclusions
|
||||
|
||||
- Entity identity and an observation's identity differ. Artifact/profile/location
|
||||
can identify a function candidate; `evidence_id` identifies one exact claim
|
||||
about it. Keep disagreeing claims instead of deduplicating them by address.
|
||||
- A session lease prevents using a handle after close/reopen. It is not a frozen
|
||||
analysis revision. In particular, Kuna's whole-project and early single-function
|
||||
reconstructions can differ under the same selected profile. Target admission
|
||||
prevents a target switch; it does not establish a consistent analysis snapshot.
|
||||
- A string spans bytes. A point-address xref and a reference anywhere inside that
|
||||
object answer different questions. A data-pointer reference is not automatically
|
||||
a runtime call, string consumption, or a reachable execution path.
|
||||
- Omission, an unsupported operation, cancellation, an unowned reference and an
|
||||
examined empty set are different outcomes. No empty set proves global absence
|
||||
unless the producing analysis actually establishes that coverage.
|
||||
- Field selection must not detach a value from the metadata that makes it
|
||||
meaningful. For example, the first prototype keeps REA's complete `string`
|
||||
metadata record as one selectable field, including its encoding status.
|
||||
- Expensive producer work should be shared with explicit identity, lifetime,
|
||||
invalidation and cancellation ownership. A client-side batch is insufficient
|
||||
evidence that native work was shared.
|
||||
|
||||
## What complex-program effectiveness would require
|
||||
|
||||
Navigation is only the first tier. Finding a string and its users is weaker than
|
||||
explaining why a branch rejects input or how data reaches a privileged operation.
|
||||
The next tier needs definition/use, call-argument and memory-flow relationships,
|
||||
with explicit analysis assumptions. The existing prototypes do **not** supply
|
||||
those relationships, and slicing C text cannot manufacture them.
|
||||
|
||||
For every harder trial, specify the analyst question, independent answer/evidence
|
||||
oracle, input artifact/profile, initial knowledge and allowed budget. Compare:
|
||||
|
||||
- whether the actual question was answered correctly;
|
||||
- decisive evidence recovered and relevant evidence omitted;
|
||||
- remaining unknowns and unsafe conclusions;
|
||||
- all calls, follow-up reads, failures, latency, CPU, family memory and estimated
|
||||
token traffic, including tool discovery and retained Evidence reads.
|
||||
|
||||
Source inspection can establish architectural opportunities. Scripted equivalent
|
||||
workflows can establish transport and engine costs. Neither alone establishes
|
||||
that a live agent reasons better on complex software; a controlled agent trial
|
||||
is a separate validation step. Likewise, byte/token reductions alone do not
|
||||
establish improved correctness or analyst productivity.
|
||||
@@ -0,0 +1,216 @@
|
||||
# First-round results
|
||||
|
||||
Continued in the [control-flow / SSA experiment](relations.md). The results below
|
||||
remain the fixed first-round baseline, including its failed mapping composition.
|
||||
|
||||
This is a completed, bounded first round of exploration, including failures.
|
||||
It does **not** establish that an agent solves complex RE tasks better than with
|
||||
the existing tools. No PR, merge, target execution or executable mutation occurred;
|
||||
Ghidra used its normal ephemeral analysis databases.
|
||||
|
||||
## Inputs and measurement
|
||||
|
||||
- REA base: `5b16f2431062eea6ffd245a9e05512790007cb13`; experiment commits retain
|
||||
the runnable code and tests. Node 24.18.0 / npm 11.16.0.
|
||||
- Kuna 1.833, executable SHA-256
|
||||
`d858918b0766435cd96b999a77f2d5a31a0cc4f07256421c883b5b9dc4dd58e4`.
|
||||
These results do not verify newer Kuna releases.
|
||||
- Ghidra 12.1.4, Temurin JDK 21.0.12.1+1, Java heap 1 GiB.
|
||||
- Installed zlib: `/usr/lib/x86_64-linux-gnu/libz.so.1.2.13`, SHA-256
|
||||
`7e2a72b4c4b38c61e6962de6e3f4a5e9ae692e732c68deead10a7ce2135a7f68`.
|
||||
- Installed SQLite: `/usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6`, SHA-256
|
||||
`2e6eef9a727f081f0d453b4e5e6cbd8b9ef8b6f86cbf7681cbad444d3b0b55c8`.
|
||||
- Real-provider jobs used the existing scope runner: one CPU, affinity to one
|
||||
allowed core, 4 GiB hard memory limit, early stop at 3 GiB. They ran serially.
|
||||
Build/type/test gates used separate bounded scopes, up to two CPUs / 4 GiB.
|
||||
- CPU is complete owned-cgroup CPU at payload exit. Memory is kernel
|
||||
`memory.peak` for the process family, **including charged file cache**, not RSS.
|
||||
The observer verified cleanup and resource counters while its scope was alive.
|
||||
- Tokens below are **offline `o200k_base` estimates**, tiktoken 0.14.0, over actual
|
||||
captured MCP payloads, including text and structured copies. They are not
|
||||
model-billed tokens. Model reasoning, tool selection, host rendering, cache
|
||||
billing and JSON-RPC framing were not measured.
|
||||
- Each table row is **one sample**, not a median or controlled causal estimate.
|
||||
OS caches were not flushed. Startup and native work varied substantially on
|
||||
this shared host. No universal latency improvement is established.
|
||||
|
||||
## Experiment 1: transport granularity
|
||||
|
||||
Task: case-sensitive literal search (`incorrect` in zlib, `malformed` in SQLite),
|
||||
select the first three provider-ordered string candidates, read at most four
|
||||
exact-start reference sites per candidate, then resolve their containing
|
||||
functions when advertised. Repeat in the same session to measure warm reads.
|
||||
Unsupported Kuna function resolution remains unknown, rather than guessed.
|
||||
|
||||
The four variants use the same application/provider observations. Native query
|
||||
planning was unchanged. `fine` reads individual facts; `batch` groups homogeneous
|
||||
reads; `workflow` composes those same fact methods in one call.
|
||||
|
||||
These captures measure the **v1 fact response shape**, introduced at `fa8d6a7a2`.
|
||||
Later selector/scope and transport-provenance clarifications are not retroactive
|
||||
performance measurements; rerun the harness to measure their current payload.
|
||||
|
||||
| Provider / target | Variant | Calls per task | Query payload tokens, first / warm | First query task ms | Warm task ms | Session CPU s | Peak family MiB |
|
||||
| ----------------- | -------- | -------------: | ---------------------------------: | ------------------: | -----------: | ------------: | --------------: |
|
||||
| Ghidra / zlib | existing | 10 | 41,565 / 41,565 | 34,926 | 429 | 50.00 | 1,444 |
|
||||
| Ghidra / zlib | fine | 10 | 23,483 / 23,483 | 34,260 | 120 | 47.37 | 1,376 |
|
||||
| Ghidra / zlib | batch | 3 | 20,114 / 20,114 | 75,202 | 218 | 52.63 | 1,360 |
|
||||
| Ghidra / zlib | workflow | 1 | 19,824 / 19,824 | 75,491 | 198 | 51.76 | 1,276 |
|
||||
| Kuna / zlib | existing | 4 | 5,638 / 5,638 | 4,079 | 44 | 16.12 | 905 |
|
||||
| Kuna / zlib | fine | 5 | 4,702 / 4,702 | 2,857 | 51 | 15.61 | 927 |
|
||||
| Kuna / zlib | batch | 3 | 3,774 / 3,774 | 2,843 | 59 | 15.14 | 911 |
|
||||
| Kuna / zlib | workflow | 1 | 3,509 / 3,509 | 1,376 | 11 | 14.34 | 922 |
|
||||
| Kuna / SQLite | existing | 4 | 6,039 / 6,039 | 5,874 | 24 | 19.15 | 994 |
|
||||
| Kuna / SQLite | fine | 5 | 4,753 / 4,753 | 13,058 | 47 | 20.40 | 989 |
|
||||
| Kuna / SQLite | batch | 3 | 3,829 / 3,829 | 25,868 | 217 | 25.85 | 1,022 |
|
||||
| Kuna / SQLite | workflow | 1 | 3,560 / 3,560 | 36,231 | 39 | 58.19 | 1,004 |
|
||||
|
||||
Session CPU/memory include MCP startup, catalog/status discovery, both tasks and
|
||||
close/cleanup. Query timers exclude the earlier `open_binary`; Ghidra lazily
|
||||
performs import/preparation during its first query. Whole-session elapsed times
|
||||
for existing/workflow were 50.67/104.15 s (Ghidra zlib), 32.46/16.63 s (Kuna zlib),
|
||||
and 19.46/159.81 s (Kuna SQLite). The SQLite workflow's `open_binary` itself took
|
||||
64.98 s versus 7.31 s in the existing-tools sample, before any experimental fact
|
||||
query. Cause is unresolved; neither host contention nor adapter overhead has
|
||||
been established as the explanation. Do not interpret these samples as a
|
||||
controlled speedup or slowdown attributable to atomization.
|
||||
|
||||
**Failure retained:** the first Ghidra batch attempt hit its 120 s provider
|
||||
request deadline during cold preparation. Its log ended in `ReaGhidraNoReturnFix`
|
||||
after reporting successful auto-analysis. The attempt cost 141.45 s elapsed,
|
||||
106.19 CPU s and 1,211 MiB peak; cleanup was verified and OOM kills were zero.
|
||||
The successful batch row excludes that failed attempt; any retry-cost comparison
|
||||
must add it. All twelve successful rows agreed on the selected strings,
|
||||
reference sites and function-resolution outcomes within each provider/target.
|
||||
That is adapter equivalence, not an independent proof of native discovery.
|
||||
|
||||
### Fixed context costs
|
||||
|
||||
The existing catalog had 207 tools; the prototype added five, retaining every
|
||||
existing tool. Actual default catalogs included output schemas: about 842k /
|
||||
843k estimated tokens. Removing only output schemas gives about 207k / 208k.
|
||||
This input-only projection is **not** an assertion about any particular model
|
||||
host: hosts can omit output schemas, deduplicate content or cache declarations.
|
||||
|
||||
For a session containing both completed tasks plus lifecycle/status traffic,
|
||||
the input-only-catalog estimate changed as follows:
|
||||
|
||||
| Provider / target | Existing tools | Workflow |
|
||||
| ----------------- | -------------: | -------: |
|
||||
| Ghidra / zlib | 319,907 | 277,458 |
|
||||
| Kuna / zlib | 245,476 | 242,249 |
|
||||
| Kuna / SQLite | 245,803 | 241,870 |
|
||||
|
||||
Thus a large reduction in query payload need not produce a large whole-task
|
||||
context reduction. Catalog/discovery and repeated envelopes deserve separate
|
||||
attention. This experiment has not yet tested a client that loads declarations
|
||||
on demand or an explicitly smaller, separately declared service surface.
|
||||
|
||||
## Experiment 2: compute sharing in native Kuna
|
||||
|
||||
Both plans first obtain a complete reference-free scan, validated by REA's
|
||||
existing string-fact owner. Then select three matching strings. The point plan
|
||||
runs `xrefs --to` separately; the inventory plan runs `strings` with attribution
|
||||
once, letting Kuna share its reference index. Full source JSON is retained
|
||||
locally, while only selected facts need be exposed to a caller.
|
||||
|
||||
| Target | Plan | Native commands | Elapsed s | CPU s | Peak family MiB |
|
||||
| ------ | ------------------ | --------------: | --------: | ----: | --------------: |
|
||||
| zlib | per address | 4 | 3.846 | 1.796 | 110.4 |
|
||||
| zlib | shared attribution | 2 | 2.838 | 1.341 | 114.4 |
|
||||
| SQLite | per address | 4 | 13.221 | 6.497 | 196.1 |
|
||||
| SQLite | shared attribution | 2 | 6.261 | 3.028 | 196.6 |
|
||||
|
||||
For these selected samples, both plans found one owner per string. Their scopes
|
||||
are nevertheless **different**: point references cover the first address;
|
||||
attribution covers any byte in the string extent. Attribution also does not
|
||||
return individual sites/kinds. These figures establish costs of the two real
|
||||
plans, not a universally equivalent-query speedup. They support investigating
|
||||
producer-level work sharing, with query selectivity and coverage kept explicit.
|
||||
This is a native-boundary experiment, not an MCP/model token measurement.
|
||||
|
||||
## Experiment 3: the point/object distinction changes correctness
|
||||
|
||||
`interior-reference.c` is compiled with `gcc -O2 -fno-pie -no-pie` and never run.
|
||||
Its `give_suffix` returns a pointer five bytes into `marker`. In this build,
|
||||
`nm` reports `marker=0x402020`, `give_suffix=0x401120`; `objdump` independently
|
||||
shows `mov $0x402025,%eax` followed by `ret` in `give_suffix`.
|
||||
|
||||
- Exact-start xref query: zero references, no owner.
|
||||
- Extent attribution: one reference, owner `give_suffix` at `0x401120`.
|
||||
|
||||
`verify-interior.py` checks the compiled symbols, input and retained-source
|
||||
digests, and both native results. This is a genuine counterexample to treating
|
||||
"no xref to the start address" as "the string has no users". Neither tool was
|
||||
wrong about its own scope. The compositional representation must carry the
|
||||
object span and reference offset/scope instead of silently equating the questions.
|
||||
|
||||
## Experiment 4: existing relations do not always bridge into code
|
||||
|
||||
On real zlib, the three selected string references belong to `inflate` at
|
||||
`0xc1e0`; their reference instructions are at `0xded7`, `0xdc6a` and `0xcd05`.
|
||||
The native single-function JSON contains 52,431 characters / 1,620 code lines,
|
||||
1,167 mapping records and 1,568 mapped instruction addresses. **None of those
|
||||
three reference instructions occurs in the printer's line mappings.**
|
||||
|
||||
The exact-address composition therefore returns **unknown**, not a guessed
|
||||
nearby line. The mapped-code run cost 8.387 s / 8.091 CPU s / 202.0 MiB, including
|
||||
discovery/references and decompilation. It did not finish the analyst question
|
||||
"which condition triggers the error?" and must not count as a successful
|
||||
context-saving answer. Single-function work uses REA's owned command deadline
|
||||
and the native `--max-fn-memory` option shown by the tested CLI's actual help.
|
||||
|
||||
The retained C text does contain the error literal at line 164, but a small
|
||||
fixed window there omits the earlier check around lines 122–124. Literal search
|
||||
and a larger manual read can help, but they are **reconstructed-text evidence**,
|
||||
not the missing machine-to-source relation. This is not proof that Kuna's line
|
||||
mapping is buggy: the contract does not promise every contributing instruction
|
||||
survives transformation into an emitted line.
|
||||
|
||||
The next useful relations to test are instruction/block ownership, control-flow
|
||||
guards and definition/use or contributing-origin edges across transformations.
|
||||
For a stronger zlib question, an independent expected mechanism is the FCHECK
|
||||
constraint in [RFC 1950 §2.2](https://www.rfc-editor.org/rfc/rfc1950.html#section-2.2):
|
||||
the network-order CMF/FLG value must be divisible by 31. Testing an agent's
|
||||
explanation against that mechanism would be a separate trial; this experiment
|
||||
does not claim to have completed that trial.
|
||||
|
||||
## Validation
|
||||
|
||||
- All **191 Kuna regression cases in 20 files passed with one worker**; the
|
||||
owning reference parser's 18 cases include typed endpoints, distinct kinds,
|
||||
unknown ownership, alias exclusion and late malformed rows. A prior two-worker
|
||||
CPU-limited run had three timing-sensitive failures in existing project/export
|
||||
tests. Those tests were not edited; the isolated cause is not established.
|
||||
- Nine experiment boundary cases passed, including stale artifact/profile/lease,
|
||||
partial batches, duplicate point lookups, paging, unsupported ownership,
|
||||
unmapped/unsafe addresses and invalid line locations.
|
||||
- Typecheck passed with one checker. Changed experiment files passed formatter
|
||||
and lint; the existing graph parser's complexity warning remains unrelated.
|
||||
- A real Kuna MCP run with the actual output-schema **off** profile passed
|
||||
paging, duplicate-selector, negative/unknown, retained-Evidence, foreign-context
|
||||
and expired-lease checks. Its live catalog had 212 tools and zero output
|
||||
schemas. Verification runs are excluded from the performance table.
|
||||
- The v2 facade was also exercised through a real Kuna MCP client: edges carry
|
||||
explicit point scope and location endpoints, caller selectors stay separate,
|
||||
and successful/error results retain the canonical untrusted-target marker.
|
||||
The same six additional verification groups passed. This v2 verification is
|
||||
not relabelled as a new performance comparison.
|
||||
- All real-provider measurements reported verified cleanup and zero OOM kills.
|
||||
No default parallel-suite stability, live-model reasoning improvement, general
|
||||
high-address code mapping, overlay support or newer Kuna compatibility is claimed.
|
||||
|
||||
## What this round supports
|
||||
|
||||
- A coherent fact includes the metadata needed to interpret it. One-field
|
||||
getters are not necessarily useful atoms.
|
||||
- Fact, transport and compute units are different. Batches/workflows can share
|
||||
one fact implementation while the producer independently shares analysis.
|
||||
- Identity and evidence are necessary, but scope and uncertainty determine
|
||||
whether joins answer the analyst's actual question.
|
||||
- Complex-program effectiveness remains **unproven**. The experiments found a
|
||||
real coverage distinction and a real composition gap, not a universal RE API.
|
||||
|
||||
Raw captures, projects, binaries and counters remain ignored under
|
||||
`.cache/atomic-re-tools/`. `summarize.py` checks selected-answer equivalence,
|
||||
payload-exit cleanup and OOM counters, and includes failed runs separately.
|
||||
@@ -0,0 +1,40 @@
|
||||
"""Run comparable real-provider tasks serially using REA's existing scope owner."""
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--provider", choices=["kuna", "ghidra"], required=True)
|
||||
parser.add_argument("--binary", required=True)
|
||||
parser.add_argument("--pattern", required=True)
|
||||
parser.add_argument("--output", required=True)
|
||||
parser.add_argument("--variants", nargs="+", choices=["baseline", "fine", "batch", "workflow"],
|
||||
default=["baseline", "fine", "batch", "workflow"])
|
||||
options = parser.parse_args()
|
||||
root = Path(options.output).resolve()
|
||||
root.mkdir(parents=True, exist_ok=True)
|
||||
completed = []
|
||||
for variant in options.variants:
|
||||
prefix = root / (options.provider + "-" + variant)
|
||||
command = [sys.executable, "experiments/analysis-efficiency/resource-run.py", "--observe-payload-exit", "--",
|
||||
"node", "experiments/atomic-re-tools/benchmark.mjs", options.provider, variant,
|
||||
options.binary, options.pattern, str(prefix)]
|
||||
with Path(str(prefix) + ".log").open("w") as output:
|
||||
result = subprocess.run(command, stdout=output, stderr=subprocess.STDOUT, check=False)
|
||||
if result.returncode:
|
||||
raise SystemExit(f"{variant} failed ({result.returncode}); retained {prefix}.log")
|
||||
report = json.loads(Path(str(prefix) + ".report.json").read_text())
|
||||
if report["failures"] or report["verification_only"]:
|
||||
raise SystemExit("Not a successful comparable performance sample: " + str(prefix))
|
||||
completed.append(str(prefix))
|
||||
(root / (options.provider + "-completed.json")).write_text(json.dumps(completed, indent=2) + "\n")
|
||||
print(json.dumps({"completed": variant, "provider": options.provider,
|
||||
"query_ms": [round(row["elapsed_ms"], 2) for row in report["outcomes"]]}), flush=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,202 @@
|
||||
// Experimental tools are added to the complete production catalog, never shipped.
|
||||
import { serveStdio } from "@modelcontextprotocol/server/stdio";
|
||||
import { z } from "zod";
|
||||
import { run } from "../../dist/main.js";
|
||||
import { createServer } from "../../dist/server/createServer.js";
|
||||
import { createEvidence } from "../../dist/domain/evidence.js";
|
||||
import { projectAnalysisError } from "../../dist/domain/analysisErrorProjection.js";
|
||||
import { toolContract } from "../../dist/contracts/toolContracts.js";
|
||||
import { TARGET_CONTENT_RESULT_META } from "../../dist/server/toolResultEncoding.js";
|
||||
import { expandProviderCapabilities } from "../../dist/contracts/toolOutputSchemaPrimitives.js";
|
||||
import { FactQueries, contextFor } from "./facts.mjs";
|
||||
|
||||
const contextSchema = z.strictObject({ ref: z.string(), lease: z.string() });
|
||||
const location = z.strictObject({
|
||||
kind: z.literal("location"),
|
||||
address: z.string().min(1),
|
||||
});
|
||||
const stringId = z.strictObject({
|
||||
kind: z.literal("string"),
|
||||
address: z.string().min(1),
|
||||
encoding: z.string().nullable(),
|
||||
});
|
||||
const paging = {
|
||||
offset: z.number().int().nonnegative().default(0),
|
||||
limit: z.number().int().positive().default(20),
|
||||
};
|
||||
const stringsInput = z.strictObject({
|
||||
query: toolContract("search_strings").inputSchema,
|
||||
...paging,
|
||||
fields: z.array(z.enum(["value", "string"])).default(["value"]),
|
||||
});
|
||||
const contracts = [
|
||||
[
|
||||
"experiment_find_strings",
|
||||
"Find string facts with scoped identities, selected fields, paging and retained source Evidence. Experimental; provider discovery coverage remains unknown.",
|
||||
stringsInput,
|
||||
"findStrings",
|
||||
],
|
||||
[
|
||||
"experiment_incoming_refs",
|
||||
"Read incoming reference facts for a batch of string identities from this session. Preserve per-target failures and unknown reference kinds. Reuse the context ref and lease returned by find_strings.",
|
||||
z.strictObject({
|
||||
context: contextSchema,
|
||||
targets: z.array(stringId),
|
||||
...paging,
|
||||
}),
|
||||
"incomingReferences",
|
||||
],
|
||||
[
|
||||
"experiment_resolve_functions",
|
||||
"Resolve a batch of instruction locations to provider-analyzed containing functions. Unsupported or failed resolution stays unknown; never guess the nearest symbol.",
|
||||
z.strictObject({ context: contextSchema, locations: z.array(location) }),
|
||||
"resolveFunctions",
|
||||
],
|
||||
[
|
||||
"experiment_trace_string_uses",
|
||||
"Compose the same string, incoming reference and containing-function facts in one call. Page limits and partial outcomes remain explicit.",
|
||||
stringsInput.extend({
|
||||
reference_limit: z.number().int().positive().default(20),
|
||||
include_functions: z.boolean().default(true),
|
||||
}),
|
||||
"traceStringUses",
|
||||
],
|
||||
];
|
||||
|
||||
const encode = (value) => ({
|
||||
_meta: TARGET_CONTENT_RESULT_META,
|
||||
content: [{ type: "text", text: JSON.stringify(value) }],
|
||||
structuredContent: value,
|
||||
...(value.status === "error" ? { isError: true } : {}),
|
||||
});
|
||||
|
||||
const createFacts = (session, analysis, signal) => {
|
||||
const status = session.status();
|
||||
const context = contextFor(
|
||||
session.activeTarget().sha256,
|
||||
session.analysisProfile("search_strings"),
|
||||
status.analysis_run.run_id,
|
||||
);
|
||||
const capabilities = expandProviderCapabilities(status.capabilities);
|
||||
const facts = new FactQueries({
|
||||
supports: (operation) =>
|
||||
capabilities.some(
|
||||
(capability) =>
|
||||
capability.operation === operation && capability.available,
|
||||
),
|
||||
context: () => context,
|
||||
observe: async (operation, parameters) => {
|
||||
const outcome = await analysis.execute(
|
||||
operation,
|
||||
toolContract(operation).inputSchema.parse(parameters),
|
||||
{ signal: signal },
|
||||
);
|
||||
if (!outcome.ok)
|
||||
return {
|
||||
ok: false,
|
||||
error: projectAnalysisError(outcome.error),
|
||||
};
|
||||
const execution = outcome.value;
|
||||
const evidence = createEvidence(
|
||||
execution.subject ?? session.activeTarget(),
|
||||
execution.provider,
|
||||
{
|
||||
operation,
|
||||
parameters,
|
||||
result: execution.result,
|
||||
analysisProfile: execution.analysisProfile,
|
||||
rawResult: execution.rawResult,
|
||||
limitations: execution.limitations,
|
||||
relationshipCoverage: execution.relationshipCoverage,
|
||||
locations: execution.locations,
|
||||
},
|
||||
);
|
||||
const recorded = session.recordEvidence(evidence);
|
||||
return recorded.ok
|
||||
? { ok: true, value: evidence }
|
||||
: {
|
||||
ok: false,
|
||||
error: projectAnalysisError(recorded.error),
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
return facts;
|
||||
};
|
||||
|
||||
await run({
|
||||
env: process.env,
|
||||
serve: serveStdio,
|
||||
writeStderr: (value) => process.stderr.write(value),
|
||||
setExitCode: (value) => {
|
||||
process.exitCode = value;
|
||||
},
|
||||
registerShutdown: (handler) => {
|
||||
const registrations = [
|
||||
[process, "SIGINT"],
|
||||
[process, "SIGTERM"],
|
||||
[process.stdin, "end"],
|
||||
[process.stdin, "close"],
|
||||
];
|
||||
for (const [emitter, event] of registrations) emitter.once(event, handler);
|
||||
return () => {
|
||||
for (const [emitter, event] of registrations) emitter.off(event, handler);
|
||||
};
|
||||
},
|
||||
createServer(source, options) {
|
||||
const server = createServer(source, options);
|
||||
const session = source.session;
|
||||
for (const [name, description, inputSchema, method] of contracts) {
|
||||
server.registerTool(
|
||||
name,
|
||||
{ description, inputSchema, annotations: { readOnlyHint: true } },
|
||||
async (input, request) => {
|
||||
const admitted = await session.withAdmittedAnalysis(
|
||||
name,
|
||||
request.mcpReq.signal,
|
||||
async (analysis) => {
|
||||
const facts = createFacts(
|
||||
session,
|
||||
analysis,
|
||||
request.mcpReq.signal,
|
||||
);
|
||||
return facts[method](inputSchema.parse(input));
|
||||
},
|
||||
);
|
||||
return encode(
|
||||
admitted.ok
|
||||
? admitted.value
|
||||
: {
|
||||
status: "error",
|
||||
error: projectAnalysisError(admitted.error),
|
||||
},
|
||||
);
|
||||
},
|
||||
);
|
||||
}
|
||||
server.registerTool(
|
||||
"experiment_read_evidence",
|
||||
{
|
||||
description:
|
||||
"Read complete retained source Evidence explicitly by evidence_id while the existing session record owner retains it. Closing the binary expires these records.",
|
||||
inputSchema: z.strictObject({ evidence_id: z.string().min(1) }),
|
||||
annotations: { readOnlyHint: true },
|
||||
},
|
||||
(input) => {
|
||||
const evidence = session.evidenceById(input.evidence_id);
|
||||
return encode(
|
||||
evidence === undefined
|
||||
? {
|
||||
status: "error",
|
||||
error: {
|
||||
code: "not_found",
|
||||
message: "Evidence is not retained in this server.",
|
||||
},
|
||||
}
|
||||
: { status: "ok", evidence },
|
||||
);
|
||||
},
|
||||
);
|
||||
return server;
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,94 @@
|
||||
"""Representation estimates and a complete ledger of this round's captured runs.
|
||||
|
||||
Run with the tokenizer environment from the first round. This does not measure
|
||||
model billing, model reasoning, or equivalent completed agent tasks.
|
||||
"""
|
||||
import importlib.metadata
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
import tiktoken
|
||||
|
||||
|
||||
def load(path):
|
||||
return json.loads(path.read_text())
|
||||
|
||||
|
||||
def resource(path):
|
||||
records = [json.loads(line) for line in path.read_text().splitlines()
|
||||
if line.startswith('{"resource_scope":')]
|
||||
assert len(records) == 1, path
|
||||
record = records[0]
|
||||
observation = record['payload_resource_observation']
|
||||
assert record['payload_resource_observation_verified'] is True
|
||||
assert observation['payload_cleanup_verified'] is True
|
||||
return {
|
||||
'log': str(path), 'elapsed_s': record['elapsed_seconds'],
|
||||
'cpu_s': observation['cpu_stat']['usage_usec'] / 1_000_000,
|
||||
'family_peak_mib_including_file_cache': observation['memory_peak_bytes'] / 2**20,
|
||||
'exit_code': observation['payload_exit_code'],
|
||||
'oom_kills': observation['resource_events']['memory.events']['oom_kill'],
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
root, output = map(Path, sys.argv[1:])
|
||||
encoder = tiktoken.get_encoding('o200k_base')
|
||||
|
||||
def tokens(value):
|
||||
return len(encoder.encode(value, disallowed_special=()))
|
||||
|
||||
def encoded(value):
|
||||
return json.dumps(value, ensure_ascii=False, separators=(',', ':'))
|
||||
|
||||
rows = []
|
||||
for name, prefix, query_file in [
|
||||
('zlib', 'relations-kuna-zlib-raw-v2', 'relations-zlib-query-v2.json'),
|
||||
('sqlite', 'relations-kuna-sqlite-raw', 'relations-sqlite-query-v2.json'),
|
||||
]:
|
||||
capture = load(root / (prefix + '.report.json'))
|
||||
query = load(root / query_file)
|
||||
raw = Path(capture['calls'][0]['file']).read_text()
|
||||
code = raw.split('\n0\nBasic Block', 1)[0].strip()
|
||||
answers = query['runs'][0]['answers']
|
||||
rows.append({
|
||||
'target': name, 'artifact_sha256': capture['target_sha256'],
|
||||
'full_code_tokens': tokens(code), 'full_code_bytes': len(code.encode()),
|
||||
'full_native_dump_tokens': tokens(raw),
|
||||
'selected_answer_tokens_with_source': [tokens(encoded({'source': query['source'], 'answer': a})) for a in answers],
|
||||
'batch_answer_tokens_with_source': tokens(encoded({'source': query['source'], 'answers': answers})),
|
||||
'parse_index_ms': query['parse_index_ms'],
|
||||
'projection_runs': [{k: v for k, v in r.items() if k != 'answers'} for r in query['runs']],
|
||||
'snapshot_coverage': query['coverage'],
|
||||
})
|
||||
ghidra = root / 'relations-ghidra-zlib-retained'
|
||||
wire = load(ghidra.with_suffix('.wire.json'))
|
||||
report = load(ghidra.with_suffix('.report.json'))
|
||||
resources = [resource(p) for p in sorted(root.glob('relations-*.log'))]
|
||||
summary = {
|
||||
'encoding': 'o200k_base', 'tiktoken': importlib.metadata.version('tiktoken'),
|
||||
'representation_estimates': rows,
|
||||
'ghidra_retained_workflow': {
|
||||
'wire_tokens_including_errors_and_exports': tokens(encoded(wire)),
|
||||
'input_only_catalog_tokens': tokens(encoded(load(ghidra.with_suffix('.catalog.json')))),
|
||||
'calls': report['calls'], 'analyst_task_completed': False,
|
||||
'reason': 'Late literal not covered by the producer-truncated high-p-code prefix.',
|
||||
},
|
||||
'run_ledger': resources,
|
||||
'scope': [
|
||||
'Raw captures are fresh native analyses; OS caches were not flushed.',
|
||||
'Reused-index runs reuse parsed facts, not a warm native analysis.',
|
||||
'Token counts cover the stated representations. The native probe has no new MCP catalog.',
|
||||
'No controlled agent A/B task, model reasoning tokens or model latency measured.',
|
||||
'Discovery, old xrefs, manual oracle construction, objdump and tokenizer CPU are outside the native/projection runs.',
|
||||
'The ledger retains failed and superseded probes rather than excluding their costs.',
|
||||
],
|
||||
}
|
||||
with output.open('x') as destination:
|
||||
destination.write(json.dumps(summary, indent=2) + '\n')
|
||||
print(json.dumps({'output': str(output), 'targets': len(rows), 'runs': len(resources)}))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,84 @@
|
||||
"""Compare completed equivalent tasks; token counts are offline estimates only."""
|
||||
import argparse
|
||||
import importlib.metadata
|
||||
import json
|
||||
from pathlib import Path
|
||||
import tiktoken
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("roots", nargs="+")
|
||||
options = parser.parse_args()
|
||||
encoding = tiktoken.get_encoding("o200k_base")
|
||||
|
||||
def tokens(value):
|
||||
return len(encoding.encode(json.dumps(value, ensure_ascii=False, separators=(",", ":")), disallowed_special=()))
|
||||
|
||||
results = []
|
||||
failed_runs = []
|
||||
equivalence = {}
|
||||
for root in map(Path, options.roots):
|
||||
for report_path in sorted(root.glob("*.report.json")):
|
||||
report = json.loads(report_path.read_text())
|
||||
if report["verification_only"]:
|
||||
continue
|
||||
prefix = str(report_path).removesuffix(".report.json")
|
||||
catalog = json.loads(Path(prefix + ".catalog.json").read_text())
|
||||
wire = json.loads(Path(prefix + ".wire.json").read_text())
|
||||
resources = [json.loads(line) for line in Path(prefix + ".log").read_text().splitlines()
|
||||
if line.startswith('{"resource_scope":')][-1]
|
||||
assert resources["payload_resource_observation_verified"], prefix
|
||||
counters = resources["payload_resource_observation"]
|
||||
assert counters["payload_cleanup_verified"], prefix
|
||||
assert counters["resource_events"]["memory.events"]["oom_kill"] == 0, prefix
|
||||
if report["failures"]:
|
||||
failed_runs.append({"provider": report["provider"], "variant": report["variant"], "target": report["binary"],
|
||||
"elapsed_seconds": resources["elapsed_seconds"], "cpu_seconds": round(counters["cpu_stat"]["usage_usec"] / 1e6, 3),
|
||||
"peak_process_family_cgroup_mib": round(counters["memory_peak_bytes"] / 1024 ** 2, 1),
|
||||
"failure_count": len(report["failures"]), "retained_prefix": prefix,
|
||||
"cleanup_verified": True, "oom_kills": 0})
|
||||
continue
|
||||
assert counters["payload_exit_code"] == 0, prefix
|
||||
key = (report["provider"], report["target_sha256"], report["pattern"])
|
||||
outcome = [{name: row[name] for name in ["strings", "references", "functions"]} for row in report["outcomes"]]
|
||||
if key in equivalence:
|
||||
assert outcome == equivalence[key], "Equivalent-task answers differ: " + prefix
|
||||
else:
|
||||
equivalence[key] = outcome
|
||||
phase_traffic = {phase: [{"request": row["request"], "response": row["response"]} for row in wire if row["phase"] == phase]
|
||||
for phase in ["cold", "warm", "lifecycle"]}
|
||||
used = {row["request"]["name"] for row in wire}
|
||||
input_only_catalog = {**catalog, "tools": [{key: value for key, value in tool.items() if key != "outputSchema"} for tool in catalog["tools"]]}
|
||||
task_traffic = [row for row in wire if row["phase"] in ["cold", "warm", "lifecycle"]]
|
||||
rows = {phase: {"calls": len(traffic), "estimated_tokens": tokens(traffic)} for phase, traffic in phase_traffic.items()}
|
||||
results.append({
|
||||
"provider": report["provider"], "variant": report["variant"], "target_sha256": report["target_sha256"],
|
||||
"pattern": report["pattern"], "catalog_tools": len(catalog["tools"]), "catalog_estimated_tokens": tokens(catalog),
|
||||
"observed_output_schema_count": sum("outputSchema" in tool for tool in catalog["tools"]),
|
||||
"input_only_catalog_estimated_tokens": tokens(input_only_catalog),
|
||||
"phases": rows,
|
||||
"query_elapsed_ms": {row["phase"]: round(row["elapsed_ms"], 2) for row in report["outcomes"]},
|
||||
"session_elapsed_seconds_including_two_tasks": resources["elapsed_seconds"],
|
||||
"session_cpu_seconds_including_two_tasks": round(counters["cpu_stat"]["usage_usec"] / 1e6, 3),
|
||||
"peak_process_family_cgroup_mib": round(counters["memory_peak_bytes"] / 1024 ** 2, 1),
|
||||
"whole_session_estimated_tokens": tokens(catalog) + tokens([{"request": row["request"], "response": row["response"]} for row in task_traffic]),
|
||||
"whole_session_with_input_only_catalog_estimated_tokens": tokens(input_only_catalog) + tokens([{"request": row["request"], "response": row["response"]} for row in task_traffic]),
|
||||
"hypothetical_selected_declarations_estimated_tokens": tokens([tool for tool in catalog["tools"] if tool["name"] in used]),
|
||||
"answer": {"strings": len(outcome[0]["strings"]), "reference_sites": sum(len(row["items"]) for row in outcome[0]["references"]), "locations": len(outcome[0]["functions"])},
|
||||
"cleanup_verified": True, "oom_kills": 0,
|
||||
})
|
||||
summary = {
|
||||
"token_method": {"encoding": "o200k_base", "tiktoken_version": importlib.metadata.version("tiktoken"),
|
||||
"estimated_not_billed": True, "scope": "Captured MCP tool catalog and request/response payloads, including text and structured copies; excludes JSON-RPC framing, model reasoning, host prompts and model tool selection."},
|
||||
"memory_method": "Kernel memory.peak for the complete owned process-family cgroup, including charged file cache; not summed RSS. Payload-exit counters verified while the observer remains alive.",
|
||||
"workload": "One cold and one warm selected navigation task per fresh session, plus startup, discovery/status, close and cleanup. Additional --verify runs excluded.",
|
||||
"equivalence": "All included variants agree on selected strings, reference sites and per-location function resolutions within each provider/target/query. This is not cross-provider truth or a live-agent reasoning evaluation.",
|
||||
"results": results,
|
||||
"failed_runs": failed_runs,
|
||||
}
|
||||
print(json.dumps(summary, indent=2))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,26 @@
|
||||
"""Independent compiled-symbol oracle for the interior-only reference experiment."""
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
binary, point_report, extent_report = map(Path, sys.argv[1:])
|
||||
point, extent = [json.loads(path.read_text()) for path in [point_report, extent_report]]
|
||||
symbols = {}
|
||||
for line in subprocess.check_output(["nm", "--defined-only", "--format=posix", str(binary)], text=True).splitlines():
|
||||
fields = line.split()
|
||||
if fields[0] in ["marker", "give_suffix"]:
|
||||
symbols[fields[0]] = "0x" + format(int(fields[2], 16), "x")
|
||||
assert set(symbols) == {"marker", "give_suffix"}
|
||||
for report in [point, extent]:
|
||||
assert not report["failures"]
|
||||
assert report["facts"]["artifact_sha256"] == hashlib.sha256(binary.read_bytes()).hexdigest()
|
||||
for call in report["calls"]:
|
||||
assert hashlib.sha256(Path(call["file"]).read_bytes()).hexdigest() == call["sha256"]
|
||||
assert report["facts"]["results"][0]["string"]["address"] == symbols["marker"]
|
||||
assert point["facts"]["results"][0]["xrefs_count"] == 0
|
||||
assert point["facts"]["results"][0]["functions"] == []
|
||||
assert extent["facts"]["results"][0]["xrefs_count"] == 1
|
||||
assert [fn["address"] for fn in extent["facts"]["results"][0]["functions"]] == [symbols["give_suffix"]]
|
||||
print(json.dumps({"verified": True, "point_references": 0, "extent_references": 1, "symbols": symbols}))
|
||||
@@ -0,0 +1,163 @@
|
||||
// Replay real captures against independent machine-byte and arithmetic checks.
|
||||
import assert from "node:assert/strict";
|
||||
import { createHash } from "node:crypto";
|
||||
import { readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { mappedCode } from "./mapped-code.mjs";
|
||||
import { parseKunaSsaDump } from "./kuna-ssa.mjs";
|
||||
import { queryLiteralUses } from "./query-ssa.mjs";
|
||||
|
||||
const [root, output] = process.argv.slice(2);
|
||||
assert.ok(root && output);
|
||||
const json = async (name) =>
|
||||
JSON.parse(await readFile(join(root, name), "utf8"));
|
||||
const sha256 = (value) => createHash("sha256").update(value).digest("hex");
|
||||
const zlib = await json("relations-zlib-query-v2.json");
|
||||
const sqlite = await json("relations-sqlite-query-v2.json");
|
||||
assert.deepEqual(zlib.runs[0].answers, zlib.runs[1].answers);
|
||||
assert.deepEqual(sqlite.runs[0].answers, sqlite.runs[1].answers);
|
||||
for (const query of [zlib, sqlite]) {
|
||||
const source = await readFile(query.source.raw.file, "utf8");
|
||||
assert.equal(sha256(source), query.source.raw.sha256);
|
||||
const snapshot = parseKunaSsaDump(source);
|
||||
for (const answer of query.runs[0].answers)
|
||||
assert.deepEqual(queryLiteralUses(snapshot, answer.literal), answer);
|
||||
}
|
||||
const old = await json("native-zlib-code-fixed.report.json");
|
||||
assert.equal(zlib.source.artifact_sha256, old.facts.artifact_sha256);
|
||||
assert.deepEqual(zlib.source.engine, old.facts.engine);
|
||||
const bodySource = old.calls.find((call) => call.arguments[0] === "decompile");
|
||||
const body = await readFile(bodySource.file, "utf8");
|
||||
assert.equal(sha256(body), bodySource.sha256);
|
||||
const bodyDocument = JSON.parse(body);
|
||||
const raw = await readFile(zlib.source.raw.file, "utf8");
|
||||
assert.equal(sha256(raw), zlib.source.raw.sha256);
|
||||
assert.equal(
|
||||
raw.split("\n0\nBasic Block", 1)[0].trim(),
|
||||
bodyDocument.functions[0].code.trim(),
|
||||
"Cross-surface emitted C differs",
|
||||
);
|
||||
const native = async () => ({
|
||||
input: bodyDocument.binary,
|
||||
stdout: { text: body },
|
||||
});
|
||||
const mappings = [];
|
||||
for (const [i, expectedLine] of [164, 551, 653].entries()) {
|
||||
const answer = zlib.runs[0].answers[i];
|
||||
const original = old.facts.results[i];
|
||||
assert.equal(answer.literal, original.string.address);
|
||||
const failed = await mappedCode(native, original);
|
||||
assert.equal(failed.status, "unknown");
|
||||
const sites = answer.direct_uses.map((id) => ({
|
||||
from: answer.operations.find((op) => op.id === id).address,
|
||||
}));
|
||||
const mapped = await mappedCode(native, {
|
||||
functions: original.functions,
|
||||
sites,
|
||||
});
|
||||
assert.equal(mapped.status, "ok");
|
||||
assert.deepEqual(mapped.exact_mapped_lines, [expectedLine]);
|
||||
mappings.push({
|
||||
literal: answer.literal,
|
||||
original_sites: original.sites.map((s) => s.from),
|
||||
semantic_sites: sites.map((s) => s.from),
|
||||
line: expectedLine,
|
||||
});
|
||||
}
|
||||
const header = zlib.runs[0].answers[0];
|
||||
assert.deepEqual(
|
||||
header.control_edges.map((e) => [e.from, e.to, e.polarity]),
|
||||
[
|
||||
[182, 446, "false"],
|
||||
[183, 446, "false"],
|
||||
],
|
||||
);
|
||||
assert.equal(header.operations.length, 21);
|
||||
assert.equal(header.coverage.dependencies.boundaries.length, 2);
|
||||
assert.ok(header.operations.some((op) => op.statement.includes("& #0x1:4")));
|
||||
assert.ok(header.operations.some((op) => op.statement.includes("/ #0x1f")));
|
||||
const match = sqlite.runs[0].answers[0];
|
||||
assert.equal(
|
||||
match.operations.find((op) => op.id === match.direct_uses[0]).kind,
|
||||
"call",
|
||||
);
|
||||
assert.ok(match.operations.some((op) => op.statement.includes("!= #0x1:4")));
|
||||
assert.deepEqual(
|
||||
match.control_edges.map((e) => e.polarity),
|
||||
["false"],
|
||||
);
|
||||
const assemblyProofs = [];
|
||||
for (const [file, patterns] of [
|
||||
[
|
||||
"relations-zlib-guard.objdump.txt",
|
||||
[/d9ec:.*je\s+ded7/u, /da32:.*jne\s+ded7/u, /0x842108421084211/u],
|
||||
],
|
||||
[
|
||||
"relations-sqlite-error.objdump.txt",
|
||||
[
|
||||
/65239:.*cmp\s+\$0x1,%ebx/u,
|
||||
/6523c:.*je\s+65281/u,
|
||||
/65284:.*lea.*11dcd8/u,
|
||||
],
|
||||
],
|
||||
]) {
|
||||
const text = await readFile(join(root, file), "utf8");
|
||||
for (const pattern of patterns) assert.match(text, pattern);
|
||||
assemblyProofs.push({ file, sha256: sha256(text) });
|
||||
}
|
||||
// Exhaustive ordinary 16-bit header domain. This executes arithmetic, not targets.
|
||||
for (let n = 0n; n < 65536n; n++) {
|
||||
const high = (n * 0x842108421084211n) >> 64n;
|
||||
const machineQ = (((n - high) >> 1n) + high) >> 4n;
|
||||
const ssaQ = (((n - n / 31n) >> 1n) + n / 31n) >> 4n;
|
||||
assert.equal(machineQ, n / 31n);
|
||||
assert.equal(ssaQ, machineQ);
|
||||
assert.equal(n !== machineQ * 31n, n % 31n !== 0n);
|
||||
}
|
||||
const ghidra = await json("relations-ghidra-zlib-retained.bundle.json");
|
||||
const flow = ghidra.records.find((r) => r.normalized_result?.native_value_flow)
|
||||
?.normalized_result.native_value_flow;
|
||||
assert.equal(flow.truncated, true);
|
||||
assert.equal(flow.operations.length, 3000);
|
||||
assert.equal(flow.known_omitted_edges, 239);
|
||||
assert.ok(
|
||||
!flow.operations.some((op) =>
|
||||
op.inputs.some((v) => v.constant_hex === "118fc4"),
|
||||
),
|
||||
);
|
||||
await writeFile(
|
||||
output,
|
||||
JSON.stringify(
|
||||
{
|
||||
mappings,
|
||||
zlib_header_selected_operations: header.operations.length,
|
||||
sqlite_selected_operations: match.operations.length,
|
||||
assemblyProofs,
|
||||
arithmetic_oracle: {
|
||||
tested_values: 65536,
|
||||
domain: "ordinary 16-bit header value",
|
||||
target_executed: false,
|
||||
},
|
||||
ghidra: {
|
||||
retained_operations: 3000,
|
||||
known_omitted_edges: 239,
|
||||
late_literal_coverage: "unknown due to truncation",
|
||||
},
|
||||
limitations: [
|
||||
"Printed-SSA observations are derived, not runtime behavior.",
|
||||
"Entry state, complete control dependencies, phi predecessor and memory alias bindings remain unknown.",
|
||||
"Source-version agreement does not prove an exact Debian source/build match.",
|
||||
],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
{ flag: "wx" },
|
||||
);
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
output,
|
||||
mapped_errors: mappings.length,
|
||||
arithmetic_values: 65536,
|
||||
}),
|
||||
);
|
||||
@@ -159,6 +159,28 @@ const xrefsDocument = z.object({
|
||||
xrefs: z.array(z.unknown()),
|
||||
});
|
||||
|
||||
/** One owning interpretation of native endpoints, identities and alias scope. */
|
||||
const parseReferenceRow = (raw: unknown, aliases: ReadonlySet<string>) => {
|
||||
const row = xrefRow.parse(raw);
|
||||
if (
|
||||
row.address !== row.address_hex ||
|
||||
row.address !== row.from_address ||
|
||||
row.from_address !== row.from_address_hex ||
|
||||
row.to_address !== row.to_address_hex
|
||||
)
|
||||
throw new Error(
|
||||
"xref endpoints disagree with numeric or directional fields",
|
||||
);
|
||||
for (const fn of [row.from_function, row.to_function])
|
||||
if (fn !== null && fn.address !== fn.address_hex)
|
||||
throw new Error("xref function address disagrees with its hex field");
|
||||
if (!aliases.has(row.to_address))
|
||||
throw new Error(
|
||||
"xref row targets neither the requested address nor an advertised alias",
|
||||
);
|
||||
return row;
|
||||
};
|
||||
|
||||
export interface KunaIncomingReferences {
|
||||
readonly addresses: readonly string[];
|
||||
readonly sha256: string;
|
||||
@@ -167,12 +189,32 @@ export interface KunaIncomingReferences {
|
||||
readonly excludedAliasRows: number;
|
||||
}
|
||||
|
||||
/** Native --to unifies forwarding aliases; REA xrefs asks for the exact address. */
|
||||
export const parseKunaIncomingReferences = (
|
||||
/** Experimental fact projection, owned by the native protocol parser. */
|
||||
export type KunaReferenceFact = {
|
||||
readonly from: string;
|
||||
readonly to: string;
|
||||
readonly kind: z.infer<typeof kind>;
|
||||
readonly fromFunction: {
|
||||
readonly address: string;
|
||||
readonly name: string;
|
||||
} | null;
|
||||
readonly toFunction: {
|
||||
readonly address: string;
|
||||
readonly name: string;
|
||||
} | null;
|
||||
readonly instruction: string;
|
||||
};
|
||||
|
||||
/** Native --to unifies aliases; these facts cover only the requested address. */
|
||||
const parseIncomingReferenceDocument = (
|
||||
text: string,
|
||||
input: string,
|
||||
target: string,
|
||||
): Result<KunaIncomingReferences, string> => {
|
||||
retainFacts: boolean,
|
||||
): Result<
|
||||
KunaIncomingReferences & { readonly facts: readonly KunaReferenceFact[] },
|
||||
string
|
||||
> => {
|
||||
try {
|
||||
const document = xrefsDocument.parse(
|
||||
parseKunaJson(text, KUNA_XREF_OUTPUT_BYTES, addressKeys),
|
||||
@@ -192,27 +234,34 @@ export const parseKunaIncomingReferences = (
|
||||
aliases.add(alias.address);
|
||||
}
|
||||
const addresses = new Set<string>();
|
||||
const facts: KunaReferenceFact[] = [];
|
||||
let excludedAliasRows = 0;
|
||||
for (const raw of document.xrefs) {
|
||||
const row = xrefRow.parse(raw);
|
||||
if (
|
||||
row.address !== row.address_hex ||
|
||||
row.address !== row.from_address ||
|
||||
row.from_address !== row.from_address_hex ||
|
||||
row.to_address !== row.to_address_hex
|
||||
)
|
||||
throw new Error(
|
||||
"xref endpoints disagree with numeric or directional fields",
|
||||
);
|
||||
for (const fn of [row.from_function, row.to_function])
|
||||
if (fn !== null && fn.address !== fn.address_hex)
|
||||
throw new Error("xref function address disagrees with its hex field");
|
||||
if (!aliases.has(row.to_address))
|
||||
throw new Error(
|
||||
"xref row targets neither the requested address nor an advertised alias",
|
||||
);
|
||||
if (row.to_address === target) addresses.add(row.from_address);
|
||||
else excludedAliasRows += 1;
|
||||
const row = parseReferenceRow(raw, aliases);
|
||||
if (row.to_address === target) {
|
||||
addresses.add(row.from_address);
|
||||
if (retainFacts)
|
||||
facts.push({
|
||||
from: row.from_address,
|
||||
to: row.to_address,
|
||||
kind: row.kind,
|
||||
fromFunction:
|
||||
row.from_function === null
|
||||
? null
|
||||
: {
|
||||
address: row.from_function.address,
|
||||
name: row.from_function.name,
|
||||
},
|
||||
toFunction:
|
||||
row.to_function === null
|
||||
? null
|
||||
: {
|
||||
address: row.to_function.address,
|
||||
name: row.to_function.name,
|
||||
},
|
||||
instruction: row.instruction,
|
||||
});
|
||||
} else excludedAliasRows += 1;
|
||||
}
|
||||
return ok({
|
||||
addresses: [...addresses].sort(compareKunaAddresses),
|
||||
@@ -220,8 +269,30 @@ export const parseKunaIncomingReferences = (
|
||||
excludedAliasRows,
|
||||
bytes: Buffer.byteLength(text),
|
||||
sha256: createHash("sha256").update(text).digest("hex"),
|
||||
facts,
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return err(cause instanceof Error ? cause.message : String(cause));
|
||||
}
|
||||
};
|
||||
|
||||
export const parseKunaIncomingReferenceFacts = (
|
||||
text: string,
|
||||
input: string,
|
||||
target: string,
|
||||
): Result<
|
||||
KunaIncomingReferences & { readonly facts: readonly KunaReferenceFact[] },
|
||||
string
|
||||
> => parseIncomingReferenceDocument(text, input, target, true);
|
||||
|
||||
/** Existing address-only contract; richer facts do not enter its retained cache. */
|
||||
export const parseKunaIncomingReferences = (
|
||||
text: string,
|
||||
input: string,
|
||||
target: string,
|
||||
): Result<KunaIncomingReferences, string> => {
|
||||
const parsed = parseIncomingReferenceDocument(text, input, target, false);
|
||||
if (!parsed.ok) return parsed;
|
||||
const { addresses, sha256, bytes, rows, excludedAliasRows } = parsed.value;
|
||||
return ok({ addresses, sha256, bytes, rows, excludedAliasRows });
|
||||
};
|
||||
@@ -4,6 +4,7 @@ import { normalizeKunaAddress } from "../../../../src/kuna/KunaAddress.js";
|
||||
import {
|
||||
parseKunaCallGraph,
|
||||
parseKunaIncomingReferences,
|
||||
parseKunaIncomingReferenceFacts,
|
||||
} from "../../../../src/kuna/KunaReferenceFiles.js";
|
||||
|
||||
const input = "/snapshot/program";
|
||||
@@ -189,6 +190,38 @@ it("keeps exact endpoints, deduplicates sites and counts excluded forwarding ali
|
||||
});
|
||||
});
|
||||
|
||||
it("preserves typed relationships and explicit unknown ownership at the producing boundary", () => {
|
||||
const value = xrefs();
|
||||
value.xrefs[1]!.kind = "data";
|
||||
const parsed = parseKunaIncomingReferenceFacts(wire(value), input, "0x2000");
|
||||
expect(parsed.ok).toBe(true);
|
||||
if (!parsed.ok) return;
|
||||
expect(parsed.value.facts).toEqual([
|
||||
{
|
||||
from: "0x1004",
|
||||
to: "0x2000",
|
||||
kind: "call",
|
||||
fromFunction: { address: "0x1000", name: "caller" },
|
||||
toFunction: null,
|
||||
instruction: "CALL",
|
||||
},
|
||||
{
|
||||
from: "0x1004",
|
||||
to: "0x2000",
|
||||
kind: "data",
|
||||
fromFunction: { address: "0x1000", name: "caller" },
|
||||
toFunction: null,
|
||||
instruction: "CALL",
|
||||
},
|
||||
]);
|
||||
const addresses = parseKunaIncomingReferences(wire(value), input, "0x2000");
|
||||
expect(addresses.ok && Object.hasOwn(addresses.value, "facts")).toBe(false);
|
||||
value.xrefs[2]!.from_address_hex = "0xdead";
|
||||
expect(parseKunaIncomingReferenceFacts(wire(value), input, "0x2000").ok).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
"count",
|
||||
|
||||
Reference in new issue
Block a user