Compare commits

..
Author SHA1 Message Date
Kaoru0822andClaude Opus 5.5 314c98f6ec test(mcp): force the reduced form when checking array item schemas
Use a large nested property with a 1 KiB budget so the schema is
reduced while its shallow array property remains, and assert the
presentation and that property before checking for missing items.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 04:31:43 +00:00
Kaoru0822andClaude Opus 5.5 b02e1c2452 fix(mcp): give the compact any-value schema an item schema
The compact input-schema profile advertises an unconstrained JSON value
as {type: [null, boolean, object, array, number, string]}. That union
declares an array type without items, which strict clients reject: VS
Code Copilot refused build_call_path with "tool parameters array type
must have items", and 21 tool input schemas carried the same shape.

Use one owned any-value schema that adds items: {} for both the
budget-inlined empty schema and reduced array items.

Fixes #3838

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 04:23:57 +00:00
126 changed files with 796 additions and 6939 deletions

No files matched your search

+1 -1
View File
@@ -344,7 +344,7 @@ Report vulnerabilities through [SECURITY.md](SECURITY.md).
## Star history
🎉 **250,000 GitHub stars — thank you!**
🎉 **200,000 GitHub stars — thank you!**
Thanks to everyone using REA, reporting bugs, requesting features, testing builds, and contributing fixes.
+2 -4
View File
@@ -343,10 +343,8 @@ an invalid supplied size remains unknown with incomplete resource coverage while
the resource's other facts are retained. Zero and nonnegative fractional sizes
are preserved.
CDP discovery has a 10-second deadline. Each WebSocket handshake and command
response has its own 20-second deadline, separate from the caller-selected
observation window. A stalled handshake or command reports `provider_timeout`;
caller cancellation still reports cancellation. WebSocket messages remain subject
CDP discovery, connection, and commands wait for the peer or caller cancellation;
a stalled peer can leave an operation waiting. WebSocket messages remain subject
to the [64 MiB limit](#request-boundary). REA does not impose a message-count or
pending-command quota. On cancellation, it closes only its own WebSocket,
waiting up to one second before terminating it. Malformed unsolicited events
-6
View File
@@ -116,12 +116,6 @@ If cleanup cannot be verified, the error retains collected observations in
`details.partial_observation` and marks their application cleanup `unverified`.
Cleanup uncertainty does not authorize signaling unrelated processes.
### Target fuse prerequisites
Active scenario capture requires that the target Electron executable honors Node CLI inspector flags (`--inspect` or `--remote-debugging-port`) so that Playwright can establish its DevTools protocol session.
When an Electron executable explicitly disables the `EnableNodeCliInspectArguments` fuse (or removes the capability), `capture-electron-scenario` fails immediately before process launch with an `AnalysisUnsupportedTargetError` ("unsupported_target") rather than waiting for the startup deadline. For targets with disabled inspector fuses, use passive CDP inspection via `inspect_electron_page` or run against an unrestricted development binary with the extracted `app.asar`.
The real verifier exercises compiled CLI and stdio MCP discovery, page/script
inspection, static/runtime reconciliation, active observation, and CLI SIGTERM
cleanup before checking the IPC, renderer crash/reload, utility/child-process,
+4 -5
View File
@@ -398,9 +398,8 @@ running server or its registration.
The npm release checked on **2026-10-11** was **6.4.0**, published from
[`a56f7bb6`](https://github.com/morluto/rea/commit/a56f7bb668ab23fec8d33f7af353ac7ed2fe1895).
The actual registry package accepted dry-run setup plans for the 17 client IDs
in that release, including Qwen Code, Pi and Hermes. Seven client IDs are
included from 6.5.0 and were absent from the checked npm 6.4.0 package: `droid`,
`dsh`, `kimi`, `qoder`, `trae`, `workbuddy` and `zcode`.
in that release, including Qwen Code, Pi and Hermes. DeepSeek Harness (`dsh`)
is included from 6.5.0 and was absent from the checked npm 6.4.0 package.
The 6.4.0 registry check does not establish 6.5.0 publication or live DeepSeek
API integration.
OpenCode's plan selected
@@ -483,8 +482,8 @@ hosted connector; see [its configuration](client-configuration.md#other-client-s
| ZCode | `zcode` |
| DeepSeek Harness | `dsh` |
`droid`, `dsh`, `kimi`, `qoder`, `trae`, `workbuddy`, and `zcode` require
`rea-agents@6.5.0` or later. The 6.4.0 registry check above does not cover them.
`dsh`, `kimi`, and `workbuddy` require a release containing them. The registry
check above does not cover them.
See [client configuration and troubleshooting](client-configuration.md) for
configuration paths, profiles, trust policies, and schema-limit workarounds.
-5
View File
@@ -95,11 +95,6 @@ pauses or resumes execution, calls `Runtime.runIfWaitingForDebugger`, launches
the target, instruments JavaScript, or invokes Electron APIs. Closing a call
closes only REA's WebSocket.
Each WebSocket handshake and command response has its own 20-second deadline,
separate from the caller-selected observation window. A stalled handshake or
command reports `provider_timeout` and retains observations already received;
caller cancellation still reports cancellation.
The direct observations are:
- `Debugger.scriptParsed`, establishing that a bounded script location was
+1 -1
View File
@@ -274,7 +274,7 @@ npx rea-agents@latest setup
## سجل النجوم
🎉 **250,000 نجمة على GitHub — شكرًا لكم!**
🎉 **200,000 نجمة على GitHub — شكرًا لكم!**
شكرًا لكل من يستخدم REA، ويبلّغ عن الأخطاء، ويقترح ميزات، ويختبر إصدارات البناء، ويساهم بالإصلاحات.
+1 -1
View File
@@ -272,7 +272,7 @@ Melde Sicherheitslücken gemäß [SECURITY.md](../../SECURITY.md).
## Sternverlauf
🎉 **250.000 GitHub-Sterne – vielen Dank!**
🎉 **200.000 GitHub-Sterne – vielen Dank!**
Danke an alle, die REA nutzen, Fehler melden, Funktionswünsche äußern, Builds testen und Fehlerbehebungen beitragen.
+1 -1
View File
@@ -285,7 +285,7 @@ npx rea-agents@latest setup
## Ιστορικό αστεριών
🎉 **250.000 αστέρια στο GitHub — σας ευχαριστούμε!**
🎉 **200.000 αστέρια στο GitHub — σας ευχαριστούμε!**
Ευχαριστούμε όλους όσοι χρησιμοποιούν το REA, αναφέρουν σφάλματα, ζητούν νέες λειτουργίες, δοκιμάζουν builds και συνεισφέρουν διορθώσεις.
+1 -1
View File
@@ -272,7 +272,7 @@ Informa de vulnerabilidades siguiendo [SECURITY.md](../../SECURITY.md).
## Historial de estrellas
🎉 **250.000 estrellas en GitHub: ¡gracias!**
🎉 **200.000 estrellas en GitHub: ¡gracias!**
Gracias a quienes usan REA, informan de errores, solicitan funciones, prueban compilaciones y contribuyen con correcciones.
+1 -1
View File
@@ -257,7 +257,7 @@ npx rea-agents@latest setup
## تاریخچهٔ ستاره‌ها
🎉 **۲۵۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
🎉 **۲۰۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
از همهٔ کسانی که از REA استفاده می‌کنند، خطاها را گزارش می‌دهند، قابلیت‌های جدید پیشنهاد می‌کنند، buildها را آزمایش می‌کنند و در بهبود پروژه مشارکت دارند سپاسگزاریم.
+1 -1
View File
@@ -276,7 +276,7 @@ Signalez les vulnérabilités via [SECURITY.md](../../SECURITY.md).
## Historique des étoiles
🎉 **250 000 étoiles GitHub — merci !**
🎉 **200 000 étoiles GitHub — merci !**
Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs.
+1 -1
View File
@@ -297,7 +297,7 @@ Laporkan kerentanan melalui [SECURITY.md](../../SECURITY.md).
## Riwayat Bintang
🎉 **250.000 bintang GitHub, terima kasih!**
🎉 **200.000 bintang GitHub, terima kasih!**
Terima kasih kepada semua orang yang menggunakan REA, melaporkan bug, meminta fitur, menguji build, dan berkontribusi dalam perbaikan.
+1 -1
View File
@@ -276,7 +276,7 @@ Segnala vulnerabilità tramite [SECURITY.md](../../SECURITY.md).
## Cronologia delle stelle
🎉 **250.000 stelle su GitHub — grazie!**
🎉 **200.000 stelle su GitHub — grazie!**
Grazie a tutti coloro che usano REA, segnalano bug, propongono funzionalità, testano le build e contribuiscono con correzioni.
+1 -1
View File
@@ -274,7 +274,7 @@ npx rea-agents@latest setup
## スター履歴
🎉 **GitHub スター 250,000 件、ありがとうございます!**
🎉 **GitHub スター 200,000 件、ありがとうございます!**
REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。
+1 -1
View File
@@ -283,7 +283,7 @@ npx rea-agents@latest setup
## 스타 기록
🎉 **GitHub 스타 250,000개, 감사합니다!**
🎉 **GitHub 스타 200,000개, 감사합니다!**
REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
+1 -1
View File
@@ -272,7 +272,7 @@ Zgłaszaj podatności zgodnie z [SECURITY.md](../../SECURITY.md).
## Historia gwiazdek
🎉 **250 000 gwiazdek na GitHubie — dziękujemy!**
🎉 **200 000 gwiazdek na GitHubie — dziękujemy!**
Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki.
+1 -1
View File
@@ -272,7 +272,7 @@ Relate vulnerabilidades conforme [SECURITY.md](../../SECURITY.md).
## Histórico de estrelas
🎉 **250.000 estrelas no GitHub — muito obrigado!**
🎉 **200.000 estrelas no GitHub — muito obrigado!**
Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções.
+1 -1
View File
@@ -272,7 +272,7 @@ npx rea-agents@latest setup
## История звёзд
🎉 **250 000 звёзд на GitHub — спасибо!**
🎉 **200 000 звёзд на GitHub — спасибо!**
Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями.
+1 -1
View File
@@ -272,7 +272,7 @@ npx rea-agents@latest setup
## ประวัติดาว
🎉 **250,000 ดาวบน GitHub — ขอบคุณทุกคน!**
🎉 **200,000 ดาวบน GitHub — ขอบคุณทุกคน!**
ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข
+1 -1
View File
@@ -277,7 +277,7 @@ Güvenlik açıklarını [SECURITY.md](../../SECURITY.md) üzerinden bildirin.
## Yıldız geçmişi
🎉 **GitHub'da 250.000 yıldız — teşekkürler!**
🎉 **GitHub'da 200.000 yıldız — teşekkürler!**
REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler.
+1 -1
View File
@@ -272,7 +272,7 @@ npx rea-agents@latest setup
## Історія зірок
🎉 **250 000 зірок на GitHub — дякуємо!**
🎉 **200 000 зірок на GitHub — дякуємо!**
Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями.
+1 -1
View File
@@ -272,7 +272,7 @@ Báo cáo lỗ hổng theo [SECURITY.md](../../SECURITY.md).
## Lịch sử sao
🎉 **250.000 sao trên GitHub — xin cảm ơn!**
🎉 **200.000 sao trên GitHub — xin cảm ơn!**
Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi.
+1 -1
View File
@@ -278,7 +278,7 @@ npx rea-agents@latest setup
## Star 歷史
🎉 **GitHub Star 達到 250,000 個,感謝大家!**
🎉 **GitHub Star 達到 200,000 個,感謝大家!**
感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。
+1 -1
View File
@@ -278,7 +278,7 @@ npx rea-agents@latest setup
## Star 历史
🎉 **GitHub Star 达到 250,000 个,感谢大家!**
🎉 **GitHub Star 达到 200,000 个,感谢大家!**
感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。
@@ -4,11 +4,9 @@ import { fileURLToPath } from "node:url";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { CdpCommandRejection } from "../../dist/browser/CdpCommandRejection.js";
import { CdpConnection } from "../../dist/browser/CdpConnection.js";
import { createBrowserScenarioProvider } from "../../dist/composition/browserScenario.js";
import { browserScenarioSchema } from "../../dist/domain/browserScenario.js";
import { AnalysisCancelledError } from "../../dist/domain/analysisErrorCore.js";
import { runScenarioCli } from "./browser-scenario-verifier.mjs";
import { requireMcpEvidenceResult } from "./mcp-verifier-results.mjs";
@@ -26,24 +24,12 @@ const alternate = {
width: 700,
height: 500,
};
const environmentExpression = `{
const expression = `JSON.stringify({
timezone: Intl.DateTimeFormat().resolvedOptions().timeZone,
locale: Intl.DateTimeFormat().resolvedOptions().locale,
dpr: devicePixelRatio, width: innerWidth, height: innerHeight
}`;
const expression = `JSON.stringify(${environmentExpression})`;
const scenarioSnapshotExpression = `JSON.stringify({
timeOrigin: performance.timeOrigin,
environment: ${environmentExpression},
run: document.body?.dataset.run ?? null
})`;
const isNavigationTransitionRejection = (cause) =>
cause instanceof CdpCommandRejection &&
cause.command === "Runtime.evaluate" &&
cause.code === -32_000 &&
cause.reportedMessage === "Inspected target navigated or closed";
const environmentScenario = ({ endpoint, targetId, origin, values, run }) =>
browserScenarioSchema.parse({
browser: { mode: "connect", cdp_endpoint: endpoint, target_id: targetId },
@@ -160,6 +146,7 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
endpoint,
targetId,
origin,
readEnvironment,
assertRestored,
});
} finally {
@@ -191,7 +178,12 @@ export async function verifyScenarioEnvironment(endpoint, targetId, origin) {
assertCapture(actionFailure.value);
assert.equal(actionFailure.value.steps.at(-1).status, "failed");
await assertRestored();
await verifyCancelledEnvironment(provider, connection, waiting);
await verifyCancelledEnvironment(
provider,
connection,
waiting,
readEnvironment,
);
await assertRestored();
return {
mocked: false,
@@ -216,6 +208,7 @@ const verifyConcurrentMcpCapture = async ({
endpoint,
targetId,
origin,
readEnvironment,
assertRestored,
}) => {
const firstScenario = environmentScenario({
@@ -248,13 +241,12 @@ const verifyConcurrentMcpCapture = async ({
try {
const first = callTool(firstScenario, firstCancellation.signal);
await waitForEnvironment(connection, expected, "A");
await waitForEnvironment(readEnvironment, connection, expected, "A");
const second = callTool(secondScenario, secondCancellation.signal);
await client.ping();
const firstSnapshot = await readScenarioSnapshot(connection);
assert.deepEqual(firstSnapshot.environment, expected);
assert.equal(firstSnapshot.run, "A");
assert.deepEqual(await readEnvironment(), expected);
assert.equal(await readScenarioRun(connection), "A");
const cancelled = callTool(
environmentScenario({
@@ -269,9 +261,8 @@ const verifyConcurrentMcpCapture = async ({
await client.ping();
cancellation.abort(new Error("cancel queued browser scenario"));
await assert.rejects(cancelled, /cancel queued browser scenario/u);
const queuedSnapshot = await readScenarioSnapshot(connection);
assert.deepEqual(queuedSnapshot.environment, expected);
assert.equal(queuedSnapshot.run, "A");
assert.deepEqual(await readEnvironment(), expected);
assert.equal(await readScenarioRun(connection), "A");
await releaseScenario(connection);
assertCapture(
@@ -279,7 +270,7 @@ const verifyConcurrentMcpCapture = async ({
expected,
);
await waitForEnvironment(connection, alternate, "B");
await waitForEnvironment(readEnvironment, connection, alternate, "B");
await releaseScenario(connection);
assertCapture(
requireMcpEvidenceResult(await second, "capture_browser_scenario"),
@@ -296,57 +287,31 @@ const verifyConcurrentMcpCapture = async ({
}
};
const waitForEnvironment = async (connection, expectedValues, run) => {
await waitForScenarioSnapshot(
connection,
(snapshot) =>
JSON.stringify(snapshot.environment) === JSON.stringify(expectedValues) &&
snapshot.run === run,
`Scenario ${run} never reached its page-controlled gate`,
);
};
const waitForScenarioSnapshot = async (connection, accept, message) => {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), 10_000);
let lastTransitionRejection;
try {
while (!controller.signal.aborted) {
try {
const snapshot = await readScenarioSnapshot(
connection,
controller.signal,
);
if (controller.signal.aborted) break;
if (accept(snapshot)) return;
} catch (cause) {
if (
controller.signal.aborted &&
cause instanceof AnalysisCancelledError
)
break;
if (!isNavigationTransitionRejection(cause)) throw cause;
lastTransitionRejection = cause;
}
await delay(25);
}
throw new Error(message, {
cause: lastTransitionRejection ?? controller.signal.reason,
});
} finally {
clearTimeout(timer);
const waitForEnvironment = async (
readEnvironment,
connection,
expectedValues,
run,
) => {
const deadline = Date.now() + 10_000;
while (Date.now() < deadline) {
if (
JSON.stringify(await readEnvironment()) ===
JSON.stringify(expectedValues) &&
(await readScenarioRun(connection)) === run
)
return;
await delay(25);
}
throw new Error(`Scenario ${run} never reached its page-controlled gate`);
};
const readScenarioSnapshot = async (connection, signal) => {
const value = await connection.send(
"Runtime.evaluate",
{ expression: scenarioSnapshotExpression, returnByValue: true },
undefined,
signal,
);
assert.equal(value.result.type, "string");
return JSON.parse(value.result.value);
const readScenarioRun = async (connection) => {
const value = await connection.send("Runtime.evaluate", {
expression: "document.body.dataset.run ?? null",
returnByValue: true,
});
return value.result.value;
};
const releaseScenario = async (connection) => {
@@ -356,26 +321,39 @@ const releaseScenario = async (connection) => {
});
};
async function verifyCancelledEnvironment(provider, connection, scenario) {
async function verifyCancelledEnvironment(
provider,
connection,
scenario,
readEnvironment,
) {
const controller = new AbortController();
const priorSnapshot = await readScenarioSnapshot(connection);
const priorNavigation = await connection.send("Runtime.evaluate", {
expression: "performance.timeOrigin",
returnByValue: true,
});
const pending = provider.captureScenario(
{ ...scenario, actions: [{ ...scenario.actions[0], timeout_ms: 30_000 }] },
{ signal: controller.signal },
);
const settlement = Promise.allSettled([pending]);
try {
await waitForScenarioSnapshot(
connection,
(snapshot) =>
snapshot.timeOrigin !== priorSnapshot.timeOrigin &&
JSON.stringify(snapshot.environment) === JSON.stringify(expected),
"Scenario never applied emulation",
);
const deadline = Date.now() + 10_000;
while (true) {
const navigation = await connection.send("Runtime.evaluate", {
expression: "performance.timeOrigin",
returnByValue: true,
});
if (
navigation.result.value !== priorNavigation.result.value &&
JSON.stringify(await readEnvironment()) === JSON.stringify(expected)
)
break;
assert.ok(Date.now() < deadline, "Scenario never applied emulation");
await delay(25);
}
await delay(100);
} finally {
controller.abort();
await settlement;
}
const cancelled = await pending;
assert.ok(
+21 -73
View File
@@ -8,12 +8,10 @@ import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
import { ABORTED, waitForAbortable } from "../process/AbortablePromise.js";
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
import { randomUUID } from "node:crypto";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
execFileOutput,
execFileOutputFailure,
} from "../process/ExecFileOutput.js";
import type { ProviderAvailability } from "../application/AnalysisProvider.js";
import type { ProviderRejectionCode } from "../contracts/providerSelection.js";
import type { JsonValue } from "../domain/jsonValue.js";
@@ -154,23 +152,14 @@ export type ApktoolConfigurationFailureCode =
| "version_unresolved"
| "unsupported_version";
/** Unresolved cleanup of a launched probe; its owner was retained separately. */
export interface ApktoolProbeCleanup {
readonly reason: string;
readonly resources: readonly string[];
}
export class ApktoolConfigurationFailure extends Error {
readonly cleanup: ApktoolProbeCleanup | undefined;
constructor(
readonly code: ApktoolConfigurationFailureCode,
message: string,
readonly diagnostics: Readonly<Record<string, JsonValue>> = {},
options?: ErrorOptions & { readonly cleanup?: ApktoolProbeCleanup },
options?: ErrorOptions,
) {
super(message, options);
this.cleanup = options?.cleanup;
}
}
@@ -239,7 +228,6 @@ export const inspectApktoolClient = async (
signal?: AbortSignal,
environment: Readonly<Record<string, string | undefined>> = process.env,
platform: NodeJS.Platform = process.platform,
onCleanupOwner?: (owner: ProviderProcessSupervisor) => void,
): Promise<ApktoolClientIdentity> => {
const hostEnvironment = snapshotEnvironment(environment, platform);
let invocation: ApktoolInvocation;
@@ -266,57 +254,23 @@ export const inspectApktoolClient = async (
let stdout: string;
let stderr: string;
try {
// The owned process group stops a shell launcher's children with it; a
// child holding inherited output pipes cannot outlive the deadline.
const output = await runOwnedCommand(
({ stdout, stderr } = await execFileOutput(
invocation.command,
invocation.arguments,
{
command: invocation.command,
arguments: invocation.arguments,
windowsVerbatimArguments: invocation.windowsVerbatimArguments,
// Apktool launchers are commonly interpreter scripts, so ownership is
// proved by the run token rather than the launcher's argv prefix.
expectedCommand: null,
runId: randomUUID(),
hostEnvironment,
timeout: VERSION_TIMEOUT_MS,
stopSignal: "SIGTERM",
maxBuffer: VERSION_MAX_OUTPUT_BYTES,
env: hostEnvironment,
...(signal === undefined ? {} : { signal }),
},
{
timeoutMs: VERSION_TIMEOUT_MS,
diagnosticBytes: VERSION_MAX_OUTPUT_BYTES,
},
signal === undefined ? {} : { signal },
);
stdout = output.stdout.text;
stderr = output.stderr.text;
));
} catch (cause) {
if (
cause instanceof OwnedCommandFailure &&
cause.cleanupOwner !== undefined
)
onCleanupOwner?.(cause.cleanupOwner);
const cleanup =
cause instanceof OwnedCommandFailure && cause.cleanupFailure !== null
? { reason: cause.cleanupFailure, resources: cause.resources }
: undefined;
const cleanupDetails = cleanup === undefined ? {} : { cleanup };
if (
signal?.aborted === true ||
(cause instanceof OwnedCommandFailure && cause.reason === "cancelled")
)
throw new AnalysisCancelledError("inspect_apktool_client", {
cause,
...cleanupDetails,
});
if (cause instanceof OwnedCommandFailure && cause.reason === "timeout")
throw new AnalysisTimeoutError(
"inspect_apktool_client",
VERSION_TIMEOUT_MS,
{ cause, ...cleanupDetails },
);
if (
cause instanceof Error &&
"code" in cause &&
(cause.code === "ENOENT" || cause.code === "EACCES")
)
const failure = execFileOutputFailure(cause);
if (signal?.aborted || failure?.code === "ABORT_ERR")
throw new AnalysisCancelledError("inspect_apktool_client", { cause });
if (failure?.code === "ENOENT")
throw new ApktoolConfigurationFailure(
"command_missing",
`The selected apktool launcher was not found: ${selection.command}`,
@@ -327,25 +281,19 @@ export const inspectApktoolClient = async (
},
{ cause },
);
const snapshot =
cause instanceof OwnedCommandFailure ? cause.snapshot : null;
throw new ApktoolConfigurationFailure(
"version_unresolved",
"The selected apktool launcher did not report a usable version",
{
selected_command: selection.command,
...(snapshot === null
...(failure === undefined
? {}
: {
exit_code: snapshot.exitCode ?? null,
stderr_excerpt: snapshot.stderr.text.slice(0, 512),
exit_code: failure.code,
stderr_excerpt: failure.stderr.slice(0, 512),
}),
...(cause instanceof OwnedCommandFailure &&
cause.reason === "output-limit"
? { max_output_bytes: VERSION_MAX_OUTPUT_BYTES }
: {}),
},
{ cause, ...cleanupDetails },
{ cause },
);
}
const identity = parseApktoolVersionOutput(stdout);
+4 -65
View File
@@ -56,7 +56,6 @@ import { WindowsBatchCommandError } from "../process/WindowsBatchCommand.js";
import {
APKTOOL_COMMAND_ENV,
ApktoolConfigurationFailure,
type ApktoolProbeCleanup,
apktoolConfigurationAvailability,
apktoolInvocation,
inspectApktoolClient,
@@ -149,19 +148,6 @@ interface DecodeLimits {
readonly maxOutputBytes: number;
}
/** A probe whose launched processes could not be confirmed stopped. */
const probeCleanupFailure = (
operation: string,
failure: ApktoolConfigurationFailure,
cleanup: ApktoolProbeCleanup,
): ProviderCleanupError =>
new ProviderCleanupError(
APKTOOL_PROVIDER_IDENTITY.id,
cleanup.resources,
{ reason: cleanup.reason, probe_failure: failure.message },
{ operation, cause: failure },
);
const runApktool = async (
operation: string,
selection: ApktoolCommandSelection,
@@ -194,9 +180,6 @@ const runApktool = async (
command: invocation.command,
arguments: invocation.arguments,
windowsVerbatimArguments: invocation.windowsVerbatimArguments,
// Apktool launchers are commonly interpreter scripts, so ownership is
// proved by the run token rather than the launcher's argv prefix.
expectedCommand: null,
runId: randomUUID(),
hostEnvironment: environment,
},
@@ -345,10 +328,6 @@ export class ApktoolProvider {
private readonly workspaces = new Set<string>();
private readonly removeWorkspace: (path: string) => Promise<void>;
readonly #pendingCommandOwners = new Map<ProviderProcessSupervisor, string>();
/** A version probe whose cleanup is unresolved is retried on close. */
readonly #retainProbeOwner = (owner: ProviderProcessSupervisor): void => {
this.#pendingCommandOwners.set(owner, "apktool version probe");
};
readonly #shutdown = new AbortController();
readonly #active = new Set<Promise<void>>();
#closing = false;
@@ -399,13 +378,7 @@ export class ApktoolProvider {
process.platform,
signal,
);
await inspectApktoolClient(
selection,
signal,
environment,
process.platform,
this.#retainProbeOwner,
);
await inspectApktoolClient(selection, signal, environment);
throwIfCancelled("inspect_apktool_client", signal);
return {
status: "available",
@@ -417,15 +390,8 @@ export class ApktoolProvider {
},
};
} catch (cause) {
if (cause instanceof ApktoolConfigurationFailure) {
if (cause.cleanup !== undefined)
throw probeCleanupFailure(
"inspect_apktool_client",
cause,
cause.cleanup,
);
if (cause instanceof ApktoolConfigurationFailure)
return apktoolConfigurationAvailability(cause);
}
throw cause;
}
}
@@ -471,13 +437,7 @@ export class ApktoolProvider {
process.platform,
signal,
);
identity = await inspectApktoolClient(
selection,
signal,
environment,
process.platform,
this.#retainProbeOwner,
);
identity = await inspectApktoolClient(selection, signal, environment);
} catch (cause) {
if (
cause instanceof AnalysisCancelledError ||
@@ -485,10 +445,6 @@ export class ApktoolProvider {
)
return err(cause);
if (cause instanceof ApktoolConfigurationFailure) {
if (cause.cleanup !== undefined)
return err(
probeCleanupFailure(request.operation, cause, cause.cleanup),
);
return err(
providerConfigurationRefusal(
APKTOOL_PROVIDER_IDENTITY.id,
@@ -541,14 +497,12 @@ export class ApktoolProvider {
async #drainAndClean(): Promise<void> {
await Promise.all(this.#active);
const failures: unknown[] = [];
const retained: string[] = [];
for (const [owner, root] of this.#pendingCommandOwners) {
try {
const stopped = await owner.stop();
if (stopped.status === "incomplete") throw new Error(stopped.reason);
this.#pendingCommandOwners.delete(owner);
} catch (cause) {
retained.push(root);
failures.push(
new Error(
`apktool process cleanup remains incomplete for ${root}: ${cause instanceof Error ? cause.message : String(cause)}`,
@@ -557,23 +511,8 @@ export class ApktoolProvider {
);
}
}
// Ownership stays retained for a later close; report it as typed cleanup.
if (failures.length > 0)
throw new ProviderCleanupError(
APKTOOL_PROVIDER_IDENTITY.id,
retained,
{
reason: failures
.map((failure) =>
failure instanceof Error ? failure.message : String(failure),
)
.join("; "),
},
{
operation: "close",
cause: new AggregateError(failures, "Apktool process cleanup failed"),
},
);
throw new AggregateError(failures, "Apktool process cleanup failed");
for (const path of this.workspaces) {
try {
await this.removeWorkspace(path);
@@ -1,107 +0,0 @@
import { execFile } from "node:child_process";
import { chmod, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { setTimeout as delay } from "node:timers/promises";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { inspectApktoolClient } from "./ApktoolConfiguration.js";
/**
* Only an exited process counts as stopped: it is gone, a zombie waiting to
* be reaped, or absent from `ps`. A failed inspection is not proof of exit.
*/
const alive = async (pid: number): Promise<boolean> => {
try {
process.kill(pid, 0);
} catch (cause: unknown) {
if ((cause as NodeJS.ErrnoException).code === "ESRCH") return false;
return true;
}
return promisify(execFile)("ps", ["-o", "stat=", "-p", `${pid}`]).then(
({ stdout }) => {
const state = stdout.trim();
return state !== "" && !state.startsWith("Z");
},
// `ps` exits 1 when no process matches; any other failure is unknown.
(cause: unknown) => (cause as { code?: unknown }).code !== 1,
);
};
const waitFor = async (check: () => Promise<boolean>, ms: number) => {
const deadline = Date.now() + ms;
while (Date.now() < deadline) {
if (await check()) return true;
await delay(25);
}
return false;
};
const readPids = async (path: string): Promise<readonly number[]> =>
readFile(path, "utf8")
.then((text) => JSON.parse(text) as number[])
.catch(() => []);
// The interpreter is the test runtime itself, so its environment (and the
// ownership run token) is readable on both Linux and macOS.
it.skipIf(process.platform === "win32")(
"stops an interpreter launcher and its pipe-holding child when the probe is cancelled",
async () => {
const root = await createTestTempDirectory("rea-apktool-probe-");
const pids = join(root, "pids.json");
const launcher = join(root, "apktool");
await writeFile(
launcher,
[
`#!${process.execPath}`,
'const { spawn } = require("node:child_process");',
'const child = spawn(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { stdio: "inherit" });',
`require("node:fs").writeFileSync(${JSON.stringify(pids)}, JSON.stringify([process.pid, child.pid]));`,
"setInterval(() => {}, 1000);",
].join("\n"),
);
await chmod(launcher, 0o755);
const controller = new AbortController();
const started = performance.now();
const probe = inspectApktoolClient(
{ command: launcher, commandSource: "environment" },
controller.signal,
process.env,
).catch((cause: unknown) => cause);
try {
expect(
await waitFor(async () => (await readPids(pids)).length === 2, 10_000),
).toBe(true);
const launched = await readPids(pids);
controller.abort();
// A regressed probe never settles; bound the wait so cleanup still runs.
const outcome = await Promise.race([
probe,
delay(15_000).then(() => "still pending"),
]);
expect(outcome).toBeInstanceOf(AnalysisCancelledError);
expect(performance.now() - started).toBeLessThan(15_000);
expect(
await waitFor(
async () =>
(await Promise.all(launched.map(alive))).every((live) => !live),
5_000,
),
).toBe(true);
} finally {
// A failed assertion must not leave the fixture's processes running.
controller.abort();
await Promise.race([probe, delay(10_000)]);
for (const pid of await readPids(pids))
if (await alive(pid))
try {
process.kill(pid, "SIGKILL");
} catch {
// Already exited between the check and the signal.
}
}
},
30_000,
);
-147
View File
@@ -1,147 +0,0 @@
import { afterEach, expect, it, vi } from "vitest";
import {
AnalysisCancelledError,
AnalysisTimeoutError,
} from "../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import {
ApktoolConfigurationFailure,
inspectApktoolClient,
} from "./ApktoolConfiguration.js";
import { ApktoolProvider } from "./ApktoolProvider.js";
vi.mock("../process/OwnedCommand.js", async (importOriginal) => ({
...(await importOriginal<typeof import("../process/OwnedCommand.js")>()),
runOwnedCommand: vi.fn(),
}));
afterEach(() => vi.mocked(runOwnedCommand).mockReset());
const selection = {
command: "/opt/apktool/apktool",
commandSource: "environment",
} as const;
const owner = {} as ProviderProcessSupervisor;
it("runs the version probe as an owned interpreter-script process group", async () => {
vi.mocked(runOwnedCommand).mockResolvedValueOnce({
stdout: { text: "2.9.3\n" },
stderr: { text: "" },
} as Awaited<ReturnType<typeof runOwnedCommand>>);
await expect(
inspectApktoolClient(selection, undefined, {}, "linux"),
).resolves.toEqual({ version: "2.9.3" });
expect(vi.mocked(runOwnedCommand).mock.lastCall?.[0]).toMatchObject({
command: "/opt/apktool/apktool",
arguments: ["--version"],
expectedCommand: null,
});
expect(vi.mocked(runOwnedCommand).mock.lastCall?.[1]).toEqual({
timeoutMs: 20_000,
diagnosticBytes: 64 * 1024,
});
});
it("reports an expired version probe as a typed timeout", async () => {
vi.mocked(runOwnedCommand).mockRejectedValueOnce(
new OwnedCommandFailure("timeout", "Command deadline elapsed."),
);
const failure = await inspectApktoolClient(
selection,
undefined,
{},
"linux",
).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(AnalysisTimeoutError);
expect(projectAnalysisError(failure as AnalysisTimeoutError)).toMatchObject({
code: "provider_timeout",
category: "timeout",
details: { timeout_ms: 20_000 },
});
});
it("retains an unresolved probe cleanup owner and reports the cleanup", async () => {
vi.mocked(runOwnedCommand).mockRejectedValueOnce(
new OwnedCommandFailure(
"timeout",
"Command deadline elapsed.",
null,
"cleanup incomplete",
undefined,
["pid:1"],
owner,
),
);
const retained: ProviderProcessSupervisor[] = [];
const failure = await inspectApktoolClient(
selection,
undefined,
{},
"linux",
(cleanupOwner) => retained.push(cleanupOwner),
).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(AnalysisTimeoutError);
expect(projectAnalysisError(failure as AnalysisTimeoutError)).toMatchObject({
code: "cleanup_incomplete",
});
expect(retained).toEqual([owner]);
});
it("reports a cancelled version probe as cancelled", async () => {
vi.mocked(runOwnedCommand).mockRejectedValueOnce(
new OwnedCommandFailure("cancelled", "Command cancelled."),
);
await expect(
inspectApktoolClient(selection, undefined, {}, "linux"),
).rejects.toBeInstanceOf(AnalysisCancelledError);
});
it("keeps unresolved probe cleanup typed when the launcher overflows its output", async () => {
const overflow = () =>
new OwnedCommandFailure(
"output-limit",
"Command diagnostic output exceeded its complete-output budget.",
null,
"process group still running",
undefined,
["pid:7", "process-group:7"],
owner,
);
vi.mocked(runOwnedCommand).mockRejectedValueOnce(overflow());
const failure = await inspectApktoolClient(
selection,
undefined,
{},
"linux",
).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(ApktoolConfigurationFailure);
expect((failure as ApktoolConfigurationFailure).cleanup).toEqual({
reason: "process group still running",
resources: ["pid:7", "process-group:7"],
});
// Both provider entry points report the unresolved processes as cleanup.
const provider = new ApktoolProvider({
environment: { REA_APKTOOL_COMMAND: "/opt/apktool/apktool" },
});
vi.mocked(runOwnedCommand).mockRejectedValueOnce(overflow());
await expect(provider.inspectAvailability()).rejects.toBeInstanceOf(
ProviderCleanupError,
);
vi.mocked(runOwnedCommand).mockRejectedValueOnce(overflow());
const result = await provider.execute({
operation: "decode_android_resources",
input: {},
} as never);
expect(result.ok ? null : result.error).toBeInstanceOf(ProviderCleanupError);
expect(result.ok ? null : projectAnalysisError(result.error)).toMatchObject({
code: "cleanup_incomplete",
details: { resources: ["pid:7", "process-group:7"] },
});
});
+1 -7
View File
@@ -1,8 +1,2 @@
/** Deep-analysis providers whose readiness can be selected explicitly by doctor. */
export const DOCTOR_PROVIDER_IDS = [
"hopper",
"ghidra",
"ida",
"jadx",
"kuna",
] as const;
export const DOCTOR_PROVIDER_IDS = ["hopper", "ghidra", "ida", "jadx"] as const;
+12 -103
View File
@@ -94,95 +94,10 @@ const failedEntry = (
};
};
/** A package directory's legacy `main` entry point, as its manifest declares it. */
type PackageMain =
| { readonly state: "declared"; readonly main: string }
| { readonly state: "none" }
| { readonly state: "unknown" };
/**
* Read each inventoried `package.json` once. Node selects a non-empty string
* `main` before a directory index; an unreadable or invalid manifest leaves
* that selection unknown.
*/
const readPackageMains = (
read: ReferenceSourceRead,
): ReadonlyMap<string, PackageMain> => {
const mains = new Map<string, PackageMain>();
for (const entry of read.entries) {
if (posix.basename(entry.path) !== "package.json") continue;
const directory = posix.dirname(entry.path);
if (entry.status !== "read" || entry.kind !== "file") {
mains.set(directory, { state: "unknown" });
continue;
}
try {
const manifest: unknown = JSON.parse(
new TextDecoder("utf-8", { fatal: true }).decode(entry.bytes),
);
const main =
typeof manifest === "object" && manifest !== null
? Reflect.get(manifest, "main")
: undefined;
mains.set(
directory,
typeof main === "string" && main !== ""
? { state: "declared", main }
: { state: "none" },
);
} catch {
mains.set(directory, { state: "unknown" });
}
}
return mains;
};
const RESOLUTION_SUFFIXES = ["", ".ts", ".js", ".mjs", ".cjs", ".tsx", ".jsx"];
/** Inventory facts an internal specifier resolves against. */
interface SpecifierContext {
readonly lookupPath: ReferenceSourcePathLookup;
readonly packageMains: ReadonlyMap<string, PackageMain>;
}
/**
* Apply a package `main` to a directory resolved through its index. Node loads
* the declared entry as a file, then as a directory index, and falls back to
* the package's own index only when neither exists.
*/
const selectPackageMain = (
index: string,
context: SpecifierContext,
): { to: string; resolution: "internal" | "unresolved" } => {
const directory = posix.dirname(index);
const manifest = context.packageMains.get(directory);
if (manifest === undefined || manifest.state === "none")
return { to: index, resolution: "internal" };
if (manifest.state === "unknown")
return { to: directory, resolution: "unresolved" };
// Node resolves `main` with path.resolve: an absolute main leaves the
// package, and a trailing separator does not make the entry a directory.
if (posix.isAbsolute(manifest.main))
return { to: posix.normalize(manifest.main), resolution: "unresolved" };
const target = posix
.normalize(posix.join(directory, manifest.main))
.replace(/(?<=.)\/+$/u, "");
// Node would load a declared entry outside the imported root.
if (logicalPathEscapesRoot(target))
return { to: target, resolution: "unresolved" };
const selected = context.lookupPath.resolveCandidates([
...RESOLUTION_SUFFIXES.map((suffix) => `${target}${suffix}`),
...RESOLUTION_SUFFIXES.filter((suffix) => suffix !== "").map((suffix) =>
posix.join(target, `index${suffix}`),
),
]);
return { to: selected ?? index, resolution: "internal" };
};
const resolveInternalSpecifier = (
fromPath: string,
specifier: string,
context: SpecifierContext,
lookupPath: ReferenceSourcePathLookup,
): { to: string; resolution: "internal" | "unresolved" } => {
if (posix.isAbsolute(specifier))
return { to: specifier, resolution: "unresolved" };
@@ -194,9 +109,10 @@ const resolveInternalSpecifier = (
// Preserve directory intent before normalization removes terminal dot segments.
const directoryReference =
specifier.endsWith("/") || [".", ".."].includes(posix.basename(specifier));
const suffixes = ["", ".ts", ".js", ".mjs", ".cjs", ".tsx", ".jsx"];
// A directory resolves only to an index with an extension; neither Node nor
// TypeScript selects an extensionless `index` file.
const candidates = RESOLUTION_SUFFIXES.flatMap((suffix) => [
const candidates = suffixes.flatMap((suffix) => [
...(directoryReference ? [] : [`${normalized}${suffix}`]),
...(suffix === "" ? [] : [posix.join(normalized, `index${suffix}`)]),
]);
@@ -208,13 +124,9 @@ const resolveInternalSpecifier = (
({ path }) => path,
),
);
const match = context.lookupPath.resolveCandidates(candidates);
if (match === undefined) return { to: normalized, resolution: "unresolved" };
const resolvedThroughIndex =
posix.basename(match).startsWith("index.") &&
posix.dirname(match).normalize("NFC") === normalized.normalize("NFC");
return resolvedThroughIndex
? selectPackageMain(match, context)
const match = lookupPath.resolveCandidates(candidates);
return match === undefined
? { to: normalized, resolution: "unresolved" }
: { to: match, resolution: "internal" };
};
@@ -222,7 +134,7 @@ type ReadFile = Extract<ReferenceSourceEntry, { status: "read"; kind: "file" }>;
const appendFile = (
entry: ReadFile,
context: SpecifierContext,
lookupPath: ReferenceSourcePathLookup,
output: ParsedReferenceSourceEntries,
): void => {
const language = detectReferenceSourceLanguage(entry.path);
@@ -249,7 +161,7 @@ const appendFile = (
const resolved = resolveInternalSpecifier(
relationship.from_path,
relationship.to,
context,
lookupPath,
);
output.relationships.push({
...relationship,
@@ -262,11 +174,11 @@ const appendFile = (
const appendReadEntry = (
entry: Extract<ReferenceSourceEntry, { status: "read" }>,
context: SpecifierContext,
lookupPath: ReferenceSourcePathLookup,
output: ParsedReferenceSourceEntries,
): void => {
if (entry.kind === "file") {
appendFile(entry, context, output);
appendFile(entry, lookupPath, output);
return;
}
const classifications = classifyReferenceSourcePath(entry.path);
@@ -309,14 +221,11 @@ export const parseReferenceSourceEntries = (
parseFailures: [],
limitations: [...read.limitations],
};
const context: SpecifierContext = {
lookupPath: createReferenceSourcePathLookup(filePaths),
packageMains: readPackageMains(read),
};
const lookupPath = createReferenceSourcePathLookup(filePaths);
for (const entry of read.entries) {
if (signal?.aborted === true) break;
if (entry.status === "failed") output.entries.push(failedEntry(entry));
else appendReadEntry(entry, context, output);
else appendReadEntry(entry, lookupPath, output);
}
return output;
};
+7 -28
View File
@@ -216,12 +216,12 @@ export class BinarySession
options: BinarySessionOpenOptions = {},
): Promise<Result<BinaryTarget, AnalysisError>> {
return this.#open(
(admitted) =>
() =>
resolveSessionOpen({
router: this.#providerRouter,
current: this.#active,
path,
options: admitted,
options,
stagedSnapshotMatches: (target, profile) =>
this.matchesSnapshot(target, profile),
}),
@@ -250,33 +250,12 @@ export class BinarySession
}
#open(
resolve: (
admitted: BinarySessionOpenOptions,
) => Promise<Result<ResolvedSessionOpen, AnalysisError>>,
options: BinarySessionOpenOptions,
resolve: () => Promise<Result<ResolvedSessionOpen, AnalysisError>>,
options: Pick<BinarySessionOpenOptions, "signal" | "snapshot">,
): Promise<Result<BinaryTarget, AnalysisError>> {
return this.#serialize(async (admittedThrough) => {
const admitted = await this.#withLoadedSnapshot(options);
if (!admitted.ok) return admitted;
return this.#openWithinTransition(
admittedThrough,
() => resolve(admitted.value),
admitted.value,
);
});
}
/** Read the staged snapshot only after earlier lifecycle transitions have settled. */
async #withLoadedSnapshot(
options: BinarySessionOpenOptions,
): Promise<Result<BinarySessionOpenOptions, AnalysisError>> {
if (options.loadSnapshot === undefined) return ok(options);
if (isAborted(options.signal))
return err(new AnalysisCancelledError("open_binary"));
const loaded = await options.loadSnapshot();
if (!loaded.ok) return loaded;
const { loadSnapshot: _loaded, ...rest } = options;
return ok({ ...rest, snapshot: loaded.value });
return this.#serialize((admittedThrough) =>
this.#openWithinTransition(admittedThrough, resolve, options),
);
}
async #openWithinTransition(
@@ -19,11 +19,6 @@ import { err, type Result } from "../../domain/result.js";
import type { SessionProviderRoute } from "./SessionProviderRouter.js";
import { SessionProviderRouter } from "./SessionProviderRouter.js";
/** Read a staged snapshot once the open holds its place in the session order. */
export type AnalysisSnapshotLoader = () => Promise<
Result<AnalysisSnapshot, AnalysisError>
>;
export interface BinarySessionOpenOptions {
readonly signal?: AbortSignal;
readonly targetKind?: BinaryTarget["kind"];
@@ -31,11 +26,6 @@ export interface BinarySessionOpenOptions {
readonly architecture?: BinaryArchitecture;
readonly entryPoint?: string;
readonly snapshot?: AnalysisSnapshot;
/**
* Load the snapshot inside the open's turn instead of before it, so calls
* dispatched after the open cannot overtake its file read.
*/
readonly loadSnapshot?: AnalysisSnapshotLoader;
readonly providerId?: AnalysisProviderSelector;
}
@@ -11,7 +11,6 @@ import type { AnalysisProviderSelector } from "../../contracts/providerSelection
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
import type { Result } from "../../domain/result.js";
import type { EvidenceIntegrityError } from "../../domain/evidenceErrors.js";
import type { AnalysisSnapshotLoader } from "./BinarySessionOpen.js";
import type { WorkflowSnapshotRecordInput } from "./BinarySessionRecords.js";
import type {
@@ -72,7 +71,6 @@ export interface BinarySessionPort
readonly architecture?: BinaryArchitecture;
readonly entryPoint?: string;
readonly snapshot?: AnalysisSnapshot;
readonly loadSnapshot?: AnalysisSnapshotLoader;
readonly providerId?: AnalysisProviderSelector;
},
): Promise<Result<BinaryTarget, AnalysisError>>;
+69 -119
View File
@@ -99,62 +99,51 @@ export const captureResources = (
return { items: [] };
}
const items: CapturedResource[] = [];
for (const resource of frameTreeResources(root, (count) =>
completeness?.exclude("resources", "invalid_protocol_value", count),
for (const tree of walkFrameTrees(root, (omittedCount) =>
completeness?.exclude("resources", "invalid_protocol_value", omittedCount),
)) {
const url = allowedSanitizedUrl(resource.url, allowedOrigins);
if (url === undefined || url.origin === null) {
completeness?.exclude(
"resources",
exclusionReasonForUrl(cdpStringValue(resource.url)),
);
continue;
}
const type = cdpStringValue(resource.type);
const mimeType = cdpStringValue(resource.mimeType);
if (type === undefined || type.length === 0 || mimeType === undefined) {
completeness?.exclude("resources", "invalid_protocol_value");
continue;
}
const rawContentSize = numberValue(resource.contentSize);
const contentSize =
rawContentSize !== undefined && rawContentSize >= 0
? rawContentSize
: null;
if (resource.contentSize !== undefined && contentSize === null)
completeness?.exclude("resources", "invalid_protocol_value");
items.push({
rawUrl: cdpStringValue(resource.url) ?? "",
url: url.url,
origin: url.origin,
type,
mime_type: mimeType,
content_size: contentSize,
});
}
return { items };
};
/**
* Yield resource records from every frame of a resource tree. A missing or
* non-array `resources` list, or a non-record entry, is reported as invalid
* rather than read as an empty frame.
*/
export const frameTreeResources = function* (
root: UnknownRecord,
onInvalid: (count: number | null) => void,
): Generator<UnknownRecord> {
for (const tree of walkFrameTrees(root, onInvalid)) {
if (!Array.isArray(tree.resources)) {
onInvalid(null);
completeness?.exclude("resources", "invalid_protocol_value", null);
continue;
}
for (const entry of tree.resources) {
const resource = recordValue(entry);
if (resource === undefined) onInvalid(1);
else yield resource;
if (resource === undefined) {
completeness?.exclude("resources", "invalid_protocol_value");
continue;
}
const url = allowedSanitizedUrl(resource.url, allowedOrigins);
if (url === undefined || url.origin === null) {
completeness?.exclude(
"resources",
exclusionReasonForUrl(cdpStringValue(resource.url)),
);
continue;
}
const type = cdpStringValue(resource.type);
const mimeType = cdpStringValue(resource.mimeType);
if (type === undefined || type.length === 0 || mimeType === undefined) {
completeness?.exclude("resources", "invalid_protocol_value");
continue;
}
const rawContentSize = numberValue(resource.contentSize);
const contentSize =
rawContentSize !== undefined && rawContentSize >= 0
? rawContentSize
: null;
if (resource.contentSize !== undefined && contentSize === null)
completeness?.exclude("resources", "invalid_protocol_value");
items.push({
rawUrl: cdpStringValue(resource.url) ?? "",
url: url.url,
origin: url.origin,
type,
mime_type: mimeType,
content_size: contentSize,
});
}
}
return { items };
};
export const walkFrameTrees = function* (
@@ -228,9 +217,7 @@ export function* captureDom(
let unresolvedSvgNodes = 0;
if (!Array.isArray(root.strings))
excludeDocumentEvidence(completeness, "invalid_protocol_value", null);
for (const document of snapshotDocuments(root.documents, (count) =>
excludeDocumentEvidence(completeness, "invalid_protocol_value", count),
)) {
for (const document of snapshotDocuments(root.documents, completeness)) {
const documentUrl = cdpSnapshotString(strings, document.documentURL);
if (documentUrl === undefined) {
excludeDocumentEvidence(completeness, "invalid_protocol_value");
@@ -249,25 +236,40 @@ export function* captureDom(
continue;
}
const baseUrl = baseUrlValue || documentUrl;
const documentNodes = snapshotDocumentNodes(document.nodes);
const documentNodes = recordValue(document.nodes);
if (documentNodes === undefined) {
excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
const {
nodeTypes: rawNodeTypes,
nodeNames: rawNodeNames,
nodeValues: rawNodeValues,
parents: rawParents,
attributes: rawAttributes,
backendNodeIds,
} = documentNodes;
if (documentNodes.excessAttributes > 0)
const rawNodeTypes = numberArray(documentNodes.nodeType);
const rawNodeNames = numberArray(documentNodes.nodeName);
const rawNodeValues = numberArray(documentNodes.nodeValue);
const rawParents = numberArray(documentNodes.parentIndex);
const rawAttributes = Array.isArray(documentNodes.attributes)
? documentNodes.attributes
: undefined;
if (
rawNodeTypes === undefined ||
rawNodeNames === undefined ||
rawNodeValues === undefined ||
rawParents === undefined ||
rawAttributes === undefined ||
rawNodeNames.length !== rawNodeTypes.length ||
rawNodeValues.length !== rawNodeTypes.length ||
rawParents.length !== rawNodeTypes.length
) {
excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
if (rawAttributes.length > rawNodeTypes.length)
excludeDocumentEvidence(
completeness,
"invalid_protocol_value",
documentNodes.excessAttributes,
rawAttributes.length - rawNodeTypes.length,
);
const backendNodeIds = Array.isArray(documentNodes.backendNodeId)
? documentNodes.backendNodeId
: [];
const baseIndex = total;
total += rawNodeTypes.length;
const ancestry = createDocumentSvgAncestry({
@@ -387,74 +389,22 @@ const excludeDocumentEvidence = (
completeness?.exclude("metadata", reason, count);
};
/**
* Read DOMSnapshot documents. A snapshot always describes at least one
* document, so a missing or empty list is invalid rather than an empty DOM.
*/
export const snapshotDocuments = (
const snapshotDocuments = (
value: unknown,
onInvalid: (count: number | null) => void,
completeness: CdpCaptureCompleteness | undefined,
): readonly UnknownRecord[] => {
if (!Array.isArray(value) || value.length === 0) {
onInvalid(null);
excludeDocumentEvidence(completeness, "invalid_protocol_value", null);
return [];
}
return value.flatMap((item) => {
const document = recordValue(item);
if (document === undefined) onInvalid(1);
if (document === undefined)
excludeDocumentEvidence(completeness, "invalid_protocol_value");
return document === undefined ? [] : [document];
});
};
/** One snapshot document's parallel node arrays, validated for every consumer. */
export interface SnapshotDocumentNodes {
readonly nodeTypes: readonly (number | undefined)[];
readonly nodeNames: readonly (number | undefined)[];
readonly nodeValues: readonly (number | undefined)[];
readonly parents: readonly (number | undefined)[];
readonly attributes: readonly unknown[];
readonly backendNodeIds: readonly unknown[];
/** Attribute rows beyond the node count describe no node. */
readonly excessAttributes: number;
}
/** Missing or misaligned node arrays make the document's nodes unknown. */
export const snapshotDocumentNodes = (
value: unknown,
): SnapshotDocumentNodes | undefined => {
const nodes = recordValue(value);
if (nodes === undefined) return undefined;
const nodeTypes = numberArray(nodes.nodeType);
const nodeNames = numberArray(nodes.nodeName);
const nodeValues = numberArray(nodes.nodeValue);
const parents = numberArray(nodes.parentIndex);
const attributes = Array.isArray(nodes.attributes)
? nodes.attributes
: undefined;
if (
nodeTypes === undefined ||
nodeNames === undefined ||
nodeValues === undefined ||
parents === undefined ||
attributes === undefined ||
nodeNames.length !== nodeTypes.length ||
nodeValues.length !== nodeTypes.length ||
parents.length !== nodeTypes.length
)
return undefined;
return {
nodeTypes,
nodeNames,
nodeValues,
parents,
attributes,
backendNodeIds: Array.isArray(nodes.backendNodeId)
? nodes.backendNodeId
: [],
excessAttributes: Math.max(0, attributes.length - nodeTypes.length),
};
};
function* captureDomMetadata(
pending: readonly PendingDomMetadata[],
completeness: CdpCaptureCompleteness | undefined,
+25 -47
View File
@@ -23,7 +23,7 @@ interface PendingCommand {
readonly sessionId?: string;
readonly resolve: (value: unknown) => void;
readonly reject: (error: AnalysisError) => void;
readonly dispose: () => void;
readonly removeAbort: () => void;
}
/** Correlated JSON-RPC transport for one CDP target operation. */
@@ -79,8 +79,6 @@ export class CdpConnection {
`CDP maxPayloadBytes must be a positive safe integer no greater than ${String(MAX_WEBSOCKET_PAYLOAD_BYTES)}.`,
);
const socket = new WebSocket(url, {
// waitForOpen owns the deadline, cancellation, cleanup, and typed errors;
// keep ws's timer disabled so handshake failures have one owner.
handshakeTimeout: 0,
maxPayload: maxPayloadBytes,
perMessageDeflate: false,
@@ -117,7 +115,7 @@ export class CdpConnection {
return () => this.#protocolFailureListeners.delete(listener);
}
/** Bound each response separately from the caller-selected observation window. */
/** Execute one command, optionally within a flat target session. */
async send(
method: string,
params: Readonly<Record<string, unknown>> = {},
@@ -132,51 +130,41 @@ export class CdpConnection {
throw new AnalysisCancelledError(this.operation);
const id = this.#nextId;
this.#nextId += 1;
const message = JSON.stringify({
id,
method,
params,
...(sessionId === undefined ? {} : { sessionId }),
});
return await new Promise((resolve, reject) => {
const onAbort = (): void => {
const pending = this.#pending.get(id);
if (pending === undefined) return;
this.#complete(id, pending);
pending.removeAbort();
this.#pending.delete(id);
reject(new AnalysisCancelledError(this.operation));
};
const timer = setTimeout(() => {
const pending = this.#pending.get(id);
if (pending === undefined) return;
this.#complete(id, pending);
reject(
new BrowserObservationError(this.operation, "timeout", {
detail: `CDP command ${method} exceeded its ${String(WEB_RUNTIME_LIMITS.commandTimeoutMs)} ms response deadline.`,
}),
);
}, WEB_RUNTIME_LIMITS.commandTimeoutMs);
signal?.addEventListener("abort", onAbort, { once: true });
this.#pending.set(id, {
method,
...(sessionId === undefined ? {} : { sessionId }),
resolve,
reject,
dispose: () => {
clearTimeout(timer);
signal?.removeEventListener("abort", onAbort);
removeAbort: () => signal?.removeEventListener("abort", onAbort),
});
this.socket.send(
JSON.stringify({
id,
method,
params,
...(sessionId === undefined ? {} : { sessionId }),
}),
(error) => {
if (error === undefined || error === null) return;
const pending = this.#pending.get(id);
if (pending === undefined) return;
this.#complete(id, pending);
reject(
new BrowserObservationError(this.operation, "disconnected", {
cause: error,
}),
);
},
});
this.socket.send(message, (error) => {
if (error === undefined || error === null) return;
const pending = this.#pending.get(id);
if (pending === undefined) return;
this.#complete(id, pending);
reject(
new BrowserObservationError(this.operation, "disconnected", {
cause: error,
}),
);
});
);
});
}
@@ -294,7 +282,7 @@ export class CdpConnection {
}
#complete(id: number, pending: PendingCommand): void {
pending.dispose();
pending.removeAbort();
this.#pending.delete(id);
}
@@ -382,21 +370,11 @@ const waitForOpen = async (
onFailure(new AnalysisCancelledError(operation));
};
const cleanup = (): void => {
clearTimeout(timer);
socket.off("open", onOpen);
socket.off("error", onFailure);
socket.off("close", onClose);
signal?.removeEventListener("abort", onAbort);
};
const timer = setTimeout(() => {
terminateSocket(socket);
cleanup();
reject(
new BrowserObservationError(operation, "timeout", {
detail: `CDP WebSocket handshake exceeded its ${String(WEB_RUNTIME_LIMITS.commandTimeoutMs)} ms deadline.`,
}),
);
}, WEB_RUNTIME_LIMITS.commandTimeoutMs);
socket.once("open", onOpen);
socket.once("error", onFailure);
socket.once("close", onClose);
+38 -45
View File
@@ -12,15 +12,11 @@ import {
delayWithCancellation,
numberValue,
recordValue,
recordsValue,
requiredRecord,
cdpStringValue,
} from "./CdpCaptureValues.js";
import {
frameTreeResources,
snapshotDocumentNodes,
snapshotDocuments,
walkFrameTrees,
} from "./CdpCaptureDocuments.js";
import { walkFrameTrees } from "./CdpCaptureDocuments.js";
import {
ingestElectronScriptEvent,
type ElectronScriptDraft,
@@ -280,34 +276,38 @@ const captureResources = async (
const root = recordValue(requiredRecord(result).frameTree);
if (root === undefined)
throw new BrowserObservationError("inspect_web_page", "protocol_error");
const pending = [root];
const resources: ElectronPageInspection["resources"] = [];
const seen = new Set<string>();
for (const resource of frameTreeResources(root, (count) =>
completeness.exclude("resources", "invalid_protocol_value", count),
)) {
const path = await authorizedElectronFile(
cdpStringValue(resource.url) ?? "",
);
if (path === undefined) {
completeness.exclude("resources", "out_of_target_scope");
continue;
while (pending.length > 0) {
const tree = pending.shift();
if (tree === undefined) break;
for (const resource of recordsValue(tree.resources)) {
const path = await authorizedElectronFile(
cdpStringValue(resource.url) ?? "",
);
if (path === undefined) {
completeness.exclude("resources", "out_of_target_scope");
continue;
}
const item = {
file_path: path,
type: cdpStringValue(resource.type) ?? "Other",
mime_type: cdpStringValue(resource.mimeType) ?? "",
content_size:
numberValue(resource.contentSize) === undefined
? null
: Math.max(0, numberValue(resource.contentSize) ?? 0),
};
const resourceKey = `electron_resource_${digestCanonicalValue(item, "CDP capture")}`;
if (seen.has(resourceKey)) continue;
seen.add(resourceKey);
resources.push({
resource_key: resourceKey,
...item,
});
}
const item = {
file_path: path,
type: cdpStringValue(resource.type) ?? "Other",
mime_type: cdpStringValue(resource.mimeType) ?? "",
content_size:
numberValue(resource.contentSize) === undefined
? null
: Math.max(0, numberValue(resource.contentSize) ?? 0),
};
const resourceKey = `electron_resource_${digestCanonicalValue(item, "CDP capture")}`;
if (seen.has(resourceKey)) continue;
seen.add(resourceKey);
resources.push({
resource_key: resourceKey,
...item,
});
pending.push(...recordsValue(tree.childFrames));
}
return resources.sort((left, right) =>
compareUnicodeCodePoints(left.resource_key, right.resource_key),
@@ -326,30 +326,23 @@ const captureDom = async (
context.signal,
),
);
const invalid = (count: number | null) =>
completeness.exclude("dom", "invalid_protocol_value", count);
if (!Array.isArray(result.strings)) invalid(null);
const strings = Array.isArray(result.strings)
? result.strings.map((value) => (typeof value === "string" ? value : ""))
: [];
const nodes: ElectronPageInspection["dom"]["nodes"] = [];
let total = 0;
for (const document of snapshotDocuments(result.documents, invalid)) {
for (const document of recordsValue(result.documents)) {
const documentUrl = stringAt(strings, document.documentURL);
if ((await authorizedElectronFile(documentUrl)) === undefined) {
completeness.exclude("dom", "out_of_target_scope");
continue;
}
// The shared snapshot owner validates the parallel node arrays.
const documentNodes = snapshotDocumentNodes(document.nodes);
if (documentNodes === undefined) {
invalid(1);
continue;
}
if (documentNodes.excessAttributes > 0)
invalid(documentNodes.excessAttributes);
const { nodeTypes, nodeNames, nodeValues, parents, attributes } =
documentNodes;
const nodeData = recordValue(document.nodes);
const nodeTypes = arrayValue(nodeData?.nodeType);
const nodeNames = arrayValue(nodeData?.nodeName);
const nodeValues = arrayValue(nodeData?.nodeValue);
const parents = arrayValue(nodeData?.parentIndex);
const attributes = arrayValue(nodeData?.attributes);
const baseIndex = nodes.length;
total += nodeTypes.length;
for (let index = 0; index < nodeTypes.length; index += 1) {
+6 -13
View File
@@ -4,7 +4,11 @@ import type {
} from "../domain/javascript/electronObservation.js";
import type { CdpConnection } from "./CdpConnection.js";
import { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js";
import { recordValue, cdpStringValue } from "./CdpCaptureValues.js";
import {
recordValue,
recordsValue,
cdpStringValue,
} from "./CdpCaptureValues.js";
import type { CdpEndpointTarget } from "./CdpEndpoint.js";
import { optionalCdpCommand } from "./CdpOptionalCommand.js";
import { authorizedElectronFile } from "./ElectronFileScope.js";
@@ -35,18 +39,7 @@ export const captureElectronWorkers = async (input: {
return [];
}
const workers: ElectronPageInspection["workers"] = [];
// A missing target list or a non-record entry is unreadable, not empty.
const targetInfos = recordValue(result)?.targetInfos;
if (!Array.isArray(targetInfos)) {
input.completeness.exclude("workers", "invalid_protocol_value", null);
return [];
}
for (const entry of targetInfos) {
const target = recordValue(entry);
if (target === undefined) {
input.completeness.exclude("workers", "invalid_protocol_value");
continue;
}
for (const target of recordsValue(recordValue(result)?.targetInfos)) {
const type = cdpStringValue(target.type) ?? "";
if (!type.includes("worker")) continue;
const openerTargetId = boundedTargetField(target.openerId);
@@ -1,167 +0,0 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect, it, vi } from "vitest";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { ELECTRON_FUSE_STATE_BYTES } from "../domain/electron/electronFuses.js";
import {
elfFuseImage,
OBSIDIAN_FUSE_BYTES,
peFuseImage,
thinFuseImage,
thinMacho,
universalFuseImage,
} from "../../tests/fixtures/electron/fuseImage.js";
import { AnalysisUnsupportedTargetError } from "../domain/analysisErrorCore.js";
import { assertNodeCliInspectArgumentsSupported } from "./PlaywrightElectronActiveFuses.js";
describe("assertNodeCliInspectArgumentsSupported", () => {
it("rejects thin Mach-O binary when EnableNodeCliInspectArguments fuse is disabled", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "disabled-app");
const image = thinFuseImage(OBSIDIAN_FUSE_BYTES); // index 3 is disabled
await writeFile(binaryPath, image.bytes);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).rejects.toThrow(AnalysisUnsupportedTargetError);
try {
await assertNodeCliInspectArgumentsSupported(binaryPath);
} catch (error) {
expect(error).toBeInstanceOf(AnalysisUnsupportedTargetError);
const targetError = error as AnalysisUnsupportedTargetError;
expect(targetError.operation).toBe("capture_electron_scenario");
expect(targetError.path).toBe(binaryPath);
expect(targetError.message).toContain("EnableNodeCliInspectArguments");
expect(targetError.remediationAction).toContain("inspect_electron_page");
}
});
it("passes when EnableNodeCliInspectArguments fuse is enabled", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "enabled-app");
const enabledFuses = [...OBSIDIAN_FUSE_BYTES];
enabledFuses[3] = ELECTRON_FUSE_STATE_BYTES.enabled;
const image = thinFuseImage(enabledFuses);
await writeFile(binaryPath, image.bytes);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).resolves.toBeUndefined();
});
it("passes when binary contains no fuse sentinel (fuse state unknown)", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "stock-node");
const image = thinMacho(0x0100000c, Buffer.from("no-sentinel-bytes-here"));
await writeFile(binaryPath, image);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).resolves.toBeUndefined();
});
it("passes gracefully when fuse reader throws a format or reading failure", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const nonMachoPath = join(root, "script.sh");
await writeFile(nonMachoPath, "#!/bin/sh\necho hello");
await expect(
assertNodeCliInspectArgumentsSupported(nonMachoPath),
).resolves.toBeUndefined();
});
it("rejects Windows PE binary when EnableNodeCliInspectArguments fuse is disabled", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "disabled-pe.exe");
const image = peFuseImage(OBSIDIAN_FUSE_BYTES);
await writeFile(binaryPath, image.bytes);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).rejects.toThrow(AnalysisUnsupportedTargetError);
});
it("rejects Linux ELF binary when EnableNodeCliInspectArguments fuse is disabled", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "disabled-elf");
const image = elfFuseImage(OBSIDIAN_FUSE_BYTES);
await writeFile(binaryPath, image.bytes);
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).rejects.toThrow(AnalysisUnsupportedTargetError);
});
it("rethrows when signal is aborted during reading", async () => {
const controller = new AbortController();
const readFuseImage = vi.fn(async () => {
controller.abort();
throw new Error("aborted during read");
});
await expect(
assertNodeCliInspectArgumentsSupported("/nonexistent/binary", {
signal: controller.signal,
readFuseImage,
}),
).rejects.toThrow();
expect(readFuseImage).toHaveBeenCalledOnce();
});
it("rejects macOS app bundle when framework binary has disabled inspect fuse", async () => {
const root = await createTestTempDirectory("rea-fuse-mac-bundle-");
const appBundle = join(root, "Obsidian.app");
const launcherBinary = join(appBundle, "Contents", "MacOS", "Obsidian");
const frameworkDir = join(
appBundle,
"Contents",
"Frameworks",
"Electron Framework.framework",
"Versions",
"A",
);
await mkdir(join(appBundle, "Contents", "MacOS"), { recursive: true });
await mkdir(frameworkDir, { recursive: true });
// Launcher binary has no sentinel:
await writeFile(launcherBinary, Buffer.from("dummy launcher binary"));
// Framework binary has the disabled fuse wire:
const frameworkBinary = join(frameworkDir, "Electron Framework");
await writeFile(frameworkBinary, thinFuseImage(OBSIDIAN_FUSE_BYTES).bytes);
await expect(
assertNodeCliInspectArgumentsSupported(launcherBinary),
).rejects.toThrow(AnalysisUnsupportedTargetError);
try {
await assertNodeCliInspectArgumentsSupported(launcherBinary);
} catch (error) {
expect(error).toBeInstanceOf(AnalysisUnsupportedTargetError);
const targetError = error as AnalysisUnsupportedTargetError;
expect(targetError.operation).toBe("capture_electron_scenario");
expect(targetError.path).toBe(launcherBinary);
expect(targetError.message).toContain("EnableNodeCliInspectArguments");
}
});
it("evaluates matching architecture slice in universal Mach-O binary", async () => {
const root = await createTestTempDirectory("rea-fuse-preflight-");
const binaryPath = join(root, "universal-app");
// universalFuseImage has left (x86_64) disabled, right (arm64) enabled
const image = universalFuseImage();
await writeFile(binaryPath, image.bytes);
if (process.arch === "arm64") {
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).resolves.toBeUndefined();
} else {
await expect(
assertNodeCliInspectArgumentsSupported(binaryPath),
).rejects.toThrow(AnalysisUnsupportedTargetError);
}
});
});
@@ -1,96 +0,0 @@
import { basename, dirname } from "node:path";
import { AnalysisUnsupportedTargetError } from "../domain/analysisErrorCore.js";
import { resolveElectronFuseTarget } from "../electron/ElectronFusePath.js";
import { readElectronFuseImage } from "../electron/ElectronFuseReader.js";
const OPERATION = "capture_electron_scenario" as const;
const NODE_CLI_INSPECT_ARGUMENTS_FUSE_INDEX = 3;
export interface InspectArgumentsPreflightOptions {
readonly signal?: AbortSignal;
readonly readFuseImage?: typeof readElectronFuseImage;
readonly resolveFuseTarget?: typeof resolveElectronFuseTarget;
}
const appBundleDirectoryOf = (executablePath: string): string | null => {
let current = dirname(executablePath);
while (true) {
if (basename(current).endsWith(".app")) return current;
const parent = dirname(current);
if (parent === current) return null;
current = parent;
}
};
/**
* Pre-flight verification for active Electron scenario capture.
*
* Playwright attaches to Electron processes using Node CLI inspect arguments
* (--inspect=0 or --remote-debugging-port=0). When an Electron executable disables
* the EnableNodeCliInspectArguments fuse, the runtime silently ignores inspect
* flags and Playwright hangs until the 60-second startup deadline expires.
*
* When the fuse wire is present and index 3 is explicitly disabled or removed,
* this check fast-fails with an AnalysisUnsupportedTargetError before any child
* process is launched. When the fuse state is unknown (missing sentinel, wire
* version != 1, or unsupported container format), launch proceeds normally.
*/
export const assertNodeCliInspectArgumentsSupported = async (
executablePath: string,
options?: InspectArgumentsPreflightOptions,
): Promise<void> => {
options?.signal?.throwIfAborted();
const reader = options?.readFuseImage ?? readElectronFuseImage;
const resolver = options?.resolveFuseTarget ?? resolveElectronFuseTarget;
try {
let targetPath = executablePath;
const bundleDir = appBundleDirectoryOf(executablePath);
if (bundleDir !== null) {
try {
const resolved = await resolver(bundleDir, options?.signal);
targetPath = resolved.file.path;
} catch (cause: unknown) {
if (options?.signal?.aborted) throw cause;
targetPath = executablePath;
}
}
const image = await reader(
targetPath,
options?.signal !== undefined ? { signal: options.signal } : undefined,
);
if (image.slices.length === 0) return;
const hostArchitecture = process.arch === "x64" ? "x86_64" : process.arch;
const targetSlice =
image.slices.find((slice) => slice.architecture === hostArchitecture) ??
image.slices[0];
if (targetSlice === undefined) return;
for (const sentinel of targetSlice.sentinels) {
if (sentinel.wire_version !== 1) continue;
const fuse = sentinel.fuses.find(
(entry) =>
entry.index === NODE_CLI_INSPECT_ARGUMENTS_FUSE_INDEX ||
entry.name === "EnableNodeCliInspectArguments",
);
if (fuse?.state === "disabled" || fuse?.state === "removed") {
throw new AnalysisUnsupportedTargetError(
OPERATION,
executablePath,
"Target binary disables the EnableNodeCliInspectArguments fuse; Node CLI inspector arguments are ignored and active scenario capture cannot attach.",
{
remediationAction:
"Use passive CDP observation tools (inspect_electron_page), or launch active capture against an Electron development build or a stock Electron runtime with the extracted app.",
},
);
}
}
} catch (cause: unknown) {
if (cause instanceof AnalysisUnsupportedTargetError) throw cause;
options?.signal?.throwIfAborted();
// Missing sentinels, format failures, or unreadable fuses leave inspection capability unknown; proceed with launch.
}
};
@@ -1,12 +1,5 @@
import { afterEach, expect, it, vi } from "vitest";
import { writeFile } from "node:fs/promises";
import { dirname, join } from "node:path";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import {
OBSIDIAN_FUSE_BYTES,
thinFuseImage,
} from "../../tests/fixtures/electron/fuseImage.js";
import { dirname } from "node:path";
import * as processOwnership from "../process/ProcessOwnership.js";
import * as electronActions from "./PlaywrightElectronActiveActions.js";
@@ -260,34 +253,3 @@ it("retains the initial snapshot when the final state read fails", async () => {
},
});
});
it("rejects active capture before launch when EnableNodeCliInspectArguments fuse is disabled", async () => {
const root = await createTestTempDirectory("rea-active-provider-fuse-");
const disabledBinary = join(root, "disabled-electron");
await writeFile(disabledBinary, thinFuseImage(OBSIDIAN_FUSE_BYTES).bytes);
const launchSpy = vi.fn(async () => {
throw new Error("launch should not be called when fuse is disabled");
});
const provider = new PlaywrightElectronActiveProvider({}, launchSpy);
const result = await provider.capture({
...electronActiveObservationInputSchema.parse({
executable_path: disabledBinary,
application_path: disabledBinary,
}),
application_root: root,
});
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Expected capture to fail");
expect(launchSpy).not.toHaveBeenCalled();
expect(result.error).toMatchObject({
_tag: "AnalysisUnsupportedTargetError",
operation: "capture_electron_scenario",
path: disabledBinary,
message: expect.stringContaining("EnableNodeCliInspectArguments"),
remediationAction: expect.stringContaining("inspect_electron_page"),
});
});
@@ -37,8 +37,6 @@ import {
runWithExecutionLimits,
} from "./PlaywrightElectronActiveActions.js";
import { projectElectronActiveCapture } from "./PlaywrightElectronActiveProjection.js";
import { readElectronFuseImage } from "../electron/ElectronFuseReader.js";
import { assertNodeCliInspectArgumentsSupported } from "./PlaywrightElectronActiveFuses.js";
const OPERATION = "capture_electron_scenario" as const;
const STARTUP_TIMEOUT_MS = 60_000;
@@ -63,7 +61,6 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
// Load Playwright only when a capture launches; MCP startup stays off its module graph.
private readonly launch: typeof electron.launch = async (options) =>
(await import("playwright-core"))._electron.launch(options),
private readonly readFuseImage: typeof readElectronFuseImage = readElectronFuseImage,
) {}
identity(): ProviderIdentity {
@@ -95,14 +92,6 @@ export class PlaywrightElectronActiveProvider implements ElectronActiveObservati
application: input.application_path,
root: input.application_root,
};
await runWithExecutionLimits(
assertNodeCliInspectArgumentsSupported(paths.executable, {
...(options.signal === undefined ? {} : { signal: options.signal }),
readFuseImage: this.readFuseImage,
}),
options.signal,
startupDeadline,
);
const captureBaseline =
await systemProcessOwnershipHost.captureBaseline?.(options.signal);
if (options.signal?.aborted)
+7 -70
View File
@@ -290,7 +290,7 @@ it.each([
},
);
it("keeps unsafe numeric lexemes in extension and mandatory schema fields", () => {
it("keeps unsafe extension numeric lexemes and rejects unsafe mandatory schema numbers", () => {
const text = JSON.stringify(historicalHar());
const result = decodeHarCapture(
text.replace('"cache":{}', '"cache":{},"_big":9007199254740993'),
@@ -302,17 +302,12 @@ it("keeps unsafe numeric lexemes in extension and mandatory schema fields", () =
literal: "9007199254740993",
});
expect(result.records[0]?.reported).toMatchObject({ _big: null });
// The pinned schema admits a valid integer the evidence cannot publish exactly.
const mandatory = decodeHarCapture(
text.replace('"bodySize":0', '"bodySize":9007199254740993'),
[],
).records[0];
expect(mandatory?.numeric_literals).toContainEqual({
pointer: "/request/bodySize",
producer_type: "json-number",
literal: "9007199254740993",
});
expect(mandatory?.reported).toMatchObject({ request: { bodySize: null } });
expect(() =>
decodeHarCapture(
text.replace('"bodySize":0', '"bodySize":9007199254740993'),
[],
),
).toThrow("HAR schema validation failed");
});
it.each(["container", "record"] as const)(
@@ -698,61 +693,3 @@ it("ignores one leading byte-order mark, as HAR 1.2 requires of readers", () =>
decodeCaptureText(Buffer.from([0x7b, 0xff, 0x7d]), "HAR"),
).toThrow("HAR is not valid UTF-8 JSON.");
});
const reportedAt = (reported: unknown, pointer: string): unknown =>
pointer
.split("/")
.slice(1)
.reduce<unknown>(
(value, key) =>
value !== null && typeof value === "object"
? Reflect.get(value, key)
: undefined,
reported,
);
it.each([
{
from: '"time":1.25',
literal: "0.123456789012345678901",
pointer: "/time",
},
{ from: '"bodySize":99', literal: "1e400", pointer: "/response/bodySize" },
])(
"admits a schema-valid number that evidence cannot publish exactly: $literal",
({ from, literal, pointer }) => {
const key = from.slice(0, from.indexOf(":") + 1);
const text = JSON.stringify(historicalHar()).replace(
from,
`${key}${literal}`,
);
expect(text).toContain(`${key}${literal}`);
const record = decodeHarCapture(text, []).records[0];
// The exact producer lexeme is retained, and the published value stays unknown.
expect(record?.numeric_literals).toContainEqual({
pointer,
producer_type: "json-number",
literal,
});
expect(reportedAt(record?.reported, pointer)).toBeNull();
},
);
it.each([
"9007199254740993.5",
"1e-400",
"1.0000000000000000001",
`1${"0".repeat(309)}.5`,
`-1${"0".repeat(309)}.5`,
])(
"judges an integer field by the lexeme when rounding or overflow would hide a fraction: %s",
(literal) => {
const text = JSON.stringify(historicalHar()).replace(
'"bodySize":99',
`"bodySize":${literal}`,
);
expect(() => decodeHarCapture(text, [])).toThrow(
"HAR schema validation failed: type (must be integer).",
);
},
);
+10 -37
View File
@@ -101,7 +101,6 @@ export const decodeHarCapture = (
};
};
/** Evidence projection when `redact` is true; the upstream schema validation copy otherwise. */
const projectHar = (
value: unknown,
sensitiveValues: readonly string[],
@@ -110,18 +109,6 @@ const projectHar = (
const redactor = new CaptureRedaction(sensitiveValues);
const numeric_literals: Projection["numeric_literals"] = [];
const binary_fields: Projection["binary_fields"] = [];
const projectNumber = (item: LosslessNumber, pointer: string): JsonValue => {
// Evidence publishes an unsafe number as null; the schema must still judge it as a number.
if (!redact) return validationNumber(item.value);
const number = redactor.number(item, pointer, redact);
if (number === null) return null;
numeric_literals.push({
pointer,
producer_type: "json-number",
literal: number.literal,
});
return number.reported;
};
const visit = (
item: unknown,
pointer: string,
@@ -140,7 +127,16 @@ const projectHar = (
redactor.harCredential(pointer, parent)
)
return redactor.credential(pointer);
if (item instanceof LosslessNumber) return projectNumber(item, pointer);
if (item instanceof LosslessNumber) {
const number = redactor.number(item, pointer, redact);
if (number === null) return null;
numeric_literals.push({
pointer,
producer_type: "json-number",
literal: number.literal,
});
return number.reported;
}
if (typeof item === "string") {
if (
redact &&
@@ -239,29 +235,6 @@ const projectHar = (
};
};
/**
* A JSON number lexeme as the validation copy carries it: its nearest double, unless rounding or
* overflow erased a fractional part, so the schema's `integer` checks the lexeme, not the double.
*/
const validationNumber = (literal: string): number => {
const nearest = Number(literal);
if (integerLexeme(literal)) return nearest;
if (Number.isFinite(nearest) && !Number.isInteger(nearest)) return nearest;
return literal.startsWith("-") ? -0.5 : 0.5;
};
/** Whether a JSON number lexeme denotes an integer, decided from its digits and exponent. */
const integerLexeme = (literal: string): boolean => {
const match = /^-?(\d+)(?:\.(\d+))?(?:[eE]([+-]?\d+))?$/.exec(literal);
if (match === null) return false;
const [, whole = "", fraction = "", exponent = "0"] = match;
const digits = `${whole}${fraction}`;
const significant = digits.replace(/0+$/, "");
if (/^0*$/.test(significant)) return true;
const trailingZeros = digits.length - significant.length;
return Number(exponent) - fraction.length + trailingZeros >= 0;
};
const decodeHarBase64 = (value: string, pointer: string): Buffer => {
const compact = value.replace(/[\t\n\r ]/g, "");
const bytes = Buffer.from(compact, "base64");
+1 -2
View File
@@ -235,8 +235,7 @@ const routesToJavaScriptAnalysis = async (
const isJavaScriptApplicationPath = async (path: string): Promise<boolean> => {
const lower = path.toLowerCase();
if (lower.endsWith(".asar")) return true;
// `A.app/` and `A.app/.` name the same bundle as `A.app`.
if (lower.replace(/(?:\/\.?)+$/u, "").endsWith(".app")) return false;
if (lower.endsWith(".app")) return false;
try {
return (await stat(path)).isDirectory();
} catch (cause: unknown) {
+55 -1
View File
@@ -191,10 +191,19 @@ describe("compact advertised input schema", () => {
const properties = schema.properties as Record<string, unknown>;
expect(properties.normalized_result).toEqual({
type: ["null", "boolean", "object", "array", "number", "string"],
items: {},
});
expect(properties.literal).toEqual({ const: {} });
const validate = new Ajv2020({ strict: false }).compile(schema);
for (const normalized_result of [null, true, {}, [], 42, "result"])
for (const normalized_result of [
null,
true,
{},
[],
[[1], "x"],
42,
"result",
])
expect(validate({ normalized_result })).toBe(true);
});
@@ -267,6 +276,7 @@ describe("compact input schema budget and reduction", () => {
type: "array",
items: {
type: ["null", "boolean", "object", "array", "number", "string"],
items: {},
},
},
label: { type: ["string", "null"] },
@@ -335,3 +345,47 @@ describe("compact input schema budget and reduction", () => {
expect(serializedBytes(schema)).toBeLessThanOrEqual(13312);
});
});
it("gives every advertised array type an item schema for strict clients", () => {
const arraysWithoutItems = (value: unknown): number => {
if (Array.isArray(value))
return value.reduce(
(count: number, child) => count + arraysWithoutItems(child),
0,
);
if (typeof value !== "object" || value === null) return 0;
const record = value as Record<string, unknown>;
const types = Array.isArray(record.type) ? record.type : [record.type];
return (
(types.includes("array") && record.items === undefined ? 1 : 0) +
Object.values(record).reduce(
(count: number, child) => count + arraysWithoutItems(child),
0,
)
);
};
// A large nested property forces the reduced form at 1 KiB while the
// shallow properties, including the array, remain advertised.
for (const [budget, expected] of [
[13312, "inline"],
[1024, "reduced"],
] as const) {
const { schema, presentation } = compactAdvertisedInputSchema(
{
type: "object",
properties: {
capture: budget === 1024 ? oversizedPairSchema() : {},
value: {},
values: { type: "array", items: {} },
},
},
budget,
);
expect(presentation, String(budget)).toBe(expected);
expect(
(schema.properties as Record<string, unknown>).values,
String(budget),
).toMatchObject({ type: "array", items: expect.any(Object) });
expect(arraysWithoutItems(schema), String(budget)).toBe(0);
}
});
+5 -4
View File
@@ -23,6 +23,9 @@ const jsonValueTypes = [
"number",
"string",
];
// Strict clients such as VS Code reject an array type without `items`, so
// the explicit any-value form also accepts every array element.
const jsonValueSchema = { type: jsonValueTypes, items: {} };
export type AdvertisedInputSchemaPresentation = "inline" | "reduced";
@@ -157,7 +160,7 @@ const inlineWithinBudget = (
yield "]";
} else if (isSchemaObject(value)) {
if (position === "schema" && Object.keys(value).length === 0) {
yield JSON.stringify({ type: jsonValueTypes });
yield JSON.stringify(jsonValueSchema);
return;
}
if (position === "schema" && typeof value.$ref === "string") {
@@ -293,9 +296,7 @@ const reduceToShallowProperties = (
...(type === undefined ? {} : { type }),
// Reduced inputs intentionally omit nested validation, but clients
// still require an item schema to accept an array declaration.
...(types?.includes("array")
? { items: { type: jsonValueTypes } }
: {}),
...(types?.includes("array") ? { items: jsonValueSchema } : {}),
...description,
},
];
-21
View File
@@ -42,27 +42,6 @@ it("omits REA_JADX_MCP_JAR from the provider registration environment when it is
expect(environment).not.toHaveProperty("REA_JADX_MCP_JAR");
});
it("carries a validated REA_KUNA_COMMAND into the provider registration environment", async () => {
expect(DOCTOR_PROVIDER_IDS).toContain("kuna");
const root = await createTestTempDirectory("rea-kuna-registration-");
const kuna = join(root, "kuna");
await writeFile(kuna, "#!/bin/sh\nprintf 'kuna 1.833\\n'\n", { mode: 0o700 });
const inspections = await createSystemDoctorHost({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
}).providerInspections?.();
expect(inspections?.find(({ id }) => id === "kuna")).toMatchObject({
available: true,
providerVersion: "1.833",
});
expect(providerRegistrationEnvironment(inspections ?? [])).toMatchObject({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
});
});
const createFakeJdk = async (root: string): Promise<string> => {
const javaHome = join(root, "fake-jdk");
const bin = join(javaHome, "bin");
-2
View File
@@ -7,7 +7,6 @@ import { fileURLToPath } from "node:url";
import { inspectJadxProvider } from "./android/JadxDoctor.js";
import { inspectSystemGhidraProvider } from "./ghidra/GhidraDoctor.js";
import { inspectIdaRegistration } from "./ida/IdaDoctor.js";
import { inspectKunaProvider } from "./kuna/KunaDoctor.js";
import { readIdaConfiguration } from "./ida/IdaConfiguration.js";
import { parseJebMcpEndpoint } from "./jeb/JebMcpEndpoint.js";
import { selectLinuxPrivateDisplayStrategy } from "./hopper/LinuxPrivateDisplayProbe.js";
@@ -46,7 +45,6 @@ export const createSystemDoctorHost = (
await inspectSystemGhidraProvider(environment),
inspectIdaRegistration(environment.REA_IDA_MCP_CONFIG),
await inspectJadxProvider(environment),
await inspectKunaProvider(environment),
],
linuxDemoRuntimeCheck: () => inspectLinuxPrivateDisplay(environment),
});
@@ -48,38 +48,6 @@ describe("Electron identity through Webpack externals", () => {
expect(electronOf(source).browser_windows).toEqual([]);
});
const GETTER_CONSUMER = "const s=n.n(n(45840));new (s().BrowserWindow)({});";
it.each([
["Reflect.set", 'Reflect.set(n,"n",e=>()=>({BrowserWindow:class{}}));'],
[
"Reflect.defineProperty",
'Reflect.defineProperty(n,"n",{value:e=>()=>0});',
],
["delete", "delete n.n;"],
["a destructuring write", "({g:n.n}={g:e=>()=>0});"],
["an unknown key", "n[key]=e=>()=>0;"],
])(
"does not infer Electron after a getter write through %s",
(_name, write) => {
expect(
electronOf(bootstrap(EXTERNAL, GETTER_CONSUMER, `${write}n(100);`))
.browser_windows,
).toEqual([]);
},
);
it("keeps the getter after a reflective write to another loader key", () => {
expect(
electronOf(
bootstrap(
EXTERNAL,
GETTER_CONSUMER,
'Object.defineProperty(n,"d",{value:1});n(100);',
),
).browser_windows,
).toHaveLength(1);
});
it("follows the loader passed to a module factory", () => {
const electron = electronOf(bootstrap(EXTERNAL, CONSUMER));
expect(electron.browser_windows).toMatchObject([
@@ -112,137 +112,3 @@ describe("Electron option evidence", () => {
);
});
});
describe("Electron empty option keys", () => {
describe.each([
["plain empty string", '[""]'],
["empty template", "[``]"],
["wrapped empty string", '["" as const]'],
])("unrelated %s option keys", (_label, key) => {
it("retains an earlier explicit webPreferences object", () => {
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow } = require("electron");
new BrowserWindow({
webPreferences: { preload: "./kept.js", sandbox: true },
${key}: true
});
`);
expect(analysis.parse_error_count).toBe(0);
expect(analysis.parse_status).toBe("complete");
expect(analysis.electron.browser_windows).toMatchObject([
{
web_preferences_status: "object-literal",
preload_path: "./kept.js",
preload_resolution_context: "module-specifier",
web_preferences: [
{
name: "preload",
value: { status: "literal", value: "./kept.js" },
},
{ name: "sandbox", value: { status: "literal", value: true } },
],
},
]);
});
it("retains an earlier explicit preload within webPreferences", () => {
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow } = require("electron");
new BrowserWindow({ webPreferences: {
preload: "./kept.js", sandbox: true, ${key}: false
} });
`);
expect(analysis.parse_error_count).toBe(0);
expect(analysis.parse_status).toBe("complete");
expect(analysis.electron.browser_windows).toMatchObject([
{
web_preferences_status: "object-literal",
preload_path: "./kept.js",
preload_resolution_context: "module-specifier",
web_preferences: [
{ name: "", value: { status: "literal", value: false } },
{
name: "preload",
value: { status: "literal", value: "./kept.js" },
},
{ name: "sandbox", value: { status: "literal", value: true } },
],
},
]);
});
it("retains an earlier explicit utility-process service name", () => {
const analysis = analyzeJavaScriptStaticSource(`
const { utilityProcess } = require("electron");
utilityProcess.fork("./worker.js", [], {
serviceName: "kept", ${key}: true
});
`);
expect(analysis.parse_error_count).toBe(0);
expect(analysis.parse_status).toBe("complete");
expect(analysis.electron.utility_processes).toMatchObject([
{ module_path: "./worker.js", service_name: "kept" },
]);
});
});
it.each([
["dynamic key", "[key]: null"],
["spread", "...options"],
])(
"keeps a preceding %s override unknown after an unrelated empty template key",
(_label, override) => {
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow, utilityProcess } = require("electron");
new BrowserWindow({
webPreferences: { preload: "./maybe.js" },
${override}, [\`\`]: true
});
new BrowserWindow({ webPreferences: {
preload: "./maybe.js", ${override}, [\`\`]: true
} });
utilityProcess.fork("./worker.js", [], {
serviceName: "maybe", ${override}, [\`\`]: true
});
`);
expect(analysis.parse_error_count).toBe(0);
expect(analysis.parse_status).toBe("partial");
expect(analysis.electron.browser_windows).toMatchObject([
{
web_preferences_status: "dynamic",
web_preferences: [],
preload_path: null,
preload_resolution_context: null,
},
{
web_preferences_status: "object-literal",
preload_path: null,
preload_resolution_context: null,
},
]);
expect(analysis.electron.browser_windows[1]?.web_preferences).toEqual(
expect.arrayContaining([
{
name: "",
value: { status: "literal", value: true, expression: null },
},
{
name: "preload",
value: { status: "literal", value: "./maybe.js", expression: null },
},
expect.objectContaining({
name: expect.stringMatching(/^\[(?:dynamic|spread)@/u),
value: expect.objectContaining({ status: "dynamic", value: null }),
}),
]),
);
expect(analysis.electron.utility_processes).toMatchObject([
{ module_path: "./worker.js", service_name: null },
]);
},
);
});
@@ -65,8 +65,9 @@ export const objectProperty = (
if (!t.isObjectExpression(object)) return { status: "dynamic" };
for (const property of object.properties.toReversed()) {
if (t.isSpreadElement(property)) return { status: "dynamic" };
const key = semanticStaticPropertyKey(property.key, property.computed);
if (key === null) return { status: "dynamic" };
const key = semanticStaticPropertyName(property.key, property.computed);
if (key === "" && !t.isStringLiteral(property.key))
return { status: "dynamic" };
if (key !== name) continue;
return t.isObjectProperty(property)
? { status: "explicit", property }
@@ -2,12 +2,7 @@ import * as t from "@babel/types";
import { javascriptBootstrapAt } from "./javascriptBootstrap.js";
import { proveJavaScriptBootstrapIntegrity } from "./javascriptBootstrapIntegrity.js";
import {
semanticStaticPropertyKey,
unwrapJavaScriptExpression,
} from "./javascriptAstValues.js";
import { collectJavaScriptIdentityWritesSteps } from "./javascriptElectronMemberWrites.js";
import { createSemanticModuleInterop } from "./javascriptModuleInterop.js";
import { semanticStaticPropertyKey } from "./javascriptAstValues.js";
import {
semanticMemberChain,
semanticRequireOrigin,
@@ -23,10 +18,7 @@ import {
moduleFactory,
modulePropertyName,
} from "./javascriptStaticAnalysisHelpers.js";
import {
completeSemanticSteps,
traverseJavaScriptAst,
} from "./javascriptSemanticTraversal.js";
import { traverseJavaScriptAst } from "./javascriptSemanticTraversal.js";
/** Loader bindings and literal externals proven for one standalone bootstrap. */
interface BootstrapExternals {
@@ -150,38 +142,60 @@ const canonicalWebpackInteropBindings = (
bootstrapBody: t.BlockStatement,
state: JavaScriptSemanticAnalysisState,
): ReadonlySet<string> => {
// The shared identity-write owner covers assignment, delete, destructuring,
// and Object/Reflect mutators; any write that may select `n` counts.
const writes = new Set<t.Node>();
const interop = createSemanticModuleInterop(state);
completeSemanticSteps(
collectJavaScriptIdentityWritesSteps(
state.program,
(root, kind) => {
const node = unwrapJavaScriptExpression(root).node;
return kind === "member" &&
t.isIdentifier(node) &&
loaderBindingIds.has(
resolveSemanticBindingState(state, node, node.name)?.bindingId ??
"",
)
? [["loader"]]
: [];
},
interop.intrinsicInvocation,
interop.staticString,
interop.isFreshObject,
(node, path) => {
if (path.length < 2 || path[1] === "n" || path[1] === null)
writes.add(node);
},
),
);
const writes = new Map<string, t.AssignmentExpression[]>();
let reflectiveWrite = false;
traverseJavaScriptAst(state.program, {
enter: (node) => {
if (t.isAssignmentExpression(node) && t.isMemberExpression(node.left)) {
const target = node.left;
const bindingId = t.isIdentifier(target.object)
? resolveSemanticBindingState(
state,
target.object,
target.object.name,
)?.bindingId
: undefined;
if (
semanticStaticPropertyKey(target.property, target.computed) === "n" &&
bindingId !== undefined &&
loaderBindingIds.has(bindingId)
)
writes.set(bindingId, [...(writes.get(bindingId) ?? []), node]);
}
if (
!t.isCallExpression(node) ||
!t.isMemberExpression(node.callee) ||
!t.isIdentifier(node.callee.object, { name: "Object" }) ||
![
"defineProperty",
"defineProperties",
"assign",
"setPrototypeOf",
].includes(
semanticStaticPropertyKey(
node.callee.property,
node.callee.computed,
) ?? "",
) ||
!t.isIdentifier(node.arguments[0])
)
return;
reflectiveWrite ||= loaderBindingIds.has(
resolveSemanticBindingState(
state,
node.arguments[0],
node.arguments[0].name,
)?.bindingId ?? "",
);
},
});
const proven = new Set<string>();
const [write] = writes;
if (reflectiveWrite) return proven;
const allWrites = [...writes.values()].flat();
const [write] = allWrites;
if (
writes.size === 1 &&
t.isAssignmentExpression(write) &&
allWrites.length === 1 &&
write !== undefined &&
isInstalledBeforeEntryDispatch(
write,
bootstrapBody,
@@ -1,108 +0,0 @@
import { expect, it } from "vitest";
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
const returned = (source: string) => {
const ir = analyzeJavaScriptSemantics(source);
const callable = ir.callables.find(({ name }) => name === "observe");
return callable?.returnSites[0]?.value;
};
it("keeps members unknown after a write through a local setter", () => {
// The setter runs on assignment and may replace any member of `this`.
expect(
returned(`
export function observe() {
const state = { value: 1, set update(next) { this.value = next; } };
state.update = 2;
return state;
}
`),
).toMatchObject({ status: "unknown" });
});
it("keeps sibling literals when the written key is an ordinary property", () => {
expect(
returned(`
export function observe() {
const state = { value: 1, update: 0 };
state.update = 2;
return state;
}
`),
).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "value",
value: { status: "literal", value: 1 },
}),
]),
});
});
it("keeps nested members unknown after a write through a nested setter", () => {
const value = returned(`
export function observe() {
const state = {
label: "kept",
nested: { value: 1, set update(next) { this.value = next; } },
};
state.nested.update = 2;
return state;
}
`);
expect(value).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "label",
value: { status: "literal", value: "kept" },
}),
expect.objectContaining({
name: "nested",
value: expect.objectContaining({ status: "unknown" }),
}),
]),
});
});
it("keeps sibling literals when an accessor is deleted rather than assigned", () => {
expect(
returned(`
export function observe() {
const state = { value: 1, set update(next) {} };
delete state.update;
return state;
}
`),
).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "value",
value: { status: "literal", value: 1 },
}),
]),
});
});
it("keeps sibling literals when a later data property replaces the setter", () => {
expect(
returned(`
export function observe() {
const state = { value: 1, set update(next) {}, update: 0 };
state.update = 2;
return state;
}
`),
).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "value",
value: { status: "literal", value: 1 },
}),
]),
});
});
@@ -134,19 +134,11 @@ it("keeps a shadowed require local through a deep member initializer", () => {
it.each([
["root.first.last()", "root.first.last"],
["root[`first`].last()", "root.first.last"],
['root["first" as const].last()', "root.first.last"],
['root[""].last()', "root..last"],
["root[``].last()", "root..last"],
['root["" as const].last()', "root..last"],
['("")[""].last()', "last"],
['("")[""]()', ""],
["root[0].last()", "root.0.last"],
["root[0x1n].last()", "root.1.last"],
["root[-1n].last()", "root.-1.last"],
["root[key].last()", "root.[computed@5].last"],
["root[`first${key}`].last()", "root.[computed@5].last"],
["root[`\\unicode`].last()", "root.[computed@5].last"],
["root?.first.last()", "root.first.last"],
[`root${ordinaryMemberChain}.last()`, `root${ordinaryMemberChain}.last`],
])("preserves exact callee syntax (case %#)", (source, expected) => {
@@ -1,59 +0,0 @@
import { expect, it } from "vitest";
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
const releases = (source: string) =>
analyzeJavaScriptSemantics(source)
.resourceOperations.filter(({ kind }) => kind === "release")
.map(({ method, linkedResourceIds, resolution }) => ({
method,
linked: linkedResourceIds.length,
resolution,
}));
it.each([
[
"a declaration",
`import { createReadStream } from "node:fs";
const stream = createReadStream("input.txt");`,
],
[
"a later assignment",
`import { createReadStream } from "node:fs";
let stream;
stream = createReadStream("input.txt");`,
],
[
"an awaited handle",
`import { open } from "node:fs/promises";
const stream = await open("input.txt");`,
],
[
"a nullish placeholder before the acquisition",
`import { open } from "node:fs/promises";
let stream = null;
try { stream = await open("input.txt"); } finally {}`,
],
])(
"keeps a release complete when the handle holds only %s",
(_name, acquire) => {
expect(releases(`${acquire}\nstream.close();`)).toEqual([
{ method: "close", linked: 1, resolution: "complete" },
]);
},
);
it.each([
["an unconditional replacement", "stream = { destroy() {} };"],
["a conditional replacement", "if (flag) stream = { destroy() {} };"],
["a destructuring replacement", "({ stream } = other);"],
])("keeps a release partial after %s", (_name, replacement) => {
expect(
releases(`
import { createReadStream } from "node:fs";
let stream = createReadStream("input.txt");
${replacement}
stream.destroy();
`),
).toEqual([{ method: "destroy", linked: 1, resolution: "partial" }]);
});
@@ -1,59 +0,0 @@
import { expect, it } from "vitest";
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
const responses = (source: string) =>
analyzeJavaScriptSemantics(source)
.requestOperations.filter(({ kind }) => kind === "response-consumer")
.map(({ method, endpoint, resolution }) => ({
method,
endpoint,
resolution,
}));
it("keeps the endpoint of a uniquely consumed request response", () => {
expect(
responses(`
async function load() {
const a = await fetch("https://example.test/a");
const b = await fetch("https://example.test/b");
return [await a.json(), await b.text()];
}
`),
).toEqual([
{
method: "json",
endpoint: "https://example.test/a",
resolution: "complete",
},
{
method: "text",
endpoint: "https://example.test/b",
resolution: "complete",
},
]);
});
it("leaves an ambiguous response consumer without an endpoint", () => {
expect(
responses(`
async function load(flag) {
let response = await fetch("https://example.test/a");
if (flag) response = await fetch("https://example.test/b");
return response.json();
}
`),
).toEqual([{ method: "json", endpoint: null, resolution: "partial" }]);
});
it("leaves a response replaced by a non-request value without an endpoint", () => {
expect(
responses(`
async function load(cachedResponse) {
let response = await fetch("https://example.test/a");
response = cachedResponse;
return response.json();
}
`),
).toEqual([expect.objectContaining({ method: "json", endpoint: null })]);
});
@@ -270,18 +270,6 @@ const collectResponseConsumers = (
? []
: (requestsByBinding.get(binding.bindingId) ?? []);
if (linked.length === 0) return;
// A consumer of exactly one request through its own result binding
// reads that request's response, so it keeps the request's endpoint.
const uniqueRequest =
linked.length === 1 &&
!linked[0]?.operation.projectedResultBindings.some(
({ bindingId }) => bindingId === binding?.bindingId,
)
? linked[0]
: undefined;
// Any other write (assignment, update, loop or destructuring target) can
// replace the response before this read, so its endpoint stays unknown.
const soleWrite = binding?.initializers.length === 1;
output.push({
requestId: `response:${String(node.start ?? -1)}:${String(node.end ?? -1)}`,
kind: "response-consumer",
@@ -299,9 +287,15 @@ const collectResponseConsumers = (
linkedRequestIds: linked
.map(({ operation }) => operation.requestId)
.sort(compareUnicodeCodePoints),
endpoint: soleWrite ? (uniqueRequest?.operation.endpoint ?? null) : null,
endpoint: null,
fields: [],
resolution: uniqueRequest === undefined ? "partial" : "complete",
resolution:
linked.length === 1 &&
!linked[0]?.operation.projectedResultBindings.some(
({ bindingId }) => bindingId === binding?.bindingId,
)
? "complete"
: "partial",
});
});
return output;
@@ -53,82 +53,6 @@ type PropertyPath = JavaScriptSemanticPropertyPath;
type ValueEffect = "write" | "escape";
/** Preserve uncertainty from property writes and references exposed to calls, yielding during traversal and fixpoint iteration. */
/** Setter keys of one object literal; a computed setter key may match any key. */
interface SetterKeys {
readonly names: ReadonlySet<string>;
readonly computed: boolean;
}
// Writes to one literal repeat; scan its properties once, including the
// common result that it defines no setter.
const setterKeysByObject = new WeakMap<t.ObjectExpression, SetterKeys | null>();
const setterKeys = (object: t.ObjectExpression): SetterKeys | null => {
const cached = setterKeysByObject.get(object);
if (cached !== undefined) return cached;
const names = new Set<string>();
let computed = false;
// Follow definition order: a later data property or method with the same
// static name replaces the accessor, so its setter no longer runs. Spreads
// and unknown computed keys may or may not replace it and keep it.
for (const property of object.properties) {
if (t.isSpreadElement(property)) continue;
const name = semanticStaticPropertyKey(property.key, property.computed);
if (t.isObjectMethod(property) && property.kind === "set") {
if (name === null) computed = true;
else names.add(name);
} else if (
name !== null &&
!(t.isObjectMethod(property) && property.kind === "get")
)
names.delete(name);
}
const keys = names.size === 0 && !computed ? null : { names, computed };
setterKeysByObject.set(object, keys);
return keys;
};
/**
* The receiver path whose own object-literal setter an assignment to `path`
* runs, following nested literal members from the binding's initializer.
*/
const localSetterReceiver = (
initializers: readonly {
readonly node: t.Node;
readonly projection: readonly unknown[];
}[],
path: readonly unknown[],
): readonly string[] | undefined => {
const key = path.at(-1);
if (typeof key !== "string") return undefined;
const receiver = path.slice(0, -1);
if (!receiver.every((segment) => typeof segment === "string"))
return undefined;
for (const { node, projection } of initializers) {
if (projection.length !== 0) continue;
let object: t.Node | undefined = unwrapJavaScriptExpression(node).node;
for (const segment of receiver) {
if (!t.isObjectExpression(object)) break;
const member: t.ObjectExpression["properties"][number] | undefined =
object.properties.findLast(
(property) =>
t.isObjectProperty(property) &&
semanticStaticPropertyKey(property.key, property.computed) ===
segment,
);
object = t.isObjectProperty(member)
? unwrapJavaScriptExpression(member.value).node
: undefined;
}
if (!t.isObjectExpression(object)) continue;
const keys = setterKeys(object);
if (keys !== null && (keys.computed || keys.names.has(key)))
return receiver as readonly string[];
}
return undefined;
};
export function* collectSemanticMemberMutationsSteps(
program: t.Program,
state: JavaScriptSemanticAnalysisState,
@@ -453,16 +377,6 @@ export function* collectSemanticMemberMutationsSteps(
current.originAt,
parents,
);
// A setter on the binding's own literal runs on assignment and may
// replace any member of its receiver, so the written key is unknown.
// Deleting the accessor never calls it.
if (
effect === "write" &&
!t.isUnaryExpression(mutation, { operator: "delete" })
) {
const receiver = localSetterReceiver(origins.initializers, path);
if (receiver !== undefined) path = [...receiver, null];
}
// Keep an antichain of exact paths. Widening distinct member keys would
// also invalidate unrelated siblings that never reach this effect.
// This map is local to the file scan. Escaping the binding id keeps a
@@ -1,73 +0,0 @@
import * as t from "@babel/types";
import { expect, it } from "vitest";
import { indexJavaScriptBindingsSteps } from "./javascriptSemanticAnalysis.js";
import { semanticObjectPropertyReferences } from "./javascriptSemanticObjectReferences.js";
import { completeSemanticSteps } from "./javascriptSemanticTraversal.js";
import { parseJavaScriptSource } from "./javascriptSourceParser.js";
const objectAndState = (source: string) => {
const parsed = parseJavaScriptSource(source, "object.js");
if (parsed === null) throw new Error("Expected a parsed fixture");
const declaration = parsed.program.body[0];
if (!t.isVariableDeclaration(declaration))
throw new Error("Expected a declaration");
const init = declaration.declarations[0]?.init;
if (!t.isObjectExpression(init)) throw new Error("Expected an object");
return {
object: init,
source,
state: completeSemanticSteps(indexJavaScriptBindingsSteps(parsed.program)),
};
};
const spelled = (
fixture: ReturnType<typeof objectAndState>,
key: string | number | null,
) =>
semanticObjectPropertyReferences(fixture.object, [key], fixture.state).map(
({ node, path }) => [
fixture.source.slice(node.start ?? 0, node.end ?? 0),
JSON.stringify(path),
],
);
it("resolves exact keys with source-order overrides, spreads and accessors", () => {
const fixture = objectAndState(
`const o = { a: 1, ...base, b: 2, a: 3, [dynamic]: 4, __proto__: proto, ...{ b: 5 }, get c() { return 6; }, c: 7 };`,
);
// Expectations match the full source-order scan these exact keys replace.
expect(spelled(fixture, "a")).toEqual([
["{ b: 5 }", '["a"]'],
["4", "[]"],
["3", "[]"],
]);
expect(spelled(fixture, "b")).toEqual([
["{ b: 5 }", '["b"]'],
["4", "[]"],
]);
expect(spelled(fixture, "c")).toEqual([
["7", "[]"],
["4", "[]"],
]);
expect(spelled(fixture, "missing")).toEqual([
["{ b: 5 }", '["missing"]'],
["proto", '["missing"]'],
["4", "[]"],
["base", '["missing"]'],
]);
// A wildcard key still scans every property.
expect(spelled(fixture, null)).toEqual([
["7", "[]"],
["{ b: 5 }", '[{"excludedKeys":["c"]}]'],
["proto", "[null]"],
["4", "[]"],
["3", "[]"],
["2", "[]"],
["base", '[{"excludedKeys":["a","b","c"]}]'],
]);
// Numeric keys use the same exact spelling as string keys.
expect(
spelled(objectAndState(`const o = { 1: "one", 2: "two" };`), 1),
).toEqual([['"one"', "[]"]]);
});
@@ -739,75 +739,6 @@ const getterResultReferences = (
return references;
};
type ObjectLiteralMember = t.ObjectExpression["properties"][number];
/** Property positions grouped by whether an exact key can reach them. */
interface ObjectPropertyIndex {
/** Spreads, computed keys and `__proto__` data entries reach every key. */
readonly keyIndependent: readonly number[];
readonly byName: ReadonlyMap<string, readonly number[]>;
}
// Mutation tracking resolves one key of the same literal per write, so the
// index removes the per-write scan of every property.
const propertyIndexes = new WeakMap<t.ObjectExpression, ObjectPropertyIndex>();
const objectPropertyIndex = (node: t.ObjectExpression): ObjectPropertyIndex => {
const cached = propertyIndexes.get(node);
if (cached !== undefined) return cached;
const keyIndependent: number[] = [];
const byName = new Map<string, number[]>();
node.properties.forEach((property, index) => {
const name = t.isSpreadElement(property)
? null
: semanticStaticPropertyKey(property.key, property.computed);
if (
name === null ||
(t.isObjectProperty(property) &&
!property.computed &&
!property.shorthand &&
name === "__proto__")
) {
keyIndependent.push(index);
return;
}
const positions = byName.get(name);
if (positions === undefined) byName.set(name, [index]);
else positions.push(index);
});
const index = { keyIndependent, byName };
propertyIndexes.set(node, index);
return index;
};
/**
* Properties in reverse source order that can affect `key`. For an exact key,
* another static name only updates bookkeeping for that other name, which
* this key never reads, so only same-name and key-independent entries remain.
*/
const relevantPropertiesInReverse = (
node: t.ObjectExpression,
key: PropertyPath[number] | undefined,
): readonly ObjectLiteralMember[] => {
if (typeof key !== "string" && typeof key !== "number")
return [...node.properties].reverse();
const { keyIndependent, byName } = objectPropertyIndex(node);
const named = byName.get(String(key)) ?? [];
const relevant: ObjectLiteralMember[] = [];
let left = keyIndependent.length - 1;
let right = named.length - 1;
while (left >= 0 || right >= 0) {
const fromLeft =
right < 0 ||
(left >= 0 && (keyIndependent[left] ?? -1) > (named[right] ?? -1));
const position = fromLeft ? keyIndependent[left--] : named[right--];
const property =
position === undefined ? undefined : node.properties[position];
if (property !== undefined) relevant.push(property);
}
return relevant;
};
export const semanticObjectPropertyReferences = (
node: t.ObjectExpression,
path: PropertyPath,
@@ -820,7 +751,7 @@ export const semanticObjectPropertyReferences = (
const accessorNames = new Set<string>();
const getterNames = new Set<string>();
const setterNames = new Set<string>();
for (const property of relevantPropertiesInReverse(node, key)) {
for (const property of [...node.properties].reverse()) {
if (t.isSpreadElement(property)) {
const selected = key ?? null;
if (typeof selected !== "object" && overwritten.has(String(selected)))
@@ -8,11 +8,7 @@ import type {
JavaScriptSemanticCallable,
JavaScriptSemanticResourceOperation,
} from "./javascriptSemanticIr.js";
import {
semanticStaticPropertyName,
unwrapJavaScriptExpression,
} from "./javascriptAstValues.js";
import { isUnmodifiedGlobalReference } from "./javascriptSemanticAmbientWrites.js";
import { semanticStaticPropertyName } from "./javascriptAstValues.js";
import {
resolveSemanticBindingState,
type JavaScriptSemanticAnalysisState,
@@ -104,58 +100,21 @@ const collectReleases = (
);
const linked =
binding === undefined ? [] : (byBinding.get(binding.bindingId) ?? []);
if (binding === undefined || linked.length === 0) return;
if (linked.length === 0) return;
output.push({
resourceId: `resource:release:${String(node.start ?? -1)}:${String(node.end ?? -1)}`,
kind: "release",
method,
location: range(node),
ownerCallableId: context.callableStack.at(-1) ?? null,
resultBindingId: binding.bindingId,
resultBindingId: binding?.bindingId ?? null,
linkedResourceIds: linked.map(resourceCandidateId).sort(),
resolution:
linked.length === 1 &&
holdsOnlyAcquisitions(binding, linked, context.state)
? "complete"
: "partial",
resolution: linked.length === 1 ? "complete" : "partial",
});
});
return output;
};
/**
* A lexical handle identifies its acquisition only while every value written
* to the binding is that acquisition itself. Any replacement, even a
* conditional or later one, leaves the released resource a candidate. A
* nullish placeholder, as in `let h = null; try { h = open() } finally {
* h?.close() }`, releases nothing else: the call is skipped or throws.
*/
const holdsOnlyAcquisitions = (
binding: JavaScriptSemanticBindingState,
linked: readonly ResourceCandidate[],
state: JavaScriptSemanticAnalysisState,
): boolean =>
[...binding.initializers, ...binding.referenceInitializers].every(
({ node, projection }) => {
if (projection.length > 0) return false;
let value = unwrapJavaScriptExpression(node).node;
if (t.isAwaitExpression(value))
value = unwrapJavaScriptExpression(value.argument).node;
return (
isNullishPlaceholder(value, state) ||
linked.some((acquisition) => acquisition.node === value)
);
},
);
const isNullishPlaceholder = (
value: t.Node,
state: JavaScriptSemanticAnalysisState,
): boolean =>
t.isNullLiteral(value) ||
t.isUnaryExpression(value, { operator: "void" }) ||
isUnmodifiedGlobalReference(value, state, "undefined");
const immutableAcquisition = (
candidate: ResourceCandidate,
): JavaScriptSemanticResourceOperation => ({
@@ -2,173 +2,6 @@ import { describe, expect, it } from "vitest";
import { analyzeJavaScriptStaticSource } from "./javascriptStaticAnalysis.js";
it("retains endpoint observations for exact template and TypeScript member keys", () => {
for (const [get, beacon] of [
['"get"', '"sendBeacon"'],
["`get`", "`sendBeacon`"],
['"get" as const', '"sendBeacon" as const'],
]) {
const analysis = analyzeJavaScriptStaticSource(`
import * as vscode from "vscode";
client[${get}]("/api/items");
navigator[${beacon}]("/api/telemetry", payload);
client[${get}]("setting");
new Headers()[${get}]("content-type");
new URLSearchParams()[${get}]("q");
vscode.workspace.getConfiguration("editor")[${get}]("tabSize");
`);
expect.soft(analysis.parse_error_count).toBe(0);
expect.soft(analysis.parse_status).toBe("complete");
expect
.soft(
analysis.endpoints.map(({ kind, value, mechanism, interpretation }) => [
kind,
value,
mechanism,
interpretation ?? null,
]),
)
.toEqual([
["network", "/api/items", "call:client.get", null],
["network", "/api/telemetry", "call:navigator.sendBeacon", null],
["network", "setting", "call:client.get", "candidate"],
]);
}
});
it("preserves preload and utility paths through exact computed join members", () => {
for (const key of ['"join"', "`join`", '"join" as const']) {
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow, utilityProcess } = require("electron");
const path = require("node:path");
new BrowserWindow({ webPreferences: {
preload: path[${key}](__dirname, "preload.js")
} });
utilityProcess.fork(path[${key}](__dirname, "worker.js"), [], {
serviceName: "kept"
});
`);
expect.soft(analysis.parse_error_count).toBe(0);
expect.soft(analysis.parse_status).toBe("complete");
expect
.soft(
analysis.role_paths.map(
({ role, path, resolution_context, mechanism }) => [
role,
path,
resolution_context,
mechanism,
],
),
)
.toEqual([
["preload", "preload.js", "filesystem-expression", "property:preload"],
]);
expect
.soft(
analysis.electron.browser_windows.map(
({ location: _location, ...finding }) => finding,
),
)
.toEqual([
{
options_status: "object-literal",
web_preferences_status: "object-literal",
web_preferences: [
{
name: "preload",
value: {
status: "literal",
value: "preload.js",
expression: null,
},
},
],
preload_path: "preload.js",
preload_resolution_context: "filesystem-expression",
module_key: null,
},
]);
expect
.soft(
analysis.electron.utility_processes.map(
({ location: _location, ...finding }) => finding,
),
)
.toEqual([
{
module_path: "worker.js",
module_resolution_context: "filesystem-expression",
module_expression: null,
service_name: "kept",
module_key: null,
},
]);
}
});
it("keeps dynamic, interpolated, and invalid template keys unresolved", () => {
for (const { key, errors } of [
{ key: "method", errors: 0 },
{ key: "`join${suffix}`", errors: 0 },
{ key: ["`", String.raw`\unicode`, "`"].join(""), errors: 4 },
]) {
const preload = `path[${key}](__dirname, "preload.js")`;
const worker = `path[${key}](__dirname, "worker.js")`;
const analysis = analyzeJavaScriptStaticSource(`
const { BrowserWindow, utilityProcess } = require("electron");
const path = require("node:path");
client[${key}]("/api/items");
navigator[${key}]("/api/telemetry", payload);
new BrowserWindow({ webPreferences: { preload: ${preload} } });
utilityProcess.fork(${worker}, [], { serviceName: "kept" });
`);
expect.soft(analysis.parse_error_count).toBe(errors);
expect.soft(analysis.parse_status).toBe("partial");
expect.soft(analysis.endpoints).toEqual([]);
expect.soft(analysis.role_paths).toEqual([]);
expect
.soft(
analysis.electron.browser_windows.map(
({ location: _location, ...finding }) => finding,
),
)
.toEqual([
{
options_status: "object-literal",
web_preferences_status: "object-literal",
web_preferences: [
{
name: "preload",
value: { status: "dynamic", value: null, expression: preload },
},
],
preload_path: null,
preload_resolution_context: null,
module_key: null,
},
]);
expect
.soft(
analysis.electron.utility_processes.map(
({ location: _location, ...finding }) => finding,
),
)
.toEqual([
{
module_path: null,
module_resolution_context: null,
module_expression: worker,
service_name: "kept",
module_key: null,
},
]);
}
});
describe("static callee boundary matching", () => {
it("matches complete callee members without suffix collisions", () => {
const analysis = analyzeJavaScriptStaticSource(`
@@ -693,8 +693,11 @@ export const calleeName = (node: t.Node): string => {
const memberPropertyName = (
node: t.MemberExpression | t.OptionalMemberExpression,
): string =>
semanticStaticPropertyKey(node.property, node.computed) ??
(node.computed ? `[computed@${String(node.property.start ?? -1)}]` : "");
node.computed &&
!t.isStringLiteral(node.property) &&
!t.isNumericLiteral(node.property)
? `[computed@${String(node.property.start ?? -1)}]`
: semanticStaticPropertyName(node.property, node.computed);
/**
* Return a string literal value without evaluating an expression. A template
@@ -23,7 +23,6 @@ const absoluteParts = [
const anchoredExpressions = [
['path.resolve(__dirname, "preload.cjs")', "preload.cjs"],
['path[`resolve`](__dirname, "preload.js")', "preload.js"],
['path.posix.resolve("ignored", __dirname, "preload.js")', "preload.js"],
['path.resolve("/ignored", __dirname, "preload.js")', "preload.js"],
[
@@ -46,8 +45,6 @@ const anchoredExpressions = [
] as const;
const unknownExpressions = [
'path[method](__dirname, "preload.js")',
'path[`join${suffix}`](__dirname, "preload.js")',
"path.resolve()",
"path.posix.resolve()",
'path.resolve("sub", "../preload.js")',
@@ -109,11 +106,6 @@ describe("inert JavaScript path resolution", () => {
"preload.js",
"filesystem-expression",
],
[
'path[`join`](__dirname, "preload.js")',
"preload.js",
"filesystem-expression",
],
[
'path.join(__dirname, "/preload.js")',
"preload.js",
@@ -24,13 +24,6 @@ type PinvokeImport = ManagedNativeBoundaryInspection["pinvoke_imports"][number];
type NativeImplementation =
ManagedNativeBoundaryInspection["native_implementations"][number];
/**
* Display label for a metadata name. Metadata may hold an empty name; the
* graph then has no label, while the node's properties keep the source value.
*/
const metadataLabel = (name: string): string | null =>
name === "" ? null : name;
/** Options for a managed contains relationship edge. */
export interface ContainsEdgeOptions {
readonly kind: string;
@@ -84,7 +77,7 @@ export const addArtifactIdentityNodes = (
),
observations: [
{
label: metadataLabel(assemblyFacts.name),
label: assemblyFacts.name,
properties: {
name: assemblyFacts.name,
version: assemblyFacts.version,
@@ -118,7 +111,7 @@ export const addArtifactIdentityNodes = (
),
observations: [
{
label: metadataLabel(module.name),
label: module.name,
properties: {
name: module.name,
mvid: module.mvid,
@@ -244,7 +237,7 @@ const typeNode = (
),
observations: [
{
label: metadataLabel(type.full_name),
label: type.full_name,
properties: {
token: type.token,
namespace: type.namespace,
@@ -273,11 +266,10 @@ const methodNode = (
),
observations: [
{
label: metadataLabel(
label:
method.declaring_type === null
? method.name
: `${method.declaring_type}.${method.name}`,
),
properties: {
token: method.token,
declaring_type_token: method.declaring_type_token,
@@ -310,11 +302,10 @@ const fieldNode = (
),
observations: [
{
label: metadataLabel(
label:
field.declaring_type === null
? field.name
: `${field.declaring_type}.${field.name}`,
),
properties: {
token: field.token,
declaring_type_token: field.declaring_type_token,
@@ -419,7 +410,7 @@ const pinvokeNode = (
),
observations: [
{
label: metadataLabel(pinvoke.import_name),
label: pinvoke.import_name,
properties: {
token: pinvoke.token,
member_token: pinvoke.member_token,
@@ -450,7 +441,7 @@ const nativeImplementationNode = (
),
observations: [
{
label: metadataLabel(implementation.name),
label: implementation.name,
properties: {
token: implementation.token,
name: implementation.name,
-101
View File
@@ -3,7 +3,6 @@ import { describe, expect, it } from "vitest";
import { readElfSymbols, readSnapshotHeader } from "./DartElf.js";
import {
REFERENCE_HASH,
buildElf32Image,
buildElf64Image,
buildStrippedElf64Image,
snapshotSection,
@@ -300,106 +299,6 @@ describe("ELF symbol-table ownership", () => {
});
});
/** Section-header field positions for each ELF class. */
const ELF_CLASSES = {
elf64: {
build: buildElf64Image,
word: {
read: (image: Buffer, at: number) => Number(image.readBigUInt64LE(at)),
write: (image: Buffer, value: number, at: number) =>
image.writeBigUInt64LE(BigInt(value), at),
},
shoff: 0x28,
shent: 64,
flags: 8,
address: 0x10,
offset: 0x18,
},
elf32: {
build: buildElf32Image,
word: {
read: (image: Buffer, at: number) => image.readUInt32LE(at),
write: (image: Buffer, value: number, at: number) =>
image.writeUInt32LE(value, at),
},
shoff: 0x20,
shent: 40,
flags: 8,
address: 0x0c,
offset: 0x10,
},
} as const;
describe.each(Object.entries(ELF_CLASSES))(
"allocated %s symbol tables",
(_, layout) => {
const { word } = layout;
/** Snapshot sections, with dynamic tables allocated at their offsets. */
const allocatedTables = (): {
image: Buffer;
headers: number[];
tables: number[];
} => {
const elf = layout.build([
{ name: "_kDartVmSnapshotData", value: 0x400, size: 52 },
]);
const section = snapshotSection(GALLERY_HASH);
const image = Buffer.concat([
elf,
Buffer.alloc(0x400 - elf.length),
section,
]);
const shoff = word.read(image, layout.shoff);
const headers = [shoff + layout.shent, shoff + 2 * layout.shent];
const tables = headers.map((header) => {
const offset = word.read(image, header + layout.offset);
word.write(image, 2, header + layout.flags);
word.write(image, offset, header + layout.address);
return offset;
});
return { image, headers, tables };
};
it("accepts tables whose mapping matches their sections", async () => {
const { image } = allocatedTables();
expect(await readElfSymbols(image, ["_kDartVmSnapshotData"])).toEqual({
symbols: [{ name: "_kDartVmSnapshotData", offset: 0x400, size: 52 }],
failures: [],
});
});
it.each([
[0, "Dynamic symbol table has conflicting PT_LOAD mappings"],
[1, "Dynamic symbol string table has conflicting PT_LOAD mappings"],
])(
"refuses table %i under a conflicting PT_LOAD interpretation",
async (table, reason) => {
const { image, tables } = allocatedTables();
const conflict = appendElfRanges(image, "program", [
{ type: 1, address: tables[table]!, offset: 0x300, size: 4 },
]);
expect(
await readElfSymbols(conflict, ["_kDartVmSnapshotData"]),
).toEqual({ symbols: [], failures: [{ reason }] });
},
);
it("refuses a table whose section offset disagrees with its mapping", async () => {
const { image, headers } = allocatedTables();
word.write(image, 0x300, headers[0]! + layout.address);
expect(await readElfSymbols(image, ["_kDartVmSnapshotData"])).toEqual({
symbols: [],
failures: [
{
reason:
"Dynamic symbol table section offset disagrees with its PT_LOAD mapping",
},
],
});
});
},
);
describe("readSnapshotHeader", () => {
it("reads a synthesized data header", async () => {
const header = readSnapshotHeader(snapshotSection(GALLERY_HASH), 0);
-33
View File
@@ -59,7 +59,6 @@ export interface ElfParseFailure {
}
const SHT_DYNSYM = 11;
const SHF_ALLOC = 2;
const PT_LOAD = 1;
const PT_DYNAMIC = 2;
/** Position-independent executable images carry PT_DYNAMIC. */
@@ -226,28 +225,6 @@ export const readElfSymbols = async (
stringOffset: number;
stringSize: number;
}[] = [];
/**
* The loader reads an allocated section through PT_LOAD, so its section
* offset must be the single file interpretation of its address range.
*/
const sectionMappingFailure = (
header: number,
offset: number,
size: number,
): string | undefined => {
const flags = is64 ? reader.u64(header + 8) : reader.u32(header + 8);
const address = is64
? reader.u64(header + 0x10)
: reader.u32(header + 0x0c);
if (flags === null || address === null)
return "has an unrepresentable section header";
if ((flags & SHF_ALLOC) === 0) return undefined;
const mapped = toOffset(address, size);
if (!mapped.ok) return mapped.error.reason;
return mapped.value === offset
? undefined
: "section offset disagrees with its PT_LOAD mapping";
};
if (!dynamicFallback) {
for (let index = 0; index < e_shnum; index += 1) {
if (index % 4096 === 0) {
@@ -292,16 +269,6 @@ export const readElfSymbols = async (
return failure(
"Dynamic symbol strings are out of bounds or not a string table",
);
const symbolMapping = sectionMappingFailure(header, offset, size);
if (symbolMapping !== undefined)
return failure(`Dynamic symbol table ${symbolMapping}`);
const stringMapping = sectionMappingFailure(
stringHeader,
stringOffset,
stringSize,
);
if (stringMapping !== undefined)
return failure(`Dynamic symbol string table ${stringMapping}`);
tables.push({
offset,
count: size / entrySize,
-57
View File
@@ -115,63 +115,6 @@ export const buildElf64Image = (
return image;
};
/** The ELF32 (ARM) counterpart of {@link buildElf64Image}, with the same layout roles. */
export const buildElf32Image = (
symbols: ReadonlyArray<{ name: string; value: number; size: number }>,
): Buffer => {
const ELF_HEADER = 52;
const PHENT = 32;
const SHENT = 40;
const SYMENT = 16;
const phoff = ELF_HEADER;
const shoff = phoff + PHENT;
const dynsymOff = shoff + 3 * SHENT + 16;
const dynstrOff = dynsymOff + symbols.length * SYMENT;
const image = Buffer.alloc(dynstrOff + 256);
image.writeUInt32LE(0x464c457f, 0);
image[4] = 1; // ELFCLASS32
image[5] = 1; // little-endian
image[6] = 1;
image.writeUInt16LE(3, 0x10);
image.writeUInt16LE(0x0028, 0x12); // e_machine ARM
image.writeUInt32LE(1, 0x14);
image.writeUInt32LE(phoff, 0x1c);
image.writeUInt32LE(shoff, 0x20);
image.writeUInt16LE(ELF_HEADER, 0x28);
image.writeUInt16LE(PHENT, 0x2a);
image.writeUInt16LE(1, 0x2c);
image.writeUInt16LE(SHENT, 0x2e);
image.writeUInt16LE(3, 0x30); // null, dynsym, dynstr
// One identity PT_LOAD covering the materialized payload.
image.writeUInt32LE(1, phoff);
image.writeUInt32LE(
Math.max(image.length, ...symbols.map(({ value, size }) => value + size)),
phoff + 0x10,
);
const dyn = shoff + SHENT;
image.writeUInt32LE(11, dyn + 4); // SHT_DYNSYM
image.writeUInt32LE(dynsymOff, dyn + 0x10);
image.writeUInt32LE(symbols.length * SYMENT, dyn + 0x14);
image.writeUInt32LE(2, dyn + 0x18); // sh_link -> strtab
image.writeUInt32LE(SYMENT, dyn + 0x24);
const str = dyn + SHENT;
image.writeUInt32LE(3, str + 4); // SHT_STRTAB
image.writeUInt32LE(dynstrOff, str + 0x10);
image.writeUInt32LE(256, str + 0x14);
let strCursor = dynstrOff + 1;
symbols.forEach((symbol, index) => {
image.write(symbol.name, strCursor, "latin1");
const base = dynsymOff + index * SYMENT;
image.writeUInt32LE(strCursor - dynstrOff, base);
image.writeUInt32LE(symbol.value, base + 4);
image.writeUInt32LE(symbol.size, base + 8);
image[base + 0xc] = 0x10; // GLOBAL
image.writeUInt16LE(1, base + 0xe); // Defined export.
strCursor += symbol.name.length + 1;
});
return image;
};
/**
* Build a section-stripped ELF64 image (e_shnum 0, ET_DYN) whose exported
* symbols are reachable only through the PT_DYNAMIC tag-value pairs, the
+5 -16
View File
@@ -69,20 +69,6 @@ const hasLegacyMipsHeaderShape = (target: BinaryTarget): boolean => {
};
/** Return a refusal for GNU ABI attributes this existing MIPS lane cannot model. */
/**
* Every MIPS lane, including PSP, needs every ABI declaration source examined:
* headers outside the captured file may hold declarations that would refuse it.
*/
export const ghidraMipsIncompleteDeclarationsReason = (
metadata: MipsElfMetadata,
): string | null =>
(metadata.limitations?.length ?? 0) > 0
? [
"MIPS ABI declaration inspection is incomplete; unsupported attributes or conflicting declaration sources may remain unknown.",
...(metadata.limitations ?? []),
].join(" ")
: null;
export const ghidraMipsGnuAttributesUnsupportedReason = (
metadata: MipsElfMetadata,
): string | null => {
@@ -178,8 +164,11 @@ export const ghidraMipsUnsupportedReason = (
if (load.symbolRelativeCount !== 0)
return "Symbol-bearing R_MIPS_REL32 is unsupported: Ghidra normalizes MIPS16 function addresses and can lose the source function-pointer mode tag. Symbol-index-zero REL32 is verified separately.";
}
const incomplete = ghidraMipsIncompleteDeclarationsReason(metadata);
if (incomplete !== null) return incomplete;
if ((metadata.limitations?.length ?? 0) > 0)
return [
"MIPS ABI declaration inspection is incomplete; unsupported attributes or conflicting declaration sources may remain unknown.",
...(metadata.limitations ?? []),
].join(" ");
const abi = metadata.abiFlags;
if (shared && abi === null && (metadata.limitations?.length ?? 0) === 0)
return null; // Inspected absence leaves FP semantics unknown, never inferred.
-3
View File
@@ -1,7 +1,6 @@
import type { BinaryTarget } from "../domain/binaryTargetTypes.js";
import {
ghidraMipsGnuAttributesUnsupportedReason,
ghidraMipsIncompleteDeclarationsReason,
ghidraMipsUnsupportedReason,
} from "./GhidraMipsProfile.js";
import { ghidraPowerPcUnsupportedReason } from "./GhidraPowerPcProfile.js";
@@ -34,8 +33,6 @@ export const ghidraProcessorUnsupportedReason = (
const gnuAttributesReason =
ghidraMipsGnuAttributesUnsupportedReason(metadata);
if (gnuAttributesReason !== null) return gnuAttributesReason;
const incomplete = ghidraMipsIncompleteDeclarationsReason(metadata);
if (incomplete !== null) return incomplete;
if (metadata.elfClass !== 32 || metadata.byteOrder !== "little")
return "The PSP Allegrex profile requires little-endian ELF32.";
if (metadata.type !== 2)
-144
View File
@@ -1,144 +0,0 @@
import { isAbsolute, join } from "node:path";
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
import { providerStartupTimeoutSchema } from "../config/providerStartupTimeout.js";
import { err, ok, type Result } from "../domain/result.js";
import {
KUNA_COMMAND_REMEDIATION,
KUNA_UNSUPPORTED_HOST_REMEDIATION,
} from "./KunaRelease.js";
/** Kuna caps its own decode lanes and worker pool at 32. */
const MAX_KUNA_JOBS = 32;
/**
* A 1.4 MB sqlite3 shell exported serially in 66-108 s on Apple M-series and
* Linux hosts; thirty minutes admits images an order of magnitude larger
* without an unbounded run. Larger images need a raised deadline or more jobs.
*/
const DEFAULT_PROJECT_TIMEOUT_MS = 30 * 60 * 1000;
/**
* A serial (`--jobs 1`) export is Kuna's reference answer: it decompiles
* callees before callers, which recovers prototypes that parallel workers
* cannot. Measured on coreutils sort, 40-60% of functions differ otherwise.
*/
const DEFAULT_JOBS = 1;
/** Measured sqlite3 exports occupy ~25 MiB; four GiB retains many projects. */
const DEFAULT_CACHE_MAX_BYTES = 4 * 1024 ** 3;
/** One project needs its .c, .h, .asm, inventory and manifest; refuse smaller caps. */
const MIN_CACHE_MAX_BYTES = 1024 ** 2;
/** Caller-selected Kuna installation and REA-owned project cache policy. */
export interface KunaConfiguration {
/** Caller-selected executable path, preserved exactly as configured. */
readonly command: string;
/** REA-owned cache root; Kuna's own spec cache lives beneath it too. */
readonly cacheRoot: string;
readonly cacheMaxBytes: number;
readonly projectTimeoutMs: number;
readonly jobs: number;
}
/** Why the configured environment cannot select a Kuna installation. */
export interface KunaConfigurationFailure {
readonly code: "not_configured" | "open_options_invalid" | "unsupported_host";
readonly reason: string;
readonly setting: string | null;
}
const failure = (
code: KunaConfigurationFailure["code"],
reason: string,
setting: string | null = null,
): Result<never, KunaConfigurationFailure> => err({ code, reason, setting });
const positiveInteger = (value: string): number | undefined =>
/^[1-9]\d*$/u.test(value) && Number.isSafeInteger(Number(value))
? Number(value)
: undefined;
/** Per-user cache directory that REA owns for Kuna on this host. */
const defaultCacheRoot = (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform,
): string => {
const home = homeDirectoryFromEnvironment(environment, platform);
if (platform === "darwin")
return join(home, "Library", "Caches", "rea", "kuna");
const xdg = environment.XDG_CACHE_HOME;
return join(
xdg !== undefined && isAbsolute(xdg) ? xdg : join(home, ".cache"),
"rea",
"kuna",
);
};
/** Parse the Kuna settings without touching the filesystem or the executable. */
export const readKunaConfiguration = (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform = process.platform,
): Result<KunaConfiguration, KunaConfigurationFailure> => {
const command = environment.REA_KUNA_COMMAND;
if (command === undefined || command.length === 0)
return failure("not_configured", KUNA_COMMAND_REMEDIATION);
if (!isAbsolute(command))
return failure(
"open_options_invalid",
`REA_KUNA_COMMAND must be an absolute path; received ${command}. ${KUNA_COMMAND_REMEDIATION}`,
"REA_KUNA_COMMAND",
);
if (platform !== "darwin" && platform !== "linux")
return failure("unsupported_host", KUNA_UNSUPPORTED_HOST_REMEDIATION);
const cacheRoot = environment.REA_KUNA_CACHE_DIR;
if (cacheRoot !== undefined && !isAbsolute(cacheRoot))
return failure(
"open_options_invalid",
`REA_KUNA_CACHE_DIR must be an absolute directory; received ${cacheRoot}.`,
"REA_KUNA_CACHE_DIR",
);
const rawMaxBytes = environment.REA_KUNA_CACHE_MAX_BYTES;
const cacheMaxBytes =
rawMaxBytes === undefined
? DEFAULT_CACHE_MAX_BYTES
: positiveInteger(rawMaxBytes);
if (cacheMaxBytes === undefined || cacheMaxBytes < MIN_CACHE_MAX_BYTES)
return failure(
"open_options_invalid",
`REA_KUNA_CACHE_MAX_BYTES must be a decimal byte count of at least ${String(MIN_CACHE_MAX_BYTES)}; received ${String(rawMaxBytes)}.`,
"REA_KUNA_CACHE_MAX_BYTES",
);
const rawTimeout = environment.REA_KUNA_PROJECT_TIMEOUT_MS;
const timeout =
rawTimeout === undefined
? undefined
: providerStartupTimeoutSchema.safeParse(rawTimeout);
if (timeout !== undefined && !timeout.success)
return failure(
"open_options_invalid",
`REA_KUNA_PROJECT_TIMEOUT_MS must be a decimal millisecond count from 1 to 2147483647; received ${String(rawTimeout)}.`,
"REA_KUNA_PROJECT_TIMEOUT_MS",
);
const rawJobs = environment.REA_KUNA_JOBS;
const jobs = rawJobs === undefined ? DEFAULT_JOBS : positiveInteger(rawJobs);
if (jobs === undefined || jobs > MAX_KUNA_JOBS)
return failure(
"open_options_invalid",
`REA_KUNA_JOBS must be an integer from 1 to ${String(MAX_KUNA_JOBS)}; received ${String(rawJobs)}.`,
"REA_KUNA_JOBS",
);
return ok({
command,
cacheRoot: cacheRoot ?? defaultCacheRoot(environment, platform),
cacheMaxBytes,
projectTimeoutMs: timeout?.data ?? DEFAULT_PROJECT_TIMEOUT_MS,
jobs,
});
};
-211
View File
@@ -1,211 +0,0 @@
import { constants } from "node:fs";
import { access, stat } from "node:fs/promises";
import { dirname } from "node:path";
import type {
DoctorProviderCheck,
DoctorProviderInspection,
} from "../application/Doctor.js";
import { readKunaConfiguration } from "./KunaConfiguration.js";
import {
inspectKunaInstallation,
type KunaReadinessFailure,
} from "./KunaInstallation.js";
import {
KUNA_COMMAND_REMEDIATION,
KUNA_PROVIDER_ID,
KUNA_UNSUPPORTED_HOST_REMEDIATION,
KUNA_VERIFIED_VERSION,
KUNA_VERSION_PROBE_REMEDIATION,
} from "./KunaRelease.js";
/** Settings a registration may carry; only those the caller set explicitly. */
const REGISTRATION_SETTINGS = [
"REA_KUNA_COMMAND",
"REA_KUNA_CACHE_DIR",
"REA_KUNA_CACHE_MAX_BYTES",
"REA_KUNA_PROJECT_TIMEOUT_MS",
"REA_KUNA_JOBS",
] as const;
/** Project Kuna readiness into the doctor contract without writing anything. */
export const inspectKunaProvider = async (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform = process.platform,
): Promise<DoctorProviderInspection> => {
const installation = await inspectKunaInstallation(environment, {
platform,
});
const configured =
environment.REA_KUNA_COMMAND !== undefined &&
environment.REA_KUNA_COMMAND.length > 0;
if (!installation.ok)
return {
id: KUNA_PROVIDER_ID,
configured,
available: false,
providerVersion: null,
registrationEnvironment: {},
checks: [installationFailure(installation.error, configured)],
};
const { value } = installation;
return {
id: KUNA_PROVIDER_ID,
configured,
available: true,
providerVersion: value.version,
registrationEnvironment: Object.fromEntries(
REGISTRATION_SETTINGS.flatMap((name) => {
const setting = environment[name];
return setting === undefined ? [] : [[name, setting]];
}),
),
checks: [
{
name: "installation",
ok: true,
code: null,
detail: `${value.executable} reports Kuna ${value.version} (sha256 ${value.sha256})${
value.verified
? "."
: `; REA verified ${KUNA_VERIFIED_VERSION}, so this release's project layout is unverified.`
}`,
remediation: null,
classification: "config_drift",
details: {
executable: value.executable,
version: value.version,
verified_version: KUNA_VERIFIED_VERSION,
sha256: value.sha256,
},
},
await cacheCheck(environment, platform),
],
};
};
const installationFailure = (
failure: KunaReadinessFailure,
configured: boolean,
): DoctorProviderCheck => ({
name: "installation",
ok: false,
code: failure.code,
detail: failure.reason,
remediation:
failure.code === "unsupported_host"
? KUNA_UNSUPPORTED_HOST_REMEDIATION
: failure.code === "version_unresolved"
? KUNA_VERSION_PROBE_REMEDIATION
: KUNA_COMMAND_REMEDIATION,
classification:
failure.code === "unsupported_host"
? "unsupported_host"
: configured
? "config_drift"
: "missing_analysis_engine",
details: failure.diagnostics,
});
/** The cache root may not exist yet; doctor reports it without creating it. */
const cacheCheck = async (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform,
): Promise<DoctorProviderCheck> => {
const configuration = readKunaConfiguration(environment, platform);
if (!configuration.ok)
return {
name: "cache",
ok: false,
code: configuration.error.code,
detail: configuration.error.reason,
remediation: KUNA_COMMAND_REMEDIATION,
classification: "config_drift",
};
const { cacheRoot, cacheMaxBytes } = configuration.value;
const details = { cache_root: cacheRoot, cache_max_bytes: cacheMaxBytes };
try {
if (!(await stat(cacheRoot)).isDirectory())
return {
name: "cache",
ok: false,
code: "open_options_invalid",
detail: `The Kuna cache root ${cacheRoot} exists but is not a directory.`,
remediation:
"Set REA_KUNA_CACHE_DIR to a directory REA may own, or remove the conflicting file.",
classification: "config_drift",
details,
};
await access(cacheRoot, constants.W_OK | constants.X_OK);
} catch (cause: unknown) {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT") {
const creatable = await nearestDirectoryWritable(cacheRoot);
return creatable.ok
? {
name: "cache",
ok: true,
code: null,
detail: `REA creates the Kuna project cache at ${cacheRoot} on first use.`,
remediation: null,
classification: "config_drift",
details,
}
: {
name: "cache",
ok: false,
code: "open_options_invalid",
detail: `REA cannot create the Kuna cache root ${cacheRoot}: ${creatable.reason}`,
remediation:
"Set REA_KUNA_CACHE_DIR to a directory the REA process user can create or write.",
classification: "config_drift",
details,
};
}
return {
name: "cache",
ok: false,
code: "open_options_invalid",
detail: `REA cannot write the Kuna cache root ${cacheRoot}: ${cause instanceof Error ? cause.message : String(cause)}`,
remediation:
"Set REA_KUNA_CACHE_DIR to a directory the REA process user can write.",
classification: "config_drift",
details,
};
}
return {
name: "cache",
ok: true,
code: null,
detail: `Kuna project exports are cached in ${cacheRoot} (at most ${String(cacheMaxBytes)} bytes).`,
remediation: null,
classification: "config_drift",
details,
};
};
/** Whether the nearest existing ancestor is a directory REA may create entries in. */
const nearestDirectoryWritable = async (
path: string,
): Promise<
{ readonly ok: true } | { readonly ok: false; readonly reason: string }
> => {
let candidate = dirname(path);
for (;;) {
try {
if (!(await stat(candidate)).isDirectory())
return { ok: false, reason: `${candidate} is not a directory.` };
await access(candidate, constants.W_OK | constants.X_OK);
return { ok: true };
} catch (cause: unknown) {
const missing =
cause instanceof Error && "code" in cause && cause.code === "ENOENT";
const parent = dirname(candidate);
if (!missing || parent === candidate)
return {
ok: false,
reason: `${candidate} is not writable: ${cause instanceof Error ? cause.message : String(cause)}`,
};
candidate = parent;
}
}
};
-191
View File
@@ -1,191 +0,0 @@
import { constants } from "node:fs";
import { access, open, realpath, stat } from "node:fs/promises";
import { join } from "node:path";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import {
execFileOutput,
execFileOutputFailure,
} from "../process/ExecFileOutput.js";
import {
readKunaConfiguration,
type KunaConfiguration,
type KunaConfigurationFailure,
} from "./KunaConfiguration.js";
import {
KUNA_COMMAND_REMEDIATION,
KUNA_VERIFIED_VERSION,
KUNA_VERSION_PROBE_REMEDIATION,
} from "./KunaRelease.js";
/** `kuna --version` exits before loading specs or an input; it needs no budget beyond this. */
const VERSION_PROBE_TIMEOUT_MS = 10_000;
const VERSION_PROBE_MAX_OUTPUT_BYTES = 64 * 1024;
/** Release CI prints exactly `kuna <MAJOR.MINOR>`; source builds print the Cargo version. */
const VERSION_LINE = /^kuna (\d+\.\d+(?:\.\d+)?)\r?\n?$/u;
/** One observed Kuna executable, identified by its bytes as well as its version. */
export interface KunaInstallation {
readonly configuration: KunaConfiguration;
/** Canonical path of the probed executable. */
readonly executable: string;
readonly version: string;
/** Whether this release is the one REA verified. */
readonly verified: boolean;
readonly sha256: string;
readonly bytes: number;
/**
* Whether the executable is a `#!` script, such as a package-manager shim.
* Its process then reports the interpreter as its command, so ownership
* checks cannot match the configured path by argv prefix.
*/
readonly script: boolean;
}
/** Why the selected Kuna installation cannot be used. */
export interface KunaReadinessFailure {
readonly code:
| KunaConfigurationFailure["code"]
| "executable_missing"
| "version_unresolved";
readonly reason: string;
readonly diagnostics: Readonly<Record<string, JsonValue>>;
}
/**
* Environment for every Kuna child. Kuna unpacks its built-in SLEIGH tree into
* the per-user cache under HOME (macOS) or XDG_CACHE_HOME (Linux), so both
* point inside REA's own cache root instead of the user's.
*/
export const kunaChildEnvironment = (
configuration: KunaConfiguration,
): NodeJS.ProcessEnv => {
const home = join(configuration.cacheRoot, "home");
return {
HOME: home,
XDG_CACHE_HOME: join(home, ".cache"),
NO_COLOR: "1",
};
};
const failure = (
code: KunaReadinessFailure["code"],
reason: string,
diagnostics: Readonly<Record<string, JsonValue>>,
): Result<never, KunaReadinessFailure> => err({ code, reason, diagnostics });
const message = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
/** Resolve, version-probe and fingerprint the configured executable. */
export const inspectKunaInstallation = async (
environment: Readonly<NodeJS.ProcessEnv>,
options: {
readonly platform?: NodeJS.Platform;
readonly signal?: AbortSignal;
/** Probe deadline; tests shorten it. */
readonly probeTimeoutMs?: number;
} = {},
): Promise<Result<KunaInstallation, KunaReadinessFailure>> => {
const configuration = readKunaConfiguration(environment, options.platform);
if (!configuration.ok)
return failure(configuration.error.code, configuration.error.reason, {
configured: configuration.error.code !== "not_configured",
...(configuration.error.setting === null
? {}
: { setting: configuration.error.setting }),
});
const { command } = configuration.value;
let executable: string;
try {
executable = await realpath(command);
await access(executable, constants.X_OK);
if (!(await stat(executable)).isFile())
return failure(
"executable_missing",
`REA_KUNA_COMMAND is not a regular file: ${command}. ${KUNA_COMMAND_REMEDIATION}`,
{ command },
);
} catch (cause: unknown) {
options.signal?.throwIfAborted();
return failure(
"executable_missing",
`Cannot execute REA_KUNA_COMMAND ${command}: ${message(cause)}. ${KUNA_COMMAND_REMEDIATION}`,
{ command },
);
}
const deadline = options.probeTimeoutMs ?? VERSION_PROBE_TIMEOUT_MS;
let stdout: string;
try {
({ stdout } = await execFileOutput(executable, ["--version"], {
env: kunaChildEnvironment(configuration.value),
timeout: deadline,
maxBuffer: VERSION_PROBE_MAX_OUTPUT_BYTES,
// Escalate to SIGKILL so an executable that ignores SIGTERM cannot hang doctor.
stopSignal: "SIGTERM",
...(options.signal === undefined ? {} : { signal: options.signal }),
}));
} catch (cause: unknown) {
options.signal?.throwIfAborted();
const output = execFileOutputFailure(cause);
return failure(
"version_unresolved",
`${executable} --version failed: ${output?.stderr.trim() || message(cause)}. ${KUNA_VERSION_PROBE_REMEDIATION}`,
{
command,
executable,
exit_code: typeof output?.code === "number" ? output.code : null,
signal: output?.signal ?? null,
killed: output?.killed ?? false,
output_truncated: output?.outputTruncated ?? false,
deadline_ms: deadline,
},
);
}
const version = VERSION_LINE.exec(stdout)?.[1];
if (version === undefined)
return failure(
"version_unresolved",
`${executable} --version printed no recognizable Kuna version. ${KUNA_VERSION_PROBE_REMEDIATION}`,
{ command, executable, stdout: stdout.slice(0, 256) },
);
let fingerprint: { readonly sha256: string; readonly bytes: number };
let script: boolean;
try {
fingerprint = await hashStableFile(executable, options.signal);
script = await startsWithShebang(executable);
} catch (cause: unknown) {
options.signal?.throwIfAborted();
return failure(
"executable_missing",
`Cannot fingerprint ${executable}: ${message(cause)}`,
{ command, executable },
);
}
return ok({
configuration: configuration.value,
executable,
version,
verified: version === KUNA_VERIFIED_VERSION,
...fingerprint,
script,
});
};
const startsWithShebang = async (path: string): Promise<boolean> => {
const handle = await open(path, "r");
try {
const prefix = Buffer.alloc(2);
const { bytesRead } = await handle.read(prefix, 0, 2, 0);
return bytesRead === 2 && prefix.toString("latin1") === "#!";
} finally {
await handle.close();
}
};
-690
View File
@@ -1,690 +0,0 @@
import { createHash, randomUUID } from "node:crypto";
import {
mkdir,
mkdtemp,
readdir,
readFile,
rename,
rm,
stat,
utimes,
} from "node:fs/promises";
import { join } from "node:path";
import canonicalize from "canonicalize";
import writeFileAtomic from "write-file-atomic";
import { z } from "zod";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import type { KunaInstallation } from "./KunaInstallation.js";
import {
exportKunaProject,
KUNA_EXPORT_OPERATION,
kunaExportFailure,
readKunaExport,
type KunaExport,
type KunaExportStatus,
type KunaProcessLauncher,
} from "./KunaProjectExport.js";
import type {
KunaInventoryFunction,
KunaProjectFunction,
} from "./KunaProjectFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
/** Bumped whenever the entry layout or manifest meaning changes. */
const CACHE_LAYOUT = 1;
/** A manifest names five files and their digests; anything larger is not ours. */
const MANIFEST_MAX_BYTES = 64 * 1024;
const KEY_PATTERN = /^[0-9a-f]{64}$/u;
const SHA256_PATTERN = /^[0-9a-f]{64}$/u;
/** Inputs that determine an export's content. Each one is part of the cache key. */
export interface KunaProjectKeyInputs {
readonly targetSha256: string;
readonly kunaSha256: string;
readonly kunaVersion: string;
/**
* `--jobs 1` is Kuna's reference answer; more jobs decompile without
* callee-first prototypes and change some bodies.
*/
readonly jobs: number;
}
/** Content address of one export: every input that can change its bytes. */
export const kunaProjectKey = (inputs: KunaProjectKeyInputs): string =>
createHash("sha256")
.update(
canonicalize({
layout: CACHE_LAYOUT,
target_sha256: inputs.targetSha256,
kuna_sha256: inputs.kunaSha256,
kuna_version: inputs.kunaVersion,
export: { command: "decompile-project", jobs: inputs.jobs },
}) ?? "",
)
.digest("hex");
const fileSchema = z.strictObject({
path: z.string().min(1),
bytes: z.number().int().nonnegative(),
sha256: z.string().regex(SHA256_PATTERN),
});
const manifestSchema = z.strictObject({
layout: z.literal(CACHE_LAYOUT),
key: z.string().regex(KEY_PATTERN),
inputs: z.strictObject({
target_sha256: z.string().regex(SHA256_PATTERN),
kuna_sha256: z.string().regex(SHA256_PATTERN),
kuna_version: z.string().min(1),
jobs: z.number().int().positive(),
}),
files: z.strictObject({
c: fileSchema,
h: fileSchema,
asm: fileSchema,
readme: fileSchema,
inventory: fileSchema,
}),
summary: z.strictObject({
decompiled: z.number().int().nonnegative(),
failed: z.number().int().nonnegative(),
}),
created_at: z.string().datetime(),
});
/** Immutable description of one published export, written once at publish. */
export type KunaProjectManifest = z.infer<typeof manifestSchema>;
/** One verified, published export. */
export interface KunaProject {
readonly key: string;
/** Directory holding the export's `.c`, `.h`, `.asm`, README and inventory. */
readonly directory: string;
readonly manifest: KunaProjectManifest;
/** Decompiled function blocks in `.c` order. */
readonly functions: readonly KunaProjectFunction[];
/** Every callable entry Kuna found, including imports it does not decompile. */
readonly inventory: readonly KunaInventoryFunction[];
}
/** One request to produce or reuse the export for a target. */
export interface KunaProjectRequest {
readonly installation: KunaInstallation;
/** Caller-selected target path; REA decompiles a verified copy of it. */
readonly targetPath: string;
readonly targetSha256: string;
readonly signal?: AbortSignal;
readonly onStatus?: (status: KunaExportStatus) => void;
}
interface StagedResult {
readonly result: Result<KunaProject, AnalysisError>;
/** Set when an owned process may still write into staging. */
readonly retainStaging: boolean;
}
interface InflightBuild {
readonly key: string;
readonly promise: Promise<Result<KunaProject, AnalysisError>>;
readonly controller: AbortController;
readonly listeners: Set<(status: KunaExportStatus) => void>;
waiters: number;
}
const isMissing = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
(cause.code === "ENOENT" || cause.code === "ENOTDIR");
/** A file an entry's manifest names is gone, replaced or changed under it. */
const isDamaged = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
["ENOENT", "ENOTDIR", "EISDIR", "ENOTFILE", "ESTALE"].includes(
String(cause.code),
);
const isPublishedByAnother = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
(cause.code === "EEXIST" || cause.code === "ENOTEMPTY");
const parseManifest = (text: string): KunaProjectManifest | undefined => {
try {
const parsed = manifestSchema.safeParse(JSON.parse(text));
return parsed.success ? parsed.data : undefined;
} catch {
return undefined;
}
};
const manifestBytes = (manifest: KunaProjectManifest): number =>
Object.values(manifest.files).reduce((sum, file) => sum + file.bytes, 0);
/**
* REA-owned on-disk cache of `kuna decompile-project` exports.
*
* Layout under the root: `projects/<key>/` holds published entries
* (`manifest.json` plus the export and inventory in `project/`), `staging/`
* holds builds whose processes may still be live, `trash/` holds retired
* workspaces and evicted entries awaiting removal,
* and `home/` is Kuna's own HOME so its spec cache stays inside REA's
* ownership.
*
* Entries are content-addressed and immutable. A build runs in private
* staging and is published by one directory rename, so a reader sees a
* complete entry or none, and concurrent publishers of one key keep the first.
* File digests are recorded once at publish and checked on every first read.
* Recency is the entry directory's mtime; publishing evicts the least recently
* used entries above the size cap, never an entry pinned by this process.
*/
export class KunaProjectCache {
readonly #root: string;
readonly #maxBytes: number;
readonly #launcher: KunaProcessLauncher | undefined;
/** Builds new requests may join, at most one per key. */
readonly #inflight = new Map<string, InflightBuild>();
/** Every build not yet settled, including abandoned ones a retry replaced. */
readonly #unfinished = new Set<InflightBuild>();
readonly #verified = new Map<string, KunaProject>();
readonly #pins = new Map<string, number>();
/** One eviction decision at a time; exports themselves remain concurrent. */
#eviction: Promise<void> = Promise.resolve();
#capacity: "within_limit" | "pinned" | "building" = "within_limit";
#maintenance: Promise<void> | undefined;
#maintenanceRequested = false;
readonly #retainedProcesses: ProviderCleanupError[] = [];
/** Only resources whose process cleanup is proven may be retried as files. */
readonly #retainedFiles = new Map<string, ProviderCleanupError>();
constructor(
root: string,
options: {
readonly maxBytes: number;
readonly launcher?: KunaProcessLauncher;
},
) {
this.#root = root;
this.#maxBytes = options.maxBytes;
this.#launcher = options.launcher;
}
/** Keep an entry from eviction while this process serves it. */
pin(key: string): () => void {
this.#pins.set(key, (this.#pins.get(key) ?? 0) + 1);
let released = false;
return () => {
if (released) return;
released = true;
const count = (this.#pins.get(key) ?? 1) - 1;
if (count === 0) {
this.#pins.delete(key);
if (this.#capacity !== "within_limit") this.#scheduleEviction();
} else this.#pins.set(key, count);
};
}
/**
* Return the verified published entry for a key, or undefined when none is
* usable. A corrupt or tampered entry is removed and reported as absent.
*/
async lookup(
key: string,
signal?: AbortSignal,
): Promise<KunaProject | undefined> {
const verified = this.#verified.get(key);
if (verified !== undefined) {
// Another process may have evicted it since this one verified it.
if (await this.#touch(key)) return verified;
this.#verified.delete(key);
return undefined;
}
const entry = join(this.#root, "projects", key);
let text: string;
try {
text = await this.#readBounded(join(entry, "manifest.json"));
} catch (cause: unknown) {
if (isMissing(cause)) return undefined;
if (!(cause instanceof RangeError)) throw cause;
await this.#discard(key);
return undefined;
}
const manifest = parseManifest(text);
if (manifest?.key !== key) {
await this.#discard(key);
return undefined;
}
const directory = join(entry, "project");
for (const file of Object.values(manifest.files)) {
let observed: { readonly sha256: string; readonly bytes: number };
try {
observed = await hashStableFile(join(directory, file.path), signal);
} catch (cause: unknown) {
signal?.throwIfAborted();
if (!isDamaged(cause)) throw cause;
await this.#discard(key);
return undefined;
}
if (observed.sha256 !== file.sha256 || observed.bytes !== file.bytes) {
await this.#discard(key);
return undefined;
}
}
const parsed = await readKunaExport(
directory,
manifest.files,
manifest.summary,
signal,
);
if (!parsed.ok) {
await this.#discard(key);
return undefined;
}
const project = { key, directory, manifest, ...parsed.value };
if (!(await this.#touch(key))) return undefined;
this.#verified.set(key, project);
return project;
}
/**
* Return the entry for a request, building and publishing it when absent.
* Concurrent requests for one key share one Kuna run; the run is cancelled
* only when every waiting request has been cancelled.
*/
async build(
request: KunaProjectRequest,
): Promise<Result<KunaProject, AnalysisError>> {
if (request.signal?.aborted === true)
return err(new AnalysisCancelledError(KUNA_EXPORT_OPERATION));
const key = kunaProjectKey({
targetSha256: request.targetSha256,
kunaSha256: request.installation.sha256,
kunaVersion: request.installation.version,
jobs: request.installation.configuration.jobs,
});
const running = this.#inflight.get(key);
// A build whose waiters all left is winding down; a new request starts afresh.
if (running !== undefined && !running.controller.signal.aborted)
return this.#await(running, request);
const controller = new AbortController();
const listeners = new Set<(status: KunaExportStatus) => void>();
const promise = this.#produce(key, request, controller.signal, (status) => {
for (const listener of listeners) listener(status);
});
const created: InflightBuild = {
key,
promise,
controller,
listeners,
waiters: 0,
};
this.#inflight.set(key, created);
this.#unfinished.add(created);
void promise.then(() => {
this.#unfinished.delete(created);
if (this.#inflight.get(key) === created) this.#inflight.delete(key);
if (this.#unfinished.size === 0 && this.#capacity === "building")
this.#scheduleEviction();
});
return this.#await(created, request);
}
/** Wait for builds, retry safe file removal, and report unresolved cleanup. */
async close(): Promise<Result<null, ProviderCleanupError>> {
await Promise.allSettled(
[...this.#unfinished].map((entry) => entry.promise),
);
while (this.#maintenance !== undefined) await this.#maintenance;
for (const path of [...this.#retainedFiles.keys()])
await this.#removeRetired(path).catch(() => undefined);
const retained = [
...this.#retainedProcesses,
...this.#retainedFiles.values(),
];
if (retained.length === 0) return ok(null);
return err(
new ProviderCleanupError(
KUNA_PROVIDER_ID,
retained.flatMap((entry) => entry.cleanupResources),
{
reason: retained
.map((entry) =>
String(entry.diagnostics?.["reason"] ?? entry.message),
)
.join("; "),
},
{ operation: KUNA_EXPORT_OPERATION },
),
);
}
#await(
entry: InflightBuild,
request: KunaProjectRequest,
): Promise<Result<KunaProject, AnalysisError>> {
entry.waiters += 1;
const onStatus = request.onStatus;
// Each waiter owns its registration, even when callers reuse a callback.
const listener =
onStatus === undefined
? undefined
: (status: KunaExportStatus) => onStatus(status);
if (listener !== undefined) entry.listeners.add(listener);
return new Promise((resolve) => {
let settled = false;
const leave = () => {
if (listener !== undefined) entry.listeners.delete(listener);
entry.waiters -= 1;
};
const onAbort = () => {
if (settled) return;
settled = true;
leave();
if (entry.waiters === 0) entry.controller.abort();
resolve(err(new AnalysisCancelledError(KUNA_EXPORT_OPERATION)));
};
request.signal?.addEventListener("abort", onAbort, { once: true });
void entry.promise.then((result) => {
request.signal?.removeEventListener("abort", onAbort);
if (settled) return;
settled = true;
leave();
resolve(result);
});
});
}
async #produce(
key: string,
request: KunaProjectRequest,
signal: AbortSignal,
onStatus: (status: KunaExportStatus) => void,
): Promise<Result<KunaProject, AnalysisError>> {
let staging: string;
try {
const existing = await this.lookup(key, signal);
if (existing !== undefined) return ok(existing);
await Promise.all(
["projects", "staging", "trash", "home"].map((name) =>
mkdir(join(this.#root, name), { recursive: true, mode: 0o700 }),
),
);
await this.#sweep(signal);
staging = await mkdtemp(join(this.#root, "staging", "build-"));
} catch (cause: unknown) {
return err(kunaExportFailure(cause, signal));
}
const built = await this.#exportAndPublish(
key,
staging,
request,
signal,
onStatus,
).catch((cause: unknown): StagedResult => ({
result: err(kunaExportFailure(cause, signal)),
retainStaging: false,
}));
if (built.retainStaging) return built.result;
let cleanupPath = staging;
try {
// Only a proven-stopped build may enter trash. A failed write, crash or
// lost cleanup identity therefore never makes live staging reclaimable.
const retired = join(this.#root, "trash", `build-${randomUUID()}`);
await rename(staging, retired);
cleanupPath = retired;
await this.#removeRetired(retired);
} catch (cause: unknown) {
if (cause instanceof ProviderCleanupError) return err(cause);
return err(this.#retainFileCleanup(cleanupPath, cause));
}
if (built.result.ok)
// The entry is published either way; a failed eviction retries on the next publish.
await this.#evict(key).catch(() => undefined);
return built.result;
}
async #exportAndPublish(
key: string,
staging: string,
request: KunaProjectRequest,
signal: AbortSignal,
onStatus: (status: KunaExportStatus) => void,
): Promise<StagedResult> {
const outcome = await exportKunaProject({
installation: request.installation,
targetPath: request.targetPath,
targetSha256: request.targetSha256,
workspace: staging,
output: join(staging, "entry", "project"),
signal,
onStatus,
launcher: this.#launcher,
});
if (outcome.kind === "retained") {
this.#retainedProcesses.push(outcome.error);
return { result: err(outcome.error), retainStaging: true };
}
return {
result: outcome.result.ok
? await this.#publish(
key,
staging,
request,
outcome.result.value,
signal,
)
: outcome.result,
retainStaging: false,
};
}
/** Record the manifest and publish the staged export under its key. */
async #publish(
key: string,
staging: string,
request: KunaProjectRequest,
exported: KunaExport,
signal: AbortSignal,
): Promise<Result<KunaProject, AnalysisError>> {
const manifest: KunaProjectManifest = {
layout: CACHE_LAYOUT,
key,
inputs: {
target_sha256: request.targetSha256,
kuna_sha256: request.installation.sha256,
kuna_version: request.installation.version,
jobs: request.installation.configuration.jobs,
},
files: exported.files,
summary: exported.summary,
created_at: new Date().toISOString(),
};
const total = manifestBytes(manifest);
if (total > this.#maxBytes)
return err(
new AnalysisResourceConstraintError(
KUNA_EXPORT_OPERATION,
"capacity",
`This export occupies ${String(total)} bytes, more than the whole Kuna cache may hold.`,
{ cache_max_bytes: this.#maxBytes, export_bytes: total },
{
remediationAction:
"Raise REA_KUNA_CACHE_MAX_BYTES above the export size, or analyze this target with another provider.",
},
),
);
await writeFileAtomic(
join(staging, "entry", "manifest.json"),
`${JSON.stringify(manifest, null, 2)}\n`,
{ mode: 0o600 },
);
const destination = join(this.#root, "projects", key);
try {
await rename(join(staging, "entry"), destination);
} catch (cause: unknown) {
if (!isPublishedByAnother(cause)) throw cause;
const winner = await this.lookup(key, signal);
if (winner !== undefined) return ok(winner);
throw cause;
}
const project = {
key,
directory: join(destination, "project"),
manifest,
functions: exported.functions,
inventory: exported.inventory,
};
this.#verified.set(key, project);
return ok(project);
}
/**
* Remove retired resources only. Age and a missing marker cannot establish
* that a process stopped, so unresolved staging is never swept.
*/
async #sweep(signal: AbortSignal): Promise<void> {
for (const name of await readdir(join(this.#root, "trash"))) {
signal.throwIfAborted();
await this.#removeRetired(join(this.#root, "trash", name)).catch(
() => undefined,
);
}
}
/** Evict least recently used entries above the cap, keeping pinned ones. */
#evict(keep: string | null): Promise<void> {
const eviction = this.#eviction.then(() => this.#evictUnpinned(keep));
this.#eviction = eviction.catch(() => undefined);
return eviction;
}
/** Coalesce deferred reclamation after callers have had a turn to pin results. */
#scheduleEviction(): void {
this.#maintenanceRequested = true;
if (this.#maintenance !== undefined) return;
const maintenance = (async () => {
while (this.#maintenanceRequested) {
await new Promise<void>((resolve) => setImmediate(resolve));
this.#maintenanceRequested = false;
await this.#evict(null).catch(() => undefined);
}
})();
this.#maintenance = maintenance;
void maintenance.then(() => {
if (this.#maintenance === maintenance) this.#maintenance = undefined;
if (this.#maintenanceRequested) this.#scheduleEviction();
});
}
async #evictUnpinned(keep: string | null): Promise<void> {
// A successful build must reach its callers before they can pin its entry.
// Protect every unfinished build while reading this eviction snapshot.
const building = new Set([...this.#unfinished].map(({ key }) => key));
const entries: { key: string; bytes: number; mtimeMs: number }[] = [];
for (const name of await readdir(join(this.#root, "projects"))) {
if (!KEY_PATTERN.test(name)) continue;
const path = join(this.#root, "projects", name);
const status = await stat(path).catch(() => undefined);
if (status === undefined) continue;
const manifest = await this.#readBounded(join(path, "manifest.json"))
.then(parseManifest)
.catch(() => undefined);
// An owned entry without a valid manifest is unusable; evict it first.
entries.push(
manifest === undefined
? { key: name, bytes: 0, mtimeMs: -1 }
: {
key: name,
bytes: manifestBytes(manifest),
mtimeMs: status.mtimeMs,
},
);
}
let total = entries.reduce((sum, entry) => sum + entry.bytes, 0);
let deferredByBuild = false;
for (const entry of entries.sort((a, b) => a.mtimeMs - b.mtimeMs)) {
if (total <= this.#maxBytes && entry.mtimeMs >= 0) break;
if (building.has(entry.key) || this.#inflight.has(entry.key)) {
deferredByBuild = true;
continue;
}
if (entry.key === keep || this.#pins.has(entry.key)) continue;
await this.#discard(entry.key);
total -= entry.bytes;
}
this.#capacity =
total <= this.#maxBytes
? "within_limit"
: deferredByBuild
? "building"
: "pinned";
}
/** Atomically retire one owned entry, then remove it. */
async #discard(key: string): Promise<void> {
this.#verified.delete(key);
const trash = join(this.#root, "trash", `${key}-${randomUUID()}`);
try {
await mkdir(join(this.#root, "trash"), { recursive: true, mode: 0o700 });
await rename(join(this.#root, "projects", key), trash);
} catch (cause: unknown) {
if (isMissing(cause)) return;
throw cause;
}
await this.#removeRetired(trash);
}
/** Keep each failed removal owned until that path is successfully removed. */
async #removeRetired(path: string): Promise<void> {
try {
await rm(path, { recursive: true, force: true });
this.#retainedFiles.delete(path);
} catch (cause: unknown) {
throw this.#retainFileCleanup(path, cause);
}
}
/** Mark an entry recently used; false when it is no longer published. */
async #touch(key: string): Promise<boolean> {
const now = new Date();
try {
await utimes(join(this.#root, "projects", key), now, now);
return true;
} catch (cause: unknown) {
// Recency is advisory; only a missing entry changes the answer.
return !isMissing(cause);
}
}
async #readBounded(path: string): Promise<string> {
const status = await stat(path);
if (status.size > MANIFEST_MAX_BYTES)
throw new RangeError(
`${path} exceeds ${String(MANIFEST_MAX_BYTES)} bytes`,
);
return readFile(path, "utf8");
}
#retainFileCleanup(path: string, cause: unknown): ProviderCleanupError {
const error = new ProviderCleanupError(
KUNA_PROVIDER_ID,
[path],
{ reason: cause instanceof Error ? cause.message : String(cause) },
{ operation: KUNA_EXPORT_OPERATION, cause },
);
this.#retainedFiles.set(path, error);
return error;
}
}
-466
View File
@@ -1,466 +0,0 @@
import { createHash, randomUUID } from "node:crypto";
import { createReadStream, createWriteStream } from "node:fs";
import { lstat, mkdir, readFile, writeFile } from "node:fs/promises";
import { basename, join } from "node:path";
import { Transform } from "node:stream";
import { pipeline } from "node:stream/promises";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisArtifactChangedError,
AnalysisCancelledError,
AnalysisResourceConstraintError,
AnalysisTimeoutError,
} from "../domain/analysisErrorCore.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import { prepareProcessOwnershipInspection } from "../process/ProcessOwnershipObservation.js";
import type {
OwnedProviderProcessSpawnOptions,
ProviderProcessSnapshot,
SpawnedOwnedProviderProcess,
} from "../process/ProviderProcess.js";
import {
kunaChildEnvironment,
type KunaInstallation,
} from "./KunaInstallation.js";
import {
parseKunaExportSummary,
parseKunaInventory,
scanKunaProjectSource,
type KunaExportSummary,
type KunaInventoryFunction,
type KunaProjectFunction,
} from "./KunaProjectFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
/** Operation name every export and cache failure reports. */
export const KUNA_EXPORT_OPERATION = "kuna:decompile-project";
/** Kuna prints one summary line per export; notes on a large image stay far below this. */
const DIAGNOSTIC_BYTES = 8 * 1024 * 1024;
/**
* `kuna functions --json` costs ~210 bytes per function; this admits about
* 600,000 functions, above the 392,814 of the largest image Kuna documents.
*/
const INVENTORY_BYTES = 128 * 1024 * 1024;
const INVENTORY_FILE = "functions.json";
/** Identity of one file an export consists of. */
export interface KunaExportFile {
readonly path: string;
readonly bytes: number;
readonly sha256: string;
}
/** The files of one export, by role. */
export interface KunaExportFiles {
readonly c: KunaExportFile;
readonly h: KunaExportFile;
readonly asm: KunaExportFile;
readonly readme: KunaExportFile;
readonly inventory: KunaExportFile;
}
/** One validated export: its files, Kuna's counts and the facts parsed from them. */
export interface KunaExport {
readonly files: KunaExportFiles;
readonly summary: KunaExportSummary;
/** Decompiled function blocks in `.c` order. */
readonly functions: readonly KunaProjectFunction[];
/** Every callable entry Kuna found, including imports it does not decompile. */
readonly inventory: readonly KunaInventoryFunction[];
}
/** The step an export is in; Kuna itself reports no progress until it finishes. */
export interface KunaExportStatus {
readonly phase: "inventory" | "decompiling" | "sealing";
/** Callable entries found by the inventory, once known. */
readonly inventoryFunctions: number | null;
}
/** Test seam for the owned process launcher; production uses the default. */
export type KunaProcessLauncher = (
options: OwnedProviderProcessSpawnOptions,
) => Promise<SpawnedOwnedProviderProcess>;
/**
* How an export ended. `retained` means an owned Kuna process could not be
* proven stopped, so it may still write into the workspace, which therefore
* stays with the caller's cleanup owner.
*/
export type KunaExportOutcome =
| {
readonly kind: "done";
readonly result: Result<KunaExport, AnalysisError>;
}
| { readonly kind: "retained"; readonly error: ProviderCleanupError };
/** One export of a target into a private workspace. */
export interface KunaExportRequest {
readonly installation: KunaInstallation;
/** Caller-selected target path; Kuna decompiles a verified copy of it. */
readonly targetPath: string;
readonly targetSha256: string;
/** Private directory for the copy, Kuna's temporary files and its output. */
readonly workspace: string;
/** Directory, inside the workspace, that receives the export's files. */
readonly output: string;
readonly signal: AbortSignal;
readonly onStatus: (status: KunaExportStatus) => void;
readonly launcher?: KunaProcessLauncher | undefined;
}
const message = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
const isMissing = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
(cause.code === "ENOENT" || cause.code === "ENOTDIR");
/** Map an unexpected failure to a typed one, preferring cancellation. */
export const kunaExportFailure = (
cause: unknown,
signal: AbortSignal,
): AnalysisError => {
if (signal.aborted)
return new AnalysisCancelledError(KUNA_EXPORT_OPERATION, { cause });
if (cause instanceof AnalysisError) return cause;
return new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
cause,
userMessage: `Kuna project cache failed: ${message(cause)}`,
});
};
const unusable = (reason: string): KunaExportOutcome => ({
kind: "done",
result: err(
new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
userMessage: `Kuna's project export is not usable: ${reason}`,
}),
),
});
/** Verify the selected installation at each execution and publication boundary. */
const verifyKunaExecutable = async (
installation: KunaInstallation,
signal: AbortSignal,
): Promise<Result<null, ProviderAdapterError>> => {
const executable = await hashStableFile(installation.executable, signal);
if (executable.sha256 === installation.sha256) return ok(null);
return err(
new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
userMessage: `${installation.executable} changed after readiness inspection; reopen the target so REA probes the current Kuna executable.`,
diagnostics: {
expected_sha256: installation.sha256,
observed_sha256: executable.sha256,
},
}),
);
};
/**
* Inventory and export one target with Kuna, then validate the result.
*
* Kuna runs on a copy hashed while copying, so the export always describes
* the digest the caller selected. Both Kuna runs share one deadline.
*/
export const exportKunaProject = async (
request: KunaExportRequest,
): Promise<KunaExportOutcome> => {
const { installation, signal } = request;
const { configuration } = installation;
const deadline = Date.now() + configuration.projectTimeoutMs;
const input = join(request.workspace, "input", basename(request.targetPath));
await mkdir(join(request.workspace, "input"), { mode: 0o700 });
await mkdir(request.output, { recursive: true, mode: 0o700 });
const copied = await copyAndHash(request.targetPath, input, signal);
if (copied !== request.targetSha256)
return {
kind: "done",
result: err(
new AnalysisArtifactChangedError(
KUNA_EXPORT_OPERATION,
request.targetPath,
"the target no longer matches the digest selected for analysis",
),
),
};
await prepareProcessOwnershipInspection(signal);
const run = (arguments_: readonly string[], diagnosticBytes: number) =>
runKuna(request, arguments_, { deadline, diagnosticBytes });
request.onStatus({ phase: "inventory", inventoryFunctions: null });
const listed = await run(["functions", input, "--json"], INVENTORY_BYTES);
if (!listed.ok) return listed.outcome;
const inventory = parseKunaInventory(listed.snapshot.stdout.text);
if (!inventory.ok) return unusable(inventory.error);
if (listed.snapshot.stdout.invalidUtf8 === true)
return unusable("the function inventory is not valid UTF-8");
await writeFile(
join(request.output, INVENTORY_FILE),
listed.snapshot.stdout.text,
{ mode: 0o600 },
);
request.onStatus({
phase: "decompiling",
inventoryFunctions: inventory.value.length,
});
const exported = await run(
[
"decompile-project",
input,
"-o",
request.output,
"--jobs",
String(configuration.jobs),
],
DIAGNOSTIC_BYTES,
);
if (!exported.ok) return exported.outcome;
request.onStatus({
phase: "sealing",
inventoryFunctions: inventory.value.length,
});
const summary = parseKunaExportSummary(exported.snapshot.stdout.text);
if (summary === undefined)
return unusable(
"Kuna did not print its `functions: N ok, M failed` summary.",
);
const stem = basename(request.targetPath);
const names = {
c: `${stem}.c`,
h: `${stem}.h`,
asm: `${stem}.asm`,
readme: "README.md",
inventory: INVENTORY_FILE,
} as const;
const files: Partial<Record<keyof KunaExportFiles, KunaExportFile>> = {};
for (const [role, name] of Object.entries(names) as [
keyof KunaExportFiles,
string,
][]) {
const path = join(request.output, name);
const status = await lstat(path).catch((cause: unknown) => {
if (isMissing(cause)) return undefined;
throw cause;
});
if (status === undefined || !status.isFile())
return unusable(`Kuna's export has no regular ${name}.`);
files[role] = { path: name, ...(await hashStableFile(path, signal)) };
}
const complete = files as KunaExportFiles;
const parsed = await readKunaExport(
request.output,
complete,
summary,
signal,
);
if (!parsed.ok) return unusable(parsed.error);
const identity = await verifyKunaExecutable(installation, signal);
if (!identity.ok) return { kind: "done", result: identity };
return {
kind: "done",
result: ok({ files: complete, summary, ...parsed.value }),
};
};
/** Parse an export's function blocks and inventory from its own files. */
export const readKunaExport = async (
directory: string,
files: KunaExportFiles,
summary: KunaExportSummary,
signal?: AbortSignal,
): Promise<
Result<
{
readonly functions: readonly KunaProjectFunction[];
readonly inventory: readonly KunaInventoryFunction[];
},
string
>
> => {
const functions = await scanKunaProjectSource(
createReadStream(
join(directory, files.c.path),
signal === undefined ? {} : { signal },
),
summary,
);
if (!functions.ok) return functions;
const inventory = parseKunaInventory(
await readFile(join(directory, files.inventory.path), {
encoding: "utf8",
...(signal === undefined ? {} : { signal }),
}),
);
if (!inventory.ok) return inventory;
return ok({ functions: functions.value, inventory: inventory.value });
};
const runKuna = async (
request: KunaExportRequest,
arguments_: readonly string[],
limits: { readonly deadline: number; readonly diagnosticBytes: number },
): Promise<
| { readonly ok: true; readonly snapshot: ProviderProcessSnapshot }
| { readonly ok: false; readonly outcome: KunaExportOutcome }
> => {
const { installation, signal } = request;
try {
const identity = await verifyKunaExecutable(installation, signal);
if (!identity.ok)
return { ok: false, outcome: { kind: "done", result: identity } };
const snapshot = await runOwnedCommand(
{
command: installation.executable,
arguments: arguments_,
runId: `rea-kuna-${randomUUID()}`,
// A script's process reports its interpreter, not the configured path.
...(installation.script ? { expectedCommand: null } : {}),
cwd: request.workspace,
hostEnvironment: kunaChildEnvironment(installation.configuration),
env: { TMPDIR: request.workspace },
},
{
timeoutMs: Math.max(1, limits.deadline - Date.now()),
diagnosticBytes: limits.diagnosticBytes,
},
{
signal,
...(request.launcher === undefined
? {}
: { launcher: request.launcher }),
},
);
return { ok: true, snapshot };
} catch (cause: unknown) {
return {
ok: false,
outcome: commandFailure(cause, request, arguments_[0] ?? "", limits),
};
}
};
/** Copy the target into the workspace and return the SHA-256 of exactly the bytes copied. */
const copyAndHash = async (
source: string,
destination: string,
signal: AbortSignal,
): Promise<string> => {
const hash = createHash("sha256");
await pipeline(
createReadStream(source),
new Transform({
transform(chunk: Buffer, _encoding, callback) {
hash.update(chunk);
callback(null, chunk);
},
}),
createWriteStream(destination, { flags: "wx", mode: 0o600 }),
{ signal },
);
return hash.digest("hex");
};
/** Map an owned-command failure to its typed error, keeping unproven cleanup owned. */
const commandFailure = (
cause: unknown,
request: KunaExportRequest,
subcommand: string,
limits: { readonly diagnosticBytes: number },
): KunaExportOutcome => {
const { installation, signal } = request;
if (!(cause instanceof OwnedCommandFailure))
return {
kind: "done",
result: err(
signal.aborted
? new AnalysisCancelledError(KUNA_EXPORT_OPERATION, { cause })
: new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
cause,
userMessage: `Cannot run ${installation.executable}: ${message(cause)}`,
}),
),
};
const snapshot = cause.snapshot;
const capturedOutput =
snapshot === null
? undefined
: {
// An inventory is evidence-sized, not diagnostic-sized; keep only its tail.
stdout: snapshot.stdout.text.slice(-4096),
stderr: snapshot.stderr.text,
truncated:
snapshot.diagnosticTruncated === true ||
snapshot.outputIncomplete === true ||
snapshot.stdout.text.length > 4096,
exit_code: snapshot.exitCode ?? null,
signal: snapshot.signal ?? null,
};
const details = {
cause,
...(capturedOutput === undefined ? {} : { capturedOutput }),
};
if (cause.cleanupFailure !== null)
return {
kind: "retained",
error: new ProviderCleanupError(
KUNA_PROVIDER_ID,
[...cause.resources, request.workspace],
{ reason: cause.cleanupFailure },
{ operation: KUNA_EXPORT_OPERATION, cause },
),
};
if (cause.reason === "cancelled")
return {
kind: "done",
result: err(new AnalysisCancelledError(KUNA_EXPORT_OPERATION, details)),
};
if (cause.reason === "timeout")
return {
kind: "done",
result: err(
new AnalysisTimeoutError(
KUNA_EXPORT_OPERATION,
installation.configuration.projectTimeoutMs,
details,
),
),
};
if (cause.reason === "output-limit")
return {
kind: "done",
result: err(
new AnalysisResourceConstraintError(
KUNA_EXPORT_OPERATION,
"capacity",
`kuna ${subcommand} wrote more than ${String(limits.diagnosticBytes)} bytes of output; the export was not retained.`,
{ output_limit_bytes: limits.diagnosticBytes },
details,
),
),
};
const exitCode = capturedOutput?.exit_code;
return {
kind: "done",
result: err(
new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
...details,
userMessage: `kuna ${subcommand} failed${exitCode === undefined || exitCode === null ? "" : ` with exit code ${String(exitCode)}`}: ${capturedOutput?.stderr.trim().split("\n").at(-1) ?? cause.message}`,
}),
),
};
};
-189
View File
@@ -1,189 +0,0 @@
import { z } from "zod";
import { err, ok, type Result } from "../domain/result.js";
/**
* One function block in a `kuna decompile-project` `.c` file. The span covers
* its `// Function:` header through the blank line that ends the block, which
* is how REA serves one function without reading the whole file.
*/
export interface KunaProjectFunction {
readonly name: string;
/** Kuna's display address, a lowercase `0x…` virtual address. */
readonly address: string;
readonly cOffset: number;
readonly cLength: number;
/** Kuna's own failure for this function; its block is then the header alone. */
readonly error: string | null;
}
// Rendered by kuna-console's `render_c`: `// Function: {name} @ 0x{addr:x}`,
// or `... @ 0x{addr:x} (error: {message})` for a failed function. A name
// can itself contain " @ ", so the address is the last such field.
const HEADER = /^\/\/ Function: (.+) @ (0x[0-9a-f]+)(?: {2}\(error: (.*)\))?$/u;
const HEADER_PREFIX = "// Function: ";
/** Kuna's completed-export summary on stdout: `functions: N ok, M failed`. */
const SUMMARY = /functions: (\d+) ok, (\d+) failed/u;
/** Function counts Kuna reported for a finished export. */
export interface KunaExportSummary {
readonly decompiled: number;
readonly failed: number;
}
/** Read the completed-file summary Kuna prints after finalizing an export. */
export const parseKunaExportSummary = (
stdout: string,
): KunaExportSummary | undefined => {
const match = SUMMARY.exec(stdout);
if (match?.[1] === undefined || match[2] === undefined) return undefined;
return { decompiled: Number(match[1]), failed: Number(match[2]) };
};
/** A header line longer than this is not one Kuna wrote. */
const MAX_HEADER_BYTES = 64 * 1024;
const PREFIX_BYTES = Buffer.from(HEADER_PREFIX);
interface Header {
readonly name: string;
readonly address: string;
readonly error: string | null;
readonly offset: number;
}
/**
* Split a finished export's `.c` into its function blocks, reading it as a
* stream so a very large export is never held in memory.
*
* The `#include` preamble precedes the first header. The header count must
* equal the functions Kuna reported, and every address must be distinct; an
* export that fails either check is not a usable function inventory.
*/
export const scanKunaProjectSource = async (
chunks: AsyncIterable<Buffer>,
summary: KunaExportSummary,
): Promise<Result<readonly KunaProjectFunction[], string>> => {
const headers: Header[] = [];
let offset = 0;
let lineStart = 0;
// Bytes of the current line, kept only while it may still be a header.
let line: Buffer[] | undefined = [];
let lineBytes = 0;
const endLine = (): string | undefined => {
if (line === undefined) return undefined;
const text = Buffer.concat(line).toString("utf8");
if (!text.startsWith(HEADER_PREFIX)) return undefined;
const match = HEADER.exec(text);
if (match?.[1] === undefined || match[2] === undefined)
return `unrecognized function header at byte ${String(lineStart)}`;
headers.push({
name: match[1],
address: match[2],
error: match[3] ?? null,
offset: lineStart,
});
return undefined;
};
for await (const chunk of chunks) {
let position = 0;
while (position < chunk.length) {
const newline = chunk.indexOf(0x0a, position);
const end = newline === -1 ? chunk.length : newline;
if (line !== undefined) {
const segment = chunk.subarray(position, end);
line.push(segment);
lineBytes += segment.length;
const seen = Buffer.concat(line).subarray(0, PREFIX_BYTES.length);
if (!PREFIX_BYTES.subarray(0, seen.length).equals(seen))
line = undefined;
else if (lineBytes > MAX_HEADER_BYTES)
return err(
`function header at byte ${String(lineStart)} is too long`,
);
}
if (newline === -1) break;
const failure = endLine();
if (failure !== undefined) return err(failure);
position = newline + 1;
lineStart = offset + position;
line = [];
lineBytes = 0;
}
offset += chunk.length;
}
const failure = endLine();
if (failure !== undefined) return err(failure);
const total = offset;
const expected = summary.decompiled + summary.failed;
if (headers.length !== expected)
return err(
`the .c holds ${String(headers.length)} function headers but Kuna reported ${String(expected)} functions`,
);
const failed = headers.filter(({ error }) => error !== null).length;
if (failed !== summary.failed)
return err(
`the .c marks ${String(failed)} failed functions but Kuna reported ${String(summary.failed)}`,
);
const addresses = new Set<string>();
const functions: KunaProjectFunction[] = [];
for (const [index, header] of headers.entries()) {
if (addresses.has(header.address))
return err(`the .c repeats address ${header.address}`);
addresses.add(header.address);
const end = headers[index + 1]?.offset ?? total;
functions.push({
name: header.name,
address: header.address,
cOffset: header.offset,
cLength: end - header.offset,
error: header.error,
});
}
return ok(functions);
};
const inventorySchema = z.object({
functions: z.array(
z.object({
name: z.string(),
address_hex: z.string().regex(/^0x[0-9a-f]+$/u),
aliases: z.array(z.string()),
size: z.number().int().nonnegative().max(Number.MAX_SAFE_INTEGER),
}),
),
});
/** One callable entry from `kuna functions --json`, including imports Kuna does not decompile. */
export interface KunaInventoryFunction {
readonly name: string;
readonly address: string;
readonly aliases: readonly string[];
/** Inventory extent: an upper bound clipped at the next entry, or 0 when unmeasured. */
readonly size: number;
}
/** Parse the callable-symbol inventory `kuna functions --json` writes. */
export const parseKunaInventory = (
text: string,
): Result<readonly KunaInventoryFunction[], string> => {
let value: unknown;
try {
value = JSON.parse(text);
} catch {
return err("the function inventory is not JSON");
}
const parsed = inventorySchema.safeParse(value);
if (!parsed.success)
return err(
`the function inventory does not match Kuna's document: ${parsed.error.issues[0]?.message ?? "invalid"}`,
);
return ok(
parsed.data.functions.map((entry) => ({
name: entry.name,
address: entry.address_hex,
aliases: entry.aliases,
size: entry.size,
})),
);
};
-16
View File
@@ -1,16 +0,0 @@
/** Kuna release whose CLI and project layout REA verified; others report as unverified. */
export const KUNA_VERIFIED_VERSION = "1.833";
/** Stable provider ID shared by readiness, doctor and Evidence. */
export const KUNA_PROVIDER_ID = "kuna";
/** Remediation when `REA_KUNA_COMMAND` is missing or not an absolute path. */
export const KUNA_COMMAND_REMEDIATION = `Set REA_KUNA_COMMAND to the absolute path of a caller-supplied kuna executable (verified with ${KUNA_VERIFIED_VERSION}). REA does not download, install or update Kuna.`;
/** Remediation when the host cannot supervise a Kuna process tree. */
export const KUNA_UNSUPPORTED_HOST_REMEDIATION =
"Use Linux or macOS for the Kuna provider. Windows process-tree ownership for Kuna is not verified.";
/** Remediation when `kuna --version` cannot establish the installed release. */
export const KUNA_VERSION_PROBE_REMEDIATION =
"Run the selected executable with --version: it must exit promptly and print `kuna <major>.<minor>`.";
@@ -339,38 +339,6 @@ it.skipIf(process.platform === "win32")(
},
);
it.skipIf(process.platform === "win32")(
"keeps the observed symlink target when dot segments resolve to the same path",
async () => {
const root = await createTestTempDirectory("rea-fs-symlink-spelling-");
await mkdir(join(root, "aaa"));
await writeFile(join(root, "target"), "content");
// Equal length, so only the target spelling distinguishes the links;
// "bbb" does not exist, so the replacement link is broken.
await symlink("aaa/../target", join(root, "link"));
const scenario = parseProcessScenario({
executable: process.execPath,
working_directory: root,
filesystem_observation_paths: [root],
limits: { file_bytes: 1 },
});
const before = await snapshotRoots(scenario);
await rm(join(root, "link"));
await symlink("bbb/../target", join(root, "link"));
const after = await snapshotRoots(scenario);
const link = (snapshot: typeof before) =>
snapshot.files.find(({ path }) => path === "root_0:link");
expect(link(before)?.symlink_target).toBe("aaa/../target");
expect(link(after)?.symlink_target).toBe("bbb/../target");
expect(
classifyFilesystemEffects(before, after).find(
({ path }) => path === "root_0:link",
)?.status,
).toBe("modified");
},
);
it("preserves creation when the selected root was observed missing before capture", async () => {
const root = await createTestTempDirectory("rea-fs-missing-root-");
const file = join(root, "created.txt");
+2 -3
View File
@@ -1,7 +1,7 @@
import { createHash } from "node:crypto";
import { constants as fsConstants } from "node:fs";
import { lstat, open, opendir, readlink } from "node:fs/promises";
import { join, relative } from "node:path";
import { dirname, join, relative, resolve } from "node:path";
import type { FileHandle } from "node:fs/promises";
import type { ProcessScenario } from "../../domain/process/processScenario.js";
import type { FilesystemCoverage } from "../../domain/process/processCaptureCoverage.js";
@@ -224,8 +224,7 @@ export const snapshotRoots = async (
if (stats === undefined) return true;
const relativePath = relative(root, path) || ".";
if (stats.isSymbolicLink()) {
// Keep the target as stored: lexical resolution can merge distinct targets.
const target = await readlink(path);
const target = resolve(dirname(path), await readlink(path));
const afterRead = await lstat(path);
if (!hasSameIdentity(stats, afterRead, "symlink")) {
truncated = true;
+13 -13
View File
@@ -12,6 +12,7 @@ import { captureProcessScenario } from "../process/capture/ProcessHarness.js";
import { ProcessCaptureResourceScope } from "../process/capture/ProcessCaptureLifecycle.js";
import { ProcessCaptureError } from "../process/capture/ProcessCaptureError.js";
import { toolContract } from "../contracts/toolContracts.js";
import type { AnalysisSnapshot } from "../domain/analysisSnapshot.js";
import { binaryTargetSummary } from "../application/binary/BinaryTargetSummary.js";
import { UnknownRegistryError } from "../domain/unknownRegistryError.js";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
@@ -177,7 +178,17 @@ const registerOpenLifecycleTool = (
openContract.name,
toolRegistrationOptions(openContract),
async (input, context) => {
const snapshotPath = input.snapshot_path;
let snapshot: AnalysisSnapshot | undefined;
if (input.snapshot_path !== undefined) {
const loaded = await readAnalysisSnapshot(
input.snapshot_path,
context.mcpReq.signal,
artifactResources,
);
if (!loaded.ok)
return server.delivery.toCallToolResult(loaded, openContract);
snapshot = loaded.value;
}
const opened = await logToolExecution(logger, openContract.name, () =>
session.open(input.path, {
signal: context.mcpReq.signal,
@@ -191,18 +202,7 @@ const registerOpenLifecycleTool = (
...(input.provider_id === undefined
? {}
: { providerId: input.provider_id }),
...(snapshotPath === undefined
? {}
: {
// Read inside the open's turn: calls issued after it must not
// overtake a file read that precedes its place in the order.
loadSnapshot: () =>
readAnalysisSnapshot(
snapshotPath,
context.mcpReq.signal,
artifactResources,
),
}),
...(snapshot === undefined ? {} : { snapshot }),
}),
);
return opened.ok
@@ -348,63 +348,6 @@ describe("CdpElectronProvider malformed source replies", () => {
);
});
describe("CdpElectronProvider malformed inventory replies", () => {
const inspectWith = async (
method: string,
reply: (fileUrl: string) => Readonly<Record<string, unknown>>,
) => {
const root = await electronFixture();
const fileUrl = pathToFileURL(join(root, "index.html")).href;
const browser = await startFakeCdpBrowser({
electronFileUrl: fileUrl,
commandResult: (command) =>
command.method === method ? reply(fileUrl) : undefined,
});
browsers.push(browser);
const result = await new CdpElectronProvider().inspectPage(
inspectElectronPageInputSchema.parse({
cdp_endpoint: browser.endpoint,
target_id: "electron-page",
observation_ms: 0,
}),
);
if (!result.ok) throw result.error;
return result.value;
};
it.each([
["dom", "DOMSnapshot.captureSnapshot", () => ({ strings: [] })],
[
"dom",
"DOMSnapshot.captureSnapshot",
(fileUrl: string) => ({
strings: [fileUrl],
documents: [{ documentURL: 0, nodes: { nodeType: null } }],
}),
],
[
"resources",
"Page.getResourceTree",
(fileUrl: string) => ({
frameTree: { frame: { id: "main", url: fileUrl }, resources: null },
}),
],
["workers", "Target.getTargets", () => ({ targetInfos: null })],
] as const)(
"keeps a malformed %s inventory incomplete (%s)",
async (section, method, reply) => {
const inspected = await inspectWith(method, reply);
expect(inspected.completeness.status).not.toBe("complete_within_window");
expect(inspected.completeness.excluded).toContainEqual(
expect.objectContaining({
section,
reason: "invalid_protocol_value",
}),
);
},
);
});
describe("CdpElectronProvider script source capture", () => {
it("retains an explicitly empty Electron producer script", async () => {
const root = await electronFixture();
@@ -1,277 +0,0 @@
import { createServer, type Socket } from "node:net";
import { afterEach, beforeEach, expect, it, vi } from "vitest";
import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js";
import { CdpConnection } from "../../../src/browser/CdpConnection.js";
import { projectAnalysisError } from "../../../src/domain/analysisErrorProjection.js";
import { inspectWebPageInputSchema } from "../../../src/domain/browserObservation.js";
import { WEB_RUNTIME_LIMITS } from "../../../src/domain/webRuntime.js";
import { V8InspectorProvider } from "../../../src/inspector/V8InspectorProvider.js";
import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js";
import { startFakeV8Inspector } from "../../fixtures/inspector/fakeV8Inspector.js";
// Keep real socket scheduling; expire only a timer whose owning phase is ready.
beforeEach(() => {
vi.spyOn(globalThis, "setTimeout");
vi.spyOn(globalThis, "clearTimeout");
});
afterEach(() => vi.restoreAllMocks());
const activeTimers = (milliseconds: number) => {
const scheduled = vi.mocked(globalThis.setTimeout);
const cleared = vi.mocked(globalThis.clearTimeout);
return scheduled.mock.calls.flatMap(([callback, delay], index) => {
const result = scheduled.mock.results[index];
if (
delay !== milliseconds ||
result?.type !== "return" ||
cleared.mock.calls.some(([timer]) => timer === result.value)
)
return [];
return [{ callback, timer: result.value }];
});
};
const expireTimer = (milliseconds: number): void => {
const timers = activeTimers(milliseconds);
expect(timers).toHaveLength(1);
const timer = timers[0];
if (timer === undefined) throw new Error("Expected one active timer");
try {
timer.callback();
if (milliseconds === WEB_RUNTIME_LIMITS.commandTimeoutMs)
expect(globalThis.clearTimeout).toHaveBeenCalledWith(timer.timer);
} finally {
// The observation callback resolves naturally without cancelling its timer.
clearTimeout(timer.timer);
}
};
it("bounds a stalled WebSocket handshake and releases its socket", async () => {
const sockets = new Set<Socket>();
const server = createServer((socket) => {
sockets.add(socket);
socket.on("close", () => sockets.delete(socket));
socket.resume();
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (address === null || typeof address === "string")
throw new Error("Expected a TCP address");
try {
const rejection = expect(
CdpConnection.connect(
`ws://127.0.0.1:${String(address.port)}/target`,
"inspect_web_page",
AbortSignal.timeout(10_000),
),
).rejects.toMatchObject({ reason: "timeout" });
await vi.waitFor(() => expect(sockets.size).toBe(1), { timeout: 5_000 });
expireTimer(WEB_RUNTIME_LIMITS.commandTimeoutMs);
await rejection;
await vi.waitFor(() => expect(sockets.size).toBe(0));
} finally {
for (const socket of sockets) socket.destroy();
await new Promise<void>((resolve) => server.close(() => resolve()));
}
});
it.each(["Target.attachToTarget", "Page.enable", "Runtime.enable"])(
"times out %s after HTTP discovery and closes the target session",
async (hangOnMethod) => {
const browser = await startFakeCdpBrowser({ hangOnMethod });
try {
const pending = new CdpBrowserProvider().inspectPage(
inspectWebPageInputSchema.parse({
cdp_endpoint: browser.endpoint,
target_id: "allowed-page",
observation_ms: 0,
}),
{ signal: AbortSignal.timeout(10_000) },
);
await vi.waitFor(
() =>
expect(browser.commands.map(({ method }) => method)).toContain(
hangOnMethod,
),
{ timeout: 5_000 },
);
expireTimer(WEB_RUNTIME_LIMITS.commandTimeoutMs);
const result = await pending;
if (result.ok) throw new Error("Expected a target command timeout");
expect(projectAnalysisError(result.error)).toMatchObject({
code: "provider_timeout",
category: "timeout",
});
expect(browser.commands.map(({ method }) => method)).toContain(
hangOnMethod,
);
await vi.waitFor(() => expect(browser.activeConnectionCount()).toBe(0));
} finally {
await browser.close();
}
},
);
it("ignores a late timed-out reply and keeps following commands correlated", async () => {
const browser = await startFakeCdpBrowser({ hangOnMethod: "Page.enable" });
const connection = await CdpConnection.connect(
browser.browserWebSocketUrl,
"inspect_web_page",
);
try {
const rejection = expect(
connection.send(
"Page.enable",
{},
undefined,
AbortSignal.timeout(10_000),
),
).rejects.toMatchObject({ reason: "timeout" });
await vi.waitFor(() => expect(browser.commands).toHaveLength(1), {
timeout: 5_000,
});
expireTimer(WEB_RUNTIME_LIMITS.commandTimeoutMs);
await rejection;
const command = browser.commands[0];
if (command === undefined)
throw new Error("Expected the timed-out command");
browser.emitRawMessage(JSON.stringify({ id: command.id, result: {} }));
await expect(connection.send("Runtime.enable")).resolves.toEqual({});
} finally {
await connection.close();
await browser.close();
}
});
it.each(["success", "rejection", "abort", "disconnect"])(
"releases handshake and command timers on %s",
async (outcome) => {
const cleared = vi.mocked(globalThis.clearTimeout);
const browser = await startFakeCdpBrowser({
...(outcome === "abort" || outcome === "success"
? { hangOnMethod: "Runtime.enable" }
: {}),
...(outcome === "disconnect" ? { closeOnMethod: "Runtime.enable" } : {}),
});
const controller = new AbortController();
const removeAbort = vi.spyOn(controller.signal, "removeEventListener");
const connection = await CdpConnection.connect(
browser.browserWebSocketUrl,
"inspect_web_page",
controller.signal,
);
try {
const pending = connection.send(
outcome === "rejection" ? "Fixture.unmodeledMethod" : "Runtime.enable",
{},
undefined,
controller.signal,
);
const assertion =
outcome === "success"
? expect(pending).resolves.toEqual({})
: expect(pending).rejects.toMatchObject({
_tag:
outcome === "abort"
? "AnalysisCancelledError"
: "BrowserObservationError",
});
if (outcome === "success") {
await vi.waitFor(() => expect(browser.commands).toHaveLength(1), {
timeout: 5_000,
});
// A healthy reply may take longer than the former 100 ms test budget.
await new Promise<void>((resolve) => setTimeout(resolve, 250));
const command = browser.commands[0];
if (command === undefined)
throw new Error("Expected a pending command");
browser.emitRawMessage(JSON.stringify({ id: command.id, result: {} }));
}
if (outcome === "abort") controller.abort();
await assertion;
const scheduled = vi.mocked(globalThis.setTimeout);
const deadlines = scheduled.mock.calls.flatMap((call, index) =>
call[1] === WEB_RUNTIME_LIMITS.commandTimeoutMs
? [scheduled.mock.results[index]?.value]
: [],
);
expect(deadlines).toHaveLength(2);
for (const timer of deadlines)
expect(cleared).toHaveBeenCalledWith(timer);
expect(removeAbort).toHaveBeenCalledTimes(2);
} finally {
await connection.close();
await browser.close();
}
},
);
it.each(["Runtime.enable", "Debugger.enable"])(
"times out Inspector %s and preserves observations received before expiry",
async (hangOnMethod) => {
const inspector = await startFakeV8Inspector({
targetUrl: "https://fixture.example/app.js",
hangOnMethod,
});
try {
const pending = new V8InspectorProvider().observe(
{
inspector_endpoint: inspector.endpoint,
target_id: inspector.targetId,
observation_ms: 0,
},
{ signal: AbortSignal.timeout(10_000) },
);
await vi.waitFor(
() =>
expect(inspector.commands.map(({ method }) => method)).toContain(
hangOnMethod,
),
{ timeout: 5_000 },
);
expireTimer(WEB_RUNTIME_LIMITS.commandTimeoutMs);
const result = await pending;
if (result.ok) throw new Error("Expected an Inspector command timeout");
expect(projectAnalysisError(result.error)).toMatchObject({
code: "provider_timeout",
category: "timeout",
});
expect(result.error.partialObservation).toMatchObject({
capture: {
events_observed: hangOnMethod === "Debugger.enable" ? 1 : 0,
},
});
} finally {
await inspector.close();
}
},
);
it("keeps the chosen observation window independent of command deadlines", async () => {
const browser = await startFakeCdpBrowser();
const observationMs = WEB_RUNTIME_LIMITS.commandTimeoutMs + 250;
try {
const pending = new CdpBrowserProvider().inspectPage(
inspectWebPageInputSchema.parse({
cdp_endpoint: browser.endpoint,
target_id: "allowed-page",
observation_ms: observationMs,
}),
);
await vi.waitFor(
() => expect(activeTimers(observationMs)).toHaveLength(1),
{
timeout: 5_000,
},
);
expect(activeTimers(WEB_RUNTIME_LIMITS.commandTimeoutMs)).toHaveLength(0);
expireTimer(observationMs);
const result = await pending;
expect(result.ok).toBe(true);
} finally {
await browser.close();
}
});
@@ -1,52 +0,0 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect } from "vitest";
import { thinMach } from "../../../src/domain/binaryTarget.fixture.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { cliTest } from "../../support/cli/cliFixture.js";
// An unknown standing provider stops native analysis before any engine runs.
const environment = {
REA_LOG_LEVEL: "silent",
REA_ANALYSIS_PROVIDER: "missing-provider",
} as const;
const fixtureApp = async () => {
const directory = await createTestTempDirectory("rea-analyze-app-path-");
const contents = join(directory, "Sample.app", "Contents");
await mkdir(join(contents, "MacOS"), { recursive: true });
await writeFile(
join(contents, "MacOS/Sample"),
thinMach(0xfeedfacf, 0x100000c),
);
await writeFile(
join(contents, "Info.plist"),
"<plist><dict><key>CFBundleExecutable</key><string>Sample</string></dict></plist>",
);
return directory;
};
describe("analyze app bundle path spellings", () => {
cliTest.for(["Sample.app", "Sample.app/", "Sample.app/.", "Sample.app//"])(
"routes %s to native provider selection",
async (path, { cli }) => {
const cwd = await fixtureApp();
const result = await cli.run({
arguments: ["analyze", path, "--json"],
cwd,
environment,
});
expect(result.exitCode).toBe(1);
expect(result.json).toMatchObject({
code: "capability_unavailable",
details: {
operation: "open_binary",
selection_reason: "unknown_provider",
requested_provider_id: "missing-provider",
},
});
},
);
});
@@ -163,33 +163,31 @@ for (const group of groups) {
environment,
timeoutMs: 20_000,
});
const planned = await run([
"setup",
...clientArguments,
"--dry-run",
"--json",
]);
expect(planned.exitCode, planned.stderr).toBe(0);
expect(planned.json).toMatchObject({
status: "planned",
plannedActions: expect.arrayContaining([
...profiles.map(({ client, config }) =>
for (const { client, config, skillDirectory } of profiles) {
const planned = await run([
"setup",
"--client",
client,
"--dry-run",
"--json",
]);
expect(planned.exitCode, client).toBe(0);
expect(planned.json, client).toMatchObject({
status: "planned",
plannedActions: expect.arrayContaining([
expect.objectContaining({
id: `configure_client:${client}`,
target: config,
}),
),
...[
...new Set(profiles.map(({ skillDirectory }) => skillDirectory)),
].map((target) => expect.objectContaining({ target })),
]),
});
for (const { config, skillDirectory } of profiles) {
await expect(access(config)).rejects.toMatchObject({ code: "ENOENT" });
expect.objectContaining({ target: skillDirectory }),
]),
});
await expect(access(skillDirectory)).rejects.toMatchObject({
code: "ENOENT",
});
}
for (const { config } of profiles)
await expect(access(config)).rejects.toMatchObject({ code: "ENOENT" });
const setup = await run(["setup", ...clientArguments, "--yes", "--json"]);
expect(setup.exitCode).toBe(0);
expect(setup.json).toMatchObject({ status: "ready" });
@@ -19,7 +19,6 @@ describe.skipIf(process.platform === "win32")(
"rea-reference-directory-imports-",
);
const root = join(workspace, "source");
const outside = join(workspace, "outside", "index.js");
const imports = [
["slash.cjs", "./lib/", "lib/index.js"],
["dot.cjs", "./lib/.", "lib/index.js"],
@@ -31,17 +30,12 @@ describe.skipIf(process.platform === "win32")(
["fallback.cjs", "./fallback", "fallback/index.js"],
["legacy-order.cjs", "./dep", "dep/index.js"],
["bare-index.cjs", "./bare", "bare.js"],
["package-main.cjs", "./pkg", "pkg/actual.js"],
["package-main-fallback.cjs", "./stale", "stale/index.js"],
["package-main-slash.cjs", "./slashed", "slashed/lib.js"],
] as const;
const unresolved = [
["missing.cjs", "./missing/", "missing/"],
["escape.cjs", "../outside/", "../outside/"],
["main.ts", "./folder.js/.", "folder.js"],
["only-bare.cjs", "./only/", "only/"],
["invalid-manifest.cjs", "./broken", "broken"],
["absolute-main.cjs", "./absolute", outside],
] as const;
const files = new Map<string, string>([
["index.js", 'module.exports = "root index";\n'],
@@ -59,22 +53,6 @@ describe.skipIf(process.platform === "win32")(
["bare.js", 'module.exports = "file before directory";\n'],
["bare/index", 'module.exports = "extensionless index";\n'],
["only/index", 'module.exports = "extensionless index";\n'],
// A package main outranks the index; a missing main falls back to it.
["pkg/package.json", '{"main":"actual.js"}\n'],
["pkg/actual.js", 'module.exports = "declared main";\n'],
["pkg/index.js", 'module.exports = "ignored index";\n'],
["stale/package.json", '{"main":"gone.js"}\n'],
["stale/index.js", 'module.exports = "legacy fallback";\n'],
["broken/package.json", '{"main":\n'],
["broken/index.js", 'module.exports = "unknown selection";\n'],
// Node resolves main with path.resolve: no trailing-slash directory
// intent, and an absolute main leaves the imported root.
["slashed/package.json", '{"main":"lib/"}\n'],
["slashed/lib.js", 'module.exports = "file main";\n'],
["slashed/lib/index.js", 'module.exports = "not selected";\n'],
["slashed/index.js", 'module.exports = "not selected";\n'],
["absolute/package.json", `${JSON.stringify({ main: outside })}\n`],
["absolute/index.js", 'module.exports = "not selected";\n'],
...[...imports, ...unresolved].map(
([path, specifier]) =>
[path, `require(${JSON.stringify(specifier)});\n`] as const,
@@ -87,6 +65,7 @@ describe.skipIf(process.platform === "win32")(
await writeFile(target, contents);
}),
);
const outside = join(workspace, "outside", "index.js");
await mkdir(dirname(outside), { recursive: true });
await writeFile(outside, 'module.exports = "outside import root";\n');
@@ -104,15 +83,9 @@ describe.skipIf(process.platform === "win32")(
expect(() =>
createRequire(join(root, "only-bare.cjs")).resolve("./only/"),
).toThrow();
expect(() =>
createRequire(join(root, "invalid-manifest.cjs")).resolve("./broken"),
).toThrow();
expect(
createRequire(join(root, "escape.cjs")).resolve("../outside/"),
).toBe(outside);
expect(
createRequire(join(root, "absolute-main.cjs")).resolve("./absolute"),
).toBe(outside);
const result = await cli.run({
arguments: ["import-reference-source", root, "--json"],
@@ -1,7 +1,6 @@
import { execFile } from "node:child_process";
import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { solidityMetadataSchema } from "../../../src/domain/evm/solidityMetadata.js";
@@ -39,12 +38,8 @@ it.each([
[
"--max-old-space-size=256",
"--disable-wasm-trap-handler",
fileURLToPath(
new URL(
"../../../dist/evm/EvmoleInterfaceWorker.js",
import.meta.url,
),
),
new URL("../../../dist/evm/EvmoleInterfaceWorker.js", import.meta.url)
.pathname,
request,
],
{ timeout: 30_000, maxBuffer: 1024 * 1024 },
@@ -416,10 +416,7 @@ it.each(["", "not-a-commit"])(
},
);
it("selects only a trusted successful ancestor from recent CI push runs", async ({
skip,
}) => {
skip(process.platform === "win32", "the fake gh is a POSIX shell script");
it("selects only a trusted successful ancestor from recent CI push runs", async () => {
const { directory, base } = await fixture();
const head = await change(directory, ["src/evm/client.ts"]);
const bin = join(directory, "fake-bin");
@@ -452,11 +449,7 @@ it("selects only a trusted successful ancestor from recent CI push runs", async
const output = join(directory, "baseline-output");
const result = await execFileAsync(
process.execPath,
[
fileURLToPath(
new URL("../../../scripts/ci/baseline.mjs", import.meta.url),
),
],
[new URL("../../../scripts/ci/baseline.mjs", import.meta.url).pathname],
{
cwd: directory,
env: {
@@ -474,8 +467,7 @@ it("selects only a trusted successful ancestor from recent CI push runs", async
expect(await readFile(output, "utf8")).toBe(`base=${base}\nreason=\n`);
});
it("returns an empty baseline when GitHub lookup fails", async ({ skip }) => {
skip(process.platform === "win32", "the fake gh is a POSIX shell script");
it("returns an empty baseline when GitHub lookup fails", async () => {
const { directory, base } = await fixture();
const head = await change(directory, ["src/evm/client.ts"]);
const bin = join(directory, "fake-bin");
@@ -485,11 +477,7 @@ it("returns an empty baseline when GitHub lookup fails", async ({ skip }) => {
await chmod(gh, 0o755);
const result = await execFileAsync(
process.execPath,
[
fileURLToPath(
new URL("../../../scripts/ci/baseline.mjs", import.meta.url),
),
],
[new URL("../../../scripts/ci/baseline.mjs", import.meta.url).pathname],
{
cwd: directory,
env: {
@@ -326,72 +326,31 @@ describe.skipIf(process.platform === "win32")(
},
);
/**
* A PSPSDK-shaped ELF32 whose PT_MIPS_ABIFLAGS record matches the PSP profile.
* Optionally declare a section-header table beyond the captured file.
*/
const pspElfBytes = ({ sectionHeadersBeyondEof = false } = {}): Buffer => {
const abiOffset = 8192;
const bytes = Buffer.alloc(abiOffset + 24);
bytes.set([0x7f, 0x45, 0x4c, 0x46, 1, 1, 1]);
bytes.writeUInt16LE(2, 16);
bytes.writeUInt16LE(8, 18);
bytes.writeUInt32LE(1, 20);
bytes.writeUInt32LE(52, 28);
bytes.writeUInt32LE(0x10a23001, 36);
bytes.writeUInt16LE(52, 40);
bytes.writeUInt16LE(32, 42);
bytes.writeUInt16LE(1, 44);
if (sectionHeadersBeyondEof) {
bytes.writeUInt32LE(bytes.length + 64, 32); // e_shoff
bytes.writeUInt16LE(40, 46); // e_shentsize
bytes.writeUInt16LE(2, 48); // e_shnum
}
bytes.writeUInt32LE(0x70000003, 52);
bytes.writeUInt32LE(abiOffset, 56);
bytes.writeUInt32LE(24, 68);
bytes.writeUInt32LE(24, 72);
// GNU readelf -x .MIPS.abiflags, not bytes generated by REA.
bytes.set(
Buffer.from("000002000101000200000000000000000100000000000000", "hex"),
abiOffset,
);
return bytes;
};
describe.skipIf(process.platform === "win32")(
"PSP inspected ABI declaration boundary",
() => {
it("refuses a PSP image whose section headers lie beyond the captured file", async () => {
const { directory, provider } = await installation();
const path = join(directory, "psp-truncated-sections.elf");
await writeFile(path, pspElfBytes({ sectionHeadersBeyondEof: true }));
const parsed = await parseBinaryTarget(path);
if (!parsed.ok) throw parsed.error;
if (
parsed.value.kind !== "executable" ||
parsed.value.architecture !== "mips"
)
throw new Error("PSP ELF was not recognized as MIPS");
expect(parsed.value.mips.abiFlags).toEqual(pspAbiFlags);
expect(parsed.value.mips.limitations).toContainEqual(
expect.stringContaining("ABI declarations in unavailable headers"),
);
expect(provider.inspectTargetSupport(parsed.value).status).toBe(
"unsupported",
);
const result = await provider.resolveAnalysisProfile(parsed.value);
expect(result.ok).toBe(false);
if (result.ok) throw new Error("Incomplete ABI inspection was admitted");
expect(result.error._tag).toBe("AnalysisUnsupportedTargetError");
expect(result.error.message).toContain(
"MIPS ABI declaration inspection is incomplete",
);
});
it("retains independently inspected PSP ABI bytes through the real target reader", async () => {
const { directory, provider } = await installation();
const bytes = pspElfBytes();
const abiOffset = 8192;
const bytes = Buffer.alloc(abiOffset + 24);
bytes.set([0x7f, 0x45, 0x4c, 0x46, 1, 1, 1]);
bytes.writeUInt16LE(2, 16);
bytes.writeUInt16LE(8, 18);
bytes.writeUInt32LE(1, 20);
bytes.writeUInt32LE(52, 28);
bytes.writeUInt32LE(0x10a23001, 36);
bytes.writeUInt16LE(52, 40);
bytes.writeUInt16LE(32, 42);
bytes.writeUInt16LE(1, 44);
bytes.writeUInt32LE(0x70000003, 52);
bytes.writeUInt32LE(abiOffset, 56);
bytes.writeUInt32LE(24, 68);
bytes.writeUInt32LE(24, 72);
// GNU readelf -x .MIPS.abiflags, not bytes generated by REA.
bytes.set(
Buffer.from("000002000101000200000000000000000100000000000000", "hex"),
abiOffset,
);
const path = join(directory, "psp-with-abi.elf");
await writeFile(path, bytes);
const parsed = await parseBinaryTarget(path);
@@ -54,8 +54,6 @@ const graphIdentities = async (source: string) => {
// source locations and every derived identity stay comparable.
it.each([
"fetch('https://api.example.com/v1/users');",
"client['get']('/api/items');",
"navigator['sendBeacon']('/api/events', payload);",
"localStorage.setItem('token', 1);",
"new Worker(new URL('./worker.js', import.meta.url));",
"const dep = require('./dep.js'); dep();",
@@ -1,7 +1,6 @@
import { execFile } from "node:child_process";
import { readFile, writeFile } from "node:fs/promises";
import { delimiter, join } from "node:path";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { parse } from "yaml";
@@ -349,7 +348,7 @@ it("binds npm and MCP publication to the same immutable release SHA", async () =
import.meta.url,
);
const result = await execFileAsync(process.execPath, [
fileURLToPath(helper),
helper.pathname,
version,
]);
expect(result.stdout).toBe(tag);
-39
View File
@@ -481,45 +481,6 @@ describe("target-free MCP workflow", () => {
}, 10_000);
});
describe("MCP snapshot open ordering", () => {
it("keeps a snapshot open ahead of later reads and closes on the same connection", async () => {
directory = await createTestTempDirectory("rea-mcp-snapshot-open-order-");
const { mcp, first, second } = await createSessionMcpHarness(
directory,
provider,
resources,
);
const snapshotPath = join(directory, "analysis.json");
await mcp.callTool({ name: "open_binary", arguments: { path: second } });
expect(
(
await mcp.callTool({
name: "close_binary",
arguments: { snapshot_path: snapshotPath },
})
).isError,
).not.toBe(true);
await mcp.callTool({ name: "open_binary", arguments: { path: first } });
// Dispatched without awaiting, like a client issuing one turn's calls.
const opening = mcp.callTool({
name: "open_binary",
arguments: { path: second, snapshot_path: snapshotPath },
});
const reading = mcp.callTool({ name: "current_document", arguments: {} });
const closing = mcp.callTool({ name: "close_binary", arguments: {} });
expect((await opening).isError, JSON.stringify(await opening)).not.toBe(
true,
);
expect(text(await reading)).toContain("second.hop");
expect((await closing).isError).not.toBe(true);
expect(
text(await mcp.callTool({ name: "binary_session", arguments: {} })),
).toContain('"open":false');
}, 10_000);
});
describe("session filesystem path boundaries over MCP", () => {
it("rejects relative snapshot and export paths with an absolute-path error", async () => {
directory = await createTestTempDirectory("rea-mcp-path-boundary-");
@@ -1,5 +1,4 @@
import { execFile } from "node:child_process";
import { fileURLToPath } from "node:url";
import { promisify } from "node:util";
import { describe, expect, it } from "vitest";
@@ -134,7 +133,7 @@ describe("Hopper API facade", () => {
it("imports without Hopper globals and gates exhaustive work during analysis", async () => {
const { stdout } = await execute(
"python3",
[fileURLToPath(probePath), fileURLToPath(bridgePath)],
[probePath.pathname, bridgePath.pathname],
{
encoding: "utf8",
timeout: 3_000,
-125
View File
@@ -1,125 +0,0 @@
import { readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
/** One function the fake exports; `error` renders Kuna's failed-function header. */
export interface FakeKunaFunction {
readonly name: string;
readonly addr: string;
readonly error?: string;
}
/** Scripted behavior of the fake `kuna` executable for one test. */
export interface FakeKunaBehavior {
/** Text printed by `--version`; `exit` fails the probe instead. */
readonly version?:
| { readonly stdout: string }
| { readonly exit: number }
| { readonly ignoreTerm: true };
readonly inventory?: "ok" | "not-json" | "hang";
readonly project?:
| "ok"
| "exit-failure"
| "hang"
| "miscounted"
| "silent"
| "block-manifest";
readonly functions?: readonly FakeKunaFunction[];
/** Extra `.asm` bytes, for capacity tests. */
readonly asmPadding?: number;
/** Milliseconds the export takes before writing its files. */
readonly delayMs?: number;
}
/**
* The fake reads its behavior from `<script>.json` and appends one line per
* invocation to `<script>.calls`. REA gives Kuna children no PATH, so the
* shebang names this Node binary directly. Output formats follow Kuna 1.833:
* `functions --json` and the `// Function:` headers of `render_c`.
*/
const SCRIPT = String.raw`
const fs = require("node:fs");
const path = require("node:path");
const behavior = JSON.parse(fs.readFileSync(__filename + ".json", "utf8"));
fs.appendFileSync(__filename + ".calls", JSON.stringify({ pid: process.pid, argv: process.argv.slice(2), home: process.env.HOME, xdg: process.env.XDG_CACHE_HOME, path: process.env.PATH ?? null }) + "\n");
const args = process.argv.slice(2);
const functions = behavior.functions ?? [{ name: "main", addr: "0x1000" }, { name: "helper", addr: "0x1040" }];
const hang = () => setInterval(() => {}, 1000);
if (args[0] === "--version") {
const version = behavior.version ?? { stdout: "kuna 1.833\n" };
if ("exit" in version) { process.stderr.write("version probe failed\n"); process.exit(version.exit); }
if ("ignoreTerm" in version) { process.on("SIGTERM", () => {}); hang(); }
else process.stdout.write(version.stdout);
} else if (args[0] === "functions") {
const mode = behavior.inventory ?? "ok";
if (mode === "hang") hang();
else if (mode === "not-json") process.stdout.write("functions: 2\n");
else {
const listed = [...functions, { name: "puts", addr: "0x2000" }].map((fn) => ({ name: fn.name, address: parseInt(fn.addr, 16), address_hex: fn.addr, aliases: [], object_location: null, size: fn.name === "puts" ? 0 : 16 }));
process.stdout.write(JSON.stringify({ binary: args[1], count: listed.length, total: listed.length, error: null, functions: listed }, null, 2) + "\n");
}
} else if (args[0] === "decompile-project") {
const input = args[1];
const output = args[args.indexOf("-o") + 1];
const stem = path.basename(input);
const mode = behavior.project ?? "ok";
if (mode === "hang") hang();
else if (mode === "exit-failure") { process.stderr.write("error: not in recognized object file format\n"); process.exit(1); }
else setTimeout(() => {
fs.mkdirSync(output, { recursive: true });
let c = '#include "' + stem + '.h"\n\n';
functions.forEach((fn, index) => {
c += fn.error === undefined
? "// Function: " + fn.name + " @ " + fn.addr + "\nint " + fn.name + "(void)\n{\n return " + index + ";\n}\n\n"
: "// Function: " + fn.name + " @ " + fn.addr + " (error: " + fn.error + ")\n\n";
});
fs.writeFileSync(path.join(output, stem + ".c"), c);
fs.writeFileSync(path.join(output, stem + ".h"), "#pragma once\n");
fs.writeFileSync(path.join(output, stem + ".asm"), "; asm\n" + "x".repeat(behavior.asmPadding ?? 0));
fs.writeFileSync(path.join(output, "README.md"), "# " + stem + "\n");
// Occupy the manifest path so REA's publication write fails.
if (mode === "block-manifest") fs.mkdirSync(path.join(output, "..", "manifest.json"));
const failed = functions.filter((fn) => fn.error !== undefined).length;
const reported = mode === "miscounted" ? functions.length + 1 : functions.length - failed;
if (mode !== "silent") process.stdout.write("wrote " + output + ": " + stem + ".c (" + c.length + " bytes); functions: " + reported + " ok, " + failed + " failed\n");
}, behavior.delayMs ?? 0);
} else {
process.stderr.write("unexpected command\n");
process.exit(2);
}
`;
/** Write a fake `kuna` into a directory and return its path. */
export const writeFakeKuna = async (
directory: string,
behavior: FakeKunaBehavior = {},
): Promise<string> => {
const path = join(directory, "kuna");
await writeFile(path, `#!${process.execPath}\n${SCRIPT}`, { mode: 0o755 });
await writeFile(`${path}.json`, JSON.stringify(behavior));
return path;
};
/** Replace the scripted behavior of an existing fake. */
export const setFakeKunaBehavior = (
path: string,
behavior: FakeKunaBehavior,
): Promise<void> => writeFile(`${path}.json`, JSON.stringify(behavior));
/** Invocations the fake recorded, oldest first. */
export const fakeKunaCalls = async (
path: string,
): Promise<
readonly {
readonly pid: number;
readonly argv: readonly string[];
readonly home: string;
readonly xdg: string;
readonly path: string | null;
}[]
> => {
const text = await readFile(`${path}.calls`, "utf8").catch(() => "");
return text
.split("\n")
.filter((line) => line.length > 0)
.map((line) => JSON.parse(line) as never);
};
@@ -1,136 +0,0 @@
import { mkdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { expect, it as test } from "vitest";
import { inspectKunaProvider } from "../../../../src/kuna/KunaDoctor.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { writeFakeKuna } from "./fakeKuna.js";
const it = test.skipIf(process.platform === "win32");
it("reports an unconfigured Kuna as a missing optional engine", async () => {
expect(await inspectKunaProvider({})).toMatchObject({
id: "kuna",
configured: false,
available: false,
providerVersion: null,
registrationEnvironment: {},
checks: [
{
name: "installation",
ok: false,
code: "not_configured",
classification: "missing_analysis_engine",
},
],
});
});
it("publishes the caller's explicit settings once the installation validates", async () => {
const root = await createTestTempDirectory("rea-kuna-doctor-");
const kuna = await writeFakeKuna(root);
const cache = join(root, "cache");
const inspection = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: cache,
REA_KUNA_JOBS: "4",
UNRELATED: "not published",
});
expect(inspection).toMatchObject({
configured: true,
available: true,
providerVersion: "1.833",
registrationEnvironment: {
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: cache,
REA_KUNA_JOBS: "4",
},
checks: [
{ name: "installation", ok: true, details: { version: "1.833" } },
{
name: "cache",
ok: true,
detail: `REA creates the Kuna project cache at ${cache} on first use.`,
},
],
});
expect(Object.keys(inspection.registrationEnvironment)).toHaveLength(3);
});
it("flags an unverified release without failing the installation", async () => {
const root = await createTestTempDirectory("rea-kuna-doctor-version-");
const kuna = await writeFakeKuna(root, {
version: { stdout: "kuna 1.950\n" },
});
const inspection = await inspectKunaProvider({ REA_KUNA_COMMAND: kuna });
expect(inspection.checks[0]).toMatchObject({
ok: true,
detail: expect.stringContaining(
"this release's project layout is unverified",
),
});
});
it("reports a cache root REA cannot use without creating anything", async () => {
const root = await createTestTempDirectory("rea-kuna-doctor-cache-");
const kuna = await writeFakeKuna(root);
const occupied = join(root, "occupied");
await writeFile(occupied, "not a directory");
const existing = join(root, "existing");
await mkdir(existing);
const blocked = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: occupied,
});
const ready = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: existing,
});
expect(blocked.checks[1]).toMatchObject({
name: "cache",
ok: false,
code: "open_options_invalid",
});
expect(ready.checks[1]).toMatchObject({
name: "cache",
ok: true,
detail: expect.stringContaining(`cached in ${existing}`),
});
});
it("reports a cache root REA cannot create under its nearest existing parent", async () => {
const root = await createTestTempDirectory("rea-kuna-doctor-create-");
const kuna = await writeFakeKuna(root);
const readOnly = join(root, "read-only");
await mkdir(readOnly, { mode: 0o500 });
const occupied = join(root, "occupied");
await writeFile(occupied, "not a directory");
const underReadOnly = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(readOnly, "nested", "cache"),
});
const underFile = await inspectKunaProvider({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(occupied, "cache"),
});
expect(underReadOnly.checks[1]).toMatchObject({
name: "cache",
ok: false,
detail: expect.stringContaining(`${readOnly} is not writable`),
});
expect(underFile.checks[1]).toMatchObject({
name: "cache",
ok: false,
// stat reports ENOTDIR for a path beneath a regular file.
detail: expect.stringContaining("not a directory"),
});
});
@@ -1,212 +0,0 @@
import { createHash } from "node:crypto";
import { chmod, readFile, realpath, symlink } from "node:fs/promises";
import { join } from "node:path";
import { expect, it as test } from "vitest";
import { readKunaConfiguration } from "../../../../src/kuna/KunaConfiguration.js";
import { inspectKunaInstallation } from "../../../../src/kuna/KunaInstallation.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { fakeKunaCalls, writeFakeKuna } from "./fakeKuna.js";
const it = test.skipIf(process.platform === "win32");
it("identifies a caller-selected kuna by version and digest", async () => {
const root = await createTestTempDirectory("rea-kuna-install-");
const kuna = await writeFakeKuna(root);
const link = join(root, "current-kuna");
await symlink(kuna, link);
const result = await inspectKunaInstallation({
REA_KUNA_COMMAND: link,
REA_KUNA_CACHE_DIR: join(root, "cache"),
});
expect(result).toMatchObject({
ok: true,
value: {
configuration: { command: link, cacheRoot: join(root, "cache") },
executable: await realpath(kuna),
version: "1.833",
verified: true,
// A `#!` launcher reports its interpreter; ownership checks must not expect the path.
script: true,
sha256: createHash("sha256")
.update(await readFile(kuna))
.digest("hex"),
},
});
// The probe runs inside REA's cache home with no inherited PATH.
expect(await fakeKunaCalls(kuna)).toEqual([
{
pid: expect.any(Number),
argv: ["--version"],
home: join(root, "cache", "home"),
xdg: join(root, "cache", "home", ".cache"),
path: null,
},
]);
});
it("accepts another release but reports it as unverified", async () => {
const root = await createTestTempDirectory("rea-kuna-unverified-");
const kuna = await writeFakeKuna(root, {
version: { stdout: "kuna 1.900\n" },
});
const result = await inspectKunaInstallation({ REA_KUNA_COMMAND: kuna });
expect(result).toMatchObject({
ok: true,
value: { version: "1.900", verified: false },
});
});
it("refuses a version line it cannot recognize", async () => {
const root = await createTestTempDirectory("rea-kuna-garbled-");
const kuna = await writeFakeKuna(root, {
version: { stdout: "Kuna decompiler, built today\n" },
});
expect(
await inspectKunaInstallation({ REA_KUNA_COMMAND: kuna }),
).toMatchObject({
ok: false,
error: {
code: "version_unresolved",
diagnostics: { stdout: "Kuna decompiler, built today\n" },
},
});
});
it("reports a failed version probe with its exit status", async () => {
const root = await createTestTempDirectory("rea-kuna-probe-exit-");
const kuna = await writeFakeKuna(root, { version: { exit: 3 } });
expect(
await inspectKunaInstallation({ REA_KUNA_COMMAND: kuna }),
).toMatchObject({
ok: false,
error: {
code: "version_unresolved",
reason: expect.stringContaining("version probe failed"),
diagnostics: { exit_code: 3, killed: false },
},
});
});
it("reports a missing or non-executable command as executable_missing", async () => {
const root = await createTestTempDirectory("rea-kuna-missing-");
const kuna = await writeFakeKuna(root);
await chmod(kuna, 0o644);
for (const command of [kuna, join(root, "absent"), root])
expect(
await inspectKunaInstallation({ REA_KUNA_COMMAND: command }),
).toMatchObject({
ok: false,
error: { code: "executable_missing", diagnostics: { command } },
});
});
it("distinguishes an unset command from an invalid one", async () => {
expect(await inspectKunaInstallation({})).toMatchObject({
ok: false,
error: { code: "not_configured", diagnostics: { configured: false } },
});
expect(
await inspectKunaInstallation({ REA_KUNA_COMMAND: "bin/kuna" }),
).toMatchObject({
ok: false,
error: {
code: "open_options_invalid",
diagnostics: { configured: true, setting: "REA_KUNA_COMMAND" },
},
});
});
it("declares Windows hosts unsupported before touching the command", async () => {
expect(
await inspectKunaInstallation(
{ REA_KUNA_COMMAND: "/opt/kuna/kuna" },
{ platform: "win32" },
),
).toMatchObject({ ok: false, error: { code: "unsupported_host" } });
});
it("parses cache, timeout and job settings at the configuration boundary", () => {
const environment = {
REA_KUNA_COMMAND: "/opt/kuna/kuna",
HOME: "/home/analyst",
};
expect(readKunaConfiguration(environment, "linux")).toMatchObject({
ok: true,
value: {
cacheRoot: "/home/analyst/.cache/rea/kuna",
cacheMaxBytes: 4 * 1024 ** 3,
projectTimeoutMs: 1_800_000,
// Kuna's reference answer is a serial export; more jobs change results.
jobs: 1,
},
});
expect(
readKunaConfiguration(
{ ...environment, XDG_CACHE_HOME: "/var/cache/analyst" },
"linux",
),
).toMatchObject({ value: { cacheRoot: "/var/cache/analyst/rea/kuna" } });
expect(readKunaConfiguration(environment, "darwin")).toMatchObject({
value: { cacheRoot: "/home/analyst/Library/Caches/rea/kuna" },
});
expect(
readKunaConfiguration(
{
...environment,
REA_KUNA_CACHE_MAX_BYTES: "2097152",
REA_KUNA_PROJECT_TIMEOUT_MS: "60000",
REA_KUNA_JOBS: "3",
},
"linux",
),
).toMatchObject({
value: { cacheMaxBytes: 2097152, projectTimeoutMs: 60000, jobs: 3 },
});
for (const [setting, value] of [
["REA_KUNA_CACHE_DIR", "relative/cache"],
["REA_KUNA_CACHE_MAX_BYTES", "1024"],
["REA_KUNA_CACHE_MAX_BYTES", "4GiB"],
["REA_KUNA_PROJECT_TIMEOUT_MS", "0"],
["REA_KUNA_JOBS", "33"],
["REA_KUNA_JOBS", "0"],
] as const)
expect(
readKunaConfiguration({ ...environment, [setting]: value }, "linux"),
).toMatchObject({
ok: false,
error: { code: "open_options_invalid", setting },
});
});
it("stops a version probe that ignores SIGTERM instead of waiting on it", async () => {
const root = await createTestTempDirectory("rea-kuna-probe-term-");
const kuna = await writeFakeKuna(root, { version: { ignoreTerm: true } });
const started = Date.now();
const result = await inspectKunaInstallation(
{ REA_KUNA_COMMAND: kuna },
// Long enough for Node to start the fake and install its SIGTERM handler.
{ probeTimeoutMs: 1500 },
);
expect(result).toMatchObject({
ok: false,
error: {
code: "version_unresolved",
diagnostics: { killed: true, deadline_ms: 1500 },
},
});
// The deadline plus ExecFileOutput's bounded kill grace, never an open-ended wait.
expect(Date.now() - started).toBeLessThan(15_000);
const pid = (await fakeKunaCalls(kuna))[0]?.pid ?? 0;
expect(() => process.kill(pid, 0)).toThrow();
}, 20_000);
@@ -1,704 +0,0 @@
import { createHash } from "node:crypto";
import {
appendFile,
chmod,
mkdir,
readdir,
readFile,
rename,
rm,
utimes,
writeFile,
} from "node:fs/promises";
import { join } from "node:path";
import { expect, it as test } from "vitest";
import { inspectKunaInstallation } from "../../../../src/kuna/KunaInstallation.js";
import { cleanupOwnedProcessGroup } from "../../../../src/process/ProcessOwnership.js";
import { spawnOwnedProviderProcess } from "../../../../src/process/ProviderProcess.js";
import type { KunaInstallation } from "../../../../src/kuna/KunaInstallation.js";
import {
KunaProjectCache,
kunaProjectKey,
} from "../../../../src/kuna/KunaProjectCache.js";
import type { KunaExportStatus } from "../../../../src/kuna/KunaProjectExport.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import {
fakeKunaCalls,
setFakeKunaBehavior,
writeFakeKuna,
type FakeKunaBehavior,
} from "./fakeKuna.js";
const it = test.skipIf(process.platform === "win32");
interface Target {
readonly targetPath: string;
readonly targetSha256: string;
}
const fixture = async (
prefix: string,
behavior: FakeKunaBehavior = {},
options: {
readonly maxBytes?: number;
readonly environment?: Readonly<Record<string, string>>;
} = {},
) => {
const root = await createTestTempDirectory(prefix);
const kuna = await writeFakeKuna(root, behavior);
const installed = await inspectKunaInstallation({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
...options.environment,
});
if (!installed.ok) throw new Error(installed.error.reason);
return {
kuna,
installation: installed.value,
cacheRoot: installed.value.configuration.cacheRoot,
cache: newCache(installed.value, options.maxBytes),
target: async (name: string, content = name): Promise<Target> => {
const targetPath = join(root, name);
await writeFile(targetPath, content);
return {
targetPath,
targetSha256: createHash("sha256").update(content).digest("hex"),
};
},
};
};
const newCache = (installation: KunaInstallation, maxBytes = 64 * 1024 ** 2) =>
new KunaProjectCache(installation.configuration.cacheRoot, {
maxBytes,
});
const keyOf = (installation: KunaInstallation, target: Target) =>
kunaProjectKey({
targetSha256: target.targetSha256,
kunaSha256: installation.sha256,
kunaVersion: installation.version,
jobs: installation.configuration.jobs,
});
const callsOf = async (kuna: string, subcommand: string) =>
(await fakeKunaCalls(kuna)).filter(({ argv }) => argv[0] === subcommand);
const entries = (cacheRoot: string, area: string) =>
readdir(join(cacheRoot, area)).catch(() => []);
const processGone = (pid: number): boolean => {
try {
process.kill(pid, 0);
return false;
} catch {
return true;
}
};
it("publishes a verified export and serves later requests from the cache", async () => {
const { kuna, installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-cache-",
{
functions: [
{ name: "main", addr: "0x1000" },
{ name: "parse @ table", addr: "0x1040" },
{
name: "broken",
addr: "0x1080",
error: "Exceeded the instruction budget",
},
],
},
{ environment: { REA_KUNA_JOBS: "1" } },
);
const input = await target("program");
const built = await cache.build({ installation, ...input });
expect(built).toMatchObject({
ok: true,
value: {
key: keyOf(installation, input),
manifest: {
layout: 1,
inputs: {
target_sha256: input.targetSha256,
kuna_sha256: installation.sha256,
kuna_version: "1.833",
jobs: 1,
},
summary: { decompiled: 2, failed: 1 },
},
functions: [
{ name: "main", address: "0x1000", error: null },
// A name may itself contain " @ "; the address is the last field.
{ name: "parse @ table", address: "0x1040", error: null },
{
name: "broken",
address: "0x1080",
error: "Exceeded the instruction budget",
},
],
// The inventory also names imports Kuna does not decompile.
inventory: [
{ name: "main", address: "0x1000", size: 16 },
{ name: "parse @ table", address: "0x1040", size: 16 },
{ name: "broken", address: "0x1080", size: 16 },
{ name: "puts", address: "0x2000", size: 0 },
],
},
});
if (!built.ok) throw built.error;
const [main, , broken] = built.value.functions;
const c = await readFile(join(built.value.directory, "program.c"));
const slice = (span: typeof main) =>
c
.subarray(span?.cOffset, (span?.cOffset ?? 0) + (span?.cLength ?? 0))
.toString("utf8");
expect(slice(main)).toBe(
"// Function: main @ 0x1000\nint main(void)\n{\n return 0;\n}\n\n",
);
expect(slice(broken)).toBe(
"// Function: broken @ 0x1080 (error: Exceeded the instruction budget)\n\n",
);
// Kuna listed and decompiled a staged copy from REA's own HOME, in reference mode.
const [listing] = await callsOf(kuna, "functions");
const [exported] = await callsOf(kuna, "decompile-project");
expect(listing?.argv).toEqual(["functions", exported?.argv[1], "--json"]);
expect(exported?.argv.slice(2)).toEqual([
"-o",
expect.stringContaining(join("cache", "staging")),
"--jobs",
"1",
]);
expect(exported?.argv[1]).not.toBe(input.targetPath);
expect(exported?.home).toBe(join(cacheRoot, "home"));
expect(await entries(cacheRoot, "staging")).toEqual([]);
// The same process and a fresh one both reuse the published entry.
await cache.build({ installation, ...input });
expect(await newCache(installation).lookup(built.value.key)).toMatchObject({
functions: built.value.functions,
inventory: built.value.inventory,
});
expect(await callsOf(kuna, "decompile-project")).toHaveLength(1);
});
it("shares one Kuna run between concurrent requests for one export", async () => {
const { kuna, installation, cache, target } = await fixture(
"rea-kuna-singleflight-",
{ delayMs: 300 },
);
const input = await target("program");
const [first, second] = await Promise.all([
cache.build({ installation, ...input }),
cache.build({ installation, ...input }),
]);
expect(first.ok && second.ok).toBe(true);
if (first.ok && second.ok)
expect(first.value.directory).toBe(second.value.directory);
expect(await callsOf(kuna, "decompile-project")).toHaveLength(1);
});
it("stops the run only when every waiting request is cancelled", async () => {
const { kuna, installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-cancel-",
{ project: "hang" },
);
const input = await target("program");
const firstCaller = new AbortController();
const secondCaller = new AbortController();
const first = cache.build({
installation,
...input,
signal: firstCaller.signal,
});
const second = cache.build({
installation,
...input,
signal: secondCaller.signal,
});
await expect.poll(() => callsOf(kuna, "decompile-project")).toHaveLength(1);
const pid = (await callsOf(kuna, "decompile-project"))[0]?.pid ?? 0;
firstCaller.abort();
expect(await first).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
// The other request still owns the run.
expect(processGone(pid)).toBe(false);
secondCaller.abort();
expect(await second).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
expect(await cache.close()).toEqual({ ok: true, value: null });
expect(processGone(pid)).toBe(true);
expect(await entries(cacheRoot, "staging")).toEqual([]);
expect(await entries(cacheRoot, "projects")).toEqual([]);
});
it("applies one deadline to the inventory and the export together", async () => {
const { kuna, installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-timeout-",
{ project: "hang" },
{ environment: { REA_KUNA_PROJECT_TIMEOUT_MS: "800" } },
);
const result = await cache.build({
installation,
...(await target("program")),
});
expect(result).toMatchObject({
ok: false,
error: { _tag: "AnalysisTimeoutError", timeoutMs: 800 },
});
const [exported] = await callsOf(kuna, "decompile-project");
expect(processGone(exported?.pid ?? 0)).toBe(true);
expect(await entries(cacheRoot, "staging")).toEqual([]);
await setFakeKunaBehavior(kuna, { inventory: "hang" });
expect(
await cache.build({ installation, ...(await target("other")) }),
).toMatchObject({ ok: false, error: { _tag: "AnalysisTimeoutError" } });
});
it("reports a failed export with Kuna's diagnostics and retains nothing", async () => {
const { installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-failure-",
{ project: "exit-failure" },
);
const result = await cache.build({
installation,
...(await target("program")),
});
expect(result).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
userMessage:
"kuna decompile-project failed with exit code 1: error: not in recognized object file format",
capturedOutput: { exit_code: 1 },
},
});
expect(await entries(cacheRoot, "projects")).toEqual([]);
expect(await entries(cacheRoot, "staging")).toEqual([]);
});
it.each([
[
"an inventory that is not Kuna's JSON document",
{ inventory: "not-json" },
"inventory is not JSON",
],
[
"an export without Kuna's completed-file summary",
{ project: "silent" },
"summary",
],
[
"an export whose headers disagree with Kuna's summary",
{ project: "miscounted" },
"2 function headers but Kuna reported 3 functions",
],
] as const)("refuses %s", async (_case, behavior, reason) => {
const { installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-refused-",
behavior,
);
expect(
await cache.build({ installation, ...(await target("program")) }),
).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
userMessage: expect.stringContaining(reason),
},
});
expect(await entries(cacheRoot, "projects")).toEqual([]);
expect(await entries(cacheRoot, "staging")).toEqual([]);
});
it("refuses to decompile a target that no longer matches its digest", async () => {
const { kuna, installation, cache, target } =
await fixture("rea-kuna-changed-");
const input = await target("program");
await appendFile(input.targetPath, "patched");
expect(await cache.build({ installation, ...input })).toMatchObject({
ok: false,
error: { _tag: "AnalysisArtifactChangedError", path: input.targetPath },
});
expect(await fakeKunaCalls(kuna)).toHaveLength(1);
});
it("removes its staging when the target disappears before the copy", async () => {
const { installation, cacheRoot, cache, target } =
await fixture("rea-kuna-vanished-");
const input = await target("program");
const result = await cache.build({
installation,
...input,
targetPath: `${input.targetPath}.gone`,
});
expect(result).toMatchObject({
ok: false,
error: {
_tag: "ProviderAdapterError",
userMessage: expect.stringContaining("ENOENT"),
},
});
expect(await entries(cacheRoot, "staging")).toEqual([]);
});
it("discards an entry whose files no longer match their manifest", async () => {
const { installation, cacheRoot, cache, target } =
await fixture("rea-kuna-tamper-");
const input = await target("program");
const built = await cache.build({ installation, ...input });
if (!built.ok) throw built.error;
await appendFile(join(built.value.directory, "program.c"), "// edited\n");
expect(await newCache(installation).lookup(built.value.key)).toBeUndefined();
expect(await entries(cacheRoot, "projects")).toEqual([]);
});
it("evicts least recently used entries above the cap, never a pinned one", async () => {
const { installation, cacheRoot, target } = await fixture("rea-kuna-evict-", {
asmPadding: 400 * 1024,
});
const cache = newCache(installation, 1024 * 1024);
const build = async (name: string, ageSeconds: number) => {
const result = await cache.build({ installation, ...(await target(name)) });
if (!result.ok) throw result.error;
const when = new Date(Date.now() - ageSeconds * 1000);
await utimes(join(cacheRoot, "projects", result.value.key), when, when);
return result.value.key;
};
const oldest = await build("oldest", 30);
const pinned = await build("pinned", 40);
const release = cache.pin(pinned);
const newest = await build("newest", 0);
// Three ~400 KiB entries exceed 1 MiB: the oldest unpinned entry goes.
expect((await entries(cacheRoot, "projects")).sort()).toEqual(
[pinned, newest].sort(),
);
expect(await cache.lookup(oldest)).toBeUndefined();
release();
await build("latest", 0);
expect(await entries(cacheRoot, "projects")).not.toContain(pinned);
});
it("refuses an export larger than the whole cache may hold", async () => {
const { installation, cacheRoot, target } = await fixture(
"rea-kuna-capacity-",
{ asmPadding: 8 * 1024 },
);
const result = await newCache(installation, 4096).build({
installation,
...(await target("program")),
});
expect(result).toMatchObject({
ok: false,
error: {
_tag: "AnalysisResourceConstraintError",
resource: "capacity",
reportedLimits: { cache_max_bytes: 4096 },
},
});
expect(await entries(cacheRoot, "projects")).toEqual([]);
});
it("keeps the first publication when another process exports the same key", async () => {
const { kuna, installation, cacheRoot, target } = await fixture(
"rea-kuna-race-",
{ delayMs: 300 },
);
const input = await target("program");
// Two cache instances stand in for two REA processes sharing one root.
const [first, second] = await Promise.all([
newCache(installation).build({ installation, ...input }),
newCache(installation).build({ installation, ...input }),
]);
expect(await callsOf(kuna, "decompile-project")).toHaveLength(2);
expect(first.ok && second.ok).toBe(true);
if (first.ok && second.ok)
expect(first.value.manifest).toEqual(second.value.manifest);
expect(await entries(cacheRoot, "projects")).toEqual([
keyOf(installation, input),
]);
expect(await entries(cacheRoot, "staging")).toEqual([]);
});
it("reports each build phase to every waiting request", async () => {
const { installation, cache, target } = await fixture("rea-kuna-phases-");
const statuses: KunaExportStatus[] = [];
const result = await cache.build({
installation,
...(await target("program")),
onStatus: (status) => statuses.push(status),
});
expect(result.ok).toBe(true);
expect(statuses).toEqual([
{ phase: "inventory", inventoryFunctions: null },
{ phase: "decompiling", inventoryFunctions: 3 },
{ phase: "sealing", inventoryFunctions: 3 },
]);
});
it("sweeps retired entries but preserves staging with unresolved process ownership", async () => {
const { installation, cacheRoot, cache, target } =
await fixture("rea-kuna-sweep-");
const abandoned = join(cacheRoot, "staging", "build-abandoned");
const recent = join(cacheRoot, "staging", "build-recent");
const evicted = join(cacheRoot, "trash", "evicted");
for (const path of [abandoned, recent, evicted])
await mkdir(path, { recursive: true });
const twoDaysAgo = new Date(Date.now() - 2 * 24 * 60 * 60 * 1000);
await utimes(abandoned, twoDaysAgo, twoDaysAgo);
expect(
(await cache.build({ installation, ...(await target("program")) })).ok,
).toBe(true);
expect(await entries(cacheRoot, "staging")).toEqual([
"build-abandoned",
"build-recent",
]);
expect(await entries(cacheRoot, "trash")).toEqual([]);
});
it.each(["sweep", "close"] as const)(
"keeps failed file cleanup owned until recovery through %s",
async (recovery) => {
const { installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-trash-cleanup-",
);
const retired = join(cacheRoot, "trash", "retired");
await mkdir(retired, { recursive: true });
await writeFile(join(retired, "evidence"), "retained export");
await chmod(retired, 0o500);
let built;
let closed;
try {
built = await cache.build({ installation, ...(await target("program")) });
closed = await cache.close();
} finally {
await chmod(retired, 0o700);
}
if (!built.ok) throw built.error;
expect(closed).toMatchObject({
ok: false,
error: { cleanupIncomplete: true, cleanupResources: [retired] },
});
expect(await readFile(join(retired, "evidence"), "utf8")).toBe(
"retained export",
);
if (recovery === "sweep") {
const next = await cache.build({
installation,
...(await target("other")),
});
if (!next.ok) throw next.error;
}
expect(await cache.close()).toEqual({ ok: true, value: null });
expect(await entries(cacheRoot, "trash")).toEqual([]);
},
);
it("keeps staging owned when a cancelled Kuna process cannot be proven stopped", async () => {
const { kuna, installation, cacheRoot, target } = await fixture(
"rea-kuna-retained-",
{ project: "hang" },
);
// Stop the process for real, but report that ownership could not be revalidated.
const cache = new KunaProjectCache(cacheRoot, {
maxBytes: 64 * 1024 ** 2,
launcher: async (options) => {
const spawned = await spawnOwnedProviderProcess(options);
if (options.arguments[0] !== "decompile-project") return spawned;
const workspace = options.cwd;
if (workspace === undefined) throw new Error("Kuna workspace is missing");
return {
...spawned,
cleanup: async () => {
await cleanupOwnedProcessGroup(spawned.ownership);
// A full disk or lost directory write access can prevent a final
// lifecycle marker. Cleanup must remain fail-closed across instances.
await chmod(workspace, 0o500);
return {
cleaned: false,
reason: "ownership could not be revalidated",
};
},
};
},
});
const caller = new AbortController();
const build = cache.build({
installation,
...(await target("program")),
signal: caller.signal,
});
await expect.poll(() => callsOf(kuna, "decompile-project")).toHaveLength(1);
caller.abort();
expect(await build).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
const closed = await cache.close();
const staged = await entries(cacheRoot, "staging");
const retained = join(cacheRoot, "staging", staged[0] ?? "missing");
// Restore access before assertions so the fixture can always remove it.
await chmod(retained, 0o700);
expect(closed).toMatchObject({
ok: false,
error: {
cleanupIncomplete: true,
diagnostics: { reason: "ownership could not be revalidated" },
},
});
expect(staged).toHaveLength(1);
if (!closed.ok) expect(closed.error.cleanupResources).toContain(retained);
const twoDaysAgo = new Date(Date.now() - 2 * 24 * 60 * 60 * 1000);
await utimes(retained, twoDaysAgo, twoDaysAgo);
await setFakeKunaBehavior(kuna, {});
const other = newCache(installation);
const built = await other.build({
installation,
...(await target("other")),
});
if (!built.ok) throw built.error;
expect(await entries(cacheRoot, "staging")).toEqual(staged);
expect(await readFile(join(retained, "input", "program"), "utf8")).toBe(
"program",
);
expect(await other.close()).toEqual({ ok: true, value: null });
// The fake process really stopped, so the test can remove its files. File
// absence still does not resolve the deliberately unproven process record.
await rm(retained, { recursive: true });
expect(await cache.close()).toMatchObject({
ok: false,
error: { diagnostics: { reason: "ownership could not be revalidated" } },
});
});
it("rebuilds an entry another cache instance evicted after this one verified it", async () => {
const { kuna, installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-evicted-elsewhere-",
);
const input = await target("program");
const built = await cache.build({ installation, ...input });
if (!built.ok) throw built.error;
// Another process retires the entry the way #discard does.
await mkdir(join(cacheRoot, "trash"), { recursive: true });
await rename(
join(cacheRoot, "projects", built.value.key),
join(cacheRoot, "trash", "evicted"),
);
expect(await cache.lookup(built.value.key)).toBeUndefined();
const rebuilt = await cache.build({ installation, ...input });
expect(rebuilt.ok).toBe(true);
expect(await callsOf(kuna, "decompile-project")).toHaveLength(2);
expect(await entries(cacheRoot, "projects")).toEqual([built.value.key]);
});
it("reports an abandoned build's unproven cleanup even after a retry replaced it", async () => {
const { kuna, installation, cacheRoot, target } = await fixture(
"rea-kuna-replaced-",
{ project: "hang" },
);
let exports = 0;
const cache = new KunaProjectCache(cacheRoot, {
maxBytes: 64 * 1024 ** 2,
launcher: async (options) => {
const spawned = await spawnOwnedProviderProcess(options);
if (options.arguments[0] !== "decompile-project" || ++exports > 1)
return spawned;
// The first export stops slowly and cannot prove its group gone.
return {
...spawned,
cleanup: async () => {
await new Promise((resolve) => setTimeout(resolve, 500));
await cleanupOwnedProcessGroup(spawned.ownership);
return { cleaned: false, reason: "first export unverified" };
},
};
},
});
const input = await target("program");
const first = new AbortController();
const firstBuild = cache.build({
installation,
...input,
signal: first.signal,
});
await expect.poll(() => callsOf(kuna, "decompile-project")).toHaveLength(1);
first.abort();
await firstBuild;
// A retry for the same key starts while the first is still stopping.
const second = new AbortController();
const secondBuild = cache.build({
installation,
...input,
signal: second.signal,
});
await expect.poll(() => callsOf(kuna, "decompile-project")).toHaveLength(2);
second.abort();
await secondBuild;
expect(await cache.close()).toMatchObject({
ok: false,
error: { diagnostics: { reason: "first export unverified" } },
});
});
it("removes its staging when publication itself fails", async () => {
const { installation, cacheRoot, cache, target } = await fixture(
"rea-kuna-publish-failure-",
{ project: "block-manifest" },
);
const result = await cache.build({
installation,
...(await target("program")),
});
expect(result).toMatchObject({
ok: false,
error: { _tag: "ProviderAdapterError" },
});
expect(await entries(cacheRoot, "staging")).toEqual([]);
expect(await entries(cacheRoot, "projects")).toEqual([]);
});
@@ -1,252 +0,0 @@
import { createHash } from "node:crypto";
import { appendFileSync } from "node:fs";
import { readFile, readdir, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { afterEach, expect, it as test, onTestFinished, vi } from "vitest";
import { inspectKunaInstallation } from "../../../../src/kuna/KunaInstallation.js";
import { KunaProjectCache } from "../../../../src/kuna/KunaProjectCache.js";
import type { KunaExportStatus } from "../../../../src/kuna/KunaProjectExport.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { fakeKunaCalls, writeFakeKuna } from "./fakeKuna.js";
// Reproduce concurrent eviction's shared directory snapshot while retaining
// real exports, publication, hashing and file removal underneath it.
const listing = vi.hoisted(() => ({
directory: "",
names: [] as string[],
gate: undefined as Promise<void> | undefined,
arrived: 0,
touchPath: "",
touchGate: undefined as Promise<void> | undefined,
touched: false,
}));
vi.mock("node:fs/promises", async (importOriginal) => {
const actual = await importOriginal<typeof import("node:fs/promises")>();
return {
...actual,
readdir: async (...args: Parameters<typeof actual.readdir>) => {
if (String(args[0]) === listing.directory && listing.gate !== undefined) {
listing.arrived += 1;
await listing.gate;
return listing.names;
}
return actual.readdir(...args);
},
utimes: async (...args: Parameters<typeof actual.utimes>) => {
await actual.utimes(...args);
if (
String(args[0]) === listing.touchPath &&
listing.touchGate !== undefined
) {
listing.touched = true;
await listing.touchGate;
}
},
};
});
afterEach(() => {
listing.directory = "";
listing.gate = undefined;
listing.names = [];
listing.arrived = 0;
listing.touchPath = "";
listing.touchGate = undefined;
listing.touched = false;
});
const it = test.skipIf(process.platform === "win32");
const barrier = () => {
let release = () => {};
const promise = new Promise<void>((resolve) => {
release = resolve;
});
return { promise, release };
};
const fixture = async (maxBytes = 64 * 1024 ** 2) => {
const root = await createTestTempDirectory("rea-kuna-concurrency-");
const kuna = await writeFakeKuna(root, {
delayMs: 300,
asmPadding: 400 * 1024,
});
const inspected = await inspectKunaInstallation({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
});
if (!inspected.ok) throw new Error(inspected.error.reason);
const installation = inspected.value;
const cache = new KunaProjectCache(installation.configuration.cacheRoot, {
maxBytes,
});
const request = async (name: string) => {
const targetPath = join(root, name);
await writeFile(targetPath, name);
return {
installation,
targetPath,
targetSha256: createHash("sha256").update(name).digest("hex"),
};
};
return {
kuna,
cache,
request,
cacheRoot: installation.configuration.cacheRoot,
};
};
it.each([true, false])(
"preserves concurrent results and reclaims capacity (pin: %s)",
async (pin) => {
const { cache, request, cacheRoot } = await fixture(600 * 1024);
const actual =
await vi.importActual<typeof import("node:fs/promises")>(
"node:fs/promises",
);
let releaseListing = () => {};
const gate = new Promise<void>((resolve) => {
releaseListing = resolve;
});
listing.directory = join(cacheRoot, "projects");
listing.gate = gate;
const releases: (() => void)[] = [];
const inputs = await Promise.all([request("first"), request("second")]);
const builds = inputs.map(async (input) => {
const result = await cache.build(input);
if (!result.ok) throw result.error;
if (pin) releases.push(cache.pin(result.value.key));
return result.value;
});
onTestFinished(async () => {
releaseListing();
await Promise.allSettled(builds);
for (const release of releases) release();
const closed = await cache.close();
if (!closed.ok) throw closed.error;
});
await expect
.poll(() => actual.readdir(listing.directory).catch(() => []))
.toHaveLength(2);
listing.names = await actual.readdir(listing.directory);
releaseListing();
const projects = await Promise.all(builds);
if (!pin) {
expect(await cache.close()).toEqual({ ok: true, value: null });
expect(await actual.readdir(listing.directory)).toHaveLength(1);
return;
}
for (const project of projects) {
expect(
await readFile(
join(project.directory, project.manifest.files.c.path),
"utf8",
),
).toContain("// Function: main @ 0x1000");
}
},
);
it.each(["decompiling", "sealing"] as const)(
"rejects an executable changed during %s",
async (phase) => {
const { kuna, cache, request, cacheRoot } = await fixture();
const input = await request("program");
const result = await cache.build({
...input,
onStatus: (status) => {
if (status.phase === phase)
appendFileSync(kuna, "\n// installation replaced\n");
},
});
expect(result).toMatchObject({
ok: false,
error: {
diagnostics: {
expected_sha256: input.installation.sha256,
observed_sha256: expect.any(String),
},
},
});
expect(await readdir(join(cacheRoot, "projects"))).toEqual([]);
expect(await cache.close()).toEqual({ ok: true, value: null });
},
);
it("protects a cache request that arrives after an eviction snapshot starts", async () => {
const { cache, cacheRoot, request } = await fixture(600 * 1024);
const actual =
await vi.importActual<typeof import("node:fs/promises")>(
"node:fs/promises",
);
const firstInput = await request("first");
const first = await cache.build(firstInput);
if (!first.ok) throw first.error;
const snapshot = barrier();
const touch = barrier();
listing.directory = join(cacheRoot, "projects");
listing.gate = snapshot.promise;
listing.touchPath = join(listing.directory, first.value.key);
listing.touchGate = touch.promise;
const releases: (() => void)[] = [];
const pinResult = async (pending: ReturnType<KunaProjectCache["build"]>) => {
const result = await pending;
if (!result.ok) throw result.error;
releases.push(cache.pin(result.value.key));
return result.value;
};
const second = pinResult(cache.build(await request("second")));
let warm: ReturnType<typeof pinResult> | undefined;
onTestFinished(async () => {
snapshot.release();
touch.release();
await Promise.allSettled(warm === undefined ? [second] : [second, warm]);
for (const release of releases) release();
const closed = await cache.close();
if (!closed.ok) throw closed.error;
});
await expect.poll(() => listing.arrived).toBe(1);
warm = pinResult(cache.build(firstInput));
await expect.poll(() => listing.touched).toBe(true);
listing.names = await actual.readdir(listing.directory);
snapshot.release();
await second;
touch.release();
const reused = await warm;
expect(
await readFile(
join(reused.directory, reused.manifest.files.c.path),
"utf8",
),
).toContain("// Function: main @ 0x1000");
});
it("keeps a shared progress callback registered for the remaining waiter", async () => {
const { kuna, cache, request } = await fixture();
const input = await request("program");
const controller = new AbortController();
const statuses: KunaExportStatus[] = [];
const onStatus = (status: KunaExportStatus) => {
statuses.push(status);
if (status.phase === "decompiling") controller.abort();
};
const first = cache.build({ ...input, signal: controller.signal, onStatus });
const second = cache.build({ ...input, onStatus });
expect(await first).toMatchObject({
ok: false,
error: { _tag: "AnalysisCancelledError" },
});
const result = await second;
if (!result.ok) throw result.error;
expect(statuses.map(({ phase }) => phase)).toContain("sealing");
expect(
(await fakeKunaCalls(kuna)).filter(
({ argv }) => argv[0] === "decompile-project",
),
).toHaveLength(1);
expect(await cache.close()).toEqual({ ok: true, value: null });
});
@@ -1,164 +0,0 @@
import { expect, it } from "vitest";
import {
parseKunaExportSummary,
parseKunaInventory,
scanKunaProjectSource,
} from "../../../../src/kuna/KunaProjectFiles.js";
const SOURCE = [
'#include "a.out.h"',
"",
"// Function: main @ 0x1000",
"int main(void)",
"{",
' return puts("// Function: not a header");',
"}",
"",
"// Function: operator new(unsigned long) @ 0x1040",
"void *operator new(unsigned long a0) { return malloc(a0); }",
"",
"// Function: sub_1080 @ 0x1080 (error: Flow exceeded maximum allowable instructions)",
"",
"",
].join("\n");
/** Feed the text in fixed-size pieces so headers straddle chunk boundaries. */
async function* chunked(text: string, size: number): AsyncGenerator<Buffer> {
const bytes = Buffer.from(text);
for (let offset = 0; offset < bytes.length; offset += size)
yield bytes.subarray(offset, offset + size);
}
it("splits Kuna's .c into function blocks at any chunk size", async () => {
// SOURCE is ASCII, so string offsets are byte offsets.
const starts = [
SOURCE.indexOf("// Function: main"),
SOURCE.indexOf("// Function: operator new"),
SOURCE.indexOf("// Function: sub_1080"),
SOURCE.length,
];
const span = (index: number) => ({
cOffset: starts[index] ?? -1,
cLength: (starts[index + 1] ?? -1) - (starts[index] ?? -1),
});
const expected = [
{ name: "main", address: "0x1000", ...span(0), error: null },
{
name: "operator new(unsigned long)",
address: "0x1040",
...span(1),
error: null,
},
{
name: "sub_1080",
address: "0x1080",
...span(2),
error: "Flow exceeded maximum allowable instructions",
},
];
expect(SOURCE.slice(starts[0], starts[1])).toBe(
'// Function: main @ 0x1000\nint main(void)\n{\n return puts("// Function: not a header");\n}\n\n',
);
for (const size of [1, 7, 64, 4096])
expect(
await scanKunaProjectSource(chunked(SOURCE, size), {
decompiled: 2,
failed: 1,
}),
).toEqual({ ok: true, value: expected });
const block = Buffer.from(SOURCE)
.subarray(expected[2]?.cOffset, SOURCE.length)
.toString();
expect(block.startsWith("// Function: sub_1080 @ 0x1080 (error:")).toBe(
true,
);
});
it.each([
[
"a header count that disagrees with Kuna's summary",
SOURCE,
{ decompiled: 3, failed: 1 },
"3 function headers but Kuna reported 4",
],
[
"a failure count that disagrees with Kuna's summary",
SOURCE,
{ decompiled: 3, failed: 0 },
"marks 1 failed functions but Kuna reported 0",
],
[
"a malformed header",
"// Function: main at 0x1000\n",
{ decompiled: 1, failed: 0 },
"unrecognized function header at byte 0",
],
[
"a repeated address",
"// Function: a @ 0x10\n\n// Function: b @ 0x10\n\n",
{ decompiled: 2, failed: 0 },
"repeats address 0x10",
],
[
"an overlong header line",
`// Function: ${"x".repeat(70_000)} @ 0x10\n`,
{ decompiled: 1, failed: 0 },
"too long",
],
] as const)("refuses %s", async (_case, source, summary, reason) => {
expect(await scanKunaProjectSource(chunked(source, 4096), summary)).toEqual({
ok: false,
error: expect.stringContaining(reason),
});
});
it("reads Kuna's completed-export summary from its stdout", () => {
expect(
parseKunaExportSummary(
"wrote ./a.out.kuna: a.out.c (325163 bytes), a.out.h (21366 bytes); functions: 172 ok, 3 failed\n",
),
).toEqual({ decompiled: 172, failed: 3 });
expect(parseKunaExportSummary("wrote ./a.out.kuna\n")).toBeUndefined();
});
it("reads the callable inventory, ignoring fields a later release adds", () => {
expect(
parseKunaInventory(
JSON.stringify({
binary: "a.out",
count: 1,
total: 1,
error: null,
functions: [
{
name: "_inflate",
address: 4294995696,
address_hex: "0x100006ef0",
aliases: ["inflate"],
object_location: null,
size: 252,
interior: false,
},
],
}),
),
).toEqual({
ok: true,
value: [
{
name: "_inflate",
address: "0x100006ef0",
aliases: ["inflate"],
size: 252,
},
],
});
expect(parseKunaInventory("functions: 1")).toEqual({
ok: false,
error: "the function inventory is not JSON",
});
expect(
parseKunaInventory(JSON.stringify({ functions: [{ name: "x" }] })),
).toMatchObject({ ok: false });
});
@@ -301,51 +301,6 @@ describe("managed application graph identity facts", () => {
),
).toEqual([]);
});
it("projects a module whose metadata name is empty without a label", () => {
const bytes = buildManagedPeFixture({ ilBody: Buffer.from([0x06, 0x2a]) });
const control = inspectManagedMembersBytes(
bytes,
managedPeFixtureTarget(bytes, "/fixture/EmptyModuleName.dll"),
);
if (control.module === null) throw new Error("Expected a Module row");
// Point the Module row's Name at the string heap's empty entry.
bytes.writeUInt16LE(0, control.module.row_offset + 2);
const binary = managedPeFixtureTarget(
bytes,
"/fixture/EmptyModuleName.dll",
);
const members = inspectManagedMembersBytes(bytes, binary);
expect(members.module?.name).toBe("");
const projected = projectManagedApplicationGraphEvidence({
managed_members: createEvidence(binary, MANAGED_STATIC_PROVIDER, {
operation: "inspect_managed_members",
parameters: {},
result: members,
rawResult: null,
limitations: members.limitations,
locations: [{ kind: "artifact-path", path: binary.path }],
}),
});
if (!projected.ok) throw projected.error;
const graph = parseJavaScriptApplicationGraph(
managedApplicationGraphResultSchema.parse(
parseEvidence(projected.value).normalized_result,
).graph,
);
expect(
graph.nodes.find(({ kind }) => kind === "managed-module")
?.observations[0],
).toMatchObject({ label: null, properties: { name: "" } });
expect(graph.nodes.map(({ kind }) => kind)).toEqual(
expect.arrayContaining([
"managed-module",
"managed-type",
"managed-method",
"managed-field",
]),
);
});
});
it("normalizes MVID casing for node identity while preserving observed values", () => {
Loaded 100 of 126 files, more files were not shown because too many files have changed in this diff. Show more