Compare commits

...
Author SHA1 Message Date
N0zoM1z0andKaoru0822 a13200b01d chore(ci): inspect structured Windows baseline spawn failures
The JSON reporter retains original errors in failureMessages but suppresses captured console output. Use unmodified baseline source and require both exact spawn failures before checking the candidate.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 15:33:21 +00:00
N0zoM1z0andKaoru0822 f3d4abd3eb chore(ci): run isolated native Windows Evidence transport proof
Verification-only branch; do not integrate the diagnostic workflow.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 15:29:32 +00:00
N0zoM1z0andKaoru0822 8809be9343 test(applications): reuse owned files for complete Evidence requests
Refresh the Windows request transport fix onto current main. Keep the already-file-backed rest trace unchanged and reuse each existing test-owned directory for the remaining two complete Evidence requests.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 15:28:45 +00:00
KenmegeandKaoru0822 40f73f62bc test(rizin): own fake provider cleanup in shared fixtures (#3735)
Retain native fixture ownership through observed bounded cleanup and preserve files and handles when cleanup cannot be confirmed. Verified all seven files on Linux and native macOS, including cleanup failure and exit-wait controls.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 15:24:55 +00:00
Ysopwnable 6ae59f3d19 fix(verifiers): launch npm through Node on Windows (#4153)
* fix(verifiers): launch npm through Node on Windows

* test(verifiers): cover Windows npm fallback
2026-10-15 23:12:29 +08:00
Kaoru0822andClaude Opus 5.5 9d6651e5b5 fix(artifacts): never treat a directory named *.map as a source map (#4131)
Artifact inventory decided source-map edges from the `.map` path suffix,
so an archive directory such as `main.js.map/` received a maps-source
edge whose parent was the sibling `main.js`, although the same
occurrence was classified as a directory container. Edge construction
now uses the occurrence's own artifact_kind, and the duplicate suffix
rule in relationFor is removed; such directories are contained by their
parent, and real source-map files keep maps-source.

Fixes #3909

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:41:04 -07:00
Kaoru0822andClaude Opus 5.5 a90c9eac6a fix(apple): keep Interface Builder objects whose archived class is unresolved (#4118)
* fix(apple): keep Interface Builder objects whose archived class is unresolved

Interface Builder keyed-archive projection skipped every object-table
entry without a resolved class name, so an archived object whose $class
reference dangled vanished from the objects, its count and its coverage,
which still reported complete; as a hierarchy root it lost its label.
Entries that carry their own $class are archived objects: they are now
kept with class_name null and the generic kind, their fields and labels
preserved. Scalars and class descriptors remain excluded.

Fixes #3958

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(apple): keep the dangling class reference of an unknown IB object

Review follow-up. An archived object kept with an unresolved class lost
its original `$class` reference, because class resolution replaces it
with a null class name. The object's attributes now retain that raw
reference when the class cannot be resolved. A graph-level test covers
the reported hierarchy-root case: the unresolved root keeps its label,
unknown kind, containment edge, object count and coverage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:40:10 -07:00
Kaoru0822andClaude Opus 5.5 cfd3c35a50 fix(kuna): keep the real lane's wide-string marker independent of layout (#4116)
On macOS arm64 the lane's UTF-8 marker ends right before the UTF-16 array,
and its last byte plus NUL decode as the UTF-16LE unit 'r'. Kuna therefore
reports "rREA wide marker", and the lane, which requires the exact text,
failed with "Required string \"REA wide marker\" was not recovered". A
leading NUL unit now ends any run the preceding bytes start, so the marker
is recovered exactly on any layout.

Refs #1957

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:38:33 -07:00
Kaoru0822andClaude Opus 5.5 3776f218e5 fix(apple): bound Interface Builder XML plists by shared value depth (#4114)
* fix(apple): bound Interface Builder XML plists by shared value depth

The Interface Builder preflight rejected XML archives whose raw element
depth exceeded 128, but the shared plist parser bounds zero-based value
depth at 128; the plist envelope and dictionary keys add elements, not
value depth. Archives whose deepest value sat at depth 127 or 128 were
therefore omitted as over budget although keyed-archive inspection
decodes them. The preflight's duplicate depth rule is removed; the
shared parser's depth limit is mapped to the existing budget omission,
so deeper archives stay explicitly omitted.

Fixes #3949

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(apple): return the decode reservation of a depth-rejected archive

Review follow-up. The removed element-depth preflight rejected an archive
before its plist representation was reserved; the shared depth limit now
rejects it during decoding, after the reservation. A large over-depth
archive therefore kept its share of the aggregate budget and could
starve later valid archives. A document omitted by the budget after
reserving now releases that reservation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:38:07 -07:00
Kaoru0822andClaude Opus 5.5 f7250fe35f fix(kuna): say why Kuna cannot answer instead of a generic message (#4113)
Kuna capability gaps such as "no code at this address" or "no function node
at this address" were built without a user message, so MCP and the CLI
showed only "This analysis is unavailable for the current target. Choose
another analysis or target." and kept the reason in details. One Kuna
helper now presents the reason as the message for every such gap, and the
unsupported-operation reason names call relationships and xrefs, which Kuna
now answers.

Refs #1957

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:37:20 -07:00
Ysopwnable cd64e24ff2 fix(verifiers): align WABT operation timeout (#4117)
* fix(verifiers): align WABT MCP request timeout

* fix(verifiers): bound packaged WABT workflow
2026-10-15 22:29:15 +08:00
Abdul Samad SiddiquiandKaoru0822 a40ee741c2 test(ghidra): align MIPS verifier ABI and snapshot expectations
Fixes #3415. Retain independently inspected GNU attribute facts and compare exact snapshot wire fields. Preserve strict metadata comparisons, specific incomplete-ABI refusals and source immutability checks.

Original implementation by @AbdulSamad94.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
2026-10-15 22:00:16 +08:00
Kaoru0822 3789d31e25 Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	tests/acceptance/applications/applicationWorkflowCli.test.ts
2026-10-14 05:14:49 +00:00
GuilhermeF 7628b0ad89 test(applications): pass Evidence requests to the CLI through files
Three application workflow CLI acceptance tests serialized complete
Evidence documents into one process argument. On Windows the command line
exceeds the process limit, so execFile rejects the spawn with
ENAMETOOLONG before the semantic trace or export-shape comparison starts.

Write those requests to a JSON file owned by the test case and pass its
path, which the CLI already accepts through the shared JSON input
boundary. Evidence stays complete; inline JSON transport keeps its
coverage in the existing parity and malformed-input tests.

Fixes #2942
2026-10-13 20:44:26 -03:00
40 changed files with 1234 additions and 1266 deletions

No files matched your search

+78 -917
View File
File diff suppressed because it is too large. Load diff
+1
View File
@@ -138,6 +138,7 @@ export const SCOPES = {
"src/domain/javascript/javascriptRuntimeStaticCandidates.ts",
"tests/boundary/filesystem/webScriptManifestPaths.test.ts",
"tests/boundary/process/ProcessEvidence.host.test.ts",
"tests/boundary/process/packageNpmInvocation.test.ts",
"tests/boundary/process/windowsOutputCapture.test.ts",
"tests/acceptance/webModuleCli.test.ts",
"tests/acceptance/webSourceLocationCli.test.ts",
+20
View File
@@ -1,5 +1,6 @@
import { execFile } from "node:child_process";
import { lstat } from "node:fs/promises";
import { dirname, join } from "node:path";
import { promisify } from "node:util";
import { Ajv2020 } from "ajv/dist/2020.js";
@@ -17,6 +18,25 @@ const loadMcpVerifierResults = () => import("./mcp-verifier-results.mjs");
/** Promisified child_process execution helper. */
export const exec = promisify(execFile);
/** Run npm through its JavaScript entry point so Windows command scripts are not required. */
export const execNpm = (arguments_, options = {}) => {
const environment = options.env ?? process.env;
const selectedEntry = environment.npm_execpath;
if (
selectedEntry !== undefined &&
(typeof selectedEntry !== "string" || selectedEntry.length === 0)
)
throw new Error("npm_execpath must name the selected npm CLI entry point");
const entry =
selectedEntry ??
(process.platform === "win32"
? join(dirname(process.execPath), "node_modules/npm/bin/npm-cli.js")
: undefined);
return entry === undefined
? exec("npm", arguments_, options)
: exec(process.execPath, [entry, ...arguments_], options);
};
/** Run a command and return its stdout. */
export const run = async (command, args, env) =>
(await exec(command, args, { env })).stdout;
+4
View File
@@ -0,0 +1,4 @@
// A single inspection can include native admission plus several WABT commands.
// The aggregate verifier covers seven complete fixtures and their failure paths.
export const WABT_OPERATION_TIMEOUT_MS = 4 * 60_000;
export const WABT_WORKFLOW_TIMEOUT_MS = 15 * 60_000;
+4 -6
View File
@@ -2,7 +2,7 @@ import { rm } from "node:fs/promises";
import { join } from "node:path";
import { pathToFileURL } from "node:url";
import { exec, json, run } from "./lib/verify-package-core.mjs";
import { exec, execNpm, json, run } from "./lib/verify-package-core.mjs";
/** Install the packed CLI globally with and without optional dependencies and verify PTY degradation. */
export async function verifyPackageInstall({
@@ -11,8 +11,7 @@ export async function verifyPackageInstall({
workspace,
environment,
}) {
await exec(
"npm",
await execNpm(
[
"install",
"--global",
@@ -53,8 +52,7 @@ export async function verifyPackageInstall({
`packaged PTY backend failed without lifecycle scripts: ${JSON.stringify(processCaptureCapability)}`,
);
const noOptionalPrefix = join(workspace, "prefix-no-optional");
await exec(
"npm",
await execNpm(
[
"install",
"--global",
@@ -108,7 +106,7 @@ export async function verifyPackageInstall({
const verifyInstalledPackageCanRepack = async (prefix, environment) => {
const installedPackageRoot = join(prefix, "lib/node_modules/rea-agents");
const repackedTarball = (
await exec("npm", ["pack", "--silent"], {
await execNpm(["pack", "--silent"], {
cwd: installedPackageRoot,
env: environment,
})
+4 -8
View File
@@ -2,7 +2,7 @@ import { mkdir } from "node:fs/promises";
import { join } from "node:path";
import { CATALOG_IDENTITY } from "../dist/catalogIdentity.js";
import { exec } from "./lib/verify-package-core.mjs";
import { exec, execNpm } from "./lib/verify-package-core.mjs";
import { verifyPackedBridge } from "./verify-packed-bridge.mjs";
/** Create a tarball and assert production packaging constraints. */
@@ -12,13 +12,9 @@ export async function verifyPackagePack({ root, workspace }) {
const tarball = join(
packDirectory,
(
await exec(
"npm",
["pack", "--silent", "--pack-destination", packDirectory],
{
cwd: root,
},
)
await execNpm(["pack", "--silent", "--pack-destination", packDirectory], {
cwd: root,
})
).stdout.trim(),
);
const packedFiles = (await exec("tar", ["-tf", tarball])).stdout.split("\n");
+3 -9
View File
@@ -28,6 +28,7 @@ import { fileURLToPath, pathToFileURL } from "node:url";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { execNpm } from "./lib/verify-package-core.mjs";
if (process.platform !== "win32" || process.arch !== "x64")
throw new Error(
@@ -82,15 +83,10 @@ let packageRoot = resolve(packageRootArgument ?? ".");
// package roots support an already-installed artifact without a second install.
if (packageRootArgument === undefined) {
try {
const npmEntry =
process.env.npm_execpath ??
join(dirname(process.execPath), "node_modules/npm/bin/npm-cli.js");
const packed = JSON.parse(
(
await exec(
process.execPath,
await execNpm(
[
npmEntry,
"pack",
"--json",
"--ignore-scripts",
@@ -107,10 +103,8 @@ if (packageRootArgument === undefined) {
"The package lane must produce one artifact.",
);
const prefix = join(workspace, "installed");
await exec(
process.execPath,
await execNpm(
[
npmEntry,
"install",
"--prefix",
prefix,
+8 -4
View File
@@ -2,7 +2,8 @@
import { join } from "node:path";
import { mkdir } from "node:fs/promises";
import { PrivateRuntimeRoot } from "../dist/process/PrivateRuntimeRoot.js";
import { exec } from "./lib/verify-package-core.mjs";
import { exec, execNpm } from "./lib/verify-package-core.mjs";
import { WABT_WORKFLOW_TIMEOUT_MS } from "./lib/wabt-verifier-timeouts.mjs";
import { verifyPackagePack } from "./verify-package-pack.mjs";
const root = await PrivateRuntimeRoot.create({
@@ -23,8 +24,7 @@ try {
XDG_CONFIG_HOME: join(home, ".config"),
XDG_CACHE_HOME: join(home, ".cache"),
};
await exec(
"npm",
await execNpm(
[
"install",
"--global",
@@ -49,7 +49,11 @@ try {
const verified = await exec(
process.execPath,
["scripts/verify-wasm-artifact.mjs", entrypoint, "--require-tools"],
{ env: environment, timeout: 180_000, maxBuffer: 4 * 1024 * 1024 },
{
env: environment,
timeout: WABT_WORKFLOW_TIMEOUT_MS,
maxBuffer: 4 * 1024 * 1024,
},
);
process.stdout.write(verified.stdout);
} finally {
+49 -84
View File
@@ -12,6 +12,7 @@ import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { parseBinaryTarget } from "../dist/application/BinaryTargetResolver.js";
import {
inspectMipsReadelf,
expectedMipsElfProfile,
inspectLegacyMipsReadelf,
assertMipsLoadedImage,
assertMipsProbe,
@@ -120,6 +121,17 @@ async function verifySoftFloat(call, symbols, byteOrder) {
);
return { storage: boundaries };
}
function assertIncompleteMipsInspection(error) {
assert.equal(error.code, "capability_unavailable");
assert.equal(error.details.selection_reason, "target_unsupported");
const rejection = error.details.rejections.find(
(candidate) => candidate.provider_id === "ghidra",
);
assert.equal(rejection?.code, "architecture_unsupported");
assert.match(rejection.reason, /ABI declaration inspection is incomplete/u);
assert.match(rejection.reason, /section header table/u);
}
const reports = [];
let primaryFailure;
try {
@@ -475,28 +487,9 @@ try {
JSON.parse(await readFile(snapshotPath, "utf8")),
);
assert.equal(snapshot.target.architecture, "mips");
const abi = independent.mips.abiFlags;
assert.deepEqual(
snapshot.binding.analysis_profile.parameters.mips_elf,
{
elf_class: independent.mips.elfClass,
byte_order: independent.mips.byteOrder,
type: independent.mips.type,
flags: independent.mips.flags,
abi_flags: {
version: abi.version,
isa_level: abi.isaLevel,
isa_revision: abi.isaRevision,
gpr_size: abi.gprSize,
cpr1_size: abi.cpr1Size,
cpr2_size: abi.cpr2Size,
fp_abi: abi.fpAbi,
isa_extension: abi.isaExtension,
ases: abi.ases,
flags1: abi.flags1,
flags2: abi.flags2,
},
},
expectedMipsElfProfile(independent.mips),
"Snapshot profile did not retain the independently inspected ELF/ABI declaration",
);
assert.equal(
@@ -617,7 +610,8 @@ try {
// PSP keeps its independent Allegrex/A-B-A checks; carved PSP is not verified.
let carved;
if (!psp) {
// Carved firmware keeps all program headers/load bytes but loses sections.
// A program-header ABI record survives carving, but GNU declarations in
// uncaptured section headers remain unknown: both adapters must refuse.
const read16 = (offset) =>
byteOrder === "little"
? sourceBytes.readUInt16LE(offset)
@@ -655,79 +649,50 @@ try {
carvedResolved.value.mips.limitations.join(" "),
/section header table/u,
);
const carvedTarget = await call("open_binary", {
path: carvedPath,
provider_id: "ghidra",
});
assert.equal(carvedTarget.sha256, carvedSha256);
const carvedImage = await call("inspect_native_load_image");
assertMipsLoadedImage(
carvedImage.observations,
byteOrder,
carvedSha256,
);
const carvedProcedures = await call("list_procedures");
assert.ok(
carvedProcedures.some(
(p) =>
BigInt(p.address) === BigInt(independent.symbols.rea_mips_entry),
const carvedError = requireMcpToolError(
await client.callTool(
{
name: "open_binary",
arguments: { path: carvedPath, provider_id: "ghidra" },
},
{ timeout: 60000 },
),
);
const carvedDossier = functionDossierSchema.parse(
await call("analyze_function", {
procedure: independent.symbols.rea_mips_entry,
}),
);
assert.ok(carvedDossier.pseudocode.trim().length > 0);
const carvedSnapshotPath = join(
workspace,
`${byteOrder}-${floatAbi}-carved.snapshot.json`,
);
await call("close_binary", { snapshot_path: carvedSnapshotPath });
const carvedSnapshot = parseAnalysisSnapshot(
JSON.parse(await readFile(carvedSnapshotPath, "utf8")),
);
assert.equal(carvedSnapshot.target.sha256, carvedSha256);
assertIncompleteMipsInspection(carvedError);
let carvedCliError;
try {
await execute(
process.execPath,
[
entrypoint,
"function",
carvedPath,
independent.symbols.rea_mips_entry,
"--provider",
"ghidra",
"--json",
],
{ env, timeout: 60000, maxBuffer: 1024 * 1024 },
);
} catch (cause) {
carvedCliError = cause;
}
assert.ok(
carvedSnapshot.evidence_bundle.records.some((record) =>
parseEvidence(record).limitations.some((item) =>
item.includes("section header table"),
),
),
);
const carvedCli = await execute(
process.execPath,
[
entrypoint,
"function",
carvedPath,
independent.symbols.rea_mips_entry,
"--provider",
"ghidra",
"--json",
],
{ env, timeout: 240000, maxBuffer: 16 * 1024 * 1024 },
);
const carvedEvidence = parseEvidence(JSON.parse(carvedCli.stdout));
assert.equal(carvedEvidence.subject?.digest.sha256, carvedSha256);
assert.ok(
carvedEvidence.limitations.some((item) =>
item.includes("section header table"),
),
);
assert.equal(
functionDossierSchema.parse(carvedEvidence.normalized_result)
.procedure.address,
carvedDossier.procedure.address,
carvedCliError &&
Number.isInteger(carvedCliError.code) &&
carvedCliError.code !== 0,
"Expected an incomplete-ABI refusal, not a timeout or success",
);
assertIncompleteMipsInspection(JSON.parse(carvedCliError.stdout));
assert.deepEqual(await readFile(carvedPath), carvedBytes);
carved = {
sha256: carvedSha256,
source_bytes: carvedBytes.length,
section_header_offset: sectionOffset,
effective_language: carvedImage.observations.language_id,
status: "expected-incomplete-abi-refusal",
limitations: carvedResolved.value.mips.limitations,
procedures: carvedProcedures.length,
cli_exit: carvedCliError.code,
mcp_error: carvedError,
};
}
reports.push({
+4 -1
View File
@@ -83,8 +83,11 @@ const FIXTURE_SOURCE = String.raw`
#include <stdio.h>
#include <stdlib.h>
static const char rea_kuna_utf8[] __attribute__((used)) = "_φ( °-°)/ REA unicode marker";
/* The leading NUL unit ends any UTF-16 run that bytes before this array would
start: the UTF-8 marker's last byte and NUL decode as the unit 'r', so on
layouts where it precedes this array Kuna reports "rREA wide marker". */
static const unsigned short rea_kuna_wide[] __attribute__((used)) = {
'R','E','A',' ','w','i','d','e',' ','m','a','r','k','e','r',0
0,'R','E','A',' ','w','i','d','e',' ','m','a','r','k','e','r',0
};
#include <string.h>
__attribute__((noinline)) unsigned rea_checksum(const unsigned char *p, size_t n) {
+2 -3
View File
@@ -1,7 +1,7 @@
import { cp, mkdir, readFile, readdir } from "node:fs/promises";
import { dirname, join, relative } from "node:path";
import { exec, pathExists } from "./lib/verify-package-core.mjs";
import { exec, execNpm, pathExists } from "./lib/verify-package-core.mjs";
/** Exercise Skills' default root discovery without ignored build outputs. */
export async function verifyRepositorySkill({ root, workspace }) {
@@ -34,8 +34,7 @@ export async function verifyRepositorySkill({ root, workspace }) {
await mkdir(consumer, { recursive: true });
await mkdir(home, { recursive: true });
const skillsVersion = "1.7.2";
await exec(
"npm",
await execNpm(
[
"exec",
"--yes",
+8 -5
View File
@@ -24,6 +24,7 @@ import {
requireMcpToolError,
} from "./lib/mcp-verifier-results.mjs";
import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
import { WABT_OPERATION_TIMEOUT_MS } from "./lib/wabt-verifier-timeouts.mjs";
const directory = process.env.REA_WABT_BIN_DIRECTORY;
if (!isAbsolute(directory ?? "")) {
@@ -76,6 +77,8 @@ try {
throw cause;
}
const client = new Client({ name: "wabt-artifact-verifier", version: "1" });
const callTool = (request) =>
client.callTool(request, { timeout: WABT_OPERATION_TIMEOUT_MS });
const transport = new StdioClientTransport({
command: process.execPath,
args: [entrypoint, "mcp"],
@@ -128,7 +131,7 @@ try {
TMP: childTempRoot,
TEMP: childTempRoot,
},
timeout: 120_000,
timeout: WABT_OPERATION_TIMEOUT_MS,
maxBuffer: 64 * 1024 * 1024,
},
)
@@ -238,7 +241,7 @@ try {
const cliValue = await inspectCli(input);
assert.equal(cliValue.category, undefined, JSON.stringify(cliValue));
const cli = parseEvidence(cliValue);
const reply = await client.callTool({
const reply = await callTool({
name: "inspect_wasm_artifact",
arguments: input,
});
@@ -332,7 +335,7 @@ try {
},
);
}
const bundleReply = await client.callTool({
const bundleReply = await callTool({
name: "get_evidence_bundle",
arguments: {},
});
@@ -362,7 +365,7 @@ try {
JSON.stringify(cliError),
);
const error = requireMcpToolError(
await client.callTool({
await callTool({
name: "inspect_wasm_artifact",
arguments: invalidInput,
}),
@@ -370,7 +373,7 @@ try {
assert.equal(error.category, "invalid_input");
const partial = error.details.partial_observation;
assert.equal(partial.kind, "retained-evidence");
const failedBundleReply = await client.callTool({
const failedBundleReply = await callTool({
name: "get_evidence_bundle",
arguments: {},
});
+10 -20
View File
@@ -18,7 +18,10 @@ import { promisify } from "node:util";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import { verifyCompleteToolCatalog } from "./lib/verify-package-core.mjs";
import {
execNpm,
verifyCompleteToolCatalog,
} from "./lib/verify-package-core.mjs";
import { verifyPackageUpdate } from "./verify-package-update.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
@@ -32,9 +35,9 @@ let report;
try {
const packed = JSON.parse(
(
await npm(
await execNpm(
["pack", "--json", "--silent", "--pack-destination", workspace],
root,
{ cwd: root, windowsHide: true },
)
).stdout,
);
@@ -63,9 +66,9 @@ try {
const tarball = join(workspace, packageResult.filename);
await access(tarball);
await npm(
await execNpm(
["install", "--no-package-lock", "--no-save", "--prefix", prefix, tarball],
workspace,
{ cwd: workspace, windowsHide: true },
);
const entry = join(
prefix,
@@ -209,9 +212,9 @@ try {
throw new Error(`Packaged uninstall did not remove ${id}`);
if (process.platform !== "win32")
await npm(
await execNpm(
["install", "--global", "--ignore-scripts", "--prefix", prefix, tarball],
workspace,
{ cwd: workspace, windowsHide: true },
);
const update = await verifyPackageUpdate({
prefix,
@@ -241,16 +244,3 @@ try {
process.stdout.write(
`${JSON.stringify({ verifier_run: await completeVerifierRun(verifierRun), ...report })}\n`,
);
function npm(arguments_, cwd) {
const npmExecPath = process.env.npm_execpath;
return npmExecPath === undefined
? exec(process.platform === "win32" ? "npm.cmd" : "npm", arguments_, {
cwd,
windowsHide: true,
})
: exec(process.execPath, [npmExecPath, ...arguments_], {
cwd,
windowsHide: true,
});
}
@@ -84,6 +84,8 @@ export const analyzeInterfaceBuilderBundle = async (input: {
recordOmittedArchive(entry.path, "max_documents");
unreportedOmittedArchives += selection.unreportedOmitted;
for (const entry of selection.selected) {
// Plist representation reserved for this document before decoding.
let plistReservation = 0;
try {
const remainingInputBytes =
MAX_AGGREGATE_INPUT_BYTES - aggregateInputBytes;
@@ -132,6 +134,7 @@ export const analyzeInterfaceBuilderBundle = async (input: {
decodedReservation.estimatedBytes,
"plist representation exceeds the aggregate Interface Builder decode budget",
);
plistReservation = decodedReservation.estimatedBytes;
binaryPrototypeKeyObserved =
decodedReservation.binaryPrototypeKeyObserved;
}
@@ -165,6 +168,9 @@ export const analyzeInterfaceBuilderBundle = async (input: {
} catch (cause: unknown) {
if (input.signal?.aborted === true) throw cause;
if (cause instanceof InterfaceBuilderDecodeBudgetExceeded) {
// A document rejected while decoding, e.g. beyond the shared depth
// limit, retains no representation; later documents keep its share.
decodeBudget.release(plistReservation);
omitted += 1;
recordOmittedArchive(entry.path, cause.reason);
continue;
@@ -700,6 +706,24 @@ const readEntry = async (
return Buffer.concat(chunks, length);
};
/**
* The shared parser bounds plist value nesting before its recursive decode and
* reports the limit as a RangeError; that is an omitted archive, not a
* malformed one.
*/
const parseXmlWithinDecodeBudget = (text: string) => {
try {
return parseXmlPropertyList(text);
} catch (cause: unknown) {
if (cause instanceof RangeError)
throw new InterfaceBuilderDecodeBudgetExceeded(
"aggregate_decode_budget_exhausted",
cause.message,
);
throw cause;
}
};
/** Project decoded plist data and dates while retaining omitted-key coverage. */
const decodePlist = (
bytes: Buffer,
@@ -716,7 +740,7 @@ const decodePlist = (
value: parseBinary(bytes),
omittedPrototypeKeys: binaryPrototypeKeyObserved ? null : 0,
}
: parseXmlPropertyList(xmlText ?? decodeXmlPlistText(bytes));
: parseXmlWithinDecodeBudget(xmlText ?? decodeXmlPlistText(bytes));
const projected = projectPlistValue(value);
return {
value: projected.value,
@@ -49,6 +49,14 @@ export class InterfaceBuilderDecodeBudget {
this.#remainingBytes -= bytes;
}
/** Return a reservation whose representation was rejected and not retained. */
release(bytes: number): void {
this.#remainingBytes = Math.min(
this.#initialBytes,
this.#remainingBytes + bytes,
);
}
/** Reserve an upper bound for one JSON-serialized string occurrence. */
reserveJsonStringOccurrence(value: string): void {
// JSON escaping uses at most six ASCII bytes per UTF-16 code unit, plus
@@ -117,7 +125,8 @@ const estimateXmlPlistExpansion = (
const xml = xmlText ?? decodeXmlPlistText(bytes);
const decodedSourceBytes = xml.length * 2;
let elementCount = 0;
let xmlDepth = 0;
// Value nesting is bounded by the shared plist parser, which counts plist
// value depth; raw element depth would also count the envelope and keys.
for (const token of xmlPreflightTokens(xml)) {
if (token.doctype) {
// Only a declaration outside opaque text can introduce an internal
@@ -130,15 +139,6 @@ const estimateXmlPlistExpansion = (
continue;
}
elementCount += 1;
if (token.closing) xmlDepth -= 1;
else if (!token.selfClosing) {
xmlDepth += 1;
if (xmlDepth > 128)
throw new InterfaceBuilderDecodeBudgetExceeded(
"aggregate_decode_budget_exhausted",
"XML plist nesting exceeds the aggregate Interface Builder decode budget",
);
}
if (decodedSourceBytes * 2 + elementCount * 128 > maxBytes)
return maxBytes + 1;
}
@@ -179,13 +179,7 @@ function* xmlPreflightTokens(xml: string) {
}
if (end === xml.length) return;
cursor = end + 1;
if (doctype || element)
yield {
doctype,
internalSubset,
closing: xml[start + 1] === "/",
selfClosing: xml[end - 1] === "/",
};
if (doctype || element) yield { doctype, internalSubset };
}
}
@@ -121,6 +121,43 @@ describe("source-map artifact relationships", () => {
);
},
);
it("keeps a directory named like a source map contained by its parent", () => {
const archive = createOccurrence(entry("app", "directory"), "app", null);
archive.artifact_id = contentId("1");
const script = createOccurrence(
entry("app/main.js", "file"),
"app/main.js",
archive.occurrence_id,
);
script.artifact_id = contentId("2");
const directory = createOccurrence(
entry("app/main.js.map", "directory"),
"app/main.js.map",
archive.occurrence_id,
);
directory.artifact_id = contentId("3");
const { edges, limitations } = createArtifactEdges(contentId("4"), [
archive,
script,
directory,
]);
expect(directory).toMatchObject({
artifact_kind: "container",
artifact_format: "directory",
});
expect(
edges.find(
({ occurrence_id }) => occurrence_id === directory.occurrence_id,
),
).toMatchObject({
relation: "contains",
parent_artifact_id: contentId("1"),
});
expect(limitations).toEqual([]);
});
});
const fat = (wide = false): Buffer =>
@@ -295,7 +295,9 @@ export const createArtifactEdges = (
const edges: ArtifactEdge[] = [];
const limitations: string[] = [];
for (const occurrence of occurrences) {
const isSourceMap = occurrence.logical_path.toLowerCase().endsWith(".map");
// Use the occurrence's own classification: a directory named `x.map` is
// a container, not a source map of its sibling.
const isSourceMap = occurrence.artifact_kind === "source-map";
const sourcePath = isSourceMap
? occurrence.logical_path.slice(0, -".map".length)
: undefined;
@@ -337,8 +339,10 @@ export const createArtifactEdges = (
relation:
occurrence.entry_kind === "slice"
? ("slice-of" as const)
: isSourceMap && mappedSource === undefined
? ("contains" as const)
: isSourceMap
? mappedSource === undefined
? ("contains" as const)
: ("maps-source" as const)
: relationFor(occurrence.logical_path),
occurrence_id: occurrence.occurrence_id,
};
@@ -587,11 +591,8 @@ const classifyFatArtifact = (
return identity.state === "mach-o" ? "mach-o-universal" : "unknown";
};
const relationFor = (path: string): ArtifactEdge["relation"] => {
const lower = path.toLowerCase();
if (lower.endsWith(".map")) return "maps-source";
if (lower.includes(".framework/")) return "embeds";
return "contains";
};
/** Relation of a non-source-map occurrence to its parent. */
const relationFor = (path: string): ArtifactEdge["relation"] =>
path.toLowerCase().includes(".framework/") ? "embeds" : "contains";
const depth = (path: string): number => path.split("/").length;
@@ -156,6 +156,90 @@ describe("compiled Interface Builder graph projection", () => {
});
});
describe("keyed archive objects with unresolved classes", () => {
it("keeps an archived object whose class reference cannot be resolved", () => {
const parsed = parseInterfaceBuilderRecords({
$archiver: "NSKeyedArchiver",
$version: 100000,
$objects: [
"$null",
{ $class: { UID: 99 }, label: "Unresolved object" },
{ $classname: "UIView", $classes: ["UIView", "NSObject"] },
{ $class: { UID: 2 }, "NS.string": "ok" },
"plain scalar",
],
$top: { root: { UID: 3 } },
});
// The class descriptor and the scalar are not archived objects.
expect(parsed.objects).toMatchObject([
{
id: "1",
kind: "other",
class_name: null,
name: "Unresolved object",
attributes: { label: "Unresolved object", $class: { UID: 99 } },
},
{ id: "3", kind: "view", class_name: "UIView" },
]);
expect(parsed.objects[1]?.attributes).not.toHaveProperty("$class");
expect(parsed.objectCount).toBe(2);
});
it("keeps an unresolved hierarchy root's label and coverage in the graph", () => {
const result = buildInterfaceBuilderAnalysis({
targetSha256: hash,
toolVersion: "test",
documents: [
{
relativePath: "Main.nib",
archiveSha256: hash,
documentKind: "nib",
raw: {
$archiver: "NSKeyedArchiver",
$version: 100000,
$objects: [
"$null",
{ $class: { UID: 99 }, label: "Unresolved object" },
{ $classname: "UIView", $classes: ["UIView", "NSObject"] },
{ $class: { UID: 2 }, "NS.string": "ok" },
],
$top: { root: { UID: 1 } },
},
},
],
limits: interfaceBuilderLimitsSchema.parse({}),
});
const objectNodes = result.graph.nodes.filter(
({ attributes }) => attributes.interface_builder_object_id !== undefined,
);
expect(objectNodes).toMatchObject([
{
kind: "unknown",
name: "Unresolved object",
attributes: { interface_builder_object_id: "1", class_name: null },
},
{ kind: "view", attributes: { interface_builder_object_id: "3" } },
]);
expect(result.documents[0]?.object_count).toBe(2);
expect(
result.graph.edges.some(
({ relation, to, resolution }) =>
relation === "contains" &&
to === objectNodes[0]?.id &&
resolution === "observed",
),
).toBe(true);
expect(result.graph.coverage).toContainEqual(
expect.objectContaining({
facet: "objects:Main.nib",
status: "complete",
examined: 2,
omitted: 0,
}),
);
});
});
describe("Interface Builder parser bounds", () => {
it("reports connections discarded by the parser's hard ceiling", () => {
const connections = Array.from({ length: 40_001 }, (_, index) => ({
@@ -327,11 +327,16 @@ const parseKeyedArchive = (
if (index === 0) continue;
const resolved = record(resolve(raw));
const className = firstString(resolved.className);
if (className === null) continue;
// An archived object names its class with its own `$class` reference;
// when that reference cannot be resolved the object is still observed,
// with an unknown class. Scalars and class descriptors are not objects.
if (className === null && !Object.hasOwn(record(raw), "$class")) continue;
objectCount += 1;
const fields = Object.fromEntries(
Object.entries(resolved).filter(([key]) => key !== "className"),
);
// Keep the unresolvable class reference itself, e.g. a dangling UID.
if (className === null) fields.$class = record(raw).$class;
const authoredId = firstString(fields.objectID, fields["object-id"]);
const objectId =
authoredId ??
@@ -357,6 +362,7 @@ const parseKeyedArchive = (
}),
);
if (
className !== null &&
/(?:Outlet|Connection|Segue|ActionConnection|ControlConnector)/iu.test(
className,
)
+2 -3
View File
@@ -4,7 +4,6 @@ import { join } from "node:path";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisCapabilityUnavailableError,
AnalysisCancelledError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
@@ -19,6 +18,7 @@ import {
} from "./KunaProjectFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
import { KUNA_FUNCTION_TEXT_BYTES } from "./KunaLimits.js";
import { kunaUnavailable } from "./KunaUnavailable.js";
/** The exact producing range for one facet, including a failed C block. */
export interface KunaProjectFileSpan {
@@ -134,8 +134,7 @@ export class KunaProjectReader {
const span = index.value.code.get(address);
if (span === undefined)
return err(
new AnalysisCapabilityUnavailableError(
KUNA_PROVIDER_ID,
kunaUnavailable(
operation,
`Kuna's export lists no code at ${address}; import slots and data entries have no listing.`,
),
+4 -19
View File
@@ -8,10 +8,7 @@ import type {
ProviderAvailability,
ProviderTargetSupport,
} from "../application/AnalysisProvider.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
} from "../domain/analysisErrorCore.js";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import {
createAnalysisProfile,
type AnalysisProfileCommitment,
@@ -33,6 +30,7 @@ import {
import { KUNA_COMMAND_REMEDIATION, KUNA_PROVIDER_ID } from "./KunaRelease.js";
import { KunaRunSlots } from "./KunaRunSlots.js";
import { KunaSessionClient } from "./KunaSessionClient.js";
import { kunaUnavailable } from "./KunaUnavailable.js";
/** Architectures REA verified Kuna on: x86-64 and AArch64 ELF and Mach-O, plus ARM Thumb. */
const VERIFIED_ARCHITECTURES: ReadonlySet<string> = new Set([
@@ -169,14 +167,7 @@ export class KunaProvider implements AnalysisProviderCandidate {
...(options?.signal === undefined ? {} : { signal: options.signal }),
});
if (!installation.ok)
return err(
new AnalysisCapabilityUnavailableError(
KUNA_PROVIDER_ID,
"open_binary",
installation.error.reason,
{ userMessage: installation.error.reason },
),
);
return err(kunaUnavailable("open_binary", installation.error.reason));
const { value } = installation;
return ok({
profile: createAnalysisProfile(
@@ -218,13 +209,7 @@ export class KunaProvider implements AnalysisProviderCandidate {
if (!configuration.ok)
return {
execute: async (operation) =>
err(
new AnalysisCapabilityUnavailableError(
KUNA_PROVIDER_ID,
operation,
configuration.error.reason,
),
),
err(kunaUnavailable(operation, configuration.error.reason)),
close: async () => ok(null),
};
return new KunaSessionClient(target, profile, {
+2 -3
View File
@@ -1,7 +1,6 @@
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisCapabilityUnavailableError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import type { JsonValue } from "../domain/jsonValue.js";
@@ -21,6 +20,7 @@ import {
} from "./KunaReferenceFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
import type { KunaPendingQuery, KunaQueryWork } from "./KunaQueryWork.js";
import { kunaUnavailable } from "./KunaUnavailable.js";
/** Charge raw JSON bytes conservatively; retained projections are smaller. */
const XREF_CACHE_BYTES = 16 * 1024 * 1024;
@@ -158,8 +158,7 @@ export class KunaReferenceQueries {
const node = graph.nodes.get(address);
if (node === undefined)
return err(
new AnalysisCapabilityUnavailableError(
KUNA_PROVIDER_ID,
kunaUnavailable(
operation,
`Kuna's graph has no function node at ${address}; its relationships are unknown.`,
),
+4 -17
View File
@@ -69,6 +69,7 @@ import {
KunaProjectReader,
type KunaProjectFileSpan,
} from "./KunaProjectReader.js";
import { kunaUnavailable } from "./KunaUnavailable.js";
/** How often a request waiting on the project export reports that it still waits. */
const WAIT_PROGRESS_INTERVAL_MS = 10_000;
@@ -278,13 +279,7 @@ export class KunaSessionClient implements AnalysisClient {
options: ExecutionOptions = {},
): Promise<Result<AnalysisExecution, AnalysisError>> {
if (this.#closing)
return err(
new AnalysisCapabilityUnavailableError(
KUNA_PROVIDER_ID,
operation,
"This Kuna session is closed.",
),
);
return err(kunaUnavailable(operation, "This Kuna session is closed."));
return this.#own(
this.#execute(operation, parameters, {
...options,
@@ -312,8 +307,7 @@ export class KunaSessionClient implements AnalysisClient {
return err(new AnalysisCancelledError(operation));
if (operation !== "health" && !isKunaOperation(operation))
return err(
new AnalysisCapabilityUnavailableError(
KUNA_PROVIDER_ID,
kunaUnavailable(
operation,
"Kuna answers procedure and string queries, pseudocode, assembly and references; this operation needs another provider.",
),
@@ -1229,14 +1223,7 @@ export class KunaSessionClient implements AnalysisClient {
this.#installation ??= inspectKunaInstallation(this.#environment).then(
(inspected): Result<KunaInstallation, AnalysisError> => {
if (!inspected.ok)
return err(
new AnalysisCapabilityUnavailableError(
KUNA_PROVIDER_ID,
"health",
inspected.error.reason,
{ userMessage: inspected.error.reason },
),
);
return err(kunaUnavailable("health", inspected.error.reason));
const committed = this.#profile?.parameters["kuna_sha256"];
if (committed !== undefined && committed !== inspected.value.sha256)
return err(
+18
View File
@@ -0,0 +1,18 @@
import { AnalysisCapabilityUnavailableError } from "../domain/analysisErrorCore.js";
import type { AnalysisErrorOptions } from "../domain/analysisErrorBase.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
/**
* A Kuna capability gap whose reason is what the caller needs to read. Without
* a user message the projection shows only "choose another analysis or
* target", which hides facts such as "no code at this address".
*/
export const kunaUnavailable = (
operation: string,
reason: string,
options: AnalysisErrorOptions = {},
): AnalysisCapabilityUnavailableError =>
new AnalysisCapabilityUnavailableError(KUNA_PROVIDER_ID, operation, reason, {
...options,
userMessage: reason,
});
@@ -850,18 +850,18 @@ describe("application workflow CLI parity", () => {
if (seed === undefined)
throw new TypeError("Expected at least one semantic relation");
const traced = await runCli([
const traced = await runCliWithRequestFile(
root,
"trace-javascript-semantics",
JSON.stringify({
{
application: analyzed.value,
query: {
seed: { kind: "semantic-node", node_id: seed },
direction: "forward-influence",
include_ambiguous_dynamic_edges: true,
},
}),
"--json",
]);
},
);
expect(traced).toMatchObject({
operation: "trace_javascript_semantics",
predicate_type: "rea.javascript-semantic-trace",
@@ -1300,18 +1300,18 @@ describe("application workflow CLI export Evidence", () => {
'export default async function make() { return { kind: "record", extra: 1 }; }',
];
const [left, right] = await analyzeComparisonSources(root, sources);
const compared = await runCli([
const compared = await runCliWithRequestFile(
root,
"compare-javascript-export-shapes",
JSON.stringify({
{
left,
right,
left_module_path: "parser.mjs",
left_export_name: "default",
right_module_path: "parser.mjs",
right_export_name: "default",
}),
"--json",
]);
},
);
expect(compared).toMatchObject({
operation: "compare_javascript_export_shapes",
normalized_result: {
@@ -1640,3 +1640,19 @@ const runCli = async (
throw cause;
}
};
/**
* Pass a request carrying complete Evidence through the CLI's file-backed JSON
* input. Serialized Evidence exceeds the Windows command-line limit when it is
* passed as one argument, so spawn fails with ENAMETOOLONG before the workflow
* starts. Inline JSON transport keeps its own coverage in the parity tests.
*/
const runCliWithRequestFile = async (
root: string,
command: string,
request: unknown,
): Promise<unknown> => {
const requestPath = join(root, "request.json");
await writeFile(requestPath, JSON.stringify(request));
return runCli([command, requestPath, "--json"]);
};
@@ -4,9 +4,10 @@ import { join } from "node:path";
import { promisify } from "node:util";
import { buildBinary } from "plist";
import { describe, expect, it } from "vitest";
import { describe, expect, it, vi } from "vitest";
import { analyzeInterfaceBuilderBundle } from "../../../src/artifacts/apple/InterfaceBuilderAnalysis.js";
import { InterfaceBuilderDecodeBudget } from "../../../src/artifacts/apple/InterfaceBuilderDecodeBudget.js";
import { encodeNibArchiveFixture } from "../../../src/artifacts/apple/NibArchive.fixture.js";
import { thinMach } from "../../../src/domain/binaryTarget.fixture.js";
import { xcodeIbtoolAvailable } from "../../fixtures/ibtool.js";
@@ -907,6 +908,57 @@ describe("aggregate Interface Builder archive retention budget", () => {
});
});
describe("Interface Builder XML plist depth", () => {
it("bounds XML archives by plist value depth, not element depth", async () => {
const root = await createTestTempDirectory("rea-ib-depth-test-");
const bundle = join(root, "Example.app");
const resources = join(bundle, "Contents", "Resources");
await mkdir(resources, { recursive: true });
// The root dict is value depth 0, so the leaf sits at `nested + 1`. The
// plist envelope and keys add XML elements but not value depth.
const document = (nested: number) =>
`<?xml version="1.0"?><plist version="1.0"><dict>${"<key>child</key><dict>".repeat(nested)}<key>leaf</key><integer>1</integer>${"</dict>".repeat(nested)}</dict></plist>`;
await writeFile(join(resources, "AtLimit.nib"), document(127));
await writeFile(join(resources, "BeyondLimit.nib"), document(128));
const release = vi.spyOn(InterfaceBuilderDecodeBudget.prototype, "release");
const result = await analyzeInterfaceBuilderBundle({
bundlePath: bundle,
targetSha256: "e".repeat(64),
});
const released = release.mock.calls.map(([bytes]) => bytes);
release.mockRestore();
// The rejected archive returns its reservation to later documents.
expect(released).toHaveLength(1);
expect(released[0]).toBeGreaterThan(0);
expect(result.documents.map(({ relative_path }) => relative_path)).toEqual([
"Contents/Resources/AtLimit.nib",
]);
expect(result.graph.coverage).toContainEqual(
expect.objectContaining({
facet: "archive_decode",
status: "partial",
reason: "aggregate_decode_budget_exhausted",
examined: 2,
omitted: 1,
}),
);
expect(result.limitations).toContain(
"Contents/Resources/BeyondLimit.nib: omitted because aggregate decode budget exhausted.",
);
});
it("restores released representation bytes up to the initial budget", () => {
const budget = new InterfaceBuilderDecodeBudget(100);
budget.reserve(60, "reserve");
budget.release(60);
expect(budget.remainingBytes).toBe(100);
budget.release(10);
expect(budget.remainingBytes).toBe(100);
});
});
describe("bounded Interface Builder archive decoding", () => {
it("counts malformed archives against the document limit", async () => {
const root = await createTestTempDirectory("rea-ib-test-");
@@ -0,0 +1,90 @@
import { execFile } from "node:child_process";
import { copyFile, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { pathToFileURL } from "node:url";
import { promisify } from "node:util";
import { expect, it } from "vitest";
const execute = promisify(execFile);
it("runs the caller-selected npm entry through Node without rewriting its inputs", async () => {
const root = await mkdtemp(join(tmpdir(), "rea-npm-invocation-"));
try {
const npmEntry = join(root, "selected npm entry.mjs");
await writeFile(
npmEntry,
`process.stdout.write(JSON.stringify({ arguments: process.argv.slice(2), cwd: process.cwd(), marker: process.env.REA_NPM_TEST_MARKER }));\n`,
);
const helper = pathToFileURL(
resolve("scripts/lib/verify-package-core.mjs"),
).href;
const program = `
const { execNpm } = await import(${JSON.stringify(helper)});
const result = await execNpm(["pack", "value with spaces"], {
cwd: ${JSON.stringify(root)},
env: { ...process.env, npm_execpath: ${JSON.stringify(npmEntry)}, REA_NPM_TEST_MARKER: "caller-selected" },
});
process.stdout.write(result.stdout);
`;
const result = await execute(
process.execPath,
["--input-type=module", "--eval", program],
{ windowsHide: true },
);
expect(JSON.parse(result.stdout)).toEqual({
arguments: ["pack", "value with spaces"],
cwd: root,
marker: "caller-selected",
});
} finally {
await rm(root, { recursive: true, force: true });
}
});
const windowsIt = process.platform === "win32" ? it : it.skip;
windowsIt(
"uses the npm entry adjacent to Node when none is selected",
async () => {
const root = await mkdtemp(join(tmpdir(), "rea-adjacent-npm-"));
try {
const node = join(root, "node.exe");
const npmEntry = join(root, "node_modules/npm/bin/npm-cli.js");
await mkdir(dirname(npmEntry), { recursive: true });
await Promise.all([
copyFile(process.execPath, node),
writeFile(
npmEntry,
`process.stdout.write(JSON.stringify({ arguments: process.argv.slice(2), cwd: process.cwd() }));\n`,
),
]);
const helper = pathToFileURL(
resolve("scripts/lib/verify-package-core.mjs"),
).href;
const program = `
const { execNpm } = await import(${JSON.stringify(helper)});
const environment = { ...process.env };
delete environment.npm_execpath;
const result = await execNpm(["pack", "value with spaces"], {
cwd: ${JSON.stringify(root)},
env: environment,
});
process.stdout.write(result.stdout);
`;
const environment = { ...process.env };
delete environment.npm_execpath;
const result = await execute(
node,
["--input-type=module", "--eval", program],
{ env: environment, windowsHide: true },
);
expect(JSON.parse(result.stdout)).toEqual({
arguments: ["pack", "value with spaces"],
cwd: root,
});
} finally {
await rm(root, { recursive: true, force: true });
}
},
);
@@ -10,6 +10,10 @@ import {
rizinTestInferiorTty,
} from "./RizinTestInferiorTty.js";
import { stopOwnedDebuggerProcess } from "../../../src/process/ownedDebuggerSession.js";
import {
ProviderProcessSupervisor,
spawnOwnedProviderProcess,
} from "../../../src/process/ProviderProcess.js";
import type {
ProviderProcessHandle,
SpawnedOwnedProviderProcess,
@@ -24,10 +28,11 @@ interface InjectedProcessControl {
exit(signal?: NodeJS.Signals): void;
}
type RizinTestManagerOptions = NonNullable<
ConstructorParameters<typeof RizinDebugSessionManager>[1]
>;
type InjectedSpawnProcess = NonNullable<
NonNullable<
ConstructorParameters<typeof RizinDebugSessionManager>[1]
>["spawnProcess"]
RizinTestManagerOptions["spawnProcess"]
>;
export const incompleteStop: typeof stopOwnedDebuggerProcess = async (
@@ -176,17 +181,47 @@ export const createInjectedProcess = (
return { spawnProcess, control };
};
export const createRizinCoverageHarness = () => {
let directory: string | undefined;
export const createRizinCoverageHarness = (
controls: {
readonly stopOwnedProcess?: (
supervisor: ProviderProcessSupervisor,
) => ReturnType<ProviderProcessSupervisor["stop"]>;
} = {},
) => {
let latestDirectory: string | undefined;
const directories = new Set<string>();
const managers: RizinDebugSessionManager[] = [];
const track = (manager: RizinDebugSessionManager): void => {
const ownedProcesses = new Set<ProviderProcessSupervisor>();
const spawnOwnedProcess: InjectedSpawnProcess = async (options) => {
const launched = await spawnOwnedProviderProcess(options);
// The fixture owns these fake children independently of the manager's
// intentionally uncertain debugger/inferior lifecycle.
ownedProcesses.add(
new ProviderProcessSupervisor(
{ ...launched, ownsProcessLifetime: true },
{ captureStdout: false, maxDiagnosticBytes: 0 },
),
);
return launched;
};
const createManager: typeof createRizinTestManager = (
environment = process.env,
options = {},
) => {
const manager = createRizinTestManager(environment, {
...options,
spawnProcess: options.spawnProcess ?? spawnOwnedProcess,
});
managers.push(manager);
return manager;
};
const makeProvider = async (
body: string,
shebang = "#!/usr/bin/env node",
): Promise<string> => {
directory = await mkdtemp(join(tmpdir(), "rea-rizin-coverage-"));
const directory = await mkdtemp(join(tmpdir(), "rea-rizin-coverage-"));
directories.add(directory);
latestDirectory = directory;
const provider = join(directory, "fake-rizin.mjs");
await writeFile(provider, `${shebang}\n${body}\n`);
await chmod(provider, 0o700);
@@ -194,13 +229,9 @@ export const createRizinCoverageHarness = () => {
};
const start = async (
provider: string,
options: ConstructorParameters<typeof RizinDebugSessionManager>[1] = {},
options: RizinTestManagerOptions = {},
): Promise<{ manager: RizinDebugSessionManager; sessionId: string }> => {
const manager = createRizinTestManager(
{ REA_RIZIN_COMMAND: provider },
options,
);
track(manager);
const manager = createManager({ REA_RIZIN_COMMAND: provider }, options);
const result = await manager.start({
path: "/tmp/coverage-target",
inferior_tty: await rizinTestInferiorTty(),
@@ -210,13 +241,12 @@ export const createRizinCoverageHarness = () => {
};
const startInjected = async (
spawnProcess: InjectedSpawnProcess,
options: ConstructorParameters<typeof RizinDebugSessionManager>[1] = {},
options: RizinTestManagerOptions = {},
) => {
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{ ...options, spawnProcess },
);
track(manager);
const started = await manager.start({
path: "/tmp/injected",
inferior_tty: await rizinTestInferiorTty(),
@@ -229,17 +259,42 @@ export const createRizinCoverageHarness = () => {
.splice(0)
.map((manager) => manager.closeAll().catch(() => undefined)),
);
if (directory !== undefined)
const failures: Error[] = [];
for (const supervisor of ownedProcesses) {
const stopped =
controls.stopOwnedProcess === undefined
? await supervisor.stop()
: await controls.stopOwnedProcess(supervisor);
if (stopped.status === "incomplete") {
failures.push(
new Error(`Fake Rizin cleanup incomplete: ${stopped.reason}`),
);
} else if (supervisor.observedExit() === undefined) {
failures.push(
new Error("Fake Rizin cleanup did not observe child exit"),
);
} else {
ownedProcesses.delete(supervisor);
}
}
// Keep ownership and fixture files available when cleanup needs a retry.
if (failures.length > 0)
throw new AggregateError(failures, "Fake Rizin fixture cleanup failed");
for (const directory of directories) {
await rm(directory, { recursive: true, force: true });
directory = undefined;
directories.delete(directory);
}
latestDirectory = undefined;
};
const missingProviderPath = (): string => join(directory!, "does-not-exist");
const missingProviderPath = (): string =>
join(latestDirectory!, "does-not-exist");
return {
cleanup,
makeProvider,
missingProviderPath,
start,
startInjected,
track,
createManager,
spawnOwnedProcess,
};
};
@@ -1,12 +1,13 @@
import { randomUUID } from "node:crypto";
import { access } from "node:fs/promises";
import { afterEach, expect, it, vi } from "vitest";
import { RizinDebugSessionManager } from "../../../src/rizin/RizinDebugSessionManager.js";
import {
createRizinTestManager,
stopFakeNoInferior,
rizinTestInferiorTty,
} from "./RizinTestInferiorTty.js";
import { rizinCommand } from "../../../src/rizin/RizinCommand.js";
import { spawnOwnedProviderProcess } from "../../../src/process/ProviderProcess.js";
import * as providerProcess from "../../../src/process/ProviderProcess.js";
import type { SpawnedOwnedProviderProcess } from "../../../src/process/ProviderProcess.js";
import {
createRizinCoverageHarness,
createInjectedProcess,
@@ -20,7 +21,8 @@ const {
missingProviderPath,
start,
startInjected,
track,
createManager,
spawnOwnedProcess,
} = createRizinCoverageHarness();
afterEach(cleanup);
@@ -40,7 +42,7 @@ it.skipIf(process.platform === "win32")(
it.skipIf(process.platform === "win32")(
"rejects unsupported, pre-cancelled, and failed startup requests",
async () => {
const windows = createRizinTestManager({}, { platform: "win32" });
const windows = createManager({}, { platform: "win32" });
expect(
(
await windows.start({
@@ -61,20 +63,20 @@ it.skipIf(process.platform === "win32")(
).toBe(false);
const failed = await makeProvider("process.exit(9);");
const manager = createRizinTestManager({
const manager = createManager({
REA_RIZIN_COMMAND: failed,
});
track(manager);
const result = await manager.start({
path: "/tmp/a",
inferior_tty: await rizinTestInferiorTty(),
});
expect(result.ok).toBe(false);
const missing = createRizinTestManager({
const missing = createManager({
REA_RIZIN_COMMAND: missingProviderPath(),
});
track(missing);
const spawnFailure = await missing.start({
path: "/tmp/a",
inferior_tty: await rizinTestInferiorTty(),
@@ -87,10 +89,10 @@ it.skipIf(process.platform === "win32")(
"returns a startup timeout when Rizin exits before its initial frame",
async () => {
const provider = await makeProvider("process.exit(0);");
const manager = createRizinTestManager({
const manager = createManager({
REA_RIZIN_COMMAND: provider,
});
track(manager);
const result = await manager.start({
path: "/tmp/early-exit",
inferior_tty: await rizinTestInferiorTty(),
@@ -110,14 +112,14 @@ it.skipIf(process.platform === "win32")(
startup: () => undefined,
beforeSpawnReturns: resolveSpawned,
});
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{
spawnProcess: injected.spawnProcess,
stopProcess: async () => ({ status: "stopped" }),
},
);
track(manager);
vi.useFakeTimers({ toFake: ["setTimeout", "clearTimeout"] });
try {
const starting = manager.start({
@@ -139,7 +141,7 @@ it.skipIf(process.platform === "win32")(
"does not spawn Rizin when cancellation arrives during TTY validation",
async () => {
let spawned = false;
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{
spawnProcess: async () => {
@@ -164,7 +166,7 @@ it.skipIf(process.platform === "win32")(
"rejects a non-TTY path before spawning Rizin",
async () => {
let spawned = false;
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{
spawnProcess: async () => {
@@ -195,7 +197,7 @@ it.skipIf(process.platform === "win32")(
releaseSpawns = resolve;
});
const injected = createInjectedProcess();
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{
spawnProcess: async (options) => {
@@ -240,10 +242,9 @@ it.skipIf(process.platform === "win32")(
const provider = await makeProvider(
'process.stdout.write("x".repeat(16 * 1024 * 1024 + 1) + "\\0"); process.stdin.resume();',
);
const manager = createRizinTestManager({
const manager = createManager({
REA_RIZIN_COMMAND: provider,
});
track(manager);
const result = await manager.start({
path: "/tmp/oversized-startup",
@@ -264,19 +265,19 @@ it.skipIf(process.platform === "win32")(
const provider = await makeProvider(
'setTimeout(() => process.stdout.write("\\0"), 500); process.stdin.resume();',
);
const manager = createRizinTestManager(
const manager = createManager(
{
REA_RIZIN_COMMAND: provider,
},
{
spawnProcess: async (options) => {
const launched = await spawnOwnedProviderProcess(options);
const launched = await spawnOwnedProcess(options);
resolveSpawned();
return launched;
},
},
);
track(manager);
const abort = new AbortController();
const starting = manager.start(
{ path: "/tmp/cancel-start", inferior_tty: await rizinTestInferiorTty() },
@@ -288,39 +289,138 @@ it.skipIf(process.platform === "win32")(
},
);
it.skipIf(process.platform === "win32")(
"preserves startup cleanup uncertainty after cancellation",
async () => {
it
.skipIf(process.platform === "win32")
.each(["is uncertain", "reports stopped before exit"] as const)(
"cleans startup fixtures after cancellation when manager stop %s",
async (stopMode) => {
const provider = await makeProvider(
"process.stdin.resume(); setInterval(() => {}, 1000);",
);
let resolveSpawned!: () => void;
const spawned = new Promise<void>((resolve) => {
resolveSpawned = resolve;
});
const provider = await makeProvider(
let ownedLaunch: SpawnedOwnedProviderProcess | undefined;
const nativeSpawn = providerProcess.spawnOwnedProviderProcess;
const spy = vi
.spyOn(providerProcess, "spawnOwnedProviderProcess")
.mockImplementation(async (options) => {
const launched = await nativeSpawn(options);
ownedLaunch = launched;
resolveSpawned();
return launched;
});
try {
const stopProcess: typeof incompleteStop =
stopMode === "is uncertain"
? incompleteStop
: async () => ({ status: "stopped" });
const manager = createManager(
{ REA_RIZIN_COMMAND: provider },
{ stopProcess },
);
const abort = new AbortController();
const starting = manager.start(
{
path: "/tmp/cancel-start",
inferior_tty: await rizinTestInferiorTty(),
},
abort.signal,
);
await spawned;
abort.abort();
const result = await starting;
expect(result.ok).toBe(false);
if (!result.ok && stopMode === "is uncertain")
expect(result.error.message).toContain("injected stop uncertainty");
await cleanup();
if (ownedLaunch === undefined)
throw new Error("Fixture launch was not observed");
expect(
ownedLaunch.process.exitCode !== null ||
ownedLaunch.process.signalCode !== null,
"harness returned before fixture child exit",
).toBe(true);
} finally {
spy.mockRestore();
// A failing regression must also reap its observed, directly owned child.
if (
ownedLaunch !== undefined &&
ownedLaunch.process.exitCode === null &&
ownedLaunch.process.signalCode === null
) {
const safety = new providerProcess.ProviderProcessSupervisor(
{ ...ownedLaunch, ownsProcessLifetime: true },
{ captureStdout: false, maxDiagnosticBytes: 0 },
);
expect((await safety.stop()).status).not.toBe("incomplete");
expect(safety.observedExit()).toBeDefined();
}
}
},
);
it.skipIf(process.platform === "win32")(
"retains owned fixture processes and files until a failed cleanup can complete",
async () => {
let failCleanup = true;
const fixture = createRizinCoverageHarness({
stopOwnedProcess: async (supervisor) =>
failCleanup
? { status: "incomplete", reason: "injected fixture stop failure" }
: supervisor.stop(),
});
const provider = await fixture.makeProvider(
"process.stdin.resume(); setInterval(() => {}, 1000);",
);
const manager = createRizinTestManager(
{ REA_RIZIN_COMMAND: provider },
{
stopProcess: incompleteStop,
spawnProcess: async (options) => {
const launched = await spawnOwnedProviderProcess(options);
resolveSpawned();
return launched;
},
},
);
track(manager);
const abort = new AbortController();
const starting = manager.start(
{ path: "/tmp/cancel-start", inferior_tty: await rizinTestInferiorTty() },
abort.signal,
);
await spawned;
abort.abort();
const result = await starting;
expect(result.ok).toBe(false);
if (!result.ok)
expect(result.error.message).toContain("injected stop uncertainty");
const launched = await fixture.spawnOwnedProcess({
command: provider,
arguments: [],
runId: randomUUID(),
stdin: "pipe",
});
try {
await expect(fixture.cleanup()).rejects.toMatchObject({
message: "Fake Rizin fixture cleanup failed",
errors: [
expect.objectContaining({
message: expect.stringContaining("injected fixture stop failure"),
}),
],
});
expect(launched.process.exitCode).toBeNull();
expect(launched.process.signalCode).toBeNull();
await access(provider);
const nextProvider = await fixture.makeProvider("process.exit(0);");
failCleanup = false;
await fixture.cleanup();
expect(
launched.process.exitCode !== null ||
launched.process.signalCode !== null,
).toBe(true);
await expect(access(provider)).rejects.toMatchObject({ code: "ENOENT" });
await expect(access(nextProvider)).rejects.toMatchObject({
code: "ENOENT",
});
} finally {
failCleanup = false;
try {
await fixture.cleanup();
} finally {
if (
launched.process.exitCode === null &&
launched.process.signalCode === null
) {
const safety = new providerProcess.ProviderProcessSupervisor(
{ ...launched, ownsProcessLifetime: true },
{ captureStdout: false, maxDiagnosticBytes: 0 },
);
expect((await safety.stop()).status).not.toBe("incomplete");
expect(safety.observedExit()).toBeDefined();
}
}
}
},
);
@@ -1,9 +1,5 @@
import { afterEach, expect, it } from "vitest";
import { RizinDebugSessionManager } from "../../../src/rizin/RizinDebugSessionManager.js";
import {
createRizinTestManager,
rizinTestInferiorTty,
} from "./RizinTestInferiorTty.js";
import { rizinTestInferiorTty } from "./RizinTestInferiorTty.js";
import {
createRizinCoverageHarness,
abortOnCheck,
@@ -11,7 +7,7 @@ import {
incompleteStop,
} from "./RizinDebugSessionCoverage.fixtures.js";
const { cleanup, makeProvider, start, startInjected, track } =
const { cleanup, makeProvider, start, startInjected, createManager } =
createRizinCoverageHarness();
afterEach(cleanup);
@@ -88,7 +84,7 @@ it.skipIf(process.platform === "win32")(
async () => {
for (const spawnError of [new Error("injected spawn failure"), "opaque"]) {
const injected = createInjectedProcess({ spawnError });
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{ spawnProcess: injected.spawnProcess },
);
@@ -102,7 +98,7 @@ it.skipIf(process.platform === "win32")(
const injected = createInjectedProcess({
startup: (control) => control.closeStdin(),
});
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{ spawnProcess: injected.spawnProcess },
);
@@ -124,7 +120,7 @@ it.skipIf(process.platform === "win32")(
beforeSpawnReturns: () => abort.abort(),
spawnError: new Error("spawn interrupted"),
});
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{ spawnProcess: injected.spawnProcess },
);
@@ -147,7 +143,7 @@ it.skipIf(process.platform === "win32")(
abort.abort();
},
});
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{ spawnProcess: injected.spawnProcess },
);
@@ -166,7 +162,7 @@ it.skipIf(process.platform === "win32")(
const injected = createInjectedProcess({
beforeSpawnReturns: () => abort.abort(),
});
const manager = createRizinTestManager(
const manager = createManager(
{ REA_RIZIN_COMMAND: "injected-rizin" },
{ spawnProcess: injected.spawnProcess },
);
@@ -1,17 +1,13 @@
import { afterEach, expect, it, vi } from "vitest";
import { AnalysisInputError } from "../../../src/domain/analysisErrorCore.js";
import { RizinDebugSessionManager } from "../../../src/rizin/RizinDebugSessionManager.js";
import {
createRizinTestManager,
rizinTestInferiorTty,
} from "./RizinTestInferiorTty.js";
import { rizinTestInferiorTty } from "./RizinTestInferiorTty.js";
import {
createRizinCoverageHarness,
abortOnCheck,
createInjectedProcess,
} from "./RizinDebugSessionCoverage.fixtures.js";
const { cleanup, startInjected, track } = createRizinCoverageHarness();
const { cleanup, startInjected, createManager } = createRizinCoverageHarness();
afterEach(cleanup);
it.skipIf(process.platform === "win32")(
@@ -1,17 +1,13 @@
import { afterEach, expect, it } from "vitest";
import { evidenceSchema } from "../../../src/domain/evidence.js";
import { RizinDebugSessionManager } from "../../../src/rizin/RizinDebugSessionManager.js";
import {
createRizinTestManager,
rizinTestInferiorTty,
} from "./RizinTestInferiorTty.js";
import { rizinTestInferiorTty } from "./RizinTestInferiorTty.js";
import {
createRizinCoverageHarness,
createInjectedProcess,
incompleteStop,
} from "./RizinDebugSessionCoverage.fixtures.js";
const { cleanup, makeProvider, start, startInjected, track } =
const { cleanup, makeProvider, start, startInjected, createManager } =
createRizinCoverageHarness();
afterEach(cleanup);
@@ -80,3 +80,41 @@ it("reports a graph beyond REA's buffer once per session instead of rebuilding i
expect(await client.close()).toEqual({ ok: true, value: null });
}
});
it("says why Kuna cannot answer instead of a generic unavailable message", async () => {
const { open } = await session("rea-kuna-unavailable-reasons-");
const client = open();
try {
const cases = [
{
operation: "procedure_assembly",
parameters: { procedure: "0x2000" },
message:
"Kuna's export lists no code at 0x2000; import slots and data entries have no listing.",
},
{
operation: "procedure_callers",
parameters: { procedure: "0x1004" },
message:
"Kuna's graph has no function node at 0x1004; its relationships are unknown.",
},
{
operation: "list_segments",
parameters: {},
message:
"Kuna answers procedure and string queries, pseudocode, assembly and references; this operation needs another provider.",
},
] as const;
for (const { operation, parameters, message } of cases) {
const result = await client.execute(operation, parameters);
if (result.ok) throw new Error(`${operation} unexpectedly succeeded`);
expect(projectAnalysisError(result.error)).toMatchObject({
code: "capability_unavailable",
message,
retryable: false,
});
}
} finally {
expect(await client.close()).toEqual({ ok: true, value: null });
}
});
+17 -2
View File
@@ -1,10 +1,25 @@
import type { MipsElfMetadata } from "../../../src/domain/binaryTargetTypes.js";
import type {
MipsAbiFlags,
MipsElfMetadata,
MipsGnuAttributes,
} from "../../../src/domain/binaryTargetTypes.js";
import type { JsonValue } from "../../../src/domain/jsonValue.js";
export type MipsReadelfMetadata = MipsElfMetadata & {
readonly abiFlags: MipsAbiFlags;
readonly gnuAttributes: MipsGnuAttributes | null;
};
export function expectedMipsElfProfile(
mips: MipsReadelfMetadata,
): Readonly<Record<string, JsonValue>>;
export type MipsFloatAbi = "hard" | "soft";
export interface MipsReadelfFacts {
mips: MipsElfMetadata;
mips: MipsReadelfMetadata & { readonly gnuAttributes: null };
symbols: Record<string, string>;
}
export function parseMipsReadelfSectionTypes(text: string): readonly string[];
export function parseMipsReadelf(
text: string,
byteOrder: string,
+65 -1
View File
@@ -23,6 +23,65 @@ const SOFT_FLOAT_SYMBOLS = [
"rea_mips_soft_caller",
];
/** Complete indexed ELF32 section types shared by the independent fixture readers. */
export function parseMipsReadelfSectionTypes(text) {
const counts = [
...text.matchAll(/^\s*Number of section headers:\s*(\d+)\s*$/gmu),
];
assert.equal(counts.length, 1, "Missing or ambiguous readelf section count");
const sectionCount = Number(counts[0][1]);
assert.ok(sectionCount > 0, "Fixture must have a section header table");
const sections = [...text.matchAll(/^[ \t]*\[[ \t]*(\d+)\][ \t]+(.+)$/gmu)];
assert.equal(
sections.length,
sectionCount,
"Incomplete readelf section table",
);
return sections.map((section, index) => {
assert.equal(Number(section[1]), index, "Ambiguous readelf section index");
const columns =
/^(?:\S+[ \t]+)?(\S+)[ \t]+[\da-f]{8}[ \t]+[\da-f]+[ \t]+[\da-f]+[ \t]+[\da-f]{2}[ \t]+(?:[A-Za-z]+[ \t]+)?\d+[ \t]+\d+[ \t]+\d+[ \t]*$/u.exec(
section[2],
);
assert.ok(columns !== null, "Malformed readelf section header");
return columns[1];
});
}
/** Independent wire-format expectation shared by generic and PSP snapshots. */
export function expectedMipsElfProfile(mips) {
const abi = mips.abiFlags;
assert.ok(abi != null, "Fixture ABI declarations must be inspected");
assert.notEqual(
mips.gnuAttributes,
undefined,
"Fixture GNU attributes must be inspected",
);
return {
elf_class: mips.elfClass,
byte_order: mips.byteOrder,
type: mips.type,
flags: mips.flags,
gnu_attributes:
mips.gnuAttributes === null
? null
: { fp_abi: mips.gnuAttributes.fpAbi, msa: mips.gnuAttributes.msa },
abi_flags: {
version: abi.version,
isa_level: abi.isaLevel,
isa_revision: abi.isaRevision,
gpr_size: abi.gprSize,
cpr1_size: abi.cpr1Size,
cpr2_size: abi.cpr2Size,
fp_abi: abi.fpAbi,
isa_extension: abi.isaExtension,
ases: abi.ases,
flags1: abi.flags1,
flags2: abi.flags2,
},
};
}
/**
* Parse only the documented GNU readelf fields needed by our tiny fixtures.
* `rel` objects carry their own symbol checks in mips-relocatable-checks.mjs.
@@ -51,6 +110,10 @@ export function parseMipsReadelf(
objectType === "rel" ? /^REL\b/u : /^EXEC\b/u,
);
assert.match(field(/^\s*Machine:\s*(.+)$/gmu, "machine"), /^MIPS\b/u);
assert.ok(
!parseMipsReadelfSectionTypes(text).includes("GNU_ATTRIBUTES"),
"Fixture unexpectedly declares GNU attributes",
);
const flags = Number(field(/^\s*Flags:\s*(0x[\da-f]+),.*$/gmu, "flags"));
assert.equal(flags, 0x70001001, "Fixture ELF flags differ from build intent");
assert.equal(field(/^ISA:\s*(.+)$/gmu, "ISA"), "MIPS32r2");
@@ -123,6 +186,7 @@ export function parseMipsReadelf(
type: objectType === "rel" ? 1 : 2,
flags,
abiFlags,
gnuAttributes: null,
},
symbols,
};
@@ -144,7 +208,7 @@ export async function inspectMipsReadelf(path, byteOrder, floatAbi = "hard") {
);
const result = await execute(
command,
["-h", "-A", "-s", "-W", path],
["-h", "-A", "-S", "-s", "-W", path],
options,
);
assert.equal(
+106 -2
View File
@@ -7,7 +7,7 @@ import {
parseMipsReadelf,
} from "./mips-checks.mjs";
// GNU readelf 2.40 output for the source-owned Clang 17 / LLD fixture.
// GNU readelf 2.44 output for the source-owned Clang 19 / LLD 19 fixture.
// This tests the reader boundary, not Ghidra; the real lane rebuilds both orders.
const readelf = `ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
@@ -21,7 +21,7 @@ const readelf = `ELF Header:
Version: 0x1
Entry point address: 0x201b0
Start of program headers: 52 (bytes into file)
Start of section headers: 2996 (bytes into file)
Start of section headers: 2624 (bytes into file)
Flags: 0x70001001, noreorder, o32, mips32r2
Size of this header: 52 (bytes)
Size of program headers: 32 (bytes)
@@ -30,6 +30,38 @@ const readelf = `ELF Header:
Number of section headers: 24
Section header string table index: 22
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .MIPS.abiflags MIPS_ABIFLAGS 00010118 000118 000018 18 A 0 0 8
[ 2] .reginfo MIPS_REGINFO 00010130 000130 000018 18 A 0 0 4
[ 3] .rodata PROGBITS 00010148 000148 00001e 00 A 0 0 1
[ 4] .text PROGBITS 00020170 000170 0000f0 00 AX 0 0 16
[ 5] .data PROGBITS 00030260 000260 000000 00 WA 0 0 16
[ 6] .got PROGBITS 00030260 000260 000008 00 WAp 0 0 16
[ 7] .sdata PROGBITS 00030268 000268 000004 00 WAp 0 0 4
[ 8] .bss NOBITS 00030270 00026c 000000 00 WA 0 0 16
[ 9] .mdebug.abi32 PROGBITS 00000000 00026c 000000 00 0 0 1
[10] .pdr PROGBITS 00000000 00026c 000060 00 0 0 4
[11] .debug_abbrev MIPS_DWARF 00000000 0002cc 0000b3 00 0 0 1
[12] .debug_info MIPS_DWARF 00000000 00037f 00015e 00 0 0 1
[13] .debug_str_offsets MIPS_DWARF 00000000 0004dd 000040 00 0 0 1
[14] .debug_str MIPS_DWARF 00000000 00051d 0000ef 01 MS 0 0 1
[15] .debug_addr MIPS_DWARF 00000000 00060c 000018 00 0 0 1
[16] .comment PROGBITS 00000000 000624 00003e 01 MS 0 0 1
[17] .debug_frame MIPS_DWARF 00000000 000664 00004c 00 0 0 4
[18] .debug_line MIPS_DWARF 00000000 0006b0 0000f8 00 0 0 1
[19] .debug_line_str MIPS_DWARF 00000000 0007a8 000098 01 MS 0 0 1
[20] .debug_aranges MIPS_DWARF 00000000 000840 000020 00 0 0 1
[21] .symtab SYMTAB 00000000 000860 000090 10 23 4 4
[22] .shstrtab STRTAB 00000000 0008f0 0000f0 00 0 0 1
[23] .strtab STRTAB 00000000 0009e0 00005f 00 0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
D (mbind), p (processor specific)
Symbol table '.symtab' contains 9 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 00000000 0 NOTYPE LOCAL DEFAULT UND
@@ -54,6 +86,15 @@ ASEs:
None
FLAGS 1: 00000000
FLAGS 2: 00000000
Static GOT:
Canonical gp value: 00038250
Reserved entries:
Address Access Value
00030260 -32752(gp) 00000000
00030264 -32748(gp) 80000000
`;
const sha256 = "a".repeat(64);
const image = {
@@ -94,6 +135,7 @@ describe("MIPS real-verifier independent expectations", () => {
elfClass: 32,
byteOrder: "little",
flags: 0x70001001,
gnuAttributes: null,
abiFlags: {
isaLevel: 32,
isaRevision: 2,
@@ -229,6 +271,67 @@ describe("MIPS real-verifier independent expectations", () => {
});
});
describe("MIPS real-verifier GNU attribute coverage", () => {
it.each(["exec", "rel"] as const)(
"reports verified GNU attribute absence for both byte orders of %s fixtures",
(objectType) => {
for (const byteOrder of ["little", "big"]) {
const text = readelf
.replace("little endian", `${byteOrder} endian`)
.replace(
"EXEC (Executable file)",
objectType === "rel"
? "REL (Relocatable file)"
: "EXEC (Executable file)",
);
expect(
parseMipsReadelf(text, byteOrder, "hard", objectType).mips
.gnuAttributes,
).toBeNull();
}
},
);
it("rejects GNU attribute sections even when their name is unrelated", () => {
expect(() =>
parseMipsReadelf(
readelf.replace(
".comment PROGBITS",
".comment GNU_ATTRIBUTES",
),
"little",
),
).toThrow(/GNU attributes/u);
});
it("uses the section type rather than its name to establish absence", () => {
expect(
parseMipsReadelf(
readelf.replace(
".comment PROGBITS",
".gnu.attributes PROGBITS",
),
"little",
).mips.gnuAttributes,
).toBeNull();
});
it.each([
[
"missing table",
readelf.replace(
/Section Headers:[\s\S]*?Key to Flags:/u,
"Key to Flags:",
),
],
["missing entry", readelf.replace(/^.*\[ 5\].*\n/gmu, "")],
["duplicate index", readelf.replace("[ 5]", "[ 4]")],
["malformed entry", readelf.replace(/^.*\[ 5\].*$/gmu, " [ 5] malformed")],
])("refuses GNU attribute absence from a %s", (_name, text) => {
expect(() => parseMipsReadelf(text, "little")).toThrow(/section/u);
});
});
describe("MIPS soft-float real-verifier expectations", () => {
const softReadelf = readelf
.replace("CPR1 size: 32", "CPR1 size: 0")
@@ -249,6 +352,7 @@ describe("MIPS soft-float real-verifier expectations", () => {
it("reads the soft-float ABI record and its fixture symbols", () => {
const parsed = parseMipsReadelf(softReadelf, "little", "soft");
expect(parsed.mips.abiFlags).toMatchObject({ cpr1Size: 0, fpAbi: 3 });
expect(parsed.mips.gnuAttributes).toBeNull();
expect(parsed.symbols.rea_mips_soft_mixed).toBe("0x20380");
});
+142 -14
View File
@@ -1,6 +1,9 @@
import assert from "node:assert/strict";
import { it } from "vitest";
import { parsePspReadelfAbi } from "./psp-fixture.mjs";
import { expectedMipsElfProfile } from "./mips-checks.mjs";
import { ghidraMipsProfileParameters } from "../../../src/ghidra/GhidraMipsProfile.js";
import type { BinaryTarget } from "../../../src/domain/binaryTargetTypes.js";
import { parsePspReadelfMips } from "./psp-fixture.mjs";
// GNU readelf 2.44 output from the maintained PSPDEV v20261001 fixture.
const observed = `MIPS ABI Flags Version: 0
@@ -16,19 +19,42 @@ FLAGS 1: 00000001
FLAGS 2: 00000000
`;
// Constructed section-table controls combined with the captured ABI declaration;
// this does not establish a real PSPSDK or Allegrex-provider run.
const metadata = `ELF Header:
Class: ELF32
Data: 2's complement, little endian
Type: EXEC (Executable file)
Machine: MIPS R3000
Flags: 0x10a23001, noreorder, allegrex, eabi32, mips2
Number of section headers: 3
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .MIPS.abiflags MIPS_ABIFLAGS 08800118 000118 000018 18 A 0 0 8
[ 2] .text PROGBITS 08800130 000130 000010 00 AX 0 0 16
${observed}`;
it("reads the independent PSP producer's single-float declaration", () => {
assert.deepEqual(parsePspReadelfAbi(observed), {
version: 0,
isaLevel: 2,
isaRevision: 0,
gprSize: 1,
cpr1Size: 1,
cpr2Size: 0,
fpAbi: 2,
isaExtension: 0,
ases: 0,
flags1: 1,
flags2: 0,
assert.deepEqual(parsePspReadelfMips(metadata), {
elfClass: 32,
byteOrder: "little",
type: 2,
flags: 0x10a23001,
gnuAttributes: null,
abiFlags: {
version: 0,
isaLevel: 2,
isaRevision: 0,
gprSize: 1,
cpr1Size: 1,
cpr2Size: 0,
fpAbi: 2,
isaExtension: 0,
ases: 0,
flags1: 1,
flags2: 0,
},
});
});
@@ -46,6 +72,108 @@ it.each([
])(
"rejects unsupported or ambiguous independent ABI output: %s",
(before, after) => {
assert.throws(() => parsePspReadelfAbi(observed.replace(before, after)));
assert.throws(() => parsePspReadelfMips(metadata.replace(before, after)));
},
);
it.each([
[
"missing table",
metadata.replace(
/Section Headers:[\s\S]*?MIPS ABI Flags/u,
"MIPS ABI Flags",
),
],
["incomplete table", metadata.replace(/^.*\[ 2\].*\n/gmu, "")],
["duplicate index", metadata.replace("[ 2]", "[ 1]")],
[
"GNU section under another name",
metadata.replace(
".text PROGBITS",
".text GNU_ATTRIBUTES",
),
],
])("refuses PSP GNU attribute absence from a %s", (_name, text) => {
assert.throws(() => parsePspReadelfMips(text), /section|GNU attributes/u);
});
it.each([
["Class: ELF32", "Class: ELF64"],
["little endian", "big endian"],
["Type: EXEC", "Type: REL"],
["Flags: 0x10a23001", "Flags: 0x70001001"],
["Class: ELF32", "Class: ELF32\n Class: ELF32"],
])("refuses contradictory or ambiguous PSP headers: %s", (before, after) => {
assert.throws(() => parsePspReadelfMips(metadata.replace(before, after)));
});
// GNU readelf 2.40 independently decoded this declaration from a constructed
// ELF32/PSP-shaped control, not a PSPSDK build or a real Allegrex import.
const gnuDeclaration = `Attribute Section: gnu
File Attributes
Tag_GNU_MIPS_ABI_FP: Hard float (single precision)
`;
const gnuMetadata = metadata.replace(
".text PROGBITS",
".text GNU_ATTRIBUTES",
);
it.each(["before", "after"])(
"retains the independent single-float GNU declaration %s the ABI dump",
(position) => {
const text =
position === "before"
? gnuMetadata.replace(observed, `${gnuDeclaration}\n${observed}`)
: `${gnuMetadata}\n${gnuDeclaration}`;
assert.deepEqual(parsePspReadelfMips(text), {
...parsePspReadelfMips(metadata),
gnuAttributes: { fpAbi: 2, msa: null },
});
},
);
it.each([
["unknown vendor", gnuDeclaration.replace("gnu", "aeabi")],
["wrong float ABI", gnuDeclaration.replace("single", "double")],
["unknown tag", `${gnuDeclaration} Tag_GNU_Unknown: 1\n`],
[
"unsupported scope",
gnuDeclaration.replace("File Attributes", "Section Attributes: 1"),
],
["duplicate declaration", `${gnuDeclaration}${gnuDeclaration}`],
])("refuses %s rather than inventing PSP GNU facts", (_name, declaration) => {
assert.throws(
() => parsePspReadelfMips(`${gnuMetadata}\n${declaration}`),
/GNU attributes/u,
);
});
it("refuses undeclared GNU attributes rather than reporting absence", () => {
assert.throws(
() => parsePspReadelfMips(`${metadata}\n${gnuDeclaration}`),
/GNU attributes/u,
);
});
it.each([
["absent", metadata],
["declared", `${gnuMetadata}\n${gnuDeclaration}`],
])(
"matches production snapshot serialization for %s GNU facts",
(_name, text) => {
const mips = parsePspReadelfMips(text);
const target = {
path: "/fixture/psp.elf",
sha256: "a".repeat(64),
kind: "executable",
format: "elf",
architecture: "mips",
availableArchitectures: ["mips"],
mips,
} satisfies BinaryTarget;
assert.deepEqual(
expectedMipsElfProfile(mips),
ghidraMipsProfileParameters(target).mips_elf,
);
},
);
+2 -2
View File
@@ -1,3 +1,3 @@
import type { MipsAbiFlags } from "../../../src/domain/binaryTargetTypes.js";
import type { MipsReadelfMetadata } from "./mips-checks.mjs";
export declare function parsePspReadelfAbi(text: string): MipsAbiFlags;
export declare function parsePspReadelfMips(text: string): MipsReadelfMetadata;
+56 -16
View File
@@ -4,6 +4,7 @@ import { createHash } from "node:crypto";
import { copyFile, readFile } from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
import { parseMipsReadelfSectionTypes } from "./mips-checks.mjs";
const execute = promisify(execFile);
const sha256 = (bytes) => createHash("sha256").update(bytes).digest("hex");
@@ -42,15 +43,11 @@ export async function buildPspFixture(directory, globalValue = 7) {
assert.equal(bytes.readUInt32LE(36), 0x10a23001);
const readelf = await execute(
"psp-readelf",
["-h", "-A", "-sW", path],
["-h", "-A", "-S", "-sW", path],
options,
);
assert.equal(readelf.stderr.trim(), "");
assert.match(readelf.stdout, /Class:\s+ELF32/u);
assert.match(readelf.stdout, /Data:\s+2's complement, little endian/u);
assert.match(readelf.stdout, /Type:\s+EXEC\b/u);
assert.match(readelf.stdout, /Flags:\s+0x10a23001\b/u);
const abiFlags = parsePspReadelfAbi(readelf.stdout);
const mips = parsePspReadelfMips(readelf.stdout);
const abiDump = await execute(
"psp-readelf",
["-x", ".MIPS.abiflags", path],
@@ -99,13 +96,7 @@ export async function buildPspFixture(directory, globalValue = 7) {
source_sha256: sourceDigests,
symbols,
flags: "0x10a23001",
mips: {
elfClass: 32,
byteOrder: "little",
type: 2,
flags: 0x10a23001,
abiFlags,
},
mips,
abi_record_hex_dump: abiDump.stdout,
global_value: globalValue,
versions,
@@ -115,8 +106,8 @@ export async function buildPspFixture(directory, globalValue = 7) {
};
}
/** Independent GNU fields for the pinned PSPSDK fixture, not REA's parser. */
export function parsePspReadelfAbi(text) {
/** Complete independent fields for the pinned PSPSDK fixture, not REA's parser. */
export function parsePspReadelfMips(text) {
const field = (expression, label) => {
const matches = [...text.matchAll(expression)];
assert.equal(
@@ -126,6 +117,13 @@ export function parsePspReadelfAbi(text) {
);
return matches[0][1].trim();
};
assert.equal(field(/^\s*Class:\s*(.+)$/gmu, "class"), "ELF32");
assert.equal(
field(/^\s*Data:.*?,\s*(little|big) endian\s*$/gmu, "byte order"),
"little",
);
assert.match(field(/^\s*Type:\s*(.+)$/gmu, "type"), /^EXEC\b/u);
assert.match(field(/^\s*Flags:\s*(.+)$/gmu, "flags"), /^0x10a23001\b/u);
const width = (name) => {
const bits = Number(
field(new RegExp(`^${name} size:\\s*(\\d+)\\s*$`, "gmu"), name),
@@ -178,7 +176,49 @@ export function parsePspReadelfAbi(text) {
},
"PSPSDK ABI declaration differs from independently inspected build intent",
);
return abi;
return {
elfClass: 32,
byteOrder: "little",
type: 2,
flags: 0x10a23001,
abiFlags: abi,
gnuAttributes: parsePspGnuAttributes(text),
};
}
// The maintained PSP fixture is single-float. Retain that independently decoded
// GNU declaration when present, and refuse other tags/vendors/scopes rather
// than confusing an uninspected or unsupported declaration with absence.
function parsePspGnuAttributes(text) {
const count = parseMipsReadelfSectionTypes(text).filter(
(type) => type === "GNU_ATTRIBUTES",
).length;
const headers = [...text.matchAll(/^Attribute Section: (.+)$/gmu)];
if (count === 0) {
assert.equal(
headers.length,
0,
"GNU attributes without a declared section",
);
return null;
}
assert.equal(count, 1, "Ambiguous GNU attributes sections");
assert.equal(headers.length, 1, "Missing or ambiguous GNU attributes output");
const header = headers[0];
assert.equal(header[1].trim(), "gnu", "Unsupported GNU attributes vendor");
const tail = text.slice(header.index + header[0].length);
// GNU attributes may precede or follow the architecture-specific ABI dump.
const end = tail.search(/^MIPS ABI Flags Version:/mu);
const lines = (end < 0 ? tail : tail.slice(0, end))
.split(/\r?\n/u)
.map((line) => line.trim())
.filter(Boolean);
assert.deepEqual(
lines,
["File Attributes", "Tag_GNU_MIPS_ABI_FP: Hard float (single precision)"],
"Unsupported or incomplete PSP GNU attributes declaration",
);
return { fpAbi: 2, msa: null };
}
/** Observations from Ghidra, not an echo of the requested profile. */
@@ -69,6 +69,12 @@ describe("curated Windows test lane", () => {
"tests/boundary/process/windowsOutputCapture.test.ts",
);
});
it("keeps the adjacent npm fallback regression in the lane", async () => {
expect(await explicitPaths("windows")).toContain(
"tests/boundary/process/packageNpmInvocation.test.ts",
);
});
});
it("keeps every explicit focused CI test path executable after file moves", async () => {