Merge latest main to include the verified uninstall fixture corrections
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
This commit is contained in:
commit
7a9991289c
77 files changed
+3984
-364
No files matched your search
@@ -100,15 +100,16 @@ limits, security/privacy/containment implications, and meaningful proof gaps. --
|
||||
The PR author is the owner by default. A different owner must explicitly accept
|
||||
ownership on this PR.
|
||||
AI agents must leave these declarations to a human to dictate to them or there could be catastrophic consequences for many users of REA.
|
||||
Draft PRs may leave them unchecked. After further changes, clear the final checkbox
|
||||
until the owner reviews the updated diff and refreshes affected validation.
|
||||
Draft PRs may leave them unchecked. These declarations record initial readiness
|
||||
when opening the PR for review; later pushes do not require a new human sign-off.
|
||||
Keep the description and validation evidence current as the work changes.
|
||||
See CONTRIBUTING.md#pull-request-ownership-and-readiness. -->
|
||||
|
||||
Human owner: <!-- @GitHub-handle -->
|
||||
|
||||
- [ ] I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
|
||||
- [ ] I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
|
||||
- [ ] I declare the current revision ready for maintainer review.
|
||||
- [ ] I declare this PR ready for initial maintainer review.
|
||||
|
||||
## Before merge
|
||||
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
name: PR readiness
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
branches: [main, "release/*"]
|
||||
types:
|
||||
[
|
||||
opened,
|
||||
edited,
|
||||
synchronize,
|
||||
reopened,
|
||||
ready_for_review,
|
||||
converted_to_draft,
|
||||
]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
pull_request_number:
|
||||
description: Open PR to recheck
|
||||
required: true
|
||||
type: string
|
||||
|
||||
permissions: {}
|
||||
|
||||
concurrency:
|
||||
group: pr-readiness-${{ github.event.pull_request.number || inputs.pull_request_number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
readiness:
|
||||
name: Validate author declaration
|
||||
runs-on: ubuntu-24.04
|
||||
timeout-minutes: 3
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
statuses: write
|
||||
steps:
|
||||
- uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0
|
||||
with:
|
||||
ref: ${{ github.event.repository.default_branch }}
|
||||
persist-credentials: false
|
||||
- name: Check initial readiness
|
||||
uses: actions/github-script@373c709c69115d41ff229c7e5df9f8788daa9553 # v9
|
||||
with:
|
||||
script: |
|
||||
const { pathToFileURL } = require("node:url");
|
||||
const entry = pathToFileURL(
|
||||
`${process.env.GITHUB_WORKSPACE}/scripts/ci/pr-readiness.mjs`,
|
||||
);
|
||||
const { runReadinessCheck } = await import(entry.href);
|
||||
await runReadinessCheck({ github, context, core });
|
||||
@@ -94,7 +94,7 @@ required to open an issue.
|
||||
Follow [human ownership and readiness](CONTRIBUTING.md#pull-request-ownership-and-readiness)
|
||||
when preparing a PR. AI-assisted contributions are welcome; a human must review
|
||||
and own the final diff, check the validation evidence and its limitations, and
|
||||
explicitly declare the current revision ready for maintainer review.
|
||||
explicitly declare the PR ready for initial maintainer review.
|
||||
|
||||
Keep the PR body concise and current, with the concrete problem, observable
|
||||
outcome, approach, and evidence. Report only checks that actually ran as executed;
|
||||
@@ -103,11 +103,11 @@ Address actionable automated review findings and explain disagreements.
|
||||
|
||||
Leave the template's **Author readiness** declarations unchecked for the human
|
||||
owner to complete. Never attest to human review on their behalf. Keep the PR in
|
||||
draft until the human declares the current diff ready. After further changes,
|
||||
clear the ready-for-review checkbox and refresh affected evidence so the human
|
||||
can review and renew the declaration. CI and automated review results support
|
||||
review; they do not establish human readiness, maintainer scope approval, or
|
||||
merge permission.
|
||||
draft until the human declares the PR initially ready. Later pushes do not
|
||||
invalidate that declaration or require another human sign-off. Keep the PR body
|
||||
and affected validation current as work changes. The readiness check validates
|
||||
the filled declaration; CI and automated review results do not establish that
|
||||
human review occurred, maintainer scope approval, or merge permission.
|
||||
|
||||
<!-- BEGIN:turborepo-agent-rules -->
|
||||
|
||||
|
||||
+18
-5
@@ -67,7 +67,7 @@ Before requesting maintainer review, the human owner must:
|
||||
passing. Follow the [verification requirements](#development-feedback-and-pr-verification)
|
||||
for required checks and real-provider coverage.
|
||||
- Complete the **Author readiness** declaration in the PR template, explicitly
|
||||
stating that the current revision is ready for review.
|
||||
stating that the PR is ready for initial review.
|
||||
|
||||
Keep the PR description concise and current. Explain the problem, observable
|
||||
outcome, approach, and useful evidence; link long logs or put them in optional
|
||||
@@ -78,10 +78,23 @@ approval or permission to merge.
|
||||
|
||||
Draft PRs may request early feedback with open questions and incomplete checks.
|
||||
Leave the readiness declaration unchecked until the human owner has completed
|
||||
it. After further changes, clear the ready-for-review checkbox until the owner
|
||||
reviews the updated diff, refreshes affected validation and the PR description,
|
||||
and renews the declaration. AI agents must leave human declarations to the human
|
||||
owner and must not attest to a human review on their behalf.
|
||||
it. The declaration records initial readiness; later pushes do not invalidate
|
||||
it or require another human sign-off. Keep the PR description and affected
|
||||
validation current as work changes. AI agents must leave human declarations to
|
||||
the human owner and must not attest to a human review on their behalf.
|
||||
|
||||
The `PR readiness` check reports pending for drafts and requires a named human
|
||||
owner and all three template declarations for non-drafts. It checks that the
|
||||
declaration is filled, not whether the code or evidence is correct. Existing PRs
|
||||
can copy the **Author readiness** section from the current template.
|
||||
Open PRs must have distinct head commits because GitHub shares commit statuses.
|
||||
|
||||
Maintainers enable merge enforcement after the workflow is merged: run
|
||||
`gh workflow run pr-readiness.yml -f pull_request_number=NUMBER` for existing
|
||||
open PRs, then add `PR readiness` from GitHub Actions to the branch's required
|
||||
status checks alongside its existing checks. Deploy the workflow to each
|
||||
protected target branch before requiring it there. The workflow reads PR
|
||||
metadata using trusted default-branch code and writes only a commit status.
|
||||
|
||||
Maintainers may return a PR to the author or close it when the problem is
|
||||
unsupported, validation is missing, or the owner cannot explain the change.
|
||||
|
||||
@@ -26,6 +26,9 @@ Readiness is `rea inspect-adb-client`: it runs `adb version` and reports the
|
||||
resolved binary, its version, and where it was selected from. A missing
|
||||
binary reports `capability_unavailable` with the `REA_ADB_PATH` remediation
|
||||
rather than attempting an installation.
|
||||
The `binary_session` inventory uses the same bounded client-version probe; it
|
||||
does not query devices, start the adb server, or establish that a requested
|
||||
serial is reachable. Device state is checked by each device operation.
|
||||
|
||||
## Answer an analyst question
|
||||
|
||||
|
||||
@@ -21,6 +21,8 @@ Readiness is `rea inspect-apktool-client`: it runs `apktool --version` and
|
||||
reports the resolved launcher, its selection source, and its version. A
|
||||
missing launcher reports `capability_unavailable` with the
|
||||
`REA_APKTOOL_COMMAND` remediation rather than attempting an installation.
|
||||
`binary_session` uses the same host probe for tool availability; it does not
|
||||
validate a particular APK or guarantee that the selected input can be decoded.
|
||||
|
||||
On Windows, a `.bat` or `.cmd` launcher (such as Chocolatey's `apktool.bat`)
|
||||
runs through an explicit `cmd.exe /d /s /c` call. That call has no AutoRun,
|
||||
|
||||
+12
-6
@@ -639,12 +639,18 @@ memory has no original file offset. For FAT Mach-O, offsets refer to the origina
|
||||
container file. Results retain Hopper's image-relative offset, the observed slice
|
||||
base, and the source executable path.
|
||||
Loader selection reads the Mach-O container header, including FAT files with a
|
||||
single architecture. For FAT64, REA validates the architecture table and selected
|
||||
Mach-O header, prepares a private thin image, and loads it with Hopper's native
|
||||
Mach-O loader. It checks the entire source's SHA-256 while copying the slice;
|
||||
source identity and reported offsets still refer to the original container.
|
||||
An ambiguous architecture subtype requires an explicitly extracted thin image;
|
||||
REA does not guess. Configured loader arguments remain explicit overrides.
|
||||
single architecture. For FAT32 and FAT64 containers, REA validates the
|
||||
architecture table and selected Mach-O header, prepares a private thin image,
|
||||
and loads it with Hopper's native Mach-O loader instead of Hopper's implicit
|
||||
FAT slice choice. For arm64 targets, REA ranks subtype bases 0 (plain arm64) and
|
||||
2 (arm64e), ignoring capability bits for ranking while retaining the raw subtype
|
||||
for slice identity. These slices' function-entry encodings decode under
|
||||
`--aarch64`, ahead of Apple's arm64e.x1 variant (base subtype 12), which is
|
||||
selected only when it is the arm64 family's only candidate. REA checks the
|
||||
entire source's SHA-256 while copying the slice; source identity and reported
|
||||
offsets still refer to the original container. Any other ambiguous architecture
|
||||
subtype requires an explicitly extracted thin image; REA does not guess.
|
||||
Configured loader arguments remain explicit overrides.
|
||||
FAT64 is distinct from the CPU architecture: FAT32 containers can contain 64-bit
|
||||
executable images. This preparation avoids the Raw Binary loader dialog observed
|
||||
with native FAT64 loading on Hopper 6.1.0-demo.
|
||||
|
||||
+14
-3
@@ -383,13 +383,24 @@ exit comparison like `settlement.elapsed_ms`.
|
||||
Every capture requires `truncation_details`, with separate accounting for:
|
||||
|
||||
- `raw_terminal`: original UTF-8 PTY chunk bytes and observed/retained chunk
|
||||
counts. A chunk that does not fit `limits.output_bytes` is omitted whole;
|
||||
a later smaller chunk can still fit.
|
||||
counts. A chunk that does not fit either its remaining original-text budget
|
||||
or normalized-text budget is omitted whole; a later smaller chunk can still
|
||||
fit. A normalized-text omission also omits its original chunk, so raw and
|
||||
normalized coverage can both be incomplete for that event.
|
||||
- `normalized_terminal`: normalized comparison-text bytes retained from raw
|
||||
chunks. This is optional when reading historical captures. A raw-fitting
|
||||
chunk is omitted if an ordered normalization stage would exceed the remaining
|
||||
`output_bytes` budget before the final normalized size is known. Its final
|
||||
size is unknown, not assumed to equal the budget. The capture records the
|
||||
omitted-frame count and a limitation; later smaller chunks remain eligible.
|
||||
- `rendered_terminal`: cumulative UTF-8 serialized-state and visible-line
|
||||
bytes, plus observed/retained frame counts. This is independent of the raw
|
||||
chunk budget and does not count JSON encoding. Repeated screen snapshots can
|
||||
exhaust it even when all raw output fits. Rendered states use retained PTY
|
||||
input, so raw omissions also limit rendered coverage.
|
||||
input, so raw omissions also limit rendered coverage. When bounded
|
||||
normalization omits a rendered frame before its final size is known,
|
||||
`unknown_size_frames` reports that count; `observed_bytes` includes only
|
||||
rendered frame sizes that were determined exactly.
|
||||
- `filesystem_before` and `filesystem_after`: file-count/depth limits,
|
||||
enumeration failures, whole-file hash budget and bytes successfully hashed.
|
||||
Directory names are read incrementally within the remaining entry capacity.
|
||||
|
||||
@@ -10,6 +10,7 @@ const HOST_TEST_OWNERS = ["apple", "package", "sqlite"];
|
||||
// workflows' release, publication and self-hosted runner safeguards.
|
||||
const GUARDED_WORKFLOWS = new Set([
|
||||
".github/workflows/pr-title.yml",
|
||||
".github/workflows/pr-readiness.yml",
|
||||
".github/workflows/ci-full-pr.yml",
|
||||
".github/workflows/release.yml",
|
||||
".github/workflows/real-ghidra-windows.yml",
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
export interface ReadinessPullRequest {
|
||||
readonly number: number;
|
||||
readonly state: "open" | "closed";
|
||||
readonly draft: boolean;
|
||||
readonly body: string | null;
|
||||
readonly head: { readonly sha: string };
|
||||
readonly base: { readonly ref: string };
|
||||
}
|
||||
export interface ReadinessResult {
|
||||
readonly state: "success" | "failure" | "pending";
|
||||
readonly description: string;
|
||||
}
|
||||
export interface ReadinessGitHub {
|
||||
readonly rest: {
|
||||
readonly pulls: {
|
||||
get(input: {
|
||||
owner: string;
|
||||
repo: string;
|
||||
pull_number: number;
|
||||
}): Promise<{ data: ReadinessPullRequest }>;
|
||||
};
|
||||
readonly repos: {
|
||||
createCommitStatus(
|
||||
input: ReadinessResult & {
|
||||
owner: string;
|
||||
repo: string;
|
||||
sha: string;
|
||||
context: string;
|
||||
target_url: string;
|
||||
},
|
||||
): Promise<unknown>;
|
||||
readonly listPullRequestsAssociatedWithCommit: unknown;
|
||||
};
|
||||
};
|
||||
paginate(
|
||||
method: unknown,
|
||||
input: {
|
||||
owner: string;
|
||||
repo: string;
|
||||
commit_sha: string;
|
||||
per_page: number;
|
||||
},
|
||||
): Promise<
|
||||
readonly Pick<ReadinessPullRequest, "number" | "state" | "head">[]
|
||||
>;
|
||||
}
|
||||
export function evaluateReadiness(
|
||||
pull: Pick<ReadinessPullRequest, "body" | "draft">,
|
||||
template: string,
|
||||
): ReadinessResult;
|
||||
export function runReadinessCheck(input: {
|
||||
github: ReadinessGitHub;
|
||||
context: {
|
||||
repo: { owner: string; repo: string };
|
||||
payload: {
|
||||
pull_request?: { number: number };
|
||||
inputs?: { pull_request_number?: string };
|
||||
};
|
||||
serverUrl: string;
|
||||
runId: number;
|
||||
};
|
||||
core: { info(message: string): void; setFailed(message: string): void };
|
||||
}): Promise<ReadinessResult | undefined>;
|
||||
@@ -0,0 +1,173 @@
|
||||
import { readFile } from "node:fs/promises";
|
||||
|
||||
const result = (state, description) => ({ state, description });
|
||||
|
||||
function readinessLines(markdown) {
|
||||
const lines = markdown.split(/\r?\n/u);
|
||||
const visible = [];
|
||||
let fence;
|
||||
let comment = false;
|
||||
for (const source of lines) {
|
||||
if (fence) {
|
||||
if (
|
||||
new RegExp(
|
||||
`^ {0,3}${fence.character}{${fence.length},}\\s*$`,
|
||||
"u",
|
||||
).test(source)
|
||||
)
|
||||
fence = undefined;
|
||||
continue;
|
||||
}
|
||||
const opening = !comment && /^ {0,3}(`{3,}|~{3,})/u.exec(source);
|
||||
if (opening) {
|
||||
fence = { character: opening[1][0], length: opening[1].length };
|
||||
continue;
|
||||
}
|
||||
let line = "";
|
||||
let offset = 0;
|
||||
while (offset < source.length) {
|
||||
const marker = comment ? "-->" : "<!--";
|
||||
const index = source.indexOf(marker, offset);
|
||||
if (!comment)
|
||||
line += source.slice(offset, index === -1 ? source.length : index);
|
||||
if (index === -1) break;
|
||||
offset = index + marker.length;
|
||||
comment = !comment;
|
||||
}
|
||||
visible.push(line);
|
||||
}
|
||||
const starts = visible.flatMap((line, index) =>
|
||||
/^## Author readiness\s*$/u.test(line) ? [index] : [],
|
||||
);
|
||||
if (starts.length !== 1) return [];
|
||||
const remaining = visible.slice(starts[0] + 1);
|
||||
const end = remaining.findIndex((line) => /^#{1,2}\s/u.test(line));
|
||||
return (end === -1 ? remaining : remaining.slice(0, end)).map((line) =>
|
||||
line.trimEnd(),
|
||||
);
|
||||
}
|
||||
|
||||
/** Check the initial declaration against the trusted repository template. */
|
||||
export function evaluateReadiness(pull, template) {
|
||||
if (pull.draft)
|
||||
return result(
|
||||
"pending",
|
||||
"Draft PR: complete Author readiness before requesting review.",
|
||||
);
|
||||
const expected = readinessLines(template).flatMap((line) => {
|
||||
const match = /^- \[ \] (.+)$/u.exec(line);
|
||||
return match ? [match[1]] : [];
|
||||
});
|
||||
if (expected.length !== 3)
|
||||
throw new Error(
|
||||
"Trusted PR template must contain three readiness declarations.",
|
||||
);
|
||||
const lines = readinessLines(pull.body ?? "");
|
||||
const owners = lines.filter((line) => line.startsWith("Human owner:"));
|
||||
if (
|
||||
owners.length !== 1 ||
|
||||
!/^Human owner: @[a-z\d](?:[a-z\d-]*[a-z\d])?$/iu.test(owners[0])
|
||||
)
|
||||
return result(
|
||||
"failure",
|
||||
"Fill Human owner with one @GitHub-handle in the Author readiness section.",
|
||||
);
|
||||
const declarations = lines.flatMap((line) => {
|
||||
const match = /^- \[([ xX])\] (.+)$/u.exec(line);
|
||||
return match
|
||||
? [{ checked: match[1].toLowerCase() === "x", text: match[2] }]
|
||||
: [];
|
||||
});
|
||||
if (
|
||||
expected.some((text) => {
|
||||
const matches = declarations.filter((item) => item.text === text);
|
||||
return matches.length !== 1 || !matches[0].checked;
|
||||
})
|
||||
)
|
||||
return result(
|
||||
"failure",
|
||||
"Complete all three Author readiness declarations from the current PR template.",
|
||||
);
|
||||
return result(
|
||||
"success",
|
||||
"Human owner and all three initial readiness declarations are filled.",
|
||||
);
|
||||
}
|
||||
|
||||
/** Publish readiness on the PR head without running contributor code. */
|
||||
export async function runReadinessCheck({ github, context, core }) {
|
||||
const number =
|
||||
context.payload.pull_request?.number ??
|
||||
Number(context.payload.inputs?.pull_request_number);
|
||||
if (!Number.isSafeInteger(number) || number <= 0)
|
||||
throw new Error("A positive pull request number is required.");
|
||||
const repository = context.repo;
|
||||
const getPull = async () =>
|
||||
(await github.rest.pulls.get({ ...repository, pull_number: number })).data;
|
||||
const pull = await getPull();
|
||||
if (pull.state !== "open" || !/^(?:main|release\/.+)$/u.test(pull.base.ref))
|
||||
return;
|
||||
const publish = (sha, verdict) =>
|
||||
github.rest.repos.createCommitStatus({
|
||||
...repository,
|
||||
sha,
|
||||
context: "PR readiness",
|
||||
...verdict,
|
||||
target_url: `${context.serverUrl}/${repository.owner}/${repository.repo}/actions/runs/${context.runId}`,
|
||||
});
|
||||
await publish(
|
||||
pull.head.sha,
|
||||
result("pending", "Checking the initial author readiness declaration."),
|
||||
);
|
||||
const template = await readFile(
|
||||
new URL("../../.github/pull_request_template.md", import.meta.url),
|
||||
"utf8",
|
||||
);
|
||||
let verdict = evaluateReadiness(pull, template);
|
||||
if (verdict.state === "success") {
|
||||
const associated = await github.paginate(
|
||||
github.rest.repos.listPullRequestsAssociatedWithCommit,
|
||||
{ ...repository, commit_sha: pull.head.sha, per_page: 100 },
|
||||
);
|
||||
if (
|
||||
!associated.some(
|
||||
(other) =>
|
||||
other.number === number &&
|
||||
other.state === "open" &&
|
||||
other.head.sha === pull.head.sha,
|
||||
)
|
||||
)
|
||||
verdict = result(
|
||||
"pending",
|
||||
"GitHub has not confirmed this PR's head association; rerun PR readiness.",
|
||||
);
|
||||
else if (
|
||||
associated.some(
|
||||
(other) =>
|
||||
other.state === "open" &&
|
||||
other.number !== number &&
|
||||
other.head.sha === pull.head.sha,
|
||||
)
|
||||
)
|
||||
verdict = result(
|
||||
"failure",
|
||||
"Open PRs must have distinct head commits because GitHub shares commit statuses.",
|
||||
);
|
||||
}
|
||||
const latest = await getPull();
|
||||
if (latest.state !== "open") return;
|
||||
if (
|
||||
latest.head.sha !== pull.head.sha ||
|
||||
latest.body !== pull.body ||
|
||||
latest.draft !== pull.draft ||
|
||||
latest.base.ref !== pull.base.ref
|
||||
)
|
||||
verdict = result(
|
||||
"pending",
|
||||
"PR changed during the check; wait for the latest run or rerun PR readiness.",
|
||||
);
|
||||
await publish(latest.head.sha, verdict);
|
||||
core.info(verdict.description);
|
||||
if (verdict.state === "failure") core.setFailed(verdict.description);
|
||||
return verdict;
|
||||
}
|
||||
@@ -55,4 +55,12 @@ describe("parseAdbVersionOutput", () => {
|
||||
installedPath: null,
|
||||
});
|
||||
});
|
||||
|
||||
it("does not accept arbitrary successful output as an adb version", () => {
|
||||
expect(
|
||||
parseAdbVersionOutput(
|
||||
"Android Debug Bridge version tomorrow\nVersion unknown\n",
|
||||
),
|
||||
).toMatchObject({ version: null });
|
||||
});
|
||||
});
|
||||
@@ -67,7 +67,9 @@ export interface AdbClientIdentity {
|
||||
* `Version ` the build version (no colon), and `Installed as` the binary path.
|
||||
*/
|
||||
export const parseAdbVersionOutput = (stdout: string): AdbClientIdentity => {
|
||||
const build = /^Version\s+(\S.*)$/mu.exec(stdout);
|
||||
const build = /^Version\s+(\d+\.\d+\.\d+(?:[-+][0-9A-Za-z.-]+)?)\s*$/mu.exec(
|
||||
stdout,
|
||||
);
|
||||
const installed = /^Installed as\s+(.+)$/mu.exec(stdout);
|
||||
return {
|
||||
version: build === null ? null : build[1]!.trim(),
|
||||
@@ -84,14 +86,34 @@ export const inspectAdbClient = async (
|
||||
signal?: AbortSignal,
|
||||
): Promise<AdbClientIdentity> => {
|
||||
try {
|
||||
const { stdout } = await execFileOutput(selection.binary, ["version"], {
|
||||
timeout: VERSION_TIMEOUT_MS,
|
||||
stopSignal: "SIGTERM",
|
||||
maxBuffer: VERSION_MAX_OUTPUT_BYTES,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
});
|
||||
return parseAdbVersionOutput(stdout);
|
||||
const { stdout, stderr } = await execFileOutput(
|
||||
selection.binary,
|
||||
["version"],
|
||||
{
|
||||
timeout: VERSION_TIMEOUT_MS,
|
||||
stopSignal: "SIGTERM",
|
||||
maxBuffer: VERSION_MAX_OUTPUT_BYTES,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
},
|
||||
);
|
||||
const identity = parseAdbVersionOutput(stdout);
|
||||
const hasRecognizedProtocolVersion =
|
||||
/^Android Debug Bridge version\s+\d+\.\d+\.\d+\s*$/mu.test(stdout);
|
||||
if (identity.version === null && !hasRecognizedProtocolVersion)
|
||||
throw new AdbConfigurationFailure(
|
||||
"version_unresolved",
|
||||
"The selected adb binary exited successfully but did not report a recognizable protocol or build version",
|
||||
{
|
||||
selected_binary: selection.binary,
|
||||
path_source: selection.pathSource,
|
||||
version_probe: "unrecognized_output",
|
||||
stdout,
|
||||
stderr,
|
||||
},
|
||||
);
|
||||
return identity;
|
||||
} catch (cause) {
|
||||
if (cause instanceof AdbConfigurationFailure) throw cause;
|
||||
const failure = execFileOutputFailure(cause);
|
||||
if (signal?.aborted || failure?.code === "ABORT_ERR")
|
||||
throw new AnalysisCancelledError("inspect_adb_client", { cause });
|
||||
|
||||
@@ -91,6 +91,36 @@ it("inspects the adb client without contacting the server", async () => {
|
||||
expect(result.contacted_adb_server).toBe(false);
|
||||
});
|
||||
|
||||
it("fails closed when adb exits successfully without a recognized version", async () => {
|
||||
const stdout = `unexpected adb output ${"x".repeat(700)}\n`;
|
||||
const stderr = `probe stderr ${"y".repeat(700)}\n`;
|
||||
const { binary } = await writeAdbStub(
|
||||
() => `printf '%s' '${stdout}'; printf '%s' '${stderr}' >&2; exit 0`,
|
||||
);
|
||||
await expect(
|
||||
providerOver(binary).inspectAvailability(),
|
||||
).resolves.toMatchObject({
|
||||
status: "unavailable",
|
||||
code: "version_unresolved",
|
||||
diagnostics: {
|
||||
version_probe: "unrecognized_output",
|
||||
stdout,
|
||||
stderr,
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts adb's recognized protocol version when the build version is absent", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
() => `printf 'Android Debug Bridge version 1.0.41\\n'`,
|
||||
);
|
||||
await expect(
|
||||
providerOver(binary).inspectAvailability(),
|
||||
).resolves.toMatchObject({
|
||||
status: "available",
|
||||
});
|
||||
});
|
||||
|
||||
it("lists devices with inferred kinds and observed daemon startup", async () => {
|
||||
const { binary } = await writeAdbStub(
|
||||
() => `if [ "$1" = "version" ]; then ${VERSION_BLOCK}; exit 0; fi
|
||||
|
||||
@@ -58,7 +58,28 @@ describe("parseApktoolVersionOutput", () => {
|
||||
});
|
||||
|
||||
it("reports null when the launcher prints nothing usable", () => {
|
||||
expect(parseApktoolVersionOutput(" \n")).toEqual({ version: null });
|
||||
expect(parseApktoolVersionOutput(" \n")).toEqual({
|
||||
version: null,
|
||||
reportedVersion: null,
|
||||
});
|
||||
expect(parseApktoolVersionOutput("successful but not a version\n")).toEqual(
|
||||
{
|
||||
version: null,
|
||||
reportedVersion: "successful but not a version",
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it("parses semantic prerelease versions without claiming they meet the minimum", () => {
|
||||
expect(parseApktoolVersionOutput("2.7.0-beta.1\n")).toEqual({
|
||||
version: "2.7.0-beta.1",
|
||||
});
|
||||
});
|
||||
|
||||
it("validates normalized semantic versions while preserving caller spelling", () => {
|
||||
expect(parseApktoolVersionOutput("v2.7.0\n")).toEqual({
|
||||
version: "v2.7.0",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ import {
|
||||
import type { ProviderAvailability } from "../application/AnalysisProvider.js";
|
||||
import type { ProviderRejectionCode } from "../contracts/providerSelection.js";
|
||||
import type { JsonValue } from "../domain/jsonValue.js";
|
||||
import { gte, valid } from "semver";
|
||||
import {
|
||||
isWindowsBatchFile,
|
||||
windowsBatchCommand,
|
||||
@@ -148,7 +149,8 @@ export type ApktoolConfigurationFailureCode =
|
||||
| "command_missing"
|
||||
| "command_unresolved"
|
||||
| "launcher_unsupported"
|
||||
| "version_unresolved";
|
||||
| "version_unresolved"
|
||||
| "unsupported_version";
|
||||
|
||||
export class ApktoolConfigurationFailure extends Error {
|
||||
constructor(
|
||||
@@ -161,9 +163,9 @@ export class ApktoolConfigurationFailure extends Error {
|
||||
}
|
||||
}
|
||||
|
||||
export interface ApktoolClientIdentity {
|
||||
readonly version: string | null;
|
||||
}
|
||||
export type ApktoolClientIdentity =
|
||||
| { readonly version: string; readonly reportedVersion?: never }
|
||||
| { readonly version: null; readonly reportedVersion: string | null };
|
||||
|
||||
/** Process creation for one apktool run on the selected host. */
|
||||
export interface ApktoolInvocation {
|
||||
@@ -199,7 +201,7 @@ export const apktoolInvocation = (
|
||||
windowsVerbatimArguments: false,
|
||||
};
|
||||
|
||||
/** Read the version from `apktool --version` output (first non-empty line). */
|
||||
/** Read the version from Apktool's single-line `--version` response. */
|
||||
export const parseApktoolVersionOutput = (
|
||||
stdout: string,
|
||||
): ApktoolClientIdentity => {
|
||||
@@ -207,7 +209,17 @@ export const parseApktoolVersionOutput = (
|
||||
.split(/\r?\n/u)
|
||||
.map((entry) => entry.trim())
|
||||
.find((entry) => entry !== "");
|
||||
return { version: line === undefined ? null : line };
|
||||
if (line === undefined || valid(line) === null)
|
||||
return { version: null, reportedVersion: line ?? null };
|
||||
return { version: line };
|
||||
};
|
||||
|
||||
const supportsDocumentedApktoolVersion = (version: string): boolean => {
|
||||
// Debian's verified 2.7.0-dirty build is a downstream build marker, not a
|
||||
// prerelease. Treat only that known marker as build metadata for the floor.
|
||||
const comparisonVersion = version.replace(/-dirty$/u, "+dirty");
|
||||
const parsedVersion = valid(comparisonVersion);
|
||||
return parsedVersion !== null && gte(parsedVersion, "2.7.0");
|
||||
};
|
||||
|
||||
/** Probe the selected apktool launcher without decoding anything. */
|
||||
@@ -239,8 +251,10 @@ export const inspectApktoolClient = async (
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
let stdout: string;
|
||||
let stderr: string;
|
||||
try {
|
||||
const { stdout } = await execFileOutput(
|
||||
({ stdout, stderr } = await execFileOutput(
|
||||
invocation.command,
|
||||
invocation.arguments,
|
||||
{
|
||||
@@ -251,8 +265,7 @@ export const inspectApktoolClient = async (
|
||||
env: hostEnvironment,
|
||||
...(signal === undefined ? {} : { signal }),
|
||||
},
|
||||
);
|
||||
return parseApktoolVersionOutput(stdout);
|
||||
));
|
||||
} catch (cause) {
|
||||
const failure = execFileOutputFailure(cause);
|
||||
if (signal?.aborted || failure?.code === "ABORT_ERR")
|
||||
@@ -283,6 +296,33 @@ export const inspectApktoolClient = async (
|
||||
{ cause },
|
||||
);
|
||||
}
|
||||
const identity = parseApktoolVersionOutput(stdout);
|
||||
if (identity.version === null)
|
||||
throw new ApktoolConfigurationFailure(
|
||||
"version_unresolved",
|
||||
"The selected apktool launcher exited successfully but did not report a recognizable version",
|
||||
{
|
||||
selected_command: selection.command,
|
||||
command_source: selection.commandSource,
|
||||
version_probe: "unrecognized_output",
|
||||
reported_version: identity.reportedVersion,
|
||||
stdout,
|
||||
stderr,
|
||||
},
|
||||
);
|
||||
if (!supportsDocumentedApktoolVersion(identity.version))
|
||||
throw new ApktoolConfigurationFailure(
|
||||
"unsupported_version",
|
||||
`The selected Apktool version ${identity.version} is older than the supported minimum 2.7.0`,
|
||||
{
|
||||
selected_command: selection.command,
|
||||
command_source: selection.commandSource,
|
||||
reported_version: identity.version,
|
||||
minimum_version: "2.7.0",
|
||||
remediation: `Use Apktool 2.7.0 or newer; see docs/apktool-resource-analysis.md.`,
|
||||
},
|
||||
);
|
||||
return identity;
|
||||
};
|
||||
|
||||
/** Availability in the shape shared by every provider. */
|
||||
@@ -295,7 +335,9 @@ const rejectionCodeOf = (
|
||||
code === "launcher_unsupported" ||
|
||||
code === "command_unresolved"
|
||||
? "not_configured"
|
||||
: "version_unresolved";
|
||||
: code === "unsupported_version"
|
||||
? "unsupported_version"
|
||||
: "version_unresolved";
|
||||
|
||||
export const apktoolConfigurationAvailability = (
|
||||
cause: ApktoolConfigurationFailure,
|
||||
|
||||
@@ -17,7 +17,7 @@ import type { ApktoolRequest } from "../domain/apktool/apktoolResourceAnalysis.j
|
||||
|
||||
const it = test.skipIf(process.platform === "win32");
|
||||
|
||||
const VERSION_BLOCK = "printf '2.7.0-stub\\n'";
|
||||
const VERSION_BLOCK = "printf '2.7.0-dirty\\n'";
|
||||
|
||||
const REAL_YML = [
|
||||
"!!brut.androlib.meta.MetaInfo",
|
||||
@@ -172,7 +172,7 @@ it("inspects the apktool launcher", async () => {
|
||||
expect(result.client).toEqual({
|
||||
command,
|
||||
command_source: "environment",
|
||||
apktool_version: "2.7.0-stub",
|
||||
apktool_version: "2.7.0-dirty",
|
||||
});
|
||||
});
|
||||
|
||||
@@ -355,6 +355,39 @@ it("reports a missing launcher as a capability gap with recovery", async () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("fails closed for unrecognized and unsupported Apktool versions", async () => {
|
||||
const invalidOutput = `unexpected Apktool output ${"x".repeat(700)}\n`;
|
||||
const unrecognized = await writeApktoolStub(
|
||||
() => `printf '%s' '${invalidOutput}'; exit 0`,
|
||||
);
|
||||
await expect(
|
||||
providerOver(unrecognized.command).inspectAvailability(),
|
||||
).resolves.toMatchObject({
|
||||
status: "unavailable",
|
||||
code: "version_unresolved",
|
||||
diagnostics: {
|
||||
reported_version: invalidOutput.trim(),
|
||||
stdout: invalidOutput,
|
||||
},
|
||||
});
|
||||
|
||||
const old = await writeApktoolStub(() => `printf '2.6.3\\n'`);
|
||||
await expect(
|
||||
providerOver(old.command).inspectAvailability(),
|
||||
).resolves.toMatchObject({
|
||||
status: "unavailable",
|
||||
code: "unsupported_version",
|
||||
});
|
||||
|
||||
const prerelease = await writeApktoolStub(() => `printf '2.7.0-beta.1\\n'`);
|
||||
await expect(
|
||||
providerOver(prerelease.command).inspectAvailability(),
|
||||
).resolves.toMatchObject({
|
||||
status: "unavailable",
|
||||
code: "unsupported_version",
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a target that is not a readable file", async () => {
|
||||
const { command, root } = await writeApktoolStub(decodeStub());
|
||||
const projected = await refusedBy(providerOver(command), {
|
||||
|
||||
@@ -97,6 +97,53 @@ const entry = (
|
||||
return found;
|
||||
};
|
||||
|
||||
describe("ADB and Apktool host availability", () => {
|
||||
it("uses only their selected owner probes", () => {
|
||||
const ready = {
|
||||
status: "available" as const,
|
||||
code: null,
|
||||
reason: null,
|
||||
diagnostics: {},
|
||||
};
|
||||
const configuredButInvalid = {
|
||||
status: "unavailable" as const,
|
||||
code: "not_configured" as const,
|
||||
reason: "REA_ADB_PATH is invalid",
|
||||
diagnostics: {},
|
||||
};
|
||||
const readyInventory = buildCapabilityInventory(status(), {
|
||||
...enabledPolicy,
|
||||
adbAnalysisAvailability: ready,
|
||||
apktoolAnalysisAvailability: ready,
|
||||
});
|
||||
expect(
|
||||
readyInventory.find(({ name }) => name === "inspect_adb_client"),
|
||||
).toMatchObject({ available: true, reason: "available" });
|
||||
expect(
|
||||
readyInventory.find(({ name }) => name === "inspect_apktool_client"),
|
||||
).toMatchObject({ available: true, reason: "available" });
|
||||
|
||||
const unavailableInventory = buildCapabilityInventory(status(), {
|
||||
...enabledPolicy,
|
||||
adbAnalysisAvailability: configuredButInvalid,
|
||||
apktoolAnalysisAvailability: {
|
||||
status: "unavailable",
|
||||
code: "unsupported_version",
|
||||
reason: "Apktool 2.6.3 is older than 2.7.0",
|
||||
diagnostics: {},
|
||||
},
|
||||
});
|
||||
expect(
|
||||
unavailableInventory.find(({ name }) => name === "inspect_adb_client"),
|
||||
).toMatchObject({ available: false, reason: "provider_missing" });
|
||||
expect(
|
||||
unavailableInventory.find(
|
||||
({ name }) => name === "decode_android_resources",
|
||||
),
|
||||
).toMatchObject({ available: false, reason: "provider_unavailable" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("capability failure reasons", () => {
|
||||
it("keeps an unavailable Android runtime distinct from a missing provider", () => {
|
||||
const inventory = buildCapabilityInventory(status(), {
|
||||
|
||||
@@ -54,6 +54,8 @@ export type AvailabilityPolicy = {
|
||||
readonly firmwareInspectionEnabled?: boolean;
|
||||
readonly firmwareExtractionEnabled?: boolean;
|
||||
readonly androidAnalysisAvailability?: ProviderAvailability;
|
||||
readonly adbAnalysisAvailability?: ProviderAvailability;
|
||||
readonly apktoolAnalysisAvailability?: ProviderAvailability;
|
||||
readonly jebAnalysisAvailability?: ProviderAvailability;
|
||||
readonly javascriptRecoveryEnabled?: boolean;
|
||||
readonly webModuleResolutionEnabled?: boolean;
|
||||
@@ -473,6 +475,16 @@ const workflowAvailabilityFor = ({
|
||||
policy.androidAnalysisAvailability?.reason ??
|
||||
"Set REA_JADX_MCP_JAR to a caller-supplied jadx-headless-mcp 0.7.1 JAR and provide a full JDK on Linux, macOS, or Windows x64 with its matching bundled native controls.",
|
||||
};
|
||||
if (kind === "adb-provider")
|
||||
return ownerProbeAvailability(
|
||||
policy.adbAnalysisAvailability,
|
||||
"Set REA_ADB_PATH to a caller-supplied adb executable or make adb available on PATH. This checks the client version only; device and serial availability depend on the request.",
|
||||
);
|
||||
if (kind === "apktool-provider")
|
||||
return ownerProbeAvailability(
|
||||
policy.apktoolAnalysisAvailability,
|
||||
"Set REA_APKTOOL_COMMAND to a caller-supplied Apktool executable or make Apktool 2.7.0 or newer available on PATH.",
|
||||
);
|
||||
if (kind === "jeb-provider")
|
||||
return policy.jebAnalysisAvailability?.status === "available"
|
||||
? { reason: "available", remediation: null }
|
||||
@@ -506,6 +518,25 @@ const workflowAvailabilityFor = ({
|
||||
return null;
|
||||
};
|
||||
|
||||
const ownerProbeAvailability = (
|
||||
observation: ProviderAvailability | undefined,
|
||||
missingRemediation: string,
|
||||
): Availability => {
|
||||
if (observation === undefined)
|
||||
return { reason: "provider_missing", remediation: missingRemediation };
|
||||
if (observation.status === "available")
|
||||
return { reason: "available", remediation: null };
|
||||
return {
|
||||
reason:
|
||||
observation.code === "unsupported_host"
|
||||
? "unsupported_host"
|
||||
: observation.code === "not_configured"
|
||||
? "provider_missing"
|
||||
: "provider_unavailable",
|
||||
remediation: observation.reason,
|
||||
};
|
||||
};
|
||||
|
||||
const electronProviderAvailability = (
|
||||
name: string,
|
||||
kind: ToolKind,
|
||||
|
||||
@@ -47,6 +47,49 @@ import { resolveProcedureAddress } from "./ProcedureAddressResolution.js";
|
||||
import { traceLiteralFeature } from "./EnhancedLiteralTracing.js";
|
||||
import { projectNativeApiInspection } from "./native/NativeApiInspection.js";
|
||||
|
||||
type BatchProcedureIdentity =
|
||||
| {
|
||||
readonly status: "resolved";
|
||||
readonly address: string;
|
||||
readonly name: string | null;
|
||||
readonly name_error?: ReturnType<typeof projectAnalysisError>;
|
||||
}
|
||||
| {
|
||||
readonly status: "unknown";
|
||||
readonly error: ReturnType<typeof projectAnalysisError>;
|
||||
};
|
||||
|
||||
type BatchResolution = {
|
||||
readonly address: string;
|
||||
readonly result: Result<string, AnalysisError>;
|
||||
};
|
||||
|
||||
type BatchReadyItem = {
|
||||
readonly address: string;
|
||||
readonly procedure: BatchProcedureIdentity;
|
||||
readonly providerProcedure: string;
|
||||
};
|
||||
|
||||
type BatchDecompileItem = {
|
||||
readonly address: string;
|
||||
readonly procedure: BatchProcedureIdentity;
|
||||
readonly status: "ok" | "error";
|
||||
readonly pseudocode?: string;
|
||||
readonly error?: ReturnType<typeof projectAnalysisError>;
|
||||
};
|
||||
|
||||
const isAnalysisCancellation = (
|
||||
error: AnalysisError,
|
||||
): error is AnalysisCancelledError => error._tag === "AnalysisCancelledError";
|
||||
|
||||
const firstAnalysisCancellation = <Value>(
|
||||
results: readonly Result<Value, AnalysisError>[],
|
||||
): AnalysisCancelledError | undefined => {
|
||||
for (const result of results)
|
||||
if (!result.ok && isAnalysisCancellation(result.error)) return result.error;
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/**
|
||||
* Composes direct provider operations into bounded reverse-engineering tools.
|
||||
* Direct callers use `execute`; adapters that own validation use
|
||||
@@ -259,52 +302,34 @@ export class EnhancedTools {
|
||||
addresses: readonly string[],
|
||||
signal?: AbortSignal,
|
||||
): EnhancedResult {
|
||||
const items = await Promise.all(
|
||||
const resolutions = await Promise.all(
|
||||
addresses.map(async (address) => {
|
||||
const resolved = await resolveProcedureAddress(
|
||||
const result = await resolveProcedureAddress(
|
||||
this.#call.bind(this),
|
||||
address,
|
||||
signal,
|
||||
);
|
||||
const procedure = resolved.ok
|
||||
? await this.#batchProcedureIdentity(resolved.value, signal)
|
||||
: {
|
||||
status: "unknown" as const,
|
||||
error: projectAnalysisError(resolved.error),
|
||||
};
|
||||
const result = await this.#call(
|
||||
"procedure_pseudo_code",
|
||||
{ procedure: resolved.ok ? resolved.value : address },
|
||||
signal,
|
||||
resolved.ok ? "provider" : undefined,
|
||||
);
|
||||
if (!result.ok)
|
||||
return {
|
||||
address,
|
||||
procedure,
|
||||
status: "error" as const,
|
||||
error: projectAnalysisError(result.error),
|
||||
};
|
||||
if (typeof result.value !== "string" || result.value.length === 0)
|
||||
return {
|
||||
address,
|
||||
procedure,
|
||||
status: "error" as const,
|
||||
error: projectAnalysisError(
|
||||
new AnalysisOutputError(
|
||||
"procedure_pseudo_code",
|
||||
"provider returned empty pseudocode",
|
||||
),
|
||||
),
|
||||
};
|
||||
return {
|
||||
address,
|
||||
procedure,
|
||||
status: "ok" as const,
|
||||
pseudocode: result.value,
|
||||
};
|
||||
return { address, result };
|
||||
}),
|
||||
);
|
||||
const resolutionCancellation = firstAnalysisCancellation(
|
||||
resolutions.map(({ result }) => result),
|
||||
);
|
||||
if (resolutionCancellation !== undefined)
|
||||
return err(resolutionCancellation);
|
||||
|
||||
const identities = await this.#batchIdentities(resolutions, signal);
|
||||
const ready: BatchReadyItem[] = [];
|
||||
for (const identity of identities) {
|
||||
if (!identity.ok) return err(identity.error);
|
||||
ready.push(identity.value);
|
||||
}
|
||||
const decompilations = await this.#batchPseudocode(ready, signal);
|
||||
const items: BatchDecompileItem[] = [];
|
||||
for (const decompilation of decompilations) {
|
||||
if (!decompilation.ok) return err(decompilation.error);
|
||||
items.push(decompilation.value);
|
||||
}
|
||||
const succeeded = items.filter(({ status }) => status === "ok").length;
|
||||
return ok({
|
||||
items,
|
||||
@@ -314,12 +339,91 @@ export class EnhancedTools {
|
||||
});
|
||||
}
|
||||
|
||||
async #batchProcedureIdentity(address: string, signal?: AbortSignal) {
|
||||
async #batchIdentities(
|
||||
resolutions: readonly BatchResolution[],
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<BatchReadyItem, AnalysisCancelledError>[]> {
|
||||
return Promise.all(
|
||||
resolutions.map(async ({ address, result }) => {
|
||||
if (!result.ok)
|
||||
return ok({
|
||||
address,
|
||||
providerProcedure: address,
|
||||
procedure: {
|
||||
status: "unknown" as const,
|
||||
error: projectAnalysisError(result.error),
|
||||
},
|
||||
});
|
||||
const identity = await this.#batchProcedureIdentity(
|
||||
result.value,
|
||||
signal,
|
||||
);
|
||||
return identity.ok
|
||||
? ok({
|
||||
address,
|
||||
providerProcedure: result.value,
|
||||
procedure: identity.value,
|
||||
})
|
||||
: err(identity.error);
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
async #batchPseudocode(
|
||||
ready: readonly BatchReadyItem[],
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<BatchDecompileItem, AnalysisCancelledError>[]> {
|
||||
return Promise.all(
|
||||
ready.map(async ({ address, procedure, providerProcedure }) => {
|
||||
const result = await this.#call(
|
||||
"procedure_pseudo_code",
|
||||
{ procedure: providerProcedure },
|
||||
signal,
|
||||
procedure.status === "resolved" ? "provider" : undefined,
|
||||
);
|
||||
if (!result.ok && isAnalysisCancellation(result.error))
|
||||
return err(result.error);
|
||||
const item: BatchDecompileItem = !result.ok
|
||||
? {
|
||||
address,
|
||||
procedure,
|
||||
status: "error",
|
||||
error: projectAnalysisError(result.error),
|
||||
}
|
||||
: typeof result.value !== "string" || result.value.length === 0
|
||||
? {
|
||||
address,
|
||||
procedure,
|
||||
status: "error",
|
||||
error: projectAnalysisError(
|
||||
new AnalysisOutputError(
|
||||
"procedure_pseudo_code",
|
||||
"provider returned empty pseudocode",
|
||||
),
|
||||
),
|
||||
}
|
||||
: {
|
||||
address,
|
||||
procedure,
|
||||
status: "ok",
|
||||
pseudocode: result.value,
|
||||
};
|
||||
return ok(item);
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
async #batchProcedureIdentity(
|
||||
address: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<BatchProcedureIdentity, AnalysisCancelledError>> {
|
||||
const observed = await this.#call("address_name", { address }, signal);
|
||||
if (!observed.ok && isAnalysisCancellation(observed.error))
|
||||
return err(observed.error);
|
||||
const named =
|
||||
observed.ok &&
|
||||
(typeof observed.value === "string" || observed.value === null);
|
||||
return {
|
||||
return ok({
|
||||
status: "resolved" as const,
|
||||
address,
|
||||
name: named ? observed.value : null,
|
||||
@@ -335,7 +439,7 @@ export class EnhancedTools {
|
||||
: observed.error,
|
||||
),
|
||||
}),
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
async #callGraph(
|
||||
@@ -385,6 +489,7 @@ export class EnhancedTools {
|
||||
"provider",
|
||||
);
|
||||
if (!result.ok) {
|
||||
if (isAnalysisCancellation(result.error)) return result;
|
||||
graph[level].push({
|
||||
address: current.address,
|
||||
status: "error",
|
||||
|
||||
@@ -28,6 +28,7 @@ it("retains batched TraceMap-resolved paths and null source positions", () => {
|
||||
text: { included: true, value: sourceMapText },
|
||||
},
|
||||
],
|
||||
unavailable_relevant_files: [],
|
||||
containers: [],
|
||||
text_bytes_read: Buffer.byteLength(sourceMapText),
|
||||
invalid_utf8_files: 0,
|
||||
|
||||
@@ -178,6 +178,7 @@ const graphCoverage = (context: JavaScriptArtifactGraphContext) => {
|
||||
);
|
||||
const unknownGap =
|
||||
(context.fileSet.source_selection?.unselected_files ?? 0) > 0 ||
|
||||
context.fileSet.unavailable_relevant_files.length > 0 ||
|
||||
context.analysis.files.some(
|
||||
({ analysis_failure, application_projection_failure }) =>
|
||||
analysis_failure !== undefined ||
|
||||
|
||||
@@ -1,7 +1,10 @@
|
||||
import type { ArtifactInventorySnapshot } from "../../domain/artifactInventorySnapshot.js";
|
||||
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
|
||||
import { htmlInlineScriptLimitation } from "./JavaScriptArtifactAnalysis.js";
|
||||
import { completeApplicationCoverage } from "../../domain/javascript/javascriptApplicationEvidenceSchemas.js";
|
||||
import {
|
||||
completeApplicationCoverage,
|
||||
partialApplicationCoverage,
|
||||
} from "../../domain/javascript/javascriptApplicationEvidenceSchemas.js";
|
||||
import type { JavaScriptModuleArtifactAnalysis } from "./JavaScriptArtifactAnalysisTypes.js";
|
||||
import { jsonValueSchema } from "../../domain/jsonValue.js";
|
||||
import { freezeJsonSnapshot } from "../../domain/immutableJson.js";
|
||||
@@ -300,8 +303,133 @@ export const addJavaScriptArtifactFiles = (
|
||||
},
|
||||
});
|
||||
}
|
||||
addUnavailableAsarEntries(context);
|
||||
};
|
||||
|
||||
const addUnavailableAsarEntries = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
): void => {
|
||||
for (const file of context.fileSet.unavailable_relevant_files) {
|
||||
if (!isAsarContainer(context, file.container_sha256)) continue;
|
||||
const limitations = [
|
||||
...(file.limitations ?? []),
|
||||
`The inventoried ${file.kind ?? "relevant"} entry has unavailable bytes; no content artifact identity can be established.`,
|
||||
];
|
||||
const entry = context.accumulator.addNode({
|
||||
kind: "asar-entry",
|
||||
identity: {
|
||||
strategy: "canonical-path",
|
||||
stability: "artifact-version",
|
||||
artifact_sha256: file.container_sha256,
|
||||
path: file.path,
|
||||
},
|
||||
observations: [
|
||||
{
|
||||
label: file.path,
|
||||
properties: {
|
||||
entry_sha256: file.sha256,
|
||||
bytes: file.bytes,
|
||||
inventory_artifact_id: file.inventory_artifact_id,
|
||||
...(file.inventory_occurrence_id === undefined
|
||||
? {}
|
||||
: { inventory_occurrence_id: file.inventory_occurrence_id }),
|
||||
hash_status: file.hash_status,
|
||||
artifact_kind: file.artifact_kind,
|
||||
...(file.kind === undefined ? {} : { kind: file.kind }),
|
||||
content_available: false,
|
||||
},
|
||||
evidence: artifactObservationEvidence({
|
||||
sha256: file.container_sha256,
|
||||
path: file.path,
|
||||
operation: "inventory-asar-unavailable-entry",
|
||||
coverage: partialApplicationCoverage([], null),
|
||||
limitations,
|
||||
}),
|
||||
},
|
||||
],
|
||||
});
|
||||
const parent =
|
||||
context.containerNodes.get(file.container_sha256) ?? context.root;
|
||||
context.accumulator.addEdge({
|
||||
source_node_id: parent.node_id,
|
||||
target_node_id: entry.node_id,
|
||||
relation: "contains",
|
||||
properties: {
|
||||
path: file.path,
|
||||
content_available: false,
|
||||
hash_status: file.hash_status,
|
||||
},
|
||||
evidence: artifactObservationEvidence({
|
||||
sha256: file.container_sha256,
|
||||
path: file.path,
|
||||
operation: "inventory-asar-unavailable-entry",
|
||||
coverage: partialApplicationCoverage([], null),
|
||||
limitations,
|
||||
}),
|
||||
});
|
||||
if (file.kind === "native-addon" || file.artifact_kind === "native-addon") {
|
||||
const addon = context.accumulator.addNode({
|
||||
kind: "native-addon",
|
||||
identity: {
|
||||
strategy: "canonical-path",
|
||||
stability: "artifact-version",
|
||||
artifact_sha256: file.container_sha256,
|
||||
path: file.path,
|
||||
},
|
||||
observations: [
|
||||
{
|
||||
label: file.path,
|
||||
properties: {
|
||||
availability: "unavailable",
|
||||
...(file.hash_status === undefined
|
||||
? {}
|
||||
: { hash_status: file.hash_status }),
|
||||
...(file.artifact_kind === undefined
|
||||
? {}
|
||||
: { artifact_kind: file.artifact_kind }),
|
||||
bytes: file.bytes,
|
||||
inventory_artifact_id: file.inventory_artifact_id,
|
||||
...(file.inventory_occurrence_id === undefined
|
||||
? {}
|
||||
: { inventory_occurrence_id: file.inventory_occurrence_id }),
|
||||
},
|
||||
evidence: artifactObservationEvidence({
|
||||
sha256: file.container_sha256,
|
||||
path: file.path,
|
||||
operation: "classify-unavailable-native-addon",
|
||||
coverage: partialApplicationCoverage([], null),
|
||||
limitations,
|
||||
}),
|
||||
},
|
||||
],
|
||||
});
|
||||
context.accumulator.addEdge({
|
||||
source_node_id: entry.node_id,
|
||||
target_node_id: addon.node_id,
|
||||
relation: "contains",
|
||||
properties: {
|
||||
classification: "native-addon",
|
||||
content_available: false,
|
||||
},
|
||||
evidence: artifactObservationEvidence({
|
||||
sha256: file.container_sha256,
|
||||
path: file.path,
|
||||
operation: "classify-unavailable-native-addon",
|
||||
coverage: partialApplicationCoverage([], null),
|
||||
limitations,
|
||||
}),
|
||||
});
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
const isAsarContainer = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
containerSha256: string,
|
||||
): boolean =>
|
||||
context.snapshot.manifest.root_format === "asar" ||
|
||||
containerSha256 !== context.snapshot.manifest.root_sha256;
|
||||
|
||||
const sourceFileRange = (
|
||||
file: JavaScriptArtifactFile,
|
||||
): JavaScriptSourceRange => {
|
||||
@@ -464,10 +592,7 @@ const createAsarEntry = (
|
||||
context: JavaScriptArtifactGraphContext,
|
||||
file: JavaScriptArtifactFile,
|
||||
): ApplicationNode | undefined => {
|
||||
const isAsar =
|
||||
context.snapshot.manifest.root_format === "asar" ||
|
||||
file.container_sha256 !== context.snapshot.manifest.root_sha256;
|
||||
if (!isAsar) return undefined;
|
||||
if (!isAsarContainer(context, file.container_sha256)) return undefined;
|
||||
return context.accumulator.addNode({
|
||||
kind: "asar-entry",
|
||||
identity: {
|
||||
|
||||
@@ -68,6 +68,56 @@ describe("artifact-local graph identity", () => {
|
||||
});
|
||||
|
||||
describe("loader-owned path lookup", () => {
|
||||
it("applies URL whitespace handling only to ESM lookup and retains declarations", () => {
|
||||
const tab = "\t";
|
||||
const files = fileMap([
|
||||
file("app/entry.mjs", "root"),
|
||||
file("app/feature.js", "root"),
|
||||
file(`app/tab${tab}name.js`, "root"),
|
||||
]);
|
||||
for (const whitespace of ["\t", "\n", "\r"]) {
|
||||
const raw = `./fea${whitespace}ture.js`;
|
||||
expect(
|
||||
resolve({
|
||||
declaredPath: raw,
|
||||
sourcePath: "app/entry.mjs",
|
||||
context: "module-specifier",
|
||||
moduleKind: "import",
|
||||
files,
|
||||
}),
|
||||
).toMatchObject({ declared_path: raw, resolved_path: "app/feature.js" });
|
||||
}
|
||||
expect(
|
||||
resolve({
|
||||
declaredPath: `./tab${tab}name.js`,
|
||||
sourcePath: "app/entry.mjs",
|
||||
context: "module-specifier",
|
||||
moduleKind: "import",
|
||||
files,
|
||||
}),
|
||||
).toMatchObject({ resolved_path: null });
|
||||
expect(
|
||||
resolve({
|
||||
declaredPath: `./tab${tab}name.js`,
|
||||
sourcePath: "app/entry.cjs",
|
||||
context: "module-specifier",
|
||||
moduleKind: "require",
|
||||
files,
|
||||
}),
|
||||
).toMatchObject({ resolved_path: "app/tab\tname.js" });
|
||||
expect(
|
||||
resolve({
|
||||
declaredPath: "./tab%09name.js",
|
||||
sourcePath: "app/entry.mjs",
|
||||
context: "module-specifier",
|
||||
moduleKind: "import",
|
||||
files,
|
||||
}),
|
||||
).toMatchObject({ resolved_path: "app/tab\tname.js" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("loader-specific module suffix lookup", () => {
|
||||
it("keeps relative ESM imports exact", () => {
|
||||
const files = fileMap([
|
||||
file("app/entry.mjs", "root"),
|
||||
@@ -186,6 +236,42 @@ describe("loader-owned path lookup", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("unavailable inventoried native add-ons", () => {
|
||||
it("retains an inventoried native add-on whose bytes are unavailable", () => {
|
||||
const unavailable = {
|
||||
path: "app/addon.node",
|
||||
container_sha256: file("app/entry.cjs", "root").container_sha256,
|
||||
sha256: null,
|
||||
bytes: 8192,
|
||||
inventory_artifact_id: null,
|
||||
inventory_occurrence_id: "missing-addon-occurrence",
|
||||
kind: "native-addon" as const,
|
||||
hash_status: "unavailable" as const,
|
||||
limitations: ["unpacked companion is missing"],
|
||||
};
|
||||
const result = resolve({
|
||||
declaredPath: "./addon.node",
|
||||
sourcePath: "app/entry.cjs",
|
||||
context: "module-specifier",
|
||||
moduleKind: "require",
|
||||
files: fileMap([file("app/entry.cjs", "root")]),
|
||||
unanalyzedFiles: {
|
||||
get: (path) => (path === unavailable.path ? unavailable : undefined),
|
||||
hasDirectory: () => false,
|
||||
},
|
||||
});
|
||||
expect(result).toMatchObject({
|
||||
declared_path: "./addon.node",
|
||||
resolution_status: "unavailable",
|
||||
candidate_path: "app/addon.node",
|
||||
limitations: [
|
||||
"unpacked companion is missing",
|
||||
expect.stringContaining("inventoried native-addon file"),
|
||||
],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("bare package root file lookup", () => {
|
||||
it("uses loader-specific bare package roots when only a sibling .js file exists", () => {
|
||||
const absentDirectory = fileMap([
|
||||
|
||||
@@ -499,6 +499,7 @@ const contextualCandidate = (
|
||||
}
|
||||
const hasUrlPathSemantics =
|
||||
context === "module-specifier" && input.moduleKind === "import";
|
||||
if (hasUrlPathSemantics) declared = htmlUrlText(declared);
|
||||
const rejected = rejectDeclaration(
|
||||
declared,
|
||||
declaresLiteralPercent(input) || hasUrlPathSemantics,
|
||||
@@ -1400,7 +1401,9 @@ const findFileCandidate = (
|
||||
candidatePath: path,
|
||||
limitations: [
|
||||
...(unanalyzed.limitations ?? []),
|
||||
`${path} exists in the inventoried artifact container (sha256 ${unanalyzed.sha256 ?? "unknown"}, ${unanalyzed.bytes === null ? "unknown size" : `${String(unanalyzed.bytes)} bytes`}) but is not a JavaScript, JSON, HTML, source map, or native add-on file; its contents were not analyzed.`,
|
||||
unanalyzed.kind === undefined
|
||||
? `${path} exists in the inventoried artifact container (sha256 ${unanalyzed.sha256 ?? "unknown"}, ${unanalyzed.bytes === null ? "unknown size" : `${String(unanalyzed.bytes)} bytes`}) but is not a JavaScript, JSON, HTML, source map, or native add-on file; its contents were not analyzed.`
|
||||
: `${path} is an inventoried ${unanalyzed.kind} file, but its bytes are unavailable (hash status ${unanalyzed.hash_status ?? "unknown"}); resolution cannot be confirmed.`,
|
||||
],
|
||||
};
|
||||
return null;
|
||||
|
||||
@@ -261,7 +261,8 @@ export const reconstructJavaScriptArtifact = async (
|
||||
),
|
||||
),
|
||||
statistics: {
|
||||
relevant_files: files.files.length,
|
||||
relevant_files:
|
||||
files.files.length + files.unavailable_relevant_files.length,
|
||||
nested_asar_containers: files.containers.length,
|
||||
text_bytes_read: files.text_bytes_read,
|
||||
invalid_utf8_files: files.invalid_utf8_files,
|
||||
|
||||
@@ -25,6 +25,43 @@ import { semanticCoverageResourceLimits } from "../../domain/javascript/javascri
|
||||
|
||||
const SHA256 = "a".repeat(64);
|
||||
|
||||
it.each([
|
||||
["plain export", "const answer = 42; export { answer };", 0],
|
||||
["aliased export", "const answer = 42; export { answer as result };", 0],
|
||||
["declaration only", "const answer = 42;", 0],
|
||||
["genuine assignment", "let answer = 0; answer = 1;", 1],
|
||||
])(
|
||||
"projects writes only for actual writes: %s",
|
||||
(_name, source, writeCount) => {
|
||||
const graph = graphFor(source);
|
||||
const binding = graph.nodes.find(
|
||||
({ kind, label }) => kind === "binding" && label === "answer",
|
||||
);
|
||||
if (binding === undefined) throw new Error("Expected answer binding");
|
||||
const writes = graph.relations.filter(
|
||||
({ relation, target_node_id }) =>
|
||||
relation === "writes" && target_node_id === binding.node_id,
|
||||
);
|
||||
expect(writes).toHaveLength(writeCount);
|
||||
const exportReference = graph.nodes.find(
|
||||
({ identity }) => identity.role_key === "reference:0:export:answer",
|
||||
);
|
||||
if (source.includes("export {")) expect(exportReference).toBeDefined();
|
||||
|
||||
const trace = queryJavaScriptSemanticGraph(graph, {
|
||||
seed: { kind: "semantic-node", node_id: binding.node_id },
|
||||
direction: "backward-provenance",
|
||||
allowed_relations: ["writes"],
|
||||
});
|
||||
expect(trace.status).toBe("found");
|
||||
expect(trace.summary).toMatchObject({
|
||||
total_seed_matches: 1,
|
||||
traversed_relations: writeCount,
|
||||
});
|
||||
expect(trace.relations).toHaveLength(writeCount);
|
||||
},
|
||||
);
|
||||
|
||||
it("projects closure and direct interprocedural flow without execution", () => {
|
||||
const graph = graphFor(`
|
||||
const outer = 2;
|
||||
|
||||
@@ -648,7 +648,7 @@ function* projectDefinitionsAndReferences(
|
||||
resolution: "resolved",
|
||||
evidence: inferredSemanticEvidenceAt(context.file, reference.location),
|
||||
});
|
||||
else
|
||||
else if (reference.role === "write")
|
||||
addRelation(context.state, {
|
||||
source: expression,
|
||||
target: binding,
|
||||
|
||||
@@ -140,13 +140,21 @@ export const readJavaScriptArtifactFiles = async (
|
||||
if (leftPriority !== rightPriority) return leftPriority - rightPriority;
|
||||
return compareUnicodeCodePoints(left.path, right.path);
|
||||
});
|
||||
const unanalyzedFiles = createUnanalyzedArtifactLookup(
|
||||
snapshot,
|
||||
context.containers,
|
||||
);
|
||||
const unavailableRelevantFiles = [...inventory.unavailableFiles]
|
||||
.sort(compareUnicodeCodePoints)
|
||||
.flatMap((path) => {
|
||||
const unavailable = unanalyzedFiles.get(path);
|
||||
return unavailable === undefined ? [] : [unavailable];
|
||||
});
|
||||
assertExpectedFilesWereVisited(expected, files);
|
||||
return {
|
||||
files,
|
||||
unanalyzed_files: createUnanalyzedArtifactLookup(
|
||||
snapshot,
|
||||
context.containers,
|
||||
),
|
||||
unanalyzed_files: unanalyzedFiles,
|
||||
unavailable_relevant_files: unavailableRelevantFiles,
|
||||
containers: context.containers.sort((left, right) =>
|
||||
compareUnicodeCodePoints(left.path, right.path),
|
||||
),
|
||||
@@ -488,6 +496,10 @@ const createUnanalyzedArtifactLookup = (
|
||||
const isOpaque = (occurrence: ArtifactOccurrence): boolean =>
|
||||
occurrence.entry_kind === "file" &&
|
||||
relevantKind(occurrence.logical_path) === undefined;
|
||||
const isUnavailableRelevant = (occurrence: ArtifactOccurrence): boolean =>
|
||||
occurrence.entry_kind === "file" &&
|
||||
occurrence.artifact_id === null &&
|
||||
relevantKind(occurrence.logical_path) !== undefined;
|
||||
return {
|
||||
get(path) {
|
||||
if (requested.has(path)) return requested.get(path);
|
||||
@@ -495,8 +507,10 @@ const createUnanalyzedArtifactLookup = (
|
||||
snapshot.occurrences[
|
||||
lowerBound(snapshot.occurrences, path, (entry) => entry.logical_path)
|
||||
];
|
||||
const kind = relevantKind(path);
|
||||
const id =
|
||||
occurrence?.logical_path === path && isOpaque(occurrence)
|
||||
occurrence?.logical_path === path &&
|
||||
(isOpaque(occurrence) || isUnavailableRelevant(occurrence))
|
||||
? occurrence.artifact_id
|
||||
: undefined;
|
||||
const candidate =
|
||||
@@ -507,7 +521,8 @@ const createUnanalyzedArtifactLookup = (
|
||||
];
|
||||
const node = candidate?.artifact_id === id ? candidate : undefined;
|
||||
const file =
|
||||
occurrence?.logical_path !== path || !isOpaque(occurrence)
|
||||
occurrence?.logical_path !== path ||
|
||||
(!isOpaque(occurrence) && !isUnavailableRelevant(occurrence))
|
||||
? undefined
|
||||
: {
|
||||
path,
|
||||
@@ -515,6 +530,10 @@ const createUnanalyzedArtifactLookup = (
|
||||
sha256: node?.sha256 ?? null,
|
||||
bytes: node?.size ?? occurrence.declared_size,
|
||||
inventory_artifact_id: occurrence.artifact_id,
|
||||
inventory_occurrence_id: occurrence.occurrence_id,
|
||||
...(kind === undefined ? {} : { kind }),
|
||||
artifact_kind: occurrence.artifact_kind,
|
||||
hash_status: occurrence.hash_status,
|
||||
limitations: occurrence.limitations,
|
||||
};
|
||||
requested.set(path, file);
|
||||
@@ -539,7 +558,7 @@ const createUnanalyzedArtifactLookup = (
|
||||
if (entry === undefined || !entry.logical_path.startsWith(prefix))
|
||||
break;
|
||||
if (
|
||||
isOpaque(entry) &&
|
||||
(isOpaque(entry) || isUnavailableRelevant(entry)) &&
|
||||
(containerSha256 === undefined ||
|
||||
containerFor(entry.logical_path) === containerSha256)
|
||||
) {
|
||||
|
||||
@@ -600,6 +600,35 @@ describe("Apple application bridge candidates", () => {
|
||||
status: "complete",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps an extensionless Mach-O without an application root as a native component", () => {
|
||||
const result = project(
|
||||
inventoryEvidence("ipa", "Fixture.ipa", [
|
||||
{ path: "Payload/Fixture.app/Info.plist" },
|
||||
macho("Payload/Fixture.app/Fixture"),
|
||||
{ path: "SwiftSupport/outside.js", format: "javascript-bundle" },
|
||||
macho("SwiftSupport/HelperTool"),
|
||||
macho("SwiftSupport/iphoneos/libswiftCore.dylib"),
|
||||
]),
|
||||
);
|
||||
|
||||
expect(result.components.executables.map(({ path }) => path)).toEqual([
|
||||
"Payload/Fixture.app/Fixture",
|
||||
"SwiftSupport/HelperTool",
|
||||
]);
|
||||
expect(result.bridge_candidates).toEqual([
|
||||
{
|
||||
source_path: "SwiftSupport/outside.js",
|
||||
native_path: "SwiftSupport/HelperTool",
|
||||
basis: "javascript-and-native-content",
|
||||
},
|
||||
{
|
||||
source_path: "SwiftSupport/outside.js",
|
||||
native_path: "SwiftSupport/iphoneos/libswiftCore.dylib",
|
||||
basis: "javascript-and-native-content",
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("Apple path conventions", () => {
|
||||
|
||||
@@ -249,6 +249,9 @@ const appleRootFormat = (format: string): RootFormat => {
|
||||
return root;
|
||||
};
|
||||
|
||||
const isNativeLibrary = (path: string): boolean =>
|
||||
/\.(?:dylib|so)$/iu.test(path);
|
||||
|
||||
const classifyComponents = (
|
||||
entries: readonly AppleInventoryEntry[],
|
||||
all: readonly Component[],
|
||||
@@ -271,11 +274,13 @@ const classifyComponents = (
|
||||
bundle_metadata: all.filter(({ path }) =>
|
||||
anatomyNameIs(lastSegment(path), "Info.plist"),
|
||||
),
|
||||
// An unrooted IPA library is listed only under native_libraries.
|
||||
executables: all.filter(
|
||||
({ path, format }) => withinApp(path) && isMachOFormat(format),
|
||||
({ path, format }) =>
|
||||
isMachOFormat(format) && (withinApp(path) || !isNativeLibrary(path)),
|
||||
),
|
||||
frameworks: all.filter(({ path }) => /\.framework\//u.test(path)),
|
||||
native_libraries: all.filter(({ path }) => /\.(?:dylib|so)$/iu.test(path)),
|
||||
native_libraries: all.filter(({ path }) => isNativeLibrary(path)),
|
||||
javascript: all.filter(({ format }) => format === "javascript-bundle"),
|
||||
signing: all.filter(
|
||||
({ path }) =>
|
||||
|
||||
@@ -140,6 +140,40 @@ it("preserves an observed empty contextBridge API object", () => {
|
||||
expect(result.parse_status).toBe("complete");
|
||||
});
|
||||
|
||||
it.each([
|
||||
[
|
||||
"explicit nested paths after an unknown spread",
|
||||
`{ ...extra, a: { b: 1 }, c: 2 }`,
|
||||
["a", "a.b", "c"],
|
||||
1,
|
||||
"dynamic",
|
||||
],
|
||||
[
|
||||
"a direct object literal",
|
||||
`{ a: { b: 1 }, c: 2 }`,
|
||||
["a", "a.b", "c"],
|
||||
0,
|
||||
"object-literal",
|
||||
],
|
||||
["a dynamic API argument", "getApi()", [], 1, "dynamic"],
|
||||
] as const)(
|
||||
"preserves directly observed nested bridge members for %s",
|
||||
(_name, api, members, unknownMembers, apiStatus) => {
|
||||
const result = analyzeJavaScriptStaticSource(`
|
||||
const { contextBridge } = require("electron");
|
||||
contextBridge.exposeInMainWorld("api", ${api});
|
||||
`);
|
||||
expect(result.electron.context_bridge_apis).toContainEqual(
|
||||
expect.objectContaining({
|
||||
api_key: "api",
|
||||
api_status: apiStatus,
|
||||
members,
|
||||
unknown_members: unknownMembers,
|
||||
}),
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
describe("wrapped contextBridge exposure methods", () => {
|
||||
it("retains the original expression for a dynamic isolated world id", () => {
|
||||
const result = analyzeJavaScriptStaticSource(
|
||||
|
||||
@@ -36,6 +36,7 @@ import {
|
||||
symbolicDefiniteExit,
|
||||
symbolicLogicalRight,
|
||||
} from "./javascriptSymbolicControlFlow.js";
|
||||
import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js";
|
||||
|
||||
const callLabel = (node: t.Node): string =>
|
||||
t.isSequenceExpression(node)
|
||||
@@ -410,7 +411,12 @@ export const collectElectronWrappedCallPaths = (
|
||||
api_key_expression: null,
|
||||
api_status: "dynamic" as const,
|
||||
resolution_basis: "wrapper-substitution" as const,
|
||||
members,
|
||||
// The direct syntax pass recursively retains literal dotted paths. The
|
||||
// symbolic resolver enumerates only immediate object keys, so its
|
||||
// replacement must not erase paths already established at this call.
|
||||
members: [
|
||||
...new Set([...(existing?.value.members ?? []), ...members]),
|
||||
].sort(compareUnicodeCodePoints),
|
||||
unknown_members: resolver.unknownMembers(object),
|
||||
module_key: null,
|
||||
location,
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { JavaScriptSourceSelection } from "./javascriptSourceSelection.js";
|
||||
import type { ArtifactOccurrence } from "../artifactGraph.js";
|
||||
|
||||
/** Whether an inventoried path has a source extension read by the analyzer. */
|
||||
export const isJavaScriptSourcePath = (path: string): boolean =>
|
||||
@@ -34,9 +35,8 @@ export interface JavaScriptArtifactFile {
|
||||
}
|
||||
|
||||
/**
|
||||
* One inventoried regular file outside the JavaScript-relevant kinds. Its presence
|
||||
* was inventoried, including unavailable or unhashed members. The source projection
|
||||
* does not read its bytes; references remain present-but-unanalyzed rather than absent.
|
||||
* One inventoried regular file not backed by analyzed source bytes. Its inventory
|
||||
* occurrence remains available even when content is opaque or unavailable.
|
||||
*/
|
||||
export interface JavaScriptUnanalyzedArtifactFile {
|
||||
readonly path: string;
|
||||
@@ -44,6 +44,14 @@ export interface JavaScriptUnanalyzedArtifactFile {
|
||||
readonly sha256: string | null;
|
||||
readonly bytes: number | null;
|
||||
readonly inventory_artifact_id: string | null;
|
||||
readonly inventory_occurrence_id?: string;
|
||||
readonly kind?: JavaScriptArtifactFileKind;
|
||||
readonly artifact_kind?: ArtifactOccurrence["artifact_kind"];
|
||||
readonly hash_status?:
|
||||
| "verified"
|
||||
| "mismatched"
|
||||
| "not-hashed"
|
||||
| "unavailable";
|
||||
readonly limitations?: readonly string[];
|
||||
}
|
||||
|
||||
@@ -66,6 +74,8 @@ export interface JavaScriptArtifactFileSet {
|
||||
readonly files: readonly JavaScriptArtifactFile[];
|
||||
/** Borrowed inventory lookup for regular files the source projection does not read. */
|
||||
readonly unanalyzed_files?: JavaScriptUnanalyzedArtifactLookup;
|
||||
/** Relevant inventoried files whose bytes were unavailable. */
|
||||
readonly unavailable_relevant_files: readonly JavaScriptUnanalyzedArtifactFile[];
|
||||
readonly containers: readonly JavaScriptArtifactContainer[];
|
||||
readonly text_bytes_read: number;
|
||||
readonly invalid_utf8_files: number;
|
||||
|
||||
@@ -16,7 +16,7 @@ export const moduleResolutionModeForJavaScriptReference = (
|
||||
if (referenceKind === "require") return "require";
|
||||
if (referenceKind === "dynamic-import") return "import";
|
||||
const extension = posix.extname(sourcePath).toLowerCase();
|
||||
if (extension === ".cts") return "require";
|
||||
if (extension === ".cjs" || extension === ".cts") return "require";
|
||||
if (extension === ".mts") return "import";
|
||||
if (extension === ".ts" || extension === ".tsx") return "compiler-dependent";
|
||||
return "import";
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
|
||||
import {
|
||||
analyzeJavaScriptSemantics,
|
||||
analyzeParsedJavaScriptSemantics,
|
||||
} from "./javascriptSemanticAnalysis.js";
|
||||
import { parseJavaScriptSource } from "./javascriptSourceParser.js";
|
||||
import { moduleResolutionModeForJavaScriptReference } from "./javascriptModuleResolutionMode.js";
|
||||
import {
|
||||
origin,
|
||||
topLevelBinding,
|
||||
@@ -56,6 +61,74 @@ describe("TypeScript import-equals links", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("CommonJS wrapper parameter redeclarations", () => {
|
||||
it("retains ambient exports and require through uninitialized top-level vars", () => {
|
||||
const path = "app/main.cjs";
|
||||
const parsed = parseJavaScriptSource(
|
||||
'var module, exports, require; module.exports = require("./dependency.cjs");',
|
||||
path,
|
||||
);
|
||||
expect(parsed).not.toBeNull();
|
||||
const ir = analyzeParsedJavaScriptSemantics(
|
||||
parsed!,
|
||||
moduleResolutionModeForJavaScriptReference(path, "static-import"),
|
||||
);
|
||||
expect(ir.moduleLinks).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
kind: "commonjs-export",
|
||||
specifier: "./dependency.cjs",
|
||||
localName: null,
|
||||
exportedName: "default",
|
||||
}),
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
it("retains the Node loader call on an uninitialized wrapper var assignment RHS", () => {
|
||||
const path = "app/main.cjs";
|
||||
const parsed = parseJavaScriptSource(
|
||||
'var require; require = require("./bootstrap.cjs");',
|
||||
path,
|
||||
);
|
||||
expect(parsed).not.toBeNull();
|
||||
const ir = analyzeParsedJavaScriptSemantics(
|
||||
parsed!,
|
||||
moduleResolutionModeForJavaScriptReference(path, "static-import"),
|
||||
);
|
||||
expect(ir.moduleLinks).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
kind: "require",
|
||||
specifier: "./bootstrap.cjs",
|
||||
}),
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
'var require = local; require("./dependency.cjs");',
|
||||
'var require; require = replacement; require("./dependency.cjs");',
|
||||
"var module; module = replacement; module.exports = 1;",
|
||||
"var exports; exports = replacement; exports.value = 1;",
|
||||
'let require; require("./dependency.cjs");',
|
||||
'function nested() { var require; require("./dependency.cjs"); }',
|
||||
])("keeps non-wrapper declarations as shadows: %s", (source) => {
|
||||
const path = "app/main.cjs";
|
||||
const parsed = parseJavaScriptSource(source, path);
|
||||
expect(parsed).not.toBeNull();
|
||||
const ir = analyzeParsedJavaScriptSemantics(
|
||||
parsed!,
|
||||
moduleResolutionModeForJavaScriptReference(path, "static-import"),
|
||||
);
|
||||
expect(
|
||||
ir.moduleLinks.filter(
|
||||
({ kind }) => kind === "require" || kind === "commonjs-export",
|
||||
),
|
||||
).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("TypeScript type-only module links", () => {
|
||||
it("retains declarations without runtime bindings or callable claims", () => {
|
||||
const ir = analyzeJavaScriptSemantics(`
|
||||
|
||||
@@ -5,6 +5,7 @@ import { BUILTIN_RESOURCE_METHODS } from "./javascriptBuiltinResourceMethods.js"
|
||||
import type { JavaScriptModuleOrigin } from "./javascriptSemanticIr.js";
|
||||
import {
|
||||
resolveSemanticBindingState,
|
||||
isUnmodifiedCommonJsWrapperBinding,
|
||||
semanticResolutionBlocked,
|
||||
semanticStableInitializer,
|
||||
type JavaScriptSemanticAnalysisState,
|
||||
@@ -46,13 +47,7 @@ export const semanticRequireLoaderAt = (
|
||||
return binding.initializers.length === 0 ? "factory-loader" : "unknown";
|
||||
if (isWrapperRequireInitializerCall(binding, node, state))
|
||||
return "commonjs-loader";
|
||||
const scope = state.scopesById.get(binding.scopeId);
|
||||
if (
|
||||
scope?.kind === "program" &&
|
||||
binding.declarationKinds.size > 0 &&
|
||||
[...binding.declarationKinds].every((kind) => kind === "var") &&
|
||||
binding.initializers.length === 0
|
||||
)
|
||||
if (isUnmodifiedCommonJsWrapperBinding(node.callee, "require", state))
|
||||
return "commonjs-loader";
|
||||
if (isSemanticCreateRequireBinding(binding, state)) return "create-require";
|
||||
const initializer = semanticStableInitializer(binding, state)?.node;
|
||||
@@ -214,10 +209,9 @@ const writesModuleRequire = (
|
||||
const value = unwrapJavaScriptExpression(node).node;
|
||||
if (!t.isIdentifier(value)) return false;
|
||||
if (value.name === "module")
|
||||
return (
|
||||
resolveSemanticBindingState(state, value, "module") === undefined &&
|
||||
!semanticResolutionBlocked(state, value, "module")
|
||||
);
|
||||
return resolveSemanticBindingState(state, value, "module") === undefined
|
||||
? !semanticResolutionBlocked(state, value, "module")
|
||||
: isUnmodifiedCommonJsWrapperBinding(value, "module", state);
|
||||
const binding = resolveSemanticBindingState(state, value, value.name);
|
||||
if (binding === undefined || visited.has(binding.bindingId)) return false;
|
||||
const next = new Set(visited).add(binding.bindingId);
|
||||
@@ -453,10 +447,17 @@ export const semanticRequireOrigin = (
|
||||
)
|
||||
)
|
||||
return undefined;
|
||||
const requireBinding = resolveSemanticBindingState(
|
||||
state,
|
||||
current.callee,
|
||||
"require",
|
||||
);
|
||||
if (
|
||||
resolveSemanticBindingState(state, current.callee, "require") !==
|
||||
undefined ||
|
||||
semanticResolutionBlocked(state, current.callee, "require") ||
|
||||
(requireBinding !== undefined &&
|
||||
!isUnmodifiedCommonJsWrapperBinding(current.callee, "require", state) &&
|
||||
!isWrapperRequireInitializerCall(requireBinding, current, state)) ||
|
||||
(requireBinding === undefined &&
|
||||
semanticResolutionBlocked(state, current.callee, "require")) ||
|
||||
(state.ambientBindingWrites.get("require") ?? []).some(
|
||||
(write) =>
|
||||
!semanticTopLevelEffectFollowsCapture(
|
||||
|
||||
@@ -253,5 +253,28 @@ export const isUnshadowedGlobal = (
|
||||
name: string,
|
||||
): boolean =>
|
||||
t.isIdentifier(node, { name }) &&
|
||||
resolveSemanticBindingState(state, node, name) === undefined &&
|
||||
!semanticResolutionBlocked(state, node, name);
|
||||
(resolveSemanticBindingState(state, node, name) === undefined
|
||||
? !semanticResolutionBlocked(state, node, name)
|
||||
: isUnmodifiedCommonJsWrapperBinding(node, name, state));
|
||||
|
||||
/** A program-level `var` redeclaration still denotes its untouched CJS wrapper parameter. */
|
||||
export const isUnmodifiedCommonJsWrapperBinding = (
|
||||
node: t.Node,
|
||||
name: string,
|
||||
state: JavaScriptSemanticAnalysisState,
|
||||
): boolean => {
|
||||
if (
|
||||
state.staticImportResolutionMode !== "require" ||
|
||||
!["module", "exports", "require"].includes(name) ||
|
||||
!t.isIdentifier(node, { name })
|
||||
)
|
||||
return false;
|
||||
const binding = resolveSemanticBindingState(state, node, name);
|
||||
if (binding === undefined || binding.initializers.length !== 0) return false;
|
||||
const scope = state.scopesById.get(binding.scopeId);
|
||||
return (
|
||||
scope?.kind === "program" &&
|
||||
binding.declarationKinds.size > 0 &&
|
||||
[...binding.declarationKinds].every((kind) => kind === "var")
|
||||
);
|
||||
};
|
||||
@@ -62,6 +62,37 @@ it("requires producer coverage accounting in every parsed capture", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("does not infer normalized retention budget from missing scenario defaults", () => {
|
||||
const capture = emptyCapture();
|
||||
const scenario = {
|
||||
...capture.manifest.scenario,
|
||||
limits: {
|
||||
files: 10_000,
|
||||
file_bytes: 10_000_000,
|
||||
processes: 1_000,
|
||||
filesystem_depth: 16,
|
||||
},
|
||||
};
|
||||
expect(() =>
|
||||
parseProcessCapture({
|
||||
...capture,
|
||||
manifest: {
|
||||
...capture.manifest,
|
||||
scenario,
|
||||
full_scenario_sha256: digestProcessCommitment(scenario),
|
||||
},
|
||||
truncation_details: {
|
||||
...capture.truncation_details,
|
||||
normalized_terminal: {
|
||||
budget_bytes: 1_000_000,
|
||||
retained_bytes: 0,
|
||||
omitted_frames: 0,
|
||||
},
|
||||
},
|
||||
}),
|
||||
).toThrow("manifest.scenario.limits.output_bytes");
|
||||
});
|
||||
|
||||
it("rejects settlement and cleanup combinations that cannot occur", () => {
|
||||
const capture = emptyCapture();
|
||||
expect(() =>
|
||||
|
||||
@@ -8,6 +8,7 @@ const byteRetentionSchema = z.strictObject({
|
||||
retained_bytes: z.number().int().nonnegative(),
|
||||
observed_frames: z.number().int().nonnegative(),
|
||||
retained_frames: z.number().int().nonnegative(),
|
||||
unknown_size_frames: z.number().int().nonnegative().optional(),
|
||||
});
|
||||
|
||||
/** Filesystem enumeration and whole-file digest coverage for one checkpoint. */
|
||||
@@ -54,8 +55,18 @@ export const processCaptureTruncationDetailsSchema = z.strictObject({
|
||||
"UTF-8 bytes of original PTY chunks; whole chunks are retained or omitted.",
|
||||
),
|
||||
rendered_terminal: byteRetentionSchema.describe(
|
||||
"Cumulative UTF-8 bytes of serialized state plus visible lines, evaluated from retained PTY input; whole frames are retained or omitted.",
|
||||
"Cumulative UTF-8 bytes of serialized state plus visible lines, evaluated from retained PTY input; whole frames are retained or omitted. When normalization exceeds a bound before the frame size is known, observed_bytes contains the exact total for size-known frames and unknown_size_frames counts the other omissions.",
|
||||
),
|
||||
normalized_terminal: z
|
||||
.strictObject({
|
||||
budget_bytes: z.number().int().nonnegative(),
|
||||
retained_bytes: z.number().int().nonnegative(),
|
||||
omitted_frames: z.number().int().nonnegative(),
|
||||
})
|
||||
.optional()
|
||||
.describe(
|
||||
"UTF-8 bytes of normalized raw-chunk text retained under output_bytes; omitted frames can have unknown final size when normalization exceeded an intermediate bound.",
|
||||
),
|
||||
filesystem_before: filesystemCoverageSchema,
|
||||
filesystem_after: filesystemCoverageSchema,
|
||||
process: z.strictObject({
|
||||
@@ -72,6 +83,9 @@ export type ProcessCaptureTruncationDetails = z.infer<
|
||||
typeof processCaptureTruncationDetailsSchema
|
||||
>;
|
||||
export type TerminalRetention = ProcessCaptureTruncationDetails["raw_terminal"];
|
||||
export type NormalizedTerminalRetention = NonNullable<
|
||||
ProcessCaptureTruncationDetails["normalized_terminal"]
|
||||
>;
|
||||
export type ProcessSamplingCoverage =
|
||||
ProcessCaptureTruncationDetails["process"];
|
||||
|
||||
@@ -88,6 +102,8 @@ export const hasCaptureTruncation = (
|
||||
): boolean =>
|
||||
omitted(details.raw_terminal) ||
|
||||
omitted(details.rendered_terminal) ||
|
||||
(details.normalized_terminal?.omitted_frames ?? 0) > 0 ||
|
||||
(details.rendered_terminal.unknown_size_frames ?? 0) > 0 ||
|
||||
filesystemTruncated(details.filesystem_before) ||
|
||||
filesystemTruncated(details.filesystem_after) ||
|
||||
details.process.sampling_partial;
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
import { digestProcessCommitment } from "./processScenario.js";
|
||||
import {
|
||||
digestProcessCommitment,
|
||||
processScenarioOutputBudgetProjectionSchema,
|
||||
} from "./processScenario.js";
|
||||
import { hasCaptureTruncation } from "./processCaptureCoverage.js";
|
||||
|
||||
import type {
|
||||
@@ -312,8 +315,31 @@ const validateCoverage = (
|
||||
retention.observed_bytes >= retention.retained_bytes &&
|
||||
(retention.observed_frames > count ||
|
||||
retention.observed_bytes === retention.retained_bytes) &&
|
||||
retention.retained_bytes <=
|
||||
retention.budget_bytes, `truncation_details.${name}`, "retained observations and budget accounting do not agree");
|
||||
retention.retained_bytes <= retention.budget_bytes &&
|
||||
(retention.unknown_size_frames ?? 0) <=
|
||||
retention.observed_frames -
|
||||
retention.retained_frames, `truncation_details.${name}`, "retained observations and budget accounting do not agree");
|
||||
}
|
||||
const normalized = details.normalized_terminal;
|
||||
if (normalized !== undefined) {
|
||||
const committedOutputBudget =
|
||||
processScenarioOutputBudgetProjectionSchema.safeParse(
|
||||
capture.manifest.scenario["limits"],
|
||||
);
|
||||
require(committedOutputBudget.success, "manifest.scenario.limits.output_bytes", "committed process scenario must include an explicit valid output_bytes budget");
|
||||
const retainedBytes = capture.frames.reduce(
|
||||
(total, frame) => total + Buffer.byteLength(frame.data),
|
||||
0,
|
||||
);
|
||||
if (committedOutputBudget.success)
|
||||
require(normalized.budget_bytes ===
|
||||
committedOutputBudget.data.output_bytes &&
|
||||
normalized.retained_bytes === retainedBytes &&
|
||||
normalized.retained_bytes <= normalized.budget_bytes &&
|
||||
normalized.omitted_frames <=
|
||||
details.raw_terminal.observed_frames -
|
||||
details.raw_terminal
|
||||
.retained_frames, "truncation_details.normalized_terminal", "normalized terminal retention must agree with the committed budget and retained frames");
|
||||
}
|
||||
for (const [name, coverage, files, checkpoint] of [
|
||||
[
|
||||
|
||||
@@ -3,6 +3,10 @@ import canonicalize from "canonicalize";
|
||||
import { z } from "zod";
|
||||
|
||||
const positiveBudget = z.number().int().safe().positive();
|
||||
/** Read the explicitly committed output budget without applying scenario defaults. */
|
||||
export const processScenarioOutputBudgetProjectionSchema = z
|
||||
.object({ output_bytes: positiveBudget })
|
||||
.passthrough();
|
||||
/**
|
||||
* Maximum xterm backing cells for one visible-plus-scrollback buffer. Each
|
||||
* xterm BufferLine stores three Uint32 words per cell (12 MB at this cap),
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
buildManagedPeFixture,
|
||||
managedPeFixtureTarget,
|
||||
} from "./ManagedPe.fixture.js";
|
||||
import { parseExceptionRegions } from "./ManagedMemberInstructionDecoder.js";
|
||||
|
||||
const managedBodyWithSections = (...sections: Buffer[]) => {
|
||||
const header = Buffer.alloc(12);
|
||||
@@ -666,6 +667,59 @@ describe("managed exception region flags and filters", () => {
|
||||
});
|
||||
|
||||
describe("managed exception section decoding", () => {
|
||||
it("reads legacy small EH DataSize without counting its header", () => {
|
||||
const section = Buffer.alloc(16);
|
||||
section[0] = 0x01;
|
||||
section[1] = 12;
|
||||
|
||||
expect(managedBodyWithSections(section)).toMatchObject({
|
||||
status: "present",
|
||||
issue: null,
|
||||
exception_regions: [expect.any(Object)],
|
||||
normalized_il_sha256: expect.any(String),
|
||||
});
|
||||
});
|
||||
|
||||
it("advances a chained legacy section by its physical size", () => {
|
||||
const legacy = Buffer.alloc(16);
|
||||
legacy[0] = 0x81;
|
||||
legacy[1] = 12;
|
||||
const conformant = Buffer.alloc(16);
|
||||
conformant[0] = 0x01;
|
||||
conformant[1] = 16;
|
||||
|
||||
expect(managedBodyWithSections(legacy, conformant)).toMatchObject({
|
||||
status: "present",
|
||||
issue: null,
|
||||
exception_regions: [expect.any(Object), expect.any(Object)],
|
||||
normalized_il_sha256: expect.any(String),
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a legacy-sized clause whose physical bytes leave the artifact", () => {
|
||||
const section = Buffer.alloc(12);
|
||||
section[0] = 0x01;
|
||||
section[1] = 12;
|
||||
|
||||
expect(parseExceptionRegions(section, 0, section.length)).toMatchObject({
|
||||
status: "malformed",
|
||||
regions: [],
|
||||
issue: "Exception section size leaves artifact",
|
||||
});
|
||||
});
|
||||
|
||||
it("still rejects a small EH DataSize that matches neither layout", () => {
|
||||
const section = Buffer.alloc(17);
|
||||
section[0] = 0x01;
|
||||
section[1] = 13;
|
||||
|
||||
expect(managedBodyWithSections(section)).toMatchObject({
|
||||
status: "malformed",
|
||||
exception_regions: [],
|
||||
issue: "Exception section size does not contain whole clauses",
|
||||
});
|
||||
});
|
||||
|
||||
it("marks an exception section with a partial clause as malformed", () => {
|
||||
const section = Buffer.alloc(17);
|
||||
section[0] = 0x01;
|
||||
@@ -695,6 +749,19 @@ describe("managed exception section decoding", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps small section reserved bytes invalid", () => {
|
||||
const section = Buffer.alloc(16);
|
||||
section[0] = 0x01;
|
||||
section[1] = 16;
|
||||
section[2] = 1;
|
||||
|
||||
expect(managedBodyWithSections(section)).toMatchObject({
|
||||
status: "malformed",
|
||||
exception_regions: [],
|
||||
issue: "Small exception section reserved bytes are nonzero",
|
||||
});
|
||||
});
|
||||
|
||||
it("preserves decoded clauses when a chained section has an unsupported kind", () => {
|
||||
const first = Buffer.alloc(16);
|
||||
first[0] = 0x81;
|
||||
|
||||
@@ -543,11 +543,36 @@ export const parseExceptionRegions = (
|
||||
"Exception section size leaves artifact",
|
||||
regions,
|
||||
);
|
||||
if ((size - 4) % clauseSize !== 0)
|
||||
let count: number;
|
||||
let sectionSize = size;
|
||||
if (fat) {
|
||||
if ((size - 4) % clauseSize !== 0)
|
||||
return malformedExceptionRegions(
|
||||
"Exception section size does not contain whole clauses",
|
||||
regions,
|
||||
);
|
||||
count = (size - 4) / clauseSize;
|
||||
} else if ((size - 4) % clauseSize === 0) {
|
||||
count = (size - 4) / clauseSize;
|
||||
} else if (size >= clauseSize && size % clauseSize === 0) {
|
||||
// CoreCLR accommodates legacy VB/MC++ sections whose DataSize omits the
|
||||
// four-byte section header. DataSize is still not the physical extent.
|
||||
count = size / clauseSize;
|
||||
sectionSize = 4 + count * clauseSize;
|
||||
} else {
|
||||
return malformedExceptionRegions(
|
||||
"Exception section size does not contain whole clauses",
|
||||
regions,
|
||||
);
|
||||
}
|
||||
if (
|
||||
sectionOffset > methodEnd - sectionSize ||
|
||||
sectionOffset > bytes.length - sectionSize
|
||||
)
|
||||
return malformedExceptionRegions(
|
||||
"Exception section size leaves artifact",
|
||||
regions,
|
||||
);
|
||||
if (
|
||||
!fat &&
|
||||
(bytes[sectionOffset + 2] !== 0 || bytes[sectionOffset + 3] !== 0)
|
||||
@@ -557,7 +582,6 @@ export const parseExceptionRegions = (
|
||||
regions,
|
||||
);
|
||||
|
||||
const count = (size - 4) / clauseSize;
|
||||
// Accumulates with the method IL bound; the body reserves what it retains.
|
||||
budget?.admitTransient(count * 256, sectionOffset);
|
||||
for (const region of readExceptionClauses(
|
||||
@@ -569,7 +593,7 @@ export const parseExceptionRegions = (
|
||||
regions.push(region);
|
||||
|
||||
hasMoreSections = (kind & 0x80) !== 0;
|
||||
sectionOffset += size;
|
||||
sectionOffset += sectionSize;
|
||||
}
|
||||
return { status: "complete", regions };
|
||||
};
|
||||
|
||||
@@ -1,6 +1,14 @@
|
||||
import { createHash } from "node:crypto";
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { hopperLoaderArgsForTarget } from "./HopperAnalysisProfile.js";
|
||||
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
|
||||
import {
|
||||
hopperLoaderArgsForTarget,
|
||||
resolveHopperAnalysisProfile,
|
||||
} from "./HopperAnalysisProfile.js";
|
||||
|
||||
it("routes physical universal Mach-O containers through Hopper's FAT loader", () => {
|
||||
const result = hopperLoaderArgsForTarget(
|
||||
@@ -21,3 +29,65 @@ it("routes physical universal Mach-O containers through Hopper's FAT loader", ()
|
||||
value: ["-l", "FAT", "--aarch64", "-l", "Mach-O"],
|
||||
});
|
||||
});
|
||||
|
||||
it("prepares a verified thin arm64e slice for a FAT32 system container", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-profile-");
|
||||
const writeUInt32 = (buffer: Buffer, offset: number, value: number): void => {
|
||||
buffer.writeUInt32BE(value, offset);
|
||||
};
|
||||
const slices = [
|
||||
{ cpuType: 0x01000007, cpuSubtype: 3 },
|
||||
{ cpuType: 0x0100000c, cpuSubtype: 0x80000002 },
|
||||
{ cpuType: 0x0100000c, cpuSubtype: 0x8000000c },
|
||||
];
|
||||
const container = Buffer.alloc(3 * 4096 + 64);
|
||||
writeUInt32(container, 0, 0xcafebabe);
|
||||
writeUInt32(container, 4, slices.length);
|
||||
slices.forEach((slice, index) => {
|
||||
const offset = 4096 * (index + 1);
|
||||
const entry = 8 + index * 20;
|
||||
writeUInt32(container, entry, slice.cpuType);
|
||||
writeUInt32(container, entry + 4, slice.cpuSubtype);
|
||||
writeUInt32(container, entry + 8, offset);
|
||||
writeUInt32(container, entry + 12, 64);
|
||||
writeUInt32(container, entry + 16, 12);
|
||||
writeUInt32(container, offset, 0xfeedfacf);
|
||||
writeUInt32(container, offset + 4, slice.cpuType);
|
||||
writeUInt32(container, offset + 8, slice.cpuSubtype);
|
||||
});
|
||||
const arm64eOffset = 8192;
|
||||
const path = join(directory, "system.fat");
|
||||
await writeFile(path, container);
|
||||
const result = await resolveHopperAnalysisProfile(
|
||||
{
|
||||
path,
|
||||
sha256: createHash("sha256").update(container).digest("hex"),
|
||||
kind: "executable",
|
||||
format: "mach-o-universal",
|
||||
architecture: "arm64",
|
||||
availableArchitectures: ["x86_64", "arm64"],
|
||||
universalSlice: { offset: arm64eOffset, size: 64 },
|
||||
},
|
||||
{
|
||||
launcherPath: path,
|
||||
loaderArgsOverride: [],
|
||||
provider: { id: "hopper", name: "Hopper Disassembler", version: null },
|
||||
},
|
||||
);
|
||||
if (!result.ok) throw new Error(result.error.message);
|
||||
const parameters = result.value.profile?.parameters;
|
||||
expect(parameters).toMatchObject({
|
||||
macho_container: "fat32",
|
||||
prepared_image: {
|
||||
method: "fat32_thin_slice",
|
||||
source_offset: arm64eOffset,
|
||||
byte_length: 64,
|
||||
cpu_type: 0x0100000c,
|
||||
cpu_subtype: 0x80000002,
|
||||
},
|
||||
loader: {
|
||||
source: "derived",
|
||||
arguments: ["-l", "Mach-O", "--aarch64"],
|
||||
},
|
||||
});
|
||||
});
|
||||
@@ -54,7 +54,7 @@ export const resolveHopperAnalysisProfile = async (
|
||||
let preparedImage: HopperMachOImage | undefined;
|
||||
if (
|
||||
target.kind === "executable" &&
|
||||
machoContainer === "fat64" &&
|
||||
(machoContainer === "fat64" || machoContainer === "fat32") &&
|
||||
options.loaderArgsOverride.length === 0
|
||||
) {
|
||||
const resolved = await resolveHopperMachOImage(target, options.signal);
|
||||
|
||||
@@ -1,11 +1,109 @@
|
||||
import { chmod, writeFile } from "node:fs/promises";
|
||||
import { createHash } from "node:crypto";
|
||||
import { chmod, readFile, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
|
||||
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
|
||||
import { resolveHopperMachOImage } from "./HopperMachOImage.js";
|
||||
import {
|
||||
prepareHopperMachOImage,
|
||||
resolveHopperMachOImage,
|
||||
} from "./HopperMachOImage.js";
|
||||
|
||||
const CPU_TYPES = { x86_64: 0x01000007, arm64: 0x0100000c } as const;
|
||||
const FAT32_ALIGNMENT = 12;
|
||||
const FAT32_SLICE_SIZE = 64;
|
||||
const ARM64E = 2;
|
||||
const ARM64E_X1 = 12;
|
||||
const ARM64E_WITH_CAPABILITIES = 0x80000002;
|
||||
const ARM64E_X1_WITH_CAPABILITIES = 0x8000000c;
|
||||
|
||||
interface FatFixtureSlice {
|
||||
readonly cpuType: number;
|
||||
readonly cpuSubtype: number;
|
||||
/** Thin-slice byte order, independent of the container's table byte order. */
|
||||
readonly little?: boolean;
|
||||
}
|
||||
|
||||
interface Fat32Fixture {
|
||||
readonly container: Buffer;
|
||||
readonly sliceOffsets: readonly number[];
|
||||
}
|
||||
|
||||
/** Synthesize a FAT32 container whose slices are 4096-aligned thin headers. */
|
||||
const buildFat32Container = (
|
||||
slices: readonly FatFixtureSlice[],
|
||||
options: { readonly littleEndianTable?: boolean } = {},
|
||||
): Fat32Fixture => {
|
||||
const littleTable = options.littleEndianTable === true;
|
||||
const writeTableUInt32 = littleTable
|
||||
? (buffer: Buffer, offset: number, value: number): number =>
|
||||
buffer.writeUInt32LE(value, offset)
|
||||
: (buffer: Buffer, offset: number, value: number): number =>
|
||||
buffer.writeUInt32BE(value, offset);
|
||||
const unit = 2 ** FAT32_ALIGNMENT;
|
||||
const sliceOffsets = slices.map((_slice, index) => unit * (index + 1));
|
||||
const lastOffset = sliceOffsets[sliceOffsets.length - 1] ?? unit;
|
||||
const container = Buffer.alloc(lastOffset + FAT32_SLICE_SIZE, 0);
|
||||
// The magic VALUE is 0xcafebabe in the table's byte order: little-endian
|
||||
// storage yields the FAT_CIGAM magic a big-endian reader observes.
|
||||
writeTableUInt32(container, 0, 0xcafebabe);
|
||||
writeTableUInt32(container, 4, slices.length);
|
||||
for (const [index, slice] of slices.entries()) {
|
||||
const entry = 8 + index * 20;
|
||||
const offset = sliceOffsets[index]!;
|
||||
writeTableUInt32(container, entry, slice.cpuType);
|
||||
writeTableUInt32(container, entry + 4, slice.cpuSubtype);
|
||||
writeTableUInt32(container, entry + 8, offset);
|
||||
writeTableUInt32(container, entry + 12, FAT32_SLICE_SIZE);
|
||||
writeTableUInt32(container, entry + 16, FAT32_ALIGNMENT);
|
||||
const writeThinUInt32 =
|
||||
slice.little === true
|
||||
? (buffer: Buffer, fieldOffset: number, value: number): number =>
|
||||
buffer.writeUInt32LE(value, fieldOffset)
|
||||
: (buffer: Buffer, fieldOffset: number, value: number): number =>
|
||||
buffer.writeUInt32BE(value, fieldOffset);
|
||||
writeThinUInt32(container, offset, 0xfeedfacf);
|
||||
writeThinUInt32(container, offset + 4, slice.cpuType);
|
||||
writeThinUInt32(container, offset + 8, slice.cpuSubtype);
|
||||
}
|
||||
return { container, sliceOffsets };
|
||||
};
|
||||
|
||||
const writeFixture = async (
|
||||
directory: string,
|
||||
name: string,
|
||||
fixture: Fat32Fixture,
|
||||
): Promise<string> => {
|
||||
const path = join(directory, name);
|
||||
await writeFile(path, fixture.container);
|
||||
return path;
|
||||
};
|
||||
|
||||
const fat32Target = (
|
||||
path: string,
|
||||
fixture: Fat32Fixture,
|
||||
architecture: "arm64" | "x86_64",
|
||||
) => ({
|
||||
path,
|
||||
sha256: createHash("sha256").update(fixture.container).digest("hex"),
|
||||
kind: "executable" as const,
|
||||
format: "mach-o-universal" as const,
|
||||
architecture,
|
||||
availableArchitectures: ["x86_64", "arm64"] as const,
|
||||
universalSlice: { offset: fixture.sliceOffsets[0]!, size: FAT32_SLICE_SIZE },
|
||||
});
|
||||
|
||||
const requireResolved = (
|
||||
result: Awaited<ReturnType<typeof resolveHopperMachOImage>>,
|
||||
) => {
|
||||
if (!result.ok)
|
||||
throw new Error(
|
||||
`FAT fixture resolution failed: ${JSON.stringify(projectAnalysisError(result.error))}`,
|
||||
);
|
||||
return result.value;
|
||||
};
|
||||
|
||||
it.skipIf(process.platform === "win32" || process.getuid?.() === 0)(
|
||||
"reports a FAT64 source that became unreadable as a host read denial",
|
||||
@@ -36,7 +134,7 @@ it.skipIf(process.platform === "win32" || process.getuid?.() === 0)(
|
||||
},
|
||||
);
|
||||
|
||||
it("rejects an ELF target before attempting FAT64 source preparation", async () => {
|
||||
it("rejects an ELF target before attempting FAT source preparation", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-mips-boundary-");
|
||||
// No file is created: format validation must precede any filesystem lookup.
|
||||
const path = join(directory, "mips.elf");
|
||||
@@ -54,6 +152,235 @@ it("rejects an ELF target before attempting FAT64 source preparation", async ()
|
||||
expect(result.error).toMatchObject({
|
||||
_tag: "BinaryTargetError",
|
||||
path,
|
||||
reason: "Hopper FAT64 image preparation requires a Mach-O target",
|
||||
reason: "Hopper FAT image preparation requires a Mach-O target",
|
||||
});
|
||||
});
|
||||
|
||||
it("selects the plain-arm64-decodable arm64e slice from an Apple-style FAT32 container", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-arm64e-");
|
||||
// /bin/* system binaries: x86_64, arm64e (cpusubtype 2), arm64e.x1 (subtype 12).
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.x86_64, cpuSubtype: 3 },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E_X1 },
|
||||
]);
|
||||
const path = await writeFixture(directory, "system.fat", fixture);
|
||||
const result = requireResolved(
|
||||
await resolveHopperMachOImage(fat32Target(path, fixture, "arm64")),
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
method: "fat32_thin_slice",
|
||||
architecture_count: 3,
|
||||
source_offset: fixture.sliceOffsets[1],
|
||||
byte_length: FAT32_SLICE_SIZE,
|
||||
cpu_type: CPU_TYPES.arm64,
|
||||
cpu_subtype: ARM64E,
|
||||
});
|
||||
});
|
||||
|
||||
it("ranks capability-bearing arm64 subtypes by base and retains raw identity", async () => {
|
||||
const directory = await createTestTempDirectory(
|
||||
"rea-hopper-fat32-capabilities-",
|
||||
);
|
||||
const fixture = buildFat32Container([
|
||||
{
|
||||
cpuType: CPU_TYPES.arm64,
|
||||
cpuSubtype: ARM64E_X1_WITH_CAPABILITIES,
|
||||
},
|
||||
{
|
||||
cpuType: CPU_TYPES.arm64,
|
||||
cpuSubtype: ARM64E_WITH_CAPABILITIES,
|
||||
},
|
||||
]);
|
||||
const path = await writeFixture(directory, "system.fat", fixture);
|
||||
const selected = requireResolved(
|
||||
await resolveHopperMachOImage(fat32Target(path, fixture, "arm64")),
|
||||
);
|
||||
expect(selected).toMatchObject({
|
||||
source_offset: fixture.sliceOffsets[1],
|
||||
cpu_subtype: ARM64E_WITH_CAPABILITIES,
|
||||
});
|
||||
});
|
||||
|
||||
it("fails closed when capability variants share the preferred arm64e base", async () => {
|
||||
const directory = await createTestTempDirectory(
|
||||
"rea-hopper-fat32-capability-ambiguity-",
|
||||
);
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: 0x80000002 },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: 0x81000002 },
|
||||
]);
|
||||
const path = await writeFixture(directory, "ambiguous.fat", fixture);
|
||||
const result = await resolveHopperMachOImage(
|
||||
fat32Target(path, fixture, "arm64"),
|
||||
);
|
||||
if (result.ok || result.error._tag !== "BinaryTargetError")
|
||||
throw new Error(
|
||||
"Expected ambiguous FAT selection to return a BinaryTargetError",
|
||||
);
|
||||
expect(result.error.reason).toContain("selection for arm64 is ambiguous");
|
||||
});
|
||||
|
||||
it("prefers cpusubtype 2 over 12 regardless of FAT32 table order", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-order-");
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E_X1 },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E },
|
||||
]);
|
||||
const path = await writeFixture(directory, "reordered.fat", fixture);
|
||||
const result = requireResolved(
|
||||
await resolveHopperMachOImage(fat32Target(path, fixture, "arm64")),
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
cpu_subtype: ARM64E,
|
||||
source_offset: fixture.sliceOffsets[1],
|
||||
});
|
||||
});
|
||||
|
||||
it("prefers a plain arm64 slice over arm64e in a FAT32 container", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-plain-");
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: 0 },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E },
|
||||
]);
|
||||
const path = await writeFixture(directory, "plain.fat", fixture);
|
||||
const result = requireResolved(
|
||||
await resolveHopperMachOImage(fat32Target(path, fixture, "arm64")),
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
cpu_subtype: 0,
|
||||
source_offset: fixture.sliceOffsets[0],
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps a lone arm64e.x1 FAT32 slice selectable to preserve today's open behavior", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-lone-");
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E_X1 },
|
||||
]);
|
||||
const path = await writeFixture(directory, "lone.fat", fixture);
|
||||
const result = requireResolved(
|
||||
await resolveHopperMachOImage(fat32Target(path, fixture, "arm64")),
|
||||
);
|
||||
expect(result).toMatchObject({ cpu_subtype: ARM64E_X1 });
|
||||
});
|
||||
|
||||
it("still fails closed when two arm64e.x1 FAT32 slices compete", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-ambig-");
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E_X1 },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E_X1 },
|
||||
]);
|
||||
const path = await writeFixture(directory, "ambiguous.fat", fixture);
|
||||
const result = await resolveHopperMachOImage(
|
||||
fat32Target(path, fixture, "arm64"),
|
||||
);
|
||||
if (result.ok) throw new Error("Ambiguous FAT32 selection succeeded");
|
||||
expect(
|
||||
projectAnalysisError(result.error).details?.reason ?? result.error,
|
||||
).toMatch(/selection.*ambiguous/u);
|
||||
});
|
||||
|
||||
it("selects the single x86_64 slice of a FAT32 container for an x86_64 target", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-intel-");
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E },
|
||||
{ cpuType: CPU_TYPES.x86_64, cpuSubtype: 3 },
|
||||
]);
|
||||
const path = await writeFixture(directory, "universal.fat", fixture);
|
||||
const result = requireResolved(
|
||||
await resolveHopperMachOImage(fat32Target(path, fixture, "x86_64")),
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
cpu_type: CPU_TYPES.x86_64,
|
||||
cpu_subtype: 3,
|
||||
source_offset: fixture.sliceOffsets[1],
|
||||
});
|
||||
});
|
||||
|
||||
it("parses a little-endian FAT32 table without tying it to slice byte order", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-le-");
|
||||
const fixture = buildFat32Container(
|
||||
[
|
||||
{ cpuType: CPU_TYPES.x86_64, cpuSubtype: 3 },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E_X1 },
|
||||
],
|
||||
{ littleEndianTable: true },
|
||||
);
|
||||
const path = await writeFixture(directory, "little.fat", fixture);
|
||||
const result = requireResolved(
|
||||
await resolveHopperMachOImage(fat32Target(path, fixture, "arm64")),
|
||||
);
|
||||
expect(result).toMatchObject({
|
||||
method: "fat32_thin_slice",
|
||||
cpu_subtype: ARM64E,
|
||||
source_offset: fixture.sliceOffsets[1],
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects a FAT32 slice whose declared range extends beyond the file", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-range-");
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E },
|
||||
]);
|
||||
fixture.container.writeUInt32BE(fixture.container.length + 16, 8 + 12);
|
||||
const path = await writeFixture(directory, "overrange.fat", fixture);
|
||||
const result = await resolveHopperMachOImage(
|
||||
fat32Target(path, fixture, "arm64"),
|
||||
);
|
||||
if (result.ok) throw new Error("An overranging FAT32 slice was accepted");
|
||||
expect(
|
||||
projectAnalysisError(result.error).details?.reason ?? result.error,
|
||||
).toMatch(/extends beyond the file/u);
|
||||
});
|
||||
|
||||
it("copies exactly the selected FAT32 slice bytes and verifies the source", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-copy-");
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.x86_64, cpuSubtype: 3 },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E_X1 },
|
||||
]);
|
||||
const path = await writeFixture(directory, "system.fat", fixture);
|
||||
const target = fat32Target(path, fixture, "arm64");
|
||||
const resolved = requireResolved(await resolveHopperMachOImage(target));
|
||||
const destination = join(directory, "image.macho");
|
||||
const prepared = await prepareHopperMachOImage(
|
||||
path,
|
||||
target.sha256,
|
||||
resolved,
|
||||
destination,
|
||||
);
|
||||
if (!prepared.ok) throw new Error(prepared.error.userMessage);
|
||||
const arm64eOffset = fixture.sliceOffsets[1]!;
|
||||
const copied = fixture.container.subarray(
|
||||
arm64eOffset,
|
||||
arm64eOffset + FAT32_SLICE_SIZE,
|
||||
);
|
||||
expect(await readFile(destination)).toEqual(copied);
|
||||
});
|
||||
|
||||
it("rejects FAT32 preparation when the source table changed after resolution", async () => {
|
||||
const directory = await createTestTempDirectory("rea-hopper-fat32-race-");
|
||||
const fixture = buildFat32Container([
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E },
|
||||
{ cpuType: CPU_TYPES.arm64, cpuSubtype: ARM64E_X1 },
|
||||
]);
|
||||
const path = await writeFixture(directory, "system.fat", fixture);
|
||||
const target = fat32Target(path, fixture, "arm64");
|
||||
const resolved = requireResolved(await resolveHopperMachOImage(target));
|
||||
const mutated = Buffer.from(fixture.container);
|
||||
mutated.writeUInt32BE(0x01000007, 8);
|
||||
await writeFile(path, mutated);
|
||||
const destination = join(directory, "image.macho");
|
||||
const prepared = await prepareHopperMachOImage(
|
||||
path,
|
||||
target.sha256,
|
||||
resolved,
|
||||
destination,
|
||||
);
|
||||
if (prepared.ok)
|
||||
throw new Error("A mutated FAT32 source passed verification");
|
||||
expect(prepared.error.userMessage).toMatch(/no longer agrees/u);
|
||||
});
|
||||
@@ -8,7 +8,10 @@ import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
|
||||
import {
|
||||
readFatSliceDeclarations,
|
||||
validateFatSlice,
|
||||
type FatSliceDeclaration,
|
||||
} from "../domain/apple/machoContainer.js";
|
||||
import { machoCpuSubtypeBase } from "../domain/apple/machoCpuIdentity.js";
|
||||
import type { HeaderArchitecture } from "../domain/binaryTargetTypes.js";
|
||||
import {
|
||||
HopperCancelledError,
|
||||
HopperStartError,
|
||||
@@ -17,7 +20,10 @@ import { err, ok, type Result } from "../domain/result.js";
|
||||
|
||||
/** Stable source coordinates committed before preparing an owned thin image. */
|
||||
export const hopperMachOImageSchema = z.strictObject({
|
||||
method: z.literal("fat64_thin_slice"),
|
||||
method: z.union([
|
||||
z.literal("fat32_thin_slice"),
|
||||
z.literal("fat64_thin_slice"),
|
||||
]),
|
||||
architecture_count: z.number().int().min(1).max(128),
|
||||
container_table_sha256: z.string().regex(/^[a-f0-9]{64}$/u),
|
||||
source_offset: z.number().int().nonnegative().max(Number.MAX_SAFE_INTEGER),
|
||||
@@ -28,7 +34,7 @@ export const hopperMachOImageSchema = z.strictObject({
|
||||
|
||||
export type HopperMachOImage = z.infer<typeof hopperMachOImageSchema>;
|
||||
|
||||
/** Validate the selected FAT64 slice without depending on Apple's lipo tool. */
|
||||
/** Validate the selected FAT slice without depending on Apple's lipo tool. */
|
||||
export const resolveHopperMachOImage = async (
|
||||
target: BinaryTarget & { readonly kind: "executable" },
|
||||
signal?: AbortSignal,
|
||||
@@ -45,14 +51,14 @@ export const resolveHopperMachOImage = async (
|
||||
);
|
||||
if (signal?.aborted) return err(new AnalysisCancelledError("open_binary"));
|
||||
if (target.format !== "mach-o" && target.format !== "mach-o-universal")
|
||||
return invalid("Hopper FAT64 image preparation requires a Mach-O target");
|
||||
return invalid("Hopper FAT image preparation requires a Mach-O target");
|
||||
let file: FileHandle;
|
||||
try {
|
||||
file = await open(target.path, "r");
|
||||
} catch (cause: unknown) {
|
||||
// Keep the OS error so a read-permission denial reports access_denied.
|
||||
return invalid(
|
||||
`Cannot read FAT64 source for Hopper image preparation: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
`Cannot read FAT source for Hopper image preparation: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
cause,
|
||||
);
|
||||
}
|
||||
@@ -64,24 +70,29 @@ export const resolveHopperMachOImage = async (
|
||||
};
|
||||
const header = await read(0, 8);
|
||||
const magic = header.length === 8 ? header.readUInt32BE(0) : 0;
|
||||
if (magic !== 0xcafebabf && magic !== 0xbfbafeca)
|
||||
const isFatMagic = [
|
||||
0xcafebabe, 0xcafebabf, 0xbebafeca, 0xbfbafeca,
|
||||
].includes(magic);
|
||||
if (!isFatMagic)
|
||||
return invalid(
|
||||
"FAT64 source header changed before Hopper image preparation",
|
||||
"FAT source header changed before Hopper image preparation",
|
||||
);
|
||||
const littleEndian = magic === 0xbfbafeca;
|
||||
const wide = magic === 0xcafebabf || magic === 0xbfbafeca;
|
||||
const littleEndian = magic === 0xbebafeca || magic === 0xbfbafeca;
|
||||
const entrySize = wide ? 32 : 20;
|
||||
const count = littleEndian
|
||||
? header.readUInt32LE(4)
|
||||
: header.readUInt32BE(4);
|
||||
const size = (await file.stat()).size;
|
||||
const tableEnd = 8 + count * 32;
|
||||
const tableEnd = 8 + count * entrySize;
|
||||
if (count === 0 || count > 128 || tableEnd > size)
|
||||
return invalid(
|
||||
"FAT64 architecture table is truncated or exceeds the supported 128 entries",
|
||||
"FAT architecture table is truncated or exceeds the supported 128 entries",
|
||||
);
|
||||
const table = await read(8, count * 32);
|
||||
const table = await read(8, count * entrySize);
|
||||
const parsed = readFatSliceDeclarations(table, {
|
||||
count,
|
||||
wide: true,
|
||||
wide,
|
||||
littleEndian,
|
||||
});
|
||||
if (parsed.status === "malformed") return invalid(parsed.reason);
|
||||
@@ -93,24 +104,26 @@ export const resolveHopperMachOImage = async (
|
||||
return err(new AnalysisCancelledError("open_binary"));
|
||||
const rangeError = validateFatSlice(slice, size, tableEnd);
|
||||
if (rangeError !== null) return invalid(rangeError);
|
||||
const reserved = littleEndian
|
||||
? table.readUInt32LE(slice.index * 32 + 28)
|
||||
: table.readUInt32BE(slice.index * 32 + 28);
|
||||
if (reserved !== 0)
|
||||
return invalid(
|
||||
`FAT64 architecture ${slice.index} has a nonzero reserved field`,
|
||||
);
|
||||
if (wide) {
|
||||
const reserved = littleEndian
|
||||
? table.readUInt32LE(slice.index * 32 + 28)
|
||||
: table.readUInt32BE(slice.index * 32 + 28);
|
||||
if (reserved !== 0)
|
||||
return invalid(
|
||||
`FAT64 architecture ${slice.index} has a nonzero reserved field`,
|
||||
);
|
||||
}
|
||||
if (slice.size < 12)
|
||||
return invalid(
|
||||
`FAT64 architecture ${slice.index} has a truncated Mach-O header`,
|
||||
`FAT architecture ${slice.index} has a truncated Mach-O header`,
|
||||
);
|
||||
const embedded = await read(slice.offset, 12);
|
||||
if (embedded.length !== 12)
|
||||
return invalid(`FAT64 architecture ${slice.index} became truncated`);
|
||||
return invalid(`FAT architecture ${slice.index} became truncated`);
|
||||
const thinMagic = embedded.readUInt32BE(0);
|
||||
if (![0xfeedface, 0xfeedfacf, 0xcefaedfe, 0xcffaedfe].includes(thinMagic))
|
||||
return invalid(
|
||||
`FAT64 architecture ${slice.index} does not contain a Mach-O image`,
|
||||
`FAT architecture ${slice.index} does not contain a Mach-O image`,
|
||||
);
|
||||
const little = thinMagic === 0xcefaedfe || thinMagic === 0xcffaedfe;
|
||||
const identity = {
|
||||
@@ -133,15 +146,15 @@ export const resolveHopperMachOImage = async (
|
||||
current !== undefined &&
|
||||
current.offset < previous.offset + previous.size
|
||||
)
|
||||
return invalid("FAT64 slices overlap");
|
||||
return invalid("FAT slices overlap");
|
||||
}
|
||||
const selected = matches.length === 1 ? matches[0] : undefined;
|
||||
const selected = selectHopperSlice(matches, target.architecture);
|
||||
if (selected === undefined)
|
||||
return invalid(
|
||||
`FAT64 selection for ${target.architecture} is ${matches.length === 0 ? "missing" : "ambiguous"}; extract the exact architecture into a thin image before opening it`,
|
||||
`FAT selection for ${target.architecture} is ${matches.length === 0 ? "missing" : "ambiguous"}; extract the exact architecture into a thin image before opening it`,
|
||||
);
|
||||
return ok({
|
||||
method: "fat64_thin_slice",
|
||||
method: wide ? "fat64_thin_slice" : "fat32_thin_slice",
|
||||
architecture_count: count,
|
||||
container_table_sha256: createHash("sha256")
|
||||
.update(header)
|
||||
@@ -154,13 +167,35 @@ export const resolveHopperMachOImage = async (
|
||||
});
|
||||
} catch (cause: unknown) {
|
||||
return invalid(
|
||||
`Cannot validate FAT64 image: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
`Cannot validate FAT image: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
);
|
||||
} finally {
|
||||
await file.close();
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Plain arm64 (cpusubtype 0) and arm64e (cpusubtype 2) function-entry encodings
|
||||
* decode under Hopper's --aarch64; Apple's arm64e.x1 variant (cpusubtype 12)
|
||||
* does not, so it is selected only when it is the arm64 family's only candidate.
|
||||
* Every other ambiguity stays fail-closed.
|
||||
*/
|
||||
const selectHopperSlice = (
|
||||
matches: readonly FatSliceDeclaration[],
|
||||
architecture: HeaderArchitecture,
|
||||
): FatSliceDeclaration | undefined => {
|
||||
if (matches.length === 1) return matches[0];
|
||||
if (architecture !== "arm64") return undefined;
|
||||
for (const cpuSubtype of [0, 2]) {
|
||||
const preferred = matches.filter(
|
||||
(slice) => machoCpuSubtypeBase(slice.cpuSubtype) === cpuSubtype,
|
||||
);
|
||||
if (preferred.length === 1) return preferred[0];
|
||||
if (preferred.length > 1) return undefined;
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
/** Copy a selected slice and verify the entire original source in the same pass. */
|
||||
export const prepareHopperMachOImage = async (
|
||||
sourcePath: string,
|
||||
@@ -170,6 +205,7 @@ export const prepareHopperMachOImage = async (
|
||||
signal?: AbortSignal,
|
||||
): Promise<Result<string, HopperStartError | HopperCancelledError>> => {
|
||||
if (signal?.aborted) return err(new HopperCancelledError());
|
||||
const tableEntrySize = image.method === "fat64_thin_slice" ? 32 : 20;
|
||||
try {
|
||||
const source = await open(sourcePath, "r");
|
||||
try {
|
||||
@@ -177,7 +213,7 @@ export const prepareHopperMachOImage = async (
|
||||
try {
|
||||
const hash = createHash("sha256");
|
||||
const tableHash = createHash("sha256");
|
||||
const tableEnd = 8 + image.architecture_count * 32;
|
||||
const tableEnd = 8 + image.architecture_count * tableEntrySize;
|
||||
const thinHeader: Buffer[] = [];
|
||||
let position = 0;
|
||||
let copied = 0;
|
||||
@@ -223,13 +259,13 @@ export const prepareHopperMachOImage = async (
|
||||
)
|
||||
return err(
|
||||
new HopperStartError({
|
||||
userMessage: `FAT64 source ${sourcePath} no longer agrees with its committed architecture table or selected Mach-O header. Open the stable source again.`,
|
||||
userMessage: `FAT source ${sourcePath} no longer agrees with its committed architecture table or selected Mach-O header. Open the stable source again.`,
|
||||
}),
|
||||
);
|
||||
if (observed !== sourceSha256 || copied !== image.byte_length)
|
||||
return err(
|
||||
new HopperStartError({
|
||||
userMessage: `FAT64 source ${sourcePath} changed before launch: expected SHA-256 ${sourceSha256}, observed ${observed}; copied ${copied} of ${image.byte_length} selected bytes. Open the stable source again.`,
|
||||
userMessage: `FAT source ${sourcePath} changed before launch: expected SHA-256 ${sourceSha256}, observed ${observed}; copied ${copied} of ${image.byte_length} selected bytes. Open the stable source again.`,
|
||||
}),
|
||||
);
|
||||
return ok(destination);
|
||||
@@ -245,7 +281,7 @@ export const prepareHopperMachOImage = async (
|
||||
: err(
|
||||
new HopperStartError({
|
||||
cause,
|
||||
userMessage: `Cannot prepare the selected FAT64 image from ${sourcePath}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
userMessage: `Cannot prepare the selected FAT image from ${sourcePath}: ${cause instanceof Error ? cause.message : String(cause)}`,
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -288,7 +288,7 @@ export class HopperProvider implements AnalysisProviderCandidate {
|
||||
...(preparedImage === undefined
|
||||
? []
|
||||
: [
|
||||
"REA loaded a verified thin Mach-O slice from the original FAT64 source. Identity and file offsets refer to the original file. The owned temporary image and its Hopper document close together, including on MCP shutdown.",
|
||||
"REA loaded a verified thin Mach-O slice from the original FAT source. Identity and file offsets refer to the original file. The owned temporary image and its Hopper document close together, including on MCP shutdown.",
|
||||
]),
|
||||
...(CAPABILITIES.find(
|
||||
(descriptor) => descriptor.operation === operation,
|
||||
|
||||
@@ -38,11 +38,13 @@ import { assertNotCancelled } from "./ProcessScenarioRuntimeValidation.js";
|
||||
import {
|
||||
normalizeProcessElapsedTime,
|
||||
normalizeProcessSamples,
|
||||
type BoundedProcessTextNormalization,
|
||||
} from "./ProcessNormalization.js";
|
||||
import { PROCESS_PROVIDER } from "../../domain/process/processEvidenceProvider.js";
|
||||
import { TerminalRenderer } from "./TerminalRenderer.js";
|
||||
import {
|
||||
hasCaptureTruncation,
|
||||
type NormalizedTerminalRetention,
|
||||
type TerminalRetention,
|
||||
type ProcessCaptureTruncationDetails,
|
||||
} from "../../domain/process/processCaptureCoverage.js";
|
||||
@@ -408,7 +410,9 @@ const captureCoverageUnknowns = (
|
||||
unknowns.push({
|
||||
scope: "terminal",
|
||||
reason:
|
||||
"Raw PTY chunks exceeded output_bytes and were omitted; rendered states also lack those inputs. See truncation_details.raw_terminal.",
|
||||
(details.normalized_terminal?.omitted_frames ?? 0) > 0
|
||||
? "Raw PTY chunks were omitted because original input exceeded output_bytes or an ordered normalization stage exceeded its remaining bound; final normalized sizes for the latter omissions are unknown. Rendered states also lack those inputs. See raw_terminal and normalized_terminal retention details, including the normalized omission count."
|
||||
: "Raw PTY chunks exceeded output_bytes and were omitted; rendered states also lack those inputs. See truncation_details.raw_terminal.",
|
||||
});
|
||||
if (
|
||||
details.rendered_terminal.observed_frames >
|
||||
@@ -417,7 +421,9 @@ const captureCoverageUnknowns = (
|
||||
unknowns.push({
|
||||
scope: "terminal",
|
||||
reason:
|
||||
"Cumulative rendered state and visible-line bytes exceeded output_bytes; whole rendered frames were omitted. Raw PTY coverage is reported separately. See truncation_details.rendered_terminal.",
|
||||
(details.rendered_terminal.unknown_size_frames ?? 0) > 0
|
||||
? "Rendered frames were omitted because a normalization stage exceeded the remaining output_bytes budget; final rendered byte totals are unknown for those frames. See truncation_details.rendered_terminal."
|
||||
: "Cumulative rendered state and visible-line bytes exceeded output_bytes; whole rendered frames were omitted. Raw PTY coverage is reported separately. See truncation_details.rendered_terminal.",
|
||||
});
|
||||
for (const [name, coverage] of [
|
||||
["before", details.filesystem_before],
|
||||
@@ -1303,45 +1309,68 @@ export const captureTerminalFrames = (options: {
|
||||
readonly started: number;
|
||||
readonly onOutput: () => void;
|
||||
readonly renderer: TerminalRenderer;
|
||||
readonly normalizeText: (value: string) => string;
|
||||
readonly normalizeText: (
|
||||
value: string,
|
||||
maxBytes: number,
|
||||
) => BoundedProcessTextNormalization;
|
||||
readonly recordEvent: RecordProcessCaptureEvent;
|
||||
}): (() => TerminalRetention) => {
|
||||
let outputBytes = 0;
|
||||
}): (() => TerminalCaptureRetention) => {
|
||||
let rawBytes = 0;
|
||||
let normalizedBytes = 0;
|
||||
let observedBytes = 0;
|
||||
let observedFrames = 0;
|
||||
let normalizedOmittedFrames = 0;
|
||||
options.terminal.onData((data) => {
|
||||
options.onOutput();
|
||||
const bytes = Buffer.byteLength(data);
|
||||
observedBytes += bytes;
|
||||
observedFrames += 1;
|
||||
if (outputBytes + bytes > options.scenario.limits.output_bytes) {
|
||||
const budget = options.scenario.limits.output_bytes;
|
||||
if (bytes > budget - rawBytes) {
|
||||
return;
|
||||
}
|
||||
outputBytes += bytes;
|
||||
const normalized = options.normalizeText(data, budget - normalizedBytes);
|
||||
if (normalized.state === "over_budget") {
|
||||
normalizedOmittedFrames += 1;
|
||||
return;
|
||||
}
|
||||
rawBytes += bytes;
|
||||
normalizedBytes += normalized.utf8Bytes;
|
||||
const atMs = normalizeProcessElapsedTime(
|
||||
monotonicProcessCaptureClock() - options.started,
|
||||
options.scenario.normalization.time_bucket_ms,
|
||||
);
|
||||
const normalized = options.normalizeText(data);
|
||||
const sequence = options.frames.length;
|
||||
options.frames.push({
|
||||
sequence,
|
||||
at_ms: atMs,
|
||||
data: normalized,
|
||||
...(normalized === data ? {} : { raw_data: data }),
|
||||
data: normalized.value,
|
||||
...(normalized.value === data ? {} : { raw_data: data }),
|
||||
});
|
||||
options.renderer.write(data, atMs);
|
||||
options.recordEvent("frames", sequence);
|
||||
});
|
||||
return () => ({
|
||||
budget_bytes: options.scenario.limits.output_bytes,
|
||||
observed_bytes: observedBytes,
|
||||
retained_bytes: outputBytes,
|
||||
observed_frames: observedFrames,
|
||||
retained_frames: options.frames.length,
|
||||
raw_terminal: {
|
||||
budget_bytes: options.scenario.limits.output_bytes,
|
||||
observed_bytes: observedBytes,
|
||||
retained_bytes: rawBytes,
|
||||
observed_frames: observedFrames,
|
||||
retained_frames: options.frames.length,
|
||||
},
|
||||
normalized_terminal: {
|
||||
budget_bytes: options.scenario.limits.output_bytes,
|
||||
retained_bytes: normalizedBytes,
|
||||
omitted_frames: normalizedOmittedFrames,
|
||||
},
|
||||
});
|
||||
};
|
||||
|
||||
export interface TerminalCaptureRetention {
|
||||
readonly raw_terminal: TerminalRetention;
|
||||
readonly normalized_terminal: NormalizedTerminalRetention;
|
||||
}
|
||||
|
||||
export const resolveProcessResult = (
|
||||
capture: PendingProcessCapture | ProcessCapture | undefined,
|
||||
executionFailure: unknown,
|
||||
|
||||
@@ -66,7 +66,12 @@ it("returns detached terminal observations", async () => {
|
||||
rows: 12,
|
||||
scrollback: 100,
|
||||
maxBytes: 100_000,
|
||||
normalize: (value) => value,
|
||||
normalize: (value, maxBytes) => {
|
||||
const utf8Bytes = Buffer.byteLength(value);
|
||||
return utf8Bytes <= maxBytes
|
||||
? { state: "normalized", value, utf8Bytes }
|
||||
: { state: "over_budget" };
|
||||
},
|
||||
recordEvent: (collection, index) =>
|
||||
observed.push(`${collection}:${String(index)}`),
|
||||
});
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
import { expect, it } from "vitest";
|
||||
import type { IPty } from "@lydell/node-pty";
|
||||
|
||||
import { parseProcessScenario } from "../../domain/process/processScenario.js";
|
||||
import type { TerminalFrame } from "../../domain/process/processCapture.js";
|
||||
import { captureTerminalFrames } from "./ProcessCaptureLifecycle.js";
|
||||
import { createBoundedProcessTextNormalizer } from "./ProcessNormalization.js";
|
||||
|
||||
it("omits an expanding chunk whole and retains a later smaller normalized chunk", () => {
|
||||
let onData: ((data: string) => void) | undefined;
|
||||
const terminal = {
|
||||
onData: (listener: (data: string) => void) => {
|
||||
onData = listener;
|
||||
return { dispose: () => undefined };
|
||||
},
|
||||
} as unknown as IPty;
|
||||
const scenario = parseProcessScenario({
|
||||
executable: "unused",
|
||||
limits: { output_bytes: 1_000 },
|
||||
normalization: {
|
||||
paths: false,
|
||||
pids: false,
|
||||
ports: false,
|
||||
patterns: [{ pattern: "a", replacement: "x".repeat(500) }],
|
||||
},
|
||||
});
|
||||
const frames: TerminalFrame[] = [];
|
||||
const renderedChunks: string[] = [];
|
||||
const journal: Array<[string, number]> = [];
|
||||
const captureRetention = captureTerminalFrames({
|
||||
terminal,
|
||||
scenario,
|
||||
frames,
|
||||
started: 0,
|
||||
onOutput: () => undefined,
|
||||
renderer: { write: (data: string) => renderedChunks.push(data) } as never,
|
||||
normalizeText: createBoundedProcessTextNormalizer(scenario, "/tmp", -1),
|
||||
recordEvent: (collection, index) => journal.push([collection, index]),
|
||||
});
|
||||
|
||||
onData?.("aaa");
|
||||
onData?.("z");
|
||||
|
||||
expect(frames).toEqual([
|
||||
{ sequence: 0, at_ms: expect.any(Number), data: "z" },
|
||||
]);
|
||||
expect(renderedChunks).toEqual(["z"]);
|
||||
expect(journal).toEqual([["frames", 0]]);
|
||||
expect(captureRetention()).toEqual({
|
||||
raw_terminal: {
|
||||
budget_bytes: 1_000,
|
||||
observed_bytes: 4,
|
||||
retained_bytes: 1,
|
||||
observed_frames: 2,
|
||||
retained_frames: 1,
|
||||
},
|
||||
normalized_terminal: {
|
||||
budget_bytes: 1_000,
|
||||
retained_bytes: 1,
|
||||
omitted_frames: 1,
|
||||
},
|
||||
});
|
||||
});
|
||||
@@ -1,5 +1,4 @@
|
||||
import type { IPty } from "@lydell/node-pty";
|
||||
import type { TerminalRetention } from "../../domain/process/processCaptureCoverage.js";
|
||||
import type { ProcessCapture } from "../../domain/process/processCaptureParsing.js";
|
||||
import type { ProcessScenario } from "../../domain/process/processScenario.js";
|
||||
import { parseProcessCapture } from "../../domain/process/processCaptureParsing.js";
|
||||
@@ -24,7 +23,10 @@ import { startProcessSampler } from "./ProcessSampling.js";
|
||||
import { snapshotRoots } from "./FilesystemSnapshot.js";
|
||||
import { TerminalRenderer } from "./TerminalRenderer.js";
|
||||
import type { ProcessTimer } from "./ProcessTimer.js";
|
||||
import { createProcessTextNormalizer } from "./ProcessNormalization.js";
|
||||
import {
|
||||
createBoundedProcessTextNormalizer,
|
||||
type BoundedProcessTextNormalization,
|
||||
} from "./ProcessNormalization.js";
|
||||
import { selectCapturedProcessGroupIds } from "../ProcessOwnershipProcessTree.js";
|
||||
import {
|
||||
awaitTerminalExit,
|
||||
@@ -46,6 +48,7 @@ import {
|
||||
describeUnverifiedCause,
|
||||
type ProcessCaptureResourceCleanupOptions,
|
||||
ProcessCaptureResourceScope,
|
||||
type TerminalCaptureRetention,
|
||||
} from "./ProcessCaptureLifecycle.js";
|
||||
import {
|
||||
assertNotCancelled,
|
||||
@@ -79,7 +82,7 @@ interface StartedCaptureRuntime {
|
||||
readonly startedAt: Date;
|
||||
readonly executableIdentity: ReturnType<typeof observeLaunchedExecutable>;
|
||||
readonly lastOutput: () => number;
|
||||
readonly rawTerminalRetention: () => TerminalRetention;
|
||||
readonly terminalRetention: () => TerminalCaptureRetention;
|
||||
readonly stopSampler: ReturnType<typeof startProcessSampler>;
|
||||
/** Signals the captured root only while its launch-time start identity still matches. */
|
||||
readonly signalRoot: (signal: "SIGTERM" | "SIGKILL") => Promise<{
|
||||
@@ -222,7 +225,10 @@ const cleanupFailedStartup = async (options: {
|
||||
|
||||
const createTerminalRenderer = (
|
||||
scenario: ProcessScenario,
|
||||
normalizeText: (value: string) => string,
|
||||
normalizeText: (
|
||||
value: string,
|
||||
maxBytes: number,
|
||||
) => BoundedProcessTextNormalization,
|
||||
recordEvent: RecordProcessCaptureEvent,
|
||||
): TerminalRenderer =>
|
||||
new TerminalRenderer({
|
||||
@@ -269,7 +275,7 @@ const startCaptureRuntime = async (
|
||||
const startedAt = new Date();
|
||||
const started = monotonicProcessCaptureClock();
|
||||
let lastOutput = started;
|
||||
const normalizeText = createProcessTextNormalizer(
|
||||
const normalizeText = createBoundedProcessTextNormalizer(
|
||||
scenario,
|
||||
options.temporaryRoot,
|
||||
() => terminal?.pid ?? -1,
|
||||
@@ -295,7 +301,7 @@ const startCaptureRuntime = async (
|
||||
scenario.executable,
|
||||
selectedExecutable,
|
||||
);
|
||||
const rawTerminalRetention = captureTerminalFrames({
|
||||
const terminalRetention = captureTerminalFrames({
|
||||
...options,
|
||||
terminal,
|
||||
started,
|
||||
@@ -324,7 +330,7 @@ const startCaptureRuntime = async (
|
||||
startedAt,
|
||||
executableIdentity,
|
||||
lastOutput: () => lastOutput,
|
||||
rawTerminalRetention,
|
||||
terminalRetention,
|
||||
stopSampler,
|
||||
signalRoot: retainRootSignaller(
|
||||
terminal.pid,
|
||||
@@ -575,7 +581,7 @@ const completeCapture = async (options: {
|
||||
before: options.before,
|
||||
after,
|
||||
truncationDetails: {
|
||||
raw_terminal: runtime.rawTerminalRetention(),
|
||||
...runtime.terminalRetention(),
|
||||
rendered_terminal: runtime.renderer.retention(),
|
||||
filesystem_before: options.before.coverage,
|
||||
filesystem_after: after.coverage,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
createBoundedProcessTextNormalizer,
|
||||
createProcessTextNormalizer,
|
||||
normalizeProcessSamples,
|
||||
normalizeProcessText,
|
||||
@@ -95,6 +96,178 @@ it("replaces caller-declared patterns literally, without expanding replacement p
|
||||
).toBe("listening on $&$&; $$; $1=alpha; tail $1");
|
||||
});
|
||||
|
||||
it("bounds ordered normalization before replacement expansion and preserves later smaller chunks", () => {
|
||||
const normalize = createBoundedProcessTextNormalizer(
|
||||
scenario({
|
||||
paths: false,
|
||||
ports: false,
|
||||
pids: false,
|
||||
patterns: [{ pattern: "a", replacement: "x".repeat(500) }],
|
||||
}),
|
||||
"/temporary",
|
||||
rootPid,
|
||||
);
|
||||
|
||||
expect(normalize("a".repeat(100), 1_000)).toEqual({ state: "over_budget" });
|
||||
expect(normalize("z", 1_000)).toEqual({
|
||||
state: "normalized",
|
||||
value: "z",
|
||||
utf8Bytes: 1,
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps ordered intermediate expansion explicit even when a later rule could shrink it", () => {
|
||||
const normalize = createBoundedProcessTextNormalizer(
|
||||
scenario({
|
||||
paths: false,
|
||||
ports: false,
|
||||
pids: false,
|
||||
patterns: [
|
||||
{ pattern: "a", replacement: "x".repeat(20) },
|
||||
{ pattern: "x", replacement: "y" },
|
||||
],
|
||||
}),
|
||||
"/temporary",
|
||||
rootPid,
|
||||
);
|
||||
|
||||
expect(normalize("a", 10)).toEqual({ state: "over_budget" });
|
||||
});
|
||||
|
||||
it("counts UTF-8 bytes across replacement boundaries like the final JavaScript string", () => {
|
||||
const splitPair = scenario({
|
||||
paths: false,
|
||||
ports: false,
|
||||
pids: false,
|
||||
patterns: [{ pattern: "\ude00", replacement: "\ude01" }],
|
||||
});
|
||||
const splitPairResult = createBoundedProcessTextNormalizer(
|
||||
splitPair,
|
||||
"/temporary",
|
||||
rootPid,
|
||||
)("😀", 4);
|
||||
expect(splitPairResult).toEqual({
|
||||
state: "normalized",
|
||||
value: normalizeProcessText("😀", splitPair, "/temporary", rootPid),
|
||||
utf8Bytes: 4,
|
||||
});
|
||||
expect(
|
||||
splitPairResult.state === "normalized" && splitPairResult.utf8Bytes,
|
||||
).toBe(
|
||||
Buffer.byteLength(
|
||||
normalizeProcessText("😀", splitPair, "/temporary", rootPid),
|
||||
),
|
||||
);
|
||||
|
||||
const joinedPair = scenario({
|
||||
paths: false,
|
||||
ports: false,
|
||||
pids: false,
|
||||
patterns: [{ pattern: "\ude00\ud83d", replacement: "" }],
|
||||
});
|
||||
const joinedInput = "😀😀";
|
||||
const joinedExpected = normalizeProcessText(
|
||||
joinedInput,
|
||||
joinedPair,
|
||||
"/temporary",
|
||||
rootPid,
|
||||
);
|
||||
expect(joinedExpected).toBe("😀");
|
||||
expect(
|
||||
createBoundedProcessTextNormalizer(
|
||||
joinedPair,
|
||||
"/temporary",
|
||||
rootPid,
|
||||
)(joinedInput, Buffer.byteLength(joinedExpected)),
|
||||
).toEqual({
|
||||
state: "normalized",
|
||||
value: joinedExpected,
|
||||
utf8Bytes: Buffer.byteLength(joinedExpected),
|
||||
});
|
||||
|
||||
const emptyPattern = scenario({
|
||||
paths: false,
|
||||
ports: false,
|
||||
pids: false,
|
||||
patterns: [{ pattern: "", replacement: "\ude00" }],
|
||||
});
|
||||
const input = "\ud83d";
|
||||
const expected = normalizeProcessText(
|
||||
input,
|
||||
emptyPattern,
|
||||
"/temporary",
|
||||
rootPid,
|
||||
);
|
||||
expect(
|
||||
createBoundedProcessTextNormalizer(
|
||||
emptyPattern,
|
||||
"/temporary",
|
||||
rootPid,
|
||||
)(input, Buffer.byteLength(expected)),
|
||||
).toEqual({
|
||||
state: "normalized",
|
||||
value: expected,
|
||||
utf8Bytes: Buffer.byteLength(expected),
|
||||
});
|
||||
expect(
|
||||
createBoundedProcessTextNormalizer(
|
||||
emptyPattern,
|
||||
"/temporary",
|
||||
rootPid,
|
||||
)(input, Buffer.byteLength(expected) - 1),
|
||||
).toEqual({ state: "over_budget" });
|
||||
|
||||
const loneSurrogate = scenario({
|
||||
paths: false,
|
||||
ports: false,
|
||||
pids: false,
|
||||
patterns: [{ pattern: "x", replacement: "y" }],
|
||||
});
|
||||
const loneInput = "\ud83dx";
|
||||
const loneExpected = normalizeProcessText(
|
||||
loneInput,
|
||||
loneSurrogate,
|
||||
"/temporary",
|
||||
rootPid,
|
||||
);
|
||||
expect(
|
||||
createBoundedProcessTextNormalizer(
|
||||
loneSurrogate,
|
||||
"/temporary",
|
||||
rootPid,
|
||||
)(loneInput, Buffer.byteLength(loneExpected)),
|
||||
).toEqual({
|
||||
state: "normalized",
|
||||
value: loneExpected,
|
||||
utf8Bytes: Buffer.byteLength(loneExpected),
|
||||
});
|
||||
});
|
||||
|
||||
it("lets a later pattern shrink text when an earlier pattern does not match", () => {
|
||||
const declared = scenario({
|
||||
paths: false,
|
||||
ports: false,
|
||||
pids: false,
|
||||
patterns: [
|
||||
{ pattern: "z", replacement: "x" },
|
||||
{ pattern: "aa", replacement: "b" },
|
||||
],
|
||||
});
|
||||
const expected = normalizeProcessText("aa", declared, "/temporary", rootPid);
|
||||
expect(expected).toBe("b");
|
||||
expect(
|
||||
createBoundedProcessTextNormalizer(
|
||||
declared,
|
||||
"/temporary",
|
||||
rootPid,
|
||||
)("aa", Buffer.byteLength(expected)),
|
||||
).toEqual({
|
||||
state: "normalized",
|
||||
value: expected,
|
||||
utf8Bytes: Buffer.byteLength(expected),
|
||||
});
|
||||
});
|
||||
|
||||
it("reuses matchers while observing the terminal PID after launch", () => {
|
||||
let pid = -1;
|
||||
const normalize = createProcessTextNormalizer(
|
||||
|
||||
@@ -3,6 +3,165 @@ import { isIP } from "node:net";
|
||||
import type { ProcessScenario } from "../../domain/process/processScenario.js";
|
||||
|
||||
import type { ProcessSample } from "../../domain/process/processCapture.js";
|
||||
|
||||
export type BoundedProcessTextNormalization =
|
||||
| {
|
||||
readonly state: "normalized";
|
||||
readonly value: string;
|
||||
readonly utf8Bytes: number;
|
||||
}
|
||||
| { readonly state: "over_budget" };
|
||||
|
||||
type RegexReplacement = (match: string, ...groups: string[]) => string;
|
||||
|
||||
type BoundedStageResult =
|
||||
| {
|
||||
readonly state: "bounded";
|
||||
readonly value: string;
|
||||
readonly utf8Bytes: number;
|
||||
}
|
||||
| { readonly state: "unchanged"; readonly value: string }
|
||||
| { readonly state: "over_budget" };
|
||||
|
||||
const createUtf8ByteCounter = (maxBytes: number) => {
|
||||
let bytes = 0;
|
||||
let pendingHighSurrogate = false;
|
||||
const addBytes = (added: number): void => {
|
||||
if (bytes > maxBytes || added > maxBytes - bytes) {
|
||||
bytes = Number.POSITIVE_INFINITY;
|
||||
return;
|
||||
}
|
||||
bytes += added;
|
||||
};
|
||||
return {
|
||||
add: (value: string): void => {
|
||||
if (bytes > maxBytes || value.length === 0) return;
|
||||
let index = 0;
|
||||
if (pendingHighSurrogate) {
|
||||
if (value.length > 0 && isLowSurrogate(value.charCodeAt(0))) {
|
||||
addBytes(4);
|
||||
index = 1;
|
||||
} else {
|
||||
addBytes(3);
|
||||
}
|
||||
pendingHighSurrogate = false;
|
||||
}
|
||||
for (; index < value.length; index += 1) {
|
||||
const code = value.charCodeAt(index);
|
||||
if (code <= 0x7f) addBytes(1);
|
||||
else if (code <= 0x7ff) addBytes(2);
|
||||
else if (isHighSurrogate(code)) {
|
||||
if (index + 1 === value.length) {
|
||||
pendingHighSurrogate = true;
|
||||
} else if (isLowSurrogate(value.charCodeAt(index + 1))) {
|
||||
addBytes(4);
|
||||
index += 1;
|
||||
} else addBytes(3);
|
||||
} else addBytes(3);
|
||||
if (bytes > maxBytes) return;
|
||||
}
|
||||
},
|
||||
finish: (): number => {
|
||||
if (pendingHighSurrogate) addBytes(3);
|
||||
return bytes;
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const isHighSurrogate = (code: number): boolean =>
|
||||
code >= 0xd800 && code <= 0xdbff;
|
||||
const isLowSurrogate = (code: number): boolean =>
|
||||
code >= 0xdc00 && code <= 0xdfff;
|
||||
|
||||
const boundedRegexReplace = (
|
||||
value: string,
|
||||
matcher: RegExp,
|
||||
replacement: RegexReplacement,
|
||||
maxBytes: number,
|
||||
): BoundedStageResult => {
|
||||
const byteCounter = createUtf8ByteCounter(maxBytes);
|
||||
let cursor = 0;
|
||||
let changed = false;
|
||||
matcher.lastIndex = 0;
|
||||
let match: RegExpExecArray | null;
|
||||
while ((match = matcher.exec(value)) !== null) {
|
||||
const index = match.index;
|
||||
const matched = match[0] ?? "";
|
||||
const nextCursor = index + matched.length;
|
||||
const replacementValue = replacement(matched, ...match.slice(1));
|
||||
changed ||= replacementValue !== matched;
|
||||
byteCounter.add(value.slice(cursor, index));
|
||||
byteCounter.add(replacementValue);
|
||||
cursor = nextCursor;
|
||||
}
|
||||
byteCounter.add(value.slice(cursor));
|
||||
const bytes = byteCounter.finish();
|
||||
if (bytes > maxBytes) {
|
||||
return changed ? { state: "over_budget" } : { state: "unchanged", value };
|
||||
}
|
||||
if (!changed) return { state: "unchanged", value };
|
||||
matcher.lastIndex = 0;
|
||||
const normalized = value.replace(matcher, replacement);
|
||||
return {
|
||||
state: "bounded",
|
||||
value: normalized,
|
||||
utf8Bytes: bytes,
|
||||
};
|
||||
};
|
||||
|
||||
const boundedLiteralReplace = (
|
||||
value: string,
|
||||
pattern: string,
|
||||
replacement: string,
|
||||
maxBytes: number,
|
||||
): BoundedStageResult => {
|
||||
const byteCounter = createUtf8ByteCounter(maxBytes);
|
||||
let cursor = 0;
|
||||
if (pattern === "") {
|
||||
if (replacement === "") return { state: "unchanged", value };
|
||||
for (let index = 0; index < value.length; index += 1) {
|
||||
byteCounter.add(replacement);
|
||||
byteCounter.add(value.slice(index, index + 1));
|
||||
}
|
||||
byteCounter.add(replacement);
|
||||
const bytes = byteCounter.finish();
|
||||
if (bytes > maxBytes) return { state: "over_budget" };
|
||||
return {
|
||||
state: "bounded",
|
||||
value: value.replaceAll(pattern, () => replacement),
|
||||
utf8Bytes: bytes,
|
||||
};
|
||||
}
|
||||
if (pattern === replacement) return { state: "unchanged", value };
|
||||
|
||||
let index = value.indexOf(pattern, cursor);
|
||||
if (index === -1) return { state: "unchanged", value };
|
||||
while (index !== -1) {
|
||||
byteCounter.add(value.slice(cursor, index));
|
||||
byteCounter.add(replacement);
|
||||
cursor = index + pattern.length;
|
||||
index = value.indexOf(pattern, cursor);
|
||||
}
|
||||
byteCounter.add(value.slice(cursor));
|
||||
const bytes = byteCounter.finish();
|
||||
if (bytes > maxBytes) return { state: "over_budget" };
|
||||
return {
|
||||
state: "bounded",
|
||||
value: value.replaceAll(pattern, () => replacement),
|
||||
utf8Bytes: bytes,
|
||||
};
|
||||
};
|
||||
|
||||
const boundedIdentity = (
|
||||
value: string,
|
||||
maxBytes: number,
|
||||
): BoundedStageResult => {
|
||||
const utf8Bytes = Buffer.byteLength(value);
|
||||
return utf8Bytes <= maxBytes
|
||||
? { state: "bounded", value, utf8Bytes }
|
||||
: { state: "unchanged", value };
|
||||
};
|
||||
|
||||
/** Bucket one elapsed process-capture timestamp under scenario normalization. */
|
||||
export const normalizeProcessElapsedTime = (
|
||||
elapsedMs: number,
|
||||
@@ -41,6 +200,84 @@ export const createProcessTextNormalizer = (
|
||||
};
|
||||
};
|
||||
|
||||
/** Normalize terminal text without materializing an intermediate over its byte budget. */
|
||||
export const createBoundedProcessTextNormalizer = (
|
||||
scenario: ProcessScenario,
|
||||
temporaryRoot: string,
|
||||
pid: number | (() => number),
|
||||
): ((value: string, maxBytes: number) => BoundedProcessTextNormalization) => {
|
||||
const pathRule = scenario.normalization.paths
|
||||
? createPathRule(scenario, temporaryRoot)
|
||||
: undefined;
|
||||
let previousPid: number | undefined;
|
||||
let pidMatcher: RegExp | undefined;
|
||||
return (value, maxBytes) => {
|
||||
let normalized = value;
|
||||
const apply = (result: BoundedStageResult): boolean => {
|
||||
if (result.state === "over_budget") return false;
|
||||
if (result.state === "bounded") normalized = result.value;
|
||||
return true;
|
||||
};
|
||||
if (pathRule !== undefined) {
|
||||
if (
|
||||
!apply(
|
||||
boundedRegexReplace(
|
||||
normalized,
|
||||
pathRule.matcher,
|
||||
pathRule.replace,
|
||||
maxBytes,
|
||||
),
|
||||
)
|
||||
)
|
||||
return { state: "over_budget" };
|
||||
}
|
||||
if (scenario.normalization.ports) {
|
||||
if (!apply(normalizePortsBounded(normalized, maxBytes)))
|
||||
return { state: "over_budget" };
|
||||
}
|
||||
if (scenario.normalization.pids) {
|
||||
const currentPid = typeof pid === "number" ? pid : pid();
|
||||
if (currentPid !== previousPid) {
|
||||
previousPid = currentPid;
|
||||
pidMatcher = createPidMatcher([currentPid]);
|
||||
}
|
||||
if (pidMatcher !== undefined) {
|
||||
if (
|
||||
!apply(
|
||||
boundedRegexReplace(
|
||||
normalized,
|
||||
pidMatcher,
|
||||
() => "<pid>",
|
||||
maxBytes,
|
||||
),
|
||||
)
|
||||
)
|
||||
return { state: "over_budget" };
|
||||
}
|
||||
}
|
||||
for (const pattern of scenario.normalization.patterns) {
|
||||
if (
|
||||
!apply(
|
||||
boundedLiteralReplace(
|
||||
normalized,
|
||||
pattern.pattern,
|
||||
pattern.replacement,
|
||||
maxBytes,
|
||||
),
|
||||
)
|
||||
)
|
||||
return { state: "over_budget" };
|
||||
}
|
||||
return boundedIdentity(normalized, maxBytes).state === "bounded"
|
||||
? {
|
||||
state: "normalized",
|
||||
value: normalized,
|
||||
utf8Bytes: Buffer.byteLength(normalized),
|
||||
}
|
||||
: { state: "over_budget" };
|
||||
};
|
||||
};
|
||||
|
||||
/** Normalize one payload when its caller does not retain a capture-local matcher. */
|
||||
export const normalizeProcessText = (
|
||||
value: string,
|
||||
@@ -50,10 +287,12 @@ export const normalizeProcessText = (
|
||||
): string => createProcessTextNormalizer(scenario, temporaryRoot, pid)(value);
|
||||
|
||||
/** Match original paths once, preferring the most specific selected identity. */
|
||||
const createPathNormalizer = (
|
||||
const createPathRule = (
|
||||
scenario: ProcessScenario,
|
||||
temporaryRoot: string,
|
||||
): ((value: string) => string) => {
|
||||
):
|
||||
| { readonly matcher: RegExp; readonly replace: RegexReplacement }
|
||||
| undefined => {
|
||||
const aliases = new Map<string, string>();
|
||||
const selected: readonly (readonly [string, string])[] = [
|
||||
[temporaryRoot, "<temporary-root>"],
|
||||
@@ -71,12 +310,24 @@ const createPathNormalizer = (
|
||||
const paths = [...aliases.keys()].sort(
|
||||
(left, right) => right.length - left.length,
|
||||
);
|
||||
if (paths.length === 0) return (value) => value;
|
||||
if (paths.length === 0) return undefined;
|
||||
const pattern = paths
|
||||
.map((path) => path.replace(/[.*+?^${}()|[\]\\]/gu, "\\$&"))
|
||||
.join("|");
|
||||
const matcher = new RegExp(pattern, "gu");
|
||||
return (value) => value.replace(matcher, (path) => aliases.get(path) ?? path);
|
||||
return {
|
||||
matcher,
|
||||
replace: (path) => aliases.get(path) ?? path,
|
||||
};
|
||||
};
|
||||
|
||||
const createPathNormalizer = (
|
||||
scenario: ProcessScenario,
|
||||
temporaryRoot: string,
|
||||
): ((value: string) => string) => {
|
||||
const rule = createPathRule(scenario, temporaryRoot);
|
||||
if (rule === undefined) return (value) => value;
|
||||
return (value) => value.replace(rule.matcher, rule.replace);
|
||||
};
|
||||
|
||||
// A port may end a sentence, but a period followed by a letter, digit, or
|
||||
@@ -85,21 +336,45 @@ const createPathNormalizer = (
|
||||
// PTY chunks are normalized independently, so a period that ends the chunk
|
||||
// stays unnormalized: its continuation may arrive in the next chunk.
|
||||
const normalizePortTokens = (value: string): string => {
|
||||
const endpointNormalized = value.replaceAll(
|
||||
/(\b[a-z][a-z0-9+.-]*:\/\/[^\s/?#"'<>]+|\blocalhost|\b(?:\d{1,3}\.){3}\d{1,3}|\[[\da-f:.%]+\]):(\d{1,5})(?![\p{L}\p{N}_@:]|\.(?:[\p{L}\p{N}_]|$))/giu,
|
||||
(match: string, endpoint: string, port: string) => {
|
||||
if (Number(port) > 65_535) return match;
|
||||
const isEndpoint = endpoint.includes("://")
|
||||
? URL.canParse(`${endpoint}:${port}`)
|
||||
: endpoint.toLowerCase() === "localhost" ||
|
||||
isIP(endpoint.replace(/^\[|\]$/gu, "")) !== 0;
|
||||
return isEndpoint ? `${endpoint}:<port>` : match;
|
||||
},
|
||||
return value
|
||||
.replaceAll(PORT_ENDPOINT_MATCHER, portEndpointReplacement)
|
||||
.replaceAll(PORT_FIELD_MATCHER, portFieldReplacement);
|
||||
};
|
||||
|
||||
const PORT_ENDPOINT_MATCHER =
|
||||
/(\b[a-z][a-z0-9+.-]*:\/\/[^\s/?#"'<>]+|\blocalhost|\b(?:\d{1,3}\.){3}\d{1,3}|\[[\da-f:.%]+\]):(\d{1,5})(?![\p{L}\p{N}_@:]|\.(?:[\p{L}\p{N}_]|$))/giu;
|
||||
const PORT_FIELD_MATCHER =
|
||||
/(\b(?:port|tcp_port|udp_port|listen)\b["']?\s*[:=]\s*)(\d{1,5})(?![\p{L}\p{N}_]|\.(?:[\p{L}\p{N}_]|$))/giu;
|
||||
|
||||
const portEndpointReplacement: RegexReplacement = (match, endpoint, port) => {
|
||||
if (Number(port) > 65_535) return match;
|
||||
const isEndpoint = endpoint.includes("://")
|
||||
? URL.canParse(`${endpoint}:${port}`)
|
||||
: endpoint.toLowerCase() === "localhost" ||
|
||||
isIP(endpoint.replace(/^\[|\]$/gu, "")) !== 0;
|
||||
return isEndpoint ? `${endpoint}:<port>` : match;
|
||||
};
|
||||
|
||||
const portFieldReplacement: RegexReplacement = (match, prefix, port) =>
|
||||
Number(port) <= 65_535 ? `${prefix}<port>` : match;
|
||||
|
||||
const normalizePortsBounded = (
|
||||
value: string,
|
||||
maxBytes: number,
|
||||
): BoundedStageResult => {
|
||||
const endpoint = boundedRegexReplace(
|
||||
value,
|
||||
PORT_ENDPOINT_MATCHER,
|
||||
portEndpointReplacement,
|
||||
maxBytes,
|
||||
);
|
||||
return endpointNormalized.replaceAll(
|
||||
/(\b(?:port|tcp_port|udp_port|listen)\b["']?\s*[:=]\s*)(\d{1,5})(?![\p{L}\p{N}_]|\.(?:[\p{L}\p{N}_]|$))/giu,
|
||||
(match: string, prefix: string, port: string) =>
|
||||
Number(port) <= 65_535 ? `${prefix}<port>` : match,
|
||||
if (endpoint.state === "over_budget") return endpoint;
|
||||
const afterEndpoint = endpoint.state === "bounded" ? endpoint.value : value;
|
||||
return boundedRegexReplace(
|
||||
afterEndpoint,
|
||||
PORT_FIELD_MATCHER,
|
||||
portFieldReplacement,
|
||||
maxBytes,
|
||||
);
|
||||
};
|
||||
|
||||
@@ -161,11 +436,18 @@ export const normalizeProcessSamples = (
|
||||
const createPidNormalizer = (
|
||||
identifiers: readonly number[],
|
||||
): ((value: string) => string) => {
|
||||
const matcher = createPidMatcher(identifiers);
|
||||
if (matcher === undefined) return (value) => value;
|
||||
return (value) => value.replace(matcher, "<pid>");
|
||||
};
|
||||
|
||||
const createPidMatcher = (
|
||||
identifiers: readonly number[],
|
||||
): RegExp | undefined => {
|
||||
const tokens = [...new Set(identifiers)]
|
||||
.filter((identifier) => Number.isSafeInteger(identifier) && identifier > 0)
|
||||
.sort((left, right) => right - left)
|
||||
.map(String);
|
||||
if (tokens.length === 0) return (value) => value;
|
||||
const matcher = new RegExp(`(?<!\\d)(?:${tokens.join("|")})(?!\\d)`, "gu");
|
||||
return (value) => value.replace(matcher, "<pid>");
|
||||
if (tokens.length === 0) return undefined;
|
||||
return new RegExp(`(?<!\\d)(?:${tokens.join("|")})(?!\\d)`, "gu");
|
||||
};
|
||||
@@ -6,6 +6,9 @@ import { expect, it, vi } from "vitest";
|
||||
|
||||
import { TerminalRenderer } from "./TerminalRenderer.js";
|
||||
import { ProcessCaptureResourceScope } from "./ProcessCaptureLifecycle.js";
|
||||
import { createBoundedProcessTextNormalizer } from "./ProcessNormalization.js";
|
||||
import { parseProcessScenario } from "../../domain/process/processScenario.js";
|
||||
import type { BoundedProcessTextNormalization } from "./ProcessNormalization.js";
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
const { Terminal } =
|
||||
@@ -13,6 +16,18 @@ const { Terminal } =
|
||||
const { SerializeAddon } =
|
||||
require("@xterm/addon-serialize") as typeof import("@xterm/addon-serialize");
|
||||
|
||||
const boundedNormalize =
|
||||
(
|
||||
normalize: (value: string) => string,
|
||||
): ((value: string, maxBytes: number) => BoundedProcessTextNormalization) =>
|
||||
(value, maxBytes) => {
|
||||
const normalized = normalize(value);
|
||||
const utf8Bytes = Buffer.byteLength(normalized);
|
||||
return utf8Bytes <= maxBytes
|
||||
? { state: "normalized", value: normalized, utf8Bytes }
|
||||
: { state: "over_budget" };
|
||||
};
|
||||
|
||||
const renderFrames = async (
|
||||
data: string,
|
||||
normalize: (value: string) => string = (value) => value,
|
||||
@@ -22,7 +37,7 @@ const renderFrames = async (
|
||||
rows: 4,
|
||||
scrollback: 10,
|
||||
maxBytes: 100_000,
|
||||
normalize,
|
||||
normalize: boundedNormalize(normalize),
|
||||
});
|
||||
renderer.write(data, 0);
|
||||
const frames = await renderer.frames();
|
||||
@@ -56,7 +71,7 @@ it("applies rendered retention cumulatively across sparse frames", async () => {
|
||||
rows: 1,
|
||||
scrollback: 0,
|
||||
maxBytes: 5,
|
||||
normalize: (value) => value,
|
||||
normalize: boundedNormalize((value) => value),
|
||||
});
|
||||
try {
|
||||
renderer.write("a", 0);
|
||||
@@ -66,10 +81,11 @@ it("applies rendered retention cumulatively across sparse frames", async () => {
|
||||
expect(frames.map(({ lines }) => lines)).toEqual([["a"]]);
|
||||
expect(renderer.retention()).toEqual({
|
||||
budget_bytes: 5,
|
||||
observed_bytes: 6,
|
||||
observed_bytes: 2,
|
||||
retained_bytes: 2,
|
||||
observed_frames: 2,
|
||||
retained_frames: 1,
|
||||
unknown_size_frames: 1,
|
||||
});
|
||||
expect(renderer.truncated()).toBe(true);
|
||||
} finally {
|
||||
@@ -84,6 +100,44 @@ it("trims only after normalization sees the full-width row", async () => {
|
||||
expect(frames[0]?.lines[0]).toBe("pid <pid>");
|
||||
});
|
||||
|
||||
it("omits rendered frames with unknown expanded size and keeps later bounded states", async () => {
|
||||
const renderer = new TerminalRenderer({
|
||||
columns: 10,
|
||||
rows: 1,
|
||||
scrollback: 0,
|
||||
maxBytes: 1_000,
|
||||
normalize: createBoundedProcessTextNormalizer(
|
||||
parseProcessScenario({
|
||||
executable: "unused",
|
||||
normalization: {
|
||||
paths: false,
|
||||
pids: false,
|
||||
ports: false,
|
||||
patterns: [{ pattern: "a", replacement: "x".repeat(2_000) }],
|
||||
},
|
||||
}),
|
||||
"/tmp",
|
||||
-1,
|
||||
),
|
||||
});
|
||||
try {
|
||||
renderer.write("a", 0);
|
||||
renderer.write("\u001b[2J\rz", 1);
|
||||
const frames = await renderer.frames();
|
||||
|
||||
expect(frames).toHaveLength(1);
|
||||
expect(frames[0]?.lines[0]).toBe("z");
|
||||
expect(renderer.retention()).toMatchObject({
|
||||
observed_frames: 2,
|
||||
retained_frames: 1,
|
||||
unknown_size_frames: 1,
|
||||
});
|
||||
expect(renderer.truncated()).toBe(true);
|
||||
} finally {
|
||||
await renderer.dispose();
|
||||
}
|
||||
});
|
||||
|
||||
it.each(["write", "resize"] as const)(
|
||||
"releases terminal resources after a queued %s observation fails",
|
||||
async (operation) => {
|
||||
@@ -95,9 +149,9 @@ it.each(["write", "resize"] as const)(
|
||||
rows: 4,
|
||||
scrollback: 10,
|
||||
maxBytes: 100_000,
|
||||
normalize: () => {
|
||||
normalize: boundedNormalize(() => {
|
||||
throw failure;
|
||||
},
|
||||
}),
|
||||
});
|
||||
const scope = new ProcessCaptureResourceScope();
|
||||
const temporaryRoot = await mkdtemp(
|
||||
@@ -147,7 +201,7 @@ it("retains disposal failures for a later cleanup retry", async () => {
|
||||
rows: 4,
|
||||
scrollback: 10,
|
||||
maxBytes: 100_000,
|
||||
normalize: (value) => value,
|
||||
normalize: boundedNormalize((value) => value),
|
||||
});
|
||||
const scope = new ProcessCaptureResourceScope();
|
||||
const temporaryRoot = await mkdtemp(join(tmpdir(), "rea-terminal-retry-"));
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { createRequire } from "node:module";
|
||||
import type { TerminalRetention } from "../../domain/process/processCaptureCoverage.js";
|
||||
import type { BoundedProcessTextNormalization } from "./ProcessNormalization.js";
|
||||
|
||||
import type {
|
||||
RecordProcessCaptureEvent,
|
||||
@@ -21,7 +22,10 @@ interface TerminalRendererOptions {
|
||||
readonly rows: number;
|
||||
readonly scrollback: number;
|
||||
readonly maxBytes: number;
|
||||
readonly normalize: (value: string) => string;
|
||||
readonly normalize: (
|
||||
value: string,
|
||||
maxBytes: number,
|
||||
) => BoundedProcessTextNormalization;
|
||||
readonly recordEvent?: RecordProcessCaptureEvent;
|
||||
}
|
||||
|
||||
@@ -42,6 +46,7 @@ export class TerminalRenderer {
|
||||
#observedBytes = 0;
|
||||
#observedFrames = 0;
|
||||
#truncated = false;
|
||||
#unknownSizeFrames = 0;
|
||||
|
||||
constructor(private readonly options: TerminalRendererOptions) {
|
||||
this.#terminal = new HeadlessPackage.Terminal({
|
||||
@@ -97,6 +102,9 @@ export class TerminalRenderer {
|
||||
retained_bytes: this.#capturedBytes,
|
||||
observed_frames: this.#observedFrames,
|
||||
retained_frames: this.#frames.length,
|
||||
...(this.#unknownSizeFrames === 0
|
||||
? {}
|
||||
: { unknown_size_frames: this.#unknownSizeFrames }),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -118,32 +126,50 @@ export class TerminalRenderer {
|
||||
#capture(atMs: number): void {
|
||||
const buffer = this.#terminal.buffer.active;
|
||||
const lines: string[] = [];
|
||||
this.#observedFrames += 1;
|
||||
let remainingBytes = this.options.maxBytes - this.#capturedBytes;
|
||||
let bytes = 0;
|
||||
for (let row = 0; row < this.#terminal.rows; row += 1) {
|
||||
const line = buffer.getLine(buffer.viewportY + row);
|
||||
const paddedLine = (
|
||||
line?.translateToString(false, 0, this.#terminal.cols) ?? ""
|
||||
).padEnd(this.#terminal.cols, " ");
|
||||
const trailingSpaceBytes = Buffer.byteLength(
|
||||
paddedLine.match(TRAILING_SPACES)?.[0] ?? "",
|
||||
);
|
||||
const normalized = this.options.normalize(
|
||||
paddedLine,
|
||||
Math.min(Number.MAX_SAFE_INTEGER, remainingBytes + trailingSpaceBytes),
|
||||
);
|
||||
if (normalized.state === "over_budget") {
|
||||
this.#unknownSizeFrames += 1;
|
||||
this.#truncated = true;
|
||||
return;
|
||||
}
|
||||
const normalizedLine = normalized.value.replace(TRAILING_SPACES, "");
|
||||
const lineBytes = Buffer.byteLength(normalizedLine);
|
||||
if (lineBytes > remainingBytes) {
|
||||
this.#unknownSizeFrames += 1;
|
||||
this.#truncated = true;
|
||||
return;
|
||||
}
|
||||
remainingBytes -= lineBytes;
|
||||
bytes += lineBytes;
|
||||
// Trailing blank cells carry no information beyond `columns`; dropping
|
||||
// them keeps sparse screens from costing a full row per blank line.
|
||||
lines.push(
|
||||
this.options
|
||||
.normalize(
|
||||
(
|
||||
line?.translateToString(false, 0, this.#terminal.cols) ?? ""
|
||||
).padEnd(this.#terminal.cols, " "),
|
||||
)
|
||||
.replace(TRAILING_SPACES, ""),
|
||||
);
|
||||
lines.push(normalizedLine);
|
||||
}
|
||||
const serializedState = this.options.normalize(
|
||||
const normalizedState = this.options.normalize(
|
||||
this.#serializeAddon.serialize(),
|
||||
remainingBytes,
|
||||
);
|
||||
const bytes =
|
||||
Buffer.byteLength(serializedState) +
|
||||
lines.reduce((total, line) => total + Buffer.byteLength(line), 0);
|
||||
this.#observedBytes += bytes;
|
||||
this.#observedFrames += 1;
|
||||
if (this.#capturedBytes + bytes > this.options.maxBytes) {
|
||||
if (normalizedState.state === "over_budget") {
|
||||
this.#unknownSizeFrames += 1;
|
||||
this.#truncated = true;
|
||||
return;
|
||||
}
|
||||
bytes += normalizedState.utf8Bytes;
|
||||
this.#observedBytes += bytes;
|
||||
this.#capturedBytes += bytes;
|
||||
const sequence = this.#frames.length;
|
||||
this.#frames.push({
|
||||
@@ -155,7 +181,7 @@ export class TerminalRenderer {
|
||||
cursor_y: buffer.cursorY,
|
||||
active_buffer: buffer.type,
|
||||
lines,
|
||||
serialized_state: serializedState,
|
||||
serialized_state: normalizedState.value,
|
||||
});
|
||||
this.options.recordEvent?.("rendered_frames", sequence);
|
||||
}
|
||||
|
||||
@@ -1,6 +1,25 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
|
||||
import { z } from "zod";
|
||||
|
||||
import { forgetPendingMcpRequest } from "./EvidenceMcpServer.js";
|
||||
import { InvestigationRecords } from "../application/investigation/InvestigationRecords.js";
|
||||
import {
|
||||
EvidenceMcpServer,
|
||||
forgetPendingMcpRequest,
|
||||
} from "./EvidenceMcpServer.js";
|
||||
import { ToolResultDelivery } from "./toolResult.js";
|
||||
|
||||
const firstText = (result: {
|
||||
readonly content: readonly {
|
||||
readonly type: string;
|
||||
readonly text?: string;
|
||||
}[];
|
||||
}): string | undefined => {
|
||||
for (const item of result.content) {
|
||||
if (item.type === "text") return item.text;
|
||||
}
|
||||
return undefined;
|
||||
};
|
||||
|
||||
describe("MCP request bookkeeping", () => {
|
||||
it("releases list, tool, and completion state when cancellation arrives", () => {
|
||||
@@ -22,3 +41,106 @@ describe("MCP request bookkeeping", () => {
|
||||
expect(pendingCompletions.has(7)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("SDK validation error delivery", () => {
|
||||
it("bounds oversized errors, retains their source, and preserves fitting and valid calls", async () => {
|
||||
const delivery = new ToolResultDelivery(4096);
|
||||
const records = new InvestigationRecords();
|
||||
const server = new EvidenceMcpServer(
|
||||
{ name: "sdk-validation-budget-test", version: "1" },
|
||||
{ capabilities: {} },
|
||||
(evidence) => records.recordEvidence(evidence),
|
||||
delivery,
|
||||
);
|
||||
server.registerTool(
|
||||
"procedure_pseudo_code",
|
||||
{ inputSchema: z.strictObject({ required: z.string() }) },
|
||||
async () => ({ content: [{ type: "text", text: "valid call" }] }),
|
||||
);
|
||||
const client = new Client({
|
||||
name: "sdk-validation-budget-client",
|
||||
version: "1",
|
||||
});
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
const oversizedKey = `unexpected_${"x".repeat(6000)}`;
|
||||
try {
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
|
||||
const oversized = await client.callTool({
|
||||
name: "procedure_pseudo_code",
|
||||
arguments: { required: "present", [oversizedKey]: true },
|
||||
});
|
||||
expect(oversized.isError).toBe(true);
|
||||
expect(oversized.structuredContent).toBeUndefined();
|
||||
expect(oversized._meta).toEqual({
|
||||
provenance: { untrusted_content: true, source: "analysis_target" },
|
||||
});
|
||||
expect(Buffer.byteLength(JSON.stringify(oversized))).toBeLessThan(4096);
|
||||
const oversizedError = z
|
||||
.object({
|
||||
error: z.object({
|
||||
code: z.string(),
|
||||
details: z.object({
|
||||
reported_limits: z.record(z.string(), z.unknown()),
|
||||
}),
|
||||
}),
|
||||
})
|
||||
.parse(JSON.parse(firstText(oversized) ?? "null"));
|
||||
expect(oversizedError.error.code).toBe("resource_constraint");
|
||||
const reportedLimits = oversizedError.error.details.reported_limits;
|
||||
expect(reportedLimits).toMatchObject({
|
||||
evidence_reference: {
|
||||
kind: "retained-evidence",
|
||||
evidence_id: expect.any(String),
|
||||
},
|
||||
diagnostic_truncated: true,
|
||||
});
|
||||
expect(reportedLimits).not.toHaveProperty("retention");
|
||||
const bundle = records.exportEvidenceBundle();
|
||||
expect(bundle.records).toHaveLength(1);
|
||||
expect(bundle.records[0]?.operation).toBe("mcp_tool_error");
|
||||
expect(bundle.records[0]?.raw_result).toMatchObject({
|
||||
isError: true,
|
||||
content: [
|
||||
{ type: "text", text: expect.stringContaining(oversizedKey) },
|
||||
],
|
||||
});
|
||||
const evidenceReference = z
|
||||
.object({ evidence_id: z.string() })
|
||||
.parse(reportedLimits.evidence_reference);
|
||||
expect(bundle.records[0]?.evidence_id).toBe(
|
||||
evidenceReference.evidence_id,
|
||||
);
|
||||
|
||||
const fitting = await client.callTool({
|
||||
name: "procedure_pseudo_code",
|
||||
arguments: { unexpected: true },
|
||||
});
|
||||
expect(fitting.isError).toBe(true);
|
||||
expect(firstText(fitting)).toContain("Input validation error");
|
||||
|
||||
const quotedKey = '"\\'.repeat(350);
|
||||
const quoted = await client.callTool({
|
||||
name: "procedure_pseudo_code",
|
||||
arguments: { required: "present", [quotedKey]: true },
|
||||
});
|
||||
expect(quoted.isError).toBe(true);
|
||||
expect(firstText(quoted)).toContain("Input validation error");
|
||||
expect(Buffer.byteLength(JSON.stringify(quoted))).toBeLessThan(4096);
|
||||
expect(records.exportEvidenceBundle().records).toHaveLength(1);
|
||||
|
||||
const valid = await client.callTool({
|
||||
name: "procedure_pseudo_code",
|
||||
arguments: { required: "present" },
|
||||
});
|
||||
expect(valid.content).toContainEqual({
|
||||
type: "text",
|
||||
text: "valid call",
|
||||
});
|
||||
} finally {
|
||||
await Promise.all([client.close(), server.close()]);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -37,6 +37,7 @@ import { toolReturnsTargetContent } from "../contracts/toolEffects.js";
|
||||
import {
|
||||
carriesTargetContentMarker,
|
||||
encodeToolResult,
|
||||
measureMcpResultWithinBudget,
|
||||
TARGET_CONTENT_RESULT_META,
|
||||
} from "./toolResultEncoding.js";
|
||||
|
||||
@@ -346,38 +347,65 @@ class ErrorEvidenceTransport implements Transport {
|
||||
if (
|
||||
!isJSONRPCResultResponse(message) ||
|
||||
!isCallToolResult(message.result) ||
|
||||
message.result.isError !== true ||
|
||||
message.result.structuredContent === undefined
|
||||
message.result.isError !== true
|
||||
)
|
||||
return message;
|
||||
const structured = jsonObjectSchema.parse(message.result.structuredContent);
|
||||
const budget = this.delivery.resultBudgetBytes;
|
||||
// Count the provenance marker exactly as the projected error did, and keep
|
||||
// it on the replacement: the retained original may hold target content.
|
||||
const targetContent = carriesTargetContentMarker(message.result);
|
||||
// The delivered error omits structuredContent, so its text and error flag count.
|
||||
const encoded = encodeToolResult(
|
||||
structured,
|
||||
budget,
|
||||
"error-text",
|
||||
targetContent,
|
||||
);
|
||||
const targetContent =
|
||||
carriesTargetContentMarker(message.result) ||
|
||||
(tool !== undefined && toolReturnsTargetContent(tool));
|
||||
// SDK validation failures have no structured carrier. Size their existing
|
||||
// text-only result without serializing the complete JSON-RPC message.
|
||||
const isStructured = message.result.structuredContent !== undefined;
|
||||
const structured = isStructured
|
||||
? jsonObjectSchema.parse(message.result.structuredContent)
|
||||
: undefined;
|
||||
const sdkResultWithMarker =
|
||||
!isStructured &&
|
||||
targetContent &&
|
||||
!carriesTargetContentMarker(message.result)
|
||||
? {
|
||||
...message.result,
|
||||
_meta: {
|
||||
...message.result._meta,
|
||||
...TARGET_CONTENT_RESULT_META,
|
||||
},
|
||||
}
|
||||
: message.result;
|
||||
const encoded =
|
||||
structured !== undefined
|
||||
? encodeToolResult(structured, budget, "error-text", targetContent)
|
||||
: measureMcpResultWithinBudget(sdkResultWithMarker, budget);
|
||||
if (encoded.ok) return message;
|
||||
const operation = tool ?? "mcp_tool_error";
|
||||
const originalError = analysisErrorProjectionSchema.safeParse(
|
||||
structured["error"],
|
||||
);
|
||||
// Retain the already projected diagnostic. Transport authentication and raw
|
||||
// request arguments are deliberately excluded: the application owns their
|
||||
// explicit sensitivity policy, and its error projection is authoritative.
|
||||
const originalResult = jsonObjectSchema.parse(message.result);
|
||||
const sdkDiagnostic = !isStructured
|
||||
? message.result.content.find((item) => item.type === "text")?.text
|
||||
: undefined;
|
||||
const retainedResult = structured ?? originalResult;
|
||||
const originalError = isStructured
|
||||
? analysisErrorProjectionSchema.safeParse(structured?.["error"])
|
||||
: undefined;
|
||||
// Retain the complete original response, including SDK validation text.
|
||||
// The SDK error contains its rendered issue, not a separate argument bag;
|
||||
// transport authentication and caller arguments are never added here.
|
||||
const evidence = createEvidence(
|
||||
undefined,
|
||||
{ id: "rea-mcp", name: "REA MCP delivery", version: "1" },
|
||||
{
|
||||
operation: "mcp_tool_error",
|
||||
parameters: { tool_name: operation },
|
||||
result: structured,
|
||||
rawResult: jsonObjectSchema.parse(message.result),
|
||||
result:
|
||||
sdkDiagnostic === undefined
|
||||
? retainedResult
|
||||
: {
|
||||
boundary: "sdk-input-validation",
|
||||
diagnostic_preview: sdkDiagnostic.slice(0, 256),
|
||||
diagnostic_truncated: sdkDiagnostic.length > 256,
|
||||
},
|
||||
rawResult: originalResult,
|
||||
},
|
||||
);
|
||||
const recorded = this.recordEvidence?.(evidence);
|
||||
@@ -388,12 +416,18 @@ class ErrorEvidenceTransport implements Transport {
|
||||
"The tool failed, but its complete error response exceeds the MCP response budget.",
|
||||
{
|
||||
boundary: "mcp-response",
|
||||
...(originalError.success
|
||||
...(originalError?.success
|
||||
? { original_error_code: originalError.data.code }
|
||||
: {}),
|
||||
result_budget_bytes: budget,
|
||||
response_bytes_at_least: encoded.bytesAtLeast,
|
||||
constraint: encoded.constraint,
|
||||
...(sdkDiagnostic === undefined
|
||||
? {}
|
||||
: {
|
||||
diagnostic_preview: sdkDiagnostic.slice(0, 256),
|
||||
diagnostic_truncated: sdkDiagnostic.length > 256,
|
||||
}),
|
||||
...(retained
|
||||
? {
|
||||
evidence_reference: {
|
||||
|
||||
@@ -577,6 +577,10 @@ export const createServer = (
|
||||
: { availabilityPolicy: availability.policy }),
|
||||
androidAnalysisAvailability: (signal) =>
|
||||
android.inspectAvailability(signal),
|
||||
adbAnalysisAvailability: (signal) =>
|
||||
adbDevice.inspectAvailability(signal),
|
||||
apktoolAnalysisAvailability: (signal) =>
|
||||
apktool.inspectAvailability(signal),
|
||||
startedAt,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -30,6 +30,12 @@ export interface SessionStatusToolOptions {
|
||||
readonly androidAnalysisAvailability: (
|
||||
signal: AbortSignal,
|
||||
) => Promise<ProviderAvailability>;
|
||||
readonly adbAnalysisAvailability?: (
|
||||
signal: AbortSignal,
|
||||
) => Promise<ProviderAvailability>;
|
||||
readonly apktoolAnalysisAvailability?: (
|
||||
signal: AbortSignal,
|
||||
) => Promise<ProviderAvailability>;
|
||||
}
|
||||
|
||||
/** Register the read-only provider and target status operation. */
|
||||
@@ -43,6 +49,8 @@ export const registerSessionStatusTool = (
|
||||
startedAt,
|
||||
availabilityPolicy,
|
||||
androidAnalysisAvailability,
|
||||
adbAnalysisAvailability,
|
||||
apktoolAnalysisAvailability,
|
||||
} = options;
|
||||
server.registerTool(
|
||||
contract.name,
|
||||
@@ -53,10 +61,17 @@ export const registerSessionStatusTool = (
|
||||
server.server,
|
||||
);
|
||||
let androidAvailability: ProviderAvailability;
|
||||
let adbAvailability: ProviderAvailability | undefined;
|
||||
let apktoolAvailability: ProviderAvailability | undefined;
|
||||
try {
|
||||
androidAvailability = await androidAnalysisAvailability(
|
||||
context.mcpReq.signal,
|
||||
);
|
||||
const [android, adb, apktool] = await Promise.all([
|
||||
androidAnalysisAvailability(context.mcpReq.signal),
|
||||
adbAnalysisAvailability?.(context.mcpReq.signal) ?? undefined,
|
||||
apktoolAnalysisAvailability?.(context.mcpReq.signal) ?? undefined,
|
||||
] as const);
|
||||
androidAvailability = android;
|
||||
adbAvailability = adb;
|
||||
apktoolAvailability = apktool;
|
||||
} catch (cause) {
|
||||
if (!context.mcpReq.signal.aborted) throw cause;
|
||||
return server.delivery.toCallToolResult(
|
||||
@@ -71,6 +86,12 @@ export const registerSessionStatusTool = (
|
||||
{
|
||||
...availabilityPolicy(),
|
||||
androidAnalysisAvailability: androidAvailability,
|
||||
...(adbAvailability === undefined
|
||||
? {}
|
||||
: { adbAnalysisAvailability: adbAvailability }),
|
||||
...(apktoolAvailability === undefined
|
||||
? {}
|
||||
: { apktoolAnalysisAvailability: apktoolAvailability }),
|
||||
},
|
||||
clientFeatures,
|
||||
);
|
||||
|
||||
@@ -155,6 +155,12 @@ export interface LifecycleToolRegistration {
|
||||
readonly androidAnalysisAvailability: (
|
||||
signal: AbortSignal,
|
||||
) => Promise<ProviderAvailability>;
|
||||
readonly adbAnalysisAvailability?: (
|
||||
signal: AbortSignal,
|
||||
) => Promise<ProviderAvailability>;
|
||||
readonly apktoolAnalysisAvailability?: (
|
||||
signal: AbortSignal,
|
||||
) => Promise<ProviderAvailability>;
|
||||
readonly closeProcessResources: () => Promise<Result<null, AnalysisError>>;
|
||||
}
|
||||
|
||||
@@ -169,6 +175,8 @@ const registerLifecycleTools = (
|
||||
startedAt,
|
||||
availabilityPolicy,
|
||||
androidAnalysisAvailability,
|
||||
adbAnalysisAvailability,
|
||||
apktoolAnalysisAvailability,
|
||||
} = registration;
|
||||
registerOpenLifecycleTool(registration, artifactResources);
|
||||
registerCloseLifecycleTool(registration);
|
||||
@@ -179,6 +187,12 @@ const registerLifecycleTools = (
|
||||
startedAt,
|
||||
availabilityPolicy,
|
||||
androidAnalysisAvailability,
|
||||
...(adbAnalysisAvailability === undefined
|
||||
? {}
|
||||
: { adbAnalysisAvailability }),
|
||||
...(apktoolAnalysisAvailability === undefined
|
||||
? {}
|
||||
: { apktoolAnalysisAvailability }),
|
||||
});
|
||||
};
|
||||
|
||||
@@ -238,6 +252,12 @@ export interface SessionToolOptions {
|
||||
readonly androidAnalysisAvailability?: (
|
||||
signal: AbortSignal,
|
||||
) => Promise<ProviderAvailability>;
|
||||
readonly adbAnalysisAvailability?: (
|
||||
signal: AbortSignal,
|
||||
) => Promise<ProviderAvailability>;
|
||||
readonly apktoolAnalysisAvailability?: (
|
||||
signal: AbortSignal,
|
||||
) => Promise<ProviderAvailability>;
|
||||
}
|
||||
|
||||
const registerContextTools = (
|
||||
@@ -317,6 +337,12 @@ export const registerSessionTools = (
|
||||
),
|
||||
androidAnalysisAvailability:
|
||||
options.androidAnalysisAvailability ?? missingAndroidAvailability,
|
||||
...(options.adbAnalysisAvailability === undefined
|
||||
? {}
|
||||
: { adbAnalysisAvailability: options.adbAnalysisAvailability }),
|
||||
...(options.apktoolAnalysisAvailability === undefined
|
||||
? {}
|
||||
: { apktoolAnalysisAvailability: options.apktoolAnalysisAvailability }),
|
||||
closeProcessResources: async () => {
|
||||
try {
|
||||
await processResources.retryCleanup();
|
||||
|
||||
@@ -63,6 +63,26 @@ export type ToolResultEncoding =
|
||||
readonly constraint: "receive-buffer" | "string-length";
|
||||
};
|
||||
|
||||
/** Measure an already-formed MCP result object without allocating its JSON text. */
|
||||
export const measureMcpResultWithinBudget = (
|
||||
value: unknown,
|
||||
budgetBytes: number,
|
||||
):
|
||||
| { readonly ok: true; readonly bytes: number }
|
||||
| {
|
||||
readonly ok: false;
|
||||
readonly bytesAtLeast: number;
|
||||
readonly constraint: "receive-buffer";
|
||||
} => {
|
||||
let bytes = 0;
|
||||
for (const part of bufferedJsonParts(jsonParts(value))) {
|
||||
bytes += Buffer.byteLength(part);
|
||||
if (bytes > budgetBytes)
|
||||
return { ok: false, bytesAtLeast: bytes, constraint: "receive-buffer" };
|
||||
}
|
||||
return { ok: true, bytes };
|
||||
};
|
||||
|
||||
/** Account for the delivered JSON representations before allocating the complete text. */
|
||||
export const encodeToolResult = (
|
||||
value: JsonValue,
|
||||
|
||||
@@ -308,6 +308,47 @@ cliTest(
|
||||
);
|
||||
|
||||
for (const surface of ["CLI", "stdio MCP"] as const) {
|
||||
cliTest(
|
||||
`retains explicit nested contextBridge members after an unknown spread through ${surface}`,
|
||||
async ({ cli }) => {
|
||||
const analyzeBridge = async (api: string) => {
|
||||
const root = await createTestTempDirectory(
|
||||
"rea-electron-spread-bridge-members-",
|
||||
);
|
||||
await writeFixtureFiles(root, {
|
||||
"preload.js": `const { contextBridge } = require("electron");
|
||||
const extra = getExtra();
|
||||
contextBridge.exposeInMainWorld("api", ${api});`,
|
||||
});
|
||||
return analyzeJavaScriptDirectory(surface, cli, root);
|
||||
};
|
||||
const withSpread = await analyzeBridge("{ ...extra, a: { b: 1 }, c: 2 }");
|
||||
const control = await analyzeBridge("{ a: { b: 1 }, c: 2 }");
|
||||
|
||||
const bridgeObservation = (application: typeof withSpread) =>
|
||||
application.graph.nodes
|
||||
.filter((node) => node.kind === "context-bridge-api")
|
||||
.flatMap((node) => node.observations)
|
||||
.find((observation) => observation.properties.api_key === "api");
|
||||
expect(bridgeObservation(withSpread)).toMatchObject({
|
||||
properties: {
|
||||
api_status: "dynamic",
|
||||
members: ["a", "a.b", "c"],
|
||||
unknown_members: 1,
|
||||
},
|
||||
});
|
||||
expect(bridgeObservation(control)).toMatchObject({
|
||||
properties: {
|
||||
api_status: "object-literal",
|
||||
members: ["a", "a.b", "c"],
|
||||
unknown_members: 0,
|
||||
},
|
||||
});
|
||||
expect(withSpread.summary.exposed_api_members).toBe(3);
|
||||
expect(control.summary.exposed_api_members).toBe(3);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
`preserves handler ambiguity and short-circuit factory paths through ${surface}`,
|
||||
async ({ cli }) => {
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
import { rm } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { expect } from "vitest";
|
||||
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { createStrippedAsarAddon } from "../../fixtures/strippedAsarAddon.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
import {
|
||||
findExportNode,
|
||||
@@ -11,6 +15,75 @@ import {
|
||||
writeFixtureFiles,
|
||||
} from "../../support/javascriptApplicationFixture.js";
|
||||
|
||||
cliTest(
|
||||
"resolves ESM specifiers containing whitespace using URL text semantics",
|
||||
async ({ cli }) => {
|
||||
const root = await createTestTempDirectory("rea-esm-whitespace-path-");
|
||||
const rawSpecifier = "./feature\tname.js";
|
||||
const encodedSpecifier = "./feature%09name.js";
|
||||
await writeFixtureFiles(root, {
|
||||
"main.mjs": `import ${JSON.stringify(rawSpecifier)};\nimport ${JSON.stringify(encodedSpecifier)};\n`,
|
||||
"featurename.js": "export const plain = 1;\n",
|
||||
"feature\tname.js": "export const value = 1;\n",
|
||||
});
|
||||
const output = await cli.run({
|
||||
arguments: ["analyze-javascript-application", root, "--json"],
|
||||
});
|
||||
expect(output.exitCode, output.stderr).toBe(0);
|
||||
const result = javascriptApplicationAnalysisResultSchema.parse(
|
||||
parseEvidence(output.json).normalized_result,
|
||||
);
|
||||
expect(
|
||||
findGraphEdge(
|
||||
result.graph,
|
||||
findSourceModule(result.graph, "main.mjs"),
|
||||
findSourceModule(result.graph, "featurename.js"),
|
||||
"imports",
|
||||
)?.properties,
|
||||
).toMatchObject({
|
||||
specifier: rawSpecifier,
|
||||
resolved_path: "featurename.js",
|
||||
});
|
||||
expect(
|
||||
findGraphEdge(
|
||||
result.graph,
|
||||
findSourceModule(result.graph, "main.mjs"),
|
||||
findSourceModule(result.graph, "feature\tname.js"),
|
||||
"imports",
|
||||
)?.properties,
|
||||
).toMatchObject({
|
||||
specifier: encodedSpecifier,
|
||||
resolved_path: "feature\tname.js",
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"retains an inventoried native addon when its ASAR companion is missing",
|
||||
async ({ cli }) => {
|
||||
const { archive, addon } = await createStrippedAsarAddon();
|
||||
await rm(join(`${archive}.unpacked`, addon));
|
||||
const output = await cli.run({
|
||||
arguments: ["analyze-javascript-application", archive, "--json"],
|
||||
});
|
||||
expect(output.exitCode, output.stderr).toBe(0);
|
||||
const result = javascriptApplicationAnalysisResultSchema.parse(
|
||||
parseEvidence(output.json).normalized_result,
|
||||
);
|
||||
expect(result.graph.coverage.status).toBe("partial");
|
||||
expect(
|
||||
result.graph.nodes.some(
|
||||
(node) =>
|
||||
node.kind === "native-addon" &&
|
||||
node.observations.some(
|
||||
({ label, properties }) =>
|
||||
label === addon && properties.availability === "unavailable",
|
||||
),
|
||||
),
|
||||
).toBe(true);
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"resolves emitted module imports without substituting Electron runtime paths",
|
||||
async ({ cli }) => {
|
||||
|
||||
@@ -1009,6 +1009,10 @@ it.each([
|
||||
".github/workflows/release.yml",
|
||||
{ workflows: true, code: true, package: false, website: false },
|
||||
],
|
||||
[
|
||||
".github/workflows/pr-readiness.yml",
|
||||
{ workflows: true, code: true, package: false, website: false },
|
||||
],
|
||||
[
|
||||
".github/workflows/real-ghidra-windows.yml",
|
||||
{ workflows: true, code: true, package: false, website: false },
|
||||
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
} from "../../../src/application/SetupClientConfiguration.js";
|
||||
import { systemUninstallHost } from "../../../src/application/Uninstall.js";
|
||||
import { supportedClients } from "../../../src/application/SupportedClients.js";
|
||||
import { CURRENT_ENTRYPOINT_PATH } from "../../../src/process/CurrentEntrypointPath.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
|
||||
describe.each([
|
||||
@@ -137,8 +138,10 @@ it("restores original BOM-prefixed JSONC bytes after update and removal", async
|
||||
await mkdir(dirname(client.configPath), { recursive: true });
|
||||
await writeFile(client.configPath, original);
|
||||
|
||||
const firstCommand = [join(home, "bin", "rea"), "mcp"];
|
||||
const updatedCommand = [join(home, "updated", "bin", "rea"), "mcp"];
|
||||
if (CURRENT_ENTRYPOINT_PATH === undefined)
|
||||
throw new Error("test runner entrypoint is unavailable");
|
||||
const firstCommand = ["rea", "mcp"];
|
||||
const updatedCommand = [process.execPath, CURRENT_ENTRYPOINT_PATH, "mcp"];
|
||||
expect(
|
||||
await configureClientConfiguration(client, {}, firstCommand),
|
||||
).toMatchObject({ status: "configured" });
|
||||
|
||||
@@ -1335,7 +1335,7 @@ describe("Codex configuration comments", () => {
|
||||
await configureClientConfiguration(
|
||||
{ name: "codex", configPath, format: "toml" },
|
||||
{},
|
||||
["/isolated prefix/bin/rea", "mcp"],
|
||||
[PRODUCT_IDENTITY.cliBinary, "mcp"],
|
||||
),
|
||||
).toMatchObject({ status: "configured" });
|
||||
const configured = await readFile(configPath, "utf8");
|
||||
@@ -1505,7 +1505,7 @@ describe("client configuration filesystem removal", () => {
|
||||
cursor,
|
||||
JSON.stringify({
|
||||
mcpServers: {
|
||||
rea: { command: "/isolated prefix/bin/rea", args: ["mcp"] },
|
||||
rea: { command: PRODUCT_IDENTITY.cliBinary, args: ["mcp"] },
|
||||
other: { command: "other" },
|
||||
},
|
||||
}),
|
||||
|
||||
+63
-1
@@ -502,10 +502,72 @@ it("keeps ASAR analysis usable when unpacked native companion bytes are absent",
|
||||
"ASAR unpacked companion bytes were unavailable; no content hash or child artifact was produced.",
|
||||
]),
|
||||
});
|
||||
if (missingNative === undefined)
|
||||
throw new Error("Inventory omitted the unavailable native companion");
|
||||
|
||||
await reconstructAsarArchive(archive, [
|
||||
const result = await reconstructAsarArchive(archive, [
|
||||
expect.objectContaining({ kind: "javascript-asset" }),
|
||||
]);
|
||||
const missingEntry = result.graph.nodes.find(
|
||||
({ kind, observations }) =>
|
||||
kind === "asar-entry" &&
|
||||
observations.some(({ label }) => label === missingNative.logical_path),
|
||||
);
|
||||
expect(missingEntry).toMatchObject({
|
||||
identity: {
|
||||
strategy: "canonical-path",
|
||||
stability: "artifact-version",
|
||||
artifact_sha256: snapshot.manifest.root_sha256,
|
||||
path: missingNative.logical_path,
|
||||
},
|
||||
observations: [
|
||||
expect.objectContaining({
|
||||
label: missingNative.logical_path,
|
||||
properties: expect.objectContaining({
|
||||
entry_sha256: null,
|
||||
bytes: missingNative.declared_size,
|
||||
inventory_artifact_id: null,
|
||||
inventory_occurrence_id: missingNative.occurrence_id,
|
||||
hash_status: "unavailable",
|
||||
artifact_kind: "native-addon",
|
||||
kind: "native-addon",
|
||||
content_available: false,
|
||||
}),
|
||||
evidence: expect.objectContaining({
|
||||
limitations: expect.arrayContaining([
|
||||
"ASAR unpacked companion bytes were unavailable; no content hash or child artifact was produced.",
|
||||
"The inventoried native-addon entry has unavailable bytes; no content artifact identity can be established.",
|
||||
]),
|
||||
}),
|
||||
}),
|
||||
],
|
||||
});
|
||||
const unavailableAddon = result.graph.nodes.find(
|
||||
({ kind, observations }) =>
|
||||
kind === "native-addon" &&
|
||||
observations.some(({ label }) => label === missingNative.logical_path),
|
||||
);
|
||||
expect(unavailableAddon).toMatchObject({
|
||||
identity: {
|
||||
strategy: "canonical-path",
|
||||
stability: "artifact-version",
|
||||
artifact_sha256: snapshot.manifest.root_sha256,
|
||||
path: missingNative.logical_path,
|
||||
},
|
||||
observations: [
|
||||
expect.objectContaining({
|
||||
label: missingNative.logical_path,
|
||||
properties: expect.objectContaining({
|
||||
availability: "unavailable",
|
||||
hash_status: "unavailable",
|
||||
artifact_kind: "native-addon",
|
||||
bytes: missingNative.declared_size,
|
||||
inventory_artifact_id: null,
|
||||
inventory_occurrence_id: missingNative.occurrence_id,
|
||||
}),
|
||||
}),
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
it("recurses into filesystem-backed ASAR containers without losing container-relative paths", async () => {
|
||||
|
||||
@@ -0,0 +1,382 @@
|
||||
import { execFile } from "node:child_process";
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { promisify } from "node:util";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { parse } from "yaml";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
evaluateReadiness,
|
||||
runReadinessCheck,
|
||||
type ReadinessGitHub,
|
||||
type ReadinessPullRequest,
|
||||
} from "../../../scripts/ci/pr-readiness.mjs";
|
||||
|
||||
const template = await readFile(
|
||||
new URL("../../../.github/pull_request_template.md", import.meta.url),
|
||||
"utf8",
|
||||
);
|
||||
const readyBody = template
|
||||
.slice(
|
||||
template.indexOf("## Author readiness"),
|
||||
template.indexOf("## Before merge"),
|
||||
)
|
||||
.replace("Human owner: <!-- @GitHub-handle -->", "Human owner: @contributor")
|
||||
.replaceAll("- [ ]", "- [x]");
|
||||
const makePull = (
|
||||
changes: Partial<ReadinessPullRequest> = {},
|
||||
): ReadinessPullRequest => ({
|
||||
number: 42,
|
||||
state: "open",
|
||||
draft: false,
|
||||
body: readyBody,
|
||||
head: { sha: "a".repeat(40) },
|
||||
base: { ref: "main" },
|
||||
...changes,
|
||||
});
|
||||
|
||||
function fixture(changes: Partial<ReadinessPullRequest> = {}) {
|
||||
const pull = makePull(changes);
|
||||
const get = vi.fn(async () => ({ data: pull }));
|
||||
const createCommitStatus = vi.fn(
|
||||
async (
|
||||
_status: Parameters<
|
||||
ReadinessGitHub["rest"]["repos"]["createCommitStatus"]
|
||||
>[0],
|
||||
) => ({}),
|
||||
);
|
||||
const paginate = vi.fn(async () => [pull]);
|
||||
const github = {
|
||||
rest: {
|
||||
pulls: { get },
|
||||
repos: {
|
||||
createCommitStatus,
|
||||
listPullRequestsAssociatedWithCommit: vi.fn(),
|
||||
},
|
||||
},
|
||||
paginate,
|
||||
};
|
||||
const context = {
|
||||
repo: { owner: "morluto", repo: "rea" },
|
||||
payload: { pull_request: { number: 42 } },
|
||||
serverUrl: "https://github.com",
|
||||
runId: 123,
|
||||
};
|
||||
const core = { info: vi.fn(), setFailed: vi.fn() };
|
||||
return { github, context, core, get, createCommitStatus, paginate, pull };
|
||||
}
|
||||
|
||||
describe("initial author declaration", () => {
|
||||
it("accepts the filled current template and uppercase checkmarks", () => {
|
||||
expect(evaluateReadiness(makePull(), template).state).toBe("success");
|
||||
expect(
|
||||
evaluateReadiness(
|
||||
makePull({ body: readyBody.replaceAll("[x]", "[X]") }),
|
||||
template,
|
||||
).state,
|
||||
).toBe("success");
|
||||
expect(
|
||||
evaluateReadiness(
|
||||
makePull({ body: readyBody.replaceAll("\n", "\r\n") }),
|
||||
template,
|
||||
).state,
|
||||
).toBe("success");
|
||||
});
|
||||
|
||||
it("allows drafts without a body or declarations", () => {
|
||||
expect(
|
||||
evaluateReadiness(makePull({ draft: true, body: null }), template).state,
|
||||
).toBe("pending");
|
||||
});
|
||||
|
||||
it.each([0, 1, 2])("requires declaration %i to be checked", (index) => {
|
||||
let position = 0;
|
||||
const body = readyBody.replaceAll("- [x]", () =>
|
||||
position++ === index ? "- [ ]" : "- [x]",
|
||||
);
|
||||
expect(evaluateReadiness(makePull({ body }), template).state).toBe(
|
||||
"failure",
|
||||
);
|
||||
});
|
||||
|
||||
it.each([
|
||||
null,
|
||||
readyBody.replace("Human owner: @contributor", "Human owner:"),
|
||||
readyBody.replace("Human owner: @contributor", "Human owner: @one @two"),
|
||||
readyBody.replace(
|
||||
"Human owner: @contributor",
|
||||
"Human owner: @automation[bot]",
|
||||
),
|
||||
readyBody + "\nHuman owner: @other\n",
|
||||
readyBody.replace("## Author readiness", "## Other"),
|
||||
readyBody + readyBody,
|
||||
`<!--\n${readyBody}\n-->`,
|
||||
`~~~markdown\n${readyBody}\n~~~`,
|
||||
`\`\`\`markdown\n${readyBody}\n\`\`\``,
|
||||
readyBody
|
||||
.split("\n")
|
||||
.map((line) => `> ${line}`)
|
||||
.join("\n"),
|
||||
readyBody.replace(
|
||||
"- [x] I declare this PR ready for initial maintainer review.",
|
||||
"",
|
||||
),
|
||||
readyBody +
|
||||
"\n- [x] I declare this PR ready for initial maintainer review.\n",
|
||||
])(
|
||||
"rejects missing, hidden, ambiguous, or incomplete declarations (%#)",
|
||||
(body) => {
|
||||
expect(evaluateReadiness(makePull({ body }), template).state).toBe(
|
||||
"failure",
|
||||
);
|
||||
},
|
||||
);
|
||||
|
||||
it("ignores checked examples in comments and fenced blocks before the actual declaration", () => {
|
||||
for (const example of [
|
||||
`<!--\n${readyBody}\n-->\n`,
|
||||
`~~~markdown\n${readyBody}\n~~~\n`,
|
||||
`\`\`\`markdown\n${readyBody}\n\`\`\`\n`,
|
||||
])
|
||||
expect(
|
||||
evaluateReadiness(makePull({ body: example + readyBody }), template)
|
||||
.state,
|
||||
).toBe("success");
|
||||
});
|
||||
|
||||
it("does not treat an HTML comment inside a code example as hiding the real declaration", () => {
|
||||
const body =
|
||||
"~~~html\n<!-- an intentionally unfinished HTML example\n~~~\n" +
|
||||
readyBody;
|
||||
expect(evaluateReadiness(makePull({ body }), template).state).toBe(
|
||||
"success",
|
||||
);
|
||||
});
|
||||
|
||||
it("fails closed when the trusted template loses a declaration", () => {
|
||||
expect(() =>
|
||||
evaluateReadiness(
|
||||
makePull(),
|
||||
template.replace(
|
||||
"- [ ] I declare this PR ready for initial maintainer review.",
|
||||
"",
|
||||
),
|
||||
),
|
||||
).toThrow("three readiness declarations");
|
||||
});
|
||||
});
|
||||
|
||||
describe("GitHub status boundary", () => {
|
||||
it("publishes failure on the actual PR head when declarations are unchecked", async () => {
|
||||
const input = fixture({ body: readyBody.replaceAll("[x]", "[ ]") });
|
||||
expect((await runReadinessCheck(input))?.state).toBe("failure");
|
||||
expect(
|
||||
input.createCommitStatus.mock.calls.map(([status]) => status),
|
||||
).toEqual([
|
||||
expect.objectContaining({
|
||||
sha: input.pull.head.sha,
|
||||
context: "PR readiness",
|
||||
state: "pending",
|
||||
}),
|
||||
expect.objectContaining({
|
||||
sha: input.pull.head.sha,
|
||||
context: "PR readiness",
|
||||
state: "failure",
|
||||
}),
|
||||
]);
|
||||
expect(input.core.setFailed).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it("keeps drafts pending without failing the workflow", async () => {
|
||||
const input = fixture({ draft: true, body: null });
|
||||
expect((await runReadinessCheck(input))?.state).toBe("pending");
|
||||
expect(input.core.setFailed).not.toHaveBeenCalled();
|
||||
expect(input.paginate).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("keeps the initial declaration valid after later pushes", async () => {
|
||||
for (const sha of ["a".repeat(40), "b".repeat(40)]) {
|
||||
const input = fixture({ head: { sha } });
|
||||
expect((await runReadinessCheck(input))?.state).toBe("success");
|
||||
expect(input.createCommitStatus).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({ sha, state: "success" }),
|
||||
);
|
||||
expect(input.github.rest.pulls).toEqual({ get: input.get });
|
||||
}
|
||||
});
|
||||
|
||||
it("does not publish stale success if the PR changes during the check", async () => {
|
||||
const input = fixture();
|
||||
input.get
|
||||
.mockResolvedValueOnce({ data: input.pull })
|
||||
.mockResolvedValueOnce({
|
||||
data: makePull({ head: { sha: "b".repeat(40) }, body: null }),
|
||||
});
|
||||
expect((await runReadinessCheck(input))?.state).toBe("pending");
|
||||
expect(input.createCommitStatus).toHaveBeenLastCalledWith(
|
||||
expect.objectContaining({ sha: "b".repeat(40), state: "pending" }),
|
||||
);
|
||||
});
|
||||
|
||||
it("prevents another open PR from lending its status to the same head", async () => {
|
||||
const input = fixture();
|
||||
input.paginate.mockResolvedValue([input.pull, makePull({ number: 43 })]);
|
||||
expect((await runReadinessCheck(input))?.state).toBe("failure");
|
||||
expect(input.core.setFailed).toHaveBeenCalledWith(
|
||||
expect.stringContaining("distinct head commits"),
|
||||
);
|
||||
});
|
||||
|
||||
it("leaves the status pending and surfaces API failure", async () => {
|
||||
const input = fixture();
|
||||
input.paginate.mockRejectedValue(new Error("GitHub unavailable"));
|
||||
await expect(runReadinessCheck(input)).rejects.toThrow(
|
||||
"GitHub unavailable",
|
||||
);
|
||||
expect(input.createCommitStatus).toHaveBeenCalledOnce();
|
||||
expect(input.createCommitStatus).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ state: "pending" }),
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps readiness unknown when GitHub has not associated the head with this PR", async () => {
|
||||
const input = fixture();
|
||||
input.paginate.mockResolvedValue([]);
|
||||
expect((await runReadinessCheck(input))?.state).toBe("pending");
|
||||
expect(input.core.setFailed).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each([{ state: "closed" as const }, { base: { ref: "other-branch" } }])(
|
||||
"ignores closed PRs and unprotected target branches (%#)",
|
||||
async (changes) => {
|
||||
const input = fixture(changes);
|
||||
await runReadinessCheck(input);
|
||||
expect(input.createCommitStatus).not.toHaveBeenCalled();
|
||||
},
|
||||
);
|
||||
|
||||
it("supports manually rechecking an existing PR", async () => {
|
||||
const input = fixture();
|
||||
await runReadinessCheck({
|
||||
...input,
|
||||
context: {
|
||||
...input.context,
|
||||
payload: { inputs: { pull_request_number: "42" } },
|
||||
},
|
||||
});
|
||||
expect(input.get).toHaveBeenCalledWith({
|
||||
owner: "morluto",
|
||||
repo: "rea",
|
||||
pull_number: 42,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
const readWorkflow = async () =>
|
||||
z
|
||||
.object({
|
||||
on: z.object({
|
||||
pull_request_target: z.object({
|
||||
branches: z.array(z.string()),
|
||||
types: z.array(z.string()),
|
||||
}),
|
||||
workflow_dispatch: z.unknown(),
|
||||
}),
|
||||
permissions: z.record(z.string(), z.string()),
|
||||
jobs: z.object({
|
||||
readiness: z.object({
|
||||
permissions: z.record(z.string(), z.string()),
|
||||
steps: z.array(
|
||||
z.object({
|
||||
uses: z.string(),
|
||||
with: z.object({
|
||||
ref: z.string().optional(),
|
||||
"persist-credentials": z.boolean().optional(),
|
||||
script: z.string().optional(),
|
||||
}),
|
||||
}),
|
||||
),
|
||||
}),
|
||||
}),
|
||||
})
|
||||
.parse(
|
||||
parse(
|
||||
await readFile(
|
||||
new URL(
|
||||
"../../../.github/workflows/pr-readiness.yml",
|
||||
import.meta.url,
|
||||
),
|
||||
"utf8",
|
||||
),
|
||||
),
|
||||
);
|
||||
|
||||
it("executes the production workflow step with trusted code and status-only write authority", async () => {
|
||||
const workflow = await readWorkflow();
|
||||
expect(workflow.on.pull_request_target.branches).toEqual([
|
||||
"main",
|
||||
"release/*",
|
||||
]);
|
||||
expect(workflow.on.pull_request_target.types).toEqual(
|
||||
expect.arrayContaining([
|
||||
"opened",
|
||||
"edited",
|
||||
"synchronize",
|
||||
"reopened",
|
||||
"ready_for_review",
|
||||
"converted_to_draft",
|
||||
]),
|
||||
);
|
||||
expect(workflow.permissions).toEqual({});
|
||||
expect(workflow.jobs.readiness.permissions).toEqual({
|
||||
contents: "read",
|
||||
"pull-requests": "read",
|
||||
statuses: "write",
|
||||
});
|
||||
const [checkout, check] = workflow.jobs.readiness.steps;
|
||||
expect(checkout?.with).toEqual({
|
||||
ref: "${{ github.event.repository.default_branch }}",
|
||||
"persist-credentials": false,
|
||||
});
|
||||
expect(check?.uses).toMatch(/^actions\/github-script@[a-f\d]{40}$/u);
|
||||
const input = fixture();
|
||||
const root = fileURLToPath(new URL("../../../", import.meta.url));
|
||||
const code = `
|
||||
import { createRequire } from "node:module";
|
||||
const require = createRequire(${JSON.stringify(new URL("../../../package.json", import.meta.url).href)});
|
||||
const pull = ${JSON.stringify(input.pull)};
|
||||
const statuses = [];
|
||||
const github = {
|
||||
rest: {
|
||||
pulls: { get: async () => ({ data: pull }) },
|
||||
repos: {
|
||||
createCommitStatus: async status => statuses.push(status),
|
||||
listPullRequestsAssociatedWithCommit: () => {},
|
||||
},
|
||||
},
|
||||
paginate: async () => [pull],
|
||||
};
|
||||
const context = ${JSON.stringify(input.context)};
|
||||
const core = {
|
||||
info: () => {},
|
||||
setFailed: message => { throw new Error(message); },
|
||||
};
|
||||
${z.string().parse(check?.with.script)}
|
||||
console.log(JSON.stringify(statuses));
|
||||
`;
|
||||
const { stdout } = await promisify(execFile)(
|
||||
process.execPath,
|
||||
["--input-type=module", "-e", code],
|
||||
{
|
||||
cwd: root,
|
||||
env: { ...process.env, GITHUB_WORKSPACE: root },
|
||||
encoding: "utf8",
|
||||
timeout: 10_000,
|
||||
maxBuffer: 1024 * 1024,
|
||||
},
|
||||
);
|
||||
const statuses = z
|
||||
.array(z.object({ state: z.string() }))
|
||||
.parse(JSON.parse(stdout));
|
||||
expect(statuses.at(-1)?.state).toBe("success");
|
||||
});
|
||||
@@ -12,6 +12,7 @@ import { createEvidence, parseEvidence } from "../../src/domain/evidence.js";
|
||||
import type { JsonValue } from "../../src/domain/jsonValue.js";
|
||||
import { analysisCliErrorEnvelopeSchema } from "../../src/contracts/errorSchemas.js";
|
||||
import { toolContract } from "../../src/contracts/toolContracts.js";
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../src/domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { javaScriptSemanticTraceResultSchema } from "../../src/domain/javascript/javascriptSemanticTraceSchemas.js";
|
||||
import { resolveJavaScriptSemanticEvidence } from "../../src/domain/javascript/javascriptSemanticGraph.js";
|
||||
|
||||
@@ -124,12 +125,88 @@ it("reports invalid nested semantic graph Evidence as caller input through CLI a
|
||||
}
|
||||
});
|
||||
|
||||
it("excludes local exports from writes-only traces through CLI and MCP", async () => {
|
||||
const root = await createTestTempDirectory("rea-semantic-export-trace-");
|
||||
await mkdir(join(root, "src"));
|
||||
await writeFile(
|
||||
join(root, "src", "main.js"),
|
||||
[
|
||||
"const answer = 42;",
|
||||
"export { answer, answer as result };",
|
||||
"let mutable = 0;",
|
||||
"mutable = 1;",
|
||||
"export { mutable };",
|
||||
].join("\n"),
|
||||
);
|
||||
|
||||
const cli = createCli({});
|
||||
const analyzed = await serveCli(cli, [
|
||||
"analyze-javascript-application",
|
||||
root,
|
||||
"--json",
|
||||
]);
|
||||
expect(analyzed.exitCode).toBe(0);
|
||||
const application = parseEvidence(JSON.parse(analyzed.stdout));
|
||||
const analysisResult = javascriptApplicationAnalysisResultSchema.parse(
|
||||
application.normalized_result,
|
||||
);
|
||||
const bindingId = (label: string): string => {
|
||||
const binding = analysisResult.semantic_graph.nodes.find(
|
||||
(node) => node.kind === "binding" && node.label === label,
|
||||
);
|
||||
if (binding === undefined)
|
||||
throw new Error(`CLI semantic graph must include the ${label} binding`);
|
||||
return binding.node_id;
|
||||
};
|
||||
|
||||
const harness = await createApplicationMcpHarness();
|
||||
try {
|
||||
const exportTrace = await assertSuccessfulTraceParity(
|
||||
cli,
|
||||
harness,
|
||||
application,
|
||||
{
|
||||
seed: { kind: "semantic-node", node_id: bindingId("answer") },
|
||||
direction: "backward-provenance",
|
||||
allowed_relations: ["writes"],
|
||||
},
|
||||
);
|
||||
expect(exportTrace.summary).toMatchObject({
|
||||
total_seed_matches: 1,
|
||||
traversed_relations: 0,
|
||||
});
|
||||
expect(exportTrace.relations).toEqual([]);
|
||||
|
||||
const assignmentTrace = await assertSuccessfulTraceParity(
|
||||
cli,
|
||||
harness,
|
||||
application,
|
||||
{
|
||||
seed: { kind: "semantic-node", node_id: bindingId("mutable") },
|
||||
direction: "backward-provenance",
|
||||
allowed_relations: ["writes"],
|
||||
},
|
||||
);
|
||||
expect(assignmentTrace.summary).toMatchObject({
|
||||
total_seed_matches: 1,
|
||||
traversed_relations: 1,
|
||||
});
|
||||
expect(assignmentTrace.relations).toHaveLength(1);
|
||||
} finally {
|
||||
await harness.close();
|
||||
}
|
||||
});
|
||||
|
||||
const assertSuccessfulTraceParity = async (
|
||||
cli: ReturnType<typeof createCli>,
|
||||
harness: Awaited<ReturnType<typeof createApplicationMcpHarness>>,
|
||||
application: ReturnType<typeof parseEvidence>,
|
||||
query: { readonly seed: unknown; readonly direction: string },
|
||||
): Promise<void> => {
|
||||
query: {
|
||||
readonly seed: unknown;
|
||||
readonly direction: string;
|
||||
readonly allowed_relations?: readonly string[];
|
||||
},
|
||||
): Promise<ReturnType<typeof javaScriptSemanticTraceResultSchema.parse>> => {
|
||||
const arguments_ = { application, query };
|
||||
const cliResponse = await serveCli(cli, [
|
||||
"trace-javascript-semantics",
|
||||
@@ -162,6 +239,7 @@ const assertSuccessfulTraceParity = async (
|
||||
).outputSchema.parse(mcpResponse.structuredContent);
|
||||
expect(mcpEvidence.normalized_result).toEqual(cliResult);
|
||||
expect(mcpEvidence).toEqual(cliEvidence);
|
||||
return cliResult;
|
||||
};
|
||||
|
||||
const assertTraceContextsResolve = (
|
||||
|
||||
@@ -2,8 +2,6 @@ import { afterEach, describe, expect, it } from "vitest";
|
||||
|
||||
import type { AnalysisOperationPort } from "../../../src/application/AnalysisProvider.js";
|
||||
import { EnhancedTools } from "../../../src/application/EnhancedTools.js";
|
||||
import { AnalysisOutputError } from "../../../src/domain/analysisErrorCore.js";
|
||||
import { err } from "../../../src/domain/result.js";
|
||||
|
||||
import {
|
||||
closeEnhancedToolResources,
|
||||
@@ -16,43 +14,15 @@ import { observed as ok } from "../../fixtures/analysisExecution.js";
|
||||
afterEach(closeEnhancedToolResources);
|
||||
|
||||
describe("enhanced MCP tools", () => {
|
||||
it("returns typed graph failures and stable unresolved-name results", async () => {
|
||||
it("returns stable unresolved-name results", async () => {
|
||||
const tools = new EnhancedTools({
|
||||
execute: (name) =>
|
||||
name === "procedure_address"
|
||||
? Promise.resolve(ok("0x1"))
|
||||
: name === "procedure_callees"
|
||||
? Promise.resolve(err(new AnalysisOutputError(name, "failed")))
|
||||
: Promise.resolve(ok(inventory({}))),
|
||||
execute: () => Promise.resolve(ok(inventory({}))),
|
||||
});
|
||||
|
||||
const graph = await tools.execute("get_call_graph", {
|
||||
address: "0x1",
|
||||
direction: "forward",
|
||||
});
|
||||
const unresolved = await tools.execute("find_xrefs_to_name", {
|
||||
name: "missing",
|
||||
});
|
||||
|
||||
expect(graph).toMatchObject({
|
||||
ok: true,
|
||||
value: {
|
||||
graph: {
|
||||
"0": [
|
||||
{
|
||||
address: "0x1",
|
||||
status: "error",
|
||||
error: {
|
||||
category: "execution_failure",
|
||||
message:
|
||||
"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.",
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
limitations: [],
|
||||
},
|
||||
});
|
||||
expect(unresolved).toEqual({
|
||||
ok: true,
|
||||
value: {
|
||||
|
||||
@@ -2,7 +2,6 @@ import { afterEach, describe, expect, it } from "vitest";
|
||||
|
||||
import type { AnalysisOperationPort } from "../../../src/application/AnalysisProvider.js";
|
||||
import { createAnalysisExecution } from "../../../src/application/AnalysisProvider.js";
|
||||
import { EnhancedTools } from "../../../src/application/EnhancedTools.js";
|
||||
import {
|
||||
AnalysisCapabilityUnavailableError,
|
||||
AnalysisCancelledError,
|
||||
@@ -11,7 +10,12 @@ import {
|
||||
import type { RelationshipCoverage } from "../../../src/domain/relationshipCoverage.js";
|
||||
import { err, ok as resultOk } from "../../../src/domain/result.js";
|
||||
|
||||
import { closeEnhancedToolResources, connect } from "./enhancedToolsHarness.js";
|
||||
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
|
||||
import {
|
||||
closeEnhancedToolResources,
|
||||
connect,
|
||||
jsonResult,
|
||||
} from "./enhancedToolsHarness.js";
|
||||
import { observed as ok } from "../../fixtures/analysisExecution.js";
|
||||
|
||||
afterEach(closeEnhancedToolResources);
|
||||
@@ -312,6 +316,98 @@ describe("trace_call_path MCP outcome semantics", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("enhanced MCP batch cancellation", () => {
|
||||
it.each([
|
||||
{
|
||||
stage: "selector resolution",
|
||||
cancelAt: "procedure_address",
|
||||
expectedCalls: ["procedure_address:0x1"],
|
||||
operation: "procedure_address",
|
||||
},
|
||||
{
|
||||
stage: "entry-label lookup",
|
||||
cancelAt: "address_name",
|
||||
expectedCalls: ["procedure_address:0x1", "address_name:0x1"],
|
||||
operation: "address_name",
|
||||
},
|
||||
{
|
||||
stage: "pseudocode read",
|
||||
cancelAt: "procedure_pseudo_code",
|
||||
expectedCalls: [
|
||||
"procedure_address:0x1",
|
||||
"address_name:0x1",
|
||||
"procedure_pseudo_code:0x1",
|
||||
],
|
||||
operation: "procedure_pseudo_code",
|
||||
},
|
||||
] as const)(
|
||||
"returns typed cancellation from $stage without dispatching a later stage",
|
||||
async ({ cancelAt, expectedCalls, operation }) => {
|
||||
const calls: string[] = [];
|
||||
const signals: boolean[] = [];
|
||||
const client = await connect({
|
||||
execute: async (name, arguments_, options) => {
|
||||
const selector = String(arguments_.procedure ?? arguments_.address);
|
||||
calls.push(`${name}:${selector}`);
|
||||
signals.push(options?.signal?.aborted ?? false);
|
||||
if (name === cancelAt)
|
||||
return err(new AnalysisCancelledError(operation));
|
||||
if (name === "procedure_address") return ok(selector);
|
||||
if (name === "address_name") return ok("fixture_name");
|
||||
if (name === "procedure_pseudo_code") return ok("return 0;");
|
||||
throw new Error(`Unexpected operation: ${name}`);
|
||||
},
|
||||
});
|
||||
|
||||
const response = await client.callTool({
|
||||
name: "batch_decompile",
|
||||
arguments: { addresses: ["0x1"] },
|
||||
});
|
||||
|
||||
expect(response.isError).toBe(true);
|
||||
expect(response.structuredContent).toBeUndefined();
|
||||
expect(parseMcpToolError(response)).toMatchObject({
|
||||
error: { code: "cancelled", details: { operation } },
|
||||
});
|
||||
expect(calls).toEqual(expectedCalls);
|
||||
expect(signals).toEqual(expectedCalls.map(() => false));
|
||||
},
|
||||
);
|
||||
|
||||
it("stops later batch stages across the input when one entry lookup is cancelled", async () => {
|
||||
const calls: string[] = [];
|
||||
const client = await connect({
|
||||
execute: async (name, arguments_) => {
|
||||
const selector = String(arguments_.procedure ?? arguments_.address);
|
||||
calls.push(`${name}:${selector}`);
|
||||
if (name === "procedure_address") return ok(selector);
|
||||
if (name === "address_name" && selector === "0x1")
|
||||
return err(new AnalysisCancelledError(name));
|
||||
if (name === "address_name") return ok("fixture_name");
|
||||
if (name === "procedure_pseudo_code") return ok("return 0;");
|
||||
throw new Error(`Unexpected operation: ${name}`);
|
||||
},
|
||||
});
|
||||
|
||||
const response = await client.callTool({
|
||||
name: "batch_decompile",
|
||||
arguments: { addresses: ["0x1", "0x2"] },
|
||||
});
|
||||
|
||||
expect(response.isError).toBe(true);
|
||||
expect(response.structuredContent).toBeUndefined();
|
||||
expect(parseMcpToolError(response)).toMatchObject({
|
||||
error: { code: "cancelled", details: { operation: "address_name" } },
|
||||
});
|
||||
expect(calls).toEqual([
|
||||
"procedure_address:0x1",
|
||||
"procedure_address:0x2",
|
||||
"address_name:0x1",
|
||||
"address_name:0x2",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("enhanced MCP batches", () => {
|
||||
it("returns every batch item in caller order with its own result", async () => {
|
||||
const addresses = Array.from(
|
||||
@@ -358,61 +454,143 @@ describe("enhanced MCP batches", () => {
|
||||
});
|
||||
|
||||
it("returns ordered typed batch failures and zero counts for empty input", async () => {
|
||||
const tools = new EnhancedTools({
|
||||
execute: (name, arguments_) =>
|
||||
const client = await connect({
|
||||
execute: async (name, arguments_) =>
|
||||
name === "procedure_address"
|
||||
? Promise.resolve(ok(String(arguments_.procedure)))
|
||||
? ok(String(arguments_.procedure))
|
||||
: name === "address_name"
|
||||
? Promise.resolve(ok(null))
|
||||
? ok(null)
|
||||
: arguments_.procedure === "0x2"
|
||||
? Promise.resolve(
|
||||
err(new AnalysisOutputError("decompile", "failed")),
|
||||
)
|
||||
: Promise.resolve(ok("pseudo")),
|
||||
? err(new AnalysisOutputError("decompile", "failed"))
|
||||
: ok("pseudo"),
|
||||
});
|
||||
|
||||
const result = await tools.execute("batch_decompile", {
|
||||
addresses: ["0x1", "0x2"],
|
||||
const result = await client.callTool({
|
||||
name: "batch_decompile",
|
||||
arguments: { addresses: ["0x1", "0x2"] },
|
||||
});
|
||||
const empty = await client.callTool({
|
||||
name: "batch_decompile",
|
||||
arguments: { addresses: [] },
|
||||
});
|
||||
const empty = await tools.execute("batch_decompile", { addresses: [] });
|
||||
|
||||
expect(result).toEqual({
|
||||
ok: true,
|
||||
value: {
|
||||
items: [
|
||||
{
|
||||
address: "0x1",
|
||||
procedure: { status: "resolved", address: "0x1", name: null },
|
||||
status: "ok",
|
||||
pseudocode: "pseudo",
|
||||
},
|
||||
{
|
||||
address: "0x2",
|
||||
procedure: { status: "resolved", address: "0x2", name: null },
|
||||
status: "error",
|
||||
error: {
|
||||
code: "unreadable_output",
|
||||
category: "execution_failure",
|
||||
details: { operation: "decompile", reason: "failed" },
|
||||
message:
|
||||
expect(result.isError).not.toBe(true);
|
||||
expect(jsonResult(result)).toEqual({
|
||||
items: [
|
||||
{
|
||||
address: "0x1",
|
||||
procedure: { status: "resolved", address: "0x1", name: null },
|
||||
status: "ok",
|
||||
pseudocode: "pseudo",
|
||||
},
|
||||
{
|
||||
address: "0x2",
|
||||
procedure: { status: "resolved", address: "0x2", name: null },
|
||||
status: "error",
|
||||
error: {
|
||||
code: "unreadable_output",
|
||||
category: "execution_failure",
|
||||
details: { operation: "decompile", reason: "failed" },
|
||||
message:
|
||||
"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.",
|
||||
retryable: false,
|
||||
remediation: {
|
||||
action:
|
||||
"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.",
|
||||
retryable: false,
|
||||
remediation: {
|
||||
action:
|
||||
"Analysis returned an unreadable result. Run `rea doctor` and review the reported details; repeating the unchanged request is not expected to succeed.",
|
||||
},
|
||||
},
|
||||
},
|
||||
],
|
||||
total: 2,
|
||||
succeeded: 1,
|
||||
failed: 1,
|
||||
},
|
||||
],
|
||||
total: 2,
|
||||
succeeded: 1,
|
||||
failed: 1,
|
||||
});
|
||||
expect(empty.isError).not.toBe(true);
|
||||
expect(jsonResult(empty)).toEqual({
|
||||
items: [],
|
||||
total: 0,
|
||||
succeeded: 0,
|
||||
failed: 0,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("enhanced MCP graph cancellation", () => {
|
||||
it("returns typed relationship cancellation without expanding queued children", async () => {
|
||||
const calls: string[] = [];
|
||||
const signals: boolean[] = [];
|
||||
const client = await connect({
|
||||
execute: async (name, arguments_, options) => {
|
||||
const selector = String(arguments_.procedure ?? arguments_.address);
|
||||
calls.push(`${name}:${selector}`);
|
||||
signals.push(options?.signal?.aborted ?? false);
|
||||
if (name === "procedure_address") return relationshipExecution("0x1");
|
||||
if (name !== "procedure_callees")
|
||||
throw new Error(`Unexpected operation: ${name}`);
|
||||
if (selector === "0x1") return relationshipExecution(["0x2", "0x3"]);
|
||||
if (selector === "0x2") return err(new AnalysisCancelledError(name));
|
||||
throw new Error("queued child 0x3 must not be expanded");
|
||||
},
|
||||
});
|
||||
expect(empty).toEqual({
|
||||
ok: true,
|
||||
value: { items: [], total: 0, succeeded: 0, failed: 0 },
|
||||
|
||||
const response = await client.callTool({
|
||||
name: "get_call_graph",
|
||||
arguments: { address: "0x1", direction: "forward" },
|
||||
});
|
||||
|
||||
expect(response.isError).toBe(true);
|
||||
expect(response.structuredContent).toBeUndefined();
|
||||
expect(parseMcpToolError(response)).toMatchObject({
|
||||
error: { code: "cancelled", details: { operation: "procedure_callees" } },
|
||||
});
|
||||
expect(calls).toEqual([
|
||||
"procedure_address:0x1",
|
||||
"procedure_callees:0x1",
|
||||
"procedure_callees:0x2",
|
||||
]);
|
||||
expect(signals).toEqual([false, false, false]);
|
||||
});
|
||||
|
||||
it("keeps ordinary provider failures as partial nodes and continues queued work", async () => {
|
||||
const calls: string[] = [];
|
||||
const client = await connect({
|
||||
execute: async (name, arguments_) => {
|
||||
const selector = String(arguments_.procedure ?? arguments_.address);
|
||||
calls.push(`${name}:${selector}`);
|
||||
if (name === "procedure_address") return relationshipExecution("0x1");
|
||||
if (name !== "procedure_callees")
|
||||
throw new Error(`Unexpected operation: ${name}`);
|
||||
if (selector === "0x1") return relationshipExecution(["0x2", "0x3"]);
|
||||
if (selector === "0x2")
|
||||
return err(new AnalysisOutputError(name, "ordinary fixture failure"));
|
||||
return relationshipExecution([]);
|
||||
},
|
||||
});
|
||||
|
||||
const response = await client.callTool({
|
||||
name: "get_call_graph",
|
||||
arguments: { address: "0x1", direction: "forward" },
|
||||
});
|
||||
|
||||
expect(response.isError).not.toBe(true);
|
||||
expect(jsonResult(response)).toMatchObject({
|
||||
graph: {
|
||||
"1": [
|
||||
{
|
||||
address: "0x2",
|
||||
status: "error",
|
||||
error: { code: "unreadable_output" },
|
||||
},
|
||||
{ address: "0x3", status: "ok", calls: [] },
|
||||
],
|
||||
},
|
||||
});
|
||||
expect(calls).toEqual([
|
||||
"procedure_address:0x1",
|
||||
"procedure_callees:0x1",
|
||||
"procedure_callees:0x2",
|
||||
"procedure_callees:0x3",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -10,7 +10,9 @@ import { z } from "zod";
|
||||
import { parseEvidence } from "../../../src/domain/evidence.js";
|
||||
import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
|
||||
import { writeJavaScriptSourceSelectionFixture } from "../../fixtures/javascriptSourceSelection.js";
|
||||
import { createStrippedAsarAddon } from "../../fixtures/strippedAsarAddon.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { writeFixtureFiles } from "../../support/javascriptApplicationFixture.js";
|
||||
|
||||
const createSelectionClient = () => {
|
||||
const client = new Client({
|
||||
@@ -30,6 +32,79 @@ const createSelectionClient = () => {
|
||||
return { client, transport };
|
||||
};
|
||||
|
||||
it("resolves ESM whitespace specifiers through the public application tool", async () => {
|
||||
const root = await createTestTempDirectory("rea-esm-whitespace-mcp-");
|
||||
const rawSpecifier = "./feature\tname.js";
|
||||
const encodedSpecifier = "./feature%09name.js";
|
||||
await writeFixtureFiles(root, {
|
||||
"main.mjs": `import ${JSON.stringify(rawSpecifier)};\nimport ${JSON.stringify(encodedSpecifier)};\n`,
|
||||
"featurename.js": "export const plain = 1;\n",
|
||||
"feature\tname.js": "export const value = 1;\n",
|
||||
});
|
||||
const { client, transport } = createSelectionClient();
|
||||
try {
|
||||
await client.connect(transport);
|
||||
const output = await client.callTool({
|
||||
name: "analyze_javascript_application",
|
||||
arguments: { input_path: root },
|
||||
});
|
||||
expect(output.isError, JSON.stringify(output.content)).not.toBe(true);
|
||||
const result = javascriptApplicationAnalysisResultSchema.parse(
|
||||
parseEvidence(output.structuredContent).normalized_result,
|
||||
);
|
||||
expect(result.graph.edges).toEqual(
|
||||
expect.arrayContaining([
|
||||
expect.objectContaining({
|
||||
properties: expect.objectContaining({
|
||||
specifier: rawSpecifier,
|
||||
resolved_path: "featurename.js",
|
||||
}),
|
||||
}),
|
||||
expect.objectContaining({
|
||||
properties: expect.objectContaining({
|
||||
specifier: encodedSpecifier,
|
||||
resolved_path: "feature\tname.js",
|
||||
}),
|
||||
}),
|
||||
]),
|
||||
);
|
||||
} finally {
|
||||
await client.close();
|
||||
await transport.close();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
it("retains missing ASAR native-addon inventory through the public application tool", async () => {
|
||||
const { archive, addon } = await createStrippedAsarAddon();
|
||||
await rm(join(`${archive}.unpacked`, addon));
|
||||
const { client, transport } = createSelectionClient();
|
||||
try {
|
||||
await client.connect(transport);
|
||||
const output = await client.callTool({
|
||||
name: "analyze_javascript_application",
|
||||
arguments: { input_path: archive },
|
||||
});
|
||||
expect(output.isError, JSON.stringify(output.content)).not.toBe(true);
|
||||
const result = javascriptApplicationAnalysisResultSchema.parse(
|
||||
parseEvidence(output.structuredContent).normalized_result,
|
||||
);
|
||||
expect(result.graph.coverage.status).toBe("partial");
|
||||
expect(
|
||||
result.graph.nodes.some(
|
||||
(node) =>
|
||||
node.kind === "native-addon" &&
|
||||
node.observations.some(
|
||||
({ label, properties }) =>
|
||||
label === addon && properties.availability === "unavailable",
|
||||
),
|
||||
),
|
||||
).toBe(true);
|
||||
} finally {
|
||||
await client.close();
|
||||
await transport.close();
|
||||
}
|
||||
}, 60_000);
|
||||
|
||||
it("advertises exact source selection and retains its scope through summary, Evidence retrieval and invalid-input recovery", async () => {
|
||||
const root = await createTestTempDirectory("rea-js-selected-mcp-");
|
||||
await writeJavaScriptSourceSelectionFixture(root);
|
||||
|
||||
@@ -20,8 +20,12 @@ const UNTRUSTED_TARGET_CONTENT = {
|
||||
provenance: { untrusted_content: true, source: "analysis_target" },
|
||||
};
|
||||
|
||||
const failureWithName = (nameBytes: number, targetContent = false) =>
|
||||
delivery.toErrorToolResult(
|
||||
const failureWithName = (
|
||||
nameBytes: number,
|
||||
targetContent = false,
|
||||
selectedDelivery = delivery,
|
||||
) =>
|
||||
selectedDelivery.toErrorToolResult(
|
||||
new AnalysisInputError("procedure_address", undefined, [
|
||||
{
|
||||
path: ["procedure"],
|
||||
@@ -260,10 +264,15 @@ it("keeps the untrusted target-content marker on an oversized error's replacemen
|
||||
|
||||
// The delivered error wire omits only the private structured carrier; its
|
||||
// `isError` flag and any provenance marker count against the result budget.
|
||||
const deliveredErrorBytes = (nameBytes: number, targetContent: boolean) => {
|
||||
const deliveredErrorBytes = (
|
||||
nameBytes: number,
|
||||
targetContent: boolean,
|
||||
selectedDelivery = delivery,
|
||||
) => {
|
||||
const { structuredContent: _carrier, ...wire } = failureWithName(
|
||||
nameBytes,
|
||||
targetContent,
|
||||
selectedDelivery,
|
||||
);
|
||||
return Buffer.byteLength(JSON.stringify(wire));
|
||||
};
|
||||
@@ -271,26 +280,28 @@ const deliveredErrorBytes = (nameBytes: number, targetContent: boolean) => {
|
||||
it.each([false, true])(
|
||||
"applies the result budget to the complete delivered error (target content: %s)",
|
||||
async (targetContent) => {
|
||||
const boundaryDelivery = new ToolResultDelivery(4096);
|
||||
const fittingName =
|
||||
delivery.resultBudgetBytes - deliveredErrorBytes(0, targetContent);
|
||||
expect(deliveredErrorBytes(fittingName, targetContent)).toBe(
|
||||
delivery.resultBudgetBytes,
|
||||
);
|
||||
boundaryDelivery.resultBudgetBytes -
|
||||
deliveredErrorBytes(0, targetContent, boundaryDelivery);
|
||||
expect(
|
||||
deliveredErrorBytes(fittingName, targetContent, boundaryDelivery),
|
||||
).toBe(boundaryDelivery.resultBudgetBytes);
|
||||
const toolName = targetContent ? "procedure_pseudo_code" : "record_unknown";
|
||||
const server = new EvidenceMcpServer(
|
||||
{ name: "error-budget-test", version: "1" },
|
||||
{ capabilities: {} },
|
||||
undefined,
|
||||
delivery,
|
||||
boundaryDelivery,
|
||||
);
|
||||
server.registerTool(toolName, { inputSchema: z.object({}) }, async () =>
|
||||
failureWithName(fittingName, targetContent, boundaryDelivery),
|
||||
);
|
||||
server.registerTool(
|
||||
"fitting_failure",
|
||||
targetContent ? "inspect_artifact" : "export_evidence_bundle",
|
||||
{ inputSchema: z.object({}) },
|
||||
async () => failureWithName(fittingName, targetContent),
|
||||
);
|
||||
server.registerTool(
|
||||
"one_byte_over_failure",
|
||||
{ inputSchema: z.object({}) },
|
||||
async () => failureWithName(fittingName + 1, targetContent),
|
||||
async () =>
|
||||
failureWithName(fittingName + 1, targetContent, boundaryDelivery),
|
||||
);
|
||||
const client = new Client({ name: "error-budget-client", version: "1" });
|
||||
const [clientTransport, serverTransport] =
|
||||
@@ -299,15 +310,19 @@ it.each([false, true])(
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
const fitting = await client.callTool({
|
||||
name: "fitting_failure",
|
||||
name: toolName,
|
||||
arguments: {},
|
||||
});
|
||||
expect(fitting.isError).toBe(true);
|
||||
expect(fitting._meta).toEqual(
|
||||
targetContent ? UNTRUSTED_TARGET_CONTENT : undefined,
|
||||
);
|
||||
expect(parseMcpToolError(fitting)).toEqual(
|
||||
failureWithName(fittingName, targetContent).structuredContent,
|
||||
failureWithName(fittingName, targetContent, boundaryDelivery)
|
||||
.structuredContent,
|
||||
);
|
||||
const over = await client.callTool({
|
||||
name: "one_byte_over_failure",
|
||||
name: targetContent ? "inspect_artifact" : "export_evidence_bundle",
|
||||
arguments: {},
|
||||
});
|
||||
expect(over.isError).toBe(true);
|
||||
|
||||
@@ -19,6 +19,8 @@ import type {
|
||||
ProviderAvailability,
|
||||
} from "../../../src/application/AnalysisProvider.js";
|
||||
import type { AndroidAnalysisPort } from "../../../src/application/android/AndroidAnalysisPort.js";
|
||||
import type { AdbDeviceAnalysisPort } from "../../../src/application/adb/AdbDeviceAnalysisPort.js";
|
||||
import type { ApktoolResourceAnalysisPort } from "../../../src/application/apktool/ApktoolResourceAnalysisPort.js";
|
||||
import { observed as ok } from "../../fixtures/analysisExecution.js";
|
||||
import { createServer } from "../../../src/server/createServer.js";
|
||||
import { silentLogger } from "../../../src/logger.js";
|
||||
@@ -26,7 +28,10 @@ import { createAnalysisProfile } from "../../../src/domain/analysisProfile.js";
|
||||
import { MAX_JSON_DEPTH } from "../../../src/domain/jsonValue.js";
|
||||
import { parseAnalysisSnapshot } from "../../../src/domain/analysisSnapshot.js";
|
||||
import { INVESTIGATION_EXAMPLES } from "../../../src/contracts/investigationExamples.js";
|
||||
import { expandProviderCapabilities } from "../../../src/contracts/toolOutputSchemaPrimitives.js";
|
||||
import {
|
||||
expandProviderCapabilities,
|
||||
toolAvailabilityInventory,
|
||||
} from "../../../src/contracts/toolOutputSchemaPrimitives.js";
|
||||
import { toolContract } from "../../../src/contracts/toolContracts.js";
|
||||
import { ok as resultOk } from "../../../src/domain/result.js";
|
||||
import {
|
||||
@@ -153,6 +158,65 @@ describe("MCP snapshot lifecycle ordering", () => {
|
||||
}, 10_000);
|
||||
});
|
||||
|
||||
describe("public provider availability", () => {
|
||||
it("reports ADB and Apktool host probes in the public session inventory", async () => {
|
||||
const closed: string[] = [];
|
||||
const session = createTestBinarySession(provider(closed));
|
||||
const available: ProviderAvailability = {
|
||||
status: "available",
|
||||
code: null,
|
||||
reason: null,
|
||||
diagnostics: { version: "1.0.41" },
|
||||
};
|
||||
const unavailable: ProviderAvailability = {
|
||||
status: "unavailable",
|
||||
code: "unsupported_version",
|
||||
reason: "Use Apktool 2.7.0 or newer.",
|
||||
diagnostics: { minimum_version: "2.7.0" },
|
||||
};
|
||||
const adbDeviceAnalysis: AdbDeviceAnalysisPort = {
|
||||
inspectAvailability: async () => available,
|
||||
close: async () => {},
|
||||
execute: async () => ok(null),
|
||||
};
|
||||
const apktoolAnalysis: ApktoolResourceAnalysisPort = {
|
||||
inspectAvailability: async () => unavailable,
|
||||
close: async () => {},
|
||||
execute: async () => ok(null),
|
||||
};
|
||||
const server = createServer(
|
||||
{ kind: "session", session },
|
||||
{ adbDeviceAnalysis, apktoolAnalysis, logger: silentLogger },
|
||||
);
|
||||
const mcp = new Client({
|
||||
name: "provider-host-readiness",
|
||||
version: "1.0.0",
|
||||
});
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
resources.push(mcp, server);
|
||||
await server.connect(serverTransport);
|
||||
await mcp.connect(clientTransport);
|
||||
|
||||
const status = structured(
|
||||
await mcp.callTool({ name: "binary_session", arguments: {} }),
|
||||
);
|
||||
const availability = toolAvailabilityInventory.parse(
|
||||
(status.result as { tool_availability: unknown }).tool_availability,
|
||||
);
|
||||
expect(
|
||||
availability.groups.find(({ names }) =>
|
||||
names.includes("inspect_adb_client"),
|
||||
),
|
||||
).toMatchObject({ available: true, reason: "available" });
|
||||
expect(
|
||||
availability.groups.find(({ names }) =>
|
||||
names.includes("inspect_apktool_client"),
|
||||
),
|
||||
).toMatchObject({ available: false, reason: "provider_unavailable" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("target-free MCP lifecycle", () => {
|
||||
it("takes the session snapshot after Android readiness inspection completes", async () => {
|
||||
directory = await createTestTempDirectory("rea-mcp-status-readiness-");
|
||||
|
||||
@@ -70,6 +70,98 @@ it("rejects oversized initial and resize terminal buffers through MCP before lau
|
||||
await client.ping();
|
||||
});
|
||||
|
||||
itWithCaptureCapability(
|
||||
"bounds normalized PTY text and preserves equal-size output through MCP",
|
||||
async () => {
|
||||
const root = await createTestTempDirectory(
|
||||
"rea-process-normalized-budget-",
|
||||
);
|
||||
const session = createTestBinarySession(createCacheProvider([]));
|
||||
const server = createServer(
|
||||
{ kind: "session", session },
|
||||
{ logger: silentLogger },
|
||||
);
|
||||
const client = new Client({
|
||||
name: "process-normalized-budget",
|
||||
version: "1",
|
||||
});
|
||||
resources.push(client, server, session);
|
||||
const [clientTransport, serverTransport] =
|
||||
InMemoryTransport.createLinkedPair();
|
||||
await server.connect(serverTransport);
|
||||
await client.connect(clientTransport);
|
||||
|
||||
const capture = async (replacement: string) => {
|
||||
const result = await client.callTool({
|
||||
name: "capture_process_scenario",
|
||||
arguments: {
|
||||
executable: process.execPath,
|
||||
working_directory: root,
|
||||
arguments: ["-e", 'process.stdout.write("a".repeat(100) + "\\n")'],
|
||||
limits: { output_bytes: 1_000 },
|
||||
normalization: {
|
||||
paths: false,
|
||||
pids: false,
|
||||
ports: false,
|
||||
patterns: [{ pattern: "a", replacement }],
|
||||
},
|
||||
settle_ms: 50,
|
||||
},
|
||||
});
|
||||
expect(result.isError, JSON.stringify(result.content)).not.toBe(true);
|
||||
return toolContract("capture_process_scenario").outputSchema.parse(
|
||||
result.structuredContent,
|
||||
).normalized_result;
|
||||
};
|
||||
|
||||
const expanded = await capture("x".repeat(500));
|
||||
expect(expanded.frames).toHaveLength(0);
|
||||
expect(expanded.truncated).toBe(true);
|
||||
expect(expanded.truncation_details.normalized_terminal).toMatchObject({
|
||||
budget_bytes: 1_000,
|
||||
retained_bytes: 0,
|
||||
omitted_frames: expect.any(Number),
|
||||
});
|
||||
expect(
|
||||
expanded.truncation_details.normalized_terminal?.omitted_frames,
|
||||
).toBeGreaterThan(0);
|
||||
expect(expanded.truncation_details.raw_terminal).toMatchObject({
|
||||
budget_bytes: 1_000,
|
||||
observed_bytes: expect.any(Number),
|
||||
retained_bytes: 0,
|
||||
observed_frames: expect.any(Number),
|
||||
retained_frames: 0,
|
||||
});
|
||||
expect(
|
||||
expanded.limitations.some((limitation) =>
|
||||
limitation.includes(
|
||||
"final normalized sizes for the latter omissions are unknown",
|
||||
),
|
||||
),
|
||||
).toBe(true);
|
||||
expect(expanded.cleanup).toMatchObject({
|
||||
owned_process_group: "verified",
|
||||
temporary_root: "removed",
|
||||
});
|
||||
|
||||
const equalSize = await capture("b");
|
||||
expect(equalSize.truncated).toBe(false);
|
||||
expect(equalSize.frames.map(({ data }) => data).join("")).toBe(
|
||||
`${"b".repeat(100)}\r\n`,
|
||||
);
|
||||
expect(equalSize.truncation_details.normalized_terminal).toMatchObject({
|
||||
budget_bytes: 1_000,
|
||||
retained_bytes: Buffer.byteLength(`${"b".repeat(100)}\r\n`),
|
||||
omitted_frames: 0,
|
||||
});
|
||||
expect(equalSize.cleanup).toMatchObject({
|
||||
owned_process_group: "verified",
|
||||
temporary_root: "removed",
|
||||
});
|
||||
},
|
||||
30_000,
|
||||
);
|
||||
|
||||
itWithCaptureCapability(
|
||||
"records process residuals linked to capture Evidence",
|
||||
async () => {
|
||||
|
||||
Reference in new issue
Block a user