scripte/rea
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into codex/ci-followups-20261015

Browse Source
This commit is contained in:
morluto committed 2026-10-15 14:11:13 +08:00
2 parents a715fb9e6d 1b036ecbff
commit 22ec8bdb2f
148 files changed
+9574 -750
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/cli.md
+3 -1
View File
@@ -314,7 +314,9 @@ diagnostics appear inside the report when `--full-output` is selected. Log
truncation alone does not change the diagnostic health or exit status.
The default terminal format is TOON. Use `--json` when saving results for a
JSON consumer. `--json` is indented on a terminal and compact when piped or
JSON consumer. Valued output flags (`--format`, `--filter-output`,
`--token-limit`, `--token-offset`) accept `--flag value` or `--flag=value`; a
missing value fails with `MISSING_OUTPUT_VALUE` before the command runs. `--json` is indented on a terminal and compact when piped or
redirected, which keeps large results smaller for agents and scripts; pipe
through `jq .` for indented files. Output selection and formatting do not change
operation status.
docs/installation.md
+31
View File
@@ -541,6 +541,37 @@ search permissions (`0700`) on a directory you own before retrying.
On supported Linux distributions, approved setup verifies Hopper's official
`.deb`, `.rpm`, or Arch package before invoking the native package manager.
The official release feed selects the current public demo without a version
pin. REA verifies the package's detached PGP signature against Hopper's stable
primary fingerprint `7C2D73BAFAF5B9A710A69DA1A26DF815099B1BCD`, then checks
that the signed package and installed launcher agree with `main_exe_hash`.
Rotating signing subkeys remain bound to that primary identity.
The feed row itself is not PGP-signed; doctor labels its display version as
`reported_release_version`, separately from the authenticated launcher identity.
The approval plan includes the caller-selected cache location. Approved setup
retains the signed archive, public key and release metadata under
`${XDG_CACHE_HOME:-$HOME/.cache}/rea/hopper-releases/<launcher-sha256>`.
An empty `XDG_CACHE_HOME` uses `$HOME/.cache`. Approved setup can repair a
damaged caller-owned entry after authenticating a fresh package; the previous
entry is preserved beside it as `<launcher-sha256>.backup-<id>`. Ordinary doctor
and launches reject damaged evidence and never repair it implicitly.
Approved publications of the same canonical cache entry share a process-backed
lease. Concurrent setup waits without moving another writer's evidence; a
failed replacement restores the previous directory when possible and reports
any retained backup path. Unavailable reads or native inspection tools do not
authorize replacing existing evidence.
Doctor and launches authenticate this evidence for offline use; an editable
digest receipt alone cannot authorize a launcher. Verification reuses at most
eight immutable package proofs per process, keyed by the complete package,
key, signature, package format and expected launcher digest. Cached file reads
still check identity and content. Without retained evidence, inspecting an
existing install requires the matching current official package and network
access. Successful live inspection can be reused within that process after
rehashing the launcher; it does not write a persistent cache or claim that a
previously observed release is still the latest. Older uncached builds require
an approved update while the feed exposes only its current release.
REA runs the supported demo build on a private Xvfb display and selects Hopper's
offered demo mode for each analysis session; it does not require the user's
desktop display. Unattended package-manager access requires
docs/javascript-artifact-reconstruction.md
+8 -1
View File
@@ -223,7 +223,10 @@ Selected bounded text is then parsed as inert data to recover:
- Webpack and Rspack chunk registrations and module factories represented as
AST literals;
- static imports, dynamic imports, CommonJS `require` calls, workers, and
service workers;
service workers (only `new Worker(new URL(path, import.meta.url))` resolves
relative to the calling module; string worker and service-worker paths use
the document base URL or Node's working directory, which static analysis
does not establish, so they keep their expression and stay unresolved);
- route, network endpoint, storage, and vendor-marker observations;
- local source-map declarations and original
source names and content digests;
@@ -629,6 +632,10 @@ href are read without surrounding whitespace or embedded tabs and newlines, and
their percent-encoded path bytes are decoded; encoded dot and separator bytes
are still rejected. CommonJS module lookups retain extension and directory
resolution.
ESM, HTML, and URL references apply URL dot segments to empty path segments:
`./assets//../app.js` selects `assets/app.js`, while a remaining empty segment
such as `./assets//app.js` stays unavailable. CommonJS keeps filesystem path
semantics and selects `app.js` for the same text.
Unresolved HTML references retain their declaration, source range, and resolution
reason in the renderer observations.
HTML script source ranges follow the HTML parser across LF, CRLF, and bare CR
docs/native-investigation.md
+12 -1
View File
@@ -378,7 +378,12 @@ complete observation.
property: `N` means `nonatomic`; a complete typed attribute string without
`N` means the Objective-C default `atomic`;
missing or incomplete attribute metadata leaves `atomicity` and any
unestablished read-only value null. `is_meta_class`, `is_root_class`, and
unestablished read-only value null. A type attribute whose quoted class name
never closes is incomplete and marks `objc_properties_categories` partial
with `property_attributes_incomplete`. A property record that cannot be
decoded, such as one with an empty or unmapped name, is reported as
`property_decode_failed` at its entry address while sibling properties are
retained. `is_meta_class`, `is_root_class`, and
`ivar_count` are also nullable when the selected provider exposes only symbol
names. Class implementation methods have null `is_required` and `is_optional`
fields because those facts apply to protocol declarations; decoded protocol
@@ -570,6 +575,12 @@ Exact and ambiguous value counts describe emitted root/object value instances;
a root and an object may contain the same value. Archive digests identify the
original bytes.
Dates are reported as `{ "$plist_type": "date", "iso8601": "<timestamp>" }`. A
date the decoder cannot represent, such as an XML date with a year beyond 9999
or a negative year, is reported as `{ "$plist_type": "date", "iso8601": null }`
with a counted limitation; the rest of the plist or archive is still decoded.
`inspect_plist` and Interface Builder decoding use the same projection.
## Locate a byte signature
Use `search_bytes` after opening a target with Ghidra to locate a known constant
licenses/npm-dependencies.txt
+6 -1
View File
@@ -2,7 +2,7 @@ REA npm dependency inventory
Prepared from the root package-lock.json; refresh this inventory when locked dependencies change.
License identifiers are upstream lockfile declarations, not a legal assessment.
Runtime includes transitive and optional platform packages; development includes build, test and website tooling.
Locked package entries: 721; runtime: 177.
Locked package entries: 722; runtime: 178.
Full runtime copyright/license/NOTICE texts: npm-runtime-notices.txt.
@algolia/abtesting@1.25.0 | development | MIT
@@ -2740,6 +2740,11 @@ oniguruma-to-es@3.1.1 | development | MIT
Package: https://www.npmjs.com/package/oniguruma-to-es/v/3.1.1
Archive: https://registry.npmjs.org/oniguruma-to-es/-/oniguruma-to-es-3.1.1.tgz
Integrity: sha512-bUH8SDvPkH3ho3dvwJwfonjlQ4R80vjyvrU8YpxuROddv55vAEJrTuCuCVUhhsHbtlD9tGGbaNApGQckXhS8iQ==
openpgp@6.3.2 | runtime | LGPL-3.0+
Lockfile path: node_modules/openpgp
Package: https://www.npmjs.com/package/openpgp/v/6.3.2
Archive: https://registry.npmjs.org/openpgp/-/openpgp-6.3.2.tgz
Integrity: sha512-wcZTzHz41LV8Y48zH/JlD1JT8YdNmpWOuMAjQ/podvvCwsjBEU37K3znJh4UQcj3hksE5z9TzvhbUbwzsGvlLQ==
oxc-parser@0.150.0 | development | MIT
Lockfile path: node_modules/oxc-parser
Package: https://www.npmjs.com/package/oxc-parser/v/0.150.0
licenses/npm-runtime-notices.txt
+175 -1
View File
@@ -1,7 +1,7 @@
REA third-party npm runtime copyright, license and NOTICE texts
Collected from integrity-verified locked npm archives. Release-specific supplements and upstream notice gaps are identified in their entries.
External engines are separately obtained; see THIRD_PARTY_NOTICES.md.
Runtime dependency fingerprint (SHA-256): 49b181923ab240c68c64d35ecdd8f93cb5cd516c765f2282d0e0eca0bcadd13e
Runtime dependency fingerprint (SHA-256): 131100b2e1e6ad5e55b10a0baf4be2d7a0838fd724fb307b9bafd73446b0d3ec
===== @babel/helper-string-parser@8.0.6 =====
Declared license: MIT
@@ -5171,6 +5171,180 @@ ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR
IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
===== openpgp@6.3.2 =====
Declared license: LGPL-3.0+
Archive: https://registry.npmjs.org/openpgp/-/openpgp-6.3.2.tgz
Integrity: sha512-wcZTzHz41LV8Y48zH/JlD1JT8YdNmpWOuMAjQ/podvvCwsjBEU37K3znJh4UQcj3hksE5z9TzvhbUbwzsGvlLQ==
Scope: npm runtime dependency. GPLv3 terms incorporated by this LGPL text are preserved in licenses/GPL-3.0.txt.
--- package/LICENSE ---
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.
===== pako@1.0.11 =====
Declared license: (MIT AND Zlib)
Archive: https://registry.npmjs.org/pako/-/pako-1.0.11.tgz
package-lock.json
Generated
+19
View File
@@ -34,6 +34,7 @@
"isomorphic-git": "1.38.7",
"jsonc-parser": "3.3.1",
"lossless-json": "4.3.1",
"openpgp": "6.3.2",
"parse5": "8.0.1",
"pino": "10.4.0",
"playwright-core": "1.58.2",
@@ -7871,6 +7872,24 @@
"regex-recursion": "^6.0.2"
}
},
"node_modules/openpgp": {
"version": "6.3.2",
"resolved": "https://registry.npmjs.org/openpgp/-/openpgp-6.3.2.tgz",
"integrity": "sha512-wcZTzHz41LV8Y48zH/JlD1JT8YdNmpWOuMAjQ/podvvCwsjBEU37K3znJh4UQcj3hksE5z9TzvhbUbwzsGvlLQ==",
"license": "LGPL-3.0+",
"engines": {
"node": ">= 18.0.0",
"typescript": ">= 5.0.0"
},
"peerDependencies": {
"@openpgp/web-stream-tools": "~0.3.0"
},
"peerDependenciesMeta": {
"@openpgp/web-stream-tools": {
"optional": true
}
}
},
"node_modules/oxc-parser": {
"version": "0.150.0",
"resolved": "https://registry.npmjs.org/oxc-parser/-/oxc-parser-0.150.0.tgz",
package.json
+1
View File
@@ -337,6 +337,7 @@
"isomorphic-git": "1.38.7",
"jsonc-parser": "3.3.1",
"lossless-json": "4.3.1",
"openpgp": "6.3.2",
"parse5": "8.0.1",
"pino": "10.4.0",
"playwright-core": "1.58.2",
scripts/hopper-demo-x11.py
+12 -10
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Inspect or activate Hopper's version-pinned demo dialog on a private X display."""
"""Inspect or activate Hopper's authenticated demo dialog on a private X display."""
from __future__ import annotations
@@ -8,6 +8,7 @@ import ctypes
import hashlib
import json
import os
import re
from pathlib import Path
import secrets
import select
@@ -18,12 +19,6 @@ import threading
import time
# Keep this digest aligned with LinuxHopper.ts and rerun real-Hopper verification
# before accepting another vendor build.
SUPPORTED_HOPPER_SHA256 = {
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
"1339f9e58377442b0c6fcb0dfc3cec20d593cc557408521fad9a00dbc6b8da13",
}
EXPECTED_SCREEN = (1280, 1024)
DIALOG_TITLES = {"Registration", "Registration — Hopper Disassembler"}
HOPPER_WINDOW_TITLE = "Hopper Disassembler"
@@ -121,6 +116,7 @@ def parse_args() -> argparse.Namespace:
parser.add_argument("--probe", action="store_true")
parser.add_argument("--owner-stdin", action="store_true")
parser.add_argument("--hopper")
parser.add_argument("--expected-launcher-sha256")
parser.add_argument("--socket")
parser.add_argument(
"--strategy",
@@ -661,7 +657,11 @@ def run_probe(args: argparse.Namespace) -> dict[str, object]:
def run_launch(args: argparse.Namespace) -> int:
if args.hopper is None or args.socket is None:
if (
args.hopper is None or args.socket is None
or args.expected_launcher_sha256 is None
or re.fullmatch(r"[a-f0-9]{64}", args.expected_launcher_sha256) is None
):
raise AdapterFailure(
INVALID_LAUNCH_COMMAND,
"invalid_launch_command",
@@ -673,7 +673,9 @@ def run_launch(args: argparse.Namespace) -> int:
hopper = Path(args.hopper)
socket = Path(args.socket)
try:
supported = digest(hopper) in SUPPORTED_HOPPER_SHA256
# The parent owns publisher/package interpretation. Recheck its admitted
# launcher bytes immediately before execution, preserving all X11 guards.
supported = digest(hopper) == args.expected_launcher_sha256
except OSError as error:
raise AdapterFailure(
INVALID_LAUNCH_COMMAND,
@@ -684,7 +686,7 @@ def run_launch(args: argparse.Namespace) -> int:
raise AdapterFailure(
UNSUPPORTED_HOPPER_BUILD,
"unsupported_hopper_build",
"unsupported Hopper binary",
"Hopper binary differs from the authenticated release",
)
command = args.command[1:] if args.command[:1] == ["--"] else args.command
if not command or Path(command[0]).resolve() != hopper.resolve():
scripts/rea.mjs
+6 -4
View File
@@ -74,12 +74,14 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
const { createCliResultOutput } =
await import("../dist/cli-runtime/cliResultOutput.js");
const {
normalizeCliOutputArguments,
renderCliOutputArgumentError,
renderEmptyFilteredCliOutput,
validateCliOutputArguments,
} = await import("../dist/cli-runtime/cliOutput.js");
guardClosedStdout();
const outputArguments = validateCliOutputArguments(args);
const cliArgs = normalizeCliOutputArguments(args);
const outputArguments = validateCliOutputArguments(cliArgs);
const configurationError = cliConfigurationError(process.env);
if (configurationError !== undefined) {
process.stderr.write(`${configurationError}\n`);
@@ -89,8 +91,8 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
process.exitCode = 1;
} else {
let wroteOutput = false;
const resultOutput = createCliResultOutput(args, process.stdout);
await createCli(process.env, resultOutput).serve(args, {
const resultOutput = createCliResultOutput(cliArgs, process.stdout);
await createCli(process.env, resultOutput).serve(cliArgs, {
stdout: (output) => {
if (resultOutput?.handled) {
if (resultOutput.failed && !stdoutClosed)
@@ -104,7 +106,7 @@ if (!(await compiledRuntimeExists(runtimeFiles))) {
exit: (code) => process.exit(stdoutClosed ? SIGPIPE_EXIT_STATUS : code),
});
if (!wroteOutput && !resultOutput?.handled)
writeStdout(renderEmptyFilteredCliOutput(args) ?? "");
writeStdout(renderEmptyFilteredCliOutput(cliArgs) ?? "");
}
}
scripts/verify-linux-setup-e2e.mjs
+15 -6
View File
@@ -13,6 +13,7 @@ import { promisify } from "node:util";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
import { assertCleanLinuxHopperRuntime } from "./lib/clean-linux-hopper-runtime.mjs";
import { isUnsupportedHopperLaunchFailure } from "./verify-package-platform.mjs";
const execFileAsync = promisify(execFile);
const verifierRun = createVerifierRun();
@@ -89,16 +90,24 @@ try {
await appendFile(modifiedHopper, new Uint8Array([0]));
await chmod(modifiedHopper, 0o700);
try {
await execFileAsync(command, ["analyze", "/usr/bin/true"], {
env: { ...process.env, HOPPER_LAUNCHER_PATH: modifiedHopper },
timeout: 30_000,
});
await execFileAsync(
command,
["analyze", "/usr/bin/true", "--provider", "hopper", "--json"],
{
env: { ...process.env, HOPPER_LAUNCHER_PATH: modifiedHopper },
timeout: 30_000,
},
);
throw new Error("modified Hopper build was not rejected");
} catch (error) {
const output = `${error?.stdout ?? ""}\n${error?.stderr ?? ""}`;
if (!output.includes("unsupported_hopper_build"))
if (
error?.code !== 1 ||
typeof error.stdout !== "string" ||
!isUnsupportedHopperLaunchFailure(JSON.parse(error.stdout))
)
throw new Error(
"modified Hopper build did not return its stable failure code",
{ cause: error },
);
}
} finally {
scripts/verify-package-environment.mjs
+9
View File
@@ -1,5 +1,6 @@
import { chmod, mkdir, symlink, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { pathToFileURL } from "node:url";
import { supportedClients } from "../dist/application/SupportedClients.js";
/** Create an isolated prefix, fake PATH, client configs, and environment. */
@@ -121,6 +122,14 @@ export const packageHopperEnvironment = (root, platform = process.platform) => {
};
return {
HOPPER_LAUNCHER_PATH: "/bin/sh",
// Feed availability must not turn this foreign-launcher rejection test into
// an unrelated networking failure. Successful vendor verification is real E2E coverage.
NODE_OPTIONS: [
process.env.NODE_OPTIONS,
`--import=${pathToFileURL(join(root, "tests/fixtures/hopperRejectionFeed.mjs")).href}`,
]
.filter(Boolean)
.join(" "),
HOPPER_LOADER_ARGS_JSON: JSON.stringify([
"-c",
'node_path=$1; shift; "$node_path" "$@"',
scripts/verify-package-platform.mjs
+20 -5
View File
@@ -88,14 +88,29 @@ export async function verifyPackagePlatform({
}
}
/** Require the declared unsupported build or an exact missing-Xvfb diagnostic, never arbitrary provider failure. */
/** One owner for unsupported-launch evidence across package and real-Hopper verification. */
export const isUnsupportedHopperLaunchFailure = (failure) => {
const details = failure?.details;
const unsupportedBuild =
details?.failure_code === "unsupported_hopper_build" &&
details.exit_code === 72;
const publisherRejection =
details?.stage === "launch" &&
details.launch_rejection?.boundary === "publisher_verification" &&
details.launch_rejection.reason === "unsupported_hopper_build" &&
details.exit_code === undefined &&
details.launcher === undefined;
return (
failure?.code === "provider_unavailable" &&
(publisherRejection || unsupportedBuild)
);
};
/** Require an exact pre-execution rejection, observed helper exit, or missing-Xvfb diagnostic. */
export const assertLinuxPackageProviderFailure = (execution) => {
const failure = json(execution.stdout);
const details = failure?.details;
const diagnostic = details?.diagnostics;
const unsupportedBuild =
details?.failure_code === "unsupported_hopper_build" &&
details.exit_code === 72;
const missingDisplayDependency =
details?.failure_code === "runtime_dependency_unavailable" &&
details.exit_code === 79 &&
@@ -108,7 +123,7 @@ export const assertLinuxPackageProviderFailure = (execution) => {
if (
execution.status !== 1 ||
failure?.code !== "provider_unavailable" ||
(!unsupportedBuild && !missingDisplayDependency)
(!isUnsupportedHopperLaunchFailure(failure) && !missingDisplayDependency)
)
throw new Error(
`packaged Linux Hopper verification did not fail closed: ${JSON.stringify(failure)}`,
scripts/verify-real-hopper-fat.mjs
+1 -1
View File
@@ -23,7 +23,7 @@ import {
requireMcpOperationResult,
} from "./lib/mcp-verifier-results.mjs";
import { snapshotHopperRuntime } from "./lib/real-hopper-cleanup.mjs";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/hopper/HopperTargetLease.js";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/process/HopperLease.js";
import { parseConfig } from "../dist/config/parseConfig.js";
import {
verifyHopperSourceBinding,
scripts/verify-real-hopper.mjs
+1 -1
View File
@@ -12,7 +12,7 @@ import {
requirePseudocode,
} from "../dist/application/RealHopperAssertions.js";
import { HOPPER_PROVIDER_IDENTITY } from "../dist/hopper/HopperProviderCapabilities.js";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/hopper/HopperTargetLease.js";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/process/HopperLease.js";
import { REA_WORKFLOW_PROVIDER } from "../dist/application/InvestigationProviders.js";
import { loadRealHopperFixtureTargets } from "./lib/real-hopper-fixture.mjs";
import { verifyHopperStringObjects } from "./lib/real-hopper-search.mjs";
src/apktool/ApktoolConfiguration.ts
+73 -21
View File
@@ -8,10 +8,12 @@ import { snapshotEnvironment } from "../process/snapshotEnvironment.js";
import { ABORTED, waitForAbortable } from "../process/AbortablePromise.js";
import { ProviderStartupDeadline } from "../process/ProviderDeadline.js";
import { randomUUID } from "node:crypto";
import {
execFileOutput,
execFileOutputFailure,
} from "../process/ExecFileOutput.js";
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
import type { ProviderAvailability } from "../application/AnalysisProvider.js";
import type { ProviderRejectionCode } from "../contracts/providerSelection.js";
import type { JsonValue } from "../domain/jsonValue.js";
@@ -152,14 +154,23 @@ export type ApktoolConfigurationFailureCode =
| "version_unresolved"
| "unsupported_version";
/** Unresolved cleanup of a launched probe; its owner was retained separately. */
export interface ApktoolProbeCleanup {
readonly reason: string;
readonly resources: readonly string[];
}
export class ApktoolConfigurationFailure extends Error {
readonly cleanup: ApktoolProbeCleanup | undefined;
constructor(
readonly code: ApktoolConfigurationFailureCode,
message: string,
readonly diagnostics: Readonly<Record<string, JsonValue>> = {},
options?: ErrorOptions,
options?: ErrorOptions & { readonly cleanup?: ApktoolProbeCleanup },
) {
super(message, options);
this.cleanup = options?.cleanup;
}
}
@@ -228,6 +239,7 @@ export const inspectApktoolClient = async (
signal?: AbortSignal,
environment: Readonly<Record<string, string | undefined>> = process.env,
platform: NodeJS.Platform = process.platform,
onCleanupOwner?: (owner: ProviderProcessSupervisor) => void,
): Promise<ApktoolClientIdentity> => {
const hostEnvironment = snapshotEnvironment(environment, platform);
let invocation: ApktoolInvocation;
@@ -254,23 +266,57 @@ export const inspectApktoolClient = async (
let stdout: string;
let stderr: string;
try {
({ stdout, stderr } = await execFileOutput(
invocation.command,
invocation.arguments,
// The owned process group stops a shell launcher's children with it; a
// child holding inherited output pipes cannot outlive the deadline.
const output = await runOwnedCommand(
{
command: invocation.command,
arguments: invocation.arguments,
windowsVerbatimArguments: invocation.windowsVerbatimArguments,
timeout: VERSION_TIMEOUT_MS,
stopSignal: "SIGTERM",
maxBuffer: VERSION_MAX_OUTPUT_BYTES,
env: hostEnvironment,
...(signal === undefined ? {} : { signal }),
// Apktool launchers are commonly interpreter scripts, so ownership is
// proved by the run token rather than the launcher's argv prefix.
expectedCommand: null,
runId: randomUUID(),
hostEnvironment,
},
));
{
timeoutMs: VERSION_TIMEOUT_MS,
diagnosticBytes: VERSION_MAX_OUTPUT_BYTES,
},
signal === undefined ? {} : { signal },
);
stdout = output.stdout.text;
stderr = output.stderr.text;
} catch (cause) {
const failure = execFileOutputFailure(cause);
if (signal?.aborted || failure?.code === "ABORT_ERR")
throw new AnalysisCancelledError("inspect_apktool_client", { cause });
if (failure?.code === "ENOENT")
if (
cause instanceof OwnedCommandFailure &&
cause.cleanupOwner !== undefined
)
onCleanupOwner?.(cause.cleanupOwner);
const cleanup =
cause instanceof OwnedCommandFailure && cause.cleanupFailure !== null
? { reason: cause.cleanupFailure, resources: cause.resources }
: undefined;
const cleanupDetails = cleanup === undefined ? {} : { cleanup };
if (
signal?.aborted === true ||
(cause instanceof OwnedCommandFailure && cause.reason === "cancelled")
)
throw new AnalysisCancelledError("inspect_apktool_client", {
cause,
...cleanupDetails,
});
if (cause instanceof OwnedCommandFailure && cause.reason === "timeout")
throw new AnalysisTimeoutError(
"inspect_apktool_client",
VERSION_TIMEOUT_MS,
{ cause, ...cleanupDetails },
);
if (
cause instanceof Error &&
"code" in cause &&
(cause.code === "ENOENT" || cause.code === "EACCES")
)
throw new ApktoolConfigurationFailure(
"command_missing",
`The selected apktool launcher was not found: ${selection.command}`,
@@ -281,19 +327,25 @@ export const inspectApktoolClient = async (
},
{ cause },
);
const snapshot =
cause instanceof OwnedCommandFailure ? cause.snapshot : null;
throw new ApktoolConfigurationFailure(
"version_unresolved",
"The selected apktool launcher did not report a usable version",
{
selected_command: selection.command,
...(failure === undefined
...(snapshot === null
? {}
: {
exit_code: failure.code,
stderr_excerpt: failure.stderr.slice(0, 512),
exit_code: snapshot.exitCode ?? null,
stderr_excerpt: snapshot.stderr.text.slice(0, 512),
}),
...(cause instanceof OwnedCommandFailure &&
cause.reason === "output-limit"
? { max_output_bytes: VERSION_MAX_OUTPUT_BYTES }
: {}),
},
{ cause },
{ cause, ...cleanupDetails },
);
}
const identity = parseApktoolVersionOutput(stdout);
src/apktool/ApktoolProvider.ts
+65 -4
View File
@@ -56,6 +56,7 @@ import { WindowsBatchCommandError } from "../process/WindowsBatchCommand.js";
import {
APKTOOL_COMMAND_ENV,
ApktoolConfigurationFailure,
type ApktoolProbeCleanup,
apktoolConfigurationAvailability,
apktoolInvocation,
inspectApktoolClient,
@@ -148,6 +149,19 @@ interface DecodeLimits {
readonly maxOutputBytes: number;
}
/** A probe whose launched processes could not be confirmed stopped. */
const probeCleanupFailure = (
operation: string,
failure: ApktoolConfigurationFailure,
cleanup: ApktoolProbeCleanup,
): ProviderCleanupError =>
new ProviderCleanupError(
APKTOOL_PROVIDER_IDENTITY.id,
cleanup.resources,
{ reason: cleanup.reason, probe_failure: failure.message },
{ operation, cause: failure },
);
const runApktool = async (
operation: string,
selection: ApktoolCommandSelection,
@@ -180,6 +194,9 @@ const runApktool = async (
command: invocation.command,
arguments: invocation.arguments,
windowsVerbatimArguments: invocation.windowsVerbatimArguments,
// Apktool launchers are commonly interpreter scripts, so ownership is
// proved by the run token rather than the launcher's argv prefix.
expectedCommand: null,
runId: randomUUID(),
hostEnvironment: environment,
},
@@ -328,6 +345,10 @@ export class ApktoolProvider {
private readonly workspaces = new Set<string>();
private readonly removeWorkspace: (path: string) => Promise<void>;
readonly #pendingCommandOwners = new Map<ProviderProcessSupervisor, string>();
/** A version probe whose cleanup is unresolved is retried on close. */
readonly #retainProbeOwner = (owner: ProviderProcessSupervisor): void => {
this.#pendingCommandOwners.set(owner, "apktool version probe");
};
readonly #shutdown = new AbortController();
readonly #active = new Set<Promise<void>>();
#closing = false;
@@ -378,7 +399,13 @@ export class ApktoolProvider {
process.platform,
signal,
);
await inspectApktoolClient(selection, signal, environment);
await inspectApktoolClient(
selection,
signal,
environment,
process.platform,
this.#retainProbeOwner,
);
throwIfCancelled("inspect_apktool_client", signal);
return {
status: "available",
@@ -390,8 +417,15 @@ export class ApktoolProvider {
},
};
} catch (cause) {
if (cause instanceof ApktoolConfigurationFailure)
if (cause instanceof ApktoolConfigurationFailure) {
if (cause.cleanup !== undefined)
throw probeCleanupFailure(
"inspect_apktool_client",
cause,
cause.cleanup,
);
return apktoolConfigurationAvailability(cause);
}
throw cause;
}
}
@@ -437,7 +471,13 @@ export class ApktoolProvider {
process.platform,
signal,
);
identity = await inspectApktoolClient(selection, signal, environment);
identity = await inspectApktoolClient(
selection,
signal,
environment,
process.platform,
this.#retainProbeOwner,
);
} catch (cause) {
if (
cause instanceof AnalysisCancelledError ||
@@ -445,6 +485,10 @@ export class ApktoolProvider {
)
return err(cause);
if (cause instanceof ApktoolConfigurationFailure) {
if (cause.cleanup !== undefined)
return err(
probeCleanupFailure(request.operation, cause, cause.cleanup),
);
return err(
providerConfigurationRefusal(
APKTOOL_PROVIDER_IDENTITY.id,
@@ -497,12 +541,14 @@ export class ApktoolProvider {
async #drainAndClean(): Promise<void> {
await Promise.all(this.#active);
const failures: unknown[] = [];
const retained: string[] = [];
for (const [owner, root] of this.#pendingCommandOwners) {
try {
const stopped = await owner.stop();
if (stopped.status === "incomplete") throw new Error(stopped.reason);
this.#pendingCommandOwners.delete(owner);
} catch (cause) {
retained.push(root);
failures.push(
new Error(
`apktool process cleanup remains incomplete for ${root}: ${cause instanceof Error ? cause.message : String(cause)}`,
@@ -511,8 +557,23 @@ export class ApktoolProvider {
);
}
}
// Ownership stays retained for a later close; report it as typed cleanup.
if (failures.length > 0)
throw new AggregateError(failures, "Apktool process cleanup failed");
throw new ProviderCleanupError(
APKTOOL_PROVIDER_IDENTITY.id,
retained,
{
reason: failures
.map((failure) =>
failure instanceof Error ? failure.message : String(failure),
)
.join("; "),
},
{
operation: "close",
cause: new AggregateError(failures, "Apktool process cleanup failed"),
},
);
for (const path of this.workspaces) {
try {
await this.removeWorkspace(path);
src/apktool/ApktoolVersionProbe.lifecycle.test.ts
+107
View File
@@ -0,0 +1,107 @@
import { execFile } from "node:child_process";
import { chmod, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { setTimeout as delay } from "node:timers/promises";
import { promisify } from "node:util";
import { expect, it } from "vitest";
import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
import { inspectApktoolClient } from "./ApktoolConfiguration.js";
/**
* Only an exited process counts as stopped: it is gone, a zombie waiting to
* be reaped, or absent from `ps`. A failed inspection is not proof of exit.
*/
const alive = async (pid: number): Promise<boolean> => {
try {
process.kill(pid, 0);
} catch (cause: unknown) {
if ((cause as NodeJS.ErrnoException).code === "ESRCH") return false;
return true;
}
return promisify(execFile)("ps", ["-o", "stat=", "-p", `${pid}`]).then(
({ stdout }) => {
const state = stdout.trim();
return state !== "" && !state.startsWith("Z");
},
// `ps` exits 1 when no process matches; any other failure is unknown.
(cause: unknown) => (cause as { code?: unknown }).code !== 1,
);
};
const waitFor = async (check: () => Promise<boolean>, ms: number) => {
const deadline = Date.now() + ms;
while (Date.now() < deadline) {
if (await check()) return true;
await delay(25);
}
return false;
};
const readPids = async (path: string): Promise<readonly number[]> =>
readFile(path, "utf8")
.then((text) => JSON.parse(text) as number[])
.catch(() => []);
// The interpreter is the test runtime itself, so its environment (and the
// ownership run token) is readable on both Linux and macOS.
it.skipIf(process.platform === "win32")(
"stops an interpreter launcher and its pipe-holding child when the probe is cancelled",
async () => {
const root = await createTestTempDirectory("rea-apktool-probe-");
const pids = join(root, "pids.json");
const launcher = join(root, "apktool");
await writeFile(
launcher,
[
`#!${process.execPath}`,
'const { spawn } = require("node:child_process");',
'const child = spawn(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { stdio: "inherit" });',
`require("node:fs").writeFileSync(${JSON.stringify(pids)}, JSON.stringify([process.pid, child.pid]));`,
"setInterval(() => {}, 1000);",
].join("\n"),
);
await chmod(launcher, 0o755);
const controller = new AbortController();
const started = performance.now();
const probe = inspectApktoolClient(
{ command: launcher, commandSource: "environment" },
controller.signal,
process.env,
).catch((cause: unknown) => cause);
try {
expect(
await waitFor(async () => (await readPids(pids)).length === 2, 10_000),
).toBe(true);
const launched = await readPids(pids);
controller.abort();
// A regressed probe never settles; bound the wait so cleanup still runs.
const outcome = await Promise.race([
probe,
delay(15_000).then(() => "still pending"),
]);
expect(outcome).toBeInstanceOf(AnalysisCancelledError);
expect(performance.now() - started).toBeLessThan(15_000);
expect(
await waitFor(
async () =>
(await Promise.all(launched.map(alive))).every((live) => !live),
5_000,
),
).toBe(true);
} finally {
// A failed assertion must not leave the fixture's processes running.
controller.abort();
await Promise.race([probe, delay(10_000)]);
for (const pid of await readPids(pids))
if (await alive(pid))
try {
process.kill(pid, "SIGKILL");
} catch {
// Already exited between the check and the signal.
}
}
},
30_000,
);
src/apktool/ApktoolVersionProbe.test.ts
+147
View File
@@ -0,0 +1,147 @@
import { afterEach, expect, it, vi } from "vitest";
import {
AnalysisCancelledError,
AnalysisTimeoutError,
} from "../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import type { ProviderProcessSupervisor } from "../process/ProviderProcess.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import {
ApktoolConfigurationFailure,
inspectApktoolClient,
} from "./ApktoolConfiguration.js";
import { ApktoolProvider } from "./ApktoolProvider.js";
vi.mock("../process/OwnedCommand.js", async (importOriginal) => ({
...(await importOriginal<typeof import("../process/OwnedCommand.js")>()),
runOwnedCommand: vi.fn(),
}));
afterEach(() => vi.mocked(runOwnedCommand).mockReset());
const selection = {
command: "/opt/apktool/apktool",
commandSource: "environment",
} as const;
const owner = {} as ProviderProcessSupervisor;
it("runs the version probe as an owned interpreter-script process group", async () => {
vi.mocked(runOwnedCommand).mockResolvedValueOnce({
stdout: { text: "2.9.3\n" },
stderr: { text: "" },
} as Awaited<ReturnType<typeof runOwnedCommand>>);
await expect(
inspectApktoolClient(selection, undefined, {}, "linux"),
).resolves.toEqual({ version: "2.9.3" });
expect(vi.mocked(runOwnedCommand).mock.lastCall?.[0]).toMatchObject({
command: "/opt/apktool/apktool",
arguments: ["--version"],
expectedCommand: null,
});
expect(vi.mocked(runOwnedCommand).mock.lastCall?.[1]).toEqual({
timeoutMs: 20_000,
diagnosticBytes: 64 * 1024,
});
});
it("reports an expired version probe as a typed timeout", async () => {
vi.mocked(runOwnedCommand).mockRejectedValueOnce(
new OwnedCommandFailure("timeout", "Command deadline elapsed."),
);
const failure = await inspectApktoolClient(
selection,
undefined,
{},
"linux",
).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(AnalysisTimeoutError);
expect(projectAnalysisError(failure as AnalysisTimeoutError)).toMatchObject({
code: "provider_timeout",
category: "timeout",
details: { timeout_ms: 20_000 },
});
});
it("retains an unresolved probe cleanup owner and reports the cleanup", async () => {
vi.mocked(runOwnedCommand).mockRejectedValueOnce(
new OwnedCommandFailure(
"timeout",
"Command deadline elapsed.",
null,
"cleanup incomplete",
undefined,
["pid:1"],
owner,
),
);
const retained: ProviderProcessSupervisor[] = [];
const failure = await inspectApktoolClient(
selection,
undefined,
{},
"linux",
(cleanupOwner) => retained.push(cleanupOwner),
).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(AnalysisTimeoutError);
expect(projectAnalysisError(failure as AnalysisTimeoutError)).toMatchObject({
code: "cleanup_incomplete",
});
expect(retained).toEqual([owner]);
});
it("reports a cancelled version probe as cancelled", async () => {
vi.mocked(runOwnedCommand).mockRejectedValueOnce(
new OwnedCommandFailure("cancelled", "Command cancelled."),
);
await expect(
inspectApktoolClient(selection, undefined, {}, "linux"),
).rejects.toBeInstanceOf(AnalysisCancelledError);
});
it("keeps unresolved probe cleanup typed when the launcher overflows its output", async () => {
const overflow = () =>
new OwnedCommandFailure(
"output-limit",
"Command diagnostic output exceeded its complete-output budget.",
null,
"process group still running",
undefined,
["pid:7", "process-group:7"],
owner,
);
vi.mocked(runOwnedCommand).mockRejectedValueOnce(overflow());
const failure = await inspectApktoolClient(
selection,
undefined,
{},
"linux",
).catch((cause: unknown) => cause);
expect(failure).toBeInstanceOf(ApktoolConfigurationFailure);
expect((failure as ApktoolConfigurationFailure).cleanup).toEqual({
reason: "process group still running",
resources: ["pid:7", "process-group:7"],
});
// Both provider entry points report the unresolved processes as cleanup.
const provider = new ApktoolProvider({
environment: { REA_APKTOOL_COMMAND: "/opt/apktool/apktool" },
});
vi.mocked(runOwnedCommand).mockRejectedValueOnce(overflow());
await expect(provider.inspectAvailability()).rejects.toBeInstanceOf(
ProviderCleanupError,
);
vi.mocked(runOwnedCommand).mockRejectedValueOnce(overflow());
const result = await provider.execute({
operation: "decode_android_resources",
input: {},
} as never);
expect(result.ok ? null : result.error).toBeInstanceOf(ProviderCleanupError);
expect(result.ok ? null : projectAnalysisError(result.error)).toMatchObject({
code: "cleanup_incomplete",
details: { resources: ["pid:7", "process-group:7"] },
});
});
src/application/Doctor.fixture.ts
+11 -1
View File
@@ -1,5 +1,6 @@
import { CATALOG_IDENTITY } from "../catalogIdentity.js";
import { PRODUCT_IDENTITY } from "../identity.js";
import { ok } from "../domain/result.js";
import type { DoctorHost } from "./Doctor.js";
/** Healthy recording host for focused doctor and projection tests. */
@@ -14,7 +15,16 @@ export const createDoctorHostFixture = (
linuxDistribution: () => Promise.resolve(undefined),
validTarget: (path) => Promise.resolve(path.includes("Hopper")),
executable: (path) => Promise.resolve(path.includes("Hopper")),
supportedLinuxHopper: () => Promise.resolve(true),
inspectLinuxHopper: () =>
Promise.resolve(
ok({
sha256: "0".repeat(64),
version: "fixture",
primaryFingerprint: "0".repeat(40),
signingFingerprint: "1".repeat(40),
source: "signed_package_cache" as const,
}),
),
linuxDemoRuntimeCheck: () =>
Promise.resolve({
name: "hopper-demo-runtime",
src/application/Doctor.test.ts
+12 -2
View File
@@ -4,6 +4,8 @@ import { SUPPORTED_NODE_VERSION_PROSE } from "../domain/runtimeVersion.js";
import type { ClientRegistrationStatus } from "./ClientRegistrationStatus.js";
import { createDoctorHostFixture as host } from "./Doctor.fixture.js";
import { runDoctor } from "./Doctor.js";
import { err } from "../domain/result.js";
import { HopperReleaseError } from "./HopperReleaseFeed.js";
describe("doctor runtime check", () => {
it("reports Node 25 as unsupported", async () => {
@@ -617,12 +619,20 @@ describe("doctor Linux and Hopper discovery", () => {
hopperSetupSupported: true,
}),
executable: (candidate) => Promise.resolve(candidate === path),
supportedLinuxHopper: () => Promise.resolve(false),
inspectLinuxHopper: () =>
Promise.resolve(
err(
new HopperReleaseError(
"unsupported_hopper_build",
"Unverified fixture build",
),
),
),
}),
);
expect(
result.checks.find(({ name }) => name === "hopper-version")?.remediation,
).toContain("Unset or update HOPPER_LAUNCHER_PATH");
).toContain("Check HOPPER_LAUNCHER_PATH");
});
});
src/application/Doctor.ts
+8 -3
View File
@@ -19,11 +19,14 @@ import {
macHopperRenamedLauncherCandidates,
} from "./MacHopperPaths.js";
import {
linuxHopperBinarySupported,
inspectLinuxHopperLauncher,
linuxSharedLibrariesAvailable,
readLinuxDistribution,
type LinuxDistribution,
} from "./LinuxHopper.js";
import type { Result } from "../domain/result.js";
import type { HopperReleaseError } from "./HopperReleaseFeed.js";
import type { VerifiedLinuxHopperLauncher } from "./LinuxHopperPublisher.js";
import {
readInstalledSkillIdentity,
type InstalledSkillIdentity,
@@ -101,7 +104,9 @@ export interface DoctorHost {
/** Route-specific target admission details, when supplied by the host. */
inspectTarget?(path: string): Promise<DoctorCheck>;
executable(path: string): Promise<boolean>;
supportedLinuxHopper(path: string): Promise<boolean>;
inspectLinuxHopper(
path: string,
): Promise<Result<VerifiedLinuxHopperLauncher, HopperReleaseError>>;
linuxDemoRuntimeCheck(): Promise<DoctorCheck>;
brewHopperPath(): Promise<string | undefined>;
manualHopperPaths(): Promise<readonly string[]>;
@@ -512,7 +517,7 @@ export const systemDoctorHost = (
),
linuxDemoRuntimeCheck:
options.linuxDemoRuntimeCheck ?? uncomposedLinuxDemoRuntimeCheck,
supportedLinuxHopper: linuxHopperBinarySupported,
inspectLinuxHopper: (path) => inspectLinuxHopperLauncher(path, environment),
async brewHopperPath() {
return probeHomebrew(async (command) => {
let caskroom: string;
src/application/DoctorDiagnostics.ts
+30 -13
View File
@@ -209,17 +209,34 @@ const optionalHopperVersionCheck = async (
hopperPath: string | undefined,
): Promise<DoctorCheck | undefined> => {
if (host.platform !== "linux" || hopperPath === undefined) return undefined;
return check(
"hopper-version",
await host.supportedLinuxHopper(hopperPath),
hopperPath,
{
remediation: unsupportedHopperRemediation(
host.configuredHopperPath === hopperPath,
),
classification: "config_drift",
},
);
const inspected = await host.inspectLinuxHopper(hopperPath);
const buildCheck = check("hopper-version", inspected.ok, hopperPath, {
remediation: unsupportedHopperRemediation(
host.configuredHopperPath === hopperPath,
),
classification:
!inspected.ok &&
[
"release_metadata",
"download",
"verification_unavailable",
"cancelled",
"cleanup_failed",
].includes(inspected.error.reason)
? "unknown"
: "config_drift",
});
return {
...buildCheck,
details: inspected.ok
? {
reported_release_version: inspected.value.version,
launcher_sha256: inspected.value.sha256,
primary_fingerprint: inspected.value.primaryFingerprint,
source: inspected.value.source,
}
: { reason: inspected.error.reason, message: inspected.error.message },
};
};
const optionalHopperDemoCheck = async (
@@ -297,8 +314,8 @@ const providerDoctorChecks = (
const unsupportedHopperRemediation = (configured: boolean): string =>
configured
? "Unset or update HOPPER_LAUNCHER_PATH, install the Hopper build supported by this REA release, or update REA."
: "Install the Hopper build supported by this REA release, or update REA for a newer Hopper build.";
? "Check HOPPER_LAUNCHER_PATH and its publisher verification evidence. Run rea setup to install and retain an authenticated official Hopper package for offline use."
: "Run rea setup to install and retain an authenticated official Hopper package for offline use.";
const parseMajor = (version: string | undefined): number =>
Number.parseInt(version?.split(".")[0] ?? "0", 10);
src/application/DoctorProviderIds.ts
+7 -1
View File
@@ -1,2 +1,8 @@
/** Deep-analysis providers whose readiness can be selected explicitly by doctor. */
export const DOCTOR_PROVIDER_IDS = ["hopper", "ghidra", "ida", "jadx"] as const;
export const DOCTOR_PROVIDER_IDS = [
"hopper",
"ghidra",
"ida",
"jadx",
"kuna",
] as const;
src/application/HopperDownload.ts
+18 -4
View File
@@ -4,9 +4,23 @@ export const HOPPER_DOWNLOAD_BYTES = 100_000_000;
/** Bound observed response bytes before retaining them, including chunked bodies. */
export const downloadHopperPackage = async (
url: string,
options: { readonly signal?: AbortSignal } = {},
options: {
readonly signal?: AbortSignal;
readonly maxBytes?: number;
readonly timeoutMs?: number;
} = {},
): Promise<{ readonly ok: boolean; readonly bytes: Uint8Array }> => {
const deadline = AbortSignal.timeout(120_000);
const timeoutMs = options.timeoutMs ?? 120_000;
if (!Number.isSafeInteger(timeoutMs) || timeoutMs < 1 || timeoutMs > 120_000)
throw new RangeError("Invalid Hopper response time budget");
const deadline = AbortSignal.timeout(timeoutMs);
const maxBytes = options.maxBytes ?? HOPPER_DOWNLOAD_BYTES;
if (
!Number.isSafeInteger(maxBytes) ||
maxBytes < 1 ||
maxBytes > HOPPER_DOWNLOAD_BYTES
)
throw new RangeError("Invalid Hopper response byte budget");
const response = await fetch(url, {
headers: { accept: "application/json", "user-agent": "rea-installer" },
signal:
@@ -17,7 +31,7 @@ export const downloadHopperPackage = async (
const refused = { ok: false, bytes: new Uint8Array() };
if (
!response.ok ||
Number(response.headers.get("content-length")) > HOPPER_DOWNLOAD_BYTES
Number(response.headers.get("content-length")) > maxBytes
) {
await response.body?.cancel();
return refused;
@@ -30,7 +44,7 @@ export const downloadHopperPackage = async (
while (true) {
const chunk = await reader.read();
if (chunk.done) return { ok: true, bytes: Buffer.concat(chunks, bytes) };
if (chunk.value.byteLength > HOPPER_DOWNLOAD_BYTES - bytes) {
if (chunk.value.byteLength > maxBytes - bytes) {
await reader.cancel();
return refused;
}
src/application/HopperReleaseFeed.test.ts
+77
View File
@@ -0,0 +1,77 @@
import { describe, expect, it } from "vitest";
import { parseLinuxHopperReleases } from "./HopperReleaseFeed.js";
const release = {
version: "99.2.1",
filename:
"https://www.hopperapp.com:443/downloader/public/Hopper-99.2.1-Linux-demo.deb",
file_length: "123",
file_hash: "a".repeat(40),
main_exe_hash: "b".repeat(64),
signature: "AQ==",
public: "1",
beta: "0",
published_at: "preserved vendor field",
};
describe("Linux Hopper release feed boundary", () => {
it("preserves future release identity and partial platform facts", () => {
const parsed = parseLinuxHopperReleases({
"Ubuntu / Mint": release,
"OS X": { signature: "Sparkle DSA" },
});
expect(parsed).toMatchObject({
ok: true,
value: {
deb: { ok: true, value: release },
rpm: { ok: false },
arch: { ok: false },
},
});
if (!parsed.ok || !parsed.value.deb.ok) throw new Error("Missing release");
expect(parsed.value.deb.value.filename).toBe(release.filename);
});
it.each([
{ signature: undefined },
{ signature: "AQ==garbage" },
{ main_exe_hash: "b".repeat(40) },
{ file_length: "100000001" },
{ file_length: "NaN" },
{ public: "0" },
{ beta: "1" },
{
filename:
"https://other.example/downloader/public/Hopper-99.2.1-Linux-demo.deb",
},
{ filename: release.filename + "?token=caller" },
{
filename: release.filename.replace("https://", "https://user:password@"),
},
{ filename: release.filename.replace(".deb", ".rpm") },
{ filename: release.filename.replace("demo.deb", "full.deb") },
])(
"rejects metadata outside the selected public package contract: %j",
(changes) => {
expect(
parseLinuxHopperReleases({
"Ubuntu / Mint": { ...release, ...changes },
}),
).toMatchObject({
ok: true,
value: { deb: { ok: false, error: { reason: "release_metadata" } } },
});
},
);
it.each([null, [], "html error page"])(
"keeps malformed feed roots unknown",
(raw) => {
expect(parseLinuxHopperReleases(raw)).toMatchObject({
ok: false,
error: { reason: "release_metadata" },
});
},
);
});
src/application/HopperReleaseFeed.ts
+125
View File
@@ -0,0 +1,125 @@
import { z } from "zod";
import { err, ok, type Result } from "../domain/result.js";
import { HOPPER_DOWNLOAD_BYTES } from "./HopperDownload.js";
/** Vendor endpoints, shared by installation and existing-build inspection. */
export const HOPPER_RELEASES_URL =
"https://www.hopperapp.com/include/files-api.php?request=releases&public=true";
export const HOPPER_PUBLIC_KEY_URL =
"https://www.hopperapp.com/hopper-public-key.asc";
export const HOPPER_DOWNLOAD_PREFIX =
"https://www.hopperapp.com:443/downloader/public/";
/** Stable publisher identity; signing subkeys are admitted through this primary. */
export const HOPPER_LINUX_PRIMARY_FINGERPRINT =
"7C2D73BAFAF5B9A710A69DA1A26DF815099B1BCD";
export type LinuxPackageFamily = "deb" | "rpm" | "arch";
const platforms = {
deb: "Ubuntu / Mint",
rpm: "Fedora",
arch: "Arch",
} as const;
const suffixes = { deb: ".deb", rpm: ".rpm", arch: ".pkg.tar.xz" } as const;
export class HopperReleaseError extends Error {
readonly _tag = "HopperReleaseError";
constructor(
readonly reason:
| "release_metadata"
| "download"
| "integrity"
| "publisher_verification_failed"
| "verification_unavailable"
| "unsupported_hopper_build"
| "cancelled"
| "cleanup_failed"
| "cache_missing",
message: string,
options?: ErrorOptions,
) {
super(message, options);
this.name = this._tag;
}
}
const sha256 = z.string().regex(/^[a-f0-9]{64}$/u);
const releaseSchema = z
.object({
version: z.string().min(1),
filename: z.string().url(),
file_length: z
.string()
.regex(/^[1-9]\d*$/u)
.refine((text) => {
const count = Number(text);
return Number.isSafeInteger(count) && count <= HOPPER_DOWNLOAD_BYTES;
}),
file_hash: z.string().regex(/^[a-f0-9]{40}$/u),
main_exe_hash: sha256,
signature: z
.string()
.min(1)
.refine(
(text) => Buffer.from(text, "base64").toString("base64") === text,
),
public: z.literal("1"),
beta: z.literal("0"),
})
.passthrough();
/** Parsed once at the feed boundary; preserve the vendor's complete release row. */
export type LinuxHopperRelease = z.infer<typeof releaseSchema>;
export type LinuxHopperReleaseFeed = Readonly<
Record<LinuxPackageFamily, Result<LinuxHopperRelease, HopperReleaseError>>
>;
export const parseLinuxHopperRelease = (
raw: unknown,
family: LinuxPackageFamily,
): Result<LinuxHopperRelease, HopperReleaseError> => {
const parsed = releaseSchema.safeParse(raw);
if (!parsed.success)
return err(
new HopperReleaseError(
"release_metadata",
"The Hopper release omitted valid package identity or signature metadata.",
),
);
const url = new URL(parsed.data.filename);
if (
url.origin !== "https://www.hopperapp.com" ||
url.username !== "" ||
url.password !== "" ||
url.search !== "" ||
url.hash !== "" ||
!url.pathname.startsWith("/downloader/public/Hopper-") ||
!url.pathname.endsWith(`-Linux-demo${suffixes[family]}`)
)
return err(
new HopperReleaseError(
"release_metadata",
"The Hopper release is outside the official public Linux demo channel.",
),
);
return ok(parsed.data);
};
export const parseLinuxHopperReleases = (
raw: unknown,
): Result<LinuxHopperReleaseFeed, HopperReleaseError> => {
const feed = z.record(z.string(), z.unknown()).safeParse(raw);
if (!feed.success)
return err(
new HopperReleaseError(
"release_metadata",
"The Hopper releases response is not an object.",
),
);
return ok({
deb: parseLinuxHopperRelease(feed.data[platforms.deb], "deb"),
rpm: parseLinuxHopperRelease(feed.data[platforms.rpm], "rpm"),
arch: parseLinuxHopperRelease(feed.data[platforms.arch], "arch"),
});
};
src/application/LinuxHopper.test.ts
+79 -25
View File
@@ -4,7 +4,6 @@ import { describe, expect, it } from "vitest";
import {
formatCommandForDisplay,
installLinuxHopper,
linuxHopperLauncherDigestSupported,
linuxPackageManagerCommands,
linuxSharedLibrariesAvailable,
parseLinuxDistribution,
@@ -18,6 +17,12 @@ import {
type LinuxPackageInstallFailure,
type LinuxPackageInstallOutcome,
} from "./LinuxHopper.js";
import { err, ok } from "../domain/result.js";
import {
HopperReleaseError,
type LinuxHopperRelease,
} from "./HopperReleaseFeed.js";
import type { VerifiedLinuxHopperPackage } from "./LinuxHopperPublisher.js";
describe("Linux package-manager command display", () => {
it("preserves argument boundaries for paths with spaces and quotes", () => {
@@ -43,6 +48,9 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
archiveOk = true;
installSucceeds = true;
cleanupFails = false;
signatureValid = true;
cacheWritable = true;
verifiedBeforeInstall = false;
packageFailure: LinuxPackageInstallFailure = {
command: "pacman -U --noconfirm /tmp/rea test/hopper.arch",
exitCode: 1,
@@ -57,6 +65,8 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
distribution = (): Promise<LinuxDistribution | undefined> =>
Promise.resolve(this.distributionValue);
release = (family: LinuxPackageFamily) =>
Promise.resolve(ok(releasesFor(new Uint8Array([1, 2, 3]))[family]));
download = (url: string): Promise<LinuxHopperDownload> => {
this.downloads.push(url);
return Promise.resolve({ ok: this.archiveOk, bytes: this.archive });
@@ -64,10 +74,52 @@ class RecordingLinuxHost implements LinuxHopperInstallHost {
createTemporaryDirectory = (): Promise<string> =>
Promise.resolve("/tmp/rea test");
writeArchive = (): Promise<void> => Promise.resolve();
verifyArchive = (family: LinuxPackageFamily, release: LinuxHopperRelease) => {
if (
createHash("sha1").update(this.archive).digest("hex") !==
release.file_hash
)
return Promise.resolve(
err(new HopperReleaseError("integrity", "Fixture byte mismatch")),
);
if (!this.signatureValid)
return Promise.resolve(
err(
new HopperReleaseError(
"publisher_verification_failed",
"Fixture signature rejection",
),
),
);
this.verifiedBeforeInstall = true;
return Promise.resolve(
ok({
family,
release,
packageSha256: createHash("sha256").update(this.archive).digest("hex"),
publicKey: "fixture",
primaryFingerprint: "0".repeat(40),
signingFingerprint: "1".repeat(40),
}),
);
};
retainVerifiedArchive = (_verified: VerifiedLinuxHopperPackage) =>
Promise.resolve(
this.cacheWritable
? ok(undefined)
: err(
new HopperReleaseError(
"verification_unavailable",
"Fixture cache permission rejection",
),
),
);
installPackage = (
family: LinuxPackageFamily,
archive: string,
): Promise<LinuxPackageInstallOutcome> => {
if (!this.verifiedBeforeInstall)
throw new Error("Package manager ran before publisher verification");
this.installed.push({ family, archive });
const outcome: LinuxPackageInstallOutcome = this.installSucceeds
? { status: "installed" }
@@ -160,20 +212,6 @@ describe("Linux Hopper host classification", () => {
});
describe("Linux Hopper installation", () => {
it.each([
[
"Debian and Arch",
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
],
["RPM", "1339f9e58377442b0c6fcb0dfc3cec20d593cc557408521fad9a00dbc6b8da13"],
])("accepts the pinned %s launcher build", (_, digest) => {
expect(linuxHopperLauncherDigestSupported(digest)).toBe(true);
});
it("rejects an unverified launcher build", () => {
expect(linuxHopperLauncherDigestSupported("0".repeat(64))).toBe(false);
});
it.each([
["deb", "apt-get", ["xvfb", "xauth", "python3", "libx11-6", "libxtst6"]],
[
@@ -239,6 +277,23 @@ describe("Linux Hopper package installation", () => {
});
describe("Linux Hopper installation failure classification", () => {
it.each([
["signatureValid", "publisher_verification_failed"],
["cacheWritable", "verification_unavailable"],
] as const)(
"stops before privileged installation when %s fails",
async (field, reason) => {
const host = new RecordingLinuxHost();
host[field] = false;
expect(await installFixture(host)).toMatchObject({
status: "failed",
reason,
verificationFailure: { reason, message: expect.any(String) },
});
expect(host.installed).toEqual([]);
expect(host.cleaned).toEqual(["/tmp/rea test"]);
},
);
it.each([
["unsupported host", "unsupported_host"],
["package request failure", "download"],
@@ -263,7 +318,7 @@ describe("Linux Hopper installation failure classification", () => {
host.launcherStatusValue = "runtime_dependencies";
if (scenario === "unsupported build")
host.launcherStatusValue = "unsupported_hopper_build";
expect(await installFixture(host)).toEqual({
expect(await installFixture(host)).toMatchObject({
status: "failed",
reason,
...(scenario === "package manager failure"
@@ -363,14 +418,16 @@ describe("Linux Hopper temporary cleanup ownership", () => {
function releasesFor(
bytes: Uint8Array,
): Record<
LinuxPackageFamily,
{ filename: string; file_length: string; file_hash: string }
> {
): Record<LinuxPackageFamily, LinuxHopperRelease> {
const release = (extension: string) => ({
filename: `https://www.hopperapp.com:443/downloader/public/Hopper.${extension}`,
version: "future-fixture",
filename: `https://www.hopperapp.com:443/downloader/public/Hopper-future-fixture-Linux-demo.${extension}`,
file_length: String(bytes.byteLength),
file_hash: createHash("sha1").update(bytes).digest("hex"),
main_exe_hash: "0".repeat(64),
signature: "AQ==",
public: "1" as const,
beta: "0" as const,
});
return {
deb: release("deb"),
@@ -379,7 +436,4 @@ function releasesFor(
};
}
const installFixture = (host: RecordingLinuxHost) =>
installLinuxHopper(host, {
releases: releasesFor(new Uint8Array([1, 2, 3])),
});
const installFixture = (host: RecordingLinuxHost) => installLinuxHopper(host);
src/application/LinuxHopper.ts
+175 -102
View File
@@ -1,55 +1,37 @@
import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import { access, mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { execFileBoundedOutput } from "../process/ExecFileBoundedOutput.js";
import { execFileOutput } from "../process/ExecFileOutput.js";
import { downloadHopperPackage } from "./HopperDownload.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import { err, type Result } from "../domain/result.js";
import {
downloadHopperPackage,
HOPPER_DOWNLOAD_BYTES,
} from "./HopperDownload.js";
HOPPER_RELEASES_URL,
HOPPER_PUBLIC_KEY_URL,
HOPPER_DOWNLOAD_PREFIX,
HOPPER_LINUX_PRIMARY_FINGERPRINT,
HopperReleaseError,
type LinuxHopperRelease,
type LinuxPackageFamily,
} from "./HopperReleaseFeed.js";
import {
LinuxHopperPublisher,
retryLinuxHopperPublisherCleanup,
type VerifiedLinuxHopperPackage,
type VerifiedLinuxHopperLauncher,
} from "./LinuxHopperPublisher.js";
export type { LinuxPackageFamily } from "./HopperReleaseFeed.js";
/** Package-manager output retained per stream without stopping the command. */
const PACKAGE_MANAGER_OUTPUT_BYTES = 64 * 1024;
const DOWNLOAD_PREFIX = "https://www.hopperapp.com:443/downloader/public/";
const SUPPORTED_HOPPER_SHA256 = new Set([
"0294ced141cc373468ee22d8343e7dac41980cb05a937994ca81c9f09afe7ded",
"1339f9e58377442b0c6fcb0dfc3cec20d593cc557408521fad9a00dbc6b8da13",
]);
interface LinuxHopperRelease {
readonly filename: string;
readonly file_length: string;
readonly file_hash: string;
}
const SUPPORTED_RELEASES: Readonly<
Record<LinuxPackageFamily, LinuxHopperRelease>
> = {
deb: {
filename: `${DOWNLOAD_PREFIX}Hopper-6.4.2-Linux-demo.deb`,
file_length: "35755772",
file_hash: "e4f79dff602648a8ff4a88b773875b6bfac0dc65",
},
rpm: {
filename: `${DOWNLOAD_PREFIX}Hopper-6.4.2-Linux-demo.rpm`,
file_length: "32264475",
file_hash: "b9a65f2f583a386d440d4401d9140aef3bf305de",
},
arch: {
filename: `${DOWNLOAD_PREFIX}Hopper-6.4.2-Linux-demo.pkg.tar.xz`,
file_length: "29405968",
file_hash: "92d780b61cab38eda95dc4112dee4fbeb0920f46",
},
};
export type LinuxPackageFamily = "deb" | "rpm" | "arch";
/** Exact stable external effects disclosed before a Linux Hopper installation. */
export interface LinuxHopperInstallDisclosure {
readonly downloadUrl: string;
readonly expectedBytes: number;
readonly expectedSha1: string;
readonly networkOrigins: readonly string[];
readonly integrity: string;
readonly commands: readonly string[];
}
@@ -58,11 +40,13 @@ export const linuxHopperInstallDisclosure = (
family: LinuxPackageFamily,
isRoot: boolean,
): LinuxHopperInstallDisclosure => {
const release = SUPPORTED_RELEASES[family];
return {
downloadUrl: release.filename,
expectedBytes: Number(release.file_length),
expectedSha1: release.file_hash,
networkOrigins: [
HOPPER_RELEASES_URL,
HOPPER_PUBLIC_KEY_URL,
HOPPER_DOWNLOAD_PREFIX,
],
integrity: `Detached PGP signature · primary ${HOPPER_LINUX_PRIMARY_FINGERPRINT} · package byte identity and installed launcher SHA-256 from the official feed`,
commands: linuxPackageManagerCommands(
family,
`<verified-hopper.${family}>`,
@@ -133,6 +117,10 @@ export type LinuxPackageInstallOutcome =
/** External effects required to install the official Linux package. */
export interface LinuxHopperInstallHost {
distribution(): Promise<LinuxDistribution | undefined>;
release(
family: LinuxPackageFamily,
signal?: AbortSignal,
): Promise<Result<LinuxHopperRelease, HopperReleaseError>>;
download(
url: string,
options: { readonly signal?: AbortSignal },
@@ -143,8 +131,23 @@ export interface LinuxHopperInstallHost {
family: LinuxPackageFamily,
archive: string,
): Promise<LinuxPackageInstallOutcome>;
launcherStatus(path: string): Promise<LinuxHopperLauncherStatus>;
verifyArchive(
family: LinuxPackageFamily,
release: LinuxHopperRelease,
archive: string,
signal?: AbortSignal,
): Promise<Result<VerifiedLinuxHopperPackage, HopperReleaseError>>;
retainVerifiedArchive(
verified: VerifiedLinuxHopperPackage,
archive: string,
signal?: AbortSignal,
): Promise<Result<void, HopperReleaseError>>;
launcherStatus(
path: string,
expectedSha256: string,
): Promise<LinuxHopperLauncherStatus>;
cleanup(path: string): Promise<void>;
close?(): Promise<void>;
}
/** Structured outcome for every expected Linux installation failure. */
@@ -163,6 +166,9 @@ export type LinuxHopperInstallResult =
| "launcher_missing"
| "runtime_dependencies"
| "unsupported_hopper_build"
| "publisher_verification_failed"
| "verification_unavailable"
| "cleanup_failed"
| "cancelled";
/**
* The failing package-manager command and its output. Present only when
@@ -170,6 +176,8 @@ export type LinuxHopperInstallResult =
* package manager said instead of assuming an authorization problem.
*/
readonly packageFailure?: LinuxPackageInstallFailure;
/** Publisher-owned failure, retained for the public setup diagnostic. */
readonly verificationFailure?: HopperReleaseError;
} & { readonly cleanupFailure?: LinuxHopperTemporaryCleanupFailure });
/** Failed removal of an owned temporary install directory. */
@@ -270,9 +278,6 @@ export const installLinuxHopper = async (
host: LinuxHopperInstallHost = systemLinuxHopperInstallHost(process.env),
options: {
readonly signal?: AbortSignal;
readonly releases?: Readonly<
Record<LinuxPackageFamily, LinuxHopperRelease>
>;
} = {},
): Promise<LinuxHopperInstallResult> => {
if (options.signal?.aborted === true)
@@ -284,16 +289,56 @@ export const installLinuxHopper = async (
let result: LinuxHopperInstallResult;
try {
result = await (async (): Promise<LinuxHopperInstallResult> => {
const release = (options.releases ?? SUPPORTED_RELEASES)[
distribution.packageFamily
];
const selected = await host.release(
distribution.packageFamily,
options.signal,
);
if (!selected.ok)
return {
status: "failed",
verificationFailure: selected.error,
reason:
selected.error.reason === "cancelled"
? "cancelled"
: "release_metadata",
};
const release = selected.value;
const archiveDownload = await host.download(release.filename, options);
if (!archiveDownload.ok) return { status: "failed", reason: "download" };
if (!packageIntegrityMatches(archiveDownload.bytes, release))
return { status: "failed", reason: "integrity" };
temporary = await host.createTemporaryDirectory();
const archive = join(temporary, `hopper.${distribution.packageFamily}`);
await host.writeArchive(archive, archiveDownload.bytes);
const verified = await host.verifyArchive(
distribution.packageFamily,
release,
archive,
options.signal,
);
if (!verified.ok)
return {
status: "failed",
verificationFailure: verified.error,
reason:
verified.error.reason === "cache_missing"
? "verification_unavailable"
: verified.error.reason,
};
const retained = await host.retainVerifiedArchive(
verified.value,
archive,
options.signal,
);
if (!retained.ok)
return {
status: "failed",
verificationFailure: retained.error,
reason:
retained.error.reason === "cache_missing"
? "verification_unavailable"
: retained.error.reason,
};
if (options.signal?.aborted === true)
return { status: "failed", reason: "cancelled" };
const install = await host.installPackage(
distribution.packageFamily,
archive,
@@ -305,7 +350,10 @@ export const installLinuxHopper = async (
packageFailure: install.failure,
};
const launcherPath = "/opt/hopper/bin/Hopper";
const launcherStatus = await host.launcherStatus(launcherPath);
const launcherStatus = await host.launcherStatus(
launcherPath,
release.main_exe_hash,
);
if (launcherStatus !== "ready")
return {
status: "failed",
@@ -360,49 +408,82 @@ export const linuxHopperLauncherCandidates = (
/** Compose package-manager and linker checks from the selected caller environment. */
export const systemLinuxHopperInstallHost = (
environment: Readonly<NodeJS.ProcessEnv>,
): LinuxHopperInstallHost => ({
distribution: readLinuxDistribution,
download: downloadHopperPackage,
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-")),
writeArchive: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
installPackage: (family, archive) =>
installSystemPackage(family, archive, environment),
async launcherStatus(path) {
try {
await access(path);
const linked = await execFileOutput("ldd", [path], { env: environment });
if (!linuxSharedLibrariesAvailable(`${linked.stdout}\n${linked.stderr}`))
return "runtime_dependencies";
return (await linuxHopperBinarySupported(path))
? "ready"
: "unsupported_hopper_build";
} catch (cause: unknown) {
// best-effort cleanup: optional launcher probing; unreadable means missing.
void cause;
return "missing";
}
},
cleanup: (path) => rm(path, { recursive: true, force: true }),
});
/** Match the Linux Hopper build validated by the packaged demo-dialog adapter. */
export const linuxHopperBinarySupported = async (
path: string,
): Promise<boolean> => {
try {
const hash = createHash("sha256");
for await (const chunk of createReadStream(path)) hash.update(chunk);
return linuxHopperLauncherDigestSupported(hash.digest("hex"));
} catch (cause: unknown) {
// best-effort cleanup: optional build probing; unreadable means unsupported.
void cause;
return false;
}
): LinuxHopperInstallHost => {
const publisher = systemLinuxHopperPublisher(environment);
return {
distribution: readLinuxDistribution,
release: (family, signal) => publisher.currentRelease(family, signal),
download: downloadHopperPackage,
createTemporaryDirectory: () => mkdtemp(join(tmpdir(), "rea-hopper-")),
writeArchive: (path, bytes) => writeFile(path, bytes, { mode: 0o600 }),
installPackage: (family, archive) =>
installSystemPackage(family, archive, environment),
verifyArchive: (family, release, archive, signal) =>
publisher.verifyArchive(family, release, archive, signal),
retainVerifiedArchive: (verified, archive, signal) =>
publisher.retainVerifiedArchive(verified, archive, signal),
async launcherStatus(path, expectedSha256) {
try {
await access(path);
const linked = await execFileOutput("ldd", [path], {
env: environment,
});
if (
!linuxSharedLibrariesAvailable(`${linked.stdout}\n${linked.stderr}`)
)
return "runtime_dependencies";
return (await hashStableFile(path)).sha256 === expectedSha256
? "ready"
: "unsupported_hopper_build";
} catch (cause: unknown) {
// best-effort cleanup: optional launcher probing; unreadable means missing.
void cause;
return "missing";
}
},
cleanup: (path) => rm(path, { recursive: true, force: true }),
close: async () => {
await publisher.close();
await retryLinuxHopperPublisherCleanup();
},
};
};
/** Match a launcher extracted from one of the pinned official Linux packages. */
export const linuxHopperLauncherDigestSupported = (digest: string): boolean =>
SUPPORTED_HOPPER_SHA256.has(digest);
export const systemLinuxHopperPublisher = (
environment: Readonly<NodeJS.ProcessEnv>,
): LinuxHopperPublisher =>
new LinuxHopperPublisher(
environment,
async () => (await readLinuxDistribution())?.packageFamily,
);
/** Read-only authentication; persistent package retention belongs to approved setup. */
export const inspectLinuxHopperLauncher = async (
path: string,
environment: Readonly<NodeJS.ProcessEnv>,
signal?: AbortSignal,
): Promise<Result<VerifiedLinuxHopperLauncher, HopperReleaseError>> => {
const publisher = systemLinuxHopperPublisher(environment);
const result = await publisher.inspectLauncher(path, signal);
try {
await publisher.close();
await retryLinuxHopperPublisherCleanup();
return result;
} catch (cause: unknown) {
return err(
new HopperReleaseError(
"cleanup_failed",
"Hopper publisher verification retains incomplete cleanup.",
{
cause: new AggregateError(
result.ok ? [cause] : [result.error, cause],
"Publisher verification outcomes",
),
},
),
);
}
};
/** Interpret ldd output conservatively so a present but broken launcher is unhealthy. */
export const linuxSharedLibrariesAvailable = (output: string): boolean =>
@@ -492,14 +573,6 @@ export const linuxPackageManagerCommands = (
});
};
const packageIntegrityMatches = (
bytes: Uint8Array,
release: LinuxHopperRelease,
): boolean =>
bytes.byteLength === Number(release.file_length) &&
bytes.byteLength <= HOPPER_DOWNLOAD_BYTES &&
createHash("sha1").update(bytes).digest("hex") === release.file_hash;
const packageFamilyFor = (
id: string,
idLike: string | undefined,
src/application/LinuxHopperPublisher.ts
+833
View File
@@ -0,0 +1,833 @@
import { createHash, randomUUID } from "node:crypto";
import { copyFile, lstat, mkdir, rename, writeFile } from "node:fs/promises";
import { isAbsolute, join } from "node:path";
import { z } from "zod";
import {
ArtifactResourceScope,
type ArtifactResourceOwner,
} from "../artifacts/ArtifactResourceScope.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import { readBoundedFileBytes } from "../process/BoundedFileBytes.js";
import { verifyDetachedPgpSignature } from "../process/DetachedPgpSignature.js";
import { execFileOutput } from "../process/ExecFileOutput.js";
import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
import { waitForAbortableDelay } from "../process/ProviderDeadline.js";
import {
acquireHopperReleaseCacheLease,
type HopperTargetLease,
} from "../process/HopperLease.js";
import {
downloadHopperPackage,
HOPPER_DOWNLOAD_BYTES,
} from "./HopperDownload.js";
import {
HOPPER_LINUX_PRIMARY_FINGERPRINT,
HOPPER_PUBLIC_KEY_URL,
HOPPER_RELEASES_URL,
HopperReleaseError,
parseLinuxHopperRelease,
parseLinuxHopperReleases,
type LinuxHopperRelease,
type LinuxPackageFamily,
} from "./HopperReleaseFeed.js";
import { withRegularFile } from "./RegularFileRead.js";
/** Metadata/key inspection budget, independent of the compressed package budget. */
const RELEASE_METADATA_BYTES = 1024 * 1024;
/** Small feed/key requests must not consume the full package-download deadline. */
const RELEASE_METADATA_TIMEOUT_MS = 10_000;
const cacheRecordSchema = z.object({
format: z.literal("rea-signed-hopper-release-v1"),
family: z.enum(["deb", "rpm", "arch"]),
release: z.unknown(),
});
export interface VerifiedLinuxHopperPackage {
readonly release: LinuxHopperRelease;
readonly family: LinuxPackageFamily;
readonly packageSha256: string;
readonly publicKey: string;
readonly primaryFingerprint: string;
readonly signingFingerprint: string;
}
export interface VerifiedLinuxHopperLauncher {
readonly sha256: string;
/** Display value from release metadata; the package signature binds bytes, not this row. */
readonly version: string;
readonly primaryFingerprint: string;
readonly signingFingerprint: string;
readonly source:
| "signed_package_cache"
| "signed_vendor_release"
| "verified_process_cache";
}
/** Retain failed ephemeral cleanup independently of the caller's lifetime. */
const pendingCleanup = new Set<ArtifactResourceScope>();
/** Immutable cryptographic facts, bounded to eight packages for this process lifetime. */
const packageProofs = new Map<
string,
Pick<VerifiedLinuxHopperPackage, "primaryFingerprint" | "signingFingerprint">
>();
const liveLauncherProofs = new Map<string, VerifiedLinuxHopperLauncher>();
const retainProof = <T>(cache: Map<string, T>, key: string, proof: T): void => {
cache.delete(key);
cache.set(key, proof);
const oldest = cache.keys().next().value;
if (cache.size > 8 && oldest !== undefined) cache.delete(oldest);
};
/** One owner for the selected cache location, shared with the approval plan. */
export const linuxHopperReleaseCacheDirectory = (
environment: Readonly<NodeJS.ProcessEnv>,
): Result<string, HopperReleaseError> => {
try {
const supplied = environment.XDG_CACHE_HOME || undefined;
if (supplied !== undefined && !isAbsolute(supplied))
return err(
new HopperReleaseError(
"verification_unavailable",
"XDG_CACHE_HOME must be an absolute filesystem path.",
),
);
return ok(
join(
supplied ??
join(homeDirectoryFromEnvironment(environment, "linux"), ".cache"),
"rea",
"hopper-releases",
),
);
} catch (cause: unknown) {
return err(
new HopperReleaseError(
"verification_unavailable",
"The Hopper release cache location could not be resolved.",
{ cause },
),
);
}
};
/** Authenticated package facts shared by setup, doctor, and the launch boundary. */
export class LinuxHopperPublisher {
readonly #resources = new ArtifactResourceScope();
#feed: ReturnType<typeof parseLinuxHopperReleases> | undefined;
constructor(
readonly environment: Readonly<NodeJS.ProcessEnv>,
readonly packageFamily: () => Promise<LinuxPackageFamily | undefined>,
) {}
get cacheRoot(): string {
const location = linuxHopperReleaseCacheDirectory(this.environment);
if (!location.ok) throw location.error;
return location.value;
}
async currentRelease(
family: LinuxPackageFamily,
signal?: AbortSignal,
): Promise<Result<LinuxHopperRelease, HopperReleaseError>> {
try {
signal?.throwIfAborted();
if (this.#feed === undefined) {
const response = await downloadHopperPackage(HOPPER_RELEASES_URL, {
...(signal === undefined ? {} : { signal }),
maxBytes: RELEASE_METADATA_BYTES,
timeoutMs: RELEASE_METADATA_TIMEOUT_MS,
});
if (!response.ok)
throw new HopperReleaseError(
"release_metadata",
"The official Hopper release feed is unavailable.",
);
this.#feed = parseLinuxHopperReleases(
JSON.parse(decodeUtf8(response.bytes)),
);
}
if (!this.#feed.ok) return this.#feed;
return this.#feed.value[family];
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "release_metadata"));
}
}
async verifyArchive(
family: LinuxPackageFamily,
release: LinuxHopperRelease,
archive: string,
signal?: AbortSignal,
suppliedPublicKey?: string,
): Promise<Result<VerifiedLinuxHopperPackage, HopperReleaseError>> {
try {
const bytes = await this.#read(archive, HOPPER_DOWNLOAD_BYTES, signal);
if (
bytes.length !== Number(release.file_length) ||
createHash("sha1").update(bytes).digest("hex") !== release.file_hash
)
return err(
new HopperReleaseError(
"integrity",
"The Hopper package does not match the selected release bytes.",
),
);
let publicKey = suppliedPublicKey;
if (publicKey === undefined) {
const response = await downloadHopperPackage(HOPPER_PUBLIC_KEY_URL, {
...(signal === undefined ? {} : { signal }),
maxBytes: RELEASE_METADATA_BYTES,
timeoutMs: RELEASE_METADATA_TIMEOUT_MS,
});
if (!response.ok)
throw new HopperReleaseError(
"verification_unavailable",
"The official Hopper public key is unavailable.",
);
publicKey = decodeUtf8(response.bytes);
}
const packageSha256 = createHash("sha256").update(bytes).digest("hex");
// Every read still validates file identity, byte bounds and selected metadata.
// Reuse only proof of the exact signed bytes/key/signature/payload interpretation.
const proofKey = createHash("sha256")
.update(
JSON.stringify([
family,
packageSha256,
publicKey,
release.signature,
release.main_exe_hash,
]),
)
.digest("hex");
signal?.throwIfAborted();
const existingProof = packageProofs.get(proofKey);
if (existingProof !== undefined)
return ok({
release,
family,
packageSha256,
publicKey,
...existingProof,
});
const verified = await verifyDetachedPgpSignature(
bytes,
Buffer.from(release.signature, "base64"),
publicKey,
HOPPER_LINUX_PRIMARY_FINGERPRINT,
signal,
);
if (!verified.ok)
return err(
new HopperReleaseError(
verified.error.reason === "cancelled"
? "cancelled"
: "publisher_verification_failed",
verified.error.message,
{ cause: verified.error },
),
);
const mainExeHash = await this.#withTemporary(async (root) => {
// Extract only after authentication, from this operation's private byte snapshot.
const snapshot = join(root, "package");
await writeFile(snapshot, bytes, { flag: "wx", mode: 0o600 });
return this.#packageLauncherHash(family, snapshot, root, signal);
});
if (mainExeHash !== release.main_exe_hash)
return err(
new HopperReleaseError(
"integrity",
`The signed ${family} package launcher SHA-256 ${mainExeHash} differs from the release main_exe_hash ${release.main_exe_hash}.`,
),
);
retainProof(packageProofs, proofKey, verified.value);
return ok({
release,
family,
packageSha256,
publicKey,
...verified.value,
});
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "verification_unavailable"));
}
}
/** Durable evidence contains the signed bytes; an editable digest receipt cannot authorize a build. */
async retainVerifiedArchive(
verified: VerifiedLinuxHopperPackage,
archive: string,
signal?: AbortSignal,
): Promise<Result<void, HopperReleaseError>> {
try {
signal?.throwIfAborted();
await mkdir(this.cacheRoot, { recursive: true, mode: 0o700 });
const parent = await lstat(this.cacheRoot);
if (
!parent.isDirectory() ||
parent.isSymbolicLink() ||
parent.uid !== process.getuid?.()
)
throw new HopperReleaseError(
"verification_unavailable",
"The Hopper release cache is not a caller-owned directory.",
);
const target = join(this.cacheRoot, verified.release.main_exe_hash);
const lease = await this.#acquireCacheLease(
verified.release.main_exe_hash,
signal,
);
await this.#withOwnedResource(
{
kind: "lease",
resource: lease.socketPath ?? "hopper-release-cache-lease",
lease,
},
async () => {
signal?.throwIfAborted();
await this.#withTemporary(async (stage) => {
await copyFile(archive, join(stage, "archive"));
if (
(await hashStableFile(join(stage, "archive"), signal)).sha256 !==
verified.packageSha256
)
throw new HopperReleaseError(
"integrity",
"The verified package changed before cache retention.",
);
await writeFile(join(stage, "public-key.asc"), verified.publicKey, {
flag: "wx",
mode: 0o600,
});
await writeFile(
join(stage, "release.json"),
JSON.stringify({
format: "rea-signed-hopper-release-v1",
family: verified.family,
release: verified.release,
}) + "\n",
{ flag: "wx", mode: 0o600 },
);
await this.#publishCacheStage(
stage,
target,
verified.release.main_exe_hash,
signal,
);
}, this.cacheRoot);
},
);
return ok(undefined);
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "verification_unavailable"));
}
}
async #publishCacheStage(
stage: string,
target: string,
digest: string,
signal?: AbortSignal,
): Promise<void> {
const existing = await this.#cachedPackage(digest, signal);
if (existing.ok) return;
if (existing.error.reason === "cache_missing") {
await rename(stage, target);
return;
}
if (
![
"integrity",
"release_metadata",
"publisher_verification_failed",
].includes(existing.error.reason)
)
throw existing.error;
signal?.throwIfAborted();
const previous = await lstat(target);
if (
!previous.isDirectory() ||
previous.isSymbolicLink() ||
previous.uid !== process.getuid?.()
)
throw existing.error;
// Only approved retention under the publication lease may repair an entry.
// Preserve malformed evidence without following links or deleting content.
const backup = `${target}.backup-${randomUUID()}`;
await rename(target, backup);
try {
const moved = await lstat(backup);
if (moved.dev !== previous.dev || moved.ino !== previous.ino)
throw new Error("The Hopper cache entry changed during repair.");
await rename(stage, target);
} catch (cause: unknown) {
const replacement = await this.#cachedPackage(digest, signal);
if (replacement.ok) return;
await this.#failedCachePublication(
target,
backup,
[cause, replacement.error],
replacement.error.reason === "cleanup_failed",
);
}
}
async #acquireCacheLease(
digest: string,
signal?: AbortSignal,
): Promise<HopperTargetLease> {
const deadline = performance.now() + 120_000;
const runId = randomUUID();
let delay = 25;
while (true) {
signal?.throwIfAborted();
const acquired = await acquireHopperReleaseCacheLease({
cacheDirectory: this.cacheRoot,
launcherSha256: digest,
runId,
});
if (acquired.acquired) return acquired.lease;
const remaining = deadline - performance.now();
if (remaining <= 0)
throw new HopperReleaseError(
"verification_unavailable",
"Another approved Hopper setup still owns this release cache. Retry after it finishes.",
);
await waitForAbortableDelay(Math.min(delay, remaining), signal);
delay = Math.min(delay * 2, 200);
}
}
async #failedCachePublication(
target: string,
backup: string,
causes: unknown[],
cleanupFailed = false,
): Promise<never> {
let preserved = backup;
try {
// Under the publication lease, restore the previous directory on failure.
// POSIX rename cannot replace another nonempty evidence directory.
await rename(backup, target);
preserved = target;
} catch (cause: unknown) {
causes.push(cause);
}
throw new HopperReleaseError(
cleanupFailed ? "cleanup_failed" : "verification_unavailable",
`The Hopper cache repair did not finish; previous evidence is preserved at ${preserved}. Retry approved setup.`,
{
cause: new AggregateError(
causes,
"Hopper cache publication and recovery outcomes",
),
},
);
}
async inspectLauncher(
path: string,
signal?: AbortSignal,
): Promise<Result<VerifiedLinuxHopperLauncher, HopperReleaseError>> {
try {
const observed = await hashStableFile(path, signal);
const cached = await this.#cachedPackage(observed.sha256, signal);
if (cached.ok) {
if ((await hashStableFile(path, signal)).sha256 !== observed.sha256)
return err(
new HopperReleaseError(
"integrity",
"The Hopper launcher changed during publisher verification.",
),
);
return ok(launcherIdentity(cached.value, "signed_package_cache"));
}
if (cached.error.reason !== "cache_missing") return cached;
signal?.throwIfAborted();
const processProof = liveLauncherProofs.get(observed.sha256);
if (processProof !== undefined)
return ok({ ...processProof, source: "verified_process_cache" });
const family = await this.packageFamily();
if (family === undefined)
return err(
new HopperReleaseError(
"verification_unavailable",
"No native Linux package family is available for Hopper publisher verification.",
),
);
const selected = await this.currentRelease(family, signal);
if (!selected.ok) return selected;
if (selected.value.main_exe_hash !== observed.sha256)
return err(
new HopperReleaseError(
"unsupported_hopper_build",
"The selected Hopper launcher does not match an authenticated cached release or the official current Linux demo.",
),
);
const verified = await this.#withTemporary(async (root) => {
const response = await downloadHopperPackage(
selected.value.filename,
signal === undefined ? {} : { signal },
);
if (!response.ok)
throw new HopperReleaseError(
"download",
"The official Hopper package is unavailable for publisher verification.",
);
const archive = join(root, "archive");
await writeFile(archive, response.bytes, { flag: "wx", mode: 0o600 });
const verified = await this.verifyArchive(
family,
selected.value,
archive,
signal,
);
if (!verified.ok) throw verified.error;
return verified.value;
});
if ((await hashStableFile(path, signal)).sha256 !== observed.sha256)
return err(
new HopperReleaseError(
"integrity",
"The Hopper launcher changed during publisher verification.",
),
);
const identity = launcherIdentity(verified, "signed_vendor_release");
retainProof(liveLauncherProofs, observed.sha256, identity);
return ok(identity);
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "verification_unavailable"));
}
}
async close(): Promise<void> {
try {
await this.#resources.close();
pendingCleanup.delete(this.#resources);
} catch (cause: unknown) {
pendingCleanup.add(this.#resources);
throw cause;
}
}
async #cachedPackage(
digest: string,
signal?: AbortSignal,
): Promise<Result<VerifiedLinuxHopperPackage, HopperReleaseError>> {
try {
const root = join(this.cacheRoot, digest);
try {
const directory = await lstat(root);
if (!directory.isDirectory() || directory.isSymbolicLink())
throw new HopperReleaseError(
"integrity",
"The retained Hopper cache entry is not a regular directory.",
);
} catch (cause: unknown) {
if (cause instanceof Error && Reflect.get(cause, "code") === "ENOENT") {
return err(
new HopperReleaseError(
"cache_missing",
"No retained signed Hopper package identifies this launcher.",
{ cause },
),
);
}
throw cause;
}
// Distinguish proven malformed/missing evidence from unavailable reads or
// inspection tools. Only the former authorize approved cache replacement.
for (const name of ["release.json", "public-key.asc", "archive"])
await this.#assertCacheFile(join(root, name));
const metadata = await this.#read(
join(root, "release.json"),
RELEASE_METADATA_BYTES,
signal,
);
let value: unknown;
try {
value = JSON.parse(decodeUtf8(metadata));
} catch (cause: unknown) {
throw new HopperReleaseError(
"release_metadata",
"The retained Hopper package metadata is malformed.",
{ cause },
);
}
const parsed = cacheRecordSchema.safeParse(value);
if (!parsed.success)
throw new HopperReleaseError(
"release_metadata",
"The retained Hopper package metadata is malformed.",
);
const release = parseLinuxHopperRelease(
parsed.data.release,
parsed.data.family,
);
if (!release.ok) return release;
if (release.value.main_exe_hash !== digest)
throw new HopperReleaseError(
"integrity",
"The retained Hopper metadata does not identify this launcher.",
);
const keyBytes = await this.#read(
join(root, "public-key.asc"),
RELEASE_METADATA_BYTES,
signal,
);
let key: string;
try {
key = decodeUtf8(keyBytes);
} catch (cause: unknown) {
throw new HopperReleaseError(
"publisher_verification_failed",
"The retained Hopper public key is malformed.",
{ cause },
);
}
return this.verifyArchive(
parsed.data.family,
release.value,
join(root, "archive"),
signal,
key,
);
} catch (cause: unknown) {
return err(releaseFailure(cause, signal, "verification_unavailable"));
}
}
async #assertCacheFile(path: string): Promise<void> {
try {
const stats = await lstat(path);
if (!stats.isFile() || stats.isSymbolicLink())
throw new HopperReleaseError(
"integrity",
`Required Hopper evidence is not a regular file: ${path}.`,
);
} catch (cause: unknown) {
if (cause instanceof Error && Reflect.get(cause, "code") === "ENOENT")
throw new HopperReleaseError(
"integrity",
`Required Hopper evidence is missing: ${path}.`,
{ cause },
);
throw cause;
}
}
async #packageLauncherHash(
family: LinuxPackageFamily,
archive: string,
root: string,
signal?: AbortSignal,
): Promise<string> {
const commandOptions = {
env: this.environment,
signal,
timeout: 120_000,
stopSignal: "SIGKILL" as const,
maxBuffer: RELEASE_METADATA_BYTES,
};
if (family === "rpm") {
const output = await execFileOutput(
"rpm",
[
"-qp",
"--nosignature",
"--nodigest",
"--qf",
"%{FILEDIGESTALGO}\n[%{FILENAMES}\t%{FILEDIGESTS}\n]",
archive,
],
commandOptions,
);
const rows = output.stdout.split("\n");
if (rows.shift() !== "8")
throw new HopperReleaseError(
"integrity",
"The signed RPM header does not use SHA-256 file digests.",
);
const matches = rows.filter((row) =>
row.startsWith("/opt/hopper/bin/Hopper\t"),
);
const digest = matches[0]?.split("\t")[1];
if (
matches.length !== 1 ||
digest === undefined ||
!/^[a-f0-9]{64}$/u.test(digest)
)
throw new HopperReleaseError(
"integrity",
"The signed RPM header has no unique Hopper launcher digest.",
);
return digest;
}
const extracted = join(root, "extracted");
await mkdir(extracted, { mode: 0o700 });
if (family === "deb")
await execFileOutput(
"dpkg-deb",
["--extract", archive, extracted],
commandOptions,
);
else
await execFileOutput(
"tar",
["-xf", archive, "-C", extracted, "--", "opt/hopper/bin/Hopper"],
commandOptions,
);
return (
await hashStableFile(
join(extracted, "opt", "hopper", "bin", "Hopper"),
signal,
)
).sha256;
}
#read(path: string, limit: number, signal?: AbortSignal): Promise<Buffer> {
return withRegularFile(
path,
async (handle, stats) => {
if (stats.size > limit)
throw new HopperReleaseError(
"integrity",
"Retained Hopper evidence exceeds its resource byte budget.",
);
const bytes = await readBoundedFileBytes(handle, stats.size, signal);
if (bytes === undefined)
throw new HopperReleaseError(
"integrity",
"Retained Hopper evidence changed during inspection.",
);
return bytes;
},
{ signal, resources: this.#resources, symlinks: "reject" },
);
}
async #withTemporary<T>(
operation: (path: string) => Promise<T>,
parent?: string,
): Promise<T> {
const root = await PrivateRuntimeRoot.create({
prefix: "rea-hopper-publisher-",
...(parent === undefined ? {} : { parent }),
});
return this.#withOwnedResource(
{
kind: "private-runtime-root",
resource: root.path,
root,
},
() => operation(root.path),
);
}
async #withOwnedResource<T>(
owner: ArtifactResourceOwner,
operation: () => Promise<T>,
): Promise<T> {
let outcome:
| { readonly ok: true; readonly value: T }
| { readonly ok: false; readonly cause: unknown };
try {
outcome = { ok: true, value: await operation() };
} catch (cause: unknown) {
outcome = { ok: false, cause };
}
const cleanup = await this.#resources.release(owner);
if (cleanup.kind === "failed") {
pendingCleanup.add(this.#resources);
const causes = outcome.ok
? [cleanup.cause]
: [outcome.cause, cleanup.cause];
throw new HopperReleaseError(
"cleanup_failed",
`Hopper publisher verification retained cleanup ownership for ${owner.resource}.`,
{
cause: new AggregateError(
causes,
"Publisher inspection and cleanup outcomes",
),
},
);
}
if (!outcome.ok) throw outcome.cause;
return outcome.value;
}
}
/** Retry only resources retained by this producer after failed cleanup. */
export const retryLinuxHopperPublisherCleanup = async (): Promise<void> => {
const causes: unknown[] = [];
const observations = [];
for (const scope of pendingCleanup) {
try {
await scope.close();
pendingCleanup.delete(scope);
} catch (cause: unknown) {
causes.push(cause);
observations.push(
ArtifactReaderFailure.cleanupObservation(
cause,
"hopper-publisher-verification",
),
);
}
}
if (causes.length > 0) {
const cleanup = {
reason: observations.map(({ reason }) => reason).join("; "),
resources: [
...new Set(observations.flatMap(({ resources }) => resources)),
],
};
throw new ArtifactReaderFailure(
"unavailable",
`Hopper publisher cleanup remains incomplete: ${cleanup.resources.join(", ")}`,
{
cause: new AggregateError(causes, "Hopper publisher cleanup failures"),
cleanup,
},
);
}
};
const decodeUtf8 = (bytes: Uint8Array): string =>
new TextDecoder("utf-8", { fatal: true }).decode(bytes);
const releaseFailure = (
cause: unknown,
signal: AbortSignal | undefined,
fallback: HopperReleaseError["reason"],
): HopperReleaseError =>
cause instanceof HopperReleaseError && cause.reason === "cleanup_failed"
? cause
: signal?.aborted === true
? new HopperReleaseError(
"cancelled",
"Hopper publisher verification was cancelled.",
{ cause },
)
: cause instanceof HopperReleaseError
? cause
: new HopperReleaseError(
fallback,
"Hopper publisher verification could not complete.",
{ cause },
);
const launcherIdentity = (
verified: VerifiedLinuxHopperPackage,
source: VerifiedLinuxHopperLauncher["source"],
): VerifiedLinuxHopperLauncher => ({
sha256: verified.release.main_exe_hash,
version: verified.release.version,
primaryFingerprint: verified.primaryFingerprint,
signingFingerprint: verified.signingFingerprint,
source,
});
src/application/ReferenceSourceImportEntries.ts
+106 -13
View File
@@ -94,10 +94,95 @@ const failedEntry = (
};
};
/** A package directory's legacy `main` entry point, as its manifest declares it. */
type PackageMain =
| { readonly state: "declared"; readonly main: string }
| { readonly state: "none" }
| { readonly state: "unknown" };
/**
* Read each inventoried `package.json` once. Node selects a non-empty string
* `main` before a directory index; an unreadable or invalid manifest leaves
* that selection unknown.
*/
const readPackageMains = (
read: ReferenceSourceRead,
): ReadonlyMap<string, PackageMain> => {
const mains = new Map<string, PackageMain>();
for (const entry of read.entries) {
if (posix.basename(entry.path) !== "package.json") continue;
const directory = posix.dirname(entry.path);
if (entry.status !== "read" || entry.kind !== "file") {
mains.set(directory, { state: "unknown" });
continue;
}
try {
const manifest: unknown = JSON.parse(
new TextDecoder("utf-8", { fatal: true }).decode(entry.bytes),
);
const main =
typeof manifest === "object" && manifest !== null
? Reflect.get(manifest, "main")
: undefined;
mains.set(
directory,
typeof main === "string" && main !== ""
? { state: "declared", main }
: { state: "none" },
);
} catch {
mains.set(directory, { state: "unknown" });
}
}
return mains;
};
const RESOLUTION_SUFFIXES = ["", ".ts", ".js", ".mjs", ".cjs", ".tsx", ".jsx"];
/** Inventory facts an internal specifier resolves against. */
interface SpecifierContext {
readonly lookupPath: ReferenceSourcePathLookup;
readonly packageMains: ReadonlyMap<string, PackageMain>;
}
/**
* Apply a package `main` to a directory resolved through its index. Node loads
* the declared entry as a file, then as a directory index, and falls back to
* the package's own index only when neither exists.
*/
const selectPackageMain = (
index: string,
context: SpecifierContext,
): { to: string; resolution: "internal" | "unresolved" } => {
const directory = posix.dirname(index);
const manifest = context.packageMains.get(directory);
if (manifest === undefined || manifest.state === "none")
return { to: index, resolution: "internal" };
if (manifest.state === "unknown")
return { to: directory, resolution: "unresolved" };
// Node resolves `main` with path.resolve: an absolute main leaves the
// package, and a trailing separator does not make the entry a directory.
if (posix.isAbsolute(manifest.main))
return { to: posix.normalize(manifest.main), resolution: "unresolved" };
const target = posix
.normalize(posix.join(directory, manifest.main))
.replace(/(?<=.)\/+$/u, "");
// Node would load a declared entry outside the imported root.
if (logicalPathEscapesRoot(target))
return { to: target, resolution: "unresolved" };
const selected = context.lookupPath.resolveCandidates([
...RESOLUTION_SUFFIXES.map((suffix) => `${target}${suffix}`),
...RESOLUTION_SUFFIXES.filter((suffix) => suffix !== "").map((suffix) =>
posix.join(target, `index${suffix}`),
),
]);
return { to: selected ?? index, resolution: "internal" };
};
const resolveInternalSpecifier = (
fromPath: string,
specifier: string,
lookupPath: ReferenceSourcePathLookup,
context: SpecifierContext,
): { to: string; resolution: "internal" | "unresolved" } => {
if (posix.isAbsolute(specifier))
return { to: specifier, resolution: "unresolved" };
@@ -109,10 +194,11 @@ const resolveInternalSpecifier = (
// Preserve directory intent before normalization removes terminal dot segments.
const directoryReference =
specifier.endsWith("/") || [".", ".."].includes(posix.basename(specifier));
const suffixes = ["", ".ts", ".js", ".mjs", ".cjs", ".tsx", ".jsx"];
const candidates = suffixes.flatMap((suffix) => [
// A directory resolves only to an index with an extension; neither Node nor
// TypeScript selects an extensionless `index` file.
const candidates = RESOLUTION_SUFFIXES.flatMap((suffix) => [
...(directoryReference ? [] : [`${normalized}${suffix}`]),
posix.join(normalized, `index${suffix}`),
...(suffix === "" ? [] : [posix.join(normalized, `index${suffix}`)]),
]);
// Retain present runtime files first; substitute only a missing counterpart.
// Counterparts cover explicit filenames, not directory/index resolution.
@@ -122,9 +208,13 @@ const resolveInternalSpecifier = (
({ path }) => path,
),
);
const match = lookupPath.resolveCandidates(candidates);
return match === undefined
? { to: normalized, resolution: "unresolved" }
const match = context.lookupPath.resolveCandidates(candidates);
if (match === undefined) return { to: normalized, resolution: "unresolved" };
const resolvedThroughIndex =
posix.basename(match).startsWith("index.") &&
posix.dirname(match).normalize("NFC") === normalized.normalize("NFC");
return resolvedThroughIndex
? selectPackageMain(match, context)
: { to: match, resolution: "internal" };
};
@@ -132,7 +222,7 @@ type ReadFile = Extract<ReferenceSourceEntry, { status: "read"; kind: "file" }>;
const appendFile = (
entry: ReadFile,
lookupPath: ReferenceSourcePathLookup,
context: SpecifierContext,
output: ParsedReferenceSourceEntries,
): void => {
const language = detectReferenceSourceLanguage(entry.path);
@@ -159,7 +249,7 @@ const appendFile = (
const resolved = resolveInternalSpecifier(
relationship.from_path,
relationship.to,
lookupPath,
context,
);
output.relationships.push({
...relationship,
@@ -172,11 +262,11 @@ const appendFile = (
const appendReadEntry = (
entry: Extract<ReferenceSourceEntry, { status: "read" }>,
lookupPath: ReferenceSourcePathLookup,
context: SpecifierContext,
output: ParsedReferenceSourceEntries,
): void => {
if (entry.kind === "file") {
appendFile(entry, lookupPath, output);
appendFile(entry, context, output);
return;
}
const classifications = classifyReferenceSourcePath(entry.path);
@@ -219,11 +309,14 @@ export const parseReferenceSourceEntries = (
parseFailures: [],
limitations: [...read.limitations],
};
const lookupPath = createReferenceSourcePathLookup(filePaths);
const context: SpecifierContext = {
lookupPath: createReferenceSourcePathLookup(filePaths),
packageMains: readPackageMains(read),
};
for (const entry of read.entries) {
if (signal?.aborted === true) break;
if (entry.status === "failed") output.entries.push(failedEntry(entry));
else appendReadEntry(entry, lookupPath, output);
else appendReadEntry(entry, context, output);
}
return output;
};
src/application/Setup.planning.test.ts
+39
View File
@@ -5,6 +5,7 @@ import type { SetupProgressEvent } from "./SetupTypes.js";
import { FakeSetupHost, options } from "./Setup.fixture.js";
import { SetupSkillInstallError } from "./SetupSkill.js";
import { err, ok } from "../domain/result.js";
import { HopperReleaseError } from "./HopperReleaseFeed.js";
const linkedSkillFailure = () =>
new SetupSkillInstallError({
@@ -16,6 +17,44 @@ const linkedSkillFailure = () =>
cause: undefined,
});
describe("Linux publisher cache approval", () => {
it.each([true, false])(
"discloses the selected cache or blocks an unresolved path before approval: %s",
async (valid) => {
const host = Object.assign(new FakeSetupHost("linux"), {
linuxHopperReleaseCache: valid
? ok("/caller/cache/rea/hopper-releases")
: err(
new HopperReleaseError(
"verification_unavailable",
"Invalid selected cache path",
),
),
});
host.distribution = {
id: "ubuntu",
versionId: "24.04",
packageFamily: "deb",
hopperSetupSupported: true,
};
const result = await runSetup(options(false, true), host);
if (valid)
expect(
result.plannedActions.find(({ kind }) => kind === "install_hopper")
?.detail,
).toContain("/caller/cache/rea/hopper-releases/<launcher-sha256>");
else
expect(result).toMatchObject({
status: "needs_human",
plannedActions: [],
});
expect(host.hopperInstalls).toBe(0);
expect(host.configurations).toBe(0);
expect(host.skillInstalls).toBe(0);
},
);
});
describe("selected skill admission", () => {
it.each(["codex", "shared"] as const)(
"keeps %s choices when an unselected client is linked",
src/application/SetupHost.ts
+39 -4
View File
@@ -20,8 +20,10 @@ import {
LinuxHopperTemporaryCleanupOwner,
readLinuxDistribution,
systemLinuxHopperInstallHost,
type LinuxHopperInstallHost,
} from "./LinuxHopper.js";
import { installMacHopper, systemMacHopperInstallHost } from "./MacHopper.js";
import { linuxHopperReleaseCacheDirectory } from "./LinuxHopperPublisher.js";
import {
clientEvidencePaths,
supportedClients,
@@ -156,6 +158,12 @@ export const systemSetupHost = (
return {
platform,
homeDirectory,
...(platform === "linux"
? {
linuxHopperReleaseCache:
linuxHopperReleaseCacheDirectory(environment),
}
: {}),
skillDestinations: (clientIds) =>
skillDestinations(homeDirectory, clientIds, environment, platform),
...setupRegistration(platform, homeDirectory, environment),
@@ -164,7 +172,7 @@ export const systemSetupHost = (
linuxDistribution: readLinuxDistribution,
close: () =>
closeSetupResources(resources, macHopperHost, doctorHost, () =>
pendingLinuxHopperCleanup.retry(linuxHopperHost.cleanup),
closeLinuxHopperInstallHost(linuxHopperHost, pendingLinuxHopperCleanup),
),
initialSetupState: async (
scope?: DoctorScope,
@@ -206,9 +214,19 @@ export const systemSetupHost = (
const result = await installLinuxHopper(linuxHopperHost);
if (result.cleanupFailure !== undefined)
pendingLinuxHopperCleanup.retain(result.cleanupFailure);
return result.status === "installed"
? { status: "installed", launcherPath: result.launcherPath }
: setupInstallFailure(result.reason, result.packageFailure);
if (result.status === "installed")
return { status: "installed", launcherPath: result.launcherPath };
const failure = setupInstallFailure(
result.reason,
result.packageFailure,
"linux",
);
return result.verificationFailure !== undefined
? {
...failure,
remediation: `${result.verificationFailure.message} ${failure.remediation}`,
}
: failure;
}
if (macHopperHost === undefined)
return setupInstallFailure("unsupported_host");
@@ -425,6 +443,23 @@ const configurationCleanupError = (
const errorMessage = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
const closeLinuxHopperInstallHost = async (
host: LinuxHopperInstallHost,
temporaryOwner: LinuxHopperTemporaryCleanupOwner,
): Promise<string | undefined> => {
const reasons: string[] = [];
const temporary = await temporaryOwner.retry(host.cleanup);
if (temporary !== undefined) reasons.push(temporary);
try {
await host.close?.();
} catch (cause: unknown) {
reasons.push(
`Linux Hopper publisher cleanup failed: ${errorMessage(cause)}`,
);
}
return reasons.length === 0 ? undefined : reasons.join("; ");
};
/** Detect supported agents from their config files or stable installation markers. */
export const detectClients = async (
home: string,
src/application/SetupInstallFailure.ts
+23 -2
View File
@@ -1,4 +1,5 @@
import { HOPPER_PUBLISHER } from "../domain/hopperPublisher.js";
import { HOPPER_LINUX_PRIMARY_FINGERPRINT } from "./HopperReleaseFeed.js";
import type { LinuxPackageInstallFailure } from "./LinuxHopper.js";
export type SetupFailureCode =
@@ -50,6 +51,8 @@ export type HopperInstallFailureReason =
| "mount"
| "bundle"
| "publisher_verification_failed"
| "verification_unavailable"
| "cleanup_failed"
| "copy";
const normalizePackageOutput = (output: string): string =>
@@ -99,6 +102,7 @@ const packageManagerRemediation = (
export const setupInstallFailure = (
reason: HopperInstallFailureReason,
packageFailure?: LinuxPackageInstallFailure,
platform: NodeJS.Platform = "darwin",
): SetupHopperInstallFailure => {
switch (reason) {
case "unsupported_host":
@@ -139,7 +143,7 @@ export const setupInstallFailure = (
status: "failed",
code: "unsupported_hopper_build",
remediation:
"The installed Hopper launcher does not match the build supported by this REA release. Unset HOPPER_LAUNCHER_PATH, reinstall through rea setup, or update REA.",
"The installed Hopper launcher does not match the authenticated vendor package. Check HOPPER_LAUNCHER_PATH and reinstall the official Hopper demo through rea setup.",
};
case "cancelled":
return {
@@ -173,7 +177,24 @@ export const setupInstallFailure = (
return {
status: "failed",
code: "publisher_verification_failed",
remediation: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`,
remediation:
platform === "linux"
? `The Hopper package signature did not verify under the pinned primary PGP fingerprint ${HOPPER_LINUX_PRIMARY_FINGERPRINT}; the package manager was not run. Download the official package and retry setup.`
: `The mounted Hopper application is not signed by the publisher REA pins (${HOPPER_PUBLISHER.bundleIdentifier}, Apple team ${HOPPER_PUBLISHER.teamIdentifier}); nothing was copied or launched. Install Hopper yourself and set HOPPER_LAUNCHER_PATH to its launcher, then rerun rea setup.`,
};
case "verification_unavailable":
return {
status: "failed",
code: "publisher_verification_failed",
remediation:
"REA could not authenticate or retain the signed Hopper package. Check vendor access, native package inspection tools and caller-owned cache permissions, then retry setup.",
};
case "cleanup_failed":
return {
status: "failed",
code: "cleanup_failed",
remediation:
"Hopper publisher verification retained temporary cleanup ownership. Retry cleanup before running setup again.",
};
case "copy":
return {
src/application/SetupPlan.ts
+15 -3
View File
@@ -161,6 +161,14 @@ export const planSetupActions = async (input: {
const client = byId.get(id);
return client === undefined ? [] : [client];
});
const hopperCache = input.host.linuxHopperReleaseCache;
if (input.discovery.installHopper && hopperCache?.ok === false)
return {
plannedActions: [],
selectedClients,
clientsToConfigure: [],
blocker: hopperCache.error.message,
};
if (
selectedClients.length > 0 &&
input.host.registrationCommandBlocker !== undefined
@@ -199,6 +207,9 @@ export const planSetupActions = async (input: {
const plannedActions = setupPlan({
platform: input.host.platform,
homeDirectory: input.host.homeDirectory,
...(hopperCache?.ok === true
? { hopperCacheDirectory: hopperCache.value }
: {}),
installHopper: input.discovery.installHopper,
installSkill: input.installSkill,
skillClientIds: input.skillClientIds,
@@ -264,6 +275,7 @@ const plannedClientWrites = (
const setupPlan = (input: {
readonly platform: NodeJS.Platform;
readonly homeDirectory: string;
readonly hopperCacheDirectory?: string;
readonly installHopper: boolean;
readonly installSkill: boolean;
readonly skillClientIds: readonly string[];
@@ -289,7 +301,7 @@ const setupPlan = (input: {
: "system package manager",
detail:
input.platform === "linux"
? "Download, verify, and install Hopper plus its Xvfb demo-session dependencies. For the supported demo build, REA uses a private display and selects Hopper's offered demo mode for each analysis session."
? `Download the current official Hopper demo, verify its PGP publisher and launcher bytes, and retain archive, public-key.asc and release.json under ${input.hopperCacheDirectory ?? `${input.homeDirectory}/.cache/rea/hopper-releases`}/<launcher-sha256> for offline verification. Preserve replaced damaged evidence beside it as <launcher-sha256>.backup-<id>. Install Hopper plus its Xvfb demo-session dependencies. REA uses a private display and selects Hopper's offered demo mode for each analysis session.`
: "Download the official Hopper package, verify it, and install it. Hopper may show its demo or license prompt when first opened.",
external: true,
operation: "install" as const,
@@ -346,9 +358,9 @@ const linuxDisclosure = (family: LinuxPackageFamily) => {
process.getuid?.() === 0,
);
return {
networkOrigins: [disclosure.downloadUrl],
networkOrigins: disclosure.networkOrigins,
commands: disclosure.commands,
integrity: `${String(disclosure.expectedBytes)} bytes · SHA-1 ${disclosure.expectedSha1}`,
integrity: disclosure.integrity,
};
};
src/application/SetupTypes.ts
+2
View File
@@ -1,5 +1,6 @@
import type { DoctorReport, DoctorScope, runDoctor } from "./Doctor.js";
import type { LinuxDistribution } from "./LinuxHopper.js";
import type { HopperReleaseError } from "./HopperReleaseFeed.js";
import type {
SkillDestination,
SkillInstallResult,
@@ -67,6 +68,7 @@ export type ClientConfigurationInspection =
export interface SetupHost {
readonly platform: NodeJS.Platform;
readonly homeDirectory: string;
readonly linuxHopperReleaseCache?: Result<string, HopperReleaseError>;
skillDestinations(clientIds: readonly string[]): readonly SkillDestination[];
readonly registrationCommand: readonly string[];
/** Why client registration cannot proceed from this host. */
src/application/javascript/JavaScriptArtifactPathResolution.test.ts
+46 -5
View File
@@ -2495,20 +2495,61 @@ describe("url-reference directory and empty-segment semantics", () => {
});
});
it.each(["./nested//worker.js", "./nested//../worker.js"])(
"does not collapse empty URL path segments: %s",
(declaredPath) => {
it.each([
["./nested//worker.js", null],
["./nested//../worker.js", "dist/nested/worker.js"],
])(
"keeps empty URL path segments significant: %s",
(declaredPath, resolvedPath) => {
// A dot segment removes the preceding empty segment, never `nested`.
expect(new URL(declaredPath, "file:///root/dist/app.mjs").pathname).toBe(
resolvedPath === null
? "/root/dist/nested//worker.js"
: `/root/${resolvedPath}`,
);
for (const context of ["url-reference", "html-reference"] as const)
expect(
resolve({
declaredPath,
sourcePath: "dist/app.mjs",
context,
files: fileMap([
file("dist/worker.js", "root"),
file("dist/nested/worker.js", "root"),
]),
}),
context,
).toMatchObject({ resolved_path: resolvedPath });
expect(
resolve({
declaredPath,
sourcePath: "dist/app.mjs",
context: "url-reference",
context: "module-specifier",
moduleKind: "import",
files: fileMap([
file("dist/worker.js", "root"),
file("dist/nested/worker.js", "root"),
]),
}),
).toMatchObject({ resolved_path: null });
).toMatchObject({ resolved_path: resolvedPath });
// CommonJS keeps filesystem path semantics.
expect(
resolve({
declaredPath,
sourcePath: "dist/app.mjs",
context: "module-specifier",
moduleKind: "require",
files: fileMap([
file("dist/worker.js", "root"),
file("dist/nested/worker.js", "root"),
]),
}),
).toMatchObject({
resolved_path:
declaredPath === "./nested//worker.js"
? "dist/nested/worker.js"
: "dist/worker.js",
});
},
);
src/application/javascript/JavaScriptArtifactPathResolution.ts
+105 -24
View File
@@ -551,22 +551,24 @@ const contextualCandidate = (
return unresolvedCandidate("rejected", [
"The URL reference path percent-decodes to NUL or to bytes that are not UTF-8.",
]);
// Empty URL segments are significant; canonical filesystem paths cannot
// preserve them, including their interaction with following dot segments.
if (decoded.includes("//"))
return unresolvedCandidate("unavailable", [
"Repeated URL path separators cannot be mapped to an exact canonical artifact file.",
]);
return exactUrlPathCandidate(
const candidate = urlRelativeCandidate(
posix.dirname(input.sourcePath),
decoded,
relativeCandidate(input.sourcePath, decoded),
"URL",
);
return typeof candidate === "string"
? exactUrlPathCandidate(decoded, candidate, "URL")
: candidate;
}
const candidate = relativeCandidate(input.sourcePath, declared);
return context === "module-specifier" && input.moduleKind === "import"
? exactUrlPathCandidate(declared, candidate, "module URL")
: candidate;
if (context === "module-specifier" && input.moduleKind === "import") {
const candidate = urlRelativeCandidate(
posix.dirname(input.sourcePath),
declared,
);
return typeof candidate === "string"
? exactUrlPathCandidate(declared, candidate, "module URL")
: candidate;
}
return relativeCandidate(input.sourcePath, declared);
};
const decodeModuleUrlPath = (
@@ -593,6 +595,78 @@ const relativeCandidate = (sourcePath: string, declared: string): string =>
? declared.slice(1)
: posix.join(posix.dirname(sourcePath), declared);
/**
* A URL path during reference resolution. Above-root `..` markers are kept
* apart from segments, so a later `..` cannot cancel an escape, while empty
* segments stay removable until the final path is formed.
*/
interface UrlPathState {
readonly escapes: number;
readonly segments: readonly string[];
}
const urlDirectoryState = (directory: string): UrlPathState => ({
escapes: 0,
segments: directory === "" || directory === "." ? [] : directory.split("/"),
});
/**
* Apply URL dot segments: `..` removes the previous segment even when it is
* empty, so `assets//../app.js` names `assets/app.js`. A final `.` or `..`
* leaves directory form, as the URL parser does.
*/
const applyUrlReference = (
base: UrlPathState,
reference: string,
): UrlPathState => {
const absolute = reference.startsWith("/");
const segments = absolute ? [] : [...base.segments];
let escapes = absolute ? 0 : base.escapes;
const parts = (absolute ? reference.slice(1) : reference).split("/");
for (const [index, part] of parts.entries()) {
if (part !== "." && part !== "..") {
segments.push(part);
continue;
}
if (part === "..")
if (segments.length > 0) segments.pop();
else escapes += 1;
if (index === parts.length - 1) segments.push("");
}
return { escapes, segments };
};
/** A relative reference replaces the base URL's final path segment. */
const urlBaseDirectory = (base: UrlPathState): UrlPathState => ({
escapes: base.escapes,
segments: base.segments.slice(0, -1),
});
/**
* Form the artifact candidate. A remaining empty segment cannot name a
* canonical artifact file; a trailing one is directory form. Confinement
* rejects a candidate that keeps an above-root marker.
*/
const urlPathCandidate = (path: UrlPathState): string | CandidateResolution => {
const segments =
path.segments.at(-1) === "" ? path.segments.slice(0, -1) : path.segments;
if (segments.includes(""))
return unresolvedCandidate("unavailable", [
"Repeated URL path separators cannot be mapped to an exact canonical artifact file.",
]);
return [
...Array.from({ length: path.escapes }, () => ".."),
...segments,
].join("/");
};
/** Resolve a URL path reference against a canonical directory. */
const urlRelativeCandidate = (
directory: string,
reference: string,
): string | CandidateResolution =>
urlPathCandidate(applyUrlReference(urlDirectoryState(directory), reference));
const bareModuleCandidate = (
input: ResolveArtifactPathInput,
): CandidateResolution => {
@@ -1066,11 +1140,12 @@ const htmlCandidate = (
return unresolvedCandidate("rejected", [
"The HTML reference path percent-decodes to NUL or to bytes that are not UTF-8.",
]);
if (declaredPath.startsWith("/")) return declaredPath.slice(1);
if (declaredPath.startsWith("/"))
return urlRelativeCandidate("", declaredPath);
if (base === undefined || base === null || base === "")
return declaredPath === ""
? input.sourcePath
: posix.join(posix.dirname(input.sourcePath), declaredPath);
: urlRelativeCandidate(posix.dirname(input.sourcePath), declaredPath);
// A local base href is a second untrusted path input; apply the same
// admission rules a declared path gets so it cannot smuggle traversal or
// separator syntax past canonicalization.
@@ -1083,12 +1158,21 @@ const htmlCandidate = (
return unresolvedCandidate("rejected", [
"The document base href path percent-decodes to NUL or to bytes that are not UTF-8.",
]);
const basePath = decodedBase.startsWith("/")
? decodedBase.slice(1)
: posix.join(posix.dirname(input.sourcePath), decodedBase);
if (declaredPath === "")
return exactUrlPathCandidate(decodedBase, basePath, "HTML");
return posix.join(htmlBaseDirectory(decodedBase, basePath), declaredPath);
// Keep the base's empty segments and escapes until the reference is
// applied; only the final path must name a canonical artifact file.
const baseState = applyUrlReference(
urlDirectoryState(posix.dirname(input.sourcePath)),
decodedBase,
);
if (declaredPath === "") {
const basePath = urlPathCandidate(baseState);
return typeof basePath === "string"
? exactUrlPathCandidate(decodedBase, basePath, "HTML")
: basePath;
}
return urlPathCandidate(
applyUrlReference(urlBaseDirectory(baseState), declaredPath),
);
};
const exactUrlPathCandidate = (
@@ -1112,9 +1196,6 @@ const urlPathIsDirectory = (path: string): boolean =>
path === "." ||
path === "..";
const htmlBaseDirectory = (base: string, basePath: string): string =>
urlPathIsDirectory(base) ? basePath : posix.dirname(basePath);
const confineCandidate = (candidate: string): string | CandidateResolution => {
const normalized = normalizeJoinedLogicalPath(candidate);
if (logicalPathEscapesRoot(normalized))
src/application/javascript/JavaScriptHtmlBaseResolution.test.ts
+21
View File
@@ -6,6 +6,7 @@ const paths = [
"assets/app.js",
"renderer/app.js",
"renderer/a/app.js",
"renderer/assets/app.js",
"app.js",
];
const files = new Map<string, JavaScriptArtifactFile>(
@@ -105,6 +106,26 @@ describe("HTML base href dot segments", () => {
},
);
});
describe("HTML base href empty segments", () => {
it("lets the script reference remove an empty segment from the base", () => {
// URL resolution keeps the base's empty segment until `..` removes it.
expect(
new URL("../app.js", new URL("./assets//", documentUrl)).pathname,
).toBe("/renderer/assets/app.js");
expect(resolve("../app.js", "./assets//")).toMatchObject({
resolution_status: "resolved",
resolved_path: "renderer/assets/app.js",
});
});
it("keeps an above-root base escape when the reference also climbs", () => {
expect(resolve("../app.js", "../../")).toMatchObject({
resolution_status: "rejected",
limitations: [
"The resolved candidate escapes the canonical artifact root.",
],
});
});
});
describe("HTML base href path syntax", () => {
it.each(["%2e%2e/secret/", "/a/%2e%2e/", "/a\\b/", "a\0b"])(
"rejects base href %j with the same admission rules a declared path gets",
src/artifacts/ArtifactResourceScope.ts
+9 -1
View File
@@ -8,8 +8,13 @@ import { ArtifactReaderFailure } from "./ArtifactReader.js";
import type { SafeOutputTree } from "./SafeOutputTree.js";
import type { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
/** One artifact resource and its format-specific cleanup owner. */
/** One artifact resource and the capability that owns its cleanup. */
export type ArtifactResourceOwner =
| {
readonly kind: "lease";
readonly resource: string;
readonly lease: { release(): Promise<void> };
}
| {
readonly kind: "javascript-source-analysis";
readonly resource: string;
@@ -156,6 +161,9 @@ export class ArtifactResourceScope {
const cleanup = async (owner: ArtifactResourceOwner): Promise<void> => {
switch (owner.kind) {
case "lease":
await owner.lease.release();
return;
case "javascript-source-analysis":
await owner.analysis.close();
return;
src/artifacts/apple/InterfaceBuilderAnalysis.ts
+27 -6
View File
@@ -7,7 +7,10 @@ import {
isUIKitRuntimeConnectionClass,
} from "../../domain/apple/interfaceBuilderKeyedArchive.js";
import { DirectoryArtifactReader } from "../DirectoryArtifactReader.js";
import { projectPlistValue } from "../../domain/apple/plistValue.js";
import {
projectPlistValue,
unknownDateLimitation,
} from "../../domain/apple/plistValue.js";
import {
omittedPrototypeKeysLimitation,
parseXmlPropertyList,
@@ -58,6 +61,7 @@ export const analyzeInterfaceBuilderBundle = async (input: {
const incompleteHierarchies = new Map<string, number>();
const unresolvedConnections = new Map<string, number>();
const prototypeKeyOmissions = new Map<string, number | null>();
const unknownDates = new Map<string, number>();
let omitted = 0;
let attempted = 0;
let aggregateInputBytes = 0;
@@ -131,16 +135,21 @@ export const analyzeInterfaceBuilderBundle = async (input: {
binaryPrototypeKeyObserved =
decodedReservation.binaryPrototypeKeyObserved;
}
const { value: raw, omittedPrototypeKeys } =
nib === null
? decodePlist(bytes, xmlText, binaryPrototypeKeyObserved)
: { value: nib.raw, omittedPrototypeKeys: 0 };
const {
value: raw,
omittedPrototypeKeys,
unknownDateCount,
} = nib === null
? decodePlist(bytes, xmlText, binaryPrototypeKeyObserved)
: { value: nib.raw, omittedPrototypeKeys: 0, unknownDateCount: 0 };
if (nib !== null && nib.omitted > 0)
incompleteHierarchies.set(entry.path, nib.omitted);
if (nib !== null && nib.unresolvedConnections > 0)
unresolvedConnections.set(entry.path, nib.unresolvedConnections);
if (omittedPrototypeKeys !== 0)
prototypeKeyOmissions.set(entry.path, omittedPrototypeKeys);
if (unknownDateCount !== 0)
unknownDates.set(entry.path, unknownDateCount);
const documentHash = createHash("sha256").update(bytes).digest("hex");
documents.push({
relativePath: entry.path,
@@ -178,6 +187,7 @@ export const analyzeInterfaceBuilderBundle = async (input: {
incompleteHierarchies,
unresolvedConnections,
prototypeKeyOmissions,
unknownDates,
invalid,
omittedArchives,
unreportedOmittedArchives,
@@ -246,6 +256,7 @@ const finalizeHierarchyCoverage = (
incompleteHierarchies: ReadonlyMap<string, number>;
unresolvedConnections: ReadonlyMap<string, number>;
prototypeKeyOmissions: ReadonlyMap<string, number | null>;
unknownDates: ReadonlyMap<string, number>;
invalid: readonly string[];
omittedArchives: readonly { path: string; reason: string }[];
unreportedOmittedArchives: number;
@@ -257,6 +268,7 @@ const finalizeHierarchyCoverage = (
incompleteHierarchies,
unresolvedConnections,
prototypeKeyOmissions,
unknownDates,
invalid,
omittedArchives,
unreportedOmittedArchives,
@@ -342,6 +354,9 @@ const finalizeHierarchyCoverage = (
...[...prototypeKeyOmissions].map(
([path, count]) => `${path}: ${omittedPrototypeKeysLimitation(count)}`,
),
...[...unknownDates].map(
([path, count]) => `${path}: ${unknownDateLimitation(count)}`,
),
...omittedArchives.map(
({ path, reason }) =>
`${path}: omitted because ${reason.replaceAll("_", " ")}.`,
@@ -693,6 +708,7 @@ const decodePlist = (
): {
readonly value: JsonValue;
readonly omittedPrototypeKeys: number | null;
readonly unknownDateCount: number;
} => {
const binary = hasBinaryPlistSignature(bytes);
const { value, omittedPrototypeKeys } = binary
@@ -701,5 +717,10 @@ const decodePlist = (
omittedPrototypeKeys: binaryPrototypeKeyObserved ? null : 0,
}
: parseXmlPropertyList(xmlText ?? decodeXmlPlistText(bytes));
return { value: projectPlistValue(value).value, omittedPrototypeKeys };
const projected = projectPlistValue(value);
return {
value: projected.value,
omittedPrototypeKeys,
unknownDateCount: projected.unknownDateCount,
};
};
src/artifacts/apple/KeyedArchiveReader.ts
+12 -4
View File
@@ -6,7 +6,10 @@ import {
AnalysisInputError,
AnalysisUnsupportedTargetError,
} from "../../domain/analysisErrorCore.js";
import { projectPlistValue } from "../../domain/apple/plistValue.js";
import {
projectPlistValue,
unknownDateLimitation,
} from "../../domain/apple/plistValue.js";
import { createPlistNumberProjection } from "../../domain/apple/plistNumbers.js";
import {
binaryPlistNumberLiterals,
@@ -75,9 +78,11 @@ export const decodeKeyedArchiveBytes = (
: 0,
}
: parseXmlPropertyList(xmlText ?? "");
const { value: plistValue, unknownRealCount } = projectPlistValue(
parsed.value,
);
const {
value: plistValue,
unknownRealCount,
unknownDateCount,
} = projectPlistValue(parsed.value);
const graph = projectKeyedArchive(plistValue, selection);
const numbers = createPlistNumberProjection(
binary
@@ -112,6 +117,9 @@ export const decodeKeyedArchiveBytes = (
: [
`${String(unknownRealCount)} non-finite real value(s) are reported as { "$plist_type": "real", "value": null } because JSON evidence cannot preserve NaN vs infinity.`,
]),
...(unknownDateCount === 0
? []
: [unknownDateLimitation(unknownDateCount)]),
],
};
};
src/cliOutput.ts
+47 -7
View File
@@ -3,13 +3,40 @@ const UNSUPPORTED_OUTPUT_COMBINATION_MESSAGE =
type StructuredOutputFormat = "json" | "jsonl" | "yaml";
const INCUR_OUTPUT_FORMATS = new Set(["toon", "json", "yaml", "md", "jsonl"]);
/** Global output flags that take a value. */
const VALUED_OUTPUT_FLAGS = new Set([
"--format",
"--filter-output",
"--token-limit",
"--token-offset",
]);
/**
* Split `--flag=value` forms of the valued output flags. Incur reads these
* flags only in spaced form, so `--format=json` otherwise reaches the command
* parser and fails as an unknown flag. Command options accept both forms.
*/
export const normalizeCliOutputArguments = (
arguments_: readonly string[],
): string[] =>
arguments_.flatMap((argument) => {
const separator = argument.indexOf("=");
if (separator < 0) return [argument];
const flag = argument.slice(0, separator);
return VALUED_OUTPUT_FLAGS.has(flag)
? [flag, argument.slice(separator + 1)]
: [argument];
});
export type CliOutputArgumentValidation =
| { readonly ok: true }
| {
readonly ok: false;
readonly format: StructuredOutputFormat;
readonly code: "UNSUPPORTED_OUTPUT_COMBINATION" | "INVALID_OUTPUT_FILTER";
readonly code:
| "UNSUPPORTED_OUTPUT_COMBINATION"
| "INVALID_OUTPUT_FILTER"
| "MISSING_OUTPUT_VALUE";
readonly message: string;
};
@@ -17,6 +44,8 @@ interface ParsedCliOutputArguments {
readonly filterOutput: string | undefined;
readonly format: string;
readonly fullOutput: boolean;
/** First valued output flag that Incur skips because it has no value. */
readonly missingValue: string | undefined;
readonly parseError: boolean;
readonly tokenCount: boolean;
readonly tokenWindow: boolean;
@@ -31,6 +60,7 @@ export const parseCliOutputArguments = (
let format: string = "toon";
let filterOutput: string | undefined;
let fullOutput = false;
let missingValue: string | undefined;
let tokenCount = false;
let tokenLimit: number | undefined;
let tokenOffset: number | undefined;
@@ -38,6 +68,7 @@ export const parseCliOutputArguments = (
filterOutput,
format,
fullOutput,
missingValue,
parseError,
tokenCount,
tokenWindow:
@@ -47,8 +78,8 @@ export const parseCliOutputArguments = (
});
for (let index = 0; index < arguments_.length; index += 1) {
const argument = arguments_[index];
// Incur scans builtins across `--` and accepts only spaced value forms.
// Mirror those effective values at the executable output boundary.
// Incur scans builtins across `--` and reads spaced value forms, which
// normalizeCliOutputArguments has produced. Mirror those effective values.
if (argument === "--json") {
format = "json";
continue;
@@ -61,14 +92,14 @@ export const parseCliOutputArguments = (
if (!INCUR_OUTPUT_FORMATS.has(value)) return parsed(true);
format = value;
index += 1;
}
} else missingValue ??= argument;
continue;
}
if (argument === "--filter-output") {
if (arguments_[index + 1]) {
filterOutput = arguments_[index + 1];
index += 1;
}
} else missingValue ??= argument;
continue;
}
if (argument === "--token-limit" || argument === "--token-offset") {
@@ -80,7 +111,7 @@ export const parseCliOutputArguments = (
if (argument === "--token-limit") tokenLimit = numericValue;
else tokenOffset = numericValue;
index += 1;
}
} else missingValue ??= argument;
continue;
}
if (argument === "--full-output") fullOutput = true;
@@ -111,7 +142,7 @@ const hasUnclosedFilterSlice = (expression: string): boolean => {
export const validateCliOutputArguments = (
arguments_: readonly string[],
): CliOutputArgumentValidation => {
const { filterOutput, format, parseError, tokenWindow } =
const { filterOutput, format, missingValue, parseError, tokenWindow } =
parseCliOutputArguments(arguments_);
if (parseError) return { ok: true };
if (filterOutput !== undefined && hasUnclosedFilterSlice(filterOutput))
@@ -132,6 +163,15 @@ export const validateCliOutputArguments = (
code: "UNSUPPORTED_OUTPUT_COMBINATION",
message: UNSUPPORTED_OUTPUT_COMBINATION_MESSAGE,
};
// Incur skips a valued flag without a value and then reports the flag
// itself as unknown. Name the missing value instead.
if (missingValue !== undefined)
return {
ok: false,
format: format === "yaml" || format === "jsonl" ? format : "json",
code: "MISSING_OUTPUT_VALUE",
message: `${missingValue} requires a value, as in ${missingValue} <value> or ${missingValue}=<value>.`,
};
return { ok: true };
};
src/doctorRuntime.test.ts
+21
View File
@@ -42,6 +42,27 @@ it("omits REA_JADX_MCP_JAR from the provider registration environment when it is
expect(environment).not.toHaveProperty("REA_JADX_MCP_JAR");
});
it("carries a validated REA_KUNA_COMMAND into the provider registration environment", async () => {
expect(DOCTOR_PROVIDER_IDS).toContain("kuna");
const root = await createTestTempDirectory("rea-kuna-registration-");
const kuna = join(root, "kuna");
await writeFile(kuna, "#!/bin/sh\nprintf 'kuna 1.833\\n'\n", { mode: 0o700 });
const inspections = await createSystemDoctorHost({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
}).providerInspections?.();
expect(inspections?.find(({ id }) => id === "kuna")).toMatchObject({
available: true,
providerVersion: "1.833",
});
expect(providerRegistrationEnvironment(inspections ?? [])).toMatchObject({
REA_KUNA_COMMAND: kuna,
REA_KUNA_CACHE_DIR: join(root, "cache"),
});
});
const createFakeJdk = async (root: string): Promise<string> => {
const javaHome = join(root, "fake-jdk");
const bin = join(javaHome, "bin");
src/doctorRuntime.ts
+2
View File
@@ -7,6 +7,7 @@ import { fileURLToPath } from "node:url";
import { inspectJadxProvider } from "./android/JadxDoctor.js";
import { inspectSystemGhidraProvider } from "./ghidra/GhidraDoctor.js";
import { inspectIdaRegistration } from "./ida/IdaDoctor.js";
import { inspectKunaProvider } from "./kuna/KunaDoctor.js";
import { readIdaConfiguration } from "./ida/IdaConfiguration.js";
import { parseJebMcpEndpoint } from "./jeb/JebMcpEndpoint.js";
import { selectLinuxPrivateDisplayStrategy } from "./hopper/LinuxPrivateDisplayProbe.js";
@@ -45,6 +46,7 @@ export const createSystemDoctorHost = (
await inspectSystemGhidraProvider(environment),
inspectIdaRegistration(environment.REA_IDA_MCP_CONFIG),
await inspectJadxProvider(environment),
await inspectKunaProvider(environment),
],
linuxDemoRuntimeCheck: () => inspectLinuxPrivateDisplay(environment),
});
src/domain/analysisErrorProjection.ts
+3
View File
@@ -422,6 +422,9 @@ const providerErrorDetails = (
return {
stage: "launch",
provider_state: "unknown",
...(error.launchRejection === undefined
? {}
: { launch_rejection: { ...error.launchRejection } }),
retry_action: error.ownerRunId === undefined ? "retry" : "unknown",
...(error.ownerRunId === undefined
? {}
src/domain/apple/plistValue.test.ts
+14
View File
@@ -2,6 +2,20 @@ import { expect, it } from "vitest";
import { projectPlistValue } from "./plistValue.js";
// Binary decoders build dates from seconds; beyond the Date range they are invalid.
it("projects an unrepresentable date as unknown and counts it", () => {
expect(
projectPlistValue({ far: new Date(Number.NaN), near: new Date(0) }),
).toEqual({
value: {
far: { $plist_type: "date", iso8601: null },
near: { $plist_type: "date", iso8601: "1970-01-01T00:00:00.000Z" },
},
unknownRealCount: 0,
unknownDateCount: 1,
});
});
it("rejects excessive plist nesting without exhausting the stack", () => {
const deep: unknown[] = [];
let cursor = deep;
src/domain/apple/plistValue.ts
+14 -3
View File
@@ -14,8 +14,14 @@ export interface ProjectedPlistValue {
readonly value: JsonValue;
/** Reals decoded as NaN, which the decoder also uses for ±infinity. */
readonly unknownRealCount: number;
/** Dates the decoder could not represent, reported without a timestamp. */
readonly unknownDateCount: number;
}
/** Limitation for results whose plist held dates the decoder could not represent. */
export const unknownDateLimitation = (count: number): string =>
`${String(count)} date value(s) are reported as { "$plist_type": "date", "iso8601": null } because the decoder could not represent them, such as XML dates with years beyond 9999 or negative years, or dates outside the JavaScript date range.`;
/**
* Project a decoded plist into JSON. Data and dates become explicit
* `$plist_type` objects; a non-finite real becomes an explicit unknown
@@ -23,6 +29,7 @@ export interface ProjectedPlistValue {
*/
export const projectPlistValue = (value: unknown): ProjectedPlistValue => {
let unknownRealCount = 0;
let unknownDateCount = 0;
const project = (item: unknown, depth: number): JsonValue => {
if (depth > MAX_PLIST_DEPTH)
throw new RangeError(
@@ -33,8 +40,12 @@ export const projectPlistValue = (value: unknown): ProjectedPlistValue => {
$plist_type: "data",
base64: Buffer.from(item).toString("base64"),
};
if (item instanceof Date)
return { $plist_type: "date", iso8601: item.toISOString() };
if (item instanceof Date) {
if (!Number.isNaN(item.getTime()))
return { $plist_type: "date", iso8601: item.toISOString() };
unknownDateCount += 1;
return { $plist_type: "date", iso8601: null };
}
if (typeof item === "number" && !Number.isFinite(item)) {
unknownRealCount += 1;
return { $plist_type: "real", value: null };
@@ -51,5 +62,5 @@ export const projectPlistValue = (value: unknown): ProjectedPlistValue => {
return z.union([z.string(), z.number(), z.boolean(), z.null()]).parse(item);
};
const projected = project(value, 0);
return { value: projected, unknownRealCount };
return { value: projected, unknownRealCount, unknownDateCount };
};
src/domain/functionComparison.test.ts
+56 -2
View File
@@ -599,11 +599,20 @@ describe("function comparison ordering controls", () => {
makeEvidence("e", rightWithoutNativeApi),
);
expect(comparison.status).toBe("unchanged");
for (const dimension of ["calls", "references", "strings_names"] as const)
for (const dimension of ["calls", "strings_names"] as const)
expect(
comparison.dimensions.find((item) => item.dimension === dimension),
).toMatchObject({ status: "unchanged" });
// Unnamed endpoints in different artifacts have no stable identity.
expect(
comparison.dimensions.find((item) => item.dimension === "references"),
).toMatchObject({
status: "unknown",
limitations: expect.arrayContaining([
expect.stringContaining("Unnamed reference endpoints"),
]),
});
expect(comparison.status).toBe("unknown");
});
});
@@ -1274,3 +1283,48 @@ it.each([
}
},
);
describe("function comparison unnamed reference endpoints", () => {
it("distinguishes unnamed reference targets by address within one artifact", () => {
const unnamedTarget = (target: string) =>
functionDossierSchema.parse({
...dossierWithReference("0x1000", "call"),
outgoing_references: [
{
...dossierWithReference("0x1000", "call").outgoing_references[0],
target_address: target,
target_procedure: null,
},
],
});
const reference = (left: string, right: string, digits = ["a", "a"]) =>
compareFunctions(
observe(digits[0] ?? "a", unnamedTarget(left)),
observe(digits[1] ?? "a", unnamedTarget(right)),
).dimensions.find(({ dimension }) => dimension === "references");
expect(reference("0x1010", "0x1020")).toMatchObject({ status: "changed" });
expect(reference("0x1010", "0x1010")).toMatchObject({
status: "unchanged",
});
// Across artifacts the same address does not identify the same target.
expect(reference("0x1010", "0x1010", ["a", "b"])).toMatchObject({
status: "unknown",
});
});
it("keeps known reference-kind changes beside unnamed endpoints", () => {
const unnamed = (kind: "call" | "data") => {
const edge = dossierWithReference("0x1000", kind).outgoing_references[0];
return functionDossierSchema.parse({
...dossierWithReference("0x1000", kind),
outgoing_references: [{ ...edge, target_procedure: null }],
});
};
expect(
compareFunctions(
observe("a", unnamed("call")),
observe("b", unnamed("data")),
).dimensions.find(({ dimension }) => dimension === "references"),
).toMatchObject({ status: "changed" });
});
});
src/domain/functionComparisonDimensions.ts
+34 -5
View File
@@ -253,13 +253,23 @@ const swiftLiteralCoverage = (
? "absent"
: "complete";
const hasUnnamedEndpoint = (snapshot: FunctionSnapshot): boolean =>
[
...snapshot.dossier.incoming_references,
...snapshot.dossier.outgoing_references,
].some(
(edge) => edge.source_procedure === null || edge.target_procedure === null,
);
const compareReferences = (
left: FunctionSnapshot,
right: FunctionSnapshot,
context: ComparisonContext,
): FunctionDimension => {
const leftProjection = referenceProjection(left);
const rightProjection = referenceProjection(right);
const sameArtifact =
left.subject.digest.sha256 === right.subject.digest.sha256;
const leftProjection = referenceProjection(left, sameArtifact);
const rightProjection = referenceProjection(right, sameArtifact);
if (leftProjection.length === 0 && rightProjection.length === 0)
return compareValues(
"references",
@@ -284,9 +294,28 @@ const compareReferences = (
"Reference endpoints differ independently of provider reference-kind metadata.",
],
});
const leftKinds = referenceKindProjection(left);
const rightKinds = referenceKindProjection(right);
if (leftKinds !== null && rightKinds !== null)
const leftKinds = referenceKindProjection(left, sameArtifact);
const rightKinds = referenceKindProjection(right, sameArtifact);
const kindsKnown = leftKinds !== null && rightKinds !== null;
// Differing known edge kinds are a change whatever the endpoint identity.
if (
kindsKnown &&
canonicalJson(leftKinds, "Function comparison") !==
canonicalJson(rightKinds, "Function comparison")
)
return compareValues("references", leftKinds, rightKinds, context);
// Equal projections with unknown endpoints do not establish equal targets.
if (!sameArtifact && (hasUnnamedEndpoint(left) || hasUnnamedEndpoint(right)))
return unresolvedDimension({
dimension: "references",
links: context.links,
leftCount: leftProjection.length,
rightCount: rightProjection.length,
limitations: [
"Unnamed reference endpoints have no stable identity across different artifacts, so equal named endpoints do not establish equal references.",
],
});
if (kindsKnown)
return compareValues("references", leftKinds, rightKinds, context);
return unresolvedDimension({
dimension: "references",
src/domain/functionComparisonNormalization.ts
+52 -14
View File
@@ -72,22 +72,52 @@ export const normalizeCfg = (
return graph;
};
export const referenceProjection = (snapshot: FunctionSnapshot) =>
sorted([
...snapshot.dossier.incoming_references.map((item) => ({
direction: "in",
source: item.source_procedure?.name ?? null,
target: item.target_procedure?.name ?? null,
type ReferenceEdge = FunctionSnapshot["dossier"]["outgoing_references"][number];
/**
* A named endpoint survives relocation. An unnamed endpoint is identified
* only by its address, which names one location only within one artifact;
* across artifacts it stays unknown (null).
*/
const referenceEndpoint = (
procedure: ReferenceEdge["source_procedure"],
address: string,
sameArtifact: boolean,
): string | { readonly address: string } | null =>
procedure !== null ? procedure.name : sameArtifact ? { address } : null;
export const referenceProjection = (
snapshot: FunctionSnapshot,
sameArtifact: boolean,
) =>
sorted(
[
...snapshot.dossier.incoming_references.map((item) => ({
direction: "in",
item,
})),
...snapshot.dossier.outgoing_references.map((item) => ({
direction: "out",
item,
})),
].map(({ direction, item }) => ({
direction,
source: referenceEndpoint(
item.source_procedure,
item.source_address,
sameArtifact,
),
target: referenceEndpoint(
item.target_procedure,
item.target_address,
sameArtifact,
),
})),
...snapshot.dossier.outgoing_references.map((item) => ({
direction: "out",
source: item.source_procedure?.name ?? null,
target: item.target_procedure?.name ?? null,
})),
]);
);
export const referenceKindProjection = (
snapshot: FunctionSnapshot,
sameArtifact: boolean,
): readonly unknown[] | null => {
const edges = [
...snapshot.dossier.incoming_references.map((edge) => ({
@@ -108,8 +138,16 @@ export const referenceKindProjection = (
);
return {
direction,
source: edge.source_procedure?.name ?? null,
target: edge.target_procedure?.name ?? null,
source: referenceEndpoint(
edge.source_procedure,
edge.source_address,
sameArtifact,
),
target: referenceEndpoint(
edge.target_procedure,
edge.target_address,
sameArtifact,
),
type: edge.kind.type,
flow: edge.kind.flow,
call: edge.kind.call,
src/domain/hopperErrors.ts
+11
View File
@@ -148,23 +148,34 @@ export class HopperProcessError extends HopperError {
}
}
/** Parsed facts about rejecting an application before execution. */
export interface HopperLaunchRejection {
/** The selected application was rejected before it was executed. */
readonly boundary: "publisher_verification";
/** Parsed failure reason supplied by the verification owner. */
readonly reason: string;
}
/** Hopper or the repository bridge could not be started. */
export class HopperStartError extends HopperError {
readonly _tag = "HopperStartError";
override readonly userMessage: string | undefined;
readonly ownerRunId: string | undefined;
readonly launcherFailure: HopperLauncherOutcome | undefined;
readonly launchRejection: HopperLaunchRejection | undefined;
constructor(
options?: ErrorOptions & {
readonly userMessage?: string;
readonly ownerRunId?: string;
readonly launcherFailure?: HopperLauncherOutcome;
readonly launchRejection?: HopperLaunchRejection;
},
) {
super("Hopper application bridge could not be started", options);
this.userMessage = options?.userMessage;
this.ownerRunId = options?.ownerRunId;
this.launcherFailure = options?.launcherFailure;
this.launchRejection = options?.launchRejection;
}
}
src/domain/javascript/javascriptSemanticAnalysis.accessor_writes.test.ts
+108
View File
@@ -0,0 +1,108 @@
import { expect, it } from "vitest";
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
const returned = (source: string) => {
const ir = analyzeJavaScriptSemantics(source);
const callable = ir.callables.find(({ name }) => name === "observe");
return callable?.returnSites[0]?.value;
};
it("keeps members unknown after a write through a local setter", () => {
// The setter runs on assignment and may replace any member of `this`.
expect(
returned(`
export function observe() {
const state = { value: 1, set update(next) { this.value = next; } };
state.update = 2;
return state;
}
`),
).toMatchObject({ status: "unknown" });
});
it("keeps sibling literals when the written key is an ordinary property", () => {
expect(
returned(`
export function observe() {
const state = { value: 1, update: 0 };
state.update = 2;
return state;
}
`),
).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "value",
value: { status: "literal", value: 1 },
}),
]),
});
});
it("keeps nested members unknown after a write through a nested setter", () => {
const value = returned(`
export function observe() {
const state = {
label: "kept",
nested: { value: 1, set update(next) { this.value = next; } },
};
state.nested.update = 2;
return state;
}
`);
expect(value).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "label",
value: { status: "literal", value: "kept" },
}),
expect.objectContaining({
name: "nested",
value: expect.objectContaining({ status: "unknown" }),
}),
]),
});
});
it("keeps sibling literals when an accessor is deleted rather than assigned", () => {
expect(
returned(`
export function observe() {
const state = { value: 1, set update(next) {} };
delete state.update;
return state;
}
`),
).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "value",
value: { status: "literal", value: 1 },
}),
]),
});
});
it("keeps sibling literals when a later data property replaces the setter", () => {
expect(
returned(`
export function observe() {
const state = { value: 1, set update(next) {}, update: 0 };
state.update = 2;
return state;
}
`),
).toMatchObject({
status: "object",
properties: expect.arrayContaining([
expect.objectContaining({
name: "value",
value: { status: "literal", value: 1 },
}),
]),
});
});
src/domain/javascript/javascriptSemanticAnalysis.invalid_templates.test.ts
+38
View File
@@ -0,0 +1,38 @@
import { expect, it } from "vitest";
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
const effects = (call: (value: string) => string, value: string) => {
const ir = analyzeJavaScriptSemantics(`
const { spawn } = require("child_process");
const { EventEmitter } = require("events");
${call(value)}
`);
return {
commands: ir.childProcessSpawns.map(({ command }) => command),
endpoints: ir.requestOperations.map(({ endpoint }) => endpoint),
events: ir.eventOperations.map(({ eventName }) => eventName),
};
};
const calls = (value: string) =>
[
`spawn(${value});`,
`fetch(${value});`,
`new EventEmitter().on(${value}, () => 1);`,
].join("\n");
it("keeps recovered templates without a cooked value dynamic", () => {
// `\\unicode` is an invalid escape: Babel recovers it with `cooked: null`.
expect(effects(calls, "`\\unicode`")).toEqual({
commands: [null],
endpoints: [null],
events: [null],
});
// Valid templates keep their cooked value, including escapes.
expect(effects(calls, "`ready\\n`")).toEqual({
commands: ["ready\n"],
endpoints: ["ready\n"],
events: ["ready\n"],
});
});
src/domain/javascript/javascriptSemanticAnalysis.response_endpoints.test.ts
+59
View File
@@ -0,0 +1,59 @@
import { expect, it } from "vitest";
import { analyzeJavaScriptSemantics } from "./javascriptSemanticAnalysis.js";
const responses = (source: string) =>
analyzeJavaScriptSemantics(source)
.requestOperations.filter(({ kind }) => kind === "response-consumer")
.map(({ method, endpoint, resolution }) => ({
method,
endpoint,
resolution,
}));
it("keeps the endpoint of a uniquely consumed request response", () => {
expect(
responses(`
async function load() {
const a = await fetch("https://example.test/a");
const b = await fetch("https://example.test/b");
return [await a.json(), await b.text()];
}
`),
).toEqual([
{
method: "json",
endpoint: "https://example.test/a",
resolution: "complete",
},
{
method: "text",
endpoint: "https://example.test/b",
resolution: "complete",
},
]);
});
it("leaves an ambiguous response consumer without an endpoint", () => {
expect(
responses(`
async function load(flag) {
let response = await fetch("https://example.test/a");
if (flag) response = await fetch("https://example.test/b");
return response.json();
}
`),
).toEqual([{ method: "json", endpoint: null, resolution: "partial" }]);
});
it("leaves a response replaced by a non-request value without an endpoint", () => {
expect(
responses(`
async function load(cachedResponse) {
let response = await fetch("https://example.test/a");
response = cachedResponse;
return response.json();
}
`),
).toEqual([expect.objectContaining({ method: "json", endpoint: null })]);
});
src/domain/javascript/javascriptSemanticDataEffectHelpers.ts
+7 -4
View File
@@ -924,7 +924,11 @@ export const dataEffectPrimitive = (
return null;
};
/** Read one static string or expression-free template. */
/**
* Read one static string or expression-free template. A template recovered
* with an invalid escape has no cooked value; its raw spelling is not a
* JavaScript string, so it stays dynamic.
*/
export const dataEffectLiteralString = (
node:
| t.Expression
@@ -935,9 +939,8 @@ export const dataEffectLiteralString = (
): string | null => {
if (node === undefined) return null;
const literal = readExactJavaScriptLiteral(node);
if (literal.found && typeof literal.value === "string") return literal.value;
return t.isTemplateLiteral(node) && node.expressions.length === 0
? (node.quasis[0]?.value.raw ?? null)
return literal.found && typeof literal.value === "string"
? literal.value
: null;
};
src/domain/javascript/javascriptSemanticDataEffects.ts
+14 -8
View File
@@ -270,6 +270,18 @@ const collectResponseConsumers = (
? []
: (requestsByBinding.get(binding.bindingId) ?? []);
if (linked.length === 0) return;
// A consumer of exactly one request through its own result binding
// reads that request's response, so it keeps the request's endpoint.
const uniqueRequest =
linked.length === 1 &&
!linked[0]?.operation.projectedResultBindings.some(
({ bindingId }) => bindingId === binding?.bindingId,
)
? linked[0]
: undefined;
// Any other write (assignment, update, loop or destructuring target) can
// replace the response before this read, so its endpoint stays unknown.
const soleWrite = binding?.initializers.length === 1;
output.push({
requestId: `response:${String(node.start ?? -1)}:${String(node.end ?? -1)}`,
kind: "response-consumer",
@@ -287,15 +299,9 @@ const collectResponseConsumers = (
linkedRequestIds: linked
.map(({ operation }) => operation.requestId)
.sort(compareUnicodeCodePoints),
endpoint: null,
endpoint: soleWrite ? (uniqueRequest?.operation.endpoint ?? null) : null,
fields: [],
resolution:
linked.length === 1 &&
!linked[0]?.operation.projectedResultBindings.some(
({ bindingId }) => bindingId === binding?.bindingId,
)
? "complete"
: "partial",
resolution: uniqueRequest === undefined ? "partial" : "complete",
});
});
return output;
src/domain/javascript/javascriptSemanticMemberMutations.ts
+86
View File
@@ -53,6 +53,82 @@ type PropertyPath = JavaScriptSemanticPropertyPath;
type ValueEffect = "write" | "escape";
/** Preserve uncertainty from property writes and references exposed to calls, yielding during traversal and fixpoint iteration. */
/** Setter keys of one object literal; a computed setter key may match any key. */
interface SetterKeys {
readonly names: ReadonlySet<string>;
readonly computed: boolean;
}
// Writes to one literal repeat; scan its properties once, including the
// common result that it defines no setter.
const setterKeysByObject = new WeakMap<t.ObjectExpression, SetterKeys | null>();
const setterKeys = (object: t.ObjectExpression): SetterKeys | null => {
const cached = setterKeysByObject.get(object);
if (cached !== undefined) return cached;
const names = new Set<string>();
let computed = false;
// Follow definition order: a later data property or method with the same
// static name replaces the accessor, so its setter no longer runs. Spreads
// and unknown computed keys may or may not replace it and keep it.
for (const property of object.properties) {
if (t.isSpreadElement(property)) continue;
const name = semanticStaticPropertyKey(property.key, property.computed);
if (t.isObjectMethod(property) && property.kind === "set") {
if (name === null) computed = true;
else names.add(name);
} else if (
name !== null &&
!(t.isObjectMethod(property) && property.kind === "get")
)
names.delete(name);
}
const keys = names.size === 0 && !computed ? null : { names, computed };
setterKeysByObject.set(object, keys);
return keys;
};
/**
* The receiver path whose own object-literal setter an assignment to `path`
* runs, following nested literal members from the binding's initializer.
*/
const localSetterReceiver = (
initializers: readonly {
readonly node: t.Node;
readonly projection: readonly unknown[];
}[],
path: readonly unknown[],
): readonly string[] | undefined => {
const key = path.at(-1);
if (typeof key !== "string") return undefined;
const receiver = path.slice(0, -1);
if (!receiver.every((segment) => typeof segment === "string"))
return undefined;
for (const { node, projection } of initializers) {
if (projection.length !== 0) continue;
let object: t.Node | undefined = unwrapJavaScriptExpression(node).node;
for (const segment of receiver) {
if (!t.isObjectExpression(object)) break;
const member: t.ObjectExpression["properties"][number] | undefined =
object.properties.findLast(
(property) =>
t.isObjectProperty(property) &&
semanticStaticPropertyKey(property.key, property.computed) ===
segment,
);
object = t.isObjectProperty(member)
? unwrapJavaScriptExpression(member.value).node
: undefined;
}
if (!t.isObjectExpression(object)) continue;
const keys = setterKeys(object);
if (keys !== null && (keys.computed || keys.names.has(key)))
return receiver as readonly string[];
}
return undefined;
};
export function* collectSemanticMemberMutationsSteps(
program: t.Program,
state: JavaScriptSemanticAnalysisState,
@@ -377,6 +453,16 @@ export function* collectSemanticMemberMutationsSteps(
current.originAt,
parents,
);
// A setter on the binding's own literal runs on assignment and may
// replace any member of its receiver, so the written key is unknown.
// Deleting the accessor never calls it.
if (
effect === "write" &&
!t.isUnaryExpression(mutation, { operator: "delete" })
) {
const receiver = localSetterReceiver(origins.initializers, path);
if (receiver !== undefined) path = [...receiver, null];
}
// Keep an antichain of exact paths. Widening distinct member keys would
// also invalidate unrelated siblings that never reach this effect.
// This map is local to the file scan. Escaping the binding id keeps a
src/domain/javascript/javascriptSemanticObjectReferences.test.ts
+73
View File
@@ -0,0 +1,73 @@
import * as t from "@babel/types";
import { expect, it } from "vitest";
import { indexJavaScriptBindingsSteps } from "./javascriptSemanticAnalysis.js";
import { semanticObjectPropertyReferences } from "./javascriptSemanticObjectReferences.js";
import { completeSemanticSteps } from "./javascriptSemanticTraversal.js";
import { parseJavaScriptSource } from "./javascriptSourceParser.js";
const objectAndState = (source: string) => {
const parsed = parseJavaScriptSource(source, "object.js");
if (parsed === null) throw new Error("Expected a parsed fixture");
const declaration = parsed.program.body[0];
if (!t.isVariableDeclaration(declaration))
throw new Error("Expected a declaration");
const init = declaration.declarations[0]?.init;
if (!t.isObjectExpression(init)) throw new Error("Expected an object");
return {
object: init,
source,
state: completeSemanticSteps(indexJavaScriptBindingsSteps(parsed.program)),
};
};
const spelled = (
fixture: ReturnType<typeof objectAndState>,
key: string | number | null,
) =>
semanticObjectPropertyReferences(fixture.object, [key], fixture.state).map(
({ node, path }) => [
fixture.source.slice(node.start ?? 0, node.end ?? 0),
JSON.stringify(path),
],
);
it("resolves exact keys with source-order overrides, spreads and accessors", () => {
const fixture = objectAndState(
`const o = { a: 1, ...base, b: 2, a: 3, [dynamic]: 4, __proto__: proto, ...{ b: 5 }, get c() { return 6; }, c: 7 };`,
);
// Expectations match the full source-order scan these exact keys replace.
expect(spelled(fixture, "a")).toEqual([
["{ b: 5 }", '["a"]'],
["4", "[]"],
["3", "[]"],
]);
expect(spelled(fixture, "b")).toEqual([
["{ b: 5 }", '["b"]'],
["4", "[]"],
]);
expect(spelled(fixture, "c")).toEqual([
["7", "[]"],
["4", "[]"],
]);
expect(spelled(fixture, "missing")).toEqual([
["{ b: 5 }", '["missing"]'],
["proto", '["missing"]'],
["4", "[]"],
["base", '["missing"]'],
]);
// A wildcard key still scans every property.
expect(spelled(fixture, null)).toEqual([
["7", "[]"],
["{ b: 5 }", '[{"excludedKeys":["c"]}]'],
["proto", "[null]"],
["4", "[]"],
["3", "[]"],
["2", "[]"],
["base", '[{"excludedKeys":["a","b","c"]}]'],
]);
// Numeric keys use the same exact spelling as string keys.
expect(
spelled(objectAndState(`const o = { 1: "one", 2: "two" };`), 1),
).toEqual([['"one"', "[]"]]);
});
src/domain/javascript/javascriptSemanticObjectReferences.ts
+70 -1
View File
@@ -739,6 +739,75 @@ const getterResultReferences = (
return references;
};
type ObjectLiteralMember = t.ObjectExpression["properties"][number];
/** Property positions grouped by whether an exact key can reach them. */
interface ObjectPropertyIndex {
/** Spreads, computed keys and `__proto__` data entries reach every key. */
readonly keyIndependent: readonly number[];
readonly byName: ReadonlyMap<string, readonly number[]>;
}
// Mutation tracking resolves one key of the same literal per write, so the
// index removes the per-write scan of every property.
const propertyIndexes = new WeakMap<t.ObjectExpression, ObjectPropertyIndex>();
const objectPropertyIndex = (node: t.ObjectExpression): ObjectPropertyIndex => {
const cached = propertyIndexes.get(node);
if (cached !== undefined) return cached;
const keyIndependent: number[] = [];
const byName = new Map<string, number[]>();
node.properties.forEach((property, index) => {
const name = t.isSpreadElement(property)
? null
: semanticStaticPropertyKey(property.key, property.computed);
if (
name === null ||
(t.isObjectProperty(property) &&
!property.computed &&
!property.shorthand &&
name === "__proto__")
) {
keyIndependent.push(index);
return;
}
const positions = byName.get(name);
if (positions === undefined) byName.set(name, [index]);
else positions.push(index);
});
const index = { keyIndependent, byName };
propertyIndexes.set(node, index);
return index;
};
/**
* Properties in reverse source order that can affect `key`. For an exact key,
* another static name only updates bookkeeping for that other name, which
* this key never reads, so only same-name and key-independent entries remain.
*/
const relevantPropertiesInReverse = (
node: t.ObjectExpression,
key: PropertyPath[number] | undefined,
): readonly ObjectLiteralMember[] => {
if (typeof key !== "string" && typeof key !== "number")
return [...node.properties].reverse();
const { keyIndependent, byName } = objectPropertyIndex(node);
const named = byName.get(String(key)) ?? [];
const relevant: ObjectLiteralMember[] = [];
let left = keyIndependent.length - 1;
let right = named.length - 1;
while (left >= 0 || right >= 0) {
const fromLeft =
right < 0 ||
(left >= 0 && (keyIndependent[left] ?? -1) > (named[right] ?? -1));
const position = fromLeft ? keyIndependent[left--] : named[right--];
const property =
position === undefined ? undefined : node.properties[position];
if (property !== undefined) relevant.push(property);
}
return relevant;
};
export const semanticObjectPropertyReferences = (
node: t.ObjectExpression,
path: PropertyPath,
@@ -751,7 +820,7 @@ export const semanticObjectPropertyReferences = (
const accessorNames = new Set<string>();
const getterNames = new Set<string>();
const setterNames = new Set<string>();
for (const property of [...node.properties].reverse()) {
for (const property of relevantPropertiesInReverse(node, key)) {
if (t.isSpreadElement(property)) {
const selected = key ?? null;
if (typeof selected !== "object" && overwritten.has(String(selected)))
src/domain/javascript/javascriptStaticAnalysisCalleeBoundary.test.ts
+3 -2
View File
@@ -27,11 +27,12 @@ describe("static callee boundary matching", () => {
["./bare.html", "call:loadFile"],
["https://optional.example.test", "call:win.loadURL"],
]);
// The document base URL is unknown, so the script path stays unresolved.
expect(
analysis.references
.filter(({ kind }) => kind === "service-worker")
.map(({ specifier }) => specifier),
).toEqual(["./worker.js"]);
.map(({ specifier, expression }) => [specifier, expression]),
).toEqual([[null, 'navigator.serviceWorker.register("./worker.js")']]);
expect(
analysis.endpoints
.filter(({ mechanism }) => mechanism.endsWith("loadURL"))
src/domain/javascript/javascriptStaticAnalysisCalls.ts
+12 -6
View File
@@ -95,19 +95,25 @@ export const inspectCall = (
addReference(context, {
node,
kind: "worker",
// Node string paths use the process cwd or filesystem root, neither of
// which is established by the analyzed source's artifact location.
// A string path is relative to the document base URL in a browser and
// to the process cwd in Node; the analyzed source's artifact location
// establishes neither. Only `new URL(path, import.meta.url)` names a
// location relative to this module.
specifier:
(workerConstructor === "node" ? undefined : first) ??
(workerConstructor === undefined || worker === undefined
workerConstructor === undefined || worker === undefined
? undefined
: workerUrlSpecifier(worker, bindings)),
: workerUrlSpecifier(worker, bindings),
});
// Service worker script URLs are also relative to the document base URL.
if (
name === "serviceWorker.register" ||
name.endsWith(".serviceWorker.register")
)
addReference(context, { node, kind: "service-worker", specifier: first });
addReference(context, {
node,
kind: "service-worker",
specifier: undefined,
});
inspectEndpointCall(node, name, first, context);
inspectStorageCall(node, name, first, context);
if (first === "audio" || first === "video")
src/domain/managed/managedApplicationGraphNodes.ts
+16 -7
View File
@@ -24,6 +24,13 @@ type PinvokeImport = ManagedNativeBoundaryInspection["pinvoke_imports"][number];
type NativeImplementation =
ManagedNativeBoundaryInspection["native_implementations"][number];
/**
* Display label for a metadata name. Metadata may hold an empty name; the
* graph then has no label, while the node's properties keep the source value.
*/
const metadataLabel = (name: string): string | null =>
name === "" ? null : name;
/** Options for a managed contains relationship edge. */
export interface ContainsEdgeOptions {
readonly kind: string;
@@ -77,7 +84,7 @@ export const addArtifactIdentityNodes = (
),
observations: [
{
label: assemblyFacts.name,
label: metadataLabel(assemblyFacts.name),
properties: {
name: assemblyFacts.name,
version: assemblyFacts.version,
@@ -111,7 +118,7 @@ export const addArtifactIdentityNodes = (
),
observations: [
{
label: module.name,
label: metadataLabel(module.name),
properties: {
name: module.name,
mvid: module.mvid,
@@ -237,7 +244,7 @@ const typeNode = (
),
observations: [
{
label: type.full_name,
label: metadataLabel(type.full_name),
properties: {
token: type.token,
namespace: type.namespace,
@@ -266,10 +273,11 @@ const methodNode = (
),
observations: [
{
label:
label: metadataLabel(
method.declaring_type === null
? method.name
: `${method.declaring_type}.${method.name}`,
),
properties: {
token: method.token,
declaring_type_token: method.declaring_type_token,
@@ -302,10 +310,11 @@ const fieldNode = (
),
observations: [
{
label:
label: metadataLabel(
field.declaring_type === null
? field.name
: `${field.declaring_type}.${field.name}`,
),
properties: {
token: field.token,
declaring_type_token: field.declaring_type_token,
@@ -410,7 +419,7 @@ const pinvokeNode = (
),
observations: [
{
label: pinvoke.import_name,
label: metadataLabel(pinvoke.import_name),
properties: {
token: pinvoke.token,
member_token: pinvoke.member_token,
@@ -441,7 +450,7 @@ const nativeImplementationNode = (
),
observations: [
{
label: implementation.name,
label: metadataLabel(implementation.name),
properties: {
token: implementation.token,
name: implementation.name,
src/domain/referenceSourceRequireBindings.ts
+87
View File
@@ -1,5 +1,11 @@
import * as t from "@babel/types";
import {
semanticStaticPropertyKey,
semanticWriteSiteTarget,
semanticWriteTargets,
unwrapJavaScriptExpression,
} from "./javascript/javascriptAstValues.js";
import { analyzeParsedJavaScriptReferences } from "./javascript/javascriptSemanticAnalysis.js";
import {
semanticBindingId,
@@ -49,6 +55,22 @@ export const unshadowedReferenceSourceRequireCalls = (
(reference.resolution === "unbound" ||
reference.bindingId === programRequireBinding),
);
const isLoaderReference = (identifier: t.Identifier): boolean => {
const start = identifier.loc?.start;
const reference =
start === undefined
? undefined
: references.get(`${start.line}:${start.column}`);
return (
(allowsUnboundRequire && reference?.resolution === "unbound") ||
(reference?.resolution === "resolved" &&
reference.bindingId === programRequireBinding)
);
};
const loaderMembersIntact = requireLoaderMembersIntact(
file.program,
isLoaderReference,
);
const calls = new Set<t.CallExpression>();
traverseJavaScriptAst(file.program, {
@@ -64,6 +86,9 @@ export const unshadowedReferenceSourceRequireCalls = (
)
)
return;
// `require.resolve` and `require.main` denote Node's members only
// while no visible write or escape can have replaced them.
if (!t.isIdentifier(node.callee) && !loaderMembersIntact) return;
const reference = references.get(`${start.line}:${start.column}`);
if (
(allowsUnboundRequire &&
@@ -80,6 +105,68 @@ export const unshadowedReferenceSourceRequireCalls = (
return calls;
};
/**
* Whether the loader's `resolve` and `main` members keep Node's identity. A
* write to either member or to a computed member, or `require` used as a
* value other than a call or member read (where a mutator or alias can
* replace members), leaves that identity unknown.
*/
const requireLoaderMembersIntact = (
program: t.Program,
isLoaderReference: (identifier: t.Identifier) => boolean,
): boolean => {
let intact = true;
const isLoaderMemberTarget = (target: t.Node): boolean => {
if (!t.isMemberExpression(target) && !t.isOptionalMemberExpression(target))
return false;
const object = unwrapJavaScriptExpression(target.object).node;
if (!t.isIdentifier(object, { name: "require" })) return false;
if (!isLoaderReference(object)) return false;
const key = semanticStaticPropertyKey(target.property, target.computed);
return key === null || key === "resolve" || key === "main";
};
traverseJavaScriptAst(program, {
enter: (node, parent, readAncestors) => {
if (!intact) return;
const site = semanticWriteSiteTarget(node);
if (site !== null) {
for (const target of semanticWriteTargets(site))
if (isLoaderMemberTarget(target)) intact = false;
return;
}
if (
!t.isIdentifier(node, { name: "require" }) ||
parent === null ||
!isLoaderReference(node)
)
return;
// Inert parentheses and TypeScript wrappers do not change how the
// loader value is used.
let used: t.Node = node;
let user: t.Node | undefined = parent;
const read = readAncestors();
const ancestors = read.at(-1) === parent ? read : [...read, parent];
for (let index = ancestors.length - 1; index >= 0; index -= 1) {
user = ancestors[index];
if (
user === undefined ||
unwrapJavaScriptExpression(user).node !== node
)
break;
used = user;
}
const usedAsLoader =
((t.isCallExpression(user) || t.isOptionalCallExpression(user)) &&
user.callee === used) ||
((t.isMemberExpression(user) || t.isOptionalMemberExpression(user)) &&
user.object === used) ||
t.isUnaryExpression(user, { operator: "typeof" });
if (!usedAsLoader) intact = false;
},
});
return intact;
};
const contains = (container: t.Node, node: t.Node): boolean =>
typeof container.start === "number" &&
typeof container.end === "number" &&
src/flutter/DartElf.test.ts
+101
View File
@@ -3,6 +3,7 @@ import { describe, expect, it } from "vitest";
import { readElfSymbols, readSnapshotHeader } from "./DartElf.js";
import {
REFERENCE_HASH,
buildElf32Image,
buildElf64Image,
buildStrippedElf64Image,
snapshotSection,
@@ -299,6 +300,106 @@ describe("ELF symbol-table ownership", () => {
});
});
/** Section-header field positions for each ELF class. */
const ELF_CLASSES = {
elf64: {
build: buildElf64Image,
word: {
read: (image: Buffer, at: number) => Number(image.readBigUInt64LE(at)),
write: (image: Buffer, value: number, at: number) =>
image.writeBigUInt64LE(BigInt(value), at),
},
shoff: 0x28,
shent: 64,
flags: 8,
address: 0x10,
offset: 0x18,
},
elf32: {
build: buildElf32Image,
word: {
read: (image: Buffer, at: number) => image.readUInt32LE(at),
write: (image: Buffer, value: number, at: number) =>
image.writeUInt32LE(value, at),
},
shoff: 0x20,
shent: 40,
flags: 8,
address: 0x0c,
offset: 0x10,
},
} as const;
describe.each(Object.entries(ELF_CLASSES))(
"allocated %s symbol tables",
(_, layout) => {
const { word } = layout;
/** Snapshot sections, with dynamic tables allocated at their offsets. */
const allocatedTables = (): {
image: Buffer;
headers: number[];
tables: number[];
} => {
const elf = layout.build([
{ name: "_kDartVmSnapshotData", value: 0x400, size: 52 },
]);
const section = snapshotSection(GALLERY_HASH);
const image = Buffer.concat([
elf,
Buffer.alloc(0x400 - elf.length),
section,
]);
const shoff = word.read(image, layout.shoff);
const headers = [shoff + layout.shent, shoff + 2 * layout.shent];
const tables = headers.map((header) => {
const offset = word.read(image, header + layout.offset);
word.write(image, 2, header + layout.flags);
word.write(image, offset, header + layout.address);
return offset;
});
return { image, headers, tables };
};
it("accepts tables whose mapping matches their sections", async () => {
const { image } = allocatedTables();
expect(await readElfSymbols(image, ["_kDartVmSnapshotData"])).toEqual({
symbols: [{ name: "_kDartVmSnapshotData", offset: 0x400, size: 52 }],
failures: [],
});
});
it.each([
[0, "Dynamic symbol table has conflicting PT_LOAD mappings"],
[1, "Dynamic symbol string table has conflicting PT_LOAD mappings"],
])(
"refuses table %i under a conflicting PT_LOAD interpretation",
async (table, reason) => {
const { image, tables } = allocatedTables();
const conflict = appendElfRanges(image, "program", [
{ type: 1, address: tables[table]!, offset: 0x300, size: 4 },
]);
expect(
await readElfSymbols(conflict, ["_kDartVmSnapshotData"]),
).toEqual({ symbols: [], failures: [{ reason }] });
},
);
it("refuses a table whose section offset disagrees with its mapping", async () => {
const { image, headers } = allocatedTables();
word.write(image, 0x300, headers[0]! + layout.address);
expect(await readElfSymbols(image, ["_kDartVmSnapshotData"])).toEqual({
symbols: [],
failures: [
{
reason:
"Dynamic symbol table section offset disagrees with its PT_LOAD mapping",
},
],
});
});
},
);
describe("readSnapshotHeader", () => {
it("reads a synthesized data header", async () => {
const header = readSnapshotHeader(snapshotSection(GALLERY_HASH), 0);
src/flutter/DartElf.ts
+33
View File
@@ -59,6 +59,7 @@ export interface ElfParseFailure {
}
const SHT_DYNSYM = 11;
const SHF_ALLOC = 2;
const PT_LOAD = 1;
const PT_DYNAMIC = 2;
/** Position-independent executable images carry PT_DYNAMIC. */
@@ -225,6 +226,28 @@ export const readElfSymbols = async (
stringOffset: number;
stringSize: number;
}[] = [];
/**
* The loader reads an allocated section through PT_LOAD, so its section
* offset must be the single file interpretation of its address range.
*/
const sectionMappingFailure = (
header: number,
offset: number,
size: number,
): string | undefined => {
const flags = is64 ? reader.u64(header + 8) : reader.u32(header + 8);
const address = is64
? reader.u64(header + 0x10)
: reader.u32(header + 0x0c);
if (flags === null || address === null)
return "has an unrepresentable section header";
if ((flags & SHF_ALLOC) === 0) return undefined;
const mapped = toOffset(address, size);
if (!mapped.ok) return mapped.error.reason;
return mapped.value === offset
? undefined
: "section offset disagrees with its PT_LOAD mapping";
};
if (!dynamicFallback) {
for (let index = 0; index < e_shnum; index += 1) {
if (index % 4096 === 0) {
@@ -269,6 +292,16 @@ export const readElfSymbols = async (
return failure(
"Dynamic symbol strings are out of bounds or not a string table",
);
const symbolMapping = sectionMappingFailure(header, offset, size);
if (symbolMapping !== undefined)
return failure(`Dynamic symbol table ${symbolMapping}`);
const stringMapping = sectionMappingFailure(
stringHeader,
stringOffset,
stringSize,
);
if (stringMapping !== undefined)
return failure(`Dynamic symbol string table ${stringMapping}`);
tables.push({
offset,
count: size / entrySize,
src/flutter/FlutterBuild.fixture.ts
+57
View File
@@ -115,6 +115,63 @@ export const buildElf64Image = (
return image;
};
/** The ELF32 (ARM) counterpart of {@link buildElf64Image}, with the same layout roles. */
export const buildElf32Image = (
symbols: ReadonlyArray<{ name: string; value: number; size: number }>,
): Buffer => {
const ELF_HEADER = 52;
const PHENT = 32;
const SHENT = 40;
const SYMENT = 16;
const phoff = ELF_HEADER;
const shoff = phoff + PHENT;
const dynsymOff = shoff + 3 * SHENT + 16;
const dynstrOff = dynsymOff + symbols.length * SYMENT;
const image = Buffer.alloc(dynstrOff + 256);
image.writeUInt32LE(0x464c457f, 0);
image[4] = 1; // ELFCLASS32
image[5] = 1; // little-endian
image[6] = 1;
image.writeUInt16LE(3, 0x10);
image.writeUInt16LE(0x0028, 0x12); // e_machine ARM
image.writeUInt32LE(1, 0x14);
image.writeUInt32LE(phoff, 0x1c);
image.writeUInt32LE(shoff, 0x20);
image.writeUInt16LE(ELF_HEADER, 0x28);
image.writeUInt16LE(PHENT, 0x2a);
image.writeUInt16LE(1, 0x2c);
image.writeUInt16LE(SHENT, 0x2e);
image.writeUInt16LE(3, 0x30); // null, dynsym, dynstr
// One identity PT_LOAD covering the materialized payload.
image.writeUInt32LE(1, phoff);
image.writeUInt32LE(
Math.max(image.length, ...symbols.map(({ value, size }) => value + size)),
phoff + 0x10,
);
const dyn = shoff + SHENT;
image.writeUInt32LE(11, dyn + 4); // SHT_DYNSYM
image.writeUInt32LE(dynsymOff, dyn + 0x10);
image.writeUInt32LE(symbols.length * SYMENT, dyn + 0x14);
image.writeUInt32LE(2, dyn + 0x18); // sh_link -> strtab
image.writeUInt32LE(SYMENT, dyn + 0x24);
const str = dyn + SHENT;
image.writeUInt32LE(3, str + 4); // SHT_STRTAB
image.writeUInt32LE(dynstrOff, str + 0x10);
image.writeUInt32LE(256, str + 0x14);
let strCursor = dynstrOff + 1;
symbols.forEach((symbol, index) => {
image.write(symbol.name, strCursor, "latin1");
const base = dynsymOff + index * SYMENT;
image.writeUInt32LE(strCursor - dynstrOff, base);
image.writeUInt32LE(symbol.value, base + 4);
image.writeUInt32LE(symbol.size, base + 8);
image[base + 0xc] = 0x10; // GLOBAL
image.writeUInt16LE(1, base + 0xe); // Defined export.
strCursor += symbol.name.length + 1;
});
return image;
};
/**
* Build a section-stripped ELF64 image (e_shnum 0, ET_DYN) whose exported
* symbols are reachable only through the PT_DYNAMIC tag-value pairs, the
src/ghidra/GhidraDoctor.test.ts
-1
View File
@@ -69,7 +69,6 @@ const doctorHost = (
validTarget: () => Promise.resolve(true),
executable: (path) =>
Promise.resolve(hopperAvailable && path.includes("Hopper")),
supportedLinuxHopper: () => Promise.resolve(true),
linuxDemoRuntimeCheck: () =>
Promise.resolve({
name: "hopper-demo-runtime",
src/hopper/BridgeLauncher.ts
+65 -3
View File
@@ -9,6 +9,8 @@ import {
HopperStartError,
} from "../domain/hopperErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { ArtifactReaderFailure } from "../artifacts/ArtifactReader.js";
import { cleanupOwnedProcessGroup } from "../process/ProcessOwnership.js";
import { execFileOutput } from "../process/ExecFileOutput.js";
import {
@@ -30,10 +32,12 @@ import {
import {
acquireHopperTargetLease,
acquireLinuxHopperApplicationLease,
} from "./HopperTargetLease.js";
import type { HopperTargetLease } from "./HopperTargetLease.js";
} from "../process/HopperLease.js";
import type { HopperTargetLease } from "../process/HopperLease.js";
import { HOPPER_SHUTDOWN_TIMEOUT_MS } from "./HopperSessionValues.js";
import { verifyMacHopperNativeHelper } from "./HopperMacPublisher.js";
import { inspectLinuxHopperLauncher } from "../application/LinuxHopper.js";
import { retryLinuxHopperPublisherCleanup } from "../application/LinuxHopperPublisher.js";
const execFileAsync = promisify(execFile);
const HOPPER_BACKGROUND_STARTUP_MS = 5_000;
@@ -73,6 +77,8 @@ export type BridgeLaunch = {
/** Application-owned capability that starts the in-Hopper bridge. */
export interface BridgeLauncher {
/** Retry owned pre-launch resources even when no application was started. */
cleanup?(): Promise<Result<null, ProviderCleanupError>>;
launch(
session: BridgeSession,
options?: { readonly signal?: AbortSignal },
@@ -110,6 +116,8 @@ export type HopperApplicationLauncherOptions =
export interface HopperApplicationLauncherDependencies {
readonly verifyNativeHelper?: typeof verifyMacHopperNativeHelper;
readonly verifyLinuxLauncher?: typeof inspectLinuxHopperLauncher;
readonly retryPublisherCleanup?: typeof retryLinuxHopperPublisherCleanup;
readonly platform?: NodeJS.Platform;
readonly acquireTargetLease?: typeof acquireHopperTargetLease;
readonly acquireApplicationLease?: typeof acquireLinuxHopperApplicationLease;
@@ -140,6 +148,32 @@ export class HopperApplicationLauncher implements BridgeLauncher {
readonly dependencies: HopperApplicationLauncherDependencies = {},
) {}
async cleanup(): Promise<Result<null, ProviderCleanupError>> {
if (!usesLinuxDemo(this.options)) return ok(null);
try {
await (
this.dependencies.retryPublisherCleanup ??
retryLinuxHopperPublisherCleanup
)();
return ok(null);
} catch (cause: unknown) {
const observation = ArtifactReaderFailure.cleanupObservation(
cause,
"hopper-publisher-verification",
);
return err(
new ProviderCleanupError(
"hopper",
observation.resources,
{
reason: observation.reason,
},
{ cause },
),
);
}
}
async launch(
session: BridgeSession,
options: { readonly signal?: AbortSignal } = {},
@@ -218,6 +252,26 @@ export class HopperApplicationLauncher implements BridgeLauncher {
if (signal?.aborted === true) return err(new HopperCancelledError());
const display = await this.#privateDisplay(signal);
if (!display.ok) return err(display.error);
let expectedLinuxSha256: string | undefined;
if (usesLinuxDemo(this.options)) {
const inspected = await (
this.dependencies.verifyLinuxLauncher ?? inspectLinuxHopperLauncher
)(this.options.launcherPath, this.options.environment, signal);
if (!inspected.ok)
return inspected.error.reason === "cancelled"
? err(new HopperCancelledError())
: err(
new HopperStartError({
userMessage: inspected.error.message,
cause: inspected.error,
launchRejection: {
boundary: "publisher_verification",
reason: inspected.error.reason,
},
}),
);
expectedLinuxSha256 = inspected.value.sha256;
}
const nativeHelperVerified =
(this.dependencies.platform ?? process.platform) === "darwin" &&
this.options.launchMode === "native" &&
@@ -236,6 +290,7 @@ export class HopperApplicationLauncher implements BridgeLauncher {
session,
argumentsForTarget,
display.strategy,
expectedLinuxSha256,
);
const started = await launchHopperProcess({
options: this.options,
@@ -449,6 +504,7 @@ export const linuxDemoLaunch = (
session: BridgeSession,
hopperArgs: readonly string[],
strategy: LinuxPrivateDisplayRunnableStrategy | undefined,
expectedSha256?: string,
):
| {
readonly command: string;
@@ -458,6 +514,10 @@ export const linuxDemoLaunch = (
| undefined => {
if (!usesLinuxDemo(options) || options.demoHelperPath === undefined)
return undefined;
if (expectedSha256 === undefined || !/^[a-f0-9]{64}$/u.test(expectedSha256))
throw new TypeError(
"Linux Hopper launch requires an authenticated launcher SHA-256",
);
const helperArguments = [
options.demoHelperPath,
"--owner-stdin",
@@ -466,6 +526,8 @@ export const linuxDemoLaunch = (
...(strategy === "user-mount-namespace" ? ["--mount-private-x11"] : []),
"--hopper",
options.launcherPath,
"--expected-launcher-sha256",
expectedSha256,
"--socket",
session.socketPath,
"--",
@@ -501,7 +563,7 @@ const hopperLaunchFailure = (
? new HopperCancelledError()
: new HopperStartError({ cause });
/** Select the version-pinned adapter from the caller's explicit launch mode. */
/** Select the publisher-verified demo adapter from the caller's explicit launch mode. */
export const usesLinuxDemo = (
options: HopperApplicationLauncherOptions,
): options is HopperApplicationLauncherOptions & {
src/hopper/HopperClient.ts
+27 -3
View File
@@ -29,6 +29,7 @@ import {
type ProviderOperationRequest,
} from "../domain/providerOperationHealth.js";
import { err, ok, type Result } from "../domain/result.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import type { JsonValue } from "../domain/jsonValue.js";
import type { ProgressReporter } from "../application/ProgressReporter.js";
import { silentLogger } from "../logger.js";
@@ -328,7 +329,8 @@ export class HopperClient {
if (
this.#socket !== undefined ||
this.#resources.launch !== undefined ||
this.#resources.runtimeRoot !== undefined
this.#resources.runtimeRoot !== undefined ||
this.#startupFailureResult?.cleanupIncomplete === true
) {
return err(
this.#startupFailureResult ??
@@ -563,7 +565,9 @@ export class HopperClient {
this.#releaseTransport(),
);
await this.#reconnectForCleanup();
return await this.#cleanup(options);
const closed = await this.#cleanup(options);
if (closed.ok) this.#startupFailureResult = undefined;
return closed;
} finally {
if (this.#startPromise === starting) this.#startPromise = undefined;
if (this.#startupController === controller) {
@@ -641,7 +645,7 @@ export class HopperClient {
this.#closing = true;
this.#ready = false;
try {
return await cleanupHopperSession({
const closed = await cleanupHopperSession({
socket: this.#socket,
resources: this.#resources,
activeRequest: this.#requests.activity(),
@@ -653,6 +657,26 @@ export class HopperClient {
retainDocument: options.retainDocument === true,
releaseTransport: () => this.#releaseTransport(),
});
const launcherClosed = await this.#options.launcher
.cleanup?.()
.catch((cause: unknown) =>
err(
new ProviderCleanupError(
"hopper",
["hopper-launcher"],
{
reason: cause instanceof Error ? cause.message : String(cause),
},
{ cause },
),
),
);
if (launcherClosed === undefined || launcherClosed.ok) return closed;
return closed.ok
? launcherClosed
: err(
analysisErrorWithCleanupFailure(closed.error, launcherClosed.error),
);
} finally {
if (this.#socket?.destroyed === true) this.#socket = undefined;
src/hopper/HopperTargetLease.test.ts
+1 -1
View File
@@ -20,7 +20,7 @@ import { afterEach, describe, expect, it } from "vitest";
import {
acquireHopperTargetLease,
acquireLinuxHopperApplicationLease,
} from "./HopperTargetLease.js";
} from "../process/HopperLease.js";
const directories: string[] = [];
src/javascript/recovery/RecoveryFiles.test.ts
+55
View File
@@ -0,0 +1,55 @@
import { execFile } from "node:child_process";
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
import { expect, it, vi } from "vitest";
import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
import { OwnedFileHandle } from "../../filesystem/OwnedFileHandle.js";
import {
openRegularFile,
RegularFileAdmissionFailure,
} from "../../filesystem/RegularFile.js";
import { readWithoutFifoWriter } from "../../../tests/fixtures/fifoInput.js";
import { createTestTempDirectory } from "../../../tests/fixtures/temporaryDirectory.js";
import { readRecoveryFile } from "./RecoveryFiles.js";
vi.mock("../../filesystem/RegularFile.js", async (importOriginal) => {
const actual =
await importOriginal<typeof import("../../filesystem/RegularFile.js")>();
return { ...actual, openRegularFile: vi.fn(actual.openRegularFile) };
});
it.skipIf(process.platform === "win32")(
"rejects a named pipe without waiting for a writer",
async () => {
const root = await createTestTempDirectory("rea-recovery-fifo-");
const regular = join(root, "regular.js");
await writeFile(regular, "module.exports = 1;\n");
await expect(readRecoveryFile(regular, 1024)).resolves.toHaveLength(20);
const fifoPath = join(root, "sample.js");
await promisify(execFile)("mkfifo", [fifoPath]);
const outcome = await readWithoutFifoWriter(fifoPath, () =>
readRecoveryFile(fifoPath, 1024).catch((cause: unknown) => cause),
);
if (outcome.state !== "completed")
throw new Error("Recovery input read waited for a FIFO writer");
expect(outcome.result).toBeInstanceOf(AnalysisOutputError);
},
);
it("keeps a failed admission close owned when cancellation is also requested", async () => {
const owner = new OwnedFileHandle({ fd: -1 } as never);
const failure = new RegularFileAdmissionFailure(
"/selected.js",
new Error("cancelled during admission"),
new Error("close failed"),
owner,
);
vi.mocked(openRegularFile).mockRejectedValueOnce(failure);
const controller = new AbortController();
controller.abort();
await expect(
readRecoveryFile("/selected.js", 1024, controller.signal),
).rejects.toBe(failure);
});
src/javascript/recovery/RecoveryFiles.ts
+23 -4
View File
@@ -1,6 +1,5 @@
import { createHash } from "node:crypto";
import { constants } from "node:fs";
import { lstat, mkdir, open, opendir, writeFile } from "node:fs/promises";
import { lstat, mkdir, opendir, writeFile } from "node:fs/promises";
import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path";
import { TextDecoder } from "node:util";
import { normalizeArtifactPath } from "../../artifacts/ArtifactPaths.js";
@@ -10,6 +9,11 @@ import {
AnalysisOutputError,
AnalysisTimeoutError,
} from "../../domain/analysisErrorCore.js";
import {
NonRegularFileReadError,
openRegularFile,
RegularFileAdmissionFailure,
} from "../../filesystem/RegularFile.js";
import { RECOVERY_LIMITS } from "./WakaruRelease.js";
const OPERATION = "recover_javascript_sources";
@@ -25,11 +29,26 @@ const scanRecoveryFile = async (
consume: (chunk: Buffer) => void,
signal?: AbortSignal,
) => {
const handle = await open(path, constants.O_RDONLY | constants.O_NOFOLLOW);
// A FIFO without a writer must be rejected before it can block the open.
const handle = await openRegularFile(path, {
symlinks: "reject",
signal,
}).catch((cause: unknown) => {
// A descriptor whose close failed stays owned by its error, even when
// cancellation was also requested.
if (cause instanceof RegularFileAdmissionFailure) throw cause;
checkRecoveryCancellation(signal);
if (cause instanceof NonRegularFileReadError)
throw new AnalysisOutputError(
OPERATION,
`Expected a regular file no larger than ${String(maximum)} bytes: ${path}`,
);
throw cause;
});
const hash = createHash("sha256");
try {
const before = await handle.stat();
if (!before.isFile() || before.size > maximum)
if (before.size > maximum)
throw new AnalysisOutputError(
OPERATION,
`Expected a regular file no larger than ${String(maximum)} bytes: ${path}`,
src/kuna/KunaConfiguration.ts
+144
View File
@@ -0,0 +1,144 @@
import { isAbsolute, join } from "node:path";
import { homeDirectoryFromEnvironment } from "../config/homeDirectory.js";
import { providerStartupTimeoutSchema } from "../config/providerStartupTimeout.js";
import { err, ok, type Result } from "../domain/result.js";
import {
KUNA_COMMAND_REMEDIATION,
KUNA_UNSUPPORTED_HOST_REMEDIATION,
} from "./KunaRelease.js";
/** Kuna caps its own decode lanes and worker pool at 32. */
const MAX_KUNA_JOBS = 32;
/**
* A 1.4 MB sqlite3 shell exported serially in 66-108 s on Apple M-series and
* Linux hosts; thirty minutes admits images an order of magnitude larger
* without an unbounded run. Larger images need a raised deadline or more jobs.
*/
const DEFAULT_PROJECT_TIMEOUT_MS = 30 * 60 * 1000;
/**
* A serial (`--jobs 1`) export is Kuna's reference answer: it decompiles
* callees before callers, which recovers prototypes that parallel workers
* cannot. Measured on coreutils sort, 40-60% of functions differ otherwise.
*/
const DEFAULT_JOBS = 1;
/** Measured sqlite3 exports occupy ~25 MiB; four GiB retains many projects. */
const DEFAULT_CACHE_MAX_BYTES = 4 * 1024 ** 3;
/** One project needs its .c, .h, .asm, inventory and manifest; refuse smaller caps. */
const MIN_CACHE_MAX_BYTES = 1024 ** 2;
/** Caller-selected Kuna installation and REA-owned project cache policy. */
export interface KunaConfiguration {
/** Caller-selected executable path, preserved exactly as configured. */
readonly command: string;
/** REA-owned cache root; Kuna's own spec cache lives beneath it too. */
readonly cacheRoot: string;
readonly cacheMaxBytes: number;
readonly projectTimeoutMs: number;
readonly jobs: number;
}
/** Why the configured environment cannot select a Kuna installation. */
export interface KunaConfigurationFailure {
readonly code: "not_configured" | "open_options_invalid" | "unsupported_host";
readonly reason: string;
readonly setting: string | null;
}
const failure = (
code: KunaConfigurationFailure["code"],
reason: string,
setting: string | null = null,
): Result<never, KunaConfigurationFailure> => err({ code, reason, setting });
const positiveInteger = (value: string): number | undefined =>
/^[1-9]\d*$/u.test(value) && Number.isSafeInteger(Number(value))
? Number(value)
: undefined;
/** Per-user cache directory that REA owns for Kuna on this host. */
const defaultCacheRoot = (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform,
): string => {
const home = homeDirectoryFromEnvironment(environment, platform);
if (platform === "darwin")
return join(home, "Library", "Caches", "rea", "kuna");
const xdg = environment.XDG_CACHE_HOME;
return join(
xdg !== undefined && isAbsolute(xdg) ? xdg : join(home, ".cache"),
"rea",
"kuna",
);
};
/** Parse the Kuna settings without touching the filesystem or the executable. */
export const readKunaConfiguration = (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform = process.platform,
): Result<KunaConfiguration, KunaConfigurationFailure> => {
const command = environment.REA_KUNA_COMMAND;
if (command === undefined || command.length === 0)
return failure("not_configured", KUNA_COMMAND_REMEDIATION);
if (!isAbsolute(command))
return failure(
"open_options_invalid",
`REA_KUNA_COMMAND must be an absolute path; received ${command}. ${KUNA_COMMAND_REMEDIATION}`,
"REA_KUNA_COMMAND",
);
if (platform !== "darwin" && platform !== "linux")
return failure("unsupported_host", KUNA_UNSUPPORTED_HOST_REMEDIATION);
const cacheRoot = environment.REA_KUNA_CACHE_DIR;
if (cacheRoot !== undefined && !isAbsolute(cacheRoot))
return failure(
"open_options_invalid",
`REA_KUNA_CACHE_DIR must be an absolute directory; received ${cacheRoot}.`,
"REA_KUNA_CACHE_DIR",
);
const rawMaxBytes = environment.REA_KUNA_CACHE_MAX_BYTES;
const cacheMaxBytes =
rawMaxBytes === undefined
? DEFAULT_CACHE_MAX_BYTES
: positiveInteger(rawMaxBytes);
if (cacheMaxBytes === undefined || cacheMaxBytes < MIN_CACHE_MAX_BYTES)
return failure(
"open_options_invalid",
`REA_KUNA_CACHE_MAX_BYTES must be a decimal byte count of at least ${String(MIN_CACHE_MAX_BYTES)}; received ${String(rawMaxBytes)}.`,
"REA_KUNA_CACHE_MAX_BYTES",
);
const rawTimeout = environment.REA_KUNA_PROJECT_TIMEOUT_MS;
const timeout =
rawTimeout === undefined
? undefined
: providerStartupTimeoutSchema.safeParse(rawTimeout);
if (timeout !== undefined && !timeout.success)
return failure(
"open_options_invalid",
`REA_KUNA_PROJECT_TIMEOUT_MS must be a decimal millisecond count from 1 to 2147483647; received ${String(rawTimeout)}.`,
"REA_KUNA_PROJECT_TIMEOUT_MS",
);
const rawJobs = environment.REA_KUNA_JOBS;
const jobs = rawJobs === undefined ? DEFAULT_JOBS : positiveInteger(rawJobs);
if (jobs === undefined || jobs > MAX_KUNA_JOBS)
return failure(
"open_options_invalid",
`REA_KUNA_JOBS must be an integer from 1 to ${String(MAX_KUNA_JOBS)}; received ${String(rawJobs)}.`,
"REA_KUNA_JOBS",
);
return ok({
command,
cacheRoot: cacheRoot ?? defaultCacheRoot(environment, platform),
cacheMaxBytes,
projectTimeoutMs: timeout?.data ?? DEFAULT_PROJECT_TIMEOUT_MS,
jobs,
});
};
src/kuna/KunaDoctor.ts
+211
View File
@@ -0,0 +1,211 @@
import { constants } from "node:fs";
import { access, stat } from "node:fs/promises";
import { dirname } from "node:path";
import type {
DoctorProviderCheck,
DoctorProviderInspection,
} from "../application/Doctor.js";
import { readKunaConfiguration } from "./KunaConfiguration.js";
import {
inspectKunaInstallation,
type KunaReadinessFailure,
} from "./KunaInstallation.js";
import {
KUNA_COMMAND_REMEDIATION,
KUNA_PROVIDER_ID,
KUNA_UNSUPPORTED_HOST_REMEDIATION,
KUNA_VERIFIED_VERSION,
KUNA_VERSION_PROBE_REMEDIATION,
} from "./KunaRelease.js";
/** Settings a registration may carry; only those the caller set explicitly. */
const REGISTRATION_SETTINGS = [
"REA_KUNA_COMMAND",
"REA_KUNA_CACHE_DIR",
"REA_KUNA_CACHE_MAX_BYTES",
"REA_KUNA_PROJECT_TIMEOUT_MS",
"REA_KUNA_JOBS",
] as const;
/** Project Kuna readiness into the doctor contract without writing anything. */
export const inspectKunaProvider = async (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform = process.platform,
): Promise<DoctorProviderInspection> => {
const installation = await inspectKunaInstallation(environment, {
platform,
});
const configured =
environment.REA_KUNA_COMMAND !== undefined &&
environment.REA_KUNA_COMMAND.length > 0;
if (!installation.ok)
return {
id: KUNA_PROVIDER_ID,
configured,
available: false,
providerVersion: null,
registrationEnvironment: {},
checks: [installationFailure(installation.error, configured)],
};
const { value } = installation;
return {
id: KUNA_PROVIDER_ID,
configured,
available: true,
providerVersion: value.version,
registrationEnvironment: Object.fromEntries(
REGISTRATION_SETTINGS.flatMap((name) => {
const setting = environment[name];
return setting === undefined ? [] : [[name, setting]];
}),
),
checks: [
{
name: "installation",
ok: true,
code: null,
detail: `${value.executable} reports Kuna ${value.version} (sha256 ${value.sha256})${
value.verified
? "."
: `; REA verified ${KUNA_VERIFIED_VERSION}, so this release's project layout is unverified.`
}`,
remediation: null,
classification: "config_drift",
details: {
executable: value.executable,
version: value.version,
verified_version: KUNA_VERIFIED_VERSION,
sha256: value.sha256,
},
},
await cacheCheck(environment, platform),
],
};
};
const installationFailure = (
failure: KunaReadinessFailure,
configured: boolean,
): DoctorProviderCheck => ({
name: "installation",
ok: false,
code: failure.code,
detail: failure.reason,
remediation:
failure.code === "unsupported_host"
? KUNA_UNSUPPORTED_HOST_REMEDIATION
: failure.code === "version_unresolved"
? KUNA_VERSION_PROBE_REMEDIATION
: KUNA_COMMAND_REMEDIATION,
classification:
failure.code === "unsupported_host"
? "unsupported_host"
: configured
? "config_drift"
: "missing_analysis_engine",
details: failure.diagnostics,
});
/** The cache root may not exist yet; doctor reports it without creating it. */
const cacheCheck = async (
environment: Readonly<NodeJS.ProcessEnv>,
platform: NodeJS.Platform,
): Promise<DoctorProviderCheck> => {
const configuration = readKunaConfiguration(environment, platform);
if (!configuration.ok)
return {
name: "cache",
ok: false,
code: configuration.error.code,
detail: configuration.error.reason,
remediation: KUNA_COMMAND_REMEDIATION,
classification: "config_drift",
};
const { cacheRoot, cacheMaxBytes } = configuration.value;
const details = { cache_root: cacheRoot, cache_max_bytes: cacheMaxBytes };
try {
if (!(await stat(cacheRoot)).isDirectory())
return {
name: "cache",
ok: false,
code: "open_options_invalid",
detail: `The Kuna cache root ${cacheRoot} exists but is not a directory.`,
remediation:
"Set REA_KUNA_CACHE_DIR to a directory REA may own, or remove the conflicting file.",
classification: "config_drift",
details,
};
await access(cacheRoot, constants.W_OK | constants.X_OK);
} catch (cause: unknown) {
if (cause instanceof Error && "code" in cause && cause.code === "ENOENT") {
const creatable = await nearestDirectoryWritable(cacheRoot);
return creatable.ok
? {
name: "cache",
ok: true,
code: null,
detail: `REA creates the Kuna project cache at ${cacheRoot} on first use.`,
remediation: null,
classification: "config_drift",
details,
}
: {
name: "cache",
ok: false,
code: "open_options_invalid",
detail: `REA cannot create the Kuna cache root ${cacheRoot}: ${creatable.reason}`,
remediation:
"Set REA_KUNA_CACHE_DIR to a directory the REA process user can create or write.",
classification: "config_drift",
details,
};
}
return {
name: "cache",
ok: false,
code: "open_options_invalid",
detail: `REA cannot write the Kuna cache root ${cacheRoot}: ${cause instanceof Error ? cause.message : String(cause)}`,
remediation:
"Set REA_KUNA_CACHE_DIR to a directory the REA process user can write.",
classification: "config_drift",
details,
};
}
return {
name: "cache",
ok: true,
code: null,
detail: `Kuna project exports are cached in ${cacheRoot} (at most ${String(cacheMaxBytes)} bytes).`,
remediation: null,
classification: "config_drift",
details,
};
};
/** Whether the nearest existing ancestor is a directory REA may create entries in. */
const nearestDirectoryWritable = async (
path: string,
): Promise<
{ readonly ok: true } | { readonly ok: false; readonly reason: string }
> => {
let candidate = dirname(path);
for (;;) {
try {
if (!(await stat(candidate)).isDirectory())
return { ok: false, reason: `${candidate} is not a directory.` };
await access(candidate, constants.W_OK | constants.X_OK);
return { ok: true };
} catch (cause: unknown) {
const missing =
cause instanceof Error && "code" in cause && cause.code === "ENOENT";
const parent = dirname(candidate);
if (!missing || parent === candidate)
return {
ok: false,
reason: `${candidate} is not writable: ${cause instanceof Error ? cause.message : String(cause)}`,
};
candidate = parent;
}
}
};
src/kuna/KunaInstallation.ts
+191
View File
@@ -0,0 +1,191 @@
import { constants } from "node:fs";
import { access, open, realpath, stat } from "node:fs/promises";
import { join } from "node:path";
import type { JsonValue } from "../domain/jsonValue.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import {
execFileOutput,
execFileOutputFailure,
} from "../process/ExecFileOutput.js";
import {
readKunaConfiguration,
type KunaConfiguration,
type KunaConfigurationFailure,
} from "./KunaConfiguration.js";
import {
KUNA_COMMAND_REMEDIATION,
KUNA_VERIFIED_VERSION,
KUNA_VERSION_PROBE_REMEDIATION,
} from "./KunaRelease.js";
/** `kuna --version` exits before loading specs or an input; it needs no budget beyond this. */
const VERSION_PROBE_TIMEOUT_MS = 10_000;
const VERSION_PROBE_MAX_OUTPUT_BYTES = 64 * 1024;
/** Release CI prints exactly `kuna <MAJOR.MINOR>`; source builds print the Cargo version. */
const VERSION_LINE = /^kuna (\d+\.\d+(?:\.\d+)?)\r?\n?$/u;
/** One observed Kuna executable, identified by its bytes as well as its version. */
export interface KunaInstallation {
readonly configuration: KunaConfiguration;
/** Canonical path of the probed executable. */
readonly executable: string;
readonly version: string;
/** Whether this release is the one REA verified. */
readonly verified: boolean;
readonly sha256: string;
readonly bytes: number;
/**
* Whether the executable is a `#!` script, such as a package-manager shim.
* Its process then reports the interpreter as its command, so ownership
* checks cannot match the configured path by argv prefix.
*/
readonly script: boolean;
}
/** Why the selected Kuna installation cannot be used. */
export interface KunaReadinessFailure {
readonly code:
| KunaConfigurationFailure["code"]
| "executable_missing"
| "version_unresolved";
readonly reason: string;
readonly diagnostics: Readonly<Record<string, JsonValue>>;
}
/**
* Environment for every Kuna child. Kuna unpacks its built-in SLEIGH tree into
* the per-user cache under HOME (macOS) or XDG_CACHE_HOME (Linux), so both
* point inside REA's own cache root instead of the user's.
*/
export const kunaChildEnvironment = (
configuration: KunaConfiguration,
): NodeJS.ProcessEnv => {
const home = join(configuration.cacheRoot, "home");
return {
HOME: home,
XDG_CACHE_HOME: join(home, ".cache"),
NO_COLOR: "1",
};
};
const failure = (
code: KunaReadinessFailure["code"],
reason: string,
diagnostics: Readonly<Record<string, JsonValue>>,
): Result<never, KunaReadinessFailure> => err({ code, reason, diagnostics });
const message = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
/** Resolve, version-probe and fingerprint the configured executable. */
export const inspectKunaInstallation = async (
environment: Readonly<NodeJS.ProcessEnv>,
options: {
readonly platform?: NodeJS.Platform;
readonly signal?: AbortSignal;
/** Probe deadline; tests shorten it. */
readonly probeTimeoutMs?: number;
} = {},
): Promise<Result<KunaInstallation, KunaReadinessFailure>> => {
const configuration = readKunaConfiguration(environment, options.platform);
if (!configuration.ok)
return failure(configuration.error.code, configuration.error.reason, {
configured: configuration.error.code !== "not_configured",
...(configuration.error.setting === null
? {}
: { setting: configuration.error.setting }),
});
const { command } = configuration.value;
let executable: string;
try {
executable = await realpath(command);
await access(executable, constants.X_OK);
if (!(await stat(executable)).isFile())
return failure(
"executable_missing",
`REA_KUNA_COMMAND is not a regular file: ${command}. ${KUNA_COMMAND_REMEDIATION}`,
{ command },
);
} catch (cause: unknown) {
options.signal?.throwIfAborted();
return failure(
"executable_missing",
`Cannot execute REA_KUNA_COMMAND ${command}: ${message(cause)}. ${KUNA_COMMAND_REMEDIATION}`,
{ command },
);
}
const deadline = options.probeTimeoutMs ?? VERSION_PROBE_TIMEOUT_MS;
let stdout: string;
try {
({ stdout } = await execFileOutput(executable, ["--version"], {
env: kunaChildEnvironment(configuration.value),
timeout: deadline,
maxBuffer: VERSION_PROBE_MAX_OUTPUT_BYTES,
// Escalate to SIGKILL so an executable that ignores SIGTERM cannot hang doctor.
stopSignal: "SIGTERM",
...(options.signal === undefined ? {} : { signal: options.signal }),
}));
} catch (cause: unknown) {
options.signal?.throwIfAborted();
const output = execFileOutputFailure(cause);
return failure(
"version_unresolved",
`${executable} --version failed: ${output?.stderr.trim() || message(cause)}. ${KUNA_VERSION_PROBE_REMEDIATION}`,
{
command,
executable,
exit_code: typeof output?.code === "number" ? output.code : null,
signal: output?.signal ?? null,
killed: output?.killed ?? false,
output_truncated: output?.outputTruncated ?? false,
deadline_ms: deadline,
},
);
}
const version = VERSION_LINE.exec(stdout)?.[1];
if (version === undefined)
return failure(
"version_unresolved",
`${executable} --version printed no recognizable Kuna version. ${KUNA_VERSION_PROBE_REMEDIATION}`,
{ command, executable, stdout: stdout.slice(0, 256) },
);
let fingerprint: { readonly sha256: string; readonly bytes: number };
let script: boolean;
try {
fingerprint = await hashStableFile(executable, options.signal);
script = await startsWithShebang(executable);
} catch (cause: unknown) {
options.signal?.throwIfAborted();
return failure(
"executable_missing",
`Cannot fingerprint ${executable}: ${message(cause)}`,
{ command, executable },
);
}
return ok({
configuration: configuration.value,
executable,
version,
verified: version === KUNA_VERIFIED_VERSION,
...fingerprint,
script,
});
};
const startsWithShebang = async (path: string): Promise<boolean> => {
const handle = await open(path, "r");
try {
const prefix = Buffer.alloc(2);
const { bytesRead } = await handle.read(prefix, 0, 2, 0);
return bytesRead === 2 && prefix.toString("latin1") === "#!";
} finally {
await handle.close();
}
};
src/kuna/KunaProjectCache.ts
+690
View File
@@ -0,0 +1,690 @@
import { createHash, randomUUID } from "node:crypto";
import {
mkdir,
mkdtemp,
readdir,
readFile,
rename,
rm,
stat,
utimes,
} from "node:fs/promises";
import { join } from "node:path";
import canonicalize from "canonicalize";
import writeFileAtomic from "write-file-atomic";
import { z } from "zod";
import type { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisCancelledError,
AnalysisResourceConstraintError,
} from "../domain/analysisErrorCore.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import type { KunaInstallation } from "./KunaInstallation.js";
import {
exportKunaProject,
KUNA_EXPORT_OPERATION,
kunaExportFailure,
readKunaExport,
type KunaExport,
type KunaExportStatus,
type KunaProcessLauncher,
} from "./KunaProjectExport.js";
import type {
KunaInventoryFunction,
KunaProjectFunction,
} from "./KunaProjectFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
/** Bumped whenever the entry layout or manifest meaning changes. */
const CACHE_LAYOUT = 1;
/** A manifest names five files and their digests; anything larger is not ours. */
const MANIFEST_MAX_BYTES = 64 * 1024;
const KEY_PATTERN = /^[0-9a-f]{64}$/u;
const SHA256_PATTERN = /^[0-9a-f]{64}$/u;
/** Inputs that determine an export's content. Each one is part of the cache key. */
export interface KunaProjectKeyInputs {
readonly targetSha256: string;
readonly kunaSha256: string;
readonly kunaVersion: string;
/**
* `--jobs 1` is Kuna's reference answer; more jobs decompile without
* callee-first prototypes and change some bodies.
*/
readonly jobs: number;
}
/** Content address of one export: every input that can change its bytes. */
export const kunaProjectKey = (inputs: KunaProjectKeyInputs): string =>
createHash("sha256")
.update(
canonicalize({
layout: CACHE_LAYOUT,
target_sha256: inputs.targetSha256,
kuna_sha256: inputs.kunaSha256,
kuna_version: inputs.kunaVersion,
export: { command: "decompile-project", jobs: inputs.jobs },
}) ?? "",
)
.digest("hex");
const fileSchema = z.strictObject({
path: z.string().min(1),
bytes: z.number().int().nonnegative(),
sha256: z.string().regex(SHA256_PATTERN),
});
const manifestSchema = z.strictObject({
layout: z.literal(CACHE_LAYOUT),
key: z.string().regex(KEY_PATTERN),
inputs: z.strictObject({
target_sha256: z.string().regex(SHA256_PATTERN),
kuna_sha256: z.string().regex(SHA256_PATTERN),
kuna_version: z.string().min(1),
jobs: z.number().int().positive(),
}),
files: z.strictObject({
c: fileSchema,
h: fileSchema,
asm: fileSchema,
readme: fileSchema,
inventory: fileSchema,
}),
summary: z.strictObject({
decompiled: z.number().int().nonnegative(),
failed: z.number().int().nonnegative(),
}),
created_at: z.string().datetime(),
});
/** Immutable description of one published export, written once at publish. */
export type KunaProjectManifest = z.infer<typeof manifestSchema>;
/** One verified, published export. */
export interface KunaProject {
readonly key: string;
/** Directory holding the export's `.c`, `.h`, `.asm`, README and inventory. */
readonly directory: string;
readonly manifest: KunaProjectManifest;
/** Decompiled function blocks in `.c` order. */
readonly functions: readonly KunaProjectFunction[];
/** Every callable entry Kuna found, including imports it does not decompile. */
readonly inventory: readonly KunaInventoryFunction[];
}
/** One request to produce or reuse the export for a target. */
export interface KunaProjectRequest {
readonly installation: KunaInstallation;
/** Caller-selected target path; REA decompiles a verified copy of it. */
readonly targetPath: string;
readonly targetSha256: string;
readonly signal?: AbortSignal;
readonly onStatus?: (status: KunaExportStatus) => void;
}
interface StagedResult {
readonly result: Result<KunaProject, AnalysisError>;
/** Set when an owned process may still write into staging. */
readonly retainStaging: boolean;
}
interface InflightBuild {
readonly key: string;
readonly promise: Promise<Result<KunaProject, AnalysisError>>;
readonly controller: AbortController;
readonly listeners: Set<(status: KunaExportStatus) => void>;
waiters: number;
}
const isMissing = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
(cause.code === "ENOENT" || cause.code === "ENOTDIR");
/** A file an entry's manifest names is gone, replaced or changed under it. */
const isDamaged = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
["ENOENT", "ENOTDIR", "EISDIR", "ENOTFILE", "ESTALE"].includes(
String(cause.code),
);
const isPublishedByAnother = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
(cause.code === "EEXIST" || cause.code === "ENOTEMPTY");
const parseManifest = (text: string): KunaProjectManifest | undefined => {
try {
const parsed = manifestSchema.safeParse(JSON.parse(text));
return parsed.success ? parsed.data : undefined;
} catch {
return undefined;
}
};
const manifestBytes = (manifest: KunaProjectManifest): number =>
Object.values(manifest.files).reduce((sum, file) => sum + file.bytes, 0);
/**
* REA-owned on-disk cache of `kuna decompile-project` exports.
*
* Layout under the root: `projects/<key>/` holds published entries
* (`manifest.json` plus the export and inventory in `project/`), `staging/`
* holds builds whose processes may still be live, `trash/` holds retired
* workspaces and evicted entries awaiting removal,
* and `home/` is Kuna's own HOME so its spec cache stays inside REA's
* ownership.
*
* Entries are content-addressed and immutable. A build runs in private
* staging and is published by one directory rename, so a reader sees a
* complete entry or none, and concurrent publishers of one key keep the first.
* File digests are recorded once at publish and checked on every first read.
* Recency is the entry directory's mtime; publishing evicts the least recently
* used entries above the size cap, never an entry pinned by this process.
*/
export class KunaProjectCache {
readonly #root: string;
readonly #maxBytes: number;
readonly #launcher: KunaProcessLauncher | undefined;
/** Builds new requests may join, at most one per key. */
readonly #inflight = new Map<string, InflightBuild>();
/** Every build not yet settled, including abandoned ones a retry replaced. */
readonly #unfinished = new Set<InflightBuild>();
readonly #verified = new Map<string, KunaProject>();
readonly #pins = new Map<string, number>();
/** One eviction decision at a time; exports themselves remain concurrent. */
#eviction: Promise<void> = Promise.resolve();
#capacity: "within_limit" | "pinned" | "building" = "within_limit";
#maintenance: Promise<void> | undefined;
#maintenanceRequested = false;
readonly #retainedProcesses: ProviderCleanupError[] = [];
/** Only resources whose process cleanup is proven may be retried as files. */
readonly #retainedFiles = new Map<string, ProviderCleanupError>();
constructor(
root: string,
options: {
readonly maxBytes: number;
readonly launcher?: KunaProcessLauncher;
},
) {
this.#root = root;
this.#maxBytes = options.maxBytes;
this.#launcher = options.launcher;
}
/** Keep an entry from eviction while this process serves it. */
pin(key: string): () => void {
this.#pins.set(key, (this.#pins.get(key) ?? 0) + 1);
let released = false;
return () => {
if (released) return;
released = true;
const count = (this.#pins.get(key) ?? 1) - 1;
if (count === 0) {
this.#pins.delete(key);
if (this.#capacity !== "within_limit") this.#scheduleEviction();
} else this.#pins.set(key, count);
};
}
/**
* Return the verified published entry for a key, or undefined when none is
* usable. A corrupt or tampered entry is removed and reported as absent.
*/
async lookup(
key: string,
signal?: AbortSignal,
): Promise<KunaProject | undefined> {
const verified = this.#verified.get(key);
if (verified !== undefined) {
// Another process may have evicted it since this one verified it.
if (await this.#touch(key)) return verified;
this.#verified.delete(key);
return undefined;
}
const entry = join(this.#root, "projects", key);
let text: string;
try {
text = await this.#readBounded(join(entry, "manifest.json"));
} catch (cause: unknown) {
if (isMissing(cause)) return undefined;
if (!(cause instanceof RangeError)) throw cause;
await this.#discard(key);
return undefined;
}
const manifest = parseManifest(text);
if (manifest?.key !== key) {
await this.#discard(key);
return undefined;
}
const directory = join(entry, "project");
for (const file of Object.values(manifest.files)) {
let observed: { readonly sha256: string; readonly bytes: number };
try {
observed = await hashStableFile(join(directory, file.path), signal);
} catch (cause: unknown) {
signal?.throwIfAborted();
if (!isDamaged(cause)) throw cause;
await this.#discard(key);
return undefined;
}
if (observed.sha256 !== file.sha256 || observed.bytes !== file.bytes) {
await this.#discard(key);
return undefined;
}
}
const parsed = await readKunaExport(
directory,
manifest.files,
manifest.summary,
signal,
);
if (!parsed.ok) {
await this.#discard(key);
return undefined;
}
const project = { key, directory, manifest, ...parsed.value };
if (!(await this.#touch(key))) return undefined;
this.#verified.set(key, project);
return project;
}
/**
* Return the entry for a request, building and publishing it when absent.
* Concurrent requests for one key share one Kuna run; the run is cancelled
* only when every waiting request has been cancelled.
*/
async build(
request: KunaProjectRequest,
): Promise<Result<KunaProject, AnalysisError>> {
if (request.signal?.aborted === true)
return err(new AnalysisCancelledError(KUNA_EXPORT_OPERATION));
const key = kunaProjectKey({
targetSha256: request.targetSha256,
kunaSha256: request.installation.sha256,
kunaVersion: request.installation.version,
jobs: request.installation.configuration.jobs,
});
const running = this.#inflight.get(key);
// A build whose waiters all left is winding down; a new request starts afresh.
if (running !== undefined && !running.controller.signal.aborted)
return this.#await(running, request);
const controller = new AbortController();
const listeners = new Set<(status: KunaExportStatus) => void>();
const promise = this.#produce(key, request, controller.signal, (status) => {
for (const listener of listeners) listener(status);
});
const created: InflightBuild = {
key,
promise,
controller,
listeners,
waiters: 0,
};
this.#inflight.set(key, created);
this.#unfinished.add(created);
void promise.then(() => {
this.#unfinished.delete(created);
if (this.#inflight.get(key) === created) this.#inflight.delete(key);
if (this.#unfinished.size === 0 && this.#capacity === "building")
this.#scheduleEviction();
});
return this.#await(created, request);
}
/** Wait for builds, retry safe file removal, and report unresolved cleanup. */
async close(): Promise<Result<null, ProviderCleanupError>> {
await Promise.allSettled(
[...this.#unfinished].map((entry) => entry.promise),
);
while (this.#maintenance !== undefined) await this.#maintenance;
for (const path of [...this.#retainedFiles.keys()])
await this.#removeRetired(path).catch(() => undefined);
const retained = [
...this.#retainedProcesses,
...this.#retainedFiles.values(),
];
if (retained.length === 0) return ok(null);
return err(
new ProviderCleanupError(
KUNA_PROVIDER_ID,
retained.flatMap((entry) => entry.cleanupResources),
{
reason: retained
.map((entry) =>
String(entry.diagnostics?.["reason"] ?? entry.message),
)
.join("; "),
},
{ operation: KUNA_EXPORT_OPERATION },
),
);
}
#await(
entry: InflightBuild,
request: KunaProjectRequest,
): Promise<Result<KunaProject, AnalysisError>> {
entry.waiters += 1;
const onStatus = request.onStatus;
// Each waiter owns its registration, even when callers reuse a callback.
const listener =
onStatus === undefined
? undefined
: (status: KunaExportStatus) => onStatus(status);
if (listener !== undefined) entry.listeners.add(listener);
return new Promise((resolve) => {
let settled = false;
const leave = () => {
if (listener !== undefined) entry.listeners.delete(listener);
entry.waiters -= 1;
};
const onAbort = () => {
if (settled) return;
settled = true;
leave();
if (entry.waiters === 0) entry.controller.abort();
resolve(err(new AnalysisCancelledError(KUNA_EXPORT_OPERATION)));
};
request.signal?.addEventListener("abort", onAbort, { once: true });
void entry.promise.then((result) => {
request.signal?.removeEventListener("abort", onAbort);
if (settled) return;
settled = true;
leave();
resolve(result);
});
});
}
async #produce(
key: string,
request: KunaProjectRequest,
signal: AbortSignal,
onStatus: (status: KunaExportStatus) => void,
): Promise<Result<KunaProject, AnalysisError>> {
let staging: string;
try {
const existing = await this.lookup(key, signal);
if (existing !== undefined) return ok(existing);
await Promise.all(
["projects", "staging", "trash", "home"].map((name) =>
mkdir(join(this.#root, name), { recursive: true, mode: 0o700 }),
),
);
await this.#sweep(signal);
staging = await mkdtemp(join(this.#root, "staging", "build-"));
} catch (cause: unknown) {
return err(kunaExportFailure(cause, signal));
}
const built = await this.#exportAndPublish(
key,
staging,
request,
signal,
onStatus,
).catch((cause: unknown): StagedResult => ({
result: err(kunaExportFailure(cause, signal)),
retainStaging: false,
}));
if (built.retainStaging) return built.result;
let cleanupPath = staging;
try {
// Only a proven-stopped build may enter trash. A failed write, crash or
// lost cleanup identity therefore never makes live staging reclaimable.
const retired = join(this.#root, "trash", `build-${randomUUID()}`);
await rename(staging, retired);
cleanupPath = retired;
await this.#removeRetired(retired);
} catch (cause: unknown) {
if (cause instanceof ProviderCleanupError) return err(cause);
return err(this.#retainFileCleanup(cleanupPath, cause));
}
if (built.result.ok)
// The entry is published either way; a failed eviction retries on the next publish.
await this.#evict(key).catch(() => undefined);
return built.result;
}
async #exportAndPublish(
key: string,
staging: string,
request: KunaProjectRequest,
signal: AbortSignal,
onStatus: (status: KunaExportStatus) => void,
): Promise<StagedResult> {
const outcome = await exportKunaProject({
installation: request.installation,
targetPath: request.targetPath,
targetSha256: request.targetSha256,
workspace: staging,
output: join(staging, "entry", "project"),
signal,
onStatus,
launcher: this.#launcher,
});
if (outcome.kind === "retained") {
this.#retainedProcesses.push(outcome.error);
return { result: err(outcome.error), retainStaging: true };
}
return {
result: outcome.result.ok
? await this.#publish(
key,
staging,
request,
outcome.result.value,
signal,
)
: outcome.result,
retainStaging: false,
};
}
/** Record the manifest and publish the staged export under its key. */
async #publish(
key: string,
staging: string,
request: KunaProjectRequest,
exported: KunaExport,
signal: AbortSignal,
): Promise<Result<KunaProject, AnalysisError>> {
const manifest: KunaProjectManifest = {
layout: CACHE_LAYOUT,
key,
inputs: {
target_sha256: request.targetSha256,
kuna_sha256: request.installation.sha256,
kuna_version: request.installation.version,
jobs: request.installation.configuration.jobs,
},
files: exported.files,
summary: exported.summary,
created_at: new Date().toISOString(),
};
const total = manifestBytes(manifest);
if (total > this.#maxBytes)
return err(
new AnalysisResourceConstraintError(
KUNA_EXPORT_OPERATION,
"capacity",
`This export occupies ${String(total)} bytes, more than the whole Kuna cache may hold.`,
{ cache_max_bytes: this.#maxBytes, export_bytes: total },
{
remediationAction:
"Raise REA_KUNA_CACHE_MAX_BYTES above the export size, or analyze this target with another provider.",
},
),
);
await writeFileAtomic(
join(staging, "entry", "manifest.json"),
`${JSON.stringify(manifest, null, 2)}\n`,
{ mode: 0o600 },
);
const destination = join(this.#root, "projects", key);
try {
await rename(join(staging, "entry"), destination);
} catch (cause: unknown) {
if (!isPublishedByAnother(cause)) throw cause;
const winner = await this.lookup(key, signal);
if (winner !== undefined) return ok(winner);
throw cause;
}
const project = {
key,
directory: join(destination, "project"),
manifest,
functions: exported.functions,
inventory: exported.inventory,
};
this.#verified.set(key, project);
return ok(project);
}
/**
* Remove retired resources only. Age and a missing marker cannot establish
* that a process stopped, so unresolved staging is never swept.
*/
async #sweep(signal: AbortSignal): Promise<void> {
for (const name of await readdir(join(this.#root, "trash"))) {
signal.throwIfAborted();
await this.#removeRetired(join(this.#root, "trash", name)).catch(
() => undefined,
);
}
}
/** Evict least recently used entries above the cap, keeping pinned ones. */
#evict(keep: string | null): Promise<void> {
const eviction = this.#eviction.then(() => this.#evictUnpinned(keep));
this.#eviction = eviction.catch(() => undefined);
return eviction;
}
/** Coalesce deferred reclamation after callers have had a turn to pin results. */
#scheduleEviction(): void {
this.#maintenanceRequested = true;
if (this.#maintenance !== undefined) return;
const maintenance = (async () => {
while (this.#maintenanceRequested) {
await new Promise<void>((resolve) => setImmediate(resolve));
this.#maintenanceRequested = false;
await this.#evict(null).catch(() => undefined);
}
})();
this.#maintenance = maintenance;
void maintenance.then(() => {
if (this.#maintenance === maintenance) this.#maintenance = undefined;
if (this.#maintenanceRequested) this.#scheduleEviction();
});
}
async #evictUnpinned(keep: string | null): Promise<void> {
// A successful build must reach its callers before they can pin its entry.
// Protect every unfinished build while reading this eviction snapshot.
const building = new Set([...this.#unfinished].map(({ key }) => key));
const entries: { key: string; bytes: number; mtimeMs: number }[] = [];
for (const name of await readdir(join(this.#root, "projects"))) {
if (!KEY_PATTERN.test(name)) continue;
const path = join(this.#root, "projects", name);
const status = await stat(path).catch(() => undefined);
if (status === undefined) continue;
const manifest = await this.#readBounded(join(path, "manifest.json"))
.then(parseManifest)
.catch(() => undefined);
// An owned entry without a valid manifest is unusable; evict it first.
entries.push(
manifest === undefined
? { key: name, bytes: 0, mtimeMs: -1 }
: {
key: name,
bytes: manifestBytes(manifest),
mtimeMs: status.mtimeMs,
},
);
}
let total = entries.reduce((sum, entry) => sum + entry.bytes, 0);
let deferredByBuild = false;
for (const entry of entries.sort((a, b) => a.mtimeMs - b.mtimeMs)) {
if (total <= this.#maxBytes && entry.mtimeMs >= 0) break;
if (building.has(entry.key) || this.#inflight.has(entry.key)) {
deferredByBuild = true;
continue;
}
if (entry.key === keep || this.#pins.has(entry.key)) continue;
await this.#discard(entry.key);
total -= entry.bytes;
}
this.#capacity =
total <= this.#maxBytes
? "within_limit"
: deferredByBuild
? "building"
: "pinned";
}
/** Atomically retire one owned entry, then remove it. */
async #discard(key: string): Promise<void> {
this.#verified.delete(key);
const trash = join(this.#root, "trash", `${key}-${randomUUID()}`);
try {
await mkdir(join(this.#root, "trash"), { recursive: true, mode: 0o700 });
await rename(join(this.#root, "projects", key), trash);
} catch (cause: unknown) {
if (isMissing(cause)) return;
throw cause;
}
await this.#removeRetired(trash);
}
/** Keep each failed removal owned until that path is successfully removed. */
async #removeRetired(path: string): Promise<void> {
try {
await rm(path, { recursive: true, force: true });
this.#retainedFiles.delete(path);
} catch (cause: unknown) {
throw this.#retainFileCleanup(path, cause);
}
}
/** Mark an entry recently used; false when it is no longer published. */
async #touch(key: string): Promise<boolean> {
const now = new Date();
try {
await utimes(join(this.#root, "projects", key), now, now);
return true;
} catch (cause: unknown) {
// Recency is advisory; only a missing entry changes the answer.
return !isMissing(cause);
}
}
async #readBounded(path: string): Promise<string> {
const status = await stat(path);
if (status.size > MANIFEST_MAX_BYTES)
throw new RangeError(
`${path} exceeds ${String(MANIFEST_MAX_BYTES)} bytes`,
);
return readFile(path, "utf8");
}
#retainFileCleanup(path: string, cause: unknown): ProviderCleanupError {
const error = new ProviderCleanupError(
KUNA_PROVIDER_ID,
[path],
{ reason: cause instanceof Error ? cause.message : String(cause) },
{ operation: KUNA_EXPORT_OPERATION, cause },
);
this.#retainedFiles.set(path, error);
return error;
}
}
src/kuna/KunaProjectExport.ts
+466
View File
@@ -0,0 +1,466 @@
import { createHash, randomUUID } from "node:crypto";
import { createReadStream, createWriteStream } from "node:fs";
import { lstat, mkdir, readFile, writeFile } from "node:fs/promises";
import { basename, join } from "node:path";
import { Transform } from "node:stream";
import { pipeline } from "node:stream/promises";
import { AnalysisError } from "../domain/analysisErrorBase.js";
import {
AnalysisArtifactChangedError,
AnalysisCancelledError,
AnalysisResourceConstraintError,
AnalysisTimeoutError,
} from "../domain/analysisErrorCore.js";
import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { ProviderCleanupError } from "../domain/providerCleanupError.js";
import { err, ok, type Result } from "../domain/result.js";
import { hashStableFile } from "../filesystem/StableFileHash.js";
import {
OwnedCommandFailure,
runOwnedCommand,
} from "../process/OwnedCommand.js";
import { prepareProcessOwnershipInspection } from "../process/ProcessOwnershipObservation.js";
import type {
OwnedProviderProcessSpawnOptions,
ProviderProcessSnapshot,
SpawnedOwnedProviderProcess,
} from "../process/ProviderProcess.js";
import {
kunaChildEnvironment,
type KunaInstallation,
} from "./KunaInstallation.js";
import {
parseKunaExportSummary,
parseKunaInventory,
scanKunaProjectSource,
type KunaExportSummary,
type KunaInventoryFunction,
type KunaProjectFunction,
} from "./KunaProjectFiles.js";
import { KUNA_PROVIDER_ID } from "./KunaRelease.js";
/** Operation name every export and cache failure reports. */
export const KUNA_EXPORT_OPERATION = "kuna:decompile-project";
/** Kuna prints one summary line per export; notes on a large image stay far below this. */
const DIAGNOSTIC_BYTES = 8 * 1024 * 1024;
/**
* `kuna functions --json` costs ~210 bytes per function; this admits about
* 600,000 functions, above the 392,814 of the largest image Kuna documents.
*/
const INVENTORY_BYTES = 128 * 1024 * 1024;
const INVENTORY_FILE = "functions.json";
/** Identity of one file an export consists of. */
export interface KunaExportFile {
readonly path: string;
readonly bytes: number;
readonly sha256: string;
}
/** The files of one export, by role. */
export interface KunaExportFiles {
readonly c: KunaExportFile;
readonly h: KunaExportFile;
readonly asm: KunaExportFile;
readonly readme: KunaExportFile;
readonly inventory: KunaExportFile;
}
/** One validated export: its files, Kuna's counts and the facts parsed from them. */
export interface KunaExport {
readonly files: KunaExportFiles;
readonly summary: KunaExportSummary;
/** Decompiled function blocks in `.c` order. */
readonly functions: readonly KunaProjectFunction[];
/** Every callable entry Kuna found, including imports it does not decompile. */
readonly inventory: readonly KunaInventoryFunction[];
}
/** The step an export is in; Kuna itself reports no progress until it finishes. */
export interface KunaExportStatus {
readonly phase: "inventory" | "decompiling" | "sealing";
/** Callable entries found by the inventory, once known. */
readonly inventoryFunctions: number | null;
}
/** Test seam for the owned process launcher; production uses the default. */
export type KunaProcessLauncher = (
options: OwnedProviderProcessSpawnOptions,
) => Promise<SpawnedOwnedProviderProcess>;
/**
* How an export ended. `retained` means an owned Kuna process could not be
* proven stopped, so it may still write into the workspace, which therefore
* stays with the caller's cleanup owner.
*/
export type KunaExportOutcome =
| {
readonly kind: "done";
readonly result: Result<KunaExport, AnalysisError>;
}
| { readonly kind: "retained"; readonly error: ProviderCleanupError };
/** One export of a target into a private workspace. */
export interface KunaExportRequest {
readonly installation: KunaInstallation;
/** Caller-selected target path; Kuna decompiles a verified copy of it. */
readonly targetPath: string;
readonly targetSha256: string;
/** Private directory for the copy, Kuna's temporary files and its output. */
readonly workspace: string;
/** Directory, inside the workspace, that receives the export's files. */
readonly output: string;
readonly signal: AbortSignal;
readonly onStatus: (status: KunaExportStatus) => void;
readonly launcher?: KunaProcessLauncher | undefined;
}
const message = (cause: unknown): string =>
cause instanceof Error ? cause.message : String(cause);
const isMissing = (cause: unknown): boolean =>
cause instanceof Error &&
"code" in cause &&
(cause.code === "ENOENT" || cause.code === "ENOTDIR");
/** Map an unexpected failure to a typed one, preferring cancellation. */
export const kunaExportFailure = (
cause: unknown,
signal: AbortSignal,
): AnalysisError => {
if (signal.aborted)
return new AnalysisCancelledError(KUNA_EXPORT_OPERATION, { cause });
if (cause instanceof AnalysisError) return cause;
return new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
cause,
userMessage: `Kuna project cache failed: ${message(cause)}`,
});
};
const unusable = (reason: string): KunaExportOutcome => ({
kind: "done",
result: err(
new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
userMessage: `Kuna's project export is not usable: ${reason}`,
}),
),
});
/** Verify the selected installation at each execution and publication boundary. */
const verifyKunaExecutable = async (
installation: KunaInstallation,
signal: AbortSignal,
): Promise<Result<null, ProviderAdapterError>> => {
const executable = await hashStableFile(installation.executable, signal);
if (executable.sha256 === installation.sha256) return ok(null);
return err(
new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
userMessage: `${installation.executable} changed after readiness inspection; reopen the target so REA probes the current Kuna executable.`,
diagnostics: {
expected_sha256: installation.sha256,
observed_sha256: executable.sha256,
},
}),
);
};
/**
* Inventory and export one target with Kuna, then validate the result.
*
* Kuna runs on a copy hashed while copying, so the export always describes
* the digest the caller selected. Both Kuna runs share one deadline.
*/
export const exportKunaProject = async (
request: KunaExportRequest,
): Promise<KunaExportOutcome> => {
const { installation, signal } = request;
const { configuration } = installation;
const deadline = Date.now() + configuration.projectTimeoutMs;
const input = join(request.workspace, "input", basename(request.targetPath));
await mkdir(join(request.workspace, "input"), { mode: 0o700 });
await mkdir(request.output, { recursive: true, mode: 0o700 });
const copied = await copyAndHash(request.targetPath, input, signal);
if (copied !== request.targetSha256)
return {
kind: "done",
result: err(
new AnalysisArtifactChangedError(
KUNA_EXPORT_OPERATION,
request.targetPath,
"the target no longer matches the digest selected for analysis",
),
),
};
await prepareProcessOwnershipInspection(signal);
const run = (arguments_: readonly string[], diagnosticBytes: number) =>
runKuna(request, arguments_, { deadline, diagnosticBytes });
request.onStatus({ phase: "inventory", inventoryFunctions: null });
const listed = await run(["functions", input, "--json"], INVENTORY_BYTES);
if (!listed.ok) return listed.outcome;
const inventory = parseKunaInventory(listed.snapshot.stdout.text);
if (!inventory.ok) return unusable(inventory.error);
if (listed.snapshot.stdout.invalidUtf8 === true)
return unusable("the function inventory is not valid UTF-8");
await writeFile(
join(request.output, INVENTORY_FILE),
listed.snapshot.stdout.text,
{ mode: 0o600 },
);
request.onStatus({
phase: "decompiling",
inventoryFunctions: inventory.value.length,
});
const exported = await run(
[
"decompile-project",
input,
"-o",
request.output,
"--jobs",
String(configuration.jobs),
],
DIAGNOSTIC_BYTES,
);
if (!exported.ok) return exported.outcome;
request.onStatus({
phase: "sealing",
inventoryFunctions: inventory.value.length,
});
const summary = parseKunaExportSummary(exported.snapshot.stdout.text);
if (summary === undefined)
return unusable(
"Kuna did not print its `functions: N ok, M failed` summary.",
);
const stem = basename(request.targetPath);
const names = {
c: `${stem}.c`,
h: `${stem}.h`,
asm: `${stem}.asm`,
readme: "README.md",
inventory: INVENTORY_FILE,
} as const;
const files: Partial<Record<keyof KunaExportFiles, KunaExportFile>> = {};
for (const [role, name] of Object.entries(names) as [
keyof KunaExportFiles,
string,
][]) {
const path = join(request.output, name);
const status = await lstat(path).catch((cause: unknown) => {
if (isMissing(cause)) return undefined;
throw cause;
});
if (status === undefined || !status.isFile())
return unusable(`Kuna's export has no regular ${name}.`);
files[role] = { path: name, ...(await hashStableFile(path, signal)) };
}
const complete = files as KunaExportFiles;
const parsed = await readKunaExport(
request.output,
complete,
summary,
signal,
);
if (!parsed.ok) return unusable(parsed.error);
const identity = await verifyKunaExecutable(installation, signal);
if (!identity.ok) return { kind: "done", result: identity };
return {
kind: "done",
result: ok({ files: complete, summary, ...parsed.value }),
};
};
/** Parse an export's function blocks and inventory from its own files. */
export const readKunaExport = async (
directory: string,
files: KunaExportFiles,
summary: KunaExportSummary,
signal?: AbortSignal,
): Promise<
Result<
{
readonly functions: readonly KunaProjectFunction[];
readonly inventory: readonly KunaInventoryFunction[];
},
string
>
> => {
const functions = await scanKunaProjectSource(
createReadStream(
join(directory, files.c.path),
signal === undefined ? {} : { signal },
),
summary,
);
if (!functions.ok) return functions;
const inventory = parseKunaInventory(
await readFile(join(directory, files.inventory.path), {
encoding: "utf8",
...(signal === undefined ? {} : { signal }),
}),
);
if (!inventory.ok) return inventory;
return ok({ functions: functions.value, inventory: inventory.value });
};
const runKuna = async (
request: KunaExportRequest,
arguments_: readonly string[],
limits: { readonly deadline: number; readonly diagnosticBytes: number },
): Promise<
| { readonly ok: true; readonly snapshot: ProviderProcessSnapshot }
| { readonly ok: false; readonly outcome: KunaExportOutcome }
> => {
const { installation, signal } = request;
try {
const identity = await verifyKunaExecutable(installation, signal);
if (!identity.ok)
return { ok: false, outcome: { kind: "done", result: identity } };
const snapshot = await runOwnedCommand(
{
command: installation.executable,
arguments: arguments_,
runId: `rea-kuna-${randomUUID()}`,
// A script's process reports its interpreter, not the configured path.
...(installation.script ? { expectedCommand: null } : {}),
cwd: request.workspace,
hostEnvironment: kunaChildEnvironment(installation.configuration),
env: { TMPDIR: request.workspace },
},
{
timeoutMs: Math.max(1, limits.deadline - Date.now()),
diagnosticBytes: limits.diagnosticBytes,
},
{
signal,
...(request.launcher === undefined
? {}
: { launcher: request.launcher }),
},
);
return { ok: true, snapshot };
} catch (cause: unknown) {
return {
ok: false,
outcome: commandFailure(cause, request, arguments_[0] ?? "", limits),
};
}
};
/** Copy the target into the workspace and return the SHA-256 of exactly the bytes copied. */
const copyAndHash = async (
source: string,
destination: string,
signal: AbortSignal,
): Promise<string> => {
const hash = createHash("sha256");
await pipeline(
createReadStream(source),
new Transform({
transform(chunk: Buffer, _encoding, callback) {
hash.update(chunk);
callback(null, chunk);
},
}),
createWriteStream(destination, { flags: "wx", mode: 0o600 }),
{ signal },
);
return hash.digest("hex");
};
/** Map an owned-command failure to its typed error, keeping unproven cleanup owned. */
const commandFailure = (
cause: unknown,
request: KunaExportRequest,
subcommand: string,
limits: { readonly diagnosticBytes: number },
): KunaExportOutcome => {
const { installation, signal } = request;
if (!(cause instanceof OwnedCommandFailure))
return {
kind: "done",
result: err(
signal.aborted
? new AnalysisCancelledError(KUNA_EXPORT_OPERATION, { cause })
: new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
cause,
userMessage: `Cannot run ${installation.executable}: ${message(cause)}`,
}),
),
};
const snapshot = cause.snapshot;
const capturedOutput =
snapshot === null
? undefined
: {
// An inventory is evidence-sized, not diagnostic-sized; keep only its tail.
stdout: snapshot.stdout.text.slice(-4096),
stderr: snapshot.stderr.text,
truncated:
snapshot.diagnosticTruncated === true ||
snapshot.outputIncomplete === true ||
snapshot.stdout.text.length > 4096,
exit_code: snapshot.exitCode ?? null,
signal: snapshot.signal ?? null,
};
const details = {
cause,
...(capturedOutput === undefined ? {} : { capturedOutput }),
};
if (cause.cleanupFailure !== null)
return {
kind: "retained",
error: new ProviderCleanupError(
KUNA_PROVIDER_ID,
[...cause.resources, request.workspace],
{ reason: cause.cleanupFailure },
{ operation: KUNA_EXPORT_OPERATION, cause },
),
};
if (cause.reason === "cancelled")
return {
kind: "done",
result: err(new AnalysisCancelledError(KUNA_EXPORT_OPERATION, details)),
};
if (cause.reason === "timeout")
return {
kind: "done",
result: err(
new AnalysisTimeoutError(
KUNA_EXPORT_OPERATION,
installation.configuration.projectTimeoutMs,
details,
),
),
};
if (cause.reason === "output-limit")
return {
kind: "done",
result: err(
new AnalysisResourceConstraintError(
KUNA_EXPORT_OPERATION,
"capacity",
`kuna ${subcommand} wrote more than ${String(limits.diagnosticBytes)} bytes of output; the export was not retained.`,
{ output_limit_bytes: limits.diagnosticBytes },
details,
),
),
};
const exitCode = capturedOutput?.exit_code;
return {
kind: "done",
result: err(
new ProviderAdapterError(KUNA_PROVIDER_ID, KUNA_EXPORT_OPERATION, {
...details,
userMessage: `kuna ${subcommand} failed${exitCode === undefined || exitCode === null ? "" : ` with exit code ${String(exitCode)}`}: ${capturedOutput?.stderr.trim().split("\n").at(-1) ?? cause.message}`,
}),
),
};
};
src/kuna/KunaProjectFiles.ts
+189
View File
@@ -0,0 +1,189 @@
import { z } from "zod";
import { err, ok, type Result } from "../domain/result.js";
/**
* One function block in a `kuna decompile-project` `.c` file. The span covers
* its `// Function:` header through the blank line that ends the block, which
* is how REA serves one function without reading the whole file.
*/
export interface KunaProjectFunction {
readonly name: string;
/** Kuna's display address, a lowercase `0x…` virtual address. */
readonly address: string;
readonly cOffset: number;
readonly cLength: number;
/** Kuna's own failure for this function; its block is then the header alone. */
readonly error: string | null;
}
// Rendered by kuna-console's `render_c`: `// Function: {name} @ 0x{addr:x}`,
// or `... @ 0x{addr:x} (error: {message})` for a failed function. A name
// can itself contain " @ ", so the address is the last such field.
const HEADER = /^\/\/ Function: (.+) @ (0x[0-9a-f]+)(?: {2}\(error: (.*)\))?$/u;
const HEADER_PREFIX = "// Function: ";
/** Kuna's completed-export summary on stdout: `functions: N ok, M failed`. */
const SUMMARY = /functions: (\d+) ok, (\d+) failed/u;
/** Function counts Kuna reported for a finished export. */
export interface KunaExportSummary {
readonly decompiled: number;
readonly failed: number;
}
/** Read the completed-file summary Kuna prints after finalizing an export. */
export const parseKunaExportSummary = (
stdout: string,
): KunaExportSummary | undefined => {
const match = SUMMARY.exec(stdout);
if (match?.[1] === undefined || match[2] === undefined) return undefined;
return { decompiled: Number(match[1]), failed: Number(match[2]) };
};
/** A header line longer than this is not one Kuna wrote. */
const MAX_HEADER_BYTES = 64 * 1024;
const PREFIX_BYTES = Buffer.from(HEADER_PREFIX);
interface Header {
readonly name: string;
readonly address: string;
readonly error: string | null;
readonly offset: number;
}
/**
* Split a finished export's `.c` into its function blocks, reading it as a
* stream so a very large export is never held in memory.
*
* The `#include` preamble precedes the first header. The header count must
* equal the functions Kuna reported, and every address must be distinct; an
* export that fails either check is not a usable function inventory.
*/
export const scanKunaProjectSource = async (
chunks: AsyncIterable<Buffer>,
summary: KunaExportSummary,
): Promise<Result<readonly KunaProjectFunction[], string>> => {
const headers: Header[] = [];
let offset = 0;
let lineStart = 0;
// Bytes of the current line, kept only while it may still be a header.
let line: Buffer[] | undefined = [];
let lineBytes = 0;
const endLine = (): string | undefined => {
if (line === undefined) return undefined;
const text = Buffer.concat(line).toString("utf8");
if (!text.startsWith(HEADER_PREFIX)) return undefined;
const match = HEADER.exec(text);
if (match?.[1] === undefined || match[2] === undefined)
return `unrecognized function header at byte ${String(lineStart)}`;
headers.push({
name: match[1],
address: match[2],
error: match[3] ?? null,
offset: lineStart,
});
return undefined;
};
for await (const chunk of chunks) {
let position = 0;
while (position < chunk.length) {
const newline = chunk.indexOf(0x0a, position);
const end = newline === -1 ? chunk.length : newline;
if (line !== undefined) {
const segment = chunk.subarray(position, end);
line.push(segment);
lineBytes += segment.length;
const seen = Buffer.concat(line).subarray(0, PREFIX_BYTES.length);
if (!PREFIX_BYTES.subarray(0, seen.length).equals(seen))
line = undefined;
else if (lineBytes > MAX_HEADER_BYTES)
return err(
`function header at byte ${String(lineStart)} is too long`,
);
}
if (newline === -1) break;
const failure = endLine();
if (failure !== undefined) return err(failure);
position = newline + 1;
lineStart = offset + position;
line = [];
lineBytes = 0;
}
offset += chunk.length;
}
const failure = endLine();
if (failure !== undefined) return err(failure);
const total = offset;
const expected = summary.decompiled + summary.failed;
if (headers.length !== expected)
return err(
`the .c holds ${String(headers.length)} function headers but Kuna reported ${String(expected)} functions`,
);
const failed = headers.filter(({ error }) => error !== null).length;
if (failed !== summary.failed)
return err(
`the .c marks ${String(failed)} failed functions but Kuna reported ${String(summary.failed)}`,
);
const addresses = new Set<string>();
const functions: KunaProjectFunction[] = [];
for (const [index, header] of headers.entries()) {
if (addresses.has(header.address))
return err(`the .c repeats address ${header.address}`);
addresses.add(header.address);
const end = headers[index + 1]?.offset ?? total;
functions.push({
name: header.name,
address: header.address,
cOffset: header.offset,
cLength: end - header.offset,
error: header.error,
});
}
return ok(functions);
};
const inventorySchema = z.object({
functions: z.array(
z.object({
name: z.string(),
address_hex: z.string().regex(/^0x[0-9a-f]+$/u),
aliases: z.array(z.string()),
size: z.number().int().nonnegative().max(Number.MAX_SAFE_INTEGER),
}),
),
});
/** One callable entry from `kuna functions --json`, including imports Kuna does not decompile. */
export interface KunaInventoryFunction {
readonly name: string;
readonly address: string;
readonly aliases: readonly string[];
/** Inventory extent: an upper bound clipped at the next entry, or 0 when unmeasured. */
readonly size: number;
}
/** Parse the callable-symbol inventory `kuna functions --json` writes. */
export const parseKunaInventory = (
text: string,
): Result<readonly KunaInventoryFunction[], string> => {
let value: unknown;
try {
value = JSON.parse(text);
} catch {
return err("the function inventory is not JSON");
}
const parsed = inventorySchema.safeParse(value);
if (!parsed.success)
return err(
`the function inventory does not match Kuna's document: ${parsed.error.issues[0]?.message ?? "invalid"}`,
);
return ok(
parsed.data.functions.map((entry) => ({
name: entry.name,
address: entry.address_hex,
aliases: entry.aliases,
size: entry.size,
})),
);
};
src/kuna/KunaRelease.ts
+16
View File
@@ -0,0 +1,16 @@
/** Kuna release whose CLI and project layout REA verified; others report as unverified. */
export const KUNA_VERIFIED_VERSION = "1.833";
/** Stable provider ID shared by readiness, doctor and Evidence. */
export const KUNA_PROVIDER_ID = "kuna";
/** Remediation when `REA_KUNA_COMMAND` is missing or not an absolute path. */
export const KUNA_COMMAND_REMEDIATION = `Set REA_KUNA_COMMAND to the absolute path of a caller-supplied kuna executable (verified with ${KUNA_VERIFIED_VERSION}). REA does not download, install or update Kuna.`;
/** Remediation when the host cannot supervise a Kuna process tree. */
export const KUNA_UNSUPPORTED_HOST_REMEDIATION =
"Use Linux or macOS for the Kuna provider. Windows process-tree ownership for Kuna is not verified.";
/** Remediation when `kuna --version` cannot establish the installed release. */
export const KUNA_VERSION_PROBE_REMEDIATION =
"Run the selected executable with --version: it must exit promptly and print `kuna <major>.<minor>`.";
src/native/AppleDispatchMetadata.test.ts
+33
View File
@@ -1,4 +1,5 @@
import { describe, expect, it } from "vitest";
import { machoWithPropertyAttributes } from "../../tests/fixtures/objcPropertyMacho.js";
import { decodeAppleDispatchMetadata } from "./AppleDispatchMetadata.js";
const base = 0x100000000n;
@@ -423,6 +424,38 @@ describe("Apple dispatch coverage ownership", () => {
});
});
describe("malformed Objective-C property records", () => {
it.each([
["an unmapped name", { secondNameOffset: 0x2000 }, "first", "0x100000518"],
["an empty name", { firstName: "" }, "second", "0x100000508"],
] as const)(
"retains the valid sibling of a property with %s",
(_, layout, kept, location) => {
const result = decodeAppleDispatchMetadata(
machoWithPropertyAttributes("Tq,N", layout),
100,
provenance,
);
expect(
result.objc_classes.map(({ name, properties }) => ({
name,
properties: properties.map((property) => property.name),
})),
).toEqual([{ name: "Fixture", properties: [kept] }]);
expect(
result.coverage.find(
({ facet }) => facet === "objc_properties_categories",
),
).toMatchObject({
status: "partial",
reason: expect.stringContaining(
`property_decode_failed at ${location}: Fixture: property`,
),
});
},
);
});
describe("malformed Apple dispatch metadata", () => {
it("retains valid class siblings when one class has an empty name", () => {
const result = decodeAppleDispatchMetadata(
src/native/AppleMachoFixups.test.ts
+13
View File
@@ -983,4 +983,17 @@ describe("Objective-C property attributes", () => {
expect(parsePropertyAttributes("count", "").atomicity).toBeNull();
expect(parsePropertyAttributes("count", "R").atomicity).toBeNull();
});
it("establishes no defaults from an unfinished quoted class name", () => {
// `,N,R` here lie inside the unterminated quote, so they are not flags.
expect(parsePropertyAttributes("title", 'T@"NSString,N,R')).toMatchObject({
type_encoding: '@"NSString,N,R',
atomicity: null,
is_readonly: null,
});
expect(parsePropertyAttributes("title", 'N,R,T@"NSString')).toMatchObject({
atomicity: "nonatomic",
is_readonly: true,
});
});
});
src/native/AppleObjcProperties.ts
+91 -23
View File
@@ -19,8 +19,21 @@ export interface ObjcPropertyReaders {
export const parsePropertyAttributes = (
name: string,
attributes: string,
): ObjcProperty => {
const tokens = attributes.length === 0 ? [] : splitAttributes(attributes);
): ObjcProperty => decodePropertyAttributes(name, attributes).property;
/**
* Decode one attribute string. A quoted class name that never closes leaves
* the string incomplete: later attributes may be inside the unfinished type,
* so a missing `N` or `R` does not establish the Objective-C defaults.
*/
const decodePropertyAttributes = (
name: string,
attributes: string,
): { readonly property: ObjcProperty; readonly complete: boolean } => {
const { tokens, complete } =
attributes.length === 0
? { tokens: [], complete: true }
: splitAttributes(attributes);
const typeAttributes = tokens.filter((token) => token[0] === "T");
const typeAttribute =
typeAttributes.length === 1 ? typeAttributes[0] : undefined;
@@ -30,7 +43,8 @@ export const parsePropertyAttributes = (
: null;
const value = (code: string): string | null =>
tokens.find((token) => token[0] === code)?.slice(1) ?? null;
return {
const defaultsEstablished = typeEncoding !== null && complete;
const property: ObjcProperty = {
name,
type_encoding: typeEncoding,
attributes: tokens.map((token) => ({
@@ -39,21 +53,27 @@ export const parsePropertyAttributes = (
})),
atomicity: tokens.includes("N")
? "nonatomic"
: typeEncoding === null
? null
: "atomic",
: defaultsEstablished
? "atomic"
: null,
is_readonly: tokens.includes("R")
? true
: typeEncoding === null
? null
: false,
: defaultsEstablished
? false
: null,
getter: value("G"),
setter: value("S"),
};
return { property, complete };
};
/** Split on commas that are outside a quoted class name in the type encoding. */
const splitAttributes = (attributes: string): string[] => {
/**
* Split on commas that are outside a quoted class name in the type encoding,
* and report whether every quote closed.
*/
const splitAttributes = (
attributes: string,
): { readonly tokens: string[]; readonly complete: boolean } => {
const tokens: string[] = [];
let current = "";
let quoted = false;
@@ -65,34 +85,68 @@ const splitAttributes = (attributes: string): string[] => {
} else current += character;
}
tokens.push(current);
return tokens.filter((token) => token.length > 0);
return {
tokens: tokens.filter((token) => token.length > 0),
complete: !quoted,
};
};
/** Decode `property_list_t { entsize_and_flags, count, property_t[] }`. */
/** One property-level decode problem, relative to the list's owner. */
interface PropertyListIssue {
readonly code: "property_decode_failed" | "property_attributes_incomplete";
readonly message: string;
readonly location: string;
}
/**
* Decode `property_list_t { entsize_and_flags, count, property_t[] }`. Each
* entry is decoded on its own, so one malformed entry is reported without
* discarding the properties decoded before or after it.
*/
export const readObjcPropertyList = (
read: ObjcPropertyReaders,
list: bigint,
admit: () => boolean,
): ObjcProperty[] => {
if (list === 0n) return [];
): {
readonly properties: ObjcProperty[];
readonly issues: PropertyListIssue[];
} => {
const properties: ObjcProperty[] = [];
const issues: PropertyListIssue[] = [];
if (list === 0n) return { properties, issues };
const entrySize = read.u32(list) & ~3;
const count = read.u32(list + 4n);
if (entrySize !== 16)
throw new TypeError(
`Unsupported Objective-C property entry size ${entrySize}`,
);
const properties: ObjcProperty[] = [];
for (let index = 0; index < count; index++) {
if (!admit()) break;
const entry = list + 8n + BigInt(index * entrySize);
properties.push(
parsePropertyAttributes(
read.string(read.pointer(entry)),
const location = `0x${entry.toString(16)}`;
try {
const name = read.string(read.pointer(entry));
if (name.length === 0) throw new TypeError("the property name is empty");
const decoded = decodePropertyAttributes(
name,
read.string(read.pointer(entry + 8n)),
),
);
);
properties.push(decoded.property);
if (!decoded.complete)
issues.push({
code: "property_attributes_incomplete",
message: `property ${name} has an unterminated quoted class name in its type attribute, so atomicity and unestablished read-only state are unknown`,
location,
});
} catch (cause: unknown) {
issues.push({
code: "property_decode_failed",
message: `property ${String(index)} could not be decoded: ${cause instanceof Error ? cause.message : String(cause)}`,
location,
});
}
}
return properties;
return { properties, issues };
};
/** Decode a property list, recording a failure for `owner` instead of throwing. */
@@ -107,7 +161,21 @@ export const readObjcPropertiesOf = (
},
): ObjcProperty[] => {
try {
return readObjcPropertyList(read, list, context.admit);
const { properties, issues } = readObjcPropertyList(
read,
list,
context.admit,
);
for (const item of issues)
context.issues.push(
issue(
"objc_properties_categories",
item.code,
`${context.owner}: ${item.message}`,
item.location,
),
);
return properties;
} catch (cause: unknown) {
context.issues.push(
issue(
src/native/parsers/plist.ts
+4
View File
@@ -4,6 +4,7 @@ import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
import type { JsonValue } from "../../domain/jsonValue.js";
import {
projectPlistValue,
unknownDateLimitation,
type ProjectedPlistValue,
} from "../../domain/apple/plistValue.js";
import {
@@ -178,6 +179,9 @@ export const parsePlistXml = (
: [
`${String(projected.unknownRealCount)} non-finite real value(s) are reported as { "$plist_type": "real", "value": null } because the XML decoder does not distinguish NaN from infinity.`,
]),
...(projected.unknownDateCount === 0
? []
: [unknownDateLimitation(projected.unknownDateCount)]),
...numberLimitations(exactIntegerCount, ambiguousNumberCount),
...prototypeKeyLimitations(omittedPrototypeKeys),
],
src/process/DetachedPgpSignature.test.ts
+128
View File
@@ -0,0 +1,128 @@
import { beforeAll, describe, expect, it } from "vitest";
import { createMessage, generateKey, readPrivateKey, sign } from "openpgp";
import { verifyDetachedPgpSignature } from "./DetachedPgpSignature.js";
const bytes = Buffer.from("binary package fixture\0\xff", "latin1");
let publicKey: string;
let fingerprint: string;
let signatures: Uint8Array[];
let signingFingerprints: string[];
beforeAll(async () => {
const generated = await generateKey({
type: "ecc",
curve: "curve25519Legacy",
userIDs: [{ name: "Fixture publisher" }],
subkeys: [{ sign: true }, { sign: true }],
format: "armored",
});
const privateKey = await readPrivateKey({ armoredKey: generated.privateKey });
publicKey = generated.publicKey;
fingerprint = privateKey.getFingerprint().toUpperCase();
signingFingerprints = privateKey.subkeys.map((subkey) =>
subkey.getFingerprint().toUpperCase(),
);
signatures = [];
for (const subkey of privateKey.subkeys)
signatures.push(
await sign({
message: await createMessage({ binary: bytes }),
signingKeys: privateKey,
signingKeyIDs: [subkey.getKeyID()],
detached: true,
format: "binary",
}),
);
});
describe("detached binary PGP publisher verification", () => {
it("accepts independently selected signing subkeys through one pinned primary", async () => {
for (const [index, signature] of signatures.entries())
expect(
await verifyDetachedPgpSignature(
bytes,
signature,
publicKey,
fingerprint,
),
).toEqual({
ok: true,
value: {
primaryFingerprint: fingerprint,
signingFingerprint: signingFingerprints[index],
},
});
});
it("rejects a valid signature belonging to another primary", async () => {
expect(
await verifyDetachedPgpSignature(
bytes,
signatures[0]!,
publicKey,
"0".repeat(40),
),
).toMatchObject({ ok: false, error: { reason: "wrong_publisher" } });
});
it("rejects tampered payload bytes", async () => {
const tampered = Buffer.from(bytes);
tampered[0] = (tampered[0] ?? 0) ^ 1;
expect(
await verifyDetachedPgpSignature(
tampered,
signatures[0]!,
publicKey,
fingerprint,
),
).toMatchObject({ ok: false, error: { reason: "invalid_signature" } });
});
it("rejects malformed signatures and public keys", async () => {
expect(
await verifyDetachedPgpSignature(
bytes,
new Uint8Array([1, 2, 3]),
publicKey,
fingerprint,
),
).toMatchObject({ ok: false, error: { reason: "invalid_signature" } });
expect(
await verifyDetachedPgpSignature(
bytes,
signatures[0]!,
"malformed",
fingerprint,
),
).toMatchObject({ ok: false, error: { reason: "invalid_signature" } });
});
it("preserves cancellation", async () => {
expect(
await verifyDetachedPgpSignature(
bytes,
signatures[0]!,
publicKey,
fingerprint,
AbortSignal.abort(),
),
).toMatchObject({ ok: false, error: { reason: "cancelled" } });
});
it("preserves cancellation while loading the publisher key, before classifying its identity", async () => {
const controller = new AbortController();
const verification = verifyDetachedPgpSignature(
bytes,
signatures[0]!,
publicKey,
"0".repeat(40),
controller.signal,
);
queueMicrotask(() => controller.abort());
expect(await verification).toMatchObject({
ok: false,
error: { reason: "cancelled" },
});
});
});
src/process/DetachedPgpSignature.ts
+101
View File
@@ -0,0 +1,101 @@
import { err, ok, type Result } from "../domain/result.js";
export class DetachedPgpSignatureError extends Error {
readonly _tag = "DetachedPgpSignatureError";
constructor(
readonly reason: "invalid_signature" | "wrong_publisher" | "cancelled",
message: string,
options?: ErrorOptions,
) {
super(message, options);
this.name = this._tag;
}
}
/** Verify the complete binary payload and its signing subkey's primary identity. */
export const verifyDetachedPgpSignature = async (
bytes: Uint8Array,
detachedSignature: Uint8Array,
armoredPublicKey: string,
primaryFingerprint: string,
signal?: AbortSignal,
): Promise<
Result<
{
readonly primaryFingerprint: string;
readonly signingFingerprint: string;
},
DetachedPgpSignatureError
>
> => {
const cancelled = () =>
err(
new DetachedPgpSignatureError(
"cancelled",
"Package verification was cancelled.",
),
);
const isAborted = () => signal?.aborted === true;
if (isAborted()) return cancelled();
try {
// Keep the cryptographic implementation out of unrelated CLI cold starts.
const pgp = await import("openpgp");
const key = await pgp.readKey({ armoredKey: armoredPublicKey });
if (isAborted()) return cancelled();
const fingerprint = key.getFingerprint().toUpperCase();
if (fingerprint !== primaryFingerprint)
return err(
new DetachedPgpSignatureError(
"wrong_publisher",
"The package signing key does not belong to the pinned publisher.",
),
);
const signature = await pgp.readSignature({
binarySignature: detachedSignature,
});
if (isAborted()) return cancelled();
if (
signature.packets.length !== 1 ||
signature.packets[0]?.signatureType !== pgp.enums.signature.binary
)
return err(
new DetachedPgpSignatureError(
"invalid_signature",
"Expected one detached binary package signature.",
),
);
const verified = await pgp.verify({
message: await pgp.createMessage({ binary: bytes }),
signature,
verificationKeys: key,
format: "binary",
});
const signed = verified.signatures[0];
if (signed === undefined || verified.signatures.length !== 1)
return err(
new DetachedPgpSignatureError(
"invalid_signature",
"The package has no unique verified signature.",
),
);
await signed.verified;
const signingKey = await key.getSigningKey(
signed.keyID,
signature.packets[0].created,
);
if (isAborted()) return cancelled();
return ok({
primaryFingerprint: fingerprint,
signingFingerprint: signingKey.getFingerprint().toUpperCase(),
});
} catch (cause: unknown) {
if (isAborted()) return cancelled();
return err(
new DetachedPgpSignatureError(
"invalid_signature",
"The package signature could not be verified.",
{ cause },
),
);
}
};
src/hopper/HopperTargetLease.ts → src/process/HopperLease.ts
+18
View File
@@ -17,6 +17,8 @@ interface HopperTargetLeaseOwner {
}
export interface HopperTargetLease {
/** Bound socket identity for cleanup diagnostics; omission remains unknown. */
readonly socketPath?: string;
readonly release: () => Promise<void>;
}
@@ -66,6 +68,21 @@ export const acquireLinuxHopperApplicationLease = (input: {
}): Promise<HopperTargetLeaseAcquisition> =>
acquireLease({ ...input, identity: { application: "linux-hopper-demo" } });
/** Serialize approved publication/repair of one canonical signed-package entry. */
export const acquireHopperReleaseCacheLease = async (input: {
readonly cacheDirectory: string;
readonly launcherSha256: string;
readonly runId: string;
}): Promise<HopperTargetLeaseAcquisition> =>
acquireLease({
runId: input.runId,
identity: {
cacheDirectory: await realpath(input.cacheDirectory),
launcherSha256: input.launcherSha256,
operation: "signed-package-retention",
},
});
const acquireLease = async (input: {
readonly identity: Readonly<Record<string, unknown>>;
readonly runId: string;
@@ -153,6 +170,7 @@ const tryAcquireLease = async (
return {
acquired: true,
lease: {
socketPath,
release: async () => {
if (released) return;
// Node removes a bound Unix socket on close. A second unlink here
tests/acceptance/applications/applicationWorkflowCli.test.ts
+34 -20
View File
@@ -541,7 +541,7 @@ it("resolves source-map URL paths once without changing their declarations", asy
await writeFile(
join(root, "app.js"),
cases.map(([url]) => `//# sourceMappingURL=${url}\n`).join("") +
'new Worker("./worker%20file.js");\n' +
'new Worker(new URL("./worker%20file.js", import.meta.url));\n' +
'navigator.serviceWorker.register("./service%20worker.js");\n',
);
await writeFile(join(root, "worker file.js"), "export const worker = 1;\n");
@@ -598,27 +598,41 @@ it("resolves source-map URL paths once without changing their declarations", asy
}),
);
}
for (const [kind, specifier, path] of [
["worker", "./worker%20file.js", "worker file.js"],
["service-worker", "./service%20worker.js", "service worker.js"],
]) {
const worker = graph.nodes.find((node) => node.kind === kind);
expect(worker?.observations).toContainEqual(
expect.objectContaining({
properties: expect.objectContaining({
declared_specifier: specifier,
resolved_path: path,
}),
const worker = graph.nodes.find((node) => node.kind === "worker");
expect(worker?.observations).toContainEqual(
expect.objectContaining({
properties: expect.objectContaining({
declared_specifier: "./worker%20file.js",
resolved_path: "worker file.js",
}),
);
expect(graph.edges).toContainEqual(
expect.objectContaining({
source_node_id: worker?.node_id,
relation: "maps_to",
properties: { resolved_path: path },
}),
);
expect(graph.edges).toContainEqual(
expect.objectContaining({
source_node_id: worker?.node_id,
relation: "maps_to",
properties: { resolved_path: "worker file.js" },
}),
);
// A service worker script URL is relative to the unestablished document base.
const serviceWorker = graph.nodes.find(
(node) => node.kind === "service-worker",
);
expect(serviceWorker?.observations).toContainEqual(
expect.objectContaining({
properties: expect.objectContaining({
declared_specifier: null,
expression: 'navigator.serviceWorker.register("./service%20worker.js")',
resolved_path: null,
}),
);
}
}),
);
expect(
graph.edges.some(
({ source_node_id, relation }) =>
source_node_id === serviceWorker?.node_id && relation === "maps_to",
),
).toBe(false);
expect((await analyzeThroughStdioMcp(root)).graph).toEqual(graph);
}, 20_000);
tests/acceptance/applications/javascriptWorkerUrlCli.test.ts
+51 -9
View File
@@ -279,12 +279,6 @@ cliTest(
specifier: "./worker.js",
resolved: "bundled/worker.js",
},
{
source: "nested/literal.mjs",
expression: 'new Worker("./literal-worker.js");',
specifier: "./literal-worker.js",
resolved: "nested/literal-worker.js",
},
];
await writeFixtureFiles(root, {
...Object.fromEntries(
@@ -431,9 +425,6 @@ cliTest(
"directory-dot.mjs": "./worker.js/.",
"directory-parent.mjs": "./worker.js/child/..",
"empty-segment.mjs": "./child//../worker.js",
"empty-worker-url.mjs": "",
"query-only-worker-url.mjs": "?cache=1",
"fragment-only-worker-url.mjs": "#worker",
"package.mjs": "./package-worker",
"external.mjs": "https://example.test/worker.js",
};
@@ -478,6 +469,57 @@ cliTest(
},
);
cliTest(
"keeps string Worker paths unresolved because the document base is unknown",
async ({ cli }) => {
const root = await createTestTempDirectory("rea-worker-document-base-");
// A browser resolves these strings against the document base URL, not the
// calling script; Node resolves them against the working directory.
const expressions = {
"scripts/main.js": 'new Worker("./worker.js")',
"scripts/service.js": 'navigator.serviceWorker.register("./sw.js")',
};
await writeFixtureFiles(root, {
"index.html": '<script src="./scripts/main.js"></script>',
...Object.fromEntries(
Object.entries(expressions).map(([path, code]) => [path, `${code};`]),
),
"worker.js": "self.postMessage('document-relative worker');",
"scripts/worker.js": "self.postMessage('script-relative decoy');",
"scripts/sw.js": "self.postMessage('script-relative decoy');",
});
const result = await analyzeJavaScriptDirectory("CLI", cli, root);
for (const [source, expression] of Object.entries(expressions)) {
const node = result.graph.nodes.find(
({ kind, observations }) =>
(kind === "worker" || kind === "service-worker") &&
observations.some(
({ evidence }) =>
evidence.location.available &&
evidence.location.value.kind === "source-range" &&
evidence.location.value.source === source,
),
);
expect(node?.observations).toContainEqual(
expect.objectContaining({
properties: expect.objectContaining({
declared_specifier: null,
expression,
resolved_path: null,
}),
}),
);
expect(
result.graph.edges.some(
({ relation, source_node_id }) =>
relation === "maps_to" && source_node_id === node?.node_id,
),
).toBe(false);
}
},
);
cliTest(
"bounds repeated ambient alias branches through the compiled CLI",
async ({ cli }) => {
tests/acceptance/providers/hopperStartupRecovery.test.ts
+1 -1
View File
@@ -10,7 +10,7 @@ import { z } from "zod";
import { parseMcpToolError } from "../../fixtures/mcpToolError.js";
import { workspaceCliTest } from "../../support/cli/workspaceCliFixture.js";
import { acquireHopperTargetLease } from "../../../src/hopper/HopperTargetLease.js";
import { acquireHopperTargetLease } from "../../../src/process/HopperLease.js";
const statusSchema = z.object({ result: z.object({ open: z.boolean() }) });
const test = workspaceCliTest.skipIf(process.platform !== "darwin");
tests/boundary/cli/cliOutput.test.ts
+80 -12
View File
@@ -7,6 +7,7 @@ import { describe, expect, it } from "vitest";
import { workspaceCliTest } from "../../support/cli/workspaceCliFixture.js";
import {
normalizeCliOutputArguments,
renderCliOutputArgumentError,
renderEmptyFilteredCliOutput,
validateCliOutputArguments,
@@ -58,6 +59,50 @@ describe("CLI output argument boundary", () => {
});
});
describe("CLI output flag value forms", () => {
it("splits only the equals forms of valued output flags", () => {
expect(
normalizeCliOutputArguments([
"inspect-sqlite-database",
"--table=a=b",
"--format=json",
"--filter-output=a.b,c[0,1]",
"--token-limit=5",
"--token-offset=",
"--json",
"--format-hint=x",
]),
).toEqual([
"inspect-sqlite-database",
"--table=a=b",
"--format",
"json",
"--filter-output",
"a.b,c[0,1]",
"--token-limit",
"5",
"--token-offset",
"",
"--json",
"--format-hint=x",
]);
});
it("reports a missing value only for a flag the parser did not consume", () => {
expect(
validateCliOutputArguments([
"capabilities",
"--json",
"--filter-output",
"--format",
]),
).toEqual({ ok: true });
expect(
validateCliOutputArguments(["capabilities", "--json", "--format"]),
).toMatchObject({ ok: false, code: "MISSING_OUTPUT_VALUE" });
});
});
describe("unclosed CLI output filter slices", () => {
it.each([
"providers[",
@@ -153,7 +198,7 @@ describe("CLI output valued-flag parsing", () => {
async ({ cli }) => {
for (const arguments_ of [
["providers", "--", "--json", "--token-limit", "5"],
["providers", "--json", "--format=toon", "--token-limit", "5"],
["providers", "--format=json", "--token-limit=5"],
]) {
const result = await cli.run({ arguments: arguments_ });
expect(result.exitCode).toBe(1);
@@ -183,22 +228,45 @@ describe("CLI output valued-flag parsing", () => {
);
workspaceCliTest(
"does not treat empty or equals-style output values as active builtins",
"names missing output values and honors equals-style output values",
async ({ cli }) => {
for (const arguments_ of [
["capabilities", "--json", "--format", ""],
["capabilities", "--json", "--filter-output", ""],
["capabilities", "--json", "--token-limit", ""],
["capabilities", "--json", "--token-limit=5"],
]) {
for (const [arguments_, flag] of [
[["capabilities", "--json", "--format", ""], "--format"],
[["capabilities", "--json", "--filter-output", ""], "--filter-output"],
[["capabilities", "--json", "--token-limit", ""], "--token-limit"],
[["capabilities", "--json", "--token-offset="], "--token-offset"],
[["capabilities", "--format"], "--format"],
] as const) {
const result = await cli.run({ arguments: arguments_ });
expect(result.exitCode).toBe(1);
expect(result.json).toBeDefined();
expect(result.stdout).not.toContain("[truncated:");
expect(JSON.stringify(result.json)).not.toContain(
"UNSUPPORTED_OUTPUT_COMBINATION",
);
expect(result.json).toMatchObject({
ok: false,
error: {
code: "MISSING_OUTPUT_VALUE",
message: expect.stringContaining(`${flag} requires a value`),
},
});
}
const equalsForms = await cli.run({
arguments: [
"capabilities",
"--format=json",
"--filter-output=capabilities[0,1]",
],
});
expect(equalsForms.exitCode).toBe(0);
expect(equalsForms.json).toMatchObject({
capabilities: [expect.any(Object)],
});
const equalsTokenWindow = await cli.run({
arguments: ["capabilities", "--json", "--token-limit=5"],
});
expect(equalsTokenWindow.exitCode).toBe(1);
expect(equalsTokenWindow.json).toMatchObject({
ok: false,
error: { code: "UNSUPPORTED_OUTPUT_COMBINATION" },
});
const validTokenWindow = await cli.run({
arguments: ["capabilities", "--json", "--token-limit", "5"],
});
tests/boundary/cli/dispatcher.test.ts
+1 -1
View File
@@ -29,7 +29,7 @@ describe("executable dispatcher", () => {
),
workspace.write(
"dist/cli-runtime/cliOutput.js",
"export const validateCliOutputArguments = () => ({ ok: true }); export const renderCliOutputArgumentError = () => ''; export const renderEmptyFilteredCliOutput = () => undefined;\n",
"export const normalizeCliOutputArguments = (args) => args; export const validateCliOutputArguments = () => ({ ok: true }); export const renderCliOutputArgumentError = () => ''; export const renderEmptyFilteredCliOutput = () => undefined;\n",
),
workspace.write(
"dist/cli-runtime/cliResultOutput.js",
tests/boundary/cli/doctorHopperDiscovery.test.ts
+1
View File
@@ -45,6 +45,7 @@ cliTest.skipIf(!["darwin", "linux"].includes(process.platform))(
? {
PATH: [join(home, "bin"), process.env.PATH ?? ""].join(delimiter),
REA_HOPPER_LDD_TEST_MARKER: loaderMarker,
NODE_OPTIONS: `--import=${new URL("../../fixtures/hopperRejectionFeed.mjs", import.meta.url).href}`,
}
: {}),
HOME: home,
tests/boundary/cli/hopperLaunchFailureCli.test.ts
+1 -1
View File
@@ -5,7 +5,7 @@ import { z } from "zod";
import { workspaceCliTest } from "../../support/cli/workspaceCliFixture.js";
import { acquireHopperTargetLease } from "../../../src/hopper/HopperTargetLease.js";
import { acquireHopperTargetLease } from "../../../src/process/HopperLease.js";
const FAILED_LAUNCHER = `#!/bin/sh
printf 'Cannot launch Hopper!\\nDocument state is unknown.\\n' >&2
tests/boundary/cli/nativeDispatchPropertyAttributes.test.ts
+114
View File
@@ -0,0 +1,114 @@
import { writeFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, expect } from "vitest";
import { machoWithPropertyAttributes } from "../../fixtures/objcPropertyMacho.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { cliTest } from "../../support/cli/cliFixture.js";
describe.skipIf(process.platform !== "darwin")(
"native dispatch property attributes",
() => {
cliTest.for([
["complete nonatomic", "Tq,N", "nonatomic", false, "complete"],
["complete default", 'T@"NSString",C', "atomic", false, "complete"],
["unfinished quote", 'T@"NSString', null, null, "partial"],
["unfinished quote after R", 'R,T@"NSString', null, true, "partial"],
] as const)(
"reports established property flags only for $0 attributes",
async ([, attributes, atomicity, readonly, facet], { cli }) => {
const directory = await createTestTempDirectory("rea-objc-property-");
const path = join(directory, "fixture.bin");
await writeFile(path, machoWithPropertyAttributes(attributes));
const output = await cli.run({
arguments: ["inspect-native-dispatch-metadata", path, "--json"],
environment: { REA_LOG_LEVEL: "silent" },
});
expect(output.exitCode).toBe(0);
const property = {
atomicity,
is_readonly: readonly,
attributes: expect.arrayContaining([
{ name: "T", value: expect.any(String) },
]),
};
expect(output.json).toMatchObject({
normalized_result: {
result: {
objc_classes: [
{
name: "Fixture",
properties: [
{ name: "first", ...property },
{ name: "second", ...property },
],
},
],
coverage: expect.arrayContaining([
expect.objectContaining({
facet: "objc_properties_categories",
status: facet,
reason:
facet === "partial"
? expect.stringContaining(
"property_attributes_incomplete",
)
: null,
}),
]),
},
},
});
},
);
},
);
describe.skipIf(process.platform !== "darwin")(
"native dispatch property record isolation",
() => {
cliTest.for([
[
"an unmapped name pointer",
{ secondNameOffset: 0x2000 },
"first",
"0x100000518",
],
["an empty name", { firstName: "" }, "second", "0x100000508"],
] as const)(
"retains the valid sibling of a property with $0",
async ([, layout, kept, location], { cli }) => {
const directory = await createTestTempDirectory("rea-objc-property-");
const path = join(directory, "fixture.bin");
await writeFile(path, machoWithPropertyAttributes("Tq,N", layout));
const output = await cli.run({
arguments: ["inspect-native-dispatch-metadata", path, "--json"],
environment: { REA_LOG_LEVEL: "silent" },
});
expect(output.exitCode).toBe(0);
expect(output.json).toMatchObject({
normalized_result: {
result: {
objc_classes: [
{
name: "Fixture",
properties: [{ name: kept, atomicity: "nonatomic" }],
},
],
coverage: expect.arrayContaining([
expect.objectContaining({
facet: "objc_properties_categories",
status: "partial",
reason: expect.stringContaining(
`property_decode_failed at ${location}: Fixture: property`,
),
}),
]),
},
},
});
},
);
},
);
tests/boundary/cli/nativePlistCli.test.ts
+4
View File
@@ -96,6 +96,8 @@ describe.skipIf(process.platform !== "darwin")(
"<key>CFBundleIdentifier</key><string>com.example.types</string>",
"<key>Blob</key><data>AAEC</data>",
"<key>Built</key><date>2020-01-01T00:00:00Z</date>",
// plutil writes years past 9999 unsigned, which the decoder rejects.
"<key>Far</key><date>11507-08-17T05:20:00Z</date>",
"<key>Ratio</key><real>nan</real>",
"<key>Nested</key><array><dict>",
"<key>Ratio</key><real>0.5</real>",
@@ -130,6 +132,7 @@ describe.skipIf(process.platform !== "darwin")(
$plist_type: "date",
iso8601: "2020-01-01T00:00:00.000Z",
},
Far: { $plist_type: "date", iso8601: null },
Ratio: { $plist_type: "real", value: null },
Nested: [
{
@@ -148,6 +151,7 @@ describe.skipIf(process.platform !== "darwin")(
limitations: [
expect.stringContaining("XML conversion"),
expect.stringContaining("2 non-finite real"),
expect.stringContaining("1 date value(s)"),
],
},
});
tests/boundary/cli/referenceSourceDirectoryImports.test.ts
+36 -1
View File
@@ -19,6 +19,7 @@ describe.skipIf(process.platform === "win32")(
"rea-reference-directory-imports-",
);
const root = join(workspace, "source");
const outside = join(workspace, "outside", "index.js");
const imports = [
["slash.cjs", "./lib/", "lib/index.js"],
["dot.cjs", "./lib/.", "lib/index.js"],
@@ -29,11 +30,18 @@ describe.skipIf(process.platform === "win32")(
["direct.cjs", "./lib.js", "lib.js"],
["fallback.cjs", "./fallback", "fallback/index.js"],
["legacy-order.cjs", "./dep", "dep/index.js"],
["bare-index.cjs", "./bare", "bare.js"],
["package-main.cjs", "./pkg", "pkg/actual.js"],
["package-main-fallback.cjs", "./stale", "stale/index.js"],
["package-main-slash.cjs", "./slashed", "slashed/lib.js"],
] as const;
const unresolved = [
["missing.cjs", "./missing/", "missing/"],
["escape.cjs", "../outside/", "../outside/"],
["main.ts", "./folder.js/.", "folder.js"],
["only-bare.cjs", "./only/", "only/"],
["invalid-manifest.cjs", "./broken", "broken"],
["absolute-main.cjs", "./absolute", outside],
] as const;
const files = new Map<string, string>([
["index.js", 'module.exports = "root index";\n'],
@@ -48,6 +56,25 @@ describe.skipIf(process.platform === "win32")(
'export const value = "emitted filename counterpart";\n',
],
["missing.js", 'module.exports = "not a directory";\n'],
["bare.js", 'module.exports = "file before directory";\n'],
["bare/index", 'module.exports = "extensionless index";\n'],
["only/index", 'module.exports = "extensionless index";\n'],
// A package main outranks the index; a missing main falls back to it.
["pkg/package.json", '{"main":"actual.js"}\n'],
["pkg/actual.js", 'module.exports = "declared main";\n'],
["pkg/index.js", 'module.exports = "ignored index";\n'],
["stale/package.json", '{"main":"gone.js"}\n'],
["stale/index.js", 'module.exports = "legacy fallback";\n'],
["broken/package.json", '{"main":\n'],
["broken/index.js", 'module.exports = "unknown selection";\n'],
// Node resolves main with path.resolve: no trailing-slash directory
// intent, and an absolute main leaves the imported root.
["slashed/package.json", '{"main":"lib/"}\n'],
["slashed/lib.js", 'module.exports = "file main";\n'],
["slashed/lib/index.js", 'module.exports = "not selected";\n'],
["slashed/index.js", 'module.exports = "not selected";\n'],
["absolute/package.json", `${JSON.stringify({ main: outside })}\n`],
["absolute/index.js", 'module.exports = "not selected";\n'],
...[...imports, ...unresolved].map(
([path, specifier]) =>
[path, `require(${JSON.stringify(specifier)});\n`] as const,
@@ -60,7 +87,6 @@ describe.skipIf(process.platform === "win32")(
await writeFile(target, contents);
}),
);
const outside = join(workspace, "outside", "index.js");
await mkdir(dirname(outside), { recursive: true });
await writeFile(outside, 'module.exports = "outside import root";\n');
@@ -75,9 +101,18 @@ describe.skipIf(process.platform === "win32")(
expect(() =>
createRequire(join(root, "main.ts")).resolve("./folder.js/."),
).toThrow();
expect(() =>
createRequire(join(root, "only-bare.cjs")).resolve("./only/"),
).toThrow();
expect(() =>
createRequire(join(root, "invalid-manifest.cjs")).resolve("./broken"),
).toThrow();
expect(
createRequire(join(root, "escape.cjs")).resolve("../outside/"),
).toBe(outside);
expect(
createRequire(join(root, "absolute-main.cjs")).resolve("./absolute"),
).toBe(outside);
const result = await cli.run({
arguments: ["import-reference-source", root, "--json"],
tests/boundary/filesystem/interfaceBuilderBundle.test.ts
+27
View File
@@ -961,6 +961,33 @@ describe("bounded Interface Builder archive decoding", () => {
);
});
it("decodes an archive whose date the decoder cannot represent", async () => {
const root = await createTestTempDirectory("rea-ib-test-");
const bundle = join(root, "Example.app");
const resources = join(bundle, "Contents", "Resources");
await mkdir(resources, { recursive: true });
// plutil writes years past 9999 unsigned, which the decoder rejects.
await writeFile(
join(resources, "Far.nib"),
'<?xml version="1.0"?><plist version="1.0"><dict><key>$archiver</key><string>NSKeyedArchiver</string><key>$objects</key><array><string>$null</string><date>11507-08-17T05:20:00Z</date></array><key>$top</key><dict/></dict></plist>',
);
const result = await analyzeInterfaceBuilderBundle({
bundlePath: bundle,
targetSha256: "f".repeat(64),
});
expect(result.documents).toHaveLength(1);
expect(result.limitations).toContainEqual(
expect.stringMatching(
/^Contents\/Resources\/Far\.nib: 1 date value\(s\)/u,
),
);
expect(result.limitations).not.toContainEqual(
expect.stringContaining("could not be decoded"),
);
});
it.skipIf(!xcodeIbtoolAvailable)(
"decodes an Xcode-compiled storyboard NIB and recovers its UI routes",
async () => {
tests/boundary/filesystem/javascript-applications/javascriptApplicationEmptyLiterals.test.ts
+3 -1
View File
@@ -148,7 +148,9 @@ it("analyzes a plain long-line loader without package metadata", async () => {
for (const node of nodes) {
expect(node.identity).toMatchObject({
strategy: "artifact-local-key",
key: node.kind === "endpoint" ? literalEndpointKey("") : "",
// A Worker string is relative to the unestablished document base, so
// its identity is the retained expression rather than a specifier.
key: node.kind === "endpoint" ? literalEndpointKey("") : 'new Worker("")',
});
expect(node.observations.map(({ label }) => label)).toEqual([null]);
}
tests/boundary/filesystem/javascript-applications/javascriptApplicationTemplateLiterals.test.ts
+1 -1
View File
@@ -55,7 +55,7 @@ const graphIdentities = async (source: string) => {
it.each([
"fetch('https://api.example.com/v1/users');",
"localStorage.setItem('token', 1);",
"new Worker('./worker.js');",
"new Worker(new URL('./worker.js', import.meta.url));",
"const dep = require('./dep.js'); dep();",
"import('./dep.js');",
])(
Loaded 100 of 148 files, more files were not shown because too many files have changed in this diff. Show more