-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · **فارسی**
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · **فارسی** · [Bahasa Indonesia](README_id.md)
# REA: مهندسی معکوس هر چیزی
@@ -241,7 +241,7 @@ npx rea-agents@latest setup
## تاریخچهٔ ستارهها
-🎉 **۴۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
+🎉 **۵۰٬۰۰۰ ستارهٔ GitHub — سپاسگزاریم!**
از همهٔ کسانی که از REA استفاده میکنند، خطاها را گزارش میدهند، قابلیتهای جدید پیشنهاد میکنند، buildها را آزمایش میکنند و در بهبود پروژه مشارکت دارند سپاسگزاریم.
diff --git a/README_fr.md b/README_fr.md
index 8fa07803b9..221e0ed68e 100644
--- a/README_fr.md
+++ b/README_fr.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · **Français** · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · **Français** · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA : rétro-ingénierie de tout
@@ -257,7 +257,7 @@ Signalez les vulnérabilités via [SECURITY.md](SECURITY.md).
## Historique des étoiles
-🎉 **40 000 étoiles GitHub — merci !**
+🎉 **50 000 étoiles GitHub — merci !**
Merci à toutes les personnes qui utilisent REA, signalent des bugs, proposent des fonctionnalités, testent les builds et contribuent des correctifs.
diff --git a/README_id.md b/README_id.md
new file mode 100644
index 0000000000..77c367a15f
--- /dev/null
+++ b/README_id.md
@@ -0,0 +1,307 @@
+
+
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · **Bahasa Indonesia**
+
+# REA: Rekayasa Balik Apa Pun
+
+### Satu MCP untuk rekayasa balik pada berbagai biner, aplikasi, dan perilaku saat runtime.
+
+**Melihat fitur yang Anda sukai. Pahami cara kerjanya hingga ke tingkat biner.**
+
+[](https://www.npmjs.com/package/rea-agents)
+[](https://github.com/morluto/rea/actions/workflows/ci.yml)
+[](docs/mcp-contracts.md#generated-catalog)
+[](#current-status)
+[](https://skills.sh/morluto/rea/reverse-engineer-anything)
+[](LICENSE)
+[](https://discord.gg/GkcryMnJDM)
+
+
+
+**[Situs Web](https://rea.tools/) · [Panduan](https://rea.tools/guides/) · [Contoh Penggunaan](https://rea.tools/showcase/)**
+
+[Mulai Cepat](#mulai-cepat) · [Cara Kerja REA](#cara-kerja-rea) · [Hal yang Dapat Anda Analisis](#hal-yang-dapat-anda-analisis) · [Contoh Penggunaan](#contoh-penggunaan) · [FAQ](#faq) · [Dokumentasi](#dokumentasi)
+
+
npx rea-agents setup
+
+
+
+
+
+
+
+
+
+
+
+
+
+---
+
+Melihat fitur dalam suatu aplikasi yang ingin Anda hadirkan di produk sendiri? Minta agen Anda menyelidikinya menggunakan REA. REA dapat memeriksa aplikasi tanpa kode sumbernya, menjelaskan cara kerja fitur tersebut, menunjukkan bukti, dan membangun versi untuk proyek Anda.
+
+REA menghubungkan agen Anda dengan berbagai alat untuk memeriksa biner native, aplikasi JavaScript dan Electron, assembly .NET, serta situs web. Anda juga dapat menggunakan alat yang sama dari terminal. Analisis dijalankan secara lokal, dan hasilnya mencakup bukti serta keterbatasan yang mendasari setiap kesimpulan.
+
+Proses penyiapan mendaftarkan REA ke agen Anda dan memasang instruksi alur kerja yang sesuai. Analisis native dapat menggunakan instalasi Hopper, Ghidra, atau IDA yang sudah ada. Proses penyiapan juga dapat memasang Hopper secara opsional setelah mendapatkan persetujuan. Analisis JavaScript statis tidak memerlukan mesin analisis native.
+
+> **[Kunjungi situs web REA](https://rea.tools/)** untuk mendapatkan petunjuk penyiapan, panduan bergambar, dan studi kasus nyata.
+
+## Mulai Cepat
+
+### Siapkan agen Anda
+
+Dengan Node.js dan npm yang telah terpasang, jalankan:
+
+```bash
+npx rea-agents setup
+```
+
+Pilih agen yang ingin digunakan, tinjau perubahan yang diusulkan, lalu setujui perubahan tersebut. Proses penyiapan menambahkan server MCP REA dan instruksi alur kerja yang sesuai, serta membuat cadangan konfigurasi yang sudah ada. Setelah itu, mulai ulang agen Anda.
+
+Penyiapan mendukung Claude Code, Codex, Cursor, Gemini CLI, Grok Build, dan [agen lainnya](docs/installation.md#supported-agents). Lihat [instalasi dan penyiapan](docs/installation.md) untuk konfigurasi penyedia dan pendaftaran MCP secara manual.
+
+### Minta agen Anda
+
+```text
+Pahami cara kerja pencarian di aplikasi Notes, tunjukkan buktinya, dan buat fitur serupa untuk proyek saya.
+```
+
+Ganti Notes dengan aplikasi target dan fitur yang ingin Anda pahami.
+
+### Gunakan terminal
+
+Periksa direktori aplikasi JavaScript/Electron yang telah diekstrak atau berkas ASAR:
+
+```bash
+npx -y rea-agents@latest analyze-javascript-application /absolute/path/to/app --json
+```
+
+Hasilnya mencakup modul, impor, batas antarkomponen Electron, dan bukti yang mendasarinya. Ganti jalur tersebut dengan lokasi target Anda, misalnya `"D:/apps/example"` di Windows.
+
+Untuk memasang perintah `rea` agar dapat digunakan secara rutin:
+
+```bash
+npm install --global rea-agents
+rea --help
+```
+
+Untuk analisis native, konfigurasikan penyedia terlebih dahulu. Lihat [Panduan CLI dan Bukti](docs/cli.md) untuk perintah native, pemilihan penyedia, snapshot, dan pembuatan skrip.
+
+### Perbarui REA
+
+REA berkembang dengan cepat, dan rilis baru sering menyertakan perbaikan bug. Pastikan instalasi Anda selalu diperbarui.
+
+Untuk CLI yang dipasang melalui npm:
+
+```bash
+rea update
+```
+
+Untuk memperbarui pendaftaran agen dan skill Anda, jalankan perintah penyiapan yang ditampilkan oleh proses pembaruan.
+
+Jika Anda menggunakan `npx`, perbarui penyiapan agen dengan:
+
+```bash
+npx rea-agents@latest setup
+```
+
+Tinjau perubahan penyiapan dan mulai ulang agen Anda. Untuk menjalankan perintah CLI sekali pakai, gunakan `npx rea-agents@latest` diikuti dengan perintah yang diinginkan.
+
+## Cara Kerja REA
+
+Agen Anda memanggil REA melalui MCP untuk memeriksa target dan menelusuri kode yang relevan. REA mengembalikan temuan beserta bukti pendukungnya. Agen kemudian menggunakan temuan tersebut untuk mengajukan pertanyaan lanjutan, menjelaskan perilaku, atau menulis dan menguji implementasi.
+
+Perintah CLI menggunakan alur kerja yang sama.
+
+
+
+[Buka ilustrasi dalam ukuran penuh](website/public/assets/figures/rea-investigation-flow.svg).
+
+
+
+## Hal yang Dapat Anda Analisis
+
+REA memerlukan Node.js 22.x (>=22.19), 24.x (>=24.11), atau 26+, beserta npm.
+
+Alat tambahan dan dukungan sistem host bergantung pada target yang dianalisis:
+
+| Target | Hasil yang diberikan REA | Persyaratan dan panduan |
+| -------------------------- | ----------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
+| Biner native | Pseudocode, assembly, string, simbol, pemanggilan, dan referensi | Hopper, Ghidra, atau IDA; [analisis native](https://rea.tools/guides/native/) |
+| Tata letak ELF offline | Bagian, segmen, simbol/relokasi asli, dan kandidat mitigasi statis | pwntools yang disediakan pemanggil pada Linux x64; [diagnostik biner](docs/binary-diagnostics.md) |
+| Bytecode EVM | Selector dispatch, offset byte, argumen yang disimpulkan, dan sifat mutabilitas | Pembawa data raw/hex lokal; [panduan EVM offline](docs/evm-bytecode.md) |
+| Crash Linux yang direkam | Catatan mentah, register/sinyal setiap thread yang direkam, dan kandidat pemetaan opsional | pwntools yang disediakan pemanggil; GDB/pwndbg opsional; [crash yang direkam](docs/recorded-crashes.md) |
+| JavaScript / Electron | Modul, impor, source map, rute, IPC, dan hubungan add-on native | Node.js dan npm; [analisis aplikasi](https://rea.tools/guides/javascript/) |
+| Situs web | Struktur halaman, skrip, pengamatan jaringan, dan tangkapan layar yang diminta | Peramban berbasis Chrome; [analisis peramban](https://rea.tools/guides/browser/) |
+| Rekaman jaringan tersimpan | Permintaan, respons, payload yang terekspos, dan lokasi sumber | HAR; mitmdump di Linux untuk rekaman native mitmproxy; [panduan rekaman](docs/web-network-captures.md) |
+| Assembly .NET | Metadata, instruksi CIL, dependensi native yang dideklarasikan, dan perbandingan build | Tidak ada (hanya analisis statis); [panduan kode terkelola](docs/managed-code-analysis.md) |
+| APK Android | Deklarasi manifest, kelas, metode hasil dekompilasi, dan referensi | JADX mode headless dan JDK lengkap di Linux/macOS/Windows x64; [panduan Android](docs/android-analysis.md) |
+| Firmware | Wilayah, hasil ekstraksi, dan pengalihan ke analisis native | Binwalk / Unblob di Linux; [panduan firmware](docs/firmware-analysis.md) |
+| Paket dan sumber daya | Inventaris berkas, digest, plist, struktur paket Apple, dan sumber daya yang diekstrak | [Panduan artefak dan JavaScript](docs/javascript-artifact-reconstruction.md), [aplikasi Apple](docs/apple-application-analysis.md) |
+| Perilaku proses | Keluaran terminal, interaksi, pengamatan proses keluar dan sistem berkas, serta perbandingan eksekusi | Linux/macOS dengan PTY native; [perekaman proses](docs/process-capture.md) |
+
+Pemeriksaan JavaScript dan .NET statis membaca berkas yang disediakan tanpa menjalankan aplikasinya. Perekaman runtime menjalankan atau berinteraksi dengan target yang dipilih menggunakan izin pengguna Anda. Setiap panduan runtime menjelaskan dampaknya.
+
+
+
+Format native dan dukungan sistem host berbeda-beda menurut penyedia. Lihat [penyiapan Hopper dan Ghidra](docs/installation.md#hopper), [panduan IDA](docs/ida-provider.md), dan [dukungan eksperimental Ghidra di Windows](docs/windows-ghidra-p0.md).
+
+Ghidra juga mendukung [analisis DOS 16-bit](docs/ghidra-dos.md).
+
+Untuk biner berukuran besar, tingkatkan batas waktu startup menggunakan `REA_GHIDRA_STARTUP_TIMEOUT_MS`.
+
+Untuk memilih penyedia, lihat [panduan CLI](docs/cli.md#choose-a-provider).
+
+Periksa [ketersediaan rilis](docs/installation.md#released-package-and-main) untuk mengetahui fitur yang ditambahkan setelah rilis npm terbaru.
+
+## Contoh Penggunaan
+
+[](https://rea.tools/showcase/)
+
+### DX-Ball: merekonstruksi perhitungan sound-pan
+
+Ikuti pemanggilan suara hingga ke fungsi pembantu yang mengubah posisi menjadi nilai pan, periksa instruksinya, lalu ubah pseudocode yang belum lengkap menjadi C. Hasil rekonstruksi tersebut lolos pada 3.205 kasus x86 asli dan mereproduksi seluruh 63 byte fungsi hasil kompilasi.
+
+[Baca studi kasus](https://rea.tools/showcase/dx-ball/) · [Repositori rekonstruksi](https://github.com/N0zoM1z0/dx-ball)
+
+### Notion: menelusuri jembatan clipboard Electron
+
+Temukan API clipboard pada renderer, telusuri jalurnya melalui preload dan IPC hingga ke proses utama, lalu periksa format clipboard kaya.
+
+[Baca studi kasus](https://rea.tools/showcase/notion/)
+
+### TH04: memulihkan perhitungan bullet-ring DOS
+
+Periksa instruksi 16-bit dari gim PC-98 asli, pulihkan perhitungan sudut tetap dan sudut yang diarahkan, lalu bandingkan C++ hasil rekonstruksi dengan keluaran historis kompiler.
+
+[Baca studi kasus](https://rea.tools/showcase/th04/) · [Repositori rekonstruksi](https://github.com/N0zoM1z0/th04)
+
+Jika Anda telah menggunakan REA untuk menganalisis sesuatu yang menarik, kami ingin melihat hasilnya. Bagikan kasus Anda melalui [issue](https://github.com/morluto/rea/issues) atau [pull request](https://github.com/morluto/rea/pulls), sertakan target, pertanyaan Anda, bagaimana REA membantu, dan apa yang Anda temukan.
+
+## FAQ
+
+
+Agen apa saja yang dapat menggunakan REA?
+
+
+Apakah saya memerlukan Hopper, Ghidra, atau IDA?
+
+
+Apakah saya harus menjalankan Hopper terlebih dahulu?
+
+
+Apa fungsi memasang skill dari skills.sh?
+
+
+Kode seperti apa yang dihasilkan REA?
+
+
+Apakah REA mengunggah aplikasi saya?
+
+
+Apa yang harus saya lakukan jika menemukan bug?
+
+## Dokumentasi
+
+Mulailah dengan [panduan praktik di situs web](https://rea.tools/guides/).
+
+Untuk mengetahui opsi, prasyarat, dan kontrak hasil secara tepat:
+
+- [Instalasi dan penyiapan](docs/installation.md): pendaftaran agen, konfigurasi penyedia, pembaruan, dan penghapusan instalasi.
+- [Kesiapan dan pemecahan masalah](docs/installation.md#check-readiness-for-your-task): mendiagnosis satu agen atau mesin analisis.
+- [CLI dan Bukti](docs/cli.md): perintah, pemilihan penyedia, snapshot, impor/ekspor, dan status keluar.
+- [Kontrak MCP](docs/mcp-contracts.md) dan [prompt agen](docs/mcp-prompts.md): hasil alat, sesi, dan investigasi terpandu.
+- [Katalog alat](docs/mcp-contracts.md#generated-catalog): inventaris alat, penyedia, dan perintah CLI yang dibuat secara otomatis.
+- [Peta jalan](docs/roadmap.md): pekerjaan yang direncanakan dan pelacak kemampuan.
+
+Laporkan kerentanan melalui [SECURITY.md](SECURITY.md).
+
+## Riwayat Bintang
+
+🎉 **50.000 bintang GitHub, terima kasih!**
+
+Terima kasih kepada semua orang yang menggunakan REA, melaporkan bug, meminta fitur, menguji build, dan berkontribusi dalam perbaikan.
+
+
+
+
+
+
+## Penafian
+
+REA menyediakan alat untuk penelitian, analisis, dan rekonstruksi rekayasa balik yang sah. Anda bertanggung jawab untuk memperoleh otorisasi yang diperlukan dan mematuhi hukum yang berlaku. Proyek ini tidak mendukung penggunaan ilegal atau tanpa izin.
+
+REA adalah proyek perangkat lunak sumber terbuka. Kami tidak menerbitkan atau mendukung mata uang kripto maupun token apa pun. Token yang menggunakan nama REA tidak berafiliasi dengan proyek ini.
+
+## Kontribusi
+
+Kami sangat menghargai bantuan Anda untuk REA! [Buat issue](https://github.com/morluto/rea/issues) untuk melaporkan bug atau menyarankan fitur, atau [kirim pull request](https://github.com/morluto/rea/pulls) untuk meningkatkan kode atau dokumentasi.
+
+Lihat [CONTRIBUTING.md](CONTRIBUTING.md) untuk penyiapan pengembangan dan pemeriksaan, [pengujian](docs/testing.md) untuk tahapan verifikasi, serta [peta arsitektur](docs/architecture.mermaid) untuk struktur proyek.
+
+## Lisensi
+
+[MIT](LICENSE)
diff --git a/README_ja.md b/README_ja.md
index b827c3c040..4628bdbe2f 100644
--- a/README_ja.md
+++ b/README_ja.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · **日本語** · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · **日本語** · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA：あらゆるものをリバースエンジニアリング
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
## スター履歴
-🎉 **GitHub スター 40,000 件、ありがとうございます！**
+🎉 **GitHub スター 50,000 件、ありがとうございます！**
REA の利用、バグ報告、機能の要望、ビルドのテスト、修正への貢献に感謝します。
diff --git a/README_ko.md b/README_ko.md
index 8767920595..0ec6ab83ba 100644
--- a/README_ko.md
+++ b/README_ko.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · **한국어** · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · **한국어** · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA: 무엇이든 리버스 엔지니어링
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
## 스타 기록
-🎉 **GitHub 스타 40,000개, 감사합니다!**
+🎉 **GitHub 스타 50,000개, 감사합니다!**
REA를 사용하고, 버그를 보고하고, 기능을 요청하고, 빌드를 테스트하고, 수정에 기여해 주신 모든 분께 감사드립니다.
diff --git a/README_pl.md b/README_pl.md
index bdd062da9b..c82e7cd381 100644
--- a/README_pl.md
+++ b/README_pl.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · **Polski** · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · **Polski** · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA: Reverse Engineer Anything
@@ -256,7 +256,7 @@ Zgłaszaj podatności zgodnie z [SECURITY.md](SECURITY.md).
## Historia gwiazdek
-🎉 **40 000 gwiazdek na GitHubie — dziękujemy!**
+🎉 **50 000 gwiazdek na GitHubie — dziękujemy!**
Dziękujemy wszystkim, którzy używają REA, zgłaszają błędy, proponują nowe funkcje, testują kompilacje i przesyłają poprawki.
diff --git a/README_pt-BR.md b/README_pt-BR.md
index 711f7d7881..5c45cd33fb 100644
--- a/README_pt-BR.md
+++ b/README_pt-BR.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · **Português (Brasil)** · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · **Português (Brasil)** · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA: Faça engenharia reversa de qualquer coisa
@@ -256,7 +256,7 @@ Relate vulnerabilidades conforme [SECURITY.md](SECURITY.md).
## Histórico de estrelas
-🎉 **40.000 estrelas no GitHub — muito obrigado!**
+🎉 **50.000 estrelas no GitHub — muito obrigado!**
Obrigado a todos que usam o REA, relatam bugs, sugerem funcionalidades, testam builds e contribuem com correções.
diff --git a/README_ru.md b/README_ru.md
index 6b44940e96..4c474a46fd 100644
--- a/README_ru.md
+++ b/README_ru.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · **Русский** · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · **Русский** · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA: Реверс-инжиниринг чего угодно
@@ -256,7 +256,7 @@ npx rea-agents@latest setup
## История звёзд
-🎉 **40 000 звёзд на GitHub — спасибо!**
+🎉 **50 000 звёзд на GitHub — спасибо!**
Спасибо всем, кто использует REA, сообщает об ошибках, предлагает новые функции, тестирует сборки и помогает с исправлениями.
diff --git a/README_th.md b/README_th.md
index c66e1a3932..b5310a77d2 100644
--- a/README_th.md
+++ b/README_th.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · **ไทย** · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · **ไทย** · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA: Reverse Engineer Anything
@@ -256,7 +256,7 @@ npx rea-agents@latest setup
## ประวัติดาว
-🎉 **40,000 ดาวบน GitHub — ขอบคุณทุกคน!**
+🎉 **50,000 ดาวบน GitHub — ขอบคุณทุกคน!**
ขอบคุณทุกคนที่ใช้ REA รายงานบั๊ก เสนอฟีเจอร์ ทดสอบบิลด์ และช่วยแก้ไข
diff --git a/README_tr.md b/README_tr.md
index ff5ae32399..717dc88f28 100644
--- a/README_tr.md
+++ b/README_tr.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · **Türkçe** · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · **Türkçe** · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA: Her şeyi tersine mühendislikle inceleyin
@@ -256,7 +256,7 @@ Güvenlik açıklarını [SECURITY.md](SECURITY.md) üzerinden bildirin.
## Yıldız geçmişi
-🎉 **GitHub'da 40.000 yıldız — teşekkürler!**
+🎉 **GitHub'da 50.000 yıldız — teşekkürler!**
REA'yı kullanan, hata bildiren, özellik talep eden, derlemeleri test eden ve düzeltmelere katkıda bulunan herkese teşekkürler.
diff --git a/README_uk.md b/README_uk.md
index 973ff9b417..f4f81beebf 100644
--- a/README_uk.md
+++ b/README_uk.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · **Українська** · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · **Українська** · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA: Реверс-інжиніринг будь-чого
@@ -256,7 +256,7 @@ npx rea-agents@latest setup
## Історія зірок
-🎉 **40 000 зірок на GitHub — дякуємо!**
+🎉 **50 000 зірок на GitHub — дякуємо!**
Дякуємо всім, хто використовує REA, повідомляє про помилки, пропонує нові функції, тестує збірки й допомагає з виправленнями.
diff --git a/README_vi.md b/README_vi.md
index 6f198f90fc..fd015d858e 100644
--- a/README_vi.md
+++ b/README_vi.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · **Tiếng Việt** · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · **Tiếng Việt** · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA: Kỹ thuật dịch ngược cho mọi thứ
@@ -256,7 +256,7 @@ Báo cáo lỗ hổng theo [SECURITY.md](SECURITY.md).
## Lịch sử sao
-🎉 **40.000 sao trên GitHub — xin cảm ơn!**
+🎉 **50.000 sao trên GitHub — xin cảm ơn!**
Cảm ơn mọi người đã sử dụng REA, báo lỗi, đề xuất tính năng, kiểm thử bản dựng và đóng góp bản sửa lỗi.
diff --git a/README_zh-TW.md b/README_zh-TW.md
index adef7444c9..fd46f28621 100644
--- a/README_zh-TW.md
+++ b/README_zh-TW.md
@@ -1,6 +1,6 @@
-[English](README.md) · [简体中文](README_zh.md) · **繁體中文** · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · [简体中文](README_zh.md) · **繁體中文** · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA：逆向分析一切
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
## Star 歷史
-🎉 **GitHub Star 達到 40,000 個，感謝大家！**
+🎉 **GitHub Star 達到 50,000 個，感謝大家！**
感謝每一位使用 REA、報告錯誤、提出功能需求、測試建置和貢獻修復的朋友。
diff --git a/README_zh.md b/README_zh.md
index baf67a1451..a8208c2cad 100644
--- a/README_zh.md
+++ b/README_zh.md
@@ -1,6 +1,6 @@
-[English](README.md) · **简体中文** · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md)
+[English](README.md) · **简体中文** · [繁體中文](README_zh-TW.md) · [日本語](README_ja.md) · [한국어](README_ko.md) · [Türkçe](README_tr.md) · [Русский](README_ru.md) · [Tiếng Việt](README_vi.md) · [ไทย](README_th.md) · [Deutsch](README_de.md) · [Español](README_es.md) · [Français](README_fr.md) · [Українська](README_uk.md) · [Polski](README_pl.md) · [Português (Brasil)](README_pt-BR.md) · [العربية](README_ar.md) · [فارسی](README_fa.md) · [Bahasa Indonesia](README_id.md)
# REA：逆向分析一切
@@ -258,7 +258,7 @@ npx rea-agents@latest setup
## Star 历史
-🎉 **GitHub Star 达到 40,000 个，感谢大家！**
+🎉 **GitHub Star 达到 50,000 个，感谢大家！**
感谢每一位使用 REA、报告错误、提出功能需求、测试构建和贡献修复的朋友。
diff --git a/docs/android-analysis.md b/docs/android-analysis.md
index efcb05c645..849aeb2b11 100644
--- a/docs/android-analysis.md
+++ b/docs/android-analysis.md
@@ -126,8 +126,13 @@ also stops target hashing before provider admission.
These operations do not execute the APK or provide split APK/AAB handling,
signature validation, full resource-table semantics, native-library analysis or
-Android runtime capture. Existing artifact inventory/extraction tools can supply
-archive evidence. Project that Evidence with the execution-free inventory tool:
+Android runtime capture. An `.aab` suffix is a ZIP archive, not an APK, so
+Android tools and the application graph do not treat it as a standalone APK.
+Existing artifact inventory/extraction tools can supply archive evidence.
+Inventory keeps ZIP names that differ only in case and records that a
+case-insensitive destination cannot store both spellings. Extraction fails
+only when that destination directory cannot keep both names. Project that
+Evidence with the execution-free inventory tool:
```sh
rea project-android-application-graph '{"inventory_evidence":[
]}'
diff --git a/docs/cli.md b/docs/cli.md
index ed5f810e98..245251a286 100644
--- a/docs/cli.md
+++ b/docs/cli.md
@@ -20,6 +20,23 @@ files when neither `--provider` nor `--snapshot` is supplied. See
[JavaScript artifact reconstruction](javascript-artifact-reconstruction.md)
for results, integrity checks and coverage.
+The complete application Evidence can be hundreds of megabytes for a real
+application. When an agent or a person reads the result, save it to a file and
+project a summary, a module page or one module from the saved Evidence:
+
+```bash
+rea analyze-javascript-application /absolute/path/to/app --json > app-evidence.json
+jq -c '{source: {kind: "inline", evidence: .}, view: {kind: "summary"}}' app-evidence.json > app-view.json
+rea inspect-analysis-view app-view.json
+```
+
+For example, a 334 MB Obsidian application Evidence file yields a summary view
+of about 10 KB. Replace the view with
+`{"kind": "page", "collection": "modules", "offset": 0, "limit": 32}` for a
+module page. See
+[JavaScript application workflows](javascript-application-workflows.md) for
+views, feature traces and comparisons that take the same saved Evidence.
+
## Native analysis
Configure [Hopper or Ghidra](installation.md#hopper), or the
@@ -179,7 +196,10 @@ user permissions; consult the chosen guide for host requirements and effects.
## Output and exit status
The default terminal format is TOON. Use `--json` when saving results for a
-JSON consumer. Output selection and formatting do not change operation status.
+JSON consumer. `--json` is indented on a terminal and compact when piped or
+redirected, which keeps large results smaller for agents and scripts; pipe
+through `jq .` for indented files. Output selection and formatting do not change
+operation status.
| Status | Meaning |
| --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
diff --git a/docs/ghidra-provider.md b/docs/ghidra-provider.md
index 0ba061177d..3151ccbb76 100644
--- a/docs/ghidra-provider.md
+++ b/docs/ghidra-provider.md
@@ -62,11 +62,14 @@ Mach-O target coverage. It requires `clang`, LLD, and `lld-link`; set
these tools before compilation. Keeping this matrix separate lets Linux
host/provider acceptance run with only the host compiler.
-`npm run verify:ghidra:windows` and its `-- --x86` variant check the source-owned native x86 and x86-64 PE
-fixture, all 25 read-only operations, target/snapshot/import SHA-256 linkage,
+`npm run verify:ghidra:windows` checks source-owned native x86-64 PE applications;
+`-- --x86` selects x86 and `-- --dll` selects DLLs, and the flags can be combined.
+These lanes check all 25 read-only operations, target/snapshot/import SHA-256 linkage,
and project, endpoint, process, and runtime cleanup. The independent native
lane checks DACLs, handle admission, cancellation, and Job Object lifecycle.
`npm run verify:ghidra:windows:package` packs and installs REA into an isolated
prefix, then checks ordinary-user CLI/MCP operations against canonical contracts.
+It accepts the same architecture and DLL flags and verifies the selected PE role
+in the analysis profile.
All lanes require matching native controls; package startup alone establishes
only package compatibility.
diff --git a/docs/installation.md b/docs/installation.md
index 77fc896d5d..18d4ba165e 100644
--- a/docs/installation.md
+++ b/docs/installation.md
@@ -95,7 +95,7 @@ npx skills add morluto/rea --skill reverse-engineer-anything
This installs agent instructions and bundled references, not REA MCP
registration or analysis engines. Follow the skill's
-[conditional connection guide](https://github.com/morluto/rea/blob/main/skill-src/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
+[conditional connection guide](https://github.com/morluto/rea/blob/main/.agents/skills/reverse-engineer-anything/SKILL.md#connect-only-when-needed).
Working tools can be used immediately. If tools are missing, inspect the current
client's registration with `doctor --client codex --json` (substitute its client
ID), then plan repairs with `setup --client codex --dry-run --json`. Show and
@@ -135,26 +135,95 @@ listed after the table because its connector is not one of these files:
| Antigravity | `antigravity` |
| GitHub Copilot CLI | `copilot_cli` |
| Command Code | `commandcode` |
+| Qwen Code | `qwen_code` |
| VS Code | `vscode` |
| Grok Build | `grok_build` |
| OMP | `omp` |
+| Pi | `pi` |
+| Hermes | `hermes` |
+
+For [Pi](https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/mcp.md),
+setup writes a `type: "stdio"` entry under `mcpServers.rea` in the user-level
+`~/.pi/agent/mcp.json` and installs the bundled skill under `~/.agents/skills`.
+Use `rea setup --client pi --dry-run --json` to inspect the plan, then
+`rea setup --client pi --yes --json` to approve it. Check the registration with
+`rea doctor --client pi --json`. Pi reads `mcp.json` as strict JSON, like its
+own loader: comments, trailing commas, a byte-order mark, or an empty existing
+file are invalid, and setup reports them instead of rewriting the file. Repair
+such a file to plain JSON first, then rerun setup.
+
+Setup follows Pi's `PI_CODING_AGENT_DIR`: unset or empty selects the default;
+`~` and `~/` expand to the home directory; absolute paths and file URLs select
+that directory; relative paths stay relative to the command's working directory.
+On Windows, native paths, `~\`, and Git Bash/MSYS, Cygwin, and WSL drive paths
+are normalized as Pi does. Run REA and Pi from the same working directory when
+using a relative override. Pi has no profiles: `OMP_PROFILE`, `PI_PROFILE`,
+`PI_CONFIG_DIR`, and OMP's `enabledServers`/`disabledServers` lists do not affect
+Pi registration. Doctor accepts an omitted `type` for stdio but reports a
+non-stdio type as invalid; `enabled: false` stays disabled.
+
+REA configures only the user scope. Pi also reads `.pi/mcp.json` in trusted
+projects; a same-name project entry can replace the user entry or override its
+enabled state and exposure. Check that file if the user registration is aligned
+but unavailable in a project. An extension that registers `/mcp` can replace
+Pi's built-in MCP support and use its own configuration instead. Run `/reload`
+in an existing Pi session after changing servers outside the session.
+
+For Qwen Code, setup registers `rea` in the `mcpServers` table of the
+user-level `~/.qwen/settings.json`, or `$QWEN_HOME/settings.json` when configured.
+Setup and doctor follow Qwen Code's tilde and working-directory-relative home
+overrides. Qwen Code also discovers personal skills in `~/.agents/skills`,
+where setup installs the bundled workflow.
+
+Gemini CLI uses `$GEMINI_CLI_HOME/.gemini/settings.json` when that home override
+is nonempty, including its shared personal skills at
+`$GEMINI_CLI_HOME/.agents/skills`. Setup and doctor check MCP allow/exclude
+policies from system defaults, user settings, trusted workspace settings, and
+system overrides. Workspace trust follows `GEMINI_CLI_TRUST_WORKSPACE`,
+`GEMINI_RESTRICTED_MODE`, the effective `security.folderTrust.enabled` setting,
+and the selected `trustedFolders.json` (including
+`GEMINI_CLI_TRUSTED_FOLDERS_PATH`). Settings in an untrusted workspace do not
+participate. A workspace or system `mcpServers.rea` replaces the entire user
+entry; setup stops if the winning definition conflicts with its planned entry.
+Review those files manually: setup never edits administrator/workspace settings
+or grants workspace trust. Trust supplied only by a running IDE is outside this
+standalone CLI check.
+
+Codex treats an unset or empty `CODEX_HOME` as `~/.codex`. Devin honors
+`XDG_CONFIG_HOME` for its `devin/mcp_config.json` on non-Windows systems.
For OMP, setup writes a `type: "stdio"` entry to the user-level
-`~/.omp/agent/mcp.json`. It follows `PI_CONFIG_DIR`, an absolute
-`PI_CODING_AGENT_DIR`, and the profile selected by `OMP_PROFILE` or
+`~/.omp/agent/mcp.json`. It follows `PI_CONFIG_DIR` and
+`PI_CODING_AGENT_DIR` (relative to the setup working directory), and the profile selected by `OMP_PROFILE` or
`PI_PROFILE` (`~/.omp/profiles//agent/mcp.json`). Setup also removes
`rea` from that file's `disabledServers` list, which would otherwise hide the
registration. Doctor treats an `enabled: false` entry as active when
`enabledServers` lists `rea`, as OMP does, unless `disabledServers` also lists
it. Run setup under each profile that should load REA.
+For Hermes, setup writes a `mcp_servers.rea` entry to `config.yaml`, preserving
+comments and unrelated settings. It resolves that file from `HERMES_HOME`, else
+from the platform default Hermes itself uses — `%LOCALAPPDATA%\hermes` on
+Windows, `~/.hermes` elsewhere — including any `HERMES_DATA_DIR_SUFFIX`. Hermes
+connects an entry unless `enabled` is false, so setup states `enabled: true`.
+
+Hermes personal skills install under the active Hermes home's `skills` directory.
+REA follows `HERMES_HOME` whitespace trimming, environment-variable expansion,
+and home expansion before resolving that profile. Setup previews the resolved
+configuration and skill destinations; doctor and uninstall use those same paths.
+
For OpenCode, setup writes the V1 `mcp.rea` entry, which OpenCode V1 and V2
both load. If the configuration already uses OpenCode V2's native
`mcp.servers` table, setup registers REA there instead and replaces any earlier
-`mcp.rea` entry from REA.
+`mcp.rea` entry from REA. Setup honors `OPENCODE_CONFIG_DIR` and
+`OPENCODE_CONFIG`, preferring the later-loaded JSONC file when both JSON and
+JSONC exist. The ordinary XDG global configuration still participates when a
+custom directory is selected. Doctor checks the merged user registration; uninstall removes
+REA-owned entries from every applicable user file so an older entry cannot
+become active again. Malformed applicable files block setup before changes.
Grok Build loads `[mcp_servers.rea]` from `$GROK_HOME/config.toml`, or from
-`~/.grok/config.toml` when `GROK_HOME` is unset. Setup edits that server
+`~/.grok/config.toml` when `GROK_HOME` is unset or empty. Setup edits that server
table, `[mcp_servers.rea.env]`, and a root `disabled_mcp_servers` entry that
names `rea`. It sets `startup_timeout_sec = 30` and leaves every other name
in that list. The shared skill installed under `~/.agents/skills` is already
@@ -194,10 +263,11 @@ save verified paths for an existing Ghidra installation.
The bundled skill is installed where each selected client discovers personal
skills: Claude Code uses `~/.claude/skills` (or
-`$CLAUDE_CONFIG_DIR/skills` when configured), while other supported clients
-use the shared `~/.agents/skills` directory. A mixed selection plans both
-paths. Selecting the skill without a client uses the shared directory. Setup
-leaves existing skill copies in other locations untouched.
+`$CLAUDE_CONFIG_DIR/skills` when configured). Gemini CLI uses `.agents/skills`
+under its selected home; Antigravity 2.0 uses `~/.gemini/config/skills`.
+Other integrations, including Qwen Code, use `~/.agents/skills`.
+A mixed selection plans each distinct destination. Selecting the skill without a client uses the shared
+directory. Setup leaves existing skill copies in other locations untouched.
`doctor --skill --json` verifies the selected copies against the bundled
instructions and references. Consumers should use `identity.skill.state`,
@@ -489,7 +559,7 @@ entry comments atomically and returns refreshed analysis. These session metadata
edits leave executable bytes unchanged and are discarded on close. GUI controls
require Hopper; Windows P0 remains read-only.
-Windows P0 admits native x86 and x86-64 PE applications on fixed local NTFS volumes.
+Windows P0 admits native x86 and x86-64 PE applications and DLLs on fixed local NTFS volumes.
The npm package bundles native Job Object ownership, protected private runtime
DACLs, and handle-based path admission; no separate addon installation is needed.
See the [Windows Ghidra P0 guide](windows-ghidra-p0.md) for verified scope and
diff --git a/docs/javascript-application-workflows.md b/docs/javascript-application-workflows.md
index cb3e1e5e9c..9b509fa604 100644
--- a/docs/javascript-application-workflows.md
+++ b/docs/javascript-application-workflows.md
@@ -90,6 +90,44 @@ keep the result ambiguous. Runtime Evidence can later corroborate an exact
mapped candidate, but structural reachability, semantic influence, runtime
observation, and causal proof remain separate claims.
+Semantic literal nodes keep their complete value in `properties.value`; literal
+queries match that field. String labels use `string literal` when that is
+shorter than the complete JSON value. A literal's `identity.role_key` commits
+its canonical JSON value as `value-sha256:` when the digest key is
+shorter; smaller values keep their existing JSON key. This keeps large values
+out of identity and display metadata without expanding short values or
+truncating their Evidence.
+
+New analyses therefore produce different IDs for hashed literals from the
+earlier payload-bearing role keys. Use IDs returned by the selected parent
+graph and read literal values from `properties.value`, rather than decoding
+role keys or labels. Previously stored graphs and their original IDs remain
+valid inputs.
+
+Endpoint nodes keep the exact endpoint in each observation's `properties.value`;
+semantic request and response nodes keep it in `properties.endpoint`. Endpoint
+identity keys use `value-sha256:` over the canonical JSON string when
+that is shorter, or when the original value starts with the reserved digest
+prefix. Other short endpoint keys remain unchanged. Endpoint labels use the
+shorter of the exact value and ` endpoint`; semantic request/response
+labels use the shorter of the endpoint and operation method. These labels are
+display metadata, not endpoint lookup values.
+
+New analyses therefore change hashed endpoint IDs and longer endpoint labels.
+Use IDs from the selected graph and read the complete endpoint properties;
+previously stored graphs still support their original IDs. Literal endpoint
+search, semantic tracing, and version comparison use the preserved values.
+
+Event and listener nodes likewise preserve the complete event name in
+`properties.event_name`. Event role keys use the same reserved-prefix string
+digest rule, scoped to the emitter; repeated registrations, removals, and
+dispatches keep one event per exact emitter/name pair. Long event labels use
+`event`, and listener labels use the shorter of `:` and
+`:listener`. Dynamic names remain `null` with explicit uncertainty. New analyses
+change hashed event IDs and longer event/listener labels; query the preserved
+name or select IDs from the parent graph. Previously stored graphs keep their
+original IDs and remain valid trace inputs.
+
## Version comparison
REA pairs entities only when a tier produces one unique candidate on each side.
@@ -145,8 +183,13 @@ Direct return expressions, including expression-bodied arrows, are evaluated
through an execution-free value lattice. Literal object fields and direct
return sites are represented in the result. Calls, dynamic spreads, computed
keys, and parser recovery remain partial or unknown.
-Objects and arrays passed to calls, constructors, or tagged templates, or used
-as method receivers, are not assumed unchanged after the invocation. Aliases
+Objects and arrays passed to calls, constructors, or tagged templates, used
+as method receivers, or stored through property targets are not assumed
+unchanged afterward. Property stores conservatively retain reference escape
+uncertainty rather than pretending to execute later writes. References
+handed back by local functions, methods, constructors, async functions, and
+generators also retain escape uncertainty, including when the call is read-only;
+this does not establish that a mutation occurred. Aliases
and shared children in spread and rest copies retain that uncertainty; copied
primitive slots and unrelated containing properties remain known. Object rest
excludes consumed keys, array rest excludes the consumed prefix for known
diff --git a/docs/managed-code-analysis.md b/docs/managed-code-analysis.md
index 74e11e55e0..e899ab6eaa 100644
--- a/docs/managed-code-analysis.md
+++ b/docs/managed-code-analysis.md
@@ -82,6 +82,10 @@ x86-64 ELF and native Windows PE targets analyzed on Linux x64 with Ghidra
See [NativeAOT recovery](ghidra-nativeaot.md) for build/configuration, recovered
metadata, derived-memory provenance, and the real verification lane.
+REA's experimental Windows Ghidra boundary admits native x86 and x86-64 PE
+applications and DLLs for static analysis. This does not establish Windows
+NativeAOT metadata-recovery coverage.
+
For NativeAOT analysis, preserve the selected native artifact's path and digest
and report recovered type data as provider observations. Linking those types
to native function addresses requires verified
diff --git a/docs/mcp-contracts.md b/docs/mcp-contracts.md
index 222aa49052..38161e1bd0 100644
--- a/docs/mcp-contracts.md
+++ b/docs/mcp-contracts.md
@@ -36,6 +36,56 @@ after SDK conversion and check advertised validation and actual calls.
Individual model APIs can impose additional nesting limits; complete producer
captures can exceed ten structural levels.
+### Compact input schema profile
+
+Some function-calling providers reject a request while any single tool's
+`parameters` schema exceeds a literal serialized size cap; Moonshot clients
+such as kimi-cli fail every request while the default catalog is attached
+([#1484](https://github.com/morluto/rea/issues/1484)). Setting the server's
+`REA_MCP_INPUT_SCHEMA_PROFILE` environment variable to `compact` renders every
+advertised input schema within a 13,312-byte budget, measured to sit below
+Moonshot's observed cap (community measurements: a 13.7 KB schema is accepted
+while a ~15 KB one is rejected). The setting is read at startup; restart the
+server to apply changes, as with `REA_MCP_MAX_RESPONSE_BYTES`. The default
+value `full` keeps the complete advertisement unchanged.
+
+The compact profile renders the same tool set, output schemas, annotations,
+and canonical server-side validation. It changes only the advertised input
+JSON Schema:
+
+- Annotation prose leaves the advertised form: nested property descriptions,
+ literal examples, defaults, and titles are dropped. The root keeps its
+ description because it states the accepted input groups.
+- Schema-local references are inlined within the budget. Kimi Code expands
+ references before sending provider requests, so sharing definitions cannot
+ establish the provider-facing size. Expansion stops at the budget before
+ allocating an oversized presentation. Unconstrained JSON fields advertise
+ all JSON value types explicitly so Kimi cannot infer string for Evidence
+ objects or arrays.
+- The few schemas whose validation structure alone exceeds the budget
+ (`compare_web_captures`, `build_reconstruction_obligation_ledger`,
+ `evaluate_reconstruction_coverage`, `project_managed_application_graph`,
+ `compare_application_versions`, `capture_browser_scenario`) are advertised
+ in a reduced form: property names, reference-resolved types (including union
+ alternatives), first-level guidance, and the root's required-field constraints,
+ with nested validation detail elided. The
+ canonical schema still rejects malformed calls server-side, so callers
+ following the reduced advertisement receive the canonical typed error
+ instead of silent acceptance.
+
+Boundary tests pin every advertised compact schema to the budget, keep them
+valid JSON Schema that still accepts every canonical example, and hold the
+set of six reduced presentations as contracts evolve. Client verification
+also checks provider-facing sizes and canonical examples after conversion by
+the actual Kimi Code CLI; local captures do not establish Moonshot acceptance.
+
+Self-contained output schemas advertise a content-bound `$id`, including their
+declared dialect. SDK validators can reuse compiled schemas across complete
+catalog refreshes and equivalent tool outputs; changing the schema changes its
+identity. Explicit schema IDs and relative external reference bases are
+preserved. This keeps every tool and validation rule in discovery and does not
+change the SDK's catalog invalidation or availability checks.
+
`compare_web_captures` accepts exactly one of two input shapes:
- Passive: `before` and `after` each contain `inspection`, the complete
@@ -192,10 +242,27 @@ the current client, so it is not a fresh-start recovery. On the same connection:
This close/reopen flow was exercised on one real Linux stdio connection after a
controlled startup timeout, followed by a successful overview and function
-analysis. It is not a Windows/macOS coverage claim. REA cleans only resources it
-owns and never switches to another provider automatically. A provider timeout,
-installation failure, or host permission denial needs its own reported recovery;
-increasing a client deadline alone does not fix those failures.
+analysis. A separate Windows x64 stdio run with Ghidra 12.1.4, JDK 21 and Node
+22.19.0 exercised the same connection and caller-selected native PE: the default
+60-second client request timed out, close succeeded, and reopening before a
+240-second request completed the cold overview in about 163 seconds. The next
+overview took about 19 ms. The cold and warm overviews reported two procedures
+and two segments; the input digest was unchanged and no owned runtime root or
+observed process-family member remained after final close. This was a small
+1,024-byte fixture, so the longer request setting remains an example, not a
+guarantee for larger targets. No macOS cold-start recovery was exercised.
+
+With a supplied progress token, the Windows run received an operation-start
+notification before timeout and start/completion notifications for the
+successful retry. It received no intermediate import or auto-analysis progress.
+These operation markers do not measure analysis work completed or extend the
+client's deadline. Each new provider session still imports the target into a
+fresh temporary project.
+
+REA cleans only resources it owns and never switches to another provider
+automatically. A provider timeout, installation failure, or host permission
+denial needs its own reported recovery; increasing a client deadline alone does
+not fix those failures.
## Tool results
@@ -243,7 +310,13 @@ Ordinary responses keep their existing complete result contract.
## Retained application Evidence inputs
`inspect_analysis_view` projects a caller-selected view of already completed
-`inspect_binary_layout` or `analyze_javascript_application` Evidence. Source is
+`inspect_binary_layout`, `analyze_javascript_application`, or `analyze_function`
+Evidence. Native views select procedure, pseudocode, assembly, basic blocks,
+comments, callers/callees, references, unresolved calls, referenced strings and
+names, the native API record, and high-pcode facets without starting a provider. Native offset
+and limit default to 0 and 64; pseudocode uses UTF-16 code units and never
+splits surrogate pairs, while other pages count rows. Unavailable facts and
+provider limitations remain explicit. Source is
an exact same-session retained reference or portable inline Evidence. Views are
a summary, a layout mitigations or linkage facet, one section/symbol/module, or
a stable page with a caller-selected positive `limit`. Module pages include
diff --git a/docs/native-investigation.md b/docs/native-investigation.md
index 4dad23b3b1..b3051cc647 100644
--- a/docs/native-investigation.md
+++ b/docs/native-investigation.md
@@ -379,7 +379,7 @@ the tool fails with a `debugger-attach-denied` reason.
Install a Ghidra 12.1.x release and the 64-bit full JDK it declares, then configure REA to
use them. Ghidra analysis supports Linux x64/arm64 and macOS x64/arm64; each installation requires
the matching native decompiler. Experimental Windows x64 P0 admits native
-x86-64 PE applications on local NTFS using bundled Job Object ownership,
+x86 and x86-64 PE applications and DLLs on local NTFS using bundled Job Object ownership,
protected runtime DACLs, and handle-based path admission. See [Windows Ghidra P0](windows-ghidra-p0.md) and
[issue #527](https://github.com/morluto/rea/issues/527).
On Linux and macOS, `annotate_native_function` atomically edits a function name
diff --git a/docs/process-capture.md b/docs/process-capture.md
index 03f17380eb..0bf6d282f3 100644
--- a/docs/process-capture.md
+++ b/docs/process-capture.md
@@ -136,6 +136,23 @@ while comparisons and trace assertions use the normalized text. Retaining the
original adds at most `limits.output_bytes` of text, because it comes from the
same admitted PTY chunks rather than a second unbounded stream.
+Rendered frame `lines` hold the visible rows after normalization, with trailing
+U+0020 spaces removed. Blank rows and internal spaces remain. `columns` records
+terminal cell width; normalization and Unicode can make string lengths differ
+from that width. `serialized_state` holds terminal serialization after the
+selected normalization. Original admitted PTY chunks are available through
+`raw_data ?? data`, as described above. Captures commit this line format in their
+comparison contract, so a capture whose lines were padded to full width does not
+compare as the same contract.
+
+Custom `normalization.patterns` replace literal text with the caller's literal
+replacement, including dollar sequences such as `$&` and `$$`. An empty pattern
+also matches the first and last string positions. Captures with custom patterns
+commit `pattern_normalization_version: "literal-replacements-v1"`. Legacy captures
+remain readable. Cross-version comparison is rejected; capture both sides with
+the same replacement semantics. Captures without custom patterns keep their
+existing comparison contract.
+
Port normalization recognizes explicit `port`, `tcp_port`, `udp_port`, and
`listen` fields, URL authorities, IP endpoints, and `localhost` endpoints, with
ports from 0 through 65535. A port may end a sentence, as in `port: 8080.`;
@@ -185,8 +202,11 @@ Trace assertions use coverage for the sources they select, so an assertion
about complete raw terminal output need not fail because rendered snapshots
were omitted. Captures missing `truncation_details` are rejected on import;
preserve them as historical files and recapture for current comparisons.
-These diagnostics use the existing scenario
-budgets; they do not introduce a separate file-hashing budget.
+`limits.file_bytes` is already the cumulative whole-file hashing budget,
+independent of `limits.output_bytes`, which bounds retained terminal content.
+Each filesystem checkpoint starts with a fresh `file_bytes` budget across all
+selected roots; it is not a per-file allowance. File contents are read in
+64 KiB chunks for hashing and are not embedded in capture Evidence.
When the host withholds an unrelated process’s ownership token, REA leaves that
process untouched and records its PID and reason in `cleanup.unverified_processes`
@@ -202,6 +222,45 @@ as well. A newly started unreadable process can prevent verified cleanup even
after the selected command exits. REA preserves this uncertainty and leaves
that process untouched.
+## Hash a report independently of terminal output
+
+Select the producer's report directory and give `file_bytes` enough room for
+the complete files you need, including other files in those roots. For example,
+save this scenario and run `rea capture-process ./scenario.json --json > capture.json`:
+
+```json
+{
+ "executable": "node",
+ "arguments": [
+ "-e",
+ "const fs=require('node:fs');fs.mkdirSync('reports',{recursive:true});fs.writeFileSync('reports/final.json',JSON.stringify({status:'complete',data:'X'.repeat(131072)}));console.log('x'.repeat(8192));"
+ ],
+ "working_directory": ".",
+ "filesystem_observation_paths": ["./reports"],
+ "limits": { "output_bytes": 1024, "file_bytes": 262144 }
+}
+```
+
+The report exceeds the terminal budget but fits the independent 256 KiB hash
+budget. Its complete SHA-256 appears in `normalized_result.files_after` at
+`root_0:final.json`; check `truncation_details.filesystem_after` for omissions
+and actual bytes hashed. The verbose terminal output is truncated without
+preventing the report digest. The same scenario object is the MCP
+`capture_process_scenario` input. REA never returns a prefix hash as a full-file
+digest: an insufficient remaining budget produces a null digest and an
+explicit omission reason.
+
+A digest binds the complete bytes observed at that checkpoint, not the
+producer's notion of a finished report. A stable periodic or running report
+can also have a complete digest. Choose the final file and verify the
+producer's documented completion marker, exit outcome and settlement
+observations before calling it final. `after_settlement` names a filesystem
+checkpoint, not a guarantee that an application finished successfully or
+that an unobserved writer will never change the file. REA checks opened-file
+identity/state around hashing and reports observed changes or unavailable
+files; cancellation stops the hash. If an external sealer produces a manifest,
+keep its completion claims and digest separate from REA's checkpoint facts.
+
## Compare two captures
Compare saved capture Evidence with:
diff --git a/docs/testing.md b/docs/testing.md
index 9730f027bb..82b529929d 100644
--- a/docs/testing.md
+++ b/docs/testing.md
@@ -88,6 +88,21 @@ entrypoint and include its verifier/fixture files in their cache inputs.
## End-to-end, integration and golden evidence
+The optional `verify:qwen-client`, `verify:pi-client`, and `verify:hermes-client`
+lanes require an installed native client. Select its executable with
+`REA_VERIFY_QWEN_COMMAND`, `REA_VERIFY_PI_COMMAND`, or `REA_VERIFY_HERMES_COMMAND`.
+They configure disposable profiles through the public REA CLI, preserve caller
+settings and backups, and exercise real stdio MCP calls and skill loading. The
+`call` mode checks complete catalog discovery and actual JavaScript Evidence;
+pass `-- chat` to check plain chat with REA enabled. Qwen also reads the full
+result that its client offloads, and Pi exercises default codemode execution.
+Use `REA_VERIFY_RUNTIME_ROOT` to select a production-only installed REA package.
+Receipts retain client versions, result digests, host coverage and owned-process
+cleanup. These POSIX lanes use a deterministic loopback model, so they do not
+prove live model-provider or native Windows compatibility. Qwen and Pi use
+caller-configured skill directories to isolate the fixture account from the OS
+home; this does not prove their default home-directory discovery.
+
Full E2E tests invoke the production command dispatcher and real providers,
without fake launchers, runners or responses. `verify:keyed-archive` writes an
actual Foundation binary archive, runs the CLI and a separate stdio MCP
diff --git a/docs/windows-ghidra-p0.md b/docs/windows-ghidra-p0.md
index ec54f5908a..5f50d832c2 100644
--- a/docs/windows-ghidra-p0.md
+++ b/docs/windows-ghidra-p0.md
@@ -17,7 +17,7 @@ additional permission flag or degraded mode is required.
- Node.js 22.x (>=22.19), 24.x (>=24.11), or 26+;
- an operator-installed Ghidra 12.1.x distribution (verified with 12.1.4);
- a 64-bit full JDK inside that installation's declared Java range (JDK 21 or newer, with no maximum, for current 12.1 releases);
-- an explicit native, non-managed, non-DLL x86 or x86-64 PE application; and
+- an explicit native, non-managed x86 or x86-64 PE application or DLL; and
- the 25 read-only Ghidra inventory, memory, and function-analysis operations.
Loaded memory reads and file offsets preserve Ghidra's observed source mappings.
@@ -38,6 +38,13 @@ REA selects its packaged bridge scripts by explicit path. Same-named files or
directories in the caller's working directory do not select a different
script or require users to clean that directory before analysis.
+Native DLL analysis preserves the PE shared-library role in the analysis profile.
+REA imports an immutable copy for static analysis; it does not load the library
+into the analyst process or execute its entry routine. The source-owned DLL
+fixtures have no entry routine or imports and expose two exported functions.
+The same native snapshot, path-admission and cleanup requirements apply to EXEs
+and DLLs.
+
## Configuration
```powershell
@@ -100,7 +107,7 @@ and artifact SHA-256 before loading the package-owned addon. Native failure
reasons remain distinct from a valid installation and an unsupported target.
PE header classification and SHA-256 come from the same open file. The Windows
-provider admits native x86 and x86-64 PE applications and rejects unsupported roles,
+provider admits native x86 and x86-64 PE applications and DLLs and rejects unsupported roles,
architectures, managed images, and malformed headers. The original selected
source coordinate is preserved for native admission.
@@ -159,7 +166,11 @@ artifact and manifest.
npm run verify:windows-native
npm run verify:ghidra:windows
npm run verify:ghidra:windows -- --x86
+npm run verify:ghidra:windows -- --dll
+npm run verify:ghidra:windows -- --x86 --dll
npm run verify:ghidra:windows:package -- --x86
+npm run verify:ghidra:windows:package -- --dll
+npm run verify:ghidra:windows:package -- --x86 --dll
npm run verify:ghidra:windows:package -- C:\fixtures\installed-rea C:\fixtures\sample.exe
```
diff --git a/package-lock.json b/package-lock.json
index e9bc42c5d9..4b668f7989 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -41,6 +41,7 @@
"stream-json": "3.7.0",
"write-file-atomic": "7.0.0",
"ws": "8.21.0",
+ "yaml": "2.9.0",
"zod": "4.4.3"
},
"bin": {
@@ -64,8 +65,7 @@
"turbo": "2.11.7",
"typescript": "7.0.2",
"vitepress": "1.6.4",
- "vitest": "5.0.3",
- "yaml": "2.9.0"
+ "vitest": "5.0.3"
},
"engines": {
"node": "^22.19.0 || ^24.11.0 || >=26.0.0"
diff --git a/package.json b/package.json
index 69cc9232b9..0c47e59016 100644
--- a/package.json
+++ b/package.json
@@ -179,6 +179,9 @@
"verify:readiness": "npm run build:cached && npm run verify:fixtures && node scripts/verify-reconstruction-readiness.mjs",
"verify:package": "npm run build:cached && node scripts/verify-package.mjs",
"verify:package:windows": "npm run build:cached && node scripts/verify-windows-package.mjs",
+ "verify:qwen-client": "npm run build:cached && node scripts/verify-qwen-client.mjs",
+ "verify:pi-client": "npm run build:cached && node scripts/verify-native-agent-client.mjs pi",
+ "verify:hermes-client": "npm run build:cached && node scripts/verify-native-agent-client.mjs hermes",
"probe:windows-capabilities": "npm run build:cached && node scripts/windows-capabilities.mjs",
"verify:published": "node scripts/verify-published-package.mjs",
"config:print": "node scripts/print-mcp-config.mjs",
@@ -269,7 +272,8 @@
"write-file-atomic": "7.0.0",
"ws": "8.21.0",
"zod": "4.4.3",
- "evmole": "0.9.3"
+ "evmole": "0.9.3",
+ "yaml": "2.9.0"
},
"overrides": {
"vitepress": {
@@ -293,7 +297,6 @@
"turbo": "2.11.7",
"typescript": "7.0.2",
"vitepress": "1.6.4",
- "vitest": "5.0.3",
- "yaml": "2.9.0"
+ "vitest": "5.0.3"
}
}
diff --git a/scripts/create-ghidra-windows-fixture.mjs b/scripts/create-ghidra-windows-fixture.mjs
index 6538ec57f8..e610b46880 100644
--- a/scripts/create-ghidra-windows-fixture.mjs
+++ b/scripts/create-ghidra-windows-fixture.mjs
@@ -3,68 +3,103 @@ import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
const root = resolve(dirname(fileURLToPath(import.meta.url)), "..");
+// Source-owned static fixtures: no compiler, target loading or execution.
for (const architecture of ["x86_64", "x86"]) {
- const x86 = architecture === "x86";
- const output = resolve(
- root,
- "build",
- "fixtures",
- x86 ? "rea-ghidra-windows-x86.exe" : "rea-ghidra-windows.exe",
- );
- const bytes = Buffer.alloc(1024);
- const peOffset = 0x80;
- const optionalHeader = peOffset + 24;
- const optionalHeaderSize = x86 ? 0xe0 : 0xf0;
- const sectionTable = optionalHeader + optionalHeaderSize;
+ for (const dll of [false, true]) {
+ const x86 = architecture === "x86";
+ const output = resolve(
+ root,
+ "build",
+ "fixtures",
+ `rea-ghidra-windows${x86 ? "-x86" : ""}.${dll ? "dll" : "exe"}`,
+ );
+ const bytes = Buffer.alloc(1024);
+ const peOffset = 0x80;
+ const optionalHeader = peOffset + 24;
+ const optionalHeaderSize = x86 ? 0xe0 : 0xf0;
+ const sectionTable = optionalHeader + optionalHeaderSize;
+ const directories = optionalHeader + (x86 ? 96 : 112);
- bytes.write("MZ", 0, "ascii");
- bytes.writeUInt32LE(peOffset, 0x3c);
- bytes.write("PE\0\0", peOffset, "binary");
- bytes.writeUInt16LE(x86 ? 0x14c : 0x8664, peOffset + 4);
- bytes.writeUInt16LE(1, peOffset + 6);
- bytes.writeUInt16LE(optionalHeaderSize, peOffset + 20);
- bytes.writeUInt16LE(x86 ? 0x102 : 0x22, peOffset + 22);
+ bytes.write("MZ", 0, "ascii");
+ bytes.writeUInt32LE(peOffset, 0x3c);
+ bytes.write("PE\0\0", peOffset, "binary");
+ bytes.writeUInt16LE(x86 ? 0x14c : 0x8664, peOffset + 4);
+ bytes.writeUInt16LE(1, peOffset + 6);
+ bytes.writeUInt16LE(optionalHeaderSize, peOffset + 20);
+ bytes.writeUInt16LE(
+ (x86 ? 0x102 : 0x22) | (dll ? 0x2000 : 0),
+ peOffset + 22,
+ );
- bytes.writeUInt16LE(x86 ? 0x10b : 0x20b, optionalHeader);
- bytes.writeUInt32LE(0x200, optionalHeader + 4);
- bytes.writeUInt32LE(0x1000, optionalHeader + 16);
- bytes.writeUInt32LE(0x1000, optionalHeader + 20);
- if (x86) bytes.writeUInt32LE(0x400000, optionalHeader + 28);
- else bytes.writeBigUInt64LE(0x1_4000_0000n, optionalHeader + 24);
- bytes.writeUInt32LE(0x1000, optionalHeader + 32);
- bytes.writeUInt32LE(0x200, optionalHeader + 36);
- bytes.writeUInt16LE(6, optionalHeader + 40);
- bytes.writeUInt16LE(6, optionalHeader + 48);
- bytes.writeUInt32LE(0x2000, optionalHeader + 56);
- bytes.writeUInt32LE(0x200, optionalHeader + 60);
- bytes.writeUInt16LE(3, optionalHeader + 68);
- bytes.writeUInt16LE(x86 ? 0x8140 : 0x8160, optionalHeader + 70);
- if (x86) {
- bytes.writeUInt32LE(0x10_0000, optionalHeader + 72);
- bytes.writeUInt32LE(0x1000, optionalHeader + 76);
- bytes.writeUInt32LE(0x10_0000, optionalHeader + 80);
- bytes.writeUInt32LE(0x1000, optionalHeader + 84);
- bytes.writeUInt32LE(16, optionalHeader + 92);
- } else {
- bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 72);
- bytes.writeBigUInt64LE(0x1000n, optionalHeader + 80);
- bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 88);
- bytes.writeBigUInt64LE(0x1000n, optionalHeader + 96);
- bytes.writeUInt32LE(16, optionalHeader + 108);
+ bytes.writeUInt16LE(x86 ? 0x10b : 0x20b, optionalHeader);
+ bytes.writeUInt32LE(0x200, optionalHeader + 4);
+ bytes.writeUInt32LE(dll ? 0 : 0x1000, optionalHeader + 16);
+ bytes.writeUInt32LE(0x1000, optionalHeader + 20);
+ if (x86) bytes.writeUInt32LE(0x400000, optionalHeader + 28);
+ else bytes.writeBigUInt64LE(0x1_4000_0000n, optionalHeader + 24);
+ bytes.writeUInt32LE(0x1000, optionalHeader + 32);
+ bytes.writeUInt32LE(0x200, optionalHeader + 36);
+ bytes.writeUInt16LE(6, optionalHeader + 40);
+ bytes.writeUInt16LE(6, optionalHeader + 48);
+ bytes.writeUInt32LE(0x2000, optionalHeader + 56);
+ bytes.writeUInt32LE(0x200, optionalHeader + 60);
+ bytes.writeUInt16LE(3, optionalHeader + 68);
+ bytes.writeUInt16LE(x86 ? 0x8140 : 0x8160, optionalHeader + 70);
+ if (x86) {
+ bytes.writeUInt32LE(0x10_0000, optionalHeader + 72);
+ bytes.writeUInt32LE(0x1000, optionalHeader + 76);
+ bytes.writeUInt32LE(0x10_0000, optionalHeader + 80);
+ bytes.writeUInt32LE(0x1000, optionalHeader + 84);
+ bytes.writeUInt32LE(16, optionalHeader + 92);
+ } else {
+ bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 72);
+ bytes.writeBigUInt64LE(0x1000n, optionalHeader + 80);
+ bytes.writeBigUInt64LE(0x10_0000n, optionalHeader + 88);
+ bytes.writeBigUInt64LE(0x1000n, optionalHeader + 96);
+ bytes.writeUInt32LE(16, optionalHeader + 108);
+ }
+
+ bytes.write(".text", sectionTable, "ascii");
+ bytes.writeUInt32LE(dll ? 0x180 : 0x16, sectionTable + 8);
+ bytes.writeUInt32LE(0x1000, sectionTable + 12);
+ bytes.writeUInt32LE(0x200, sectionTable + 16);
+ bytes.writeUInt32LE(0x200, sectionTable + 20);
+ bytes.writeUInt32LE(0x6000_0020, sectionTable + 36);
+
+ // EXE entry returns zero. DLL caller adds one to the callee's 42.
+ bytes.set(
+ dll
+ ? [0xe8, 0x0b, 0, 0, 0, 0x83, 0xc0, 1, 0xc3]
+ : [0xe8, 0x0b, 0, 0, 0, 0x31, 0xc0, 0xc3],
+ 0x200,
+ );
+ bytes.set([0xb8, 0x2a, 0, 0, 0, 0xc3], 0x210);
+
+ if (dll) {
+ // Export both functions; an entry-point-free library is discovered by exports.
+ bytes.writeUInt32LE(0x1040, directories);
+ bytes.writeUInt32LE(0x130, directories + 4);
+ const exports = 0x240;
+ bytes.writeUInt32LE(0x1100, exports + 12);
+ bytes.writeUInt32LE(1, exports + 16);
+ bytes.writeUInt32LE(2, exports + 20);
+ bytes.writeUInt32LE(2, exports + 24);
+ bytes.writeUInt32LE(0x1080, exports + 28);
+ bytes.writeUInt32LE(0x1088, exports + 32);
+ bytes.writeUInt32LE(0x1090, exports + 36);
+ bytes.writeUInt32LE(0x1000, 0x280);
+ bytes.writeUInt32LE(0x1010, 0x284);
+ bytes.writeUInt32LE(0x1120, 0x288);
+ bytes.writeUInt32LE(0x1140, 0x28c);
+ bytes.writeUInt16LE(0, 0x290);
+ bytes.writeUInt16LE(1, 0x292);
+ bytes.write("rea-fixture.dll\0", 0x300, "binary");
+ bytes.write("fixture_caller\0", 0x320, "binary");
+ bytes.write("fixture_return42\0", 0x340, "binary");
+ }
+
+ await mkdir(dirname(output), { recursive: true });
+ await writeFile(output, bytes);
+ if (!x86 && !dll) process.stdout.write(`${output}\n`);
}
-
- bytes.write(".text", sectionTable, "ascii");
- bytes.writeUInt32LE(0x16, sectionTable + 8);
- bytes.writeUInt32LE(0x1000, sectionTable + 12);
- bytes.writeUInt32LE(0x200, sectionTable + 16);
- bytes.writeUInt32LE(0x200, sectionTable + 20);
- bytes.writeUInt32LE(0x6000_0020, sectionTable + 36);
-
- // entry calls the second function, returns zero; second returns 42.
- bytes.set([0xe8, 0x0b, 0, 0, 0, 0x31, 0xc0, 0xc3], 0x200);
- bytes.set([0xb8, 0x2a, 0, 0, 0, 0xc3], 0x210);
-
- await mkdir(dirname(output), { recursive: true });
- await writeFile(output, bytes);
- if (!x86) process.stdout.write(`${output}\n`);
}
diff --git a/scripts/generate-package-metadata.mjs b/scripts/generate-package-metadata.mjs
index 8247dac1c2..0130401b17 100644
--- a/scripts/generate-package-metadata.mjs
+++ b/scripts/generate-package-metadata.mjs
@@ -17,7 +17,7 @@ const lock = JSON.parse(
await readFile(join(root, "package-lock.json"), "utf8"),
);
const skill = await readFile(
- join(root, "skill-src/reverse-engineer-anything/SKILL.md"),
+ join(root, ".agents/skills/reverse-engineer-anything/SKILL.md"),
"utf8",
);
const skillVersion = /^\s{2}version:\s*"([^"]+)"\s*$/mu.exec(skill)?.[1];
diff --git a/scripts/generate-skill-metadata.mjs b/scripts/generate-skill-metadata.mjs
index 5bb2e3ccff..3ca0a467db 100644
--- a/scripts/generate-skill-metadata.mjs
+++ b/scripts/generate-skill-metadata.mjs
@@ -10,7 +10,7 @@ for (const argument of arguments_)
throw new Error(`Unknown skill metadata option: ${argument}`);
const root = join(dirname(fileURLToPath(import.meta.url)), "..");
-const sourceRoot = join(root, "skill-src/reverse-engineer-anything");
+const sourceRoot = join(root, ".agents/skills/reverse-engineer-anything");
const outputRoot = join(root, "skills/reverse-engineer-anything");
const check = arguments_.has("--check");
const current = (
@@ -25,7 +25,7 @@ const { PRODUCT_IDENTITY } = await import(
);
if (/^\s{2}(?:tool_count|catalog_digest):/mu.test(current))
throw new Error(
- "Catalog metadata belongs in the generated skill, not skill-src",
+ "Catalog metadata belongs in the generated skill, not the authored source",
);
const versionLine = /^ {2}version: "[^"\r\n]+"$/mu;
if (!versionLine.test(current))
diff --git a/scripts/verify-binary-layout.mjs b/scripts/verify-binary-layout.mjs
index 996eb39666..1f3b5e4bc4 100644
--- a/scripts/verify-binary-layout.mjs
+++ b/scripts/verify-binary-layout.mjs
@@ -1283,7 +1283,7 @@ async function inspect(
evidence.provider.version,
"pwntools@4.15.0;pyelftools@0.33;unicorn@2.1.2",
);
- assert.deepEqual(evidence.normalized_result, evidence.raw_result);
+ assert.equal(evidence.raw_result, null);
assert.equal(evidence.normalized_result.diagnostics.truncated, false);
return evidence.normalized_result;
}
diff --git a/scripts/verify-native-agent-client.mjs b/scripts/verify-native-agent-client.mjs
new file mode 100644
index 0000000000..f561e353e7
--- /dev/null
+++ b/scripts/verify-native-agent-client.mjs
@@ -0,0 +1,456 @@
+import assert from "node:assert/strict";
+import { createHash } from "node:crypto";
+import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { fileURLToPath } from "node:url";
+import {
+ TOOL_CONTRACTS,
+ toolContract,
+} from "../dist/contracts/toolContracts.js";
+import { exec } from "./lib/verify-package-core.mjs";
+import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
+import { createOpenAiModelFixture } from "./verify/clients/model-fixture.mjs";
+
+const client = process.argv[2];
+const mode = process.argv[3] ?? "call";
+assert(["pi", "hermes"].includes(client), "Select pi or hermes.");
+assert(["chat", "call"].includes(mode), "Select chat or call.");
+assert(
+ process.platform !== "win32",
+ "This real-client lane requires POSIX; native Windows is unverified.",
+);
+const repo = fileURLToPath(new URL("..", import.meta.url));
+const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
+const command =
+ process.env[`REA_VERIFY_${client.toUpperCase()}_COMMAND`] ?? client;
+const lab = await mkdtemp(join(tmpdir(), `rea-${client}-client-`));
+const account = join(lab, "account");
+const profile = join(lab, "selected profile");
+const workspace = join(lab, "workspace");
+const target = join(workspace, "fixture α测试");
+const artifactPath = join(workspace, "evidence.json");
+const skillPath = join(
+ client === "pi" ? join(account, ".agents") : profile,
+ "skills",
+ "reverse-engineer-anything",
+ "SKILL.md",
+);
+const verifier = createVerifierRun();
+await Promise.all([
+ mkdir(account),
+ mkdir(profile),
+ mkdir(target, { recursive: true }),
+]);
+await writeFile(
+ join(target, "app.js"),
+ 'export function result() { return { clientCompatibility: "REA_CLIENT_FIXTURE", count: 7 }; }\n',
+);
+const environment = {
+ PATH: process.env.PATH ?? "",
+ USERPROFILE: account,
+ [client === "pi" ? "PI_CODING_AGENT_DIR" : "HERMES_HOME"]: profile,
+ PI_OFFLINE: "1",
+ REA_PROCESS_RUN_ID: verifier.run_id,
+};
+const configPath = join(profile, client === "pi" ? "mcp.json" : "config.yaml");
+const original =
+ client === "pi"
+ ? JSON.stringify({
+ mcpServers: { other: { command: "unrelated-server", enabled: false } },
+ })
+ : "# Keep caller settings.\nmcp_servers:\n other:\n command: unrelated-server\n enabled: false\n";
+await writeFile(configPath, original);
+if (client === "pi") {
+ // The CLI fixture account and native os.homedir() differ; use Pi's caller-owned skill setting.
+ await writeFile(
+ join(profile, "settings.json"),
+ JSON.stringify({ skills: [join(account, ".agents", "skills")] }),
+ );
+}
+const clientVersion = (
+ await exec(command, ["--version"], { env: environment, timeout: 30_000 })
+).stdout.trim();
+for (const args of [
+ ["setup", "--client", client, "--dry-run"],
+ ["setup", "--client", client, "--yes"],
+ ["doctor", "--client", client, "--skill"],
+]) {
+ const result = await exec(
+ process.execPath,
+ [join(runtimeRoot, "scripts/rea.mjs"), ...args, "--json"],
+ { env: environment, cwd: workspace, timeout: 60_000 },
+ );
+ const parsed = JSON.parse(result.stdout);
+ if (args[0] === "doctor") assert.equal(parsed.healthy, true);
+ else
+ assert.equal(
+ parsed.status,
+ args.includes("--dry-run") ? "planned" : "ready",
+ );
+ await writeFile(
+ join(lab, `${args[0]}${args.includes("--dry-run") ? "-plan" : ""}.json`),
+ result.stdout,
+ );
+}
+assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
+const installedConfiguration = await readFile(configPath, "utf8");
+assert(installedConfiguration.includes("unrelated-server"));
+if (client === "hermes")
+ assert(installedConfiguration.includes("# Keep caller settings."));
+const repeated = await exec(
+ process.execPath,
+ [
+ join(runtimeRoot, "scripts/rea.mjs"),
+ "setup",
+ "--client",
+ client,
+ "--yes",
+ "--json",
+ ],
+ { env: environment, cwd: workspace, timeout: 60_000 },
+);
+assert.deepEqual(JSON.parse(repeated.stdout).appliedActions, []);
+assert.equal(await readFile(configPath, "utf8"), installedConfiguration);
+assert.equal(await readFile(`${configPath}.rea.backup`, "utf8"), original);
+assert(
+ (await readFile(skillPath, "utf8")).includes(
+ "name: reverse-engineer-anything",
+ ),
+);
+let evidenceSeen = false;
+let skillLoaded = false;
+let skillRequested = false;
+let callRequested = false;
+let searchRequested = false;
+let describeRequested = false;
+let finalSeen = false;
+let catalog;
+let artifactDigest;
+const extractEvidence = (value) => {
+ if (typeof value === "string") {
+ if (
+ value.startsWith("")
+ ) {
+ // Hermes wraps external MCP content with its own instruction banner.
+ // Decode only that observed envelope, retaining the original request artifact.
+ const start = value.indexOf("\n\n");
+ if (start >= 0)
+ return extractEvidence(
+ value
+ .slice(start + 2, value.lastIndexOf(" "))
+ .trim(),
+ );
+ }
+ try {
+ return extractEvidence(JSON.parse(value));
+ } catch {
+ return undefined;
+ }
+ }
+ if (value && typeof value === "object") {
+ if (
+ typeof value.evidence_id === "string" &&
+ value.normalized_result?.graph?.nodes
+ )
+ return value;
+ for (const child of Object.values(value)) {
+ const found = extractEvidence(child);
+ if (found) return found;
+ }
+ }
+};
+const validateEvidence = async (value) => {
+ const evidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(value);
+ assert.equal(evidence.subject.local_path, target);
+ const shape = evidence.normalized_result.graph.nodes
+ .flatMap((n) => n.observations)
+ .map((o) => o.properties)
+ .find(
+ (p) =>
+ p.semantic_role === "export-return-shapes" &&
+ p.exported_name === "result",
+ );
+ assert(shape?.return_shape_coverage?.projection_complete);
+ const fields = shape.static_return_shapes.flatMap((s) => s.fields);
+ assert(
+ fields.some(
+ (f) =>
+ f.path === "/clientCompatibility" &&
+ f.state === "literal" &&
+ f.value === "REA_CLIENT_FIXTURE",
+ ),
+ );
+ assert(
+ fields.some(
+ (f) => f.path === "/count" && f.state === "literal" && f.value === 7,
+ ),
+ );
+ const serialized = JSON.stringify(evidence);
+ await writeFile(join(lab, "evidence.json"), serialized);
+ artifactDigest = createHash("sha256").update(serialized).digest("hex");
+ evidenceSeen = true;
+};
+const fixture = await createOpenAiModelFixture({
+ directory: lab,
+ prefix: `${client}-${mode}`,
+ onRequest: async ({ body, tools, requestIndex }) => {
+ assert(
+ requestIndex <= 6,
+ "Client did not complete the bounded fixture workflow.",
+ );
+ const messages = body.messages ?? [];
+ const toolMessages = messages.filter((m) => m.role === "tool");
+ const serializedTools = JSON.stringify(toolMessages);
+ const call = (name, args) => ({
+ role: "assistant",
+ content: null,
+ tool_calls: [
+ {
+ id: `call_rea_${requestIndex}`,
+ type: "function",
+ function: { name, arguments: JSON.stringify(args) },
+ },
+ ],
+ });
+ if (
+ client === "pi" &&
+ callRequested &&
+ serializedTools.includes("REA_CLIENT_FIXTURE")
+ ) {
+ const fact = toolMessages
+ .flatMap((m) => {
+ const text =
+ typeof m.content === "string"
+ ? m.content
+ : JSON.stringify(m.content);
+ const marker = text.indexOf('{"REA_CLIENT_FIXTURE":true');
+ if (marker < 0) return [];
+ const end = text.indexOf("}\n", marker);
+ try {
+ return [
+ JSON.parse(text.slice(marker, end < 0 ? undefined : end + 1)),
+ ];
+ } catch {
+ return [];
+ }
+ })
+ .find((v) => Array.isArray(v.catalog));
+ assert(
+ fact,
+ "Pi codemode must return its actual catalog and skill-read receipt.",
+ );
+ catalog = fact.catalog;
+ skillLoaded = fact.skill_loaded === true;
+ await validateEvidence(JSON.parse(await readFile(artifactPath, "utf8")));
+ } else if (client === "hermes") {
+ const search = tools.find((t) => t.name === "tool_search");
+ const exposed = tools
+ .filter((t) => t.name.startsWith("mcp__rea__"))
+ .map((t) => t.name.slice(10));
+ const deferred = [
+ ...(search?.description ?? "").matchAll(/^- mcp__rea__(\w+):/gmu),
+ ].map((m) => m[1]);
+ const native = [...new Set([...exposed, ...deferred])];
+ // Hermes advertises MCP prompt utilities beside server tools.
+ if (native.length)
+ catalog = native.filter(
+ (name) => !["get_prompt", "list_prompts"].includes(name),
+ );
+ skillLoaded ||=
+ serializedTools.includes(skillPath) &&
+ serializedTools.includes("reverse-engineer-anything");
+ for (const message of toolMessages) {
+ const evidence = extractEvidence(message.content);
+ if (evidence) await validateEvidence(evidence);
+ }
+ }
+ if (mode === "chat" || evidenceSeen) {
+ finalSeen = true;
+ return { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" };
+ }
+ if (client === "pi") {
+ const tool = tools.find((t) => t.name === "codemode");
+ assert(
+ tool,
+ `Native Pi codemode unavailable: ${tools.map((t) => t.name).join(", ")}`,
+ );
+ assert(
+ JSON.stringify(messages).includes(skillPath),
+ "Pi must advertise the installed REA skill.",
+ );
+ callRequested = true;
+ return call(tool.name, {
+ code: `await describeNamespace("rea"); const skill = await tools.read({path:${JSON.stringify(skillPath)}}); const result = await tools.mcp__rea__analyze_javascript_application({input_path:${JSON.stringify(target)},format:"directory"}); if(result.isError) throw new Error(JSON.stringify(result)); const evidence = result.structuredContent ?? JSON.parse(result.content.find(c=>c.type==="text").text); await tools.write({path:${JSON.stringify(artifactPath)},content:JSON.stringify(evidence)}); text({REA_CLIENT_FIXTURE:true,skill_loaded:typeof skill==="string"&&skill.includes("name: reverse-engineer-anything"),catalog:ALL_TOOLS.filter(t=>t.name.startsWith("mcp__rea__")).map(t=>t.name.slice(10))});`,
+ });
+ }
+ if (!skillRequested) {
+ const tool = tools.find((t) => t.name === "skill_view");
+ assert(tool, "Hermes skill_view must be available.");
+ skillRequested = true;
+ return call(tool.name, { name: "reverse-engineer-anything" });
+ }
+ if (!searchRequested) {
+ const tool = tools.find((t) => t.name === "tool_search");
+ assert(tool, "Hermes native deferred tool_search must be available.");
+ searchRequested = true;
+ return call(tool.name, { queries: ["analyze_javascript_application"] });
+ }
+ assert(
+ serializedTools.includes("mcp__rea__analyze_javascript_application"),
+ "Hermes must discover the analysis tool.",
+ );
+ if (!describeRequested) {
+ const tool = tools.find((t) => t.name === "tool_describe");
+ assert(tool, "Hermes must load the actual parameter schema.");
+ describeRequested = true;
+ return call(tool.name, {
+ names: ["mcp__rea__analyze_javascript_application"],
+ });
+ }
+ const tool = tools.find((t) => t.name === "tool_call");
+ assert(tool, "Hermes native deferred tool_call must be available.");
+ callRequested = true;
+ return call(tool.name, {
+ calls: [
+ {
+ name: "mcp__rea__analyze_javascript_application",
+ arguments: { input_path: target, format: "directory" },
+ },
+ ],
+ });
+ },
+});
+if (client === "pi") {
+ await writeFile(
+ join(profile, "models.json"),
+ JSON.stringify({
+ providers: {
+ "rea-fixture": {
+ baseUrl: fixture.baseUrl,
+ api: "openai-completions",
+ apiKey: "local-fixture-key",
+ models: [
+ {
+ id: "rea-client-fixture",
+ reasoning: false,
+ contextWindow: 200000,
+ maxTokens: 4096,
+ },
+ ],
+ },
+ },
+ }),
+ );
+} else {
+ const configured = await readFile(configPath, "utf8");
+ await writeFile(
+ configPath,
+ configured +
+ `\nmodel:\n default: rea-client-fixture\n provider: custom\n base_url: ${fixture.baseUrl}\nagent:\n max_turns: 6\n`,
+ );
+}
+const args =
+ client === "pi"
+ ? [
+ "--offline",
+ "--provider",
+ "rea-fixture",
+ "--model",
+ "rea-client-fixture",
+ "--no-session",
+ "--no-context-files",
+ "--mode",
+ "json",
+ "-p",
+ mode === "chat"
+ ? "Say hello briefly."
+ : `Use REA to analyze ${target}.`,
+ ]
+ : [
+ "--oneshot",
+ mode === "chat"
+ ? "Say hello briefly."
+ : `Use REA to analyze ${target}.`,
+ "--provider",
+ "custom",
+ "--model",
+ "rea-client-fixture",
+ "--reasoning",
+ "none",
+ ];
+let failure;
+try {
+ const execution = exec(command, args, {
+ env: {
+ ...environment,
+ OPENAI_API_KEY: "local-fixture-key",
+ OPENAI_BASE_URL: fixture.baseUrl,
+ },
+ cwd: workspace,
+ timeout: 120_000,
+ maxBuffer: 10 * 1024 * 1024,
+ });
+ // Pi consumes piped input before its first prompt; an unused open pipe is not EOF.
+ execution.child.stdin.end();
+ const result = await execution;
+ await writeFile(join(lab, "client.stdout"), result.stdout);
+ await writeFile(join(lab, "client.stderr"), result.stderr);
+ assert.equal(fixture.failure, undefined);
+ assert(finalSeen);
+ assert(result.stdout.includes("REA_CLIENT_COMPATIBILITY_OK"));
+ if (mode === "call") {
+ assert(evidenceSeen);
+ assert(
+ skillLoaded,
+ "Native client must actually read the installed skill.",
+ );
+ assert.deepEqual(
+ catalog?.sort(),
+ TOOL_CONTRACTS.map((t) => t.name).sort(),
+ "The complete REA catalog must survive native client discovery.",
+ );
+ }
+} catch (cause) {
+ failure = cause;
+ if (typeof cause?.stdout === "string")
+ await writeFile(join(lab, "client.stdout"), cause.stdout);
+ if (typeof cause?.stderr === "string")
+ await writeFile(join(lab, "client.stderr"), cause.stderr);
+} finally {
+ await fixture.close();
+ const receipt = {
+ client,
+ clientVersion,
+ mode,
+ passed: failure === undefined,
+ artifacts: lab,
+ runtimeRoot,
+ host: {
+ platform: process.platform,
+ architecture: process.arch,
+ node: process.versions.node,
+ },
+ modelFixture:
+ "loopback deterministic OpenAI-compatible endpoint; no live provider claim",
+ skillDiscovery:
+ client === "pi"
+ ? "caller-configured isolated shared directory; default OS home discovery unverified"
+ : "native HERMES_HOME personal directory",
+ catalogSize: catalog?.length,
+ evidenceSeen,
+ skillLoaded,
+ artifactDigest,
+ finalSeen,
+ requests: fixture.requests,
+ probes: fixture.probes,
+ fixtureFailure: fixture.failure,
+ verifier: await completeVerifierRun(verifier),
+ };
+ await writeFile(join(lab, "receipt.json"), JSON.stringify(receipt, null, 2));
+ console.log(JSON.stringify(receipt));
+}
+if (failure !== undefined) throw failure;
diff --git a/scripts/verify-package-environment.mjs b/scripts/verify-package-environment.mjs
index f9ee010af8..cd84655b2e 100644
--- a/scripts/verify-package-environment.mjs
+++ b/scripts/verify-package-environment.mjs
@@ -85,7 +85,12 @@ export async function verifyPackageEnvironment({
PI_CODING_AGENT_DIR: undefined,
PI_CONFIG_DIR: undefined,
PI_PROFILE: undefined,
+ QWEN_HOME: undefined,
OPENCODE_CONFIG: undefined,
+ OPENCODE_CONFIG_DIR: undefined,
+ GEMINI_CLI_HOME: undefined,
+ GEMINI_CLI_SYSTEM_SETTINGS_PATH: undefined,
+ GEMINI_CLI_SYSTEM_DEFAULTS_PATH: undefined,
XDG_CONFIG_HOME: join(home, ".config"),
PATH: `${fakeBin}:${process.env.PATH ?? ""}`,
REA_ANALYSIS_PROVIDER: "auto",
diff --git a/scripts/verify-package-install.mjs b/scripts/verify-package-install.mjs
index 544ff93008..722549e5d0 100644
--- a/scripts/verify-package-install.mjs
+++ b/scripts/verify-package-install.mjs
@@ -13,7 +13,16 @@ export async function verifyPackageInstall({
}) {
await exec(
"npm",
- ["install", "--global", "--ignore-scripts", "--prefix", prefix, tarball],
+ [
+ "install",
+ "--global",
+ "--ignore-scripts",
+ "--omit=dev",
+ "--install-strategy=nested",
+ "--prefix",
+ prefix,
+ tarball,
+ ],
{ env: environment },
);
const cli = join(prefix, "bin", "rea");
diff --git a/scripts/verify-package-pack.mjs b/scripts/verify-package-pack.mjs
index 96f05417b9..fc9676745b 100644
--- a/scripts/verify-package-pack.mjs
+++ b/scripts/verify-package-pack.mjs
@@ -38,7 +38,7 @@ export async function verifyPackagePack({ root, workspace }) {
throw new Error("package did not expose both rea command entry points");
if (packedManifest.mcpName !== "io.github.morluto/rea")
throw new Error("package did not retain its MCP Registry ownership marker");
- if (packedFiles.some((path) => path.startsWith("package/skill-src/")))
+ if (packedFiles.some((path) => path.startsWith("package/.agents/")))
throw new Error(
"package included authored skill sources instead of only the generated bundle",
);
diff --git a/scripts/verify-package.mjs b/scripts/verify-package.mjs
index 24923f7f6e..b70b06064b 100644
--- a/scripts/verify-package.mjs
+++ b/scripts/verify-package.mjs
@@ -17,6 +17,7 @@ import { verifyPackageMcp } from "./verify-package-mcp.mjs";
import { verifyPackagePack } from "./verify-package-pack.mjs";
import { verifyPackagePlatform } from "./verify-package-platform.mjs";
import { verifyPackageSetup } from "./verify-package-setup.mjs";
+import { verifyRepositorySkill } from "./verify-repository-skill.mjs";
import { verifyManaged } from "./verify-package-managed.mjs";
import { verifyUnknownProvider } from "./verify-package-unknown-provider.mjs";
import { completeVerifierRun, createVerifierRun } from "./lib/verifier-run.mjs";
@@ -34,9 +35,11 @@ let tarball;
let mcpStartup;
let mcpModuleLoading;
let update;
+let repositorySkill;
let report;
try {
+ repositorySkill = await verifyRepositorySkill({ root, workspace });
({ tarball } = await verifyPackagePack({ root, workspace }));
const environmentData = await verifyPackageEnvironment({
root,
@@ -156,6 +159,7 @@ try {
configSymlinkLifecycle: supportedSetupHost,
skill: supportedSetupHost,
skillReferences: supportedSetupHost,
+ repositorySkill,
mcpTools: TOOL_CONTRACTS.length,
mcpPrompts: prompts.names.length,
promptCompletion: true,
diff --git a/scripts/verify-packaged-ghidra-windows.mjs b/scripts/verify-packaged-ghidra-windows.mjs
index 17d29c4ada..be21fde120 100644
--- a/scripts/verify-packaged-ghidra-windows.mjs
+++ b/scripts/verify-packaged-ghidra-windows.mjs
@@ -15,7 +15,7 @@ import {
import { tmpdir } from "node:os";
import { dirname, join, resolve } from "node:path";
import { promisify } from "node:util";
-import { pathToFileURL } from "node:url";
+import { fileURLToPath, pathToFileURL } from "node:url";
import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
@@ -24,10 +24,50 @@ if (process.platform !== "win32" || process.arch !== "x64")
throw new Error(
"Packaged Windows Ghidra acceptance requires a Windows x64 host.",
);
-const workspace = await mkdtemp(join(tmpdir(), "rea-packaged-ghidra-"));
const exec = promisify(execFile);
-const x86Fixture = process.argv[2] === "--x86";
-const packageRootArgument = x86Fixture ? undefined : process.argv[2];
+const arguments_ = process.argv.slice(2);
+const internalWorkspace = arguments_[0] === "--internal-workspace";
+const fixtureArguments = internalWorkspace ? arguments_.slice(2) : arguments_;
+const x86Fixture = fixtureArguments.includes("--x86");
+const dllFixture = fixtureArguments.includes("--dll");
+const [packageRootArgument, targetArgument, reportArgument] =
+ fixtureArguments.filter(
+ (argument) => argument !== "--x86" && argument !== "--dll",
+ );
+// Windows locks the loaded package-owned addon until the verifier exits.
+// Keep installation cleanup in a parent that never loads that addon.
+if (!internalWorkspace) {
+ const ownedWorkspace = await mkdtemp(join(tmpdir(), "rea-packaged-ghidra-"));
+ let result;
+ try {
+ result = await exec(
+ process.execPath,
+ [
+ fileURLToPath(import.meta.url),
+ "--internal-workspace",
+ ownedWorkspace,
+ ...arguments_,
+ ],
+ { timeout: 900_000, maxBuffer: 8 * 1024 * 1024 },
+ );
+ } catch (cause) {
+ if (typeof cause.stdout === "string") process.stdout.write(cause.stdout);
+ if (typeof cause.stderr === "string") process.stderr.write(cause.stderr);
+ throw cause;
+ } finally {
+ await rm(ownedWorkspace, { recursive: true, force: true });
+ }
+ const report = JSON.parse(result.stdout);
+ report.workspaceCleanup = true;
+ report.ok = true;
+ if (reportArgument !== undefined)
+ await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
+ process.stdout.write(`${JSON.stringify(report)}\n`);
+ process.stderr.write(result.stderr);
+ process.exit(0);
+}
+assert.ok(arguments_[1] !== undefined, "Missing verifier workspace.");
+const workspace = resolve(arguments_[1]);
let packageRoot = resolve(packageRootArgument ?? ".");
// The default lane verifies the npm artifact in an isolated prefix. Explicit
// package roots support an already-installed artifact without a second install.
@@ -75,17 +115,17 @@ if (packageRootArgument === undefined) {
);
packageRoot = join(prefix, "node_modules", "rea-agents");
} catch (cause) {
- await rm(workspace, { recursive: true, force: true });
- throw cause;
+ throw new Error("Packaged Windows verifier installation failed.", {
+ cause,
+ });
}
}
const target = resolve(
- process.argv[3] ??
- (x86Fixture
- ? "build/fixtures/rea-ghidra-windows-x86.exe"
- : "build/fixtures/rea-ghidra-windows.exe"),
+ targetArgument ??
+ `build/fixtures/rea-ghidra-windows${x86Fixture ? "-x86" : ""}.${dllFixture ? "dll" : "exe"}`,
);
let TOOL_CONTRACTS, native, token, sha256;
+let parsedTarget;
try {
({ TOOL_CONTRACTS } = await import(
pathToFileURL(join(packageRoot, "dist/contracts/toolContracts.js"))
@@ -107,9 +147,25 @@ try {
);
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
sha256 = digest(await readFile(target));
+ const { parseBinaryTarget } = await import(
+ pathToFileURL(join(packageRoot, "dist/application/BinaryTargetResolver.js"))
+ );
+ parsedTarget = await parseBinaryTarget(target);
+ assert.equal(parsedTarget.ok, true);
+ assert.equal(parsedTarget.value.format, "pe");
+ assert.equal(parsedTarget.value.managed, false);
+ if (targetArgument === undefined) {
+ assert.equal(
+ parsedTarget.value.architecture,
+ x86Fixture ? "x86" : "x86_64",
+ );
+ assert.equal(
+ parsedTarget.value.executableRole,
+ dllFixture ? "shared-library" : "application",
+ );
+ }
} catch (cause) {
- await rm(workspace, { recursive: true, force: true });
- throw cause;
+ throw new Error("Packaged Windows verifier prerequisites failed.", { cause });
}
const digest = (bytes) => createHash("sha256").update(bytes).digest("hex");
const runtimeParent = join(workspace, "runtime with spaces");
@@ -145,6 +201,8 @@ const report = {
architecture: "x64",
nonAdmin: true,
targetSha256: sha256,
+ targetArchitecture: parsedTarget.value.architecture,
+ targetRole: parsedTarget.value.executableRole,
cli: {},
mcpOperations: [],
nativeControls: "available",
@@ -208,8 +266,8 @@ try {
});
const call = async (name, arguments_) => {
report.lastOperation = name;
- if (process.argv[4] !== undefined)
- await writeFile(process.argv[4], `${JSON.stringify(report)}\n`);
+ if (reportArgument !== undefined)
+ await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
const contract = TOOL_CONTRACTS.find((value) => value.name === name);
assert.ok(contract !== undefined, `Missing canonical contract: ${name}`);
const argumentsParsed = contract.inputSchema.parse(arguments_);
@@ -230,6 +288,25 @@ try {
await call("list_names", {});
const procedures = await call("list_procedures", {});
assert.ok(procedures.length > 0);
+ if (targetArgument === undefined) {
+ const calleeAddress = x86Fixture ? "0x401010" : "0x140001010";
+ const calleeCode = await call("procedure_pseudo_code", {
+ procedure: calleeAddress,
+ });
+ assert.match(calleeCode, /\b(?:42|0x2a)\b/iu);
+ if (dllFixture) {
+ const callerAddress = x86Fixture ? "0x401000" : "0x140001000";
+ assert.ok(
+ (
+ await call("procedure_callees", { procedure: callerAddress })
+ ).includes(calleeAddress),
+ );
+ assert.match(
+ await call("procedure_pseudo_code", { procedure: callerAddress }),
+ /\+\s*1\b/u,
+ );
+ }
+ }
assert.ok((await call("list_segments", {})).length > 0);
await call("list_strings", {});
const procedure =
@@ -296,6 +373,12 @@ try {
"Packaged verification omitted an admitted operation.",
);
const session = await call("binary_session", {});
+ assert.equal(
+ session.analysis_provider_binding?.analysis_profile?.parameters
+ ?.executable_role,
+ parsedTarget.value.executableRole,
+ "MCP profile must preserve the selected PE role.",
+ );
assert.equal(
session.analysis_provider_binding?.selection_source,
"auto-single-candidate",
@@ -326,9 +409,9 @@ try {
assert.deepEqual(await readdir(join(callerDirectory, name)), []);
report.callerScriptCollisionsPreserved = true;
report.runtimeCleanup = true;
- report.ok = true;
- if (process.argv[4] !== undefined)
- await writeFile(process.argv[4], `${JSON.stringify(report)}\n`);
+ // Only the parent can attest success after removing the installation.
+ if (reportArgument !== undefined)
+ await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
process.stdout.write(`${JSON.stringify(report)}\n`);
} catch (cause) {
report.failure = {
@@ -338,9 +421,7 @@ try {
.replaceAll(target, "")
.replaceAll(workspace, ""),
};
- if (process.argv[4] !== undefined)
- await writeFile(process.argv[4], `${JSON.stringify(report)}\n`);
+ if (reportArgument !== undefined)
+ await writeFile(reportArgument, `${JSON.stringify(report)}\n`);
throw cause;
-} finally {
- await rm(workspace, { recursive: true, force: true });
}
diff --git a/scripts/verify-qwen-client.mjs b/scripts/verify-qwen-client.mjs
new file mode 100644
index 0000000000..4b85470d68
--- /dev/null
+++ b/scripts/verify-qwen-client.mjs
@@ -0,0 +1,403 @@
+import assert from "node:assert/strict";
+import { createOpenAiModelFixture } from "./verify/clients/model-fixture.mjs";
+import {
+ mkdir,
+ writeFile,
+ readFile,
+ mkdtemp,
+ realpath,
+} from "node:fs/promises";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { fileURLToPath } from "node:url";
+import { createHash } from "node:crypto";
+import { exec } from "./lib/verify-package-core.mjs";
+import { createVerifierRun, completeVerifierRun } from "./lib/verifier-run.mjs";
+import {
+ TOOL_CONTRACTS,
+ toolContract,
+} from "../dist/contracts/toolContracts.js";
+
+assert(
+ process.platform !== "win32",
+ "The Qwen real-client verifier currently requires a POSIX host; native Windows is unverified.",
+);
+const repo = fileURLToPath(new URL("..", import.meta.url));
+const runtimeRoot = process.env.REA_VERIFY_RUNTIME_ROOT ?? repo;
+const lab = await mkdtemp(join(tmpdir(), "rea-qwen-client-"));
+const client = "qwen";
+const mode = process.argv[2] ?? "call";
+assert(
+ ["chat", "call"].includes(mode),
+ "Usage: node scripts/verify-qwen-client.mjs [chat|call]",
+);
+const command = process.env.REA_VERIFY_QWEN_COMMAND ?? "qwen";
+const verifier = createVerifierRun();
+const profile = join(lab, "selected profile");
+const account = join(lab, "account");
+const runtime = join(profile, "runtime");
+const workspace = join(lab, "workspace");
+const sharedSkills = join(account, ".agents", "skills");
+await Promise.all([
+ mkdir(profile, { recursive: true }),
+ mkdir(account),
+ mkdir(workspace),
+]);
+const baseEnvironment = {
+ PATH: process.env.PATH ?? "",
+ USERPROFILE: account,
+ QWEN_HOME: profile,
+ QWEN_RUNTIME_DIR: runtime,
+ REA_PROCESS_RUN_ID: verifier.run_id,
+};
+let clientVersion;
+try {
+ clientVersion = (
+ await exec(command, ["--version"], {
+ env: baseEnvironment,
+ timeout: 30_000,
+ })
+ ).stdout.trim();
+} catch (cause) {
+ throw new Error(
+ `Qwen client preflight failed for ${command}; install Qwen Code or set REA_VERIFY_QWEN_COMMAND. ${String(cause)}`,
+ );
+}
+const original = JSON.stringify(
+ {
+ security: {
+ auth: { selectedType: "openai" },
+ folderTrust: { enabled: false },
+ },
+ telemetry: { enabled: false },
+ model: { name: "rea-client-fixture" },
+ // Qwen uses os.homedir(), while REA's isolated CLI account uses USERPROFILE.
+ // This caller-owned setting lets the real client load the REA-installed bundle.
+ skills: { directories: [sharedSkills] },
+ mcpServers: { other: { command: "unrelated-server", disabled: true } },
+ },
+ null,
+ 2,
+);
+await writeFile(join(profile, "settings.json"), original);
+for (const step of [
+ ["setup", "--client", "qwen_code", "--dry-run"],
+ ["setup", "--client", "qwen_code", "--yes"],
+ ["doctor", "--client", "qwen_code", "--skill"],
+]) {
+ const response = await exec(
+ process.execPath,
+ [join(runtimeRoot, "scripts/rea.mjs"), ...step, "--json"],
+ { env: baseEnvironment, cwd: repo, timeout: 60_000 },
+ );
+ const result = JSON.parse(response.stdout);
+ if (step.includes("--dry-run"))
+ assert(
+ result.plannedActions.every(
+ (a) =>
+ a.target.startsWith(profile + "/") ||
+ a.target.startsWith(sharedSkills + "/"),
+ ),
+ );
+ await writeFile(
+ join(lab, `${step[0]}${step.includes("--dry-run") ? "-plan" : ""}.json`),
+ response.stdout,
+ );
+}
+assert.equal(
+ await readFile(join(profile, "settings.json.rea.backup"), "utf8"),
+ original,
+);
+const configured = JSON.parse(
+ await readFile(join(profile, "settings.json"), "utf8"),
+);
+assert.deepEqual(configured.model, { name: "rea-client-fixture" });
+assert.deepEqual(configured.skills, { directories: [sharedSkills] });
+assert.deepEqual(configured.mcpServers.other, {
+ command: "unrelated-server",
+ disabled: true,
+});
+const target = join(lab, "fixture α测试");
+await mkdir(target, { recursive: true });
+await writeFile(
+ `${target}/app.js`,
+ 'export function result() { return { clientCompatibility: "REA_CLIENT_FIXTURE", count: 7 }; }\n',
+);
+let evidenceSeen = false;
+let skillSeen = false;
+let finalSeen = false;
+let skillRequested = false;
+let searchRequested = false;
+let artifactReadByAgent = false;
+let artifactPath;
+let selectedSkillLoaded = false;
+let artifactDigest;
+const extractEvidence = (value) => {
+ if (typeof value === "string") {
+ try {
+ return extractEvidence(JSON.parse(value));
+ } catch {
+ return null;
+ }
+ }
+ if (Array.isArray(value)) {
+ for (const item of value) {
+ const result = extractEvidence(item);
+ if (result) return result;
+ }
+ } else if (value && typeof value === "object") {
+ if (
+ typeof value.evidence_id === "string" &&
+ value.normalized_result?.graph?.nodes?.length > 0
+ )
+ return value;
+ for (const nested of Object.values(value)) {
+ const result = extractEvidence(nested);
+ if (result) return result;
+ }
+ }
+ return null;
+};
+const fixture = await createOpenAiModelFixture({
+ directory: lab,
+ prefix: `${client}-${mode}`,
+ onRequest: async ({ body, tools, requestIndex }) => {
+ skillSeen ||= JSON.stringify(body.messages ?? []).includes(
+ "reverse-engineer-anything",
+ );
+ selectedSkillLoaded ||= JSON.stringify(body.messages ?? []).includes(
+ join(sharedSkills, "reverse-engineer-anything"),
+ );
+ const toolText = JSON.stringify(
+ (body.messages ?? []).filter((m) => m.role === "tool"),
+ );
+ if (
+ toolText.includes("REA_CLIENT_FIXTURE") &&
+ /(?:\\?"REA_CLIENT_FIXTURE\\?"\s*:\s*true)/u.test(toolText)
+ ) {
+ artifactReadByAgent = true;
+ assert(artifactPath);
+ const artifact = await readFile(artifactPath);
+ artifactDigest = createHash("sha256").update(artifact).digest("hex");
+ const evidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(JSON.parse(artifact.toString("utf8")));
+ assert(extractEvidence(evidence));
+ assert.equal(evidence.subject.local_path, target);
+ const shape = evidence.normalized_result.graph.nodes
+ .flatMap((n) => n.observations)
+ .map((o) => o.properties)
+ .find(
+ (p) =>
+ p.semantic_role === "export-return-shapes" &&
+ p.exported_name === "result",
+ );
+ assert(shape?.return_shape_coverage?.projection_complete);
+ const fields = shape.static_return_shapes.flatMap((s) => s.fields);
+ assert(
+ fields.some(
+ (f) =>
+ f.path === "/clientCompatibility" &&
+ f.state === "literal" &&
+ f.value === "REA_CLIENT_FIXTURE",
+ ),
+ );
+ assert(
+ fields.some(
+ (f) => f.path === "/count" && f.state === "literal" && f.value === 7,
+ ),
+ );
+ evidenceSeen = true;
+ await writeFile(
+ `${lab}/${client}-${mode}-evidence.json`,
+ JSON.stringify(evidence),
+ );
+ }
+ for (const message of body.messages ?? []) {
+ if (message.role !== "tool") continue;
+ const evidence = extractEvidence(message.content);
+ if (evidence) {
+ assert(
+ JSON.stringify(evidence.normalized_result).includes(
+ "REA_CLIENT_FIXTURE",
+ ),
+ );
+ evidenceSeen = true;
+ await writeFile(
+ `${lab}/${client}-${mode}-evidence.json`,
+ JSON.stringify(evidence),
+ );
+ }
+ }
+ const selected = tools.find((t) =>
+ t.name.includes("analyze_javascript_application"),
+ );
+ const reviewedNames = [
+ ...JSON.stringify(
+ (body.messages ?? []).filter((m) => m.role === "tool"),
+ ).matchAll(
+ /(?:\\?"name\\?"\s*:\s*\\?")([^"\\]*analyze_javascript_application)(?:\\?")/g,
+ ),
+ ]
+ .map((m) => m[1])
+ .sort((a, b) => b.length - a.length);
+ const search = tools.find((t) => t.name === "tool_search");
+ const invoke = tools.find((t) => t.name === "tool_call");
+ const skill = tools.find((t) => t.name === "skill");
+ const shell = tools.find((t) => t.name === "run_shell_command");
+ const persisted = toolText.match(
+ /Full output saved to: ([^\\"\n]+)(?:\\n|\n)/u,
+ );
+ const call = (name, args) => ({
+ role: "assistant",
+ content: null,
+ tool_calls: [
+ {
+ id: `call_rea_fixture_${requestIndex}`,
+ type: "function",
+ function: { name, arguments: JSON.stringify(args) },
+ },
+ ],
+ });
+ let message;
+ if (mode === "chat" || evidenceSeen) {
+ message = { role: "assistant", content: "REA_CLIENT_COMPATIBILITY_OK" };
+ finalSeen = true;
+ } else if (skill && !skillRequested) {
+ skillRequested = true;
+ message = call(skill.name, { skill: "reverse-engineer-anything" });
+ } else if (persisted && shell) {
+ artifactPath = persisted[1];
+ assert(
+ (await realpath(artifactPath)).startsWith(
+ (await realpath(runtime)) + "/",
+ ),
+ );
+ const script = `const fs=require('node:fs');const e=JSON.parse(fs.readFileSync(${JSON.stringify(artifactPath)},'utf8'));console.log(JSON.stringify({REA_CLIENT_FIXTURE:JSON.stringify(e.normalized_result).includes('REA_CLIENT_FIXTURE'),evidence_id:e.evidence_id,graph_nodes:e.normalized_result.graph.nodes.length}));`;
+ const quote = (value) => "'" + value.replaceAll("'", "'\\''") + "'";
+ message = call(shell.name, {
+ command: `node -e ${quote(script)}`,
+ description:
+ "Read the complete saved REA Evidence and verify the fixture facts.",
+ });
+ } else if (selected)
+ message = call(selected.name, {
+ input_path: target,
+ format: "directory",
+ });
+ else if (reviewedNames.length && invoke)
+ message = call(invoke.name, {
+ name: reviewedNames[0],
+ arguments: { input_path: target, format: "directory" },
+ });
+ else if (search && !searchRequested) {
+ searchRequested = true;
+ message = call(search.name, { query: "analyze_javascript_application" });
+ } else
+ throw new Error(
+ `REA analysis tool not exposed; tools: ${tools.map((t) => t.name).join(", ")}`,
+ );
+ return message;
+ },
+});
+const endpoint = fixture.baseUrl;
+const prompt =
+ mode === "chat"
+ ? "Say hello briefly."
+ : `Use REA to analyze the JavaScript directory ${target}. Report the result.`;
+const env = {
+ ...baseEnvironment,
+ OPENAI_API_KEY: "local-fixture-key",
+ OPENAI_BASE_URL: endpoint,
+};
+const args = [
+ "--auth-type",
+ "openai",
+ "--openai-base-url",
+ endpoint,
+ "--model",
+ "rea-client-fixture",
+ "--approval-mode",
+ mode === "chat" ? "default" : "yolo",
+ "--allowed-mcp-server-names",
+ "rea",
+ "--output-format",
+ "stream-json",
+ "--max-session-turns",
+ "8",
+ "-p",
+ prompt,
+];
+let response;
+let failure;
+try {
+ response = await exec(command, args, {
+ env,
+ cwd: workspace,
+ timeout: 120_000,
+ maxBuffer: 10 * 1024 * 1024,
+ });
+ await writeFile(join(lab, `${mode}.stdout`), response.stdout);
+ await writeFile(join(lab, `${mode}.stderr`), response.stderr);
+ const initialized = response.stdout
+ .split("\n")
+ .filter(Boolean)
+ .map((line) => JSON.parse(line))
+ .find((v) => v.type === "system" && v.subtype === "init");
+ assert(initialized);
+ const observed = initialized.tools
+ .filter((n) => n.startsWith("mcp__rea__"))
+ .sort();
+ assert.deepEqual(
+ observed,
+ TOOL_CONTRACTS.map((c) => `mcp__rea__${c.name}`).sort(),
+ "Qwen must discover the complete REA catalog",
+ );
+ assert.equal(fixture.failure, undefined);
+ assert(finalSeen);
+ if (mode === "call") {
+ assert(evidenceSeen);
+ assert(selectedSkillLoaded);
+ assert(artifactReadByAgent);
+ }
+} catch (cause) {
+ failure = cause;
+ if (typeof cause?.stdout === "string")
+ await writeFile(join(lab, `${mode}.stdout`), cause.stdout);
+ if (typeof cause?.stderr === "string")
+ await writeFile(join(lab, `${mode}.stderr`), cause.stderr);
+} finally {
+ await fixture.close();
+ const lineage = await completeVerifierRun(verifier);
+ const receipt = {
+ client: "Qwen Code",
+ clientVersion,
+ mode,
+ passed: failure === undefined,
+ artifacts: lab,
+ runtimeRoot,
+ host: {
+ platform: process.platform,
+ architecture: process.arch,
+ node: process.versions.node,
+ },
+ modelFixture:
+ "loopback deterministic OpenAI-compatible endpoint; no live model-provider claim",
+ skillDiscovery:
+ "caller-configured isolated shared directory; default OS home discovery unverified",
+ catalogSize: TOOL_CONTRACTS.length,
+ requests: fixture.requests,
+ probes: fixture.probes,
+ evidenceSeen,
+ selectedSkillLoaded,
+ artifactReadByAgent,
+ artifactPath,
+ artifactDigest,
+ finalSeen,
+ fixtureFailure: fixture.failure,
+ verifier: lineage,
+ };
+ await writeFile(join(lab, "receipt.json"), JSON.stringify(receipt, null, 2));
+ console.log(JSON.stringify(receipt));
+}
+if (failure !== undefined) throw failure;
diff --git a/scripts/verify-real-ghidra-windows.mjs b/scripts/verify-real-ghidra-windows.mjs
index 1c5b3b829a..3332ea4e2d 100644
--- a/scripts/verify-real-ghidra-windows.mjs
+++ b/scripts/verify-real-ghidra-windows.mjs
@@ -62,21 +62,21 @@ if (
);
const root = resolve(fileURLToPath(new URL("..", import.meta.url)));
-const architecture = process.argv[2] === "--x86" ? "x86" : "x86_64";
+const architecture = process.argv.includes("--x86") ? "x86" : "x86_64";
+const dllFixture = process.argv.includes("--dll");
+const role = dllFixture ? "shared-library" : "application";
const targetPath = resolve(
root,
"build",
"fixtures",
- architecture === "x86"
- ? "rea-ghidra-windows-x86.exe"
- : "rea-ghidra-windows.exe",
+ `rea-ghidra-windows${architecture === "x86" ? "-x86" : ""}.${dllFixture ? "dll" : "exe"}`,
);
const target = await parseBinaryTarget(targetPath);
if (
!target.ok ||
target.value.format !== "pe" ||
target.value.architecture !== architecture ||
- target.value.executableRole !== "application" ||
+ target.value.executableRole !== role ||
target.value.managed !== false
)
throw new Error(
@@ -89,6 +89,8 @@ const profile = await resolveGhidraAnalysisProfile(
);
if (!profile.ok || profile.value.profile === null)
throw new Error("Windows Ghidra profile could not be committed");
+if (profile.value.profile.parameters.executable_role !== role)
+ throw new Error("Windows Ghidra profile lost the observed PE role");
const client = new GhidraClient({
platform: installation.platform,
@@ -162,6 +164,19 @@ try {
throw new Error(
"Windows fixture decompilation lost its known return value",
);
+ if (dllFixture) {
+ const callerAddress = architecture === "x86" ? "0x401000" : "0x140001000";
+ const callees = await functionOperation("procedure_callees", {
+ procedure: callerAddress,
+ });
+ if (!callees.includes(callee.address))
+ throw new Error("DLL export caller lost its controlled call reference");
+ const callerCode = await functionOperation("procedure_pseudo_code", {
+ procedure: callerAddress,
+ });
+ if (typeof callerCode !== "string" || !/\+\s*1\b/u.test(callerCode))
+ throw new Error("DLL export caller lost its addition of one");
+ }
await inventory("inspect_native_load_image", {});
const memory = await inventory("read_bytes", {
@@ -254,7 +269,7 @@ try {
limitations: [
"approved-non-sensitive-fixtures-only",
"windows-x64-local-ntfs-only",
- "native-x86-and-x86-64-pe-applications-only",
+ "native-x86-and-x86-64-pe-applications-and-dlls-only",
"no-gui-or-mutation-authority",
],
};
diff --git a/scripts/verify-repository-skill.mjs b/scripts/verify-repository-skill.mjs
new file mode 100644
index 0000000000..8340bce7b6
--- /dev/null
+++ b/scripts/verify-repository-skill.mjs
@@ -0,0 +1,97 @@
+import { cp, mkdir, readFile, readdir } from "node:fs/promises";
+import { dirname, join, relative } from "node:path";
+
+import { exec, pathExists } from "./lib/verify-package-core.mjs";
+
+/** Exercise Skills' default root discovery without ignored build outputs. */
+export async function verifyRepositorySkill({ root, workspace }) {
+ const snapshot = join(workspace, "repository");
+ const consumer = join(workspace, "skill-consumer");
+ const home = join(workspace, "skill-consumer-home");
+ const tracked = (
+ await exec(
+ "git",
+ ["ls-files", "--cached", "--format=%(objectmode) %(path)", "-z"],
+ { cwd: root },
+ )
+ ).stdout
+ .split("\0")
+ .filter(Boolean);
+ for (const entry of tracked) {
+ const mode = entry.slice(0, 6);
+ const path = entry.slice(7);
+ const destination = join(snapshot, path);
+ // A GitHub clone without submodule initialization has only these directories.
+ if (mode === "160000") {
+ await mkdir(destination, { recursive: true });
+ continue;
+ }
+ await mkdir(dirname(destination), { recursive: true });
+ await cp(join(root, path), destination, { verbatimSymlinks: true });
+ }
+ if (await pathExists(join(snapshot, "skills")))
+ throw new Error("repository skill check must run without generated skills");
+ await mkdir(consumer, { recursive: true });
+ await mkdir(home, { recursive: true });
+ const skillsVersion = "1.7.2";
+ await exec(
+ "npm",
+ [
+ "exec",
+ "--yes",
+ "--package",
+ `skills@${skillsVersion}`,
+ "--",
+ "skills",
+ "add",
+ snapshot,
+ "--skill",
+ "reverse-engineer-anything",
+ "--agent",
+ "codex",
+ "--yes",
+ ],
+ {
+ cwd: consumer,
+ env: {
+ ...process.env,
+ HOME: home,
+ USERPROFILE: home,
+ XDG_CONFIG_HOME: join(home, ".config"),
+ CODEX_HOME: join(home, ".codex"),
+ DO_NOT_TRACK: "1",
+ SKILLS_NO_TELEMETRY: "1",
+ CI: "1",
+ },
+ timeout: 120_000,
+ },
+ );
+ const authoredRoot = join(
+ snapshot,
+ ".agents/skills/reverse-engineer-anything",
+ );
+ const installedSkills = join(consumer, ".agents/skills");
+ const installedRoot = join(installedSkills, "reverse-engineer-anything");
+ const files = await filePaths(authoredRoot);
+ if (
+ JSON.stringify(await readdir(installedSkills)) !==
+ JSON.stringify(["reverse-engineer-anything"]) ||
+ JSON.stringify(await filePaths(installedRoot)) !== JSON.stringify(files)
+ )
+ throw new Error(
+ "Skills did not install exactly the selected public bundle",
+ );
+ for (const path of files) {
+ const authored = await readFile(join(authoredRoot, path));
+ if (!authored.equals(await readFile(join(installedRoot, path))))
+ throw new Error(`Skills changed authored skill content: ${path}`);
+ }
+ return { skillsVersion, files };
+}
+
+async function filePaths(root) {
+ return (await readdir(root, { recursive: true, withFileTypes: true }))
+ .filter((entry) => entry.isFile())
+ .map((entry) => relative(root, join(entry.parentPath, entry.name)))
+ .sort();
+}
diff --git a/scripts/verify/apple/macos-bundle.mjs b/scripts/verify/apple/macos-bundle.mjs
index c10abcb2e0..e2a1308421 100644
--- a/scripts/verify/apple/macos-bundle.mjs
+++ b/scripts/verify/apple/macos-bundle.mjs
@@ -17,6 +17,7 @@ import {
import { verifyDyldEnvironment } from "./dyld-environment.mjs";
import { verifyDylibResolution } from "./macos-dylib-e2e.mjs";
import { verifyMachOOccurrences } from "./macho-occurrences.mjs";
+import { verifyNativeProvenance } from "./native-provenance.mjs";
const exec = promisify(execFile);
@@ -69,6 +70,7 @@ const EXPECTED_BUNDLES = [
let report;
try {
const { app, executable } = await buildMacosBundleFixture(root);
+ const nativeProvenance = await verifyNativeProvenance(app, executable);
const machoOccurrences = await verifyMachOOccurrences(app, executable);
const zip = join(root, "MacFixture.zip");
await exec("/usr/bin/ditto", ["-c", "-k", "--keepParent", app, zip]);
@@ -166,6 +168,7 @@ try {
containers: containers.map(({ format }) => format),
bundles: EXPECTED_BUNDLES.length,
dmg_detached: true,
+ native_provenance: nativeProvenance,
macho_occurrences: machoOccurrences,
dylib_resolution: dylibs,
dyld_environment: dyldEnvironment,
diff --git a/scripts/verify/apple/native-provenance.mjs b/scripts/verify/apple/native-provenance.mjs
new file mode 100644
index 0000000000..ad7bf6cc13
--- /dev/null
+++ b/scripts/verify/apple/native-provenance.mjs
@@ -0,0 +1,71 @@
+import assert from "node:assert/strict";
+import { createHash } from "node:crypto";
+import { readFile } from "node:fs/promises";
+
+import { toolContract } from "../../../dist/contracts/toolContracts.js";
+import {
+ artifactCliEvidence,
+ withArtifactMcp,
+} from "../../lib/artifact-e2e.mjs";
+
+/** Inspect the compiled bundle with real native utilities through both adapters. */
+export async function verifyNativeProvenance(app, executable) {
+ const original = await readFile(executable);
+ const sha256 = createHash("sha256").update(original).digest("hex");
+ const symbols = ["$s4main3FooV", "plain_symbol"];
+ const operations = [
+ ["inspect_macho", []],
+ ["inspect_signature", []],
+ ["inspect_plist", []],
+ ["list_architectures", []],
+ ["demangle_swift", symbols],
+ ];
+ const observations = [];
+ await withArtifactMcp(app, async (client) => {
+ const definitions = new Map(
+ (await client.listTools()).tools.map((tool) => [tool.name, tool]),
+ );
+ for (const [operation, arguments_] of operations) {
+ const cli = toolContract(operation).outputSchema.parse(
+ await artifactCliEvidence(
+ operation.replaceAll("_", "-"),
+ app,
+ arguments_,
+ ),
+ );
+ const response = await client.callTool(
+ {
+ name: operation,
+ arguments: operation === "demangle_swift" ? { symbols } : {},
+ },
+ { toolDefinition: definitions.get(operation) },
+ );
+ assert.notEqual(response.isError, true, JSON.stringify(response));
+ const mcp = toolContract(operation).outputSchema.parse(
+ response.structuredContent,
+ );
+ assert.equal(cli.raw_result, null);
+ assert.equal(mcp.raw_result, null);
+ assert.ok(cli.normalized_result.provenance.length > 0);
+ assert.deepEqual(mcp.normalized_result, cli.normalized_result);
+ assert.deepEqual(
+ JSON.parse(response.content.find((item) => item.type === "text").text),
+ response.structuredContent,
+ );
+ await client.ping();
+ observations.push({
+ operation,
+ commands: cli.normalized_result.provenance.length,
+ cli_bytes: Buffer.byteLength(JSON.stringify(cli)),
+ mcp_bytes: Buffer.byteLength(JSON.stringify(response)),
+ });
+ }
+ });
+ assert.deepEqual(await readFile(executable), original);
+ return {
+ real_native_utilities: true,
+ target_sha256: sha256,
+ bytes_unchanged: true,
+ observations,
+ };
+}
diff --git a/scripts/verify/clients/model-fixture.mjs b/scripts/verify/clients/model-fixture.mjs
new file mode 100644
index 0000000000..cdc99a31c0
--- /dev/null
+++ b/scripts/verify/clients/model-fixture.mjs
@@ -0,0 +1,171 @@
+import assert from "node:assert/strict";
+import { createServer } from "node:http";
+import { writeFile } from "node:fs/promises";
+import { join } from "node:path";
+
+/** A loopback model fixture exercises real clients without claiming live-provider compatibility. */
+export const createOpenAiModelFixture = async ({
+ directory,
+ prefix,
+ onRequest,
+}) => {
+ const requests = [];
+ const probes = [];
+ let incoming = 0;
+ let failure;
+ const server = createServer(async (request, response) => {
+ try {
+ if (request.method === "GET" && request.url?.endsWith("/models")) {
+ response.setHeader("Content-Type", "application/json");
+ response.end(
+ JSON.stringify({
+ object: "list",
+ data: [
+ {
+ id: "rea-client-fixture",
+ object: "model",
+ owned_by: "local-fixture",
+ },
+ ],
+ }),
+ );
+ return;
+ }
+ const chunks = [];
+ let bytes = 0;
+ for await (const chunk of request) {
+ bytes += chunk.length;
+ assert(
+ bytes <= 10 * 1024 * 1024,
+ "Local model fixture exceeded its 10 MiB request resource budget",
+ );
+ chunks.push(chunk);
+ }
+ const rawBody = Buffer.concat(chunks).toString("utf8");
+ const body = rawBody.length === 0 ? null : JSON.parse(rawBody);
+ await writeFile(
+ join(directory, `${prefix}-incoming-${++incoming}.json`),
+ JSON.stringify({ method: request.method, path: request.url, body }),
+ );
+ if (request.url !== "/v1/chat/completions") {
+ // Hermes probes custom endpoints for Ollama/llama.cpp metadata.
+ // Unsupported HTTP endpoints return 404, as on an OpenAI-only server.
+ probes.push({
+ method: request.method,
+ path: request.url,
+ body,
+ status: 404,
+ });
+ response.statusCode = 404;
+ response.setHeader("Content-Type", "application/json");
+ response.end(
+ JSON.stringify({
+ error:
+ "This OpenAI fixture exposes models and chat completions only.",
+ }),
+ );
+ return;
+ }
+ assert.equal(request.method, "POST");
+ assert(body !== null && typeof body === "object" && !Array.isArray(body));
+ assert(Array.isArray(body.messages));
+ const tools = (body.tools ?? [])
+ .filter((t) => t.type === "function")
+ .map((t) => t.function ?? t);
+ requests.push({
+ path: request.url,
+ tools: tools.map((t) => t.name),
+ stream: body.stream,
+ messageRoles: body.messages.map((m) => m.role),
+ });
+ await writeFile(
+ join(directory, `${prefix}-request-${requests.length}.json`),
+ JSON.stringify(body),
+ );
+ const message = await onRequest({
+ body,
+ tools,
+ requestIndex: requests.length,
+ });
+ const base = {
+ id: `chatcmpl-rea-${requests.length}`,
+ object: "chat.completion",
+ created: Math.floor(Date.now() / 1000),
+ model: body.model,
+ };
+ const finish = message.tool_calls ? "tool_calls" : "stop";
+ const usage = {
+ prompt_tokens: 100,
+ completion_tokens: 20,
+ total_tokens: 120,
+ };
+ if (body.stream) {
+ response.setHeader("Content-Type", "text/event-stream");
+ const emit = (delta, finishReason = null) =>
+ response.write(
+ `data: ${JSON.stringify({ ...base, object: "chat.completion.chunk", choices: [{ index: 0, delta, finish_reason: finishReason }] })}\n\n`,
+ );
+ if (message.tool_calls) {
+ const fragments = message.tool_calls.map((call, index) => {
+ const split = Math.floor(call.function.arguments.length / 2);
+ return {
+ first: {
+ index,
+ ...call,
+ function: {
+ name: call.function.name,
+ arguments: call.function.arguments.slice(0, split),
+ },
+ },
+ second: {
+ index,
+ function: { arguments: call.function.arguments.slice(split) },
+ },
+ };
+ });
+ emit({
+ role: "assistant",
+ content: null,
+ tool_calls: fragments.map((f) => f.first),
+ });
+ emit({ tool_calls: fragments.map((f) => f.second) });
+ } else emit(message);
+ emit({}, finish);
+ response.write(
+ `data: ${JSON.stringify({ ...base, object: "chat.completion.chunk", choices: [], usage })}\n\n`,
+ );
+ response.end("data: [DONE]\n\n");
+ } else {
+ response.setHeader("Content-Type", "application/json");
+ response.end(
+ JSON.stringify({
+ ...base,
+ choices: [{ index: 0, message, finish_reason: finish }],
+ usage,
+ }),
+ );
+ }
+ } catch (cause) {
+ failure = String(cause);
+ response.statusCode = 400;
+ response.setHeader("Content-Type", "application/json");
+ response.end(
+ JSON.stringify({
+ error: { message: failure, type: "invalid_request_error" },
+ }),
+ );
+ }
+ });
+ await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
+ const address = server.address();
+ assert(address !== null && typeof address !== "string");
+ return {
+ baseUrl: `http://127.0.0.1:${address.port}/v1`,
+ requests,
+ probes,
+ get failure() {
+ return failure;
+ },
+ close: () => new Promise((resolve) => server.close(resolve)),
+ };
+};
diff --git a/scripts/verify/javascript/output.mjs b/scripts/verify/javascript/output.mjs
index 13dfa916c7..dbdfdc3402 100644
--- a/scripts/verify/javascript/output.mjs
+++ b/scripts/verify/javascript/output.mjs
@@ -54,15 +54,15 @@ try {
data: { ...evidence, normalized_result: { payload: [marker] } },
meta: metadata,
};
- const skeleton = JSON.stringify(template, null, format === "json" ? 2 : 0);
+ // The destination is a redirected file, so both formats are compact.
+ const skeleton = JSON.stringify(template);
const encodedMarker = JSON.stringify(marker);
const markerIndex = skeleton.indexOf(encodedMarker);
assert.ok(markerIndex >= 0);
assert.equal(markerIndex, skeleton.lastIndexOf(encodedMarker));
const prefix = skeleton.slice(0, markerIndex);
const suffix = `${skeleton.slice(markerIndex + encodedMarker.length)}\n`;
- const indentation = prefix.slice(prefix.lastIndexOf("\n") + 1);
- const separator = format === "json" ? `,\n${indentation}` : ",";
+ const separator = ",";
const expectedHash = createHash("sha256");
expectedHash.update(prefix);
for (let index = 0; index < count; index += 1) {
diff --git a/scripts/verify/recorded-crash/public.mjs b/scripts/verify/recorded-crash/public.mjs
index 76713bf9f4..ecdfd14cd2 100644
--- a/scripts/verify/recorded-crash/public.mjs
+++ b/scripts/verify/recorded-crash/public.mjs
@@ -80,7 +80,7 @@ export async function connectRecordedCrash({ entrypoint, environment }) {
evidence.confidence,
debuggerContext ? "derived" : "observed",
);
- assert.deepEqual(evidence.raw_result, report);
+ assert.equal(evidence.raw_result, null);
assert.equal(evidence.subject.local_path, path);
assert.equal(report.artifact.path, path);
assert.equal(
diff --git a/src/application/ClientConfigurationDocument.ts b/src/application/ClientConfigurationDocument.ts
index 5ae6ad98cc..c7f13ba661 100644
--- a/src/application/ClientConfigurationDocument.ts
+++ b/src/application/ClientConfigurationDocument.ts
@@ -1,4 +1,9 @@
import { parse as parseToml, stringify as stringifyToml } from "smol-toml";
+import {
+ parse as parseYaml,
+ parseDocument as parseYamlDocument,
+ stringify as stringifyYaml,
+} from "yaml";
import { isDeepStrictEqual } from "node:util";
import {
applyEdits,
@@ -70,6 +75,7 @@ export const clientConfigurationServersKey = (
switch (format) {
case "toml":
case "grok":
+ case "hermes":
return "mcp_servers";
case "opencode":
return "mcp";
@@ -93,6 +99,12 @@ const parseDocument = (
): Record => {
if (format === "toml" || format === "grok")
return objectSchema.parse(parseToml(text));
+ // Pi loads `mcp.json` with a strict `JSON.parse` and no BOM tolerance, so
+ // comments, trailing commas, a BOM, or an empty existing file must be
+ // rejected instead of registering into a document Pi cannot load.
+ if (format === "pi") return objectSchema.parse(JSON.parse(text));
+ // Hermes stores MCP servers in `mcp_servers` inside a YAML document.
+ if (format === "hermes") return objectSchema.parse(parseYaml(text));
// An empty file, which some clients create before any server is added,
// holds no settings to preserve.
if (isEmptyClientConfigurationText(text)) return {};
@@ -1265,6 +1277,8 @@ export const serializeClientConfiguration = (
return serializeCodexConfiguration(document, originalText);
if (format === "grok")
return serializeGrokConfiguration(document, originalText);
+ if (format === "hermes")
+ return serializeHermesConfiguration(document, originalText);
// An empty original holds no comments or settings to preserve. Writing a
// fresh document also keeps a lone BOM from shifting past the closing brace.
if (
@@ -1303,6 +1317,34 @@ export const legacyClientServerPath = (
? [parsed.serversPath[0] ?? "mcp", name]
: undefined;
+/**
+ * Keep an existing Hermes `config.yaml` intact aside from the REA server entry.
+ * Hermes config files carry user comments and unrelated settings, so the rea
+ * entry is upserted into the parsed YAML document (which retains formatting and
+ * comments) rather than re-serializing the whole object.
+ */
+const serializeHermesConfiguration = (
+ document: Record,
+ originalText: string | undefined,
+): string => {
+ if (
+ originalText === undefined ||
+ isEmptyClientConfigurationText(originalText)
+ )
+ return stringifyYaml(document);
+ const serversKey = clientConfigurationServersKey("hermes");
+ const servers = document[serversKey];
+ const entry =
+ typeof servers === "object" && servers !== null
+ ? (servers as Record)[PRODUCT_IDENTITY.mcpServerKey]
+ : undefined;
+ const yamlDocument = parseYamlDocument(originalText);
+ if (entry === undefined)
+ yamlDocument.deleteIn([serversKey, PRODUCT_IDENTITY.mcpServerKey]);
+ else yamlDocument.setIn([serversKey, PRODUCT_IDENTITY.mcpServerKey], entry);
+ return yamlDocument.toString();
+};
+
/** Build the stdio entry shape expected by one client's configuration dialect. */
export const clientRegistrationEntry = (
format: ClientRegistrationDialect,
@@ -1331,6 +1373,7 @@ export const clientRegistrationEntry = (
};
case "vscode":
case "omp":
+ case "pi":
return {
type: "stdio",
command: executable,
@@ -1353,6 +1396,15 @@ export const clientRegistrationEntry = (
args,
...(Object.keys(environment).length === 0 ? {} : { env: environment }),
};
+ case "hermes":
+ // Hermes reads stdio servers from `mcp_servers.` and connects them
+ // unless `enabled` is false, so the entry states it explicitly.
+ return {
+ command: executable,
+ args,
+ ...(Object.keys(environment).length === 0 ? {} : { env: environment }),
+ enabled: true,
+ };
default:
return {
command: executable,
@@ -1361,3 +1413,45 @@ export const clientRegistrationEntry = (
};
}
};
+
+/** Parse one Gemini policy at its source boundary before consolidating scopes. */
+export const parseGeminiMcpPolicy = (parsed: ClientConfigurationDocument) =>
+ z
+ .object({
+ allowed: z.array(z.string()).optional(),
+ excluded: z.array(z.string()).optional(),
+ })
+ .passthrough()
+ .parse(parsed.document.mcp ?? {});
+
+/** Gemini consolidates excludes by union and allowlists by normalized intersection. */
+export const geminiServerPolicyBlock = (
+ policies: readonly ReturnType[],
+ serverKey: string,
+): string | undefined => {
+ if (policies.some(({ excluded }) => excluded?.includes(serverKey)))
+ return `Gemini mcp.excluded blocks ${serverKey}. Review the selected settings policies before rerunning setup.`;
+ const allowlists = policies.flatMap(({ allowed }) =>
+ allowed === undefined ? [] : [allowed],
+ );
+ const allowed = allowlists.reduce(
+ (previous, current) => {
+ if (previous === undefined) return current;
+ const normalized = new Set(
+ current.map((name) => name.toLowerCase().trim()),
+ );
+ return previous.filter((name) =>
+ normalized.has(name.toLowerCase().trim()),
+ );
+ },
+ undefined,
+ );
+ // Gemini's manager applies the consolidated allowlist only when nonempty.
+ if (
+ allowed !== undefined &&
+ allowed.length > 0 &&
+ !allowed.includes(serverKey)
+ )
+ return `Gemini mcp.allowed does not include ${serverKey}. Review the selected settings policies before rerunning setup.`;
+ return undefined;
+};
diff --git a/src/application/ClientConfigurationFiles.ts b/src/application/ClientConfigurationFiles.ts
new file mode 100644
index 0000000000..5982f53a92
--- /dev/null
+++ b/src/application/ClientConfigurationFiles.ts
@@ -0,0 +1,108 @@
+import { err, ok, type Result } from "../domain/result.js";
+import { readFile } from "node:fs/promises";
+import {
+ effectiveClientServer,
+ parseClientConfiguration,
+ type ClientConfigurationDocument,
+} from "./ClientConfigurationDocument.js";
+import type { SetupClient } from "./SupportedClients.js";
+
+/** A named source failure that setup can explain without losing the bad path. */
+export interface ClientConfigurationFileError {
+ readonly kind: "unreadable" | "malformed";
+ readonly path: string;
+ readonly detail: string;
+}
+
+/** Preserve named read failures and distinguish a missing source from an empty file. */
+export const readConfigurationText = async (
+ path: string,
+): Promise> => {
+ try {
+ return ok(await readFile(path, "utf8"));
+ } catch (cause: unknown) {
+ if (cause instanceof Error && "code" in cause && cause.code === "ENOENT")
+ return ok(undefined);
+ return err({
+ kind: "unreadable",
+ path,
+ detail: `Configuration ${path} could not be read: ${cause instanceof Error ? cause.message : String(cause)}`,
+ });
+ }
+};
+
+/** Read and parse one configuration at its source boundary. */
+export const readConfigurationFile = async (
+ path: string,
+ format: SetupClient["format"],
+): Promise<
+ Result
+> => {
+ const source = await readConfigurationText(path);
+ if (!source.ok) return source;
+ const content = source.value;
+ if (content === undefined) return ok(undefined);
+ try {
+ return ok(parseClientConfiguration(content, format));
+ } catch (cause: unknown) {
+ return err({
+ kind: "malformed",
+ path,
+ detail: `Configuration ${path} is malformed: ${cause instanceof Error ? cause.message : String(cause)}`,
+ });
+ }
+};
+
+/** Read every applicable user file; malformed lower-priority files also stop setup. */
+export const readClientConfigurationFiles = async (
+ client: SetupClient,
+): Promise<
+ Result
+> => {
+ const documents: ClientConfigurationDocument[] = [];
+ for (const path of client.configPaths ?? [client.configPath]) {
+ const document = await readConfigurationFile(path, client.format);
+ if (!document.ok) return document;
+ if (document.value !== undefined) documents.push(document.value);
+ }
+ return ok(documents);
+};
+
+/** OpenCode merges server objects in file order, replacing arrays and scalar values. */
+const mergeConfigurationValue = (previous: unknown, next: unknown): unknown => {
+ if (!isObject(previous) || !isObject(next)) return next;
+ return Object.fromEntries(
+ [...new Set([...Object.keys(previous), ...Object.keys(next)])].map(
+ (key) => [
+ key,
+ Object.hasOwn(next, key)
+ ? mergeConfigurationValue(previous[key], next[key])
+ : previous[key],
+ ],
+ ),
+ );
+};
+const isObject = (value: unknown): value is Record =>
+ typeof value === "object" && value !== null && !Array.isArray(value);
+
+/** Project a client's effective user registration without modifying its source files. */
+export const effectiveClientConfiguration = (
+ documents: readonly ClientConfigurationDocument[],
+): ClientConfigurationDocument | undefined => {
+ const last = documents.at(-1);
+ if (last === undefined) return undefined;
+ if (documents.length === 1) return last;
+ const servers: Record = {};
+ for (const document of documents) {
+ const names = new Set([
+ ...Object.keys(document.legacyServers),
+ ...Object.keys(document.servers),
+ ]);
+ for (const name of names)
+ servers[name] = mergeConfigurationValue(
+ servers[name],
+ effectiveClientServer(document, name),
+ );
+ }
+ return { ...last, servers, legacyServers: {} };
+};
diff --git a/src/application/ClientRegistrationStatus.ts b/src/application/ClientRegistrationStatus.ts
index a657f1df4f..1b1bb6d6e2 100644
--- a/src/application/ClientRegistrationStatus.ts
+++ b/src/application/ClientRegistrationStatus.ts
@@ -1,10 +1,14 @@
+import { readClientPolicyBlock } from "./GeminiClientSettings.js";
+import {
+ effectiveClientConfiguration,
+ readClientConfigurationFiles,
+} from "./ClientConfigurationFiles.js";
import {
clientServerForcedEnabled,
clientServerListedDisabled,
effectiveClientServer,
- parseClientConfiguration,
} from "./ClientConfigurationDocument.js";
-import { access, readFile } from "node:fs/promises";
+import { access } from "node:fs/promises";
import { resolve } from "node:path";
import { z } from "zod";
@@ -16,6 +20,7 @@ import {
npxRegistrationCommand,
} from "./ClientRegistrationIdentity.js";
import {
+ clientEvidencePaths,
manualRegistrationRemediation,
supportedClients,
} from "./SupportedClients.js";
@@ -78,28 +83,19 @@ export const readClientRegistrationStatuses = async (
for (const client of supportedClients(
home,
options.platform,
- options.environment === undefined
- ? undefined
- : {
- APPDATA: options.environment.APPDATA,
- CLAUDE_CONFIG_DIR: options.environment.CLAUDE_CONFIG_DIR,
- CODEX_HOME: options.environment.CODEX_HOME,
- COPILOT_HOME: options.environment.COPILOT_HOME,
- GROK_HOME: options.environment.GROK_HOME,
- OMP_PROFILE: options.environment.OMP_PROFILE,
- OPENCODE_CONFIG: options.environment.OPENCODE_CONFIG,
- PI_CODING_AGENT_DIR: options.environment.PI_CODING_AGENT_DIR,
- PI_CONFIG_DIR: options.environment.PI_CONFIG_DIR,
- PI_PROFILE: options.environment.PI_PROFILE,
- SAND_DATA_ROOT: options.environment.SAND_DATA_ROOT,
- XDG_CONFIG_HOME: options.environment.XDG_CONFIG_HOME,
- },
+ options.environment,
)) {
- if (
- !(await exists(client.markerPath)) &&
- !(await exists(client.configPath))
- )
+ if (client.configPathError !== undefined) {
+ statuses.push({
+ client: client.name,
+ config_path: client.configPath,
+ command: [],
+ state: "invalid",
+ remediation: client.configPathError,
+ });
continue;
+ }
+ if (!(await clientEvidencePresent(client))) continue;
const manualRemediation = manualRegistrationRemediation(client.name);
if (client.format === "unsupported") {
if (manualRemediation !== undefined)
@@ -113,8 +109,36 @@ export const readClientRegistrationStatuses = async (
continue;
}
try {
- const content = await readFile(client.configPath, "utf8");
- const parsed = parseClientConfiguration(content, client.format);
+ const files = await readClientConfigurationFiles(client);
+ if (!files.ok) {
+ statuses.push({
+ client: client.name,
+ config_path: client.configPath,
+ command: [],
+ state: "invalid",
+ remediation: files.error.detail,
+ });
+ continue;
+ }
+ const parsed = effectiveClientConfiguration(files.value);
+ if (parsed === undefined) {
+ statuses.push(
+ unavailableStatus(client.name, client.configPath, "missing"),
+ );
+ continue;
+ }
+ const policy = await readClientPolicyBlock(client, parsed);
+ if (!policy.ok) {
+ statuses.push({
+ client: client.name,
+ config_path: client.configPath,
+ command: [],
+ state: "invalid",
+ remediation: policy.error.detail,
+ });
+ continue;
+ }
+ const policyBlock = policy.value;
const raw = effectiveClientServer(parsed, PRODUCT_IDENTITY.mcpServerKey);
if (raw === undefined) {
statuses.push(
@@ -139,9 +163,14 @@ export const readClientRegistrationStatuses = async (
options.platform ?? process.platform,
clientServerForcedEnabled(parsed, PRODUCT_IDENTITY.mcpServerKey),
) &&
- !clientServerListedDisabled(parsed, PRODUCT_IDENTITY.mcpServerKey)
+ !clientServerListedDisabled(
+ parsed,
+ PRODUCT_IDENTITY.mcpServerKey,
+ ) &&
+ policyBlock === undefined
? "aligned"
: "stale",
+ policyBlock,
),
);
} catch (cause: unknown) {
@@ -168,7 +197,7 @@ const registrationAligned = (
): boolean => {
const command = [registration.command, ...registration.args];
if (
- registration.disabled === true ||
+ (client.format !== "grok" && registration.disabled === true) ||
(registration.enabled === false && !forcedEnabled)
)
return false;
@@ -181,9 +210,9 @@ const registrationAligned = (
)
return false;
if (client.format === "vscode" && registration.type !== "stdio") return false;
- // OMP infers stdio for a command entry without an explicit type.
+ // OMP and Pi infer stdio for a command entry without an explicit type.
if (
- client.format === "omp" &&
+ (client.format === "omp" || client.format === "pi") &&
registration.type !== undefined &&
registration.type !== "stdio"
)
@@ -245,7 +274,7 @@ const parseRegistration = (
if (client.format === "commandcode" && registration.transport !== "stdio")
throw new TypeError("Expected an stdio registration");
if (
- client.format === "omp" &&
+ (client.format === "omp" || client.format === "pi") &&
registration.type !== undefined &&
registration.type !== "stdio"
)
@@ -261,10 +290,17 @@ const configuredStatus = (
configPath: string,
command: RegistrationCommand,
state: "aligned" | "stale",
+ policyRemediation?: string,
): ClientRegistrationStatus =>
state === "aligned"
? { client, config_path: configPath, command, state, remediation: null }
- : { client, config_path: configPath, command, state, remediation };
+ : {
+ client,
+ config_path: configPath,
+ command,
+ state,
+ remediation: policyRemediation ?? remediation,
+ };
const unavailableStatus = (
client: string,
@@ -278,6 +314,12 @@ const unavailableStatus = (
remediation,
});
+const clientEvidencePresent = async (client: SetupClient): Promise => {
+ for (const path of clientEvidencePaths(client))
+ if (await exists(path)) return true;
+ return false;
+};
+
const exists = async (path: string | undefined): Promise => {
if (path === undefined) return false;
try {
diff --git a/src/application/Doctor.ts b/src/application/Doctor.ts
index 88dfda4c8a..cf6fd2a260 100644
--- a/src/application/Doctor.ts
+++ b/src/application/Doctor.ts
@@ -16,7 +16,6 @@ import {
type LinuxDistribution,
} from "./LinuxHopper.js";
import {
- claudeCodeSkillsDirectory,
readInstalledSkillIdentity,
type InstalledSkillIdentity,
} from "./SetupSkill.js";
@@ -441,7 +440,8 @@ export const systemDoctorHost = (
readInstalledSkillIdentity(
homeDirectory,
clientIds,
- claudeCodeSkillsDirectory(homeDirectory, environment),
+ environment,
+ platform,
),
clientRegistrations: () =>
readClientRegistrationStatuses(homeDirectory, undefined, {
diff --git a/src/application/GeminiClientSettings.ts b/src/application/GeminiClientSettings.ts
new file mode 100644
index 0000000000..eecb33b537
--- /dev/null
+++ b/src/application/GeminiClientSettings.ts
@@ -0,0 +1,202 @@
+import { realpath } from "node:fs/promises";
+import { dirname, resolve } from "node:path";
+import { stripComments } from "jsonc-parser";
+import { z } from "zod";
+import { err, ok, type Result } from "../domain/result.js";
+import { PRODUCT_IDENTITY } from "../identity.js";
+import { npxRegistrationCommand } from "./ClientRegistrationIdentity.js";
+import {
+ clientConfigurationValuesEqual,
+ clientRegistrationEntry,
+ geminiServerPolicyBlock,
+ parseGeminiMcpPolicy,
+ type ClientConfigurationDocument,
+} from "./ClientConfigurationDocument.js";
+import {
+ readConfigurationFile,
+ readConfigurationText,
+ type ClientConfigurationFileError,
+} from "./ClientConfigurationFiles.js";
+import type { SetupClient } from "./SupportedClients.js";
+
+type SettingsSource = { path: string; document: ClientConfigurationDocument };
+const folderTrustSchema = z
+ .object({
+ security: z
+ .object({
+ folderTrust: z
+ .object({ enabled: z.boolean().optional() })
+ .passthrough()
+ .optional(),
+ })
+ .passthrough()
+ .optional(),
+ })
+ .passthrough();
+const trustRulesSchema = z.record(
+ z.string(),
+ z.enum(["TRUST_FOLDER", "TRUST_PARENT", "DO_NOT_TRUST"]),
+);
+const sourceFailure = (
+ path: string,
+ cause: unknown,
+): ClientConfigurationFileError => ({
+ kind: "malformed",
+ path,
+ detail: `Gemini settings in ${path} are malformed: ${cause instanceof Error ? cause.message : String(cause)}`,
+});
+
+// Gemini resolves symlinks when possible and compares macOS/Windows paths without case.
+const normalizedPath = async (
+ path: string,
+ platform: NodeJS.Platform,
+): Promise => {
+ const physical = await realpath(path).catch(() => path);
+ const absolute = resolve(physical).replaceAll("\\", "/");
+ return platform === "win32" || platform === "darwin"
+ ? absolute.toLowerCase()
+ : absolute;
+};
+
+const workspaceTrusted = async (
+ settings: NonNullable,
+ sources: readonly SettingsSource[],
+): Promise> => {
+ let enabled = true;
+ for (const { path, document } of sources) {
+ const parsed = folderTrustSchema.safeParse(document.document);
+ if (!parsed.success) return err(sourceFailure(path, parsed.error));
+ enabled = parsed.data.security?.folderTrust?.enabled ?? enabled;
+ }
+ if (settings.trustOverride !== undefined) return ok(settings.trustOverride);
+ if (!enabled) return ok(true);
+ const text = await readConfigurationText(settings.trustedFoldersPath);
+ if (!text.ok) return text;
+ if (text.value === undefined) return ok(false);
+ let rules: z.output;
+ try {
+ // Gemini strips comments, but rejects trailing commas, BOMs and invalid trust levels.
+ rules = trustRulesSchema.parse(JSON.parse(stripComments(text.value)));
+ } catch (cause: unknown) {
+ return err(sourceFailure(settings.trustedFoldersPath, cause));
+ }
+ const workspace = await normalizedPath(
+ settings.workspaceDirectory,
+ settings.platform,
+ );
+ let longest = -1;
+ let trusted = false;
+ for (const [path, level] of Object.entries(rules)) {
+ const effective = await normalizedPath(
+ level === "TRUST_PARENT" ? dirname(path) : path,
+ settings.platform,
+ );
+ if (
+ (workspace === effective ||
+ workspace.startsWith(
+ effective.endsWith("/") ? effective : `${effective}/`,
+ )) &&
+ path.length > longest
+ ) {
+ longest = path.length;
+ trusted = level !== "DO_NOT_TRUST";
+ }
+ }
+ return ok(trusted);
+};
+
+/** Check effective Gemini scopes while preserving their authority and source identity. */
+export const readClientPolicyBlock = async (
+ client: SetupClient,
+ userDocument?: ClientConfigurationDocument,
+ desiredRegistration?: unknown,
+): Promise> => {
+ if (client.name !== "gemini_cli" || client.geminiSettings === undefined)
+ return ok(undefined);
+ const settings = client.geminiSettings;
+ const sources: SettingsSource[] = [];
+ const higher: SettingsSource[] = [];
+ for (const path of [
+ settings.defaultsPath,
+ client.configPath,
+ settings.systemPath,
+ ]) {
+ const parsed =
+ path === client.configPath && userDocument !== undefined
+ ? ok(userDocument)
+ : await readConfigurationFile(path, "json");
+ if (!parsed.ok) return parsed;
+ if (parsed.value !== undefined)
+ sources.push({ path, document: parsed.value });
+ }
+ const workspace = await normalizedPath(
+ settings.workspaceDirectory,
+ settings.platform,
+ );
+ const home = await normalizedPath(
+ dirname(dirname(client.configPath)),
+ settings.platform,
+ );
+ if (workspace !== home) {
+ const path = resolve(
+ settings.workspaceDirectory,
+ ".gemini",
+ "settings.json",
+ );
+ const document = await readConfigurationFile(path, "json");
+ if (!document.ok) return document;
+ if (document.value !== undefined) {
+ const trusted = await workspaceTrusted(settings, sources);
+ if (!trusted.ok) return trusted;
+ if (trusted.value) {
+ const source = { path, document: document.value };
+ const systemIndex = sources.findIndex(
+ (item) => item.path === settings.systemPath,
+ );
+ sources.splice(
+ systemIndex < 0 ? sources.length : systemIndex,
+ 0,
+ source,
+ );
+ higher.push(source);
+ }
+ }
+ }
+ const system = sources.find(({ path }) => path === settings.systemPath);
+ if (system !== undefined) higher.push(system);
+ const policies: ReturnType[] = [];
+ for (const { path, document } of sources) {
+ try {
+ policies.push(parseGeminiMcpPolicy(document));
+ } catch (cause: unknown) {
+ return err(sourceFailure(path, cause));
+ }
+ }
+ const policy = geminiServerPolicyBlock(
+ policies,
+ PRODUCT_IDENTITY.mcpServerKey,
+ );
+ if (policy !== undefined)
+ return ok(
+ `${policy} Settings sources: ${sources.map(({ path }) => path).join(", ")}.`,
+ );
+ // mcpServers uses shallow merge: the entire same-name definition is replaced.
+ const override = higher.findLast(({ document }) =>
+ Object.hasOwn(document.servers, PRODUCT_IDENTITY.mcpServerKey),
+ );
+ const desired =
+ desiredRegistration ??
+ userDocument?.servers[PRODUCT_IDENTITY.mcpServerKey] ??
+ clientRegistrationEntry("json", npxRegistrationCommand(), {});
+ if (
+ override !== undefined &&
+ !clientConfigurationValuesEqual(
+ override.document.servers[PRODUCT_IDENTITY.mcpServerKey],
+ desired,
+ )
+ )
+ return ok(
+ `Gemini mcpServers.${PRODUCT_IDENTITY.mcpServerKey} in ${override.path} overrides the user registration. Review that settings file before rerunning setup; REA does not modify system or workspace settings.`,
+ );
+ return ok(undefined);
+};
diff --git a/src/application/ReferenceSourceVcsAdapter.ts b/src/application/ReferenceSourceVcsAdapter.ts
index 42ab4e7388..3ddad25d71 100644
--- a/src/application/ReferenceSourceVcsAdapter.ts
+++ b/src/application/ReferenceSourceVcsAdapter.ts
@@ -2,8 +2,6 @@ import fs from "node:fs";
import { lstat, readFile } from "node:fs/promises";
import { join, resolve } from "node:path";
-import { resolveRef } from "isomorphic-git";
-
export type ReferenceSourceVcsInfo =
| {
readonly kind: "git";
@@ -42,6 +40,8 @@ export const readReferenceSourceVcs = async (
};
const resolveSourceHead = async (root: string): Promise => {
+ // isomorphic-git is loaded on first use so CLI and MCP startup skip it.
+ const { resolveRef } = await import("isomorphic-git");
const dotgit = join(root, ".git");
if (!(await lstat(dotgit)).isFile())
return resolveRef({ fs, dir: root, ref: "HEAD" });
diff --git a/src/application/Setup.fixture.ts b/src/application/Setup.fixture.ts
index b15b7940ec..bf9a978104 100644
--- a/src/application/Setup.fixture.ts
+++ b/src/application/Setup.fixture.ts
@@ -1,3 +1,4 @@
+import { skillDestinations } from "./SetupSkill.js";
import {
type ClientConfigurationInspection,
type ClientConfigurationResult,
@@ -18,7 +19,9 @@ import { setupRegistrationCommand } from "./SetupHost.js";
export class FakeSetupHost implements SetupHost {
readonly platform: NodeJS.Platform;
readonly homeDirectory = "/fixture/home";
- readonly claudeCodeSkillsDirectory = "/fixture/home/.claude/skills";
+ skillDestinations(clientIds: readonly string[]) {
+ return skillDestinations(this.homeDirectory, clientIds);
+ }
readonly registrationCommand: readonly string[];
nodeVersion = "24.18.0";
version: string | undefined = "14.5";
diff --git a/src/application/SetupClientConfiguration.ts b/src/application/SetupClientConfiguration.ts
index 9519808962..5cbf952060 100644
--- a/src/application/SetupClientConfiguration.ts
+++ b/src/application/SetupClientConfiguration.ts
@@ -1,3 +1,5 @@
+import { readClientPolicyBlock } from "./GeminiClientSettings.js";
+import { readClientConfigurationFiles } from "./ClientConfigurationFiles.js";
import {
clientRegistrationEntry,
clientConfigurationValuesEqual,
@@ -33,6 +35,8 @@ export const configureClientConfiguration = (
environment: SetupProviderEnvironment = {},
command: readonly string[] = npxRegistrationCommand(),
): Promise => {
+ if (client.configPathError !== undefined)
+ return Promise.resolve({ status: "failed", reason: "path" });
if (client.format === undefined || client.format === "unsupported")
return Promise.resolve({ status: "failed", reason: "readback" });
return configureClientDocument(client, environment, command, client.format);
@@ -44,6 +48,15 @@ const configureClientDocument = async (
command: readonly string[],
format: NonNullable,
): Promise => {
+ const files = await readClientConfigurationFiles(client);
+ if (!files.ok) return { status: "failed", reason: "readback" };
+ const policy = await readClientPolicyBlock(
+ client,
+ undefined,
+ clientConfigurationDesired(client, environment, command, client.format),
+ );
+ if (!policy.ok || policy.value !== undefined)
+ return { status: "failed", reason: "readback" };
const transactionPath = await resolveClientConfigTransactionPath(
client.configPath,
);
@@ -61,6 +74,13 @@ const configureClientDocument = async (
original ?? (format === "toml" || format === "grok" ? "" : "{}"),
format,
);
+ const policy = await readClientPolicyBlock(
+ client,
+ parsed,
+ clientConfigurationDesired(client, environment, command, parsed.dialect),
+ );
+ if (!policy.ok || policy.value !== undefined)
+ return { status: "failed", reason: "readback" };
} catch (cause: unknown) {
// Malformed existing configuration fails the readback gate.
void cause;
@@ -154,10 +174,14 @@ export const clientConfigurationAligned = async (
providerEnvironment: SetupProviderEnvironment,
command: readonly string[],
): Promise => {
+ if (client.configPathError !== undefined) return false;
try {
+ const files = await readClientConfigurationFiles(client);
+ if (!files.ok) return false;
const original = await readFile(client.configPath, "utf8");
const parsed = parseClientConfiguration(original, client.format);
- return registrationCurrent(
+ const policy = await readClientPolicyBlock(
+ client,
parsed,
clientConfigurationDesired(
client,
@@ -166,6 +190,19 @@ export const clientConfigurationAligned = async (
parsed.dialect,
),
);
+ return (
+ policy.ok &&
+ policy.value === undefined &&
+ registrationCurrent(
+ parsed,
+ clientConfigurationDesired(
+ client,
+ providerEnvironment,
+ command,
+ parsed.dialect,
+ ),
+ )
+ );
} catch (cause: unknown) {
// Unreadable configuration is treated as not aligned so setup repairs it.
void cause;
@@ -179,7 +216,25 @@ export const inspectClientConfiguration = async (
providerEnvironment: SetupProviderEnvironment,
command: readonly string[],
): Promise => {
+ if (client.configPathError !== undefined)
+ return { status: "invalid", remediation: client.configPathError };
if (client.format === "unsupported") return { status: "already_current" };
+ const files = await readClientConfigurationFiles(client);
+ if (!files.ok) return { status: "invalid", remediation: files.error.detail };
+ const policy = await readClientPolicyBlock(
+ client,
+ undefined,
+ clientConfigurationDesired(
+ client,
+ providerEnvironment,
+ command,
+ client.format,
+ ),
+ );
+ if (!policy.ok)
+ return { status: "invalid", remediation: policy.error.detail };
+ if (policy.value !== undefined)
+ return { status: "invalid", remediation: policy.value };
const transactionPath = await resolveClientConfigTransactionPath(
client.configPath,
);
@@ -202,6 +257,20 @@ export const inspectClientConfiguration = async (
}
try {
const parsed = parseClientConfiguration(original, client.format);
+ const policy = await readClientPolicyBlock(
+ client,
+ parsed,
+ clientConfigurationDesired(
+ client,
+ providerEnvironment,
+ command,
+ parsed.dialect,
+ ),
+ );
+ if (!policy.ok)
+ return { status: "invalid", remediation: policy.error.detail };
+ if (policy.value !== undefined)
+ return { status: "invalid", remediation: policy.value };
const desired = clientConfigurationDesired(
client,
providerEnvironment,
diff --git a/src/application/SetupHost.ts b/src/application/SetupHost.ts
index 068b8c012a..ed789fec03 100644
--- a/src/application/SetupHost.ts
+++ b/src/application/SetupHost.ts
@@ -14,9 +14,13 @@ import {
systemLinuxHopperInstallHost,
} from "./LinuxHopper.js";
import { installMacHopper, systemMacHopperInstallHost } from "./MacHopper.js";
-import { supportedClients, type SetupClient } from "./SupportedClients.js";
import {
- claudeCodeSkillsDirectory as resolveClaudeCodeSkillsDirectory,
+ clientEvidencePaths,
+ supportedClients,
+ type SetupClient,
+} from "./SupportedClients.js";
+import {
+ skillDestinations,
canonicalSkillNeedsInstall,
installCanonicalSkill,
} from "./SetupSkill.js";
@@ -90,14 +94,11 @@ export const systemSetupHost = (
const doctorHost = selectedDoctorHost ?? systemDoctorHost({ environment });
const platform = doctorHost.platform;
const { homeDirectory } = doctorHost;
- const claudeSkillsDirectory = resolveClaudeCodeSkillsDirectory(
- homeDirectory,
- environment,
- );
return {
platform,
homeDirectory,
- claudeCodeSkillsDirectory: claudeSkillsDirectory,
+ skillDestinations: (clientIds) =>
+ skillDestinations(homeDirectory, clientIds, environment, platform),
registrationCommand: setupRegistrationCommand(
platform,
environment.npm_command === "exec",
@@ -153,10 +154,11 @@ export const systemSetupHost = (
canonicalSkillNeedsInstall(
homeDirectory,
clientIds,
- claudeSkillsDirectory,
+ environment,
+ platform,
),
installSkill: (clientIds) =>
- installCanonicalSkill(homeDirectory, clientIds, claudeSkillsDirectory),
+ installCanonicalSkill(homeDirectory, clientIds, environment, platform),
doctor: (scope) => runDoctor(undefined, doctorHost, scope),
};
};
@@ -169,17 +171,20 @@ export const detectClients = async (
): Promise => {
const detected: SetupClient[] = [];
for (const candidate of supportedClients(home, platform, environment)) {
- const [hasConfig, hasMarker] = await Promise.all([
- exists(candidate.configPath),
- candidate.markerPath === undefined ? false : exists(candidate.markerPath),
- ]);
- if (hasConfig || hasMarker) detected.push(candidate);
+ if (candidate.configPathError !== undefined) continue;
+ if (await clientEvidencePresent(candidate)) detected.push(candidate);
}
return detected;
};
const major = (version: string): number =>
Number.parseInt(version.split(".")[0] ?? "0", 10);
+const clientEvidencePresent = async (client: SetupClient): Promise => {
+ for (const path of clientEvidencePaths(client))
+ if (await exists(path)) return true;
+ return false;
+};
+
const exists = async (path: string): Promise => {
try {
await access(path);
diff --git a/src/application/SetupPlan.ts b/src/application/SetupPlan.ts
index fc0d978a37..20f1e77a22 100644
--- a/src/application/SetupPlan.ts
+++ b/src/application/SetupPlan.ts
@@ -5,7 +5,7 @@ import {
linuxHopperInstallDisclosure,
type LinuxPackageFamily,
} from "./LinuxHopper.js";
-import { skillDestinations } from "./SetupSkill.js";
+import type { SkillDestination } from "./SetupSkill.js";
import { macHopperInstallDisclosure } from "./MacHopper.js";
import type {
SetupAction,
@@ -188,7 +188,7 @@ export const planSetupActions = async (input: {
installHopper: input.discovery.installHopper,
installSkill: input.installSkill,
skillClientIds: input.skillClientIds,
- claudeCodeSkillsDirectory: input.host.claudeCodeSkillsDirectory,
+ skillDestinations: input.host.skillDestinations(input.skillClientIds),
clients: clientPlans,
providerEnvironment: input.providerEnvironment,
...(input.discovery.linuxPackageFamily === undefined
@@ -210,7 +210,7 @@ const setupPlan = (input: {
readonly installHopper: boolean;
readonly installSkill: boolean;
readonly skillClientIds: readonly string[];
- readonly claudeCodeSkillsDirectory: string;
+ readonly skillDestinations: readonly SkillDestination[];
readonly clients: readonly {
readonly client: SetupClient;
readonly operation: "create" | "update";
@@ -268,15 +268,8 @@ const setupPlan = (input: {
...(backupPath === undefined ? {} : { backupPath }),
})),
...(input.installSkill
- ? skillDestinations(
- input.homeDirectory,
- input.skillClientIds,
- input.claudeCodeSkillsDirectory,
- ).map(({ client, path }): SetupAction => ({
- id:
- client === "claude_code"
- ? "install_skill:claude_code"
- : "install_skill",
+ ? input.skillDestinations.map(({ client, path }): SetupAction => ({
+ id: client === "shared" ? "install_skill" : `install_skill:${client}`,
kind: "install_skill",
label:
client === "claude_code"
diff --git a/src/application/SetupSkill.ts b/src/application/SetupSkill.ts
index 19a306f9aa..70924d28f7 100644
--- a/src/application/SetupSkill.ts
+++ b/src/application/SetupSkill.ts
@@ -3,6 +3,7 @@ import { dirname, join } from "node:path";
import writeFileAtomic from "write-file-atomic";
import { PRODUCT_IDENTITY } from "../identity.js";
+import { clientSkillDirectories } from "./SupportedClients.js";
const SKILL_FILES = [
"SKILL.md",
@@ -21,7 +22,7 @@ interface CanonicalSkillFile {
/** Managed skill location selected for a client family. */
export interface SkillDestination {
- readonly client: "shared" | "claude_code";
+ readonly client: string;
readonly path: string;
}
@@ -32,44 +33,19 @@ export interface InstalledSkillIdentity {
readonly canonical: boolean;
}
-/** Resolve Claude Code personal skills from the selected environment. */
-export const claudeCodeSkillsDirectory = (
- home: string,
- environment: { readonly CLAUDE_CONFIG_DIR?: string } = {},
-): string =>
- join(environment.CLAUDE_CONFIG_DIR ?? join(home, ".claude"), "skills");
-
/** Resolve the skill roots required by a client selection, or all owned roots. */
export const skillDestinations = (
home: string,
clientIds: readonly string[] | undefined,
- claudeSkillsDirectory = claudeCodeSkillsDirectory(home),
-): readonly SkillDestination[] => {
- const includeClaude =
- clientIds === undefined || clientIds.includes("claude_code");
- const includeShared =
- clientIds === undefined ||
- clientIds.length === 0 ||
- clientIds.some((clientId) => clientId !== "claude_code");
- return [
- ...(includeShared
- ? [
- {
- client: "shared" as const,
- path: join(home, ".agents/skills", PRODUCT_IDENTITY.skillName),
- },
- ]
- : []),
- ...(includeClaude
- ? [
- {
- client: "claude_code" as const,
- path: join(claudeSkillsDirectory, PRODUCT_IDENTITY.skillName),
- },
- ]
- : []),
- ];
-};
+ environment: Readonly = {},
+ platform: NodeJS.Platform = process.platform,
+): readonly SkillDestination[] =>
+ clientSkillDirectories(home, clientIds, environment, platform).map(
+ ({ client, directory }) => ({
+ client,
+ path: join(directory, PRODUCT_IDENTITY.skillName),
+ }),
+ );
const readOptionalText = async (path: string): Promise => {
try {
@@ -84,10 +60,11 @@ const readOptionalText = async (path: string): Promise => {
const canonicalSkillFiles = async (
home: string,
clientIds: readonly string[],
- claudeSkillsDirectory?: string,
+ environment: Readonly = {},
+ platform: NodeJS.Platform = process.platform,
): Promise =>
Promise.all(
- skillDestinations(home, clientIds, claudeSkillsDirectory).flatMap(
+ skillDestinations(home, clientIds, environment, platform).flatMap(
({ path: root }) =>
SKILL_FILES.map(async (relativePath) => {
const destination = join(root, relativePath);
@@ -110,11 +87,12 @@ const canonicalSkillFiles = async (
export const canonicalSkillNeedsInstall = async (
home: string,
clientIds: readonly string[] = [],
- claudeSkillsDirectory?: string,
+ environment: Readonly = {},
+ platform: NodeJS.Platform = process.platform,
): Promise => {
try {
return (
- await canonicalSkillFiles(home, clientIds, claudeSkillsDirectory)
+ await canonicalSkillFiles(home, clientIds, environment, platform)
).some(({ content, original }) => original !== content);
} catch (cause: unknown) {
// Unreadable skill state fails open to install so setup can repair it.
@@ -139,14 +117,16 @@ const restoreSkillFiles = async (
export const installCanonicalSkill = async (
home: string,
clientIds: readonly string[] = [],
- claudeSkillsDirectory?: string,
+ environment: Readonly = {},
+ platform: NodeJS.Platform = process.platform,
): Promise<"installed" | "unchanged" | "failed"> => {
let changed: readonly CanonicalSkillFile[] = [];
try {
const canonical = await canonicalSkillFiles(
home,
clientIds,
- claudeSkillsDirectory,
+ environment,
+ platform,
);
changed = canonical.filter(({ content, original }) => original !== content);
if (changed.length === 0) return "unchanged";
@@ -179,12 +159,14 @@ export const installCanonicalSkill = async (
export const readInstalledSkillIdentity = async (
home: string,
clientIds: readonly string[],
- claudeSkillsDirectory?: string,
+ environment: Readonly = {},
+ platform: NodeJS.Platform = process.platform,
): Promise => {
const destinations = skillDestinations(
home,
clientIds,
- claudeSkillsDirectory,
+ environment,
+ platform,
);
let content: string | undefined;
for (const destination of destinations) {
@@ -202,7 +184,8 @@ export const readInstalledSkillIdentity = async (
canonical: !(await canonicalSkillNeedsInstall(
home,
clientIds,
- claudeSkillsDirectory,
+ environment,
+ platform,
)),
version: /^\s{2}version:\s*"([^"]+)"\s*$/mu.exec(content)?.[1] ?? null,
toolCount: toolCount === undefined ? null : Number.parseInt(toolCount, 10),
diff --git a/src/application/SetupTypes.ts b/src/application/SetupTypes.ts
index 08eb0df994..8a1b999613 100644
--- a/src/application/SetupTypes.ts
+++ b/src/application/SetupTypes.ts
@@ -1,5 +1,6 @@
import type { DoctorReport, DoctorScope, runDoctor } from "./Doctor.js";
import type { LinuxDistribution } from "./LinuxHopper.js";
+import type { SkillDestination } from "./SetupSkill.js";
import type { SetupClient } from "./SupportedClients.js";
import type {
SetupFailureCode,
@@ -42,7 +43,7 @@ export type ClientConfigurationInspection =
export interface SetupHost {
readonly platform: NodeJS.Platform;
readonly homeDirectory: string;
- readonly claudeCodeSkillsDirectory: string;
+ skillDestinations(clientIds: readonly string[]): readonly SkillDestination[];
readonly registrationCommand: readonly string[];
readonly nodeVersion: string;
macosVersion(): Promise;
diff --git a/src/application/SupportedClients.ts b/src/application/SupportedClients.ts
index 97467a227c..dfeee34b7c 100644
--- a/src/application/SupportedClients.ts
+++ b/src/application/SupportedClients.ts
@@ -1,4 +1,5 @@
-import { isAbsolute, join } from "node:path";
+import { dirname, isAbsolute, join, resolve, win32 } from "node:path";
+import { fileURLToPath } from "node:url";
import { lstatSync } from "node:fs";
/** One supported client configuration location. */
@@ -6,6 +7,19 @@ export interface SetupClient {
readonly name: string;
readonly displayName?: string;
readonly configPath: string;
+ /** User configuration files in increasing precedence, including the write target. */
+ readonly configPaths?: readonly string[];
+ /** Read-only Gemini scopes and trust inputs; never setup write targets. */
+ readonly geminiSettings?: {
+ readonly systemPath: string;
+ readonly defaultsPath: string;
+ readonly workspaceDirectory: string;
+ readonly trustedFoldersPath: string;
+ readonly trustOverride?: boolean;
+ readonly platform: NodeJS.Platform;
+ };
+ /** Unresolved path evidence only; consumers must not perform I/O on it. */
+ readonly configPathError?: string;
readonly markerPath?: string;
readonly format?:
| "json"
@@ -16,6 +30,8 @@ export interface SetupClient {
| "commandcode"
| "grok"
| "omp"
+ | "pi"
+ | "hermes"
| "unsupported";
}
@@ -24,20 +40,7 @@ type ClientPath = readonly string[] | ((paths: ClientPathContext) => string);
interface ClientPathContext {
readonly home: string;
readonly platform: NodeJS.Platform;
- readonly env: {
- readonly APPDATA?: string | undefined;
- readonly CLAUDE_CONFIG_DIR?: string | undefined;
- readonly CODEX_HOME?: string | undefined;
- readonly COPILOT_HOME?: string | undefined;
- readonly GROK_HOME?: string | undefined;
- readonly OMP_PROFILE?: string | undefined;
- readonly OPENCODE_CONFIG?: string | undefined;
- readonly PI_CODING_AGENT_DIR?: string | undefined;
- readonly PI_CONFIG_DIR?: string | undefined;
- readonly PI_PROFILE?: string | undefined;
- readonly SAND_DATA_ROOT?: string | undefined;
- readonly XDG_CONFIG_HOME?: string | undefined;
- };
+ readonly env: Readonly;
}
interface ClientDefinition {
@@ -46,6 +49,11 @@ interface ClientDefinition {
readonly configPath: ClientPath;
readonly markerPath: ClientPath;
readonly format: NonNullable;
+ readonly skillPath?: ClientPath;
+ readonly configPaths?: (context: ClientPathContext) => readonly string[];
+ readonly geminiSettings?: (
+ context: ClientPathContext,
+ ) => NonNullable;
}
const vscodeUserDirectory = ({
@@ -83,10 +91,73 @@ const claudeCodeMarkerDirectory = ({ home, env }: ClientPathContext): string =>
env.CLAUDE_CONFIG_DIR ?? join(home, ".claude");
const codexDirectory = ({ home, env }: ClientPathContext): string =>
- env.CODEX_HOME ?? join(home, ".codex");
+ env.CODEX_HOME || join(home, ".codex");
+
+/** Match Qwen Code's home override, including tilde and cwd-relative paths. */
+const qwenCodeDirectory = ({ home, env }: ClientPathContext): string => {
+ const override = env.QWEN_HOME;
+ if (!override) return join(home, ".qwen");
+ const expanded =
+ override === "~"
+ ? home
+ : override.startsWith("~/") || override.startsWith("~\\")
+ ? join(
+ home,
+ ...override
+ .slice(2)
+ .split(/[/\\]+/)
+ .filter(Boolean),
+ )
+ : override;
+ return isAbsolute(expanded) ? expanded : resolve(expanded);
+};
const grokDirectory = ({ home, env }: ClientPathContext): string =>
- env.GROK_HOME ?? join(home, ".grok");
+ env.GROK_HOME || join(home, ".grok");
+
+/**
+ * Hermes Agent home. `HERMES_HOME` wins; otherwise the platform default mirrors
+ * Hermes's own resolver (`%LOCALAPPDATA%\\hermes` on Windows, `~/.hermes`
+ * elsewhere), including the optional `HERMES_DATA_DIR_SUFFIX` that is appended
+ * to the literal `hermes`/`.hermes` directory name.
+ */
+const hermesDirectory = ({
+ home,
+ platform,
+ env,
+}: ClientPathContext): string => {
+ const override = env.HERMES_HOME?.trim();
+ if (override) {
+ const expandedVariables = override.replace(
+ /\$(\w+|\{[^}]*\})/gu,
+ (match, variable: string) => {
+ const key = variable.startsWith("{") ? variable.slice(1, -1) : variable;
+ return env[key] ?? match;
+ },
+ );
+ const expanded =
+ platform === "win32"
+ ? expandedVariables.replace(
+ /%([^%]+)%/gu,
+ (match, variable: string) => env[variable] ?? match,
+ )
+ : expandedVariables;
+ if (expanded === "~") return home;
+ if (
+ expanded.startsWith("~/") ||
+ (platform === "win32" && expanded.startsWith("~\\"))
+ )
+ return join(home, expanded.slice(2));
+ return resolve(expanded);
+ }
+ const suffix = env.HERMES_DATA_DIR_SUFFIX ?? "";
+ return platform === "win32"
+ ? join(
+ env.LOCALAPPDATA?.trim() || join(home, "AppData", "Local"),
+ `hermes${suffix}`,
+ )
+ : join(home, `.hermes${suffix}`);
+};
/** Grok Bot uses an absolute SAND_DATA_ROOT; anything else stays ~/.grokbot. */
const grokBotDirectory = ({ home, env }: ClientPathContext): string => {
@@ -132,17 +203,49 @@ const ompProfile = ({ env }: ClientPathContext): string | undefined => {
/**
* The OMP agent directory whose `mcp.json` holds user-scope MCP servers. OMP
* joins PI_CONFIG_DIR to the home directory, and a named profile ignores
- * PI_CODING_AGENT_DIR. OMP resolves a relative agent directory against each
- * process's working directory, so only an absolute override names one file.
+ * PI_CODING_AGENT_DIR. Relative overrides use the setup process working directory.
*/
const ompAgentDirectory = (context: ClientPathContext): string => {
const root = join(context.home, context.env.PI_CONFIG_DIR || ".omp");
const profile = ompProfile(context);
if (profile !== undefined) return join(root, "profiles", profile, "agent");
const override = context.env.PI_CODING_AGENT_DIR;
- return override !== undefined && override !== "" && isAbsolute(override)
- ? override
- : join(root, "agent");
+ return override ? resolve(override) : join(root, "agent");
+};
+
+/**
+ * Pi's public getAgentDir()/normalizePath semantics, not OMP's profiles.
+ * Keep relative overrides relative: filesystem consumers resolve them in cwd.
+ */
+const piAgentDirectory = ({
+ home,
+ platform,
+ env,
+}: ClientPathContext): string => {
+ const paths = platform === "win32" ? win32 : { join };
+ let directory = env.PI_CODING_AGENT_DIR;
+ if (!directory) return paths.join(home, ".pi", "agent");
+ if (
+ platform === "win32" &&
+ directory.startsWith("/") &&
+ !directory.startsWith("//") &&
+ !directory.includes("\\")
+ ) {
+ const drive = /^\/(?:mnt\/|cygdrive\/)?([a-z])(?:\/(.*))?$/iu.exec(
+ directory,
+ );
+ if (drive !== null)
+ directory = `${drive[1]?.toUpperCase()}:\\${drive[2]?.replaceAll("/", "\\") ?? ""}`;
+ }
+ if (directory === "~") return home;
+ if (
+ directory.startsWith("~/") ||
+ (platform === "win32" && directory.startsWith("~\\"))
+ )
+ return paths.join(home, directory.slice(2));
+ if (directory.startsWith("file://"))
+ return fileURLToPath(directory, { windows: platform === "win32" });
+ return directory;
};
const copilotDirectory = ({ home, env }: ClientPathContext): string =>
@@ -151,21 +254,66 @@ const copilotDirectory = ({ home, env }: ClientPathContext): string =>
const commandCodeDirectory = ({ home }: ClientPathContext): string =>
join(home, ".commandcode");
-const opencodeDirectory = ({ home, env }: ClientPathContext): string =>
- join(env.XDG_CONFIG_HOME ?? join(home, ".config"), "opencode");
+/**
+ * OpenCode appends `OPENCODE_CONFIG_DIR` only when the value is non-empty.
+ * An empty variable is the same as unset and leaves the XDG directory in place.
+ */
+const openCodeOverrideDirectory = (
+ env: ClientPathContext["env"],
+): string | undefined => {
+ const directory = env.OPENCODE_CONFIG_DIR;
+ return directory === undefined || directory === "" ? undefined : directory;
+};
+
+const opencodeDirectory = (context: ClientPathContext): string =>
+ openCodeOverrideDirectory(context.env) ??
+ join(
+ context.env.XDG_CONFIG_HOME ?? join(context.home, ".config"),
+ "opencode",
+ );
+
+/** OpenCode loads global JSON before JSONC, then explicit file and directory overrides. */
+const openCodeConfigurationPaths = (
+ context: ClientPathContext,
+): readonly string[] => {
+ const directory = join(
+ context.env.XDG_CONFIG_HOME ?? join(context.home, ".config"),
+ "opencode",
+ );
+ const overrideDirectory = openCodeOverrideDirectory(context.env);
+ const paths = [
+ join(directory, "config.json"),
+ join(directory, "opencode.json"),
+ join(directory, "opencode.jsonc"),
+ ...(context.env.OPENCODE_CONFIG ? [context.env.OPENCODE_CONFIG] : []),
+ ...(overrideDirectory !== undefined
+ ? [
+ join(overrideDirectory, "opencode.json"),
+ join(overrideDirectory, "opencode.jsonc"),
+ ]
+ : []),
+ ];
+ const resolved = paths.map((path) => resolve(path));
+ return resolved.filter((path, index) => resolved.lastIndexOf(path) === index);
+};
const existingOpenCodeConfigPath = (context: ClientPathContext): string => {
- if (context.env.OPENCODE_CONFIG !== undefined)
- return context.env.OPENCODE_CONFIG;
- const directory = opencodeDirectory(context);
- const candidates = [
- "opencode.json",
- "opencode.jsonc",
- ".opencode.json",
- ".opencode.jsonc",
- ].map((filename) => join(directory, filename));
+ // The selected directory is loaded after an explicit file by OpenCode V1.
+ const overrideDirectory = openCodeOverrideDirectory(context.env);
+ const candidates =
+ overrideDirectory !== undefined
+ ? [
+ join(overrideDirectory, "opencode.json"),
+ join(overrideDirectory, "opencode.jsonc"),
+ ].map((path) => resolve(path))
+ : context.env.OPENCODE_CONFIG
+ ? [resolve(context.env.OPENCODE_CONFIG)]
+ : [
+ join(opencodeDirectory(context), "opencode.json"),
+ join(opencodeDirectory(context), "opencode.jsonc"),
+ ].map((path) => resolve(path));
return (
- candidates.find((path) => {
+ [...candidates].reverse().find((path) => {
try {
lstatSync(path);
return true;
@@ -177,21 +325,60 @@ const existingOpenCodeConfigPath = (context: ClientPathContext): string => {
);
}
}) ??
- candidates[0] ??
- join(directory, "opencode.json")
+ resolve(
+ overrideDirectory !== undefined
+ ? join(overrideDirectory, "opencode.json")
+ : context.env.OPENCODE_CONFIG ||
+ join(opencodeDirectory(context), "opencode.json"),
+ )
);
};
+const geminiSettings = ({
+ home,
+ platform,
+ env,
+}: ClientPathContext): NonNullable => {
+ const system =
+ env.GEMINI_CLI_SYSTEM_SETTINGS_PATH ||
+ (platform === "darwin"
+ ? "/Library/Application Support/GeminiCli/settings.json"
+ : platform === "win32"
+ ? "C:\\ProgramData\\gemini-cli\\settings.json"
+ : "/etc/gemini-cli/settings.json");
+ const trustOverride =
+ env.GEMINI_RESTRICTED_MODE === "true" ||
+ env.GEMINI_CLI_TRUST_WORKSPACE === "false"
+ ? false
+ : env.GEMINI_CLI_TRUST_WORKSPACE === "true"
+ ? true
+ : undefined;
+ return {
+ systemPath: system,
+ defaultsPath:
+ env.GEMINI_CLI_SYSTEM_DEFAULTS_PATH ||
+ join(dirname(system), "system-defaults.json"),
+ workspaceDirectory: process.cwd(),
+ trustedFoldersPath:
+ env.GEMINI_CLI_TRUSTED_FOLDERS_PATH ||
+ join(env.GEMINI_CLI_HOME || home, ".gemini", "trustedFolders.json"),
+ ...(trustOverride === undefined ? {} : { trustOverride }),
+ platform,
+ };
+};
+
const devinDirectory = ({ home, platform, env }: ClientPathContext): string =>
platform === "win32"
? join(env.APPDATA ?? join(home, "AppData", "Roaming"), "devin")
- : join(home, ".config", "devin");
+ : join(env.XDG_CONFIG_HOME || join(home, ".config"), "devin");
/** Stable product metadata used to derive setup discovery and documentation. */
export const SUPPORTED_CLIENT_DEFINITIONS = [
{
name: "claude_code",
displayName: "Claude Code",
+ skillPath: ({ home, env }: ClientPathContext) =>
+ join(env.CLAUDE_CONFIG_DIR ?? join(home, ".claude"), "skills"),
configPath: (context: ClientPathContext) =>
join(claudeCodeConfigDirectory(context), ".claude.json"),
markerPath: claudeCodeMarkerDirectory,
@@ -223,8 +410,13 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
{
name: "gemini_cli",
displayName: "Gemini CLI",
- configPath: [".gemini", "settings.json"],
- markerPath: [".gemini"],
+ geminiSettings,
+ configPath: ({ home, env }: ClientPathContext) =>
+ join(env.GEMINI_CLI_HOME || home, ".gemini", "settings.json"),
+ markerPath: ({ home, env }: ClientPathContext) =>
+ join(env.GEMINI_CLI_HOME || home, ".gemini"),
+ skillPath: ({ home, env }: ClientPathContext) =>
+ join(env.GEMINI_CLI_HOME || home, ".agents", "skills"),
format: "json",
},
{
@@ -246,12 +438,14 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
name: "opencode",
displayName: "OpenCode",
configPath: existingOpenCodeConfigPath,
+ configPaths: openCodeConfigurationPaths,
markerPath: opencodeDirectory,
format: "opencode",
},
{
name: "antigravity",
displayName: "Antigravity",
+ skillPath: [".gemini", "config", "skills"],
configPath: [".gemini", "config", "mcp_config.json"],
markerPath: [".gemini", "config"],
format: "json",
@@ -272,6 +466,14 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
markerPath: commandCodeDirectory,
format: "commandcode",
},
+ {
+ name: "qwen_code",
+ displayName: "Qwen Code",
+ configPath: (context: ClientPathContext) =>
+ join(qwenCodeDirectory(context), "settings.json"),
+ markerPath: qwenCodeDirectory,
+ format: "json",
+ },
{
name: "vscode",
displayName: "VS Code",
@@ -296,6 +498,27 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
markerPath: ompAgentDirectory,
format: "omp",
},
+ {
+ name: "pi",
+ displayName: "Pi",
+ configPath: (context: ClientPathContext) =>
+ (context.platform === "win32" ? win32 : { join }).join(
+ piAgentDirectory(context),
+ "mcp.json",
+ ),
+ markerPath: piAgentDirectory,
+ format: "pi",
+ },
+ {
+ name: "hermes",
+ displayName: "Hermes",
+ skillPath: (context: ClientPathContext) =>
+ join(hermesDirectory(context), "skills"),
+ configPath: (context: ClientPathContext) =>
+ join(hermesDirectory(context), "config.yaml"),
+ markerPath: hermesDirectory,
+ format: "hermes",
+ },
{
name: "grok_bot",
displayName: "Grok Bot",
@@ -305,6 +528,23 @@ export const SUPPORTED_CLIENT_DEFINITIONS = [
},
] as const satisfies readonly ClientDefinition[];
+/**
+ * Paths whose presence means doctor and setup should inspect the client.
+ * Layered sources count: OpenCode still loads its XDG files when a custom
+ * directory override does not exist.
+ */
+export const clientEvidencePaths = (
+ client: Pick,
+): readonly string[] => [
+ ...new Set(
+ [
+ client.configPath,
+ ...(client.configPaths ?? []),
+ client.markerPath,
+ ].filter((path): path is string => path !== undefined && path.length > 0),
+ ),
+];
+
const resolvePath = (path: ClientPath, context: ClientPathContext): string =>
typeof path === "function" ? path(context) : join(context.home, ...path);
@@ -312,27 +552,81 @@ const resolvePath = (path: ClientPath, context: ClientPathContext): string =>
export const supportedClients = (
home: string,
platform: NodeJS.Platform = process.platform,
- env: ClientPathContext["env"] = {
- APPDATA: process.env.APPDATA,
- CLAUDE_CONFIG_DIR: process.env.CLAUDE_CONFIG_DIR,
- CODEX_HOME: process.env.CODEX_HOME,
- COPILOT_HOME: process.env.COPILOT_HOME,
- GROK_HOME: process.env.GROK_HOME,
- OMP_PROFILE: process.env.OMP_PROFILE,
- OPENCODE_CONFIG: process.env.OPENCODE_CONFIG,
- PI_CODING_AGENT_DIR: process.env.PI_CODING_AGENT_DIR,
- PI_CONFIG_DIR: process.env.PI_CONFIG_DIR,
- PI_PROFILE: process.env.PI_PROFILE,
- SAND_DATA_ROOT: process.env.SAND_DATA_ROOT,
- XDG_CONFIG_HOME: process.env.XDG_CONFIG_HOME,
- },
+ env: ClientPathContext["env"] = process.env,
): readonly SetupClient[] => {
const context = { home, platform, env };
- return SUPPORTED_CLIENT_DEFINITIONS.map((definition) => ({
- name: definition.name,
- displayName: definition.displayName,
- configPath: resolvePath(definition.configPath, context),
- markerPath: resolvePath(definition.markerPath, context),
- format: definition.format,
- }));
+ const definitions: readonly ClientDefinition[] = SUPPORTED_CLIENT_DEFINITIONS;
+ return definitions.map((definition) => {
+ try {
+ const configPath = resolvePath(definition.configPath, context);
+ return {
+ name: definition.name,
+ displayName: definition.displayName,
+ configPath,
+ ...(definition.geminiSettings === undefined
+ ? {}
+ : { geminiSettings: definition.geminiSettings(context) }),
+ ...(definition.configPaths === undefined
+ ? {}
+ : {
+ configPaths: [
+ ...new Set([...definition.configPaths(context), configPath]),
+ ],
+ }),
+ markerPath: resolvePath(definition.markerPath, context),
+ format: definition.format,
+ };
+ } catch (cause: unknown) {
+ if (definition.name !== "pi") throw cause;
+ return {
+ name: definition.name,
+ displayName: definition.displayName,
+ format: definition.format,
+ configPath: env.PI_CODING_AGENT_DIR ?? "",
+ configPathError: `Invalid PI_CODING_AGENT_DIR: ${cause instanceof Error ? cause.message : String(cause)}. Repair the override before configuring Pi.`,
+ };
+ }
+ });
+};
+
+/** Resolve personal skill roots using the same client environment as setup. */
+export const clientSkillDirectories = (
+ home: string,
+ clientIds: readonly string[] | undefined,
+ environment: Readonly = {},
+ platform: NodeJS.Platform = process.platform,
+): readonly { readonly client: string; readonly directory: string }[] => {
+ const definitions: readonly ClientDefinition[] = SUPPORTED_CLIENT_DEFINITIONS;
+ const selected =
+ clientIds === undefined
+ ? [
+ "shared",
+ ...definitions
+ .filter(({ skillPath }) => skillPath !== undefined)
+ .map(({ name }) => name),
+ ]
+ : clientIds.length === 0
+ ? ["shared"]
+ : clientIds;
+ const destinations = new Map();
+ for (const client of selected) {
+ const definition = definitions.find(({ name }) => name === client);
+ const directory =
+ definition?.skillPath === undefined
+ ? join(home, ".agents", "skills")
+ : resolvePath(definition.skillPath, {
+ home,
+ platform,
+ env: environment,
+ });
+ if (!destinations.has(directory))
+ destinations.set(directory, {
+ client:
+ directory === join(home, ".agents", "skills") ? "shared" : client,
+ directory,
+ });
+ }
+ return [...destinations.values()].sort((left, right) =>
+ left.directory.localeCompare(right.directory),
+ );
};
diff --git a/src/application/Uninstall.ts b/src/application/Uninstall.ts
index 9848e91dae..bc8b180dc6 100644
--- a/src/application/Uninstall.ts
+++ b/src/application/Uninstall.ts
@@ -15,7 +15,7 @@ import writeFileAtomic from "write-file-atomic";
import { z } from "zod";
import { PRODUCT_IDENTITY } from "../identity.js";
-import { claudeCodeSkillsDirectory, skillDestinations } from "./SetupSkill.js";
+import { skillDestinations } from "./SetupSkill.js";
import { isOwnedClientRegistrationCommand } from "./ClientRegistrationIdentity.js";
import { resolveClientConfigTransactionPath } from "./ClientConfigPath.js";
import {
@@ -124,11 +124,12 @@ export const runUninstall = async (
status: "retained",
detail: "Hopper is not owned by REA uninstall.",
});
+ const reported = collapseUninstallItems(items);
return {
- status: items.some(({ status }) => status === "failed")
+ status: reported.some(({ status }) => status === "failed")
? "failed"
: "complete",
- items,
+ items: reported,
};
};
@@ -171,7 +172,16 @@ export const systemUninstallHost = (
selectedHome ?? homeDirectoryFromEnvironment(environment, platform);
return {
clients: () =>
- Promise.resolve(supportedClients(home, platform, environment)),
+ Promise.resolve(
+ supportedClients(home, platform, environment).flatMap((client) =>
+ client.configPaths === undefined
+ ? [client]
+ : client.configPaths.map((configPath) => ({
+ ...client,
+ configPath,
+ })),
+ ),
+ ),
inspectClient: async (client) => {
const read = await readClientConfiguration(client, fileSystem);
return read.kind === "item" && read.item.status === "failed"
@@ -179,7 +189,8 @@ export const systemUninstallHost = (
: undefined;
},
removeClient: (client) => removeClient(client, fileSystem),
- removeSkill: () => removeManagedSkills(home, fileSystem, environment),
+ removeSkill: () =>
+ removeManagedSkills(home, fileSystem, environment, platform),
purgeData: async () => [
await removeManagedPath(join(home, ".rea/cache"), "cache", fileSystem),
await removeManagedPath(join(home, ".rea/state"), "state", fileSystem),
@@ -200,6 +211,8 @@ const readClientConfiguration = async (
client: SetupClient,
fileSystem: UninstallFileSystem,
): Promise => {
+ if (client.configPathError !== undefined)
+ return itemRead(item(client.name, "failed", client.configPathError));
if (client.format === "unsupported")
return itemRead(
item(
@@ -404,18 +417,16 @@ const removeManagedSkills = async (
home: string,
fileSystem: UninstallFileSystem,
environment: Readonly,
+ platform: NodeJS.Platform,
): Promise => {
const results = await Promise.all(
- skillDestinations(
- home,
- undefined,
- claudeCodeSkillsDirectory(home, environment),
- ).map(({ client, path }) =>
- removeManagedPath(
- path,
- client === "claude_code" ? "Claude Code skill" : "skill",
- fileSystem,
- ),
+ skillDestinations(home, undefined, environment, platform).map(
+ ({ client, path }) =>
+ removeManagedPath(
+ path,
+ client === "claude_code" ? "Claude Code skill" : "skill",
+ fileSystem,
+ ),
),
);
const failed = results.find(({ status }) => status === "failed");
@@ -451,6 +462,68 @@ const item = (
status: UninstallItem["status"],
detail: string,
): UninstallItem => ({ name, status, detail });
+
+/**
+ * Layered clients are removed file by file, but callers look up one status by
+ * client name. A missing earlier file must not hide a later removal.
+ */
+const collapseUninstallItems = (
+ items: readonly UninstallItem[],
+): UninstallItem[] => {
+ const collapsed: UninstallItem[] = [];
+ for (const next of items) {
+ const index = collapsed.findIndex(({ name }) => name === next.name);
+ const current = collapsed[index];
+ if (current === undefined) {
+ collapsed.push(next);
+ continue;
+ }
+ collapsed[index] = mergeUninstallItems(current, next);
+ }
+ return collapsed;
+};
+
+const mergeUninstallItems = (
+ current: UninstallItem,
+ next: UninstallItem,
+): UninstallItem => {
+ const status = preferredUninstallStatus(current.status, next.status);
+ const details = [current, next]
+ .filter(
+ (entry) =>
+ entry.status === status ||
+ (status === "removed" && entry.status === "retained"),
+ )
+ .map(({ detail }) => detail);
+ return item(current.name, status, [...new Set(details)].join(" "));
+};
+
+const preferredUninstallStatus = (
+ current: UninstallItem["status"],
+ next: UninstallItem["status"],
+): UninstallItem["status"] =>
+ uninstallStatusPriority(next) > uninstallStatusPriority(current)
+ ? next
+ : current;
+
+const uninstallStatusPriority = (status: UninstallItem["status"]): number => {
+ switch (status) {
+ case "failed":
+ return 3;
+ case "removed":
+ return 2;
+ case "retained":
+ return 1;
+ case "skipped":
+ return 0;
+ default: {
+ const exhaustive: never = status;
+ throw new TypeError(
+ `Unhandled uninstall item status: ${String(exhaustive)}`,
+ );
+ }
+ }
+};
const registrationSchema = z
.object({ command: z.string(), args: z.array(z.string()) })
.passthrough();
diff --git a/src/application/Utf8JsonInput.ts b/src/application/Utf8JsonInput.ts
index 86204e3c90..0e8ac8bb6f 100644
--- a/src/application/Utf8JsonInput.ts
+++ b/src/application/Utf8JsonInput.ts
@@ -7,6 +7,13 @@ import { safeParseJson, type SafeJsonParseResult } from "../domain/safeJson.js";
export const JSON_INPUT_RESOURCE_REMEDIATION =
"Provide a smaller valid JSON value. For Evidence workflows, re-analyze a smaller selection of the original target and use its complete Evidence; splitting JSON text or trimming Evidence fields does not produce valid workflow input.";
+/**
+ * Strict JSON input keeps rejecting a leading byte-order mark, but names it:
+ * the parser's own message quotes the invisible U+FEFF as the bad token.
+ */
+export const JSON_BYTE_ORDER_MARK_MESSAGE =
+ "JSON input begins with a UTF-8 byte-order mark (EF BB BF), which strict JSON input does not accept. Save the file as UTF-8 without a byte-order mark.";
+
/** Decode JSON bytes without confusing a runtime string limit with malformed input. */
export const parseUtf8Json = (
bytes: Uint8Array,
@@ -77,5 +84,7 @@ export const parseUtf8Json = (
cause,
};
}
+ if (text.startsWith("\uFEFF"))
+ return { ok: false, error: JSON_BYTE_ORDER_MARK_MESSAGE, cause: undefined };
return safeParseJson(text);
};
diff --git a/src/application/analysisView/AnalysisViewService.test.ts b/src/application/analysisView/AnalysisViewService.test.ts
index 29289efae8..b361e162fa 100644
--- a/src/application/analysisView/AnalysisViewService.test.ts
+++ b/src/application/analysisView/AnalysisViewService.test.ts
@@ -2,6 +2,8 @@ import { expect, it } from "vitest";
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
import { createEvidence, parseEvidence } from "../../domain/evidence.js";
+import { functionDossierSchema } from "../../domain/hopperValues.js";
+import { ghidraFunctionDossier } from "../../domain/ghidraValues.fixture.js";
import {
analysisViewJavaScriptAnalysisWithSource,
analysisViewJavaScriptEvidence,
@@ -9,6 +11,96 @@ import {
} from "../../../tests/fixtures/analysisView.js";
import { inspectAnalysisView } from "./AnalysisViewService.js";
+it.each(["/fixtures/unknown.exe", ""])(
+ "preserves recorded empty native identity fields: path=%j",
+ (path) => {
+ const original = functionDossierSchema.parse(ghidraFunctionDossier());
+ const parent = createEvidence(
+ { path, format: "pe", sha256: "c".repeat(64) },
+ { id: "ghidra", name: "Ghidra", version: "12.1.4" },
+ {
+ operation: "analyze_function",
+ parameters: {},
+ result: {
+ ...original,
+ procedure: {
+ ...original.procedure,
+ address: "",
+ body: { available: false, reason: "Not recorded" },
+ },
+ },
+ },
+ );
+ const result = inspectAnalysisView({
+ source: { kind: "inline", evidence: parent },
+ view: {
+ kind: "native",
+ facet: "procedure",
+ offset: 0,
+ limit: 64,
+ },
+ });
+ if (!result.ok) throw result.error;
+ expect(result.value.normalized_result).toMatchObject({
+ procedure_address: "",
+ artifact: { path },
+ item: { address: "" },
+ });
+ expect(result.value.locations).toEqual(
+ path.length > 0 ? [{ kind: "artifact-path", path }] : [],
+ );
+ },
+);
+
+it("projects a small native view from an authenticated >10 MiB retained dossier", () => {
+ const original = functionDossierSchema.parse(ghidraFunctionDossier());
+ const parent = createEvidence(
+ { path: "/fixtures/large.exe", format: "pe", sha256: "c".repeat(64) },
+ { id: "ghidra", name: "Ghidra", version: "12.1.4" },
+ {
+ operation: "analyze_function",
+ parameters: { address: "0x401000" },
+ result: { ...original, pseudocode: "X".repeat(11 * 1024 * 1024) },
+ limitations: ["Synthetic fixture"],
+ },
+ );
+ const input = {
+ source: {
+ kind: "retained-evidence" as const,
+ evidence_id: parent.evidence_id,
+ },
+ view: {
+ kind: "native" as const,
+ facet: "pseudocode" as const,
+ offset: 0,
+ limit: 128,
+ },
+ };
+ const view = inspectAnalysisView(input, (id) =>
+ id === parent.evidence_id ? parent : undefined,
+ );
+ if (!view.ok) throw view.error;
+ expect(view.value.evidence_links).toEqual([parent.evidence_id]);
+ expect(view.value.normalized_result).toMatchObject({
+ kind: "native",
+ parent_operation: "analyze_function",
+ parent_evidence_id: parent.evidence_id,
+ procedure_address: "0x401000",
+ artifact: { path: "/fixtures/large.exe", sha256: "c".repeat(64) },
+ coverage: { total: 11 * 1024 * 1024, examined: 128, next_offset: 128 },
+ });
+ expect(view.value.locations).toEqual([
+ { kind: "artifact-path", path: "/fixtures/large.exe" },
+ { kind: "address", address: "0x401000" },
+ ]);
+ expect(JSON.stringify(view.value).length).toBeLessThan(12000);
+ expect(JSON.stringify(parent).length).toBeGreaterThan(10 * 1024 * 1024);
+ expect(inspectAnalysisView(input, () => undefined)).toMatchObject({
+ ok: false,
+ error: { _tag: "EvidenceIntegrityError" },
+ });
+});
+
it("projects inline layout Evidence and retains a derived view record", () => {
const parent = analysisViewLayoutEvidence();
const result = inspectAnalysisView({
diff --git a/src/application/analysisView/AnalysisViewService.ts b/src/application/analysisView/AnalysisViewService.ts
index 0d10d5fa8d..7af6fb24a7 100644
--- a/src/application/analysisView/AnalysisViewService.ts
+++ b/src/application/analysisView/AnalysisViewService.ts
@@ -53,6 +53,13 @@ export const inspectAnalysisViewValidated = (
operation: parent.operation,
normalizedResult: parent.normalized_result,
limitations: parent.limitations,
+ artifact:
+ parent.subject === null
+ ? null
+ : {
+ path: parent.subject.local_path,
+ sha256: parent.subject.digest.sha256,
+ },
},
input.view,
);
@@ -126,7 +133,16 @@ const createAnalysisViewEvidence = (
confidence: "derived",
authority: parent.authority,
limitations: result.limitations,
- locations: [{ kind: "artifact-path", path: result.artifact.path }],
+ locations: [
+ ...(result.artifact.path.length > 0
+ ? [{ kind: "artifact-path" as const, path: result.artifact.path }]
+ : []),
+ ...(result.kind === "native" &&
+ result.procedure_address !== null &&
+ result.procedure_address.length > 0
+ ? [{ kind: "address" as const, address: result.procedure_address }]
+ : []),
+ ],
evidenceLinks: [parent.evidence_id],
});
};
diff --git a/src/application/binaryDiagnostics/BinaryLayoutService.test.ts b/src/application/binaryDiagnostics/BinaryLayoutService.test.ts
index e8f3e2c6dd..e65da65f67 100644
--- a/src/application/binaryDiagnostics/BinaryLayoutService.test.ts
+++ b/src/application/binaryDiagnostics/BinaryLayoutService.test.ts
@@ -5,6 +5,7 @@ import {
BINARY_LAYOUT_TEST_PROVIDER,
} from "../../../tests/fixtures/binaryDiagnostics/layout.js";
import { AnalysisCapabilityUnavailableError } from "../../domain/analysisErrorCore.js";
+import { parseEvidence } from "../../domain/evidence.js";
import { err, ok } from "../../domain/result.js";
it.each([
@@ -28,6 +29,19 @@ it.each([
},
);
+it("records the decoded layout once in its Evidence", async () => {
+ const layout = binaryLayoutFixture();
+ const service = new BinaryLayoutService({
+ identity: BINARY_LAYOUT_TEST_PROVIDER,
+ inspect: () => Promise.resolve(ok(layout)),
+ });
+ const result = await service.inspect({ path: layout.artifact.path });
+ if (!result.ok) throw result.error;
+ const evidence = parseEvidence(result.value);
+ expect(evidence.normalized_result).toEqual(layout);
+ expect(evidence.raw_result).toBeNull();
+});
+
it("preserves a provider's actionable unsupported reason", async () => {
const failure = new AnalysisCapabilityUnavailableError(
"fixture-layout",
diff --git a/src/application/binaryDiagnostics/BinaryLayoutService.ts b/src/application/binaryDiagnostics/BinaryLayoutService.ts
index 11bfeb76e9..bdb07da56a 100644
--- a/src/application/binaryDiagnostics/BinaryLayoutService.ts
+++ b/src/application/binaryDiagnostics/BinaryLayoutService.ts
@@ -78,7 +78,8 @@ export class BinaryLayoutService {
operation: OPERATION,
parameters: input.data,
result: value,
- rawResult: value,
+ // The decoded report is the only representation; do not repeat it.
+ rawResult: null,
confidence: "observed",
limitations: value.limitations,
locations: [{ kind: "artifact-path", path: value.artifact.path }],
diff --git a/src/application/binaryDiagnostics/RecordedCrashService.test.ts b/src/application/binaryDiagnostics/RecordedCrashService.test.ts
index 6060aef332..0ab49ee1ba 100644
--- a/src/application/binaryDiagnostics/RecordedCrashService.test.ts
+++ b/src/application/binaryDiagnostics/RecordedCrashService.test.ts
@@ -24,6 +24,7 @@ it("labels an envelope containing debugger-derived maps as derived", async () =>
const evidence = parseEvidence(result.value);
expect(evidence.confidence).toBe("derived");
expect(evidence.normalized_result).toEqual(fixture);
+ expect(evidence.raw_result).toBeNull();
});
it.each([
diff --git a/src/application/binaryDiagnostics/RecordedCrashService.ts b/src/application/binaryDiagnostics/RecordedCrashService.ts
index cfc9278dc1..5a4fff645d 100644
--- a/src/application/binaryDiagnostics/RecordedCrashService.ts
+++ b/src/application/binaryDiagnostics/RecordedCrashService.ts
@@ -83,7 +83,8 @@ export class RecordedCrashService {
operation: OPERATION,
parameters: input.data,
result: value,
- rawResult: value,
+ // The decoded report is the only representation; do not repeat it.
+ rawResult: null,
confidence:
value.debugger.status === "available" ? "derived" : "observed",
limitations: value.limitations,
diff --git a/src/application/javascript/JavaScriptArtifactGraphFindings.ts b/src/application/javascript/JavaScriptArtifactGraphFindings.ts
index cff140220a..41f480daf3 100644
--- a/src/application/javascript/JavaScriptArtifactGraphFindings.ts
+++ b/src/application/javascript/JavaScriptArtifactGraphFindings.ts
@@ -1,5 +1,6 @@
import { createHash } from "node:crypto";
+import { compactStringIdentityKey } from "../../domain/compactStringIdentity.js";
import type { ApplicationNode } from "../../domain/javascript/javascriptApplicationGraphSchemas.js";
import type { JavaScriptStaticAnalysis } from "../../domain/javascript/javascriptStaticAnalysisTypes.js";
import type { JavaScriptArtifactFile } from "../../domain/javascript/javascriptArtifactFiles.js";
@@ -162,16 +163,15 @@ const addEndpoint = (
const source =
sourceNodeFor(context, input.file.path, input.value.module_key) ??
input.asset;
+ const key = compactStringIdentityKey(input.value.value);
+ const label = `${input.value.kind} endpoint`;
const node = context.accumulator.addNode({
kind: "endpoint",
- identity: artifactLocalIdentity(
- input.file.sha256,
- input.value.kind,
- input.value.value,
- ),
+ identity: artifactLocalIdentity(input.file.sha256, input.value.kind, key),
observations: [
{
- label: input.value.value,
+ label:
+ input.value.value.length <= label.length ? input.value.value : label,
properties: {
endpoint_kind: input.value.kind,
value: input.value.value,
diff --git a/src/application/javascript/JavaScriptSemanticGraphAsyncProjection.ts b/src/application/javascript/JavaScriptSemanticGraphAsyncProjection.ts
index a8814281f3..f54c5c4f31 100644
--- a/src/application/javascript/JavaScriptSemanticGraphAsyncProjection.ts
+++ b/src/application/javascript/JavaScriptSemanticGraphAsyncProjection.ts
@@ -1,3 +1,4 @@
+import { compactStringIdentityKey } from "../../domain/compactStringIdentity.js";
import { createJavaScriptSemanticGraphUnknown } from "../../domain/javascript/javascriptSemanticGraph.js";
import type { JavaScriptSemanticGraphNode } from "../../domain/javascript/javascriptSemanticGraphSchemas.js";
import type {
@@ -21,7 +22,7 @@ export const projectSemanticEvents = (
const eventKey =
operation.eventName === null
? operation.eventId
- : `${operation.emitterKey}\u0000${operation.eventName}`;
+ : `${operation.emitterKey}\u0000${compactStringIdentityKey(operation.eventName)}`;
const event =
eventNodes.get(eventKey) ?? addEventNode(context, operation, eventKey);
if (event === null) continue;
@@ -59,7 +60,11 @@ const addEventNode = (
kind: "event",
roleKey: `event:${eventKey}`,
location: operation.eventName === null ? operation.location : null,
- label: operation.eventName,
+ label:
+ operation.eventName === null ||
+ operation.eventName.length <= "event".length
+ ? operation.eventName
+ : "event",
functionNodeId: null,
properties: {
emitter_key: operation.emitterKey,
@@ -78,7 +83,9 @@ const addListenerNode = (
label:
operation.eventName === null
? operation.method
- : `${operation.method}:${operation.eventName}`,
+ : operation.eventName.length <= "listener".length
+ ? `${operation.method}:${operation.eventName}`
+ : `${operation.method}:listener`,
functionNodeId:
operation.ownerCallableId === null
? null
diff --git a/src/application/javascript/JavaScriptSemanticGraphBuilder.ts b/src/application/javascript/JavaScriptSemanticGraphBuilder.ts
index a951a7a55e..45ded50518 100644
--- a/src/application/javascript/JavaScriptSemanticGraphBuilder.ts
+++ b/src/application/javascript/JavaScriptSemanticGraphBuilder.ts
@@ -175,7 +175,21 @@ export const createJavaScriptSemanticGraphProjection =
): Value => {
bindSemanticGraphApplicationNodes(state, applicationGraph);
if (state.roots.size === 0) addFallbackRoot(rootArtifactSha256, state);
+ const relations = [...state.relations.values()];
const unknowns = [...state.unknowns.values()];
+ type Family = (typeof JAVASCRIPT_SEMANTIC_RELATION_FAMILIES)[number];
+ const retainedByFamily = new Map();
+ const unknownIdsByFamily = new Map();
+ for (const relation of relations) {
+ const family = JAVASCRIPT_SEMANTIC_RELATION_FAMILY[relation.relation];
+ retainedByFamily.set(family, (retainedByFamily.get(family) ?? 0) + 1);
+ }
+ for (const unknown of unknowns) {
+ const identifiers = unknownIdsByFamily.get(unknown.family);
+ if (identifiers === undefined)
+ unknownIdsByFamily.set(unknown.family, [unknown.unknown_id]);
+ else identifiers.push(unknown.unknown_id);
+ }
const graph = factory({
schema: "JavaScriptSemanticRelationGraph",
root_artifact_sha256: rootArtifactSha256,
@@ -183,7 +197,7 @@ export const createJavaScriptSemanticGraphProjection =
root_node_ids: [...state.roots],
evidence_contexts: state.evidenceContexts.contexts,
nodes: [...state.nodes.values()],
- relations: [...state.relations.values()],
+ relations,
fingerprints,
unknowns,
coverage: {
@@ -204,15 +218,9 @@ export const createJavaScriptSemanticGraphProjection =
families: JAVASCRIPT_SEMANTIC_RELATION_FAMILIES.map((family) => ({
family,
status: semanticFamilyStatus(family),
- retained_relations: [...state.relations.values()].filter(
- (relation) =>
- JAVASCRIPT_SEMANTIC_RELATION_FAMILY[relation.relation] ===
- family,
- ).length,
+ retained_relations: retainedByFamily.get(family) ?? 0,
omitted_relations: truncatedFiles > 0 ? null : 0,
- unknown_ids: unknowns
- .filter((unknown) => unknown.family === family)
- .map(({ unknown_id: identifier }) => identifier),
+ unknown_ids: unknownIdsByFamily.get(family) ?? [],
})),
},
limitations: [
diff --git a/src/application/javascript/JavaScriptSemanticGraphDataProjection.ts b/src/application/javascript/JavaScriptSemanticGraphDataProjection.ts
index 51efcd2ec6..b1c44e8f8c 100644
--- a/src/application/javascript/JavaScriptSemanticGraphDataProjection.ts
+++ b/src/application/javascript/JavaScriptSemanticGraphDataProjection.ts
@@ -137,7 +137,11 @@ const addRequestNode = (
kind: operation.kind === "request" ? "request" : "response",
roleKey: operation.requestId,
location: operation.location,
- label: operation.endpoint ?? operation.method,
+ label:
+ operation.endpoint !== null &&
+ operation.endpoint.length <= operation.method.length
+ ? operation.endpoint
+ : operation.method,
functionNodeId:
operation.ownerCallableId === null
? null
diff --git a/src/application/javascript/JavaScriptSemanticGraphValueProjection.ts b/src/application/javascript/JavaScriptSemanticGraphValueProjection.ts
index b8002fcd0c..febe23d247 100644
--- a/src/application/javascript/JavaScriptSemanticGraphValueProjection.ts
+++ b/src/application/javascript/JavaScriptSemanticGraphValueProjection.ts
@@ -2,6 +2,7 @@ import {
semanticContainer,
semanticPropertyPointer,
} from "../../domain/javascript/javascriptSemanticSlots.js";
+import { digestCanonicalValue } from "../../domain/canonicalDigest.js";
import type { JavaScriptSemanticBinding } from "../../domain/javascript/javascriptSemanticIr.js";
import type {
JavaScriptSemanticProperty,
@@ -153,16 +154,27 @@ const addLiteralNode = (
binding: JavaScriptSemanticBinding,
value: string | number | boolean | null,
role: string,
-) =>
- retainSemanticGraphNode(context.state, context.file, {
+) => {
+ const encodedValue = JSON.stringify(value);
+ const valueDigest = `value-sha256:${digestCanonicalValue(value, "Semantic literal")}`;
+ const valueKey =
+ encodedValue.length <= valueDigest.length ? encodedValue : valueDigest;
+ const stringLabel = "string literal";
+ return retainSemanticGraphNode(context.state, context.file, {
kind: "literal",
- roleKey: `literal:${binding.bindingId}:${role}:${JSON.stringify(value)}`,
+ // Identity needs an exact value commitment, not another payload copy.
+ // Keep the complete literal in properties for queries and Evidence export.
+ roleKey: `literal:${binding.bindingId}:${role}:${valueKey}`,
location: binding.definitions[0]?.location ?? null,
- label: JSON.stringify(value),
+ label:
+ typeof value === "string" && encodedValue.length > stringLabel.length
+ ? stringLabel
+ : encodedValue,
functionNodeId:
context.bindingNodes.get(binding.bindingId)?.function_node_id ?? null,
properties: { value },
});
+};
/** Create one canonical slot for a root binding and exact static property path. */
export const semanticPropertySlot = (
diff --git a/src/application/process/ProcessCli.test.ts b/src/application/process/ProcessCli.test.ts
index 88ab756ad7..34e777d31d 100644
--- a/src/application/process/ProcessCli.test.ts
+++ b/src/application/process/ProcessCli.test.ts
@@ -6,6 +6,7 @@ import {
captureProcessScenarioFile,
projectProcessCliError,
} from "./ProcessCli.js";
+import { JSON_BYTE_ORDER_MARK_MESSAGE } from "../Utf8JsonInput.js";
const roots: string[] = [];
afterEach(async () => {
@@ -87,4 +88,14 @@ describe("process CLI input diagnostics", () => {
}),
});
});
+
+ it("names a leading byte-order mark instead of quoting it as a token", async () => {
+ const path = join(await fixture(), "scenario.json");
+ await writeFile(path, '\uFEFF{"executable":"/bin/echo"}');
+ expect(await captureProcessScenarioFile(path)).toEqual({
+ error: "Process command failed",
+ category: "invalid_input",
+ message: `Process input file is not valid UTF-8 JSON: ${path} (${JSON_BYTE_ORDER_MARK_MESSAGE}). Repair the file, then try again.`,
+ });
+ });
});
diff --git a/src/application/process/ProcessCli.ts b/src/application/process/ProcessCli.ts
index 0640293783..bbc4667eef 100644
--- a/src/application/process/ProcessCli.ts
+++ b/src/application/process/ProcessCli.ts
@@ -19,6 +19,7 @@ import { createProgressReporter } from "../ProgressReporter.js";
import { captureProcessScenario } from "../../process/capture/ProcessHarness.js";
import { PROCESS_PROVIDER } from "../../domain/process/processEvidenceProvider.js";
import { createProcessCaptureEvidence } from "./ProcessEvidence.js";
+import { JSON_BYTE_ORDER_MARK_MESSAGE } from "../Utf8JsonInput.js";
/** Safe process-command failure returned to the CLI adapter. */
export interface ProcessCliErrorOutput {
@@ -191,6 +192,8 @@ const readJson = async (path: string): Promise => {
fatal: true,
ignoreBOM: true,
}).decode(bytes);
+ if (text.startsWith("\uFEFF"))
+ throw new Error(JSON_BYTE_ORDER_MARK_MESSAGE);
const parsed: unknown = JSON.parse(text);
return parsed;
} catch (cause: unknown) {
diff --git a/src/artifacts/ArtifactPaths.test.ts b/src/artifacts/ArtifactPaths.test.ts
new file mode 100644
index 0000000000..4834cf2d4e
--- /dev/null
+++ b/src/artifacts/ArtifactPaths.test.ts
@@ -0,0 +1,83 @@
+import { describe, expect, it } from "vitest";
+
+import {
+ annotateEnglishUnicodeCaseCollisions,
+ ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION,
+} from "./ArtifactPaths.js";
+
+const NOTE =
+ "a case-insensitive destination cannot store both. This comparison is not a filesystem observation.";
+
+const annotate = (paths: readonly string[]) => {
+ const occurrences = paths.map((logical_path) => ({
+ logical_path,
+ limitations: [] as string[],
+ }));
+ const limitation = annotateEnglishUnicodeCaseCollisions(occurrences);
+ return {
+ limitation,
+ byPath: new Map(
+ occurrences.map(({ logical_path, limitations }) => [
+ logical_path,
+ limitations,
+ ]),
+ ),
+ };
+};
+
+describe("English Unicode case collisions", () => {
+ it("names the other spelling of a colliding file", () => {
+ const { limitation, byPath } = annotate([".", "Main.js", "main.js", "x"]);
+ expect(limitation).toBe(ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION);
+ expect(byPath.get("Main.js")).toEqual([
+ `Logical path Main.js collides under English (en-US) Unicode case folding with main.js; ${NOTE}`,
+ ]);
+ expect(byPath.get("main.js")).toEqual([
+ `Logical path main.js collides under English (en-US) Unicode case folding with Main.js; ${NOTE}`,
+ ]);
+ expect(byPath.get("x")).toEqual([]);
+ });
+
+ it("records a directory collision once at the diverging segment", () => {
+ const { byPath } = annotate([
+ "res/Foo",
+ "res/Foo/a.txt",
+ "res/foo",
+ "res/foo/b.txt",
+ ]);
+ expect(byPath.get("res/Foo")).toEqual([
+ `Logical path res/Foo collides under English (en-US) Unicode case folding with res/foo; ${NOTE}`,
+ ]);
+ expect(byPath.get("res/foo/b.txt")).toEqual([
+ `Logical path res/foo/b.txt is under res/foo, which collides under English (en-US) Unicode case folding with res/Foo; ${NOTE}`,
+ ]);
+ });
+
+ it("counts further spellings, each of which is its own occurrence", () => {
+ const { byPath } = annotate(["A.txt", "a.txt", "a.TXT"]);
+ expect(byPath.get("a.txt")).toEqual([
+ `Logical path a.txt collides under English (en-US) Unicode case folding with A.txt and 1 other spelling; ${NOTE}`,
+ ]);
+ });
+
+ it("stays linear for large case-variant subtrees", () => {
+ const count = 5_000;
+ const paths = Array.from({ length: count }, (_, index) => [
+ `Foo/a${String(index)}.txt`,
+ `foo/b${String(index)}.txt`,
+ ]).flat();
+ const { byPath } = annotate(paths);
+ for (const path of paths) {
+ const limitations = byPath.get(path) ?? [];
+ expect(limitations).toHaveLength(1);
+ expect(limitations[0]?.length).toBeLessThan(300);
+ }
+ });
+
+ it("walks names with more components than the call stack holds", () => {
+ const deep = Array.from({ length: 30_000 }, () => "a").join("/");
+ const { byPath } = annotate([deep, `${deep}/X`, `${deep}/x`]);
+ expect(byPath.get(`${deep}/X`)).toHaveLength(1);
+ expect(byPath.get(deep)).toEqual([]);
+ });
+});
diff --git a/src/artifacts/ArtifactPaths.ts b/src/artifacts/ArtifactPaths.ts
index d5ec35cb4b..cabcb60cf7 100644
--- a/src/artifacts/ArtifactPaths.ts
+++ b/src/artifacts/ArtifactPaths.ts
@@ -1,9 +1,24 @@
import { posix } from "node:path";
+import { compareUnicodeCodePoints } from "../domain/unicodeCodePointOrder.js";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
const DRIVE_OR_UNC = /^(?:[A-Za-z]:|\\|\/\/)/u;
+/**
+ * English Unicode case folding used only to describe logical names a
+ * case-insensitive destination might collapse. It is not a filesystem case table.
+ */
+const ENGLISH_UNICODE_CASE_NOTE =
+ "This comparison is not a filesystem observation.";
+
+/** Inventory limitation when logical names collide under English Unicode case folding. */
+export const ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION = `Logical archive names collide under English (en-US) Unicode case folding; a case-insensitive destination cannot store both spellings. ${ENGLISH_UNICODE_CASE_NOTE}`;
+
+/** User-facing prefix for a destination that cannot keep two logical spellings. */
+export const DESTINATION_CASE_COLLISION_PREFIX =
+ "Destination filesystem cannot store both ";
+
/** Normalize an untrusted logical archive path without touching filesystem. */
export const normalizeArtifactPath = (input: string): string => {
if (
@@ -30,7 +45,7 @@ export const normalizeArtifactPath = (input: string): string => {
return normalized;
};
-/** Reject exact, Unicode-normalized, case-folded, and prefix collisions. */
+/** Reject exact and same-spelling prefix collisions. Case-distinct names stay distinct. */
export class ArtifactPathRegistry {
readonly #root: PathTrieNode = { kind: undefined, children: new Map() };
@@ -43,16 +58,10 @@ export class ArtifactPathRegistry {
"path",
`Artifact prefix conflict: ${path}`,
);
- const foldedPart = part.toLocaleLowerCase("en-US");
- let child = node.children.get(foldedPart);
- if (child !== undefined && child.spelling !== part)
- throw new ArtifactReaderFailure(
- "path",
- `Artifact path collision: ${path} differs only in case from ${[...parts.slice(0, index), child.spelling].join("/")}`,
- );
+ let child = node.children.get(part);
if (child === undefined) {
- child = { spelling: part, kind: undefined, children: new Map() };
- node.children.set(foldedPart, child);
+ child = { kind: undefined, children: new Map() };
+ node.children.set(part, child);
}
node = child;
if (index === parts.length - 1) {
@@ -73,7 +82,169 @@ export class ArtifactPathRegistry {
}
interface PathTrieNode {
- readonly spelling?: string;
kind: "file" | "directory" | "symlink" | "slice" | undefined;
readonly children: Map;
}
+
+interface CaseNode {
+ readonly bySpelling: Map;
+ readonly spellingsByFold: Map;
+ readonly terminals: string[];
+}
+
+const emptyCaseNode = (): CaseNode => ({
+ bySpelling: new Map(),
+ spellingsByFold: new Map(),
+ terminals: [],
+});
+
+const englishCaseFold = (part: string): string =>
+ part.toLocaleLowerCase("en-US");
+
+/**
+ * Record English (en-US) Unicode case collisions on each affected occurrence.
+ * Returns the inventory limitation when any logical names collide.
+ */
+export const annotateEnglishUnicodeCaseCollisions = (
+ occurrences: readonly {
+ readonly logical_path: string;
+ limitations: string[];
+ }[],
+): string | undefined => {
+ const collisions = englishUnicodeCaseCollisions(
+ occurrences.map(({ logical_path }) => logical_path),
+ );
+ if (collisions.size === 0) return undefined;
+ for (const occurrence of occurrences)
+ for (const collision of collisions.get(occurrence.logical_path) ?? [])
+ occurrence.limitations.push(
+ caseCollisionLimitation(occurrence.logical_path, collision),
+ );
+ return ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION;
+};
+
+/** One logical path segment whose spelling folds to another sibling's. */
+interface CaseCollision {
+ /** Logical path through the colliding segment. */
+ readonly prefix: string;
+ /** First other sibling spelling, as a logical path, in code-point order. */
+ readonly other: string;
+ /** Count of further sibling spellings, each its own occurrence. */
+ readonly additional: number;
+}
+
+const caseCollisionLimitation = (
+ path: string,
+ { prefix, other, additional }: CaseCollision,
+): string => {
+ const others =
+ additional === 0
+ ? other
+ : `${other} and ${String(additional)} other spelling${additional === 1 ? "" : "s"}`;
+ const subject =
+ prefix === path
+ ? `Logical path ${path} collides`
+ : `Logical path ${path} is under ${prefix}, which collides`;
+ return `${subject} under English (en-US) Unicode case folding with ${others}; a case-insensitive destination cannot store both. ${ENGLISH_UNICODE_CASE_NOTE}`;
+};
+
+/**
+ * Name the directory-listing spelling that differs from the requested segment
+ * only by English (en-US) Unicode case. Exact matches are the same logical name.
+ */
+export const destinationCaseCollisionMessage = (
+ requestedPath: string,
+ existingNames: readonly string[],
+): string | undefined => {
+ const slash = requestedPath.lastIndexOf("/");
+ const segment = slash < 0 ? requestedPath : requestedPath.slice(slash + 1);
+ if (existingNames.includes(segment)) return undefined;
+ const folded = englishCaseFold(segment);
+ const existing = existingNames
+ .filter((name) => englishCaseFold(name) === folded)
+ .sort(compareUnicodeCodePoints)[0];
+ if (existing === undefined) return undefined;
+ const parent = slash < 0 ? "" : requestedPath.slice(0, slash + 1);
+ return `${DESTINATION_CASE_COLLISION_PREFIX}${requestedPath} and ${parent}${existing}; inventory retains both logical names.`;
+};
+
+const englishUnicodeCaseCollisions = (
+ paths: readonly string[],
+): ReadonlyMap => {
+ const root = emptyCaseNode();
+ for (const path of paths) {
+ if (path.length === 0 || path === ".") continue;
+ let node = root;
+ for (const part of path.split("/")) {
+ const folded = englishCaseFold(part);
+ const spellings = node.spellingsByFold.get(folded);
+ if (spellings === undefined) node.spellingsByFold.set(folded, [part]);
+ else if (!spellings.includes(part)) spellings.push(part);
+ let child = node.bySpelling.get(part);
+ if (child === undefined) {
+ child = emptyCaseNode();
+ node.bySpelling.set(part, child);
+ }
+ node = child;
+ }
+ node.terminals.push(path);
+ }
+ // A collision is recorded once, at the segment where spellings diverge, and
+ // shared down the subtree. Pairing every descendant of one spelling with
+ // every descendant of another grew quadratically with the subtree sizes.
+ // The walk is iterative because a ZIP name can hold more components than
+ // the call stack allows.
+ const collisions = new Map();
+ const pending: {
+ readonly node: CaseNode;
+ readonly prefix: string | undefined;
+ readonly chain: CollisionChain | undefined;
+ }[] = [{ node: root, prefix: undefined, chain: undefined }];
+ for (let item = pending.pop(); item !== undefined; item = pending.pop()) {
+ const { node, prefix, chain } = item;
+ if (chain !== undefined)
+ for (const path of node.terminals)
+ collisions.set(path, collisionsAlong(chain));
+ for (const spellings of node.spellingsByFold.values())
+ spellings.sort(compareUnicodeCodePoints);
+ for (const [spelling, child] of node.bySpelling) {
+ const childPrefix =
+ prefix === undefined ? spelling : `${prefix}/${spelling}`;
+ const spellings =
+ node.spellingsByFold.get(englishCaseFold(spelling)) ?? [];
+ const other = spellings[0] === spelling ? spellings[1] : spellings[0];
+ pending.push({
+ node: child,
+ prefix: childPrefix,
+ chain:
+ other === undefined
+ ? chain
+ : {
+ collision: {
+ prefix: childPrefix,
+ other: prefix === undefined ? other : `${prefix}/${other}`,
+ additional: spellings.length - 2,
+ },
+ parent: chain,
+ },
+ });
+ }
+ }
+ return collisions;
+};
+
+/** Collisions on one path, linked from the deepest segment to the root. */
+interface CollisionChain {
+ readonly collision: CaseCollision;
+ readonly parent: CollisionChain | undefined;
+}
+
+const collisionsAlong = (chain: CollisionChain): CaseCollision[] => {
+ const collisions: CaseCollision[] = [];
+ let link: CollisionChain | undefined = chain;
+ while (link !== undefined) {
+ collisions.push(link.collision);
+ link = link.parent;
+ }
+ return collisions.reverse();
+};
diff --git a/src/artifacts/ArtifactProvider.ts b/src/artifacts/ArtifactProvider.ts
index fe5d24ef0f..821455903b 100644
--- a/src/artifacts/ArtifactProvider.ts
+++ b/src/artifacts/ArtifactProvider.ts
@@ -194,10 +194,8 @@ class ArtifactClient implements AnalysisClient {
this.target.sourcePath ?? this.target.path,
result.manifest,
),
- locations: result.artifacts.map(({ relative_path: path }) => ({
- kind: "artifact-path" as const,
- path,
- })),
+ // Each extracted file's path is already in artifacts.
+ locations: [],
}),
);
}
@@ -211,10 +209,8 @@ class ArtifactClient implements AnalysisClient {
this.target.sourcePath ?? this.target.path,
result.manifest,
),
- locations: result.occurrences.map(({ logical_path: path }) => ({
- kind: "artifact-path" as const,
- path,
- })),
+ // Each occurrence's logical path is already in the result.
+ locations: [],
}),
);
} catch (cause: unknown) {
@@ -323,17 +319,14 @@ class ArtifactClient implements AnalysisClient {
this.target.sourcePath ?? this.target.path,
inventory.manifest,
);
- const locations = inventory.occurrences.map(({ logical_path: path }) => ({
- kind: "artifact-path" as const,
- path,
- }));
const inventoryEvidence = createEvidence(subject, ARTIFACT_GRAPH_PROVIDER, {
operation: "inventory_artifact",
parameters: inventoryParameters,
result: inventory,
rawResult: null,
limitations: inventory.limitations,
- locations,
+ // The nested inventory already names every occurrence's logical path.
+ locations: [],
});
const result = createArtifactInspection(inventoryEvidence);
await options?.progress?.report({
@@ -347,7 +340,7 @@ class ArtifactClient implements AnalysisClient {
rawResult: null,
limitations: result.limitations,
subject,
- locations,
+ locations: [],
}),
);
}
diff --git a/src/artifacts/SafeOutputTree.ts b/src/artifacts/SafeOutputTree.ts
index fafb6fee4a..05ce0ac680 100644
--- a/src/artifacts/SafeOutputTree.ts
+++ b/src/artifacts/SafeOutputTree.ts
@@ -4,6 +4,7 @@ import {
lstat,
mkdir,
open,
+ readdir,
realpath,
rm,
type FileHandle,
@@ -14,6 +15,7 @@ import { streamChunkToBuffer } from "./StreamBytes.js";
import {
ArtifactPathRegistry,
+ destinationCaseCollisionMessage,
normalizeArtifactPath,
} from "./ArtifactPaths.js";
import { ArtifactReaderFailure } from "./ArtifactReader.js";
@@ -134,7 +136,8 @@ export class SafeOutputTree {
constants.O_WRONLY |
constants.O_NOFOLLOW,
0o600,
- ).catch((cause: unknown) => {
+ ).catch(async (cause: unknown) => {
+ await throwIfDestinationCaseCollision(dirname(destination), path, cause);
throw new ArtifactReaderFailure(
"path",
`Could not exclusively create extraction path: ${path}`,
@@ -245,17 +248,28 @@ export class SafeOutputTree {
if (fileName === undefined)
throw new ArtifactReaderFailure("path", "Invalid extraction path");
let current = this.#outputRoot;
+ let logicalParent = "";
for (const part of parts) {
+ const parentDirectory = current;
+ const logicalPath =
+ logicalParent.length === 0 ? part : `${logicalParent}/${part}`;
current = join(current, part);
+ let created = true;
await mkdir(current, { mode: 0o700 }).catch((cause: unknown) => {
if (!isAlreadyExists(cause)) throw cause;
+ created = false;
});
+ if (!created)
+ await throwIfDestinationCaseCollision(parentDirectory, logicalPath);
const metadata = await lstat(current);
- if (!metadata.isDirectory() || metadata.isSymbolicLink())
+ if (!metadata.isDirectory() || metadata.isSymbolicLink()) {
+ await throwIfDestinationCaseCollision(parentDirectory, logicalPath);
throw new ArtifactReaderFailure(
"path",
`Unsafe extraction parent: ${relativePath}`,
);
+ }
+ logicalParent = logicalPath;
}
return join(current, fileName);
}
@@ -330,6 +344,23 @@ const isNotFound = (cause: unknown): boolean =>
const isAlreadyExists = (cause: unknown): boolean =>
cause instanceof Error && "code" in cause && cause.code === "EEXIST";
+/** Fail when this directory already holds another spelling of the requested segment. */
+const throwIfDestinationCaseCollision = async (
+ parentDirectory: string,
+ logicalPath: string,
+ cause?: unknown,
+): Promise => {
+ const names = await readdir(parentDirectory).catch(() => undefined);
+ if (names === undefined) return;
+ const message = destinationCaseCollisionMessage(logicalPath, names);
+ if (message === undefined) return;
+ throw new ArtifactReaderFailure(
+ "path",
+ message,
+ cause === undefined ? undefined : { cause },
+ );
+};
+
const abortIfNeeded = (signal?: AbortSignal): void => {
if (signal?.aborted === true)
throw new ArtifactReaderFailure(
diff --git a/src/artifacts/apple/InterfaceBuilderDecodeBudget.ts b/src/artifacts/apple/InterfaceBuilderDecodeBudget.ts
index d8b6f3a932..a43529634f 100644
--- a/src/artifacts/apple/InterfaceBuilderDecodeBudget.ts
+++ b/src/artifacts/apple/InterfaceBuilderDecodeBudget.ts
@@ -147,10 +147,45 @@ const estimateXmlPlistExpansion = (
return decodedSourceBytes * 2 + elementCount * 128;
};
-/** Count every expanded binary-plist reference before calling plist.parseBinary. */
-const estimateBinaryPlistExpansion = (
+/** Resource a binary plist preflight found over its budget, or a malformed reference cycle. */
+export type BinaryPlistExpansionFailure = "cycle" | "depth" | "expansion";
+
+const interfaceBuilderBudgetFailure = (
+ _kind: BinaryPlistExpansionFailure,
+ message: string,
+): Error =>
+ new InterfaceBuilderDecodeBudgetExceeded(
+ "aggregate_decode_budget_exhausted",
+ message,
+ );
+
+/** How a caller names and reports binary plist expansion failures. */
+export interface BinaryPlistExpansionOptions {
+ /** Budget named in failure messages. */
+ readonly budget?: string;
+ /** Error for budget, depth, and reference-cycle failures. */
+ readonly fail?: (kind: BinaryPlistExpansionFailure, message: string) => Error;
+ /**
+ * Structural bounds: "strict" keeps objects and the offset table before the
+ * trailer; "decoder" admits exactly what plist.parseBinary can read, so
+ * expansion is bounded without rejecting archives that decoder accepts.
+ */
+ readonly bounds?: "strict" | "decoder";
+}
+
+/**
+ * Count every expanded binary-plist reference before calling plist.parseBinary,
+ * which recurses on each reference and copies shared containers. Malformed
+ * structure throws TypeError; budget, depth, and cycle failures use `fail`.
+ */
+export const estimateBinaryPlistExpansion = (
bytes: Buffer,
maxBytes: number,
+ {
+ budget = "the aggregate Interface Builder decode budget",
+ fail = interfaceBuilderBudgetFailure,
+ bounds = "strict",
+ }: BinaryPlistExpansionOptions = {},
): number => {
if (bytes.length < 40)
throw new TypeError("binary plist trailer is truncated");
@@ -172,20 +207,19 @@ const estimateBinaryPlistExpansion = (
const objectCount = readInteger(trailer + 8, 8);
const topObject = readInteger(trailer + 16, 8);
const offsetTable = readInteger(trailer + 24, 8);
+ const tableEnd = bounds === "strict" ? trailer : bytes.length;
+ const objectsEnd = bounds === "strict" ? offsetTable : bytes.length;
if (
objectCount < 1 ||
topObject < 0 ||
topObject >= objectCount ||
offsetTable < 8 ||
- offsetTable + objectCount * offsetSize > trailer
+ offsetTable + objectCount * offsetSize > tableEnd
)
throw new TypeError("binary plist trailer or object table is invalid");
let estimate = bytes.length * 2 + objectCount * 16;
if (estimate > maxBytes)
- throw new InterfaceBuilderDecodeBudgetExceeded(
- "aggregate_decode_budget_exhausted",
- "binary plist object table exceeds the aggregate Interface Builder decode budget",
- );
+ throw fail("expansion", `binary plist object table exceeds ${budget}`);
const active = new Set();
const pending: Array<{ object: number; depth: number; exit: boolean }> = [
@@ -201,26 +235,20 @@ const estimateBinaryPlistExpansion = (
}
pendingObjectCount -= 1;
if (entry.depth > 128)
- throw new InterfaceBuilderDecodeBudgetExceeded(
- "aggregate_decode_budget_exhausted",
- "binary plist nesting exceeds the aggregate Interface Builder decode budget",
- );
+ throw fail("depth", `binary plist nesting exceeds ${budget}`);
if (active.has(entry.object))
- throw new InterfaceBuilderDecodeBudgetExceeded(
- "aggregate_decode_budget_exhausted",
- "binary plist reference cycle exceeds the aggregate Interface Builder decode budget",
- );
+ throw fail("cycle", `binary plist reference cycle exceeds ${budget}`);
estimate += 96;
if (estimate > maxBytes)
- throw new InterfaceBuilderDecodeBudgetExceeded(
- "aggregate_decode_budget_exhausted",
- "binary plist reference expansion exceeds the aggregate Interface Builder decode budget",
+ throw fail(
+ "expansion",
+ `binary plist reference expansion exceeds ${budget}`,
);
const offset = readInteger(
offsetTable + entry.object * offsetSize,
offsetSize,
);
- if (offset < 8 || offset >= offsetTable)
+ if (offset < 8 || offset >= objectsEnd)
throw new TypeError("binary plist object offset is invalid");
const marker = bytes[offset] ?? 0;
const type = marker >> 4;
@@ -234,31 +262,31 @@ const estimateBinaryPlistExpansion = (
const integerSize = 1 << (extMarker & 0x0f);
size = readInteger(cursor, integerSize);
cursor += integerSize;
- if (size < 0 || cursor > offsetTable)
+ if (size < 0 || cursor > objectsEnd)
throw new TypeError("binary plist extended object size is invalid");
}
if (type === 4 || type === 5 || type === 6) {
const byteLength = type === 6 ? size * 2 : size;
- if (cursor + byteLength > offsetTable)
+ if (cursor + byteLength > objectsEnd)
throw new TypeError("binary plist scalar object is truncated");
estimate += type === 4 ? size + 4 * Math.ceil(size / 3) : byteLength * 2;
if (estimate > maxBytes)
- throw new InterfaceBuilderDecodeBudgetExceeded(
- "aggregate_decode_budget_exhausted",
- "binary plist scalar expansion exceeds the aggregate Interface Builder decode budget",
+ throw fail(
+ "expansion",
+ `binary plist scalar expansion exceeds ${budget}`,
);
continue;
}
if (type !== 10 && type !== 13) continue;
const childCount = type === 13 ? size * 2 : size;
const refsEnd = cursor + childCount * referenceSize;
- if (!Number.isSafeInteger(refsEnd) || refsEnd > offsetTable)
+ if (!Number.isSafeInteger(refsEnd) || refsEnd > objectsEnd)
throw new TypeError("binary plist reference table is truncated");
const availableNodes = Math.floor((maxBytes - estimate) / 96);
if (childCount > availableNodes - pendingObjectCount)
- throw new InterfaceBuilderDecodeBudgetExceeded(
- "aggregate_decode_budget_exhausted",
- "binary plist reference expansion exceeds the aggregate Interface Builder decode budget",
+ throw fail(
+ "expansion",
+ `binary plist reference expansion exceeds ${budget}`,
);
active.add(entry.object);
pending.push({ object: entry.object, depth: entry.depth, exit: true });
diff --git a/src/artifacts/apple/KeyedArchiveReader.test.ts b/src/artifacts/apple/KeyedArchiveReader.test.ts
index dd569a6e51..c6a1319f17 100644
--- a/src/artifacts/apple/KeyedArchiveReader.test.ts
+++ b/src/artifacts/apple/KeyedArchiveReader.test.ts
@@ -103,3 +103,88 @@ describe("inert keyed archive decoding", () => {
);
});
});
+
+/** Encode a bplist00 whose objects reference each other by index. */
+const binaryPlist = (objects: readonly Buffer[]): Buffer => {
+ const header = Buffer.from("bplist00");
+ const offsets: number[] = [];
+ let position = header.length;
+ for (const object of objects) {
+ offsets.push(position);
+ position += object.length;
+ }
+ const table = Buffer.alloc(objects.length * 4);
+ offsets.forEach((offset, index) => table.writeUInt32BE(offset, index * 4));
+ const trailer = Buffer.alloc(32);
+ trailer[6] = 4;
+ trailer[7] = 2;
+ trailer.writeBigUInt64BE(BigInt(objects.length), 8);
+ trailer.writeBigUInt64BE(0n, 16);
+ trailer.writeBigUInt64BE(BigInt(position), 24);
+ return Buffer.concat([header, ...objects, table, trailer]);
+};
+const plistArray = (references: readonly number[]): Buffer => {
+ const bytes = Buffer.alloc(1 + references.length * 2);
+ bytes[0] = 0xa0 | references.length;
+ references.forEach((reference, index) =>
+ bytes.writeUInt16BE(reference, 1 + index * 2),
+ );
+ return bytes;
+};
+const plistString = (value: string): Buffer =>
+ Buffer.concat([Buffer.from([0x50 | value.length]), Buffer.from(value)]);
+// {"$archiver": "NSKeyedArchiver", "$top": {"root": UID 1}, "$objects": object 8}
+const keyedRoot = (payload: readonly Buffer[]): Buffer =>
+ binaryPlist([
+ Buffer.from([0xd3, 0, 1, 0, 3, 0, 4, 0, 2, 0, 8, 0, 5]),
+ plistString("$archiver"),
+ Buffer.concat([
+ Buffer.from([0x5f, 0x10, 15]),
+ Buffer.from("NSKeyedArchiver"),
+ ]),
+ plistString("$objects"),
+ plistString("$top"),
+ Buffer.from([0xd1, 0, 6, 0, 7]),
+ plistString("root"),
+ Buffer.from([0x80, 1]),
+ ...payload,
+ ]);
+
+describe("binary plist reference preflight", () => {
+ const selection = { offset: 0, limit: 10 };
+
+ it("reports a reference cycle as malformed, not as an oversized archive", () => {
+ expect(() =>
+ decodeKeyedArchiveBytes(keyedRoot([plistArray([8])]), selection),
+ ).toThrow(
+ new TypeError(
+ "binary plist object references form a cycle, which a property list cannot represent",
+ ),
+ );
+ });
+
+ it("rejects exponential shared-container expansion before decoding", () => {
+ // Forty levels of [next, next] expand to 2^40 leaves in a few hundred bytes.
+ const levels = Array.from({ length: 40 }, (_, index) =>
+ plistArray([9 + index, 9 + index]),
+ );
+ expect(() =>
+ decodeKeyedArchiveBytes(
+ keyedRoot([...levels, plistString("x")]),
+ selection,
+ ),
+ ).toThrow(RangeError);
+ });
+
+ it("decodes shared leaf references", () => {
+ const decoded = decodeKeyedArchiveBytes(
+ keyedRoot([plistArray([9, 9, 9]), plistString("$null")]),
+ selection,
+ );
+ expect(decoded.objects.map(({ id, value }) => [id, value])).toEqual([
+ [0, "$null"],
+ [1, "$null"],
+ [2, "$null"],
+ ]);
+ });
+});
diff --git a/src/artifacts/apple/KeyedArchiveReader.ts b/src/artifacts/apple/KeyedArchiveReader.ts
index 729215929c..fefc6d97a7 100644
--- a/src/artifacts/apple/KeyedArchiveReader.ts
+++ b/src/artifacts/apple/KeyedArchiveReader.ts
@@ -28,8 +28,11 @@ import { ArtifactReaderFailure } from "../ArtifactReader.js";
import type { ArtifactEntry } from "../ArtifactReader.js";
import { decodeXmlPlistText } from "../../domain/propertyListXmlText.js";
+import { estimateBinaryPlistExpansion } from "./InterfaceBuilderDecodeBudget.js";
const MAX_BYTES = 64 * 1024 * 1024;
+/** Decoded representation budget, the same 8:1 ratio Interface Builder uses. */
+const MAX_DECODE_BYTES = 8 * MAX_BYTES;
/** Parse inert plist bytes, preserving data and dates with explicit typed values. */
export const decodeKeyedArchiveBytes = (
@@ -45,6 +48,20 @@ export const decodeKeyedArchiveBytes = (
);
const binary = bytes.subarray(0, 8).toString("ascii") === "bplist00";
const xmlText = binary ? undefined : decodeXmlPlistText(bytes);
+ // plist.parseBinary recurses on every object reference and copies shared
+ // containers, so a few hundred bytes can exhaust memory or the stack.
+ // RangeError reports a resource limit; a reference cycle is malformed.
+ if (binary)
+ estimateBinaryPlistExpansion(bytes, MAX_DECODE_BYTES, {
+ budget: "the keyed archive decode budget",
+ fail: (kind, message) =>
+ kind === "cycle"
+ ? new TypeError(
+ "binary plist object references form a cycle, which a property list cannot represent",
+ )
+ : new RangeError(message),
+ bounds: "decoder",
+ });
const parsed = binary
? { value: parseBinary(bytes), omittedPrototypeKeys: 0 }
: parseXmlPropertyList(xmlText ?? "");
diff --git a/src/artifacts/inventory/ArtifactGraphConstruction.ts b/src/artifacts/inventory/ArtifactGraphConstruction.ts
index 55d391017d..2b85cd3a3c 100644
--- a/src/artifacts/inventory/ArtifactGraphConstruction.ts
+++ b/src/artifacts/inventory/ArtifactGraphConstruction.ts
@@ -249,16 +249,14 @@ export const createArtifactEdges = (
rootArtifactId;
const semantic = {
parent_artifact_id: parentArtifactId,
- child_artifact_id: occurrence.artifact_id,
relation:
occurrence.entry_kind === "slice"
? ("slice-of" as const)
: relationFor(occurrence.logical_path),
occurrence_id: occurrence.occurrence_id,
- logical_path: occurrence.logical_path,
};
edges.push({
- edge_id: artifactEdgeId(semantic),
+ edge_id: artifactEdgeId(semantic, occurrence),
...semantic,
producer: occurrence.entry_kind === "slice" ? (producer ?? null) : null,
ordinal: edges.length,
diff --git a/src/artifacts/inventory/scanCanonical.ts b/src/artifacts/inventory/scanCanonical.ts
index abd5246934..0719e58f4c 100644
--- a/src/artifacts/inventory/scanCanonical.ts
+++ b/src/artifacts/inventory/scanCanonical.ts
@@ -73,12 +73,12 @@ export const scanCanonicalArtifactInventory = async (
try {
if (reader instanceof AsarArtifactReader)
await reader.prepareContainer(rootDigest?.sha256, options.signal);
- const { nodes, occurrences, pendingContradictions } = await scanReader(
- reader,
- ownedReaders,
- options.signal,
- integrity,
- );
+ const {
+ nodes,
+ occurrences,
+ pendingContradictions,
+ caseCollisionLimitation,
+ } = await scanReader(reader, ownedReaders, options.signal, integrity);
outcome = {
kind: "completed",
snapshot: await buildInventorySnapshot({
@@ -91,6 +91,7 @@ export const scanCanonicalArtifactInventory = async (
nodes,
occurrences,
pendingContradictions,
+ caseCollisionLimitation,
}),
};
} catch (cause: unknown) {
@@ -130,6 +131,7 @@ interface SnapshotBuildInput {
readonly nodes: Map;
readonly occurrences: MutableOccurrence[];
readonly pendingContradictions: PendingIntegrityContradiction[];
+ readonly caseCollisionLimitation: string | undefined;
}
const buildInventorySnapshot = async (
@@ -202,7 +204,12 @@ const buildInventorySnapshot = async (
edges: orderedEdges,
provenance: reader?.provenance() ?? [],
integrity_contradictions: integrityContradictions,
- limitations: buildLimitations(rootFormat, reader, integrityContradictions),
+ limitations: buildLimitations(
+ rootFormat,
+ reader,
+ integrityContradictions,
+ input.caseCollisionLimitation,
+ ),
};
};
@@ -310,8 +317,10 @@ const buildLimitations = (
rootFormat: ArtifactOccurrence["artifact_format"],
reader: ArtifactReader | undefined,
integrityContradictions: readonly IntegrityContradiction[],
+ caseCollisionLimitation: string | undefined,
): string[] => [
...inventoryLimitations(rootFormat, reader),
+ ...(caseCollisionLimitation === undefined ? [] : [caseCollisionLimitation]),
...(integrityContradictions.length === 0
? []
: [
diff --git a/src/artifacts/inventory/scanReader.ts b/src/artifacts/inventory/scanReader.ts
index 1ee691a74f..89b1b628e8 100644
--- a/src/artifacts/inventory/scanReader.ts
+++ b/src/artifacts/inventory/scanReader.ts
@@ -1,6 +1,7 @@
import { isAbsolute } from "node:path";
import {
+ annotateEnglishUnicodeCaseCollisions,
ArtifactPathRegistry,
normalizeArtifactPath,
} from "../ArtifactPaths.js";
@@ -64,10 +65,16 @@ export const scanReader = async (
readonly nodes: Map;
readonly occurrences: MutableOccurrence[];
readonly pendingContradictions: PendingIntegrityContradiction[];
+ readonly caseCollisionLimitation: string | undefined;
}> => {
const { nodes, occurrences, pendingContradictions } = emptyScan();
if (reader === undefined)
- return { nodes, occurrences, pendingContradictions };
+ return {
+ nodes,
+ occurrences,
+ pendingContradictions,
+ caseCollisionLimitation: undefined,
+ };
const context: ScanContext = {
reader,
signal,
@@ -90,7 +97,12 @@ export const scanReader = async (
context.occurrenceByPath,
context.expandedContainerIds,
)?.occurrence_id ?? null;
- return { nodes, occurrences, pendingContradictions };
+ return {
+ nodes,
+ occurrences,
+ pendingContradictions,
+ caseCollisionLimitation: annotateEnglishUnicodeCaseCollisions(occurrences),
+ };
};
const visitArtifactEntries = async (
diff --git a/src/browser/history/HarCapture.test.ts b/src/browser/history/HarCapture.test.ts
index 4c29504a8a..2eb1fe21a0 100644
--- a/src/browser/history/HarCapture.test.ts
+++ b/src/browser/history/HarCapture.test.ts
@@ -1,7 +1,7 @@
import { createHash } from "node:crypto";
import { expect, it } from "vitest";
import { historicalHar } from "../../../tests/fixtures/historicalHar.js";
-import { decodeHarCapture } from "./HarCapture.js";
+import { decodeHarCapture, decodeHarText } from "./HarCapture.js";
it.each([
{ values: [] },
@@ -558,3 +558,23 @@ it.each([
});
},
);
+
+it("ignores one leading byte-order mark, as HAR 1.2 requires of readers", () => {
+ const text = JSON.stringify(historicalHar());
+ const body = Buffer.from(text, "utf8");
+ const marked = Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), body]);
+ expect(decodeHarText(marked)).toBe(text);
+ expect(decodeHarCapture(decodeHarText(marked), [])).toEqual(
+ decodeHarCapture(decodeHarText(body), []),
+ );
+ // A second mark is content, not an encoding signature.
+ expect(() =>
+ decodeHarCapture(
+ decodeHarText(Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), marked])),
+ [],
+ ),
+ ).toThrow("HAR is malformed JSON.");
+ expect(() => decodeHarText(Buffer.from([0x7b, 0xff, 0x7d]))).toThrow(
+ "HAR is not valid UTF-8 JSON.",
+ );
+});
diff --git a/src/browser/history/HarCapture.ts b/src/browser/history/HarCapture.ts
index ddc31567c0..f3a83af15d 100644
--- a/src/browser/history/HarCapture.ts
+++ b/src/browser/history/HarCapture.ts
@@ -1,4 +1,5 @@
import { createHash } from "node:crypto";
+import { TextDecoder } from "node:util";
import { isSafeNumber, LosslessNumber } from "lossless-json";
import { parseHarJson } from "./HarJson.js";
import type { JsonValue } from "../../domain/jsonValue.js";
@@ -25,6 +26,19 @@ type Projection = Pick<
"reported" | "numeric_literals" | "binary_fields" | "redactions"
>;
+/**
+ * Decode retained HAR bytes as UTF-8. HAR 1.2 lets writers emit a byte-order
+ * mark and requires readers to ignore it, so one leading BOM is skipped.
+ */
+export const decodeHarText = (bytes: Uint8Array): string => {
+ try {
+ // ignoreBOM: false (the default) is what skips the mark; true keeps it.
+ return new TextDecoder("utf-8", { fatal: true }).decode(bytes);
+ } catch {
+ throw new CaptureFormatError("format", "HAR is not valid UTF-8 JSON.");
+ }
+};
+
/** Decode a retained HAR with exact numeric lexemes and unchanged upstream schema validation. */
export const decodeHarCapture = (
text: string,
diff --git a/src/browser/history/HarCaptureProcess.ts b/src/browser/history/HarCaptureProcess.ts
index 0751d0b80f..1e4c66c761 100644
--- a/src/browser/history/HarCaptureProcess.ts
+++ b/src/browser/history/HarCaptureProcess.ts
@@ -1,10 +1,9 @@
import { readFile, writeFile } from "node:fs/promises";
-import { TextDecoder } from "node:util";
import { z } from "zod";
import { readStableArtifact } from "../../artifacts/readStableArtifact.js";
import { WEB_NETWORK_CAPTURE_LIMITS } from "../../domain/webNetworkCapture.js";
import { CaptureFormatError } from "./CaptureFormatError.js";
-import { decodeHarCapture } from "./HarCapture.js";
+import { decodeHarCapture, decodeHarText } from "./HarCapture.js";
const requestSchema = z.strictObject({
snapshot_path: z.string(),
@@ -23,17 +22,12 @@ const main = async (): Promise => {
request.snapshot_path,
WEB_NETWORK_CAPTURE_LIMITS.inputBytes,
);
- let text: string;
- try {
- text = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(
- snapshot.bytes,
- );
- } catch {
- throw new CaptureFormatError("format", "HAR is not valid UTF-8 JSON.");
- }
reply = {
ok: true,
- value: decodeHarCapture(text, request.sensitive_values),
+ value: decodeHarCapture(
+ decodeHarText(snapshot.bytes),
+ request.sensitive_values,
+ ),
};
} catch (cause: unknown) {
reply =
diff --git a/src/cli/analysisViewCommands.ts b/src/cli/analysisViewCommands.ts
index a0deb02290..a193d27400 100644
--- a/src/cli/analysisViewCommands.ts
+++ b/src/cli/analysisViewCommands.ts
@@ -18,7 +18,7 @@ export const registerAnalysisViewCommands = (
): void => {
cli.command(CLI_COMMANDS.inspectAnalysisView, {
description:
- "Project a selected view of completed layout or JavaScript application Evidence JSON",
+ "Project a selected view of completed layout, JavaScript application or native function Evidence JSON",
args: z.object({
inputJson: z.string().describe("Inline workflow JSON or JSON file path"),
}),
diff --git a/src/cli/jsonOutput.ts b/src/cli/jsonOutput.ts
index b42bd82bdf..f7422a7bf2 100644
--- a/src/cli/jsonOutput.ts
+++ b/src/cli/jsonOutput.ts
@@ -4,16 +4,27 @@ import { bufferedJsonParts, jsonParts } from "../domain/jsonSerialization.js";
type JsonOutputFormat = "json" | "jsonl";
-/** Write ordinary JSON without constructing a document-sized string. */
+/**
+ * Write ordinary JSON without constructing a document-sized string.
+ *
+ * `json` is indented for a terminal reader. Piped or redirected output, which
+ * agents and scripts consume, is compact: indentation adds no information and
+ * measured 16-45% more model tokens on real results. The value is identical.
+ */
export const writeJsonOutput = async (
value: unknown,
destination: Writable,
format: JsonOutputFormat = "json",
): Promise => {
- for (const part of bufferedJsonParts(jsonOutputParts(value, format)))
+ for (const part of bufferedJsonParts(
+ jsonOutputParts(value, format, isTerminal(destination)),
+ ))
await writePart(destination, part);
};
+const isTerminal = (destination: Writable): boolean =>
+ "isTTY" in destination && destination.isTTY === true;
+
const writePart = (destination: Writable, part: string): Promise =>
new Promise((resolve, reject) => {
const onError = (cause: unknown): void => reject(cause);
@@ -39,6 +50,7 @@ const writePart = (destination: Writable, part: string): Promise =>
export function* jsonOutputParts(
value: unknown,
format: JsonOutputFormat,
+ pretty = true,
): Generator {
if (format === "jsonl" && Array.isArray(value)) {
for (const item of value) {
@@ -47,6 +59,6 @@ export function* jsonOutputParts(
}
return;
}
- yield* jsonParts(value, format === "json");
+ yield* jsonParts(value, pretty && format === "json");
yield "\n";
}
diff --git a/src/cliBrowserScenarioCommands.ts b/src/cliBrowserScenarioCommands.ts
index f98e415cb2..a1a1f86e15 100644
--- a/src/cliBrowserScenarioCommands.ts
+++ b/src/cliBrowserScenarioCommands.ts
@@ -2,7 +2,6 @@ import { Cli } from "incur";
import { z } from "zod";
import { captureBrowserScenario } from "./application/BrowserScenarioCaptureService.js";
-import { createBrowserScenarioProvider } from "./composition/browserScenario.js";
import { CLI_COMMANDS } from "./cliCommandNames.js";
import { parseCliJsonInput, resolveCliJsonPaths } from "./cliJsonInput.js";
import { logCliCommand } from "./cliLogging.js";
@@ -44,6 +43,9 @@ export const registerBrowserScenarioCommands = (
projectInputIssues(scenario.error.issues, input.value),
),
);
+ // Load Playwright only for this command, not during CLI startup.
+ const { createBrowserScenarioProvider } =
+ await import("./composition/browserScenario.js");
const result = await captureBrowserScenario(
createBrowserScenarioProvider(environment),
scenario.data,
diff --git a/src/cliJsonFile.ts b/src/cliJsonFile.ts
index ea75f7c295..c0d474f698 100644
--- a/src/cliJsonFile.ts
+++ b/src/cliJsonFile.ts
@@ -8,6 +8,7 @@ import streamValues from "stream-json/streamers/stream-values.js";
import { withRegularFile } from "./application/RegularFileRead.js";
import {
+ JSON_BYTE_ORDER_MARK_MESSAGE,
JSON_INPUT_RESOURCE_REMEDIATION,
parseUtf8Json,
} from "./application/Utf8JsonInput.js";
@@ -211,12 +212,22 @@ async function* decodedChunks(
prefix?: Uint8Array,
): AsyncGenerator {
const decoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });
+ let leading = true;
+ const checked = (text: string): string => {
+ if (leading && text.length > 0) {
+ leading = false;
+ if (text.startsWith("\uFEFF")) throw new JsonByteOrderMarkError();
+ }
+ return text;
+ };
if (prefix !== undefined)
for (let offset = 0; offset < prefix.length; offset += READ_CHUNK_BYTES) {
signal?.throwIfAborted();
- yield decoder.decode(prefix.subarray(offset, offset + READ_CHUNK_BYTES), {
- stream: true,
- });
+ yield checked(
+ decoder.decode(prefix.subarray(offset, offset + READ_CHUNK_BYTES), {
+ stream: true,
+ }),
+ );
}
const bytes = Buffer.allocUnsafe(READ_CHUNK_BYTES);
while (true) {
@@ -224,9 +235,18 @@ async function* decodedChunks(
const read = await handle.read(bytes, 0, bytes.length, null);
signal?.throwIfAborted();
if (read.bytesRead === 0) break;
- yield decoder.decode(bytes.subarray(0, read.bytesRead), { stream: true });
+ yield checked(
+ decoder.decode(bytes.subarray(0, read.bytesRead), { stream: true }),
+ );
+ }
+ yield checked(decoder.decode());
+}
+
+/** A streamed JSON file begins with a byte-order mark; reported as invalid JSON. */
+class JsonByteOrderMarkError extends SyntaxError {
+ constructor() {
+ super(JSON_BYTE_ORDER_MARK_MESSAGE);
}
- yield decoder.decode();
}
const readPrefix = async (
diff --git a/src/config/environment.ts b/src/config/environment.ts
index f743de5761..3e02f68a28 100644
--- a/src/config/environment.ts
+++ b/src/config/environment.ts
@@ -5,11 +5,13 @@ import { ConfigurationError } from "../domain/configurationErrors.js";
import { err, ok, type Result } from "../domain/result.js";
import { analysisProviderSelectorSchema } from "../contracts/providerSelection.js";
import { ghidraStartupTimeoutSchema } from "./ghidraStartupTimeout.js";
+import { mcpInputSchemaProfileSchema } from "./mcpInputSchemaProfile.js";
import { mcpResponseBudgetSchema } from "./mcpResponseBudget.js";
const environmentSchema = z.object({
REA_ANALYSIS_PROVIDER: analysisProviderSelectorSchema.default("auto"),
REA_MCP_MAX_RESPONSE_BYTES: mcpResponseBudgetSchema.optional(),
+ REA_MCP_INPUT_SCHEMA_PROFILE: mcpInputSchemaProfileSchema.optional(),
REA_IDA_MCP_CONFIG: z
.string()
.min(1)
diff --git a/src/config/mcpInputSchemaProfile.ts b/src/config/mcpInputSchemaProfile.ts
new file mode 100644
index 0000000000..321803b5b6
--- /dev/null
+++ b/src/config/mcpInputSchemaProfile.ts
@@ -0,0 +1,38 @@
+import { z } from "zod";
+
+import { ConfigurationError } from "../domain/configurationErrors.js";
+import { err, ok, type Result } from "../domain/result.js";
+
+// Moonshot's function-calling API rejects requests while any single tool
+// parameters schema exceeds its literal serialized size cap. Community
+// measurements bound the cap between ~13.7 KB (accepted) and ~15 KB
+// (rejected); 13 KiB leaves margin under the lower bound. See
+// https://github.com/morluto/rea/issues/1484 for the measured evidence.
+export const COMPACT_INPUT_SCHEMA_BUDGET_BYTES = 13 * 1024;
+
+export const mcpInputSchemaProfileSchema = z.enum(["full", "compact"]);
+
+const profileConstraint =
+ 'REA_MCP_INPUT_SCHEMA_PROFILE must be "full" (default) or "compact". The compact profile advertises input schemas without nested annotation prose for providers that cap each tool schema\'s serialized size.';
+
+export type McpInputSchemaProfile = z.infer;
+
+/** Parse the advertised input schema presentation selection. */
+export const parseMcpInputSchemaProfile = (
+ value: string | undefined,
+): Result => {
+ if (value === undefined) return ok(undefined);
+ const parsed = mcpInputSchemaProfileSchema.safeParse(value);
+ return parsed.success
+ ? ok(parsed.data)
+ : err(
+ new ConfigurationError(profileConstraint, {
+ settings: [
+ {
+ setting: "REA_MCP_INPUT_SCHEMA_PROFILE",
+ constraint: profileConstraint,
+ },
+ ],
+ }),
+ );
+};
diff --git a/src/config/parseConfig.ts b/src/config/parseConfig.ts
index 074ed991a4..da6de5be2e 100644
--- a/src/config/parseConfig.ts
+++ b/src/config/parseConfig.ts
@@ -50,6 +50,9 @@ export const parseConfig = (
...(env.REA_MCP_MAX_RESPONSE_BYTES === undefined
? {}
: { mcpMaxResponseBytes: env.REA_MCP_MAX_RESPONSE_BYTES }),
+ ...(env.REA_MCP_INPUT_SCHEMA_PROFILE === undefined
+ ? {}
+ : { mcpInputSchemaProfile: env.REA_MCP_INPUT_SCHEMA_PROFILE }),
analysisProvider: env.REA_ANALYSIS_PROVIDER,
...(env.REA_IDA_MCP_CONFIG === undefined
? {}
diff --git a/src/config/types.ts b/src/config/types.ts
index 37696ff4cf..706db8e5cf 100644
--- a/src/config/types.ts
+++ b/src/config/types.ts
@@ -2,8 +2,11 @@ import type { LogLevel } from "../logger.js";
import type { ReferenceSourcePolicy } from "../domain/referenceSourcePolicy.js";
import type { AnalysisProviderSelector } from "../contracts/providerSelection.js";
+import type { McpInputSchemaProfile } from "./mcpInputSchemaProfile.js";
+
export interface AppConfig {
readonly mcpMaxResponseBytes?: number;
+ readonly mcpInputSchemaProfile?: McpInputSchemaProfile;
readonly analysisProvider: AnalysisProviderSelector;
readonly idaMcpConfigPath?: string;
readonly ghidraInstallDir: string | undefined;
diff --git a/src/contracts/analysisViewToolContracts.ts b/src/contracts/analysisViewToolContracts.ts
index b8f455767a..0edf10f604 100644
--- a/src/contracts/analysisViewToolContracts.ts
+++ b/src/contracts/analysisViewToolContracts.ts
@@ -8,14 +8,14 @@ import { evidenceResultOf } from "./toolOutputSchemaPrimitives.js";
const RETAINED_EXAMPLE_ID = `ev_${"a".repeat(64)}`;
-/** Selected views of already completed layout and JavaScript analysis Evidence. */
+/** Selected views of already completed layout, JavaScript and native function Evidence. */
export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
{
name: "inspect_analysis_view",
...toolContractMetadata("inspect_analysis_view"),
kind: "application",
description:
- "Inspect a selected view of completed binary-layout or JavaScript application Evidence without repeating analysis. Use an exact same-session retained evidence_id or portable inline Evidence. Select a summary, layout mitigations/linkage facet, one section/symbol/module, or a stable page with a positive limit. Module pages contain JavaScript assets, bundled modules and source modules; use node_id when paths are ambiguous or unavailable. Returns selected facts inline with artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the retained Evidence when needed.",
+ "Inspect selected views of completed binary-layout, JavaScript application or native analyze_function Evidence without repeating analysis. Use an exact same-session retained evidence_id or portable inline Evidence. Select a summary, layout mitigations/linkage facet, one section/symbol/module, or a stable page. Module pages include assets, bundled and source modules; use node_id for ambiguous or unavailable paths. Native views select procedure, pseudocode, assembly, basic blocks, comments, callers/callees, references, unresolved calls, referenced strings/names, native API and high-pcode. Use nonnegative offset and a positive caller-selected limit (defaults 0 and 64); pseudocode uses UTF-16 code units, other native pages use rows. Returns artifact identity, parent Evidence ID, a distinct view digest, coverage, limitations and unknowns. Actual serialized size determines MCP transport admission; choose a smaller page or export the complete retained Evidence if needed.",
inputSchema: inspectAnalysisViewInputSchema,
outputSchema: evidenceResultOf(analysisViewResultSchema),
examples: [
@@ -43,6 +43,21 @@ export const ANALYSIS_VIEW_TOOL_CONTRACTS = [
},
},
},
+ {
+ title: "Page native high-pcode operations from retained Evidence",
+ input: {
+ source: {
+ kind: "retained-evidence",
+ evidence_id: RETAINED_EXAMPLE_ID,
+ },
+ view: {
+ kind: "native",
+ facet: "value_flow_operations",
+ offset: 0,
+ limit: 32,
+ },
+ },
+ },
{
title: "Page module identities from retained application Evidence",
input: {
diff --git a/src/contracts/artifactToolContracts.ts b/src/contracts/artifactToolContracts.ts
index a6926a19fb..744ea34c3f 100644
--- a/src/contracts/artifactToolContracts.ts
+++ b/src/contracts/artifactToolContracts.ts
@@ -63,7 +63,7 @@ const artifact = <
export const ARTIFACT_TOOL_CONTRACTS = [
artifact(
"inspect_artifact",
- "Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph, source Evidence, observations, relationships, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
+ "Inspect an active archive or application package in one call. Returns the complete content-addressed artifact graph once, as the nested inventory Evidence, with root and integrity observations, integrity contradictions, hypotheses, unexplored branches, limitations, and next probes inline. Read-only DMG mounting is used automatically on supported macOS hosts. Integrity mismatches fail by default; record-and-continue keeps mismatches explicitly untrusted. It does not extract files; use extract_artifact to materialize its regular contents.",
artifactInventoryInputSchema,
),
artifact(
diff --git a/src/contracts/compactInputSchema.test.ts b/src/contracts/compactInputSchema.test.ts
new file mode 100644
index 0000000000..3fcfc42652
--- /dev/null
+++ b/src/contracts/compactInputSchema.test.ts
@@ -0,0 +1,314 @@
+import { Ajv2020 } from "ajv/dist/2020.js";
+import { describe, expect, it } from "vitest";
+
+import { compactAdvertisedInputSchema } from "./compactInputSchema.js";
+
+const serializedBytes = (value: unknown): number =>
+ Buffer.byteLength(JSON.stringify(value), "utf8");
+
+const repeatedObservation = {
+ type: "object",
+ properties: {
+ captured_url: { type: "string", minLength: 1 },
+ captured_at: { type: "string", minLength: 1 },
+ status_code: { type: "integer", minimum: 100, maximum: 599 },
+ headers: {
+ type: "object",
+ additionalProperties: { type: "string" },
+ minProperties: 1,
+ },
+ body_digest: { type: "string", minLength: 64, maxLength: 64 },
+ },
+ required: ["captured_url", "captured_at", "status_code"],
+ additionalProperties: false,
+} as const;
+
+const annotatedSchema = {
+ type: "object",
+ description: "Root group guidance stays.",
+ properties: {
+ query: {
+ type: "string",
+ description: "Field guidance leaves.",
+ examples: ["seed"],
+ default: "seed",
+ title: "Query",
+ minLength: 1,
+ },
+ limits: {
+ type: "object",
+ description: "Nested guidance leaves.",
+ properties: {
+ max_results: {
+ type: "integer",
+ description: "Leaves.",
+ minimum: 1,
+ maximum: 500,
+ default: 50,
+ },
+ },
+ required: ["max_results"],
+ additionalProperties: false,
+ },
+ },
+ required: ["query"],
+ dependentRequired: { limits: ["query"] },
+ additionalProperties: false,
+} as const;
+
+const oversizedPairSchema = (): Record => {
+ const deepStructure = {
+ type: "object",
+ description: "Nested guidance that leaves.",
+ properties: Object.fromEntries(
+ Array.from({ length: 40 }, (_, index) => [
+ `field_${index}`,
+ { type: "string", minLength: 1, description: `Field ${index}` },
+ ]),
+ ),
+ required: Array.from({ length: 40 }, (_, index) => `field_${index}`),
+ additionalProperties: false,
+ };
+ return {
+ type: "object",
+ description: "Provide a complete capture pair.",
+ properties: {
+ before: structuredClone(deepStructure),
+ after: structuredClone(deepStructure),
+ },
+ required: ["before", "after"],
+ additionalProperties: false,
+ };
+};
+
+describe("compact advertised input schema", () => {
+ it("drops nested annotation prose while keeping validation keywords", () => {
+ const { schema, presentation } = compactAdvertisedInputSchema(
+ structuredClone(annotatedSchema),
+ 13 * 1024,
+ );
+ expect(presentation).toBe("inline");
+ expect(schema.description).toBe("Root group guidance stays.");
+ const query = (schema.properties as Record)
+ .query as Record;
+ expect(query).toEqual({ type: "string", minLength: 1 });
+ const limits = (schema.properties as Record)
+ .limits as Record;
+ expect(limits.description).toBeUndefined();
+ expect(limits.required).toEqual(["max_results"]);
+ expect(schema.dependentRequired).toEqual({ limits: ["query"] });
+ expect(schema.required).toEqual(["query"]);
+ });
+
+ it("keeps annotation-shaped keys inside literal const and enum data", () => {
+ const literalMode = {
+ description: "saved data",
+ default: { keep: "me" },
+ };
+ const { schema } = compactAdvertisedInputSchema(
+ {
+ type: "object",
+ properties: {
+ mode: { enum: [literalMode] },
+ pinned: { const: { description: "literal" } },
+ },
+ },
+ 13 * 1024,
+ );
+ const properties = schema.properties as Record;
+ expect((properties.mode as Record).enum).toEqual([
+ literalMode,
+ ]);
+ expect((properties.pinned as Record).const).toEqual({
+ description: "literal",
+ });
+ });
+
+ it("inlines local references without changing validation or literal data", () => {
+ const original = {
+ type: "object",
+ $defs: { "observation/~": structuredClone(repeatedObservation) },
+ properties: {
+ before: { $ref: "#/$defs/observation~1~0" },
+ after: { $ref: "#/$defs/observation~1~0" },
+ literal: { const: { $ref: "saved data", $defs: { keep: true } } },
+ },
+ required: ["before", "after"],
+ additionalProperties: false,
+ };
+ const { schema, presentation } = compactAdvertisedInputSchema(
+ original,
+ 13312,
+ );
+ expect(presentation).toBe("inline");
+ expect(schema.$defs).toBeUndefined();
+ const ajv = new Ajv2020({ strict: false });
+ const canonical = ajv.compile(original);
+ const compact = ajv.compile(schema);
+ const observation = {
+ captured_url: "https://example.com",
+ captured_at: "now",
+ status_code: 200,
+ };
+ for (const input of [
+ { before: observation, after: observation },
+ { before: observation, after: { ...observation, status_code: 99 } },
+ {
+ before: observation,
+ after: observation,
+ literal: original.properties.literal.const,
+ },
+ ])
+ expect(compact(input)).toBe(canonical(input));
+ });
+
+ it("keeps unconstrained JSON values explicit for client type inference", () => {
+ const { schema } = compactAdvertisedInputSchema(
+ {
+ type: "object",
+ properties: { normalized_result: {}, literal: { const: {} } },
+ },
+ 13312,
+ );
+ const properties = schema.properties as Record;
+ expect(properties.normalized_result).toEqual({
+ type: ["null", "boolean", "object", "array", "number", "string"],
+ });
+ expect(properties.literal).toEqual({ const: {} });
+ const validate = new Ajv2020({ strict: false }).compile(schema);
+ for (const normalized_result of [null, true, {}, [], 42, "result"])
+ expect(validate({ normalized_result })).toBe(true);
+ });
+
+ it("preserves independent validation constraints on reference siblings", () => {
+ const original = {
+ type: "object",
+ $defs: { text: { type: "string", minLength: 3 } },
+ properties: {
+ query: { $ref: "#/$defs/text", minLength: 1, maxLength: 5 },
+ },
+ };
+ const { schema } = compactAdvertisedInputSchema(original, 13312);
+ const validate = new Ajv2020({ strict: false }).compile(schema);
+ expect(validate({ query: "a" })).toBe(false);
+ expect(validate({ query: "abc" })).toBe(true);
+ expect(validate({ query: "abcdef" })).toBe(false);
+ });
+});
+
+describe("compact input schema budget and reduction", () => {
+ it("budgets repeated-reference expansion before materializing it", () => {
+ const definitions: Record = {
+ leaf: structuredClone(repeatedObservation),
+ };
+ for (let index = 0; index < 30; index++) {
+ const $ref = `#/$defs/${index === 0 ? "leaf" : `level${index - 1}`}`;
+ definitions[`level${index}`] = {
+ type: "object",
+ properties: { left: { $ref }, right: { $ref } },
+ };
+ }
+ const { schema, presentation } = compactAdvertisedInputSchema(
+ {
+ type: "object",
+ $defs: definitions,
+ properties: { tree: { $ref: "#/$defs/level29" } },
+ },
+ 1024,
+ );
+ expect(presentation).toBe("reduced");
+ expect(serializedBytes(schema)).toBeLessThanOrEqual(1024);
+ expect(schema.properties).toEqual({ tree: { type: "object" } });
+ });
+
+ it("retains referenced object, array and nullable types in reduced inputs", () => {
+ const original = {
+ type: "object",
+ $defs: {
+ pair: oversizedPairSchema(),
+ observations: {
+ type: "array",
+ items: structuredClone(repeatedObservation),
+ },
+ nullable: { anyOf: [{ type: "string" }, { type: "null" }] },
+ },
+ properties: {
+ capture: { $ref: "#/$defs/pair", description: "Complete capture." },
+ observations: { $ref: "#/$defs/observations" },
+ label: { $ref: "#/$defs/nullable" },
+ },
+ };
+ const { schema, presentation } = compactAdvertisedInputSchema(
+ original,
+ 1024,
+ );
+ expect(presentation).toBe("reduced");
+ expect(schema.properties).toEqual({
+ capture: { type: "object", description: "Complete capture." },
+ observations: { type: "array" },
+ label: { type: ["string", "null"] },
+ });
+ const validate = new Ajv2020({ strict: false }).compile(schema);
+ expect(validate({ capture: {}, observations: [], label: null })).toBe(true);
+ expect(validate({ capture: {}, observations: [], label: "capture" })).toBe(
+ true,
+ );
+ expect(validate({ capture: "{}", observations: [], label: null })).toBe(
+ false,
+ );
+ });
+
+ it("reduces structurally oversized schemas to the shallow property presentation", () => {
+ const compacted = compactAdvertisedInputSchema(oversizedPairSchema(), 1024);
+ expect(compacted.presentation).toBe("reduced");
+ expect(serializedBytes(compacted.schema)).toBeLessThanOrEqual(1024);
+ expect(compacted.schema.type).toBe("object");
+ expect(compacted.schema.description).toContain(
+ "Provide a complete capture pair.",
+ );
+ expect(compacted.schema.description).toContain(
+ "compact input schema profile",
+ );
+ const properties = compacted.schema.properties as Record;
+ expect(properties.before).toEqual({
+ type: "object",
+ description: "Nested guidance that leaves.",
+ });
+ expect(properties.after).toEqual(properties.before);
+ expect(compacted.schema.required).toEqual(["before", "after"]);
+ });
+
+ it("produces deterministic output for equal input", () => {
+ const schema = {
+ type: "object",
+ properties: {
+ before: structuredClone(repeatedObservation),
+ after: structuredClone(repeatedObservation),
+ },
+ };
+ expect(compactAdvertisedInputSchema(schema, 13 * 1024)).toEqual(
+ compactAdvertisedInputSchema(structuredClone(schema), 13 * 1024),
+ );
+ });
+
+ it("keeps inline presentations under the measured provider budget", () => {
+ const { schema, presentation } = compactAdvertisedInputSchema(
+ {
+ type: "object",
+ description: "Group guidance.",
+ properties: {
+ before: structuredClone(repeatedObservation),
+ after: structuredClone(repeatedObservation),
+ normalization: {
+ type: "object",
+ description: "Rules.",
+ properties: { rules: { type: "array", items: { type: "string" } } },
+ },
+ },
+ },
+ 13312,
+ );
+ expect(presentation).toBe("inline");
+ expect(serializedBytes(schema)).toBeLessThanOrEqual(13312);
+ });
+});
diff --git a/src/contracts/compactInputSchema.ts b/src/contracts/compactInputSchema.ts
new file mode 100644
index 0000000000..321ea83235
--- /dev/null
+++ b/src/contracts/compactInputSchema.ts
@@ -0,0 +1,311 @@
+import {
+ schemaArrays,
+ schemaChildren,
+ schemaMaps,
+} from "./inputSchemaPresentation.js";
+
+// Kimi expands schema-local references before sending tools to Moonshot.
+// Compact advertisements must fit after expansion, independently of client
+// reference handling. Canonical server-side validation remains unchanged.
+
+/** Annotation keys that carry guidance but no validation semantics. */
+const annotationKeys = new Set(["description", "examples", "default", "title"]);
+
+/** Keys whose values are literal data, never nested schemas to rewrite. */
+const dataKeys = new Set(["const", "enum", "examples", "default"]);
+const definitionKeys = new Set(["$defs", "definitions"]);
+// An empty schema accepts every JSON value. Kimi otherwise infers string.
+const jsonValueTypes = [
+ "null",
+ "boolean",
+ "object",
+ "array",
+ "number",
+ "string",
+];
+
+export type AdvertisedInputSchemaPresentation = "inline" | "reduced";
+
+export interface CompactedInputSchema {
+ readonly schema: Record;
+ readonly presentation: AdvertisedInputSchemaPresentation;
+}
+
+const isSchemaObject = (value: unknown): value is Record =>
+ typeof value === "object" && value !== null && !Array.isArray(value);
+
+const serializedBytes = (value: unknown): number =>
+ Buffer.byteLength(JSON.stringify(value), "utf8");
+
+/** Rewrite every schema-valued position; literal data stays opaque. */
+const rebuildSchemaPositions = (
+ node: unknown,
+ rebuild: (child: unknown) => unknown,
+): unknown => {
+ if (Array.isArray(node)) return node.map((child) => rebuild(child));
+ if (!isSchemaObject(node)) return node;
+ const out: Record = {};
+ for (const [key, value] of Object.entries(node)) {
+ if (dataKeys.has(key)) {
+ out[key] = value;
+ continue;
+ }
+ if (schemaMaps.has(key) && isSchemaObject(value)) {
+ out[key] = Object.fromEntries(
+ Object.entries(value).map(([name, child]) => [name, rebuild(child)]),
+ );
+ continue;
+ }
+ if (schemaArrays.has(key) && Array.isArray(value)) {
+ out[key] = value.map((child) => rebuild(child));
+ continue;
+ }
+ if (schemaChildren.has(key)) {
+ out[key] = Array.isArray(value)
+ ? value.map((child) => rebuild(child))
+ : rebuild(value);
+ continue;
+ }
+ out[key] = value;
+ }
+ return out;
+};
+
+/**
+ * Drop annotation prose from every subschema while preserving all validation
+ * keywords. The root keeps its own description because the root-union
+ * projection encodes accepted input groups there; the root's echoed literal
+ * examples leave with the rest of the annotation prose.
+ */
+const stripAnnotations = (
+ schema: Record,
+): Record => {
+ const rebuilt = rebuildSchemaPositions(
+ schema,
+ stripChildAnnotations,
+ ) as Record;
+ const out: Record = {};
+ for (const [key, value] of Object.entries(rebuilt)) {
+ if (annotationKeys.has(key) && key !== "description") continue;
+ out[key] = value;
+ }
+ return out;
+};
+
+const stripChildAnnotations = (child: unknown): unknown => {
+ const rebuilt = rebuildSchemaPositions(child, stripChildAnnotations);
+ if (!isSchemaObject(rebuilt)) return rebuilt;
+ const kept: Record = {};
+ for (const [key, value] of Object.entries(rebuilt))
+ if (!annotationKeys.has(key)) kept[key] = value;
+ return kept;
+};
+
+const resolveLocalReference = (
+ root: Record,
+ reference: string,
+): unknown => {
+ if (reference === "#") return root;
+ if (!reference.startsWith("#/")) return undefined;
+ let value: unknown = root;
+ for (const token of reference.slice(2).split("/")) {
+ const key = token.replaceAll("~1", "/").replaceAll("~0", "~");
+ if (!isSchemaObject(value) || !Object.hasOwn(value, key)) return undefined;
+ value = value[key];
+ }
+ return value;
+};
+
+/** Stop serialization at the budget before allocating an expanded schema. */
+const inlineWithinBudget = (
+ root: Record,
+ budgetBytes: number,
+): Record | undefined => {
+ let resolved = true;
+ function* serialize(
+ value: unknown,
+ position: "schema" | "map" | "data",
+ active: ReadonlySet,
+ ): Generator {
+ if (Array.isArray(value)) {
+ yield "[";
+ for (const [index, child] of value.entries()) {
+ if (index > 0) yield ",";
+ yield* serialize(child, position, active);
+ }
+ yield "]";
+ } else if (isSchemaObject(value)) {
+ if (position === "schema" && Object.keys(value).length === 0) {
+ yield JSON.stringify({ type: jsonValueTypes });
+ return;
+ }
+ if (position === "schema" && typeof value.$ref === "string") {
+ const target = resolveLocalReference(root, value.$ref);
+ if (target === undefined || active.has(value.$ref)) {
+ resolved = false;
+ return;
+ }
+ const { $ref, ...siblings } = value;
+ // Reference siblings are independent constraints in draft 2020-12.
+ // Merging their keys into the target would overwrite validation rules.
+ yield* serialize(
+ Object.keys(siblings).length === 0
+ ? target
+ : { allOf: [target, siblings] },
+ "schema",
+ new Set([...active, $ref]),
+ );
+ return;
+ }
+ yield "{";
+ let first = true;
+ for (const [key, child] of Object.entries(value)) {
+ if (position === "schema" && definitionKeys.has(key)) continue;
+ if (!first) yield ",";
+ first = false;
+ yield JSON.stringify(key) + ":";
+ const childPosition =
+ position === "map"
+ ? "schema"
+ : position === "schema" && schemaMaps.has(key)
+ ? "map"
+ : position === "schema" &&
+ (schemaArrays.has(key) || schemaChildren.has(key))
+ ? "schema"
+ : "data";
+ yield* serialize(child, childPosition, active);
+ }
+ yield "}";
+ } else {
+ yield JSON.stringify(value);
+ }
+ }
+ const chunks: string[] = [];
+ let bytes = 0;
+ for (const chunk of serialize(root, "schema", new Set())) {
+ bytes += Buffer.byteLength(chunk, "utf8");
+ if (bytes > budgetBytes || !resolved) return undefined;
+ chunks.push(chunk);
+ }
+ return resolved
+ ? (JSON.parse(chunks.join("")) as Record)
+ : undefined;
+};
+
+/** Preserve every possible type, including referenced and nullable unions. */
+const detectedTypes = (
+ schema: unknown,
+ root: Record,
+ active: ReadonlySet = new Set(),
+): readonly string[] | undefined => {
+ if (!isSchemaObject(schema)) return undefined;
+ if (typeof schema.$ref === "string") {
+ if (active.has(schema.$ref)) return undefined;
+ return detectedTypes(
+ resolveLocalReference(root, schema.$ref),
+ root,
+ new Set([...active, schema.$ref]),
+ );
+ }
+ if (typeof schema.type === "string") return [schema.type];
+ if (
+ Array.isArray(schema.type) &&
+ schema.type.every((type) => typeof type === "string")
+ )
+ return schema.type;
+ const alternatives = Array.isArray(schema.anyOf)
+ ? schema.anyOf
+ : Array.isArray(schema.oneOf)
+ ? schema.oneOf
+ : [];
+ if (alternatives.length === 0) return undefined;
+ const types = alternatives.map((branch) =>
+ detectedTypes(branch, root, active),
+ );
+ if (types.some((type) => type === undefined)) return undefined;
+ return [...new Set(types.flatMap((type) => type ?? []))];
+};
+
+const REDUCED_ANNOTATION =
+ "Advertised in reduced form by the compact input schema profile; the server still validates complete canonical inputs.";
+
+/**
+ * Present only the root property shape with per-property guidance. Used for
+ * schemas whose validation structure alone exceeds the compact budget; the
+ * canonical schema still rejects malformed calls server-side.
+ */
+const reduceToShallowProperties = (
+ schema: Record,
+): Record => {
+ const properties = isSchemaObject(schema.properties) ? schema.properties : {};
+ const description = [
+ typeof schema.description === "string" ? schema.description : undefined,
+ REDUCED_ANNOTATION,
+ ]
+ .filter((text) => text !== undefined)
+ .join(" ");
+ const $schema =
+ typeof schema.$schema === "string" ? { $schema: schema.$schema } : {};
+ const shallow = Object.fromEntries(
+ Object.entries(properties).map(([name, value]) => {
+ const types = detectedTypes(value, schema);
+ const type = types?.length === 1 ? types[0] : types;
+ const description =
+ isSchemaObject(value) && typeof value.description === "string"
+ ? { description: value.description }
+ : {};
+ return [
+ name,
+ { ...(type === undefined ? {} : { type }), ...description },
+ ];
+ }),
+ );
+ return {
+ ...$schema,
+ type: "object",
+ description,
+ properties: shallow,
+ ...(Array.isArray(schema.required)
+ ? { required: [...schema.required] }
+ : {}),
+ ...(typeof schema.minProperties === "number"
+ ? { minProperties: schema.minProperties }
+ : {}),
+ ...(isSchemaObject(schema.dependentRequired)
+ ? { dependentRequired: { ...schema.dependentRequired } }
+ : {}),
+ ...(schema.additionalProperties === false
+ ? { additionalProperties: false }
+ : {}),
+ };
+};
+
+/**
+ * Render an advertised input schema within a per-schema byte budget for
+ * providers that validate the literal serialized schema size. Validation
+ * keywords and the canonical contract are preserved; only
+ * annotation prose and, for structurally oversized schemas, nested
+ * validation detail leave the advertised form.
+ */
+export const compactAdvertisedInputSchema = (
+ schema: Record,
+ budgetBytes: number,
+): CompactedInputSchema => {
+ const inline = inlineWithinBudget(stripAnnotations(schema), budgetBytes);
+ if (inline !== undefined) return { schema: inline, presentation: "inline" };
+ const reduced = reduceToShallowProperties(schema);
+ if (serializedBytes(reduced) <= budgetBytes)
+ return { schema: reduced, presentation: "reduced" };
+ // A future contract with an extreme root-property count still advertises
+ // an honest, budget-fitting shape instead of an oversized one.
+ return {
+ schema: reduceToShallowProperties({
+ type: "object",
+ description:
+ (typeof schema.description === "string"
+ ? `${schema.description} `
+ : "") + REDUCED_ANNOTATION,
+ }),
+ presentation: "reduced",
+ };
+};
diff --git a/src/contracts/inputSchemaPresentation.ts b/src/contracts/inputSchemaPresentation.ts
index 8b1bd5b9ea..9663dd8329 100644
--- a/src/contracts/inputSchemaPresentation.ts
+++ b/src/contracts/inputSchemaPresentation.ts
@@ -1,6 +1,6 @@
import { isDeepStrictEqual } from "node:util";
-const schemaMaps = new Set([
+export const schemaMaps = new Set([
"$defs",
"definitions",
"properties",
@@ -8,8 +8,8 @@ const schemaMaps = new Set([
"dependentSchemas",
"dependencies",
]);
-const schemaArrays = new Set(["allOf", "anyOf", "oneOf", "prefixItems"]);
-const schemaChildren = new Set([
+export const schemaArrays = new Set(["allOf", "anyOf", "oneOf", "prefixItems"]);
+export const schemaChildren = new Set([
"items",
"additionalItems",
"contains",
@@ -59,7 +59,7 @@ const referencedDescription = (
const describeSchema = (
schema: unknown,
root: Readonly>,
- describeProperty: (property: string) => string,
+ describeProperty: (property: string) => string | undefined,
): unknown => {
if (!isObject(schema)) return schema;
return Object.fromEntries(
@@ -70,16 +70,16 @@ const describeSchema = (
Object.fromEntries(
Object.entries(value).map(([name, child]) => {
const described = describeSchema(child, root, describeProperty);
+ if (key !== "properties" || !isObject(described))
+ return [name, described];
+ const description =
+ referencedDescription(root, described) ??
+ describeProperty(name);
return [
name,
- key === "properties" && isObject(described)
- ? {
- ...described,
- description:
- referencedDescription(root, described) ??
- describeProperty(name),
- }
- : described,
+ description === undefined
+ ? described
+ : { ...described, description },
];
}),
),
@@ -223,7 +223,7 @@ const flattenRootObjectUnion = (
/** Describe canonical input fields and present the root as one object. */
export const presentInputJsonSchema = (
root: Record,
- describeProperty: (property: string) => string,
+ describeProperty: (property: string) => string | undefined,
): Record => {
const flattened = flattenRootObjectUnion(root);
const described = describeSchema(flattened, flattened, describeProperty);
diff --git a/src/contracts/toolInputReferences.test.ts b/src/contracts/toolInputReferences.test.ts
index 4441ba9f76..c661fe040f 100644
--- a/src/contracts/toolInputReferences.test.ts
+++ b/src/contracts/toolInputReferences.test.ts
@@ -97,9 +97,14 @@ describe("advertised input references", () => {
before: { description: "Exact byte range in the selected artifact." },
after: { description: "Exact byte range in the selected artifact." },
metadata: { default: literal, examples: [literal] },
- constructor: { description: "Value for constructor." },
},
});
+ // Inherited object keys never supply a description.
+ expect(inline).not.toHaveProperty([
+ "properties",
+ "constructor",
+ "description",
+ ]);
const ajv =
target === "draft-2020-12"
? new Ajv2020({ strict: false })
diff --git a/src/contracts/toolOutputReferences.test.ts b/src/contracts/toolOutputReferences.test.ts
index 70c1f5ea5e..cda49f1d5e 100644
--- a/src/contracts/toolOutputReferences.test.ts
+++ b/src/contracts/toolOutputReferences.test.ts
@@ -1,11 +1,69 @@
import { Ajv } from "ajv";
import { Ajv2020 } from "ajv/dist/2020.js";
+import { AjvJsonSchemaValidator } from "@modelcontextprotocol/client/validators/ajv";
import { expect, it } from "vitest";
import { z } from "zod";
import { toolContract } from "./toolContracts.js";
import { toolOutputSchemaWithMetadata } from "./toolSchemaMetadata.js";
+it("keeps dialect-specific validators distinct after JSON schema serialization", () => {
+ const contract = {
+ ...toolContract("binary_session"),
+ outputSchema: z.strictObject({ value: z.number().int().min(1).max(8) }),
+ };
+ const output = toolOutputSchemaWithMetadata(contract);
+ const provider = new AjvJsonSchemaValidator();
+ for (const target of ["draft-2020-12", "draft-07"] as const) {
+ const projected = output["~standard"].jsonSchema.output({ target });
+ const parsed = z
+ .record(z.string(), z.unknown())
+ .parse(JSON.parse(JSON.stringify(projected)));
+ const validate = provider.getValidator(parsed);
+ expect(validate({ value: 8 }).valid).toBe(true);
+ expect(validate({ value: 9 }).valid).toBe(false);
+ expect(validate({ value: 1.5 }).valid).toBe(false);
+ }
+});
+
+it("preserves explicit schema identity and relative reference resolution", () => {
+ const value = z.string();
+ const canonical = z.strictObject({ value });
+ const contract = {
+ ...toolContract("binary_session"),
+ outputSchema: canonical,
+ };
+ const projected = toolOutputSchemaWithMetadata(contract)[
+ "~standard"
+ ].jsonSchema.output({
+ target: "draft-2020-12",
+ libraryOptions: {
+ override: (context: {
+ readonly zodSchema: unknown;
+ readonly jsonSchema: Record;
+ }) => {
+ if (context.zodSchema !== value) return;
+ delete context.jsonSchema.type;
+ context.jsonSchema.$ref = "child.json";
+ },
+ },
+ });
+ const ajv = new Ajv2020({ strict: false });
+ ajv.addSchema({ $id: "child.json", type: "string", minLength: 3 });
+ const validate = ajv.compile(projected);
+ expect(validate({ value: "abc" })).toBe(true);
+ expect(validate({ value: "ab" })).toBe(false);
+
+ const identified = toolOutputSchemaWithMetadata({
+ ...contract,
+ outputSchema: canonical.meta({ $id: "https://example.test/result" }),
+ })["~standard"].jsonSchema.output({ target: "draft-2020-12" });
+ expect(identified.$id).toBe("https://example.test/result");
+ const explicit = ajv.compile(identified);
+ expect(explicit({ value: "abc" })).toBe(true);
+ expect(explicit({ value: 1 })).toBe(false);
+});
+
it.each(["draft-2020-12", "draft-07"] as const)(
"preserves refined numeric bounds and strict repeated objects through %s references",
(target) => {
diff --git a/src/contracts/toolSchemaMetadata.ts b/src/contracts/toolSchemaMetadata.ts
index f2d395ece8..34089a929f 100644
--- a/src/contracts/toolSchemaMetadata.ts
+++ b/src/contracts/toolSchemaMetadata.ts
@@ -2,6 +2,7 @@ import { z } from "zod";
import type { ToolContract } from "./toolContractTypes.js";
import { presentInputJsonSchema } from "./inputSchemaPresentation.js";
+import { digestCanonicalValue } from "../domain/canonicalDigest.js";
const PROPERTY_DESCRIPTIONS: Readonly> = {
addresses: "Ordered provider-normalized procedure addresses to analyze.",
@@ -69,20 +70,95 @@ export const toolInputSchemaWithMetadata = (
},
);
-/** Share repeated output definitions without changing canonical validation. */
+/** Share output definitions and content-bound validator identities on the wire. */
export const toolOutputSchemaWithMetadata = (
contract: Contract,
): Contract["outputSchema"] =>
withAdvertisedJsonSchema(
contract.outputSchema,
"output",
- (project) => (options) =>
- project({
+ (project) => (options) => {
+ const shared = project({
...options,
libraryOptions: { reused: "ref", ...options.libraryOptions },
- }),
+ });
+ // An explicit ID owns the reference base. Relative external references
+ // also depend on that base, so leave those projections unchanged.
+ if ("$id" in shared || hasRelativeSchemaReference(shared)) return shared;
+ return {
+ ...shared,
+ $id: `urn:rea:tool-output:sha256:${digestCanonicalValue(
+ { target: options.target, schema: shared },
+ "Tool output schema",
+ )}`,
+ };
+ },
);
+/**
+ * Apply a wire-presentation transform to one schema's advertised input
+ * projection. The wrapper preserves the schema's own parser and standard
+ * metadata, so both Zod registrations and plain standard schemas keep their
+ * canonical validation while only the advertised JSON changes.
+ */
+export const transformAdvertisedInputJsonSchema = (
+ schema: Schema,
+ transform: (projected: Record) => Record,
+): Schema => {
+ const standard = Reflect.get(schema as object, "~standard");
+ if (!isSchemaRecord(standard)) return schema;
+ const jsonSchema = Reflect.get(standard, "jsonSchema");
+ if (!isSchemaRecord(jsonSchema)) return schema;
+ const project = Reflect.get(jsonSchema, "input");
+ if (typeof project !== "function") return schema;
+ const byTarget = new Map>();
+ const transformedInput = (options: unknown): unknown => {
+ const target = Reflect.get(options as object, "target");
+ if (typeof target === "string") {
+ const cached = byTarget.get(target);
+ if (cached !== undefined) return cached;
+ }
+ const projected = (project as (options: unknown) => unknown)(options);
+ const transformed = isSchemaRecord(projected)
+ ? transform(projected)
+ : projected;
+ if (typeof target === "string" && isSchemaRecord(transformed))
+ byTarget.set(target, transformed);
+ return transformed;
+ };
+ return {
+ ...(schema as object),
+ "~standard": {
+ ...standard,
+ jsonSchema: { ...jsonSchema, input: transformedInput },
+ },
+ } as Schema;
+};
+
+const hasRelativeSchemaReference = (
+ schema: Record,
+): boolean => {
+ const pending: unknown[] = [schema];
+ const visited = new WeakSet();
+ while (pending.length > 0) {
+ const current = pending.pop();
+ if (current === null || typeof current !== "object" || visited.has(current))
+ continue;
+ visited.add(current);
+ for (const [key, value] of Object.entries(current)) {
+ if (
+ ["$id", "$ref", "$dynamicRef", "$recursiveRef"].includes(key) &&
+ typeof value === "string" &&
+ !value.startsWith("#") &&
+ !URL.canParse(value)
+ )
+ return true;
+ if (value !== null && typeof value === "object") pending.push(value);
+ }
+ }
+ return false;
+};
+
const withAdvertisedJsonSchema = (
canonical: Schema,
io: "input" | "output",
@@ -128,7 +204,10 @@ const memoizeByTarget = (
};
};
-const fallbackPropertyDescription = (property: string): string => {
+const isSchemaRecord = (value: unknown): value is Record =>
+ typeof value === "object" && value !== null && !Array.isArray(value);
+
+const fallbackPropertyDescription = (property: string): string | undefined => {
const explicit = Object.hasOwn(PROPERTY_DESCRIPTIONS, property)
? PROPERTY_DESCRIPTIONS[property]
: undefined;
@@ -156,5 +235,6 @@ const fallbackPropertyDescription = (property: string): string => {
return `Local filesystem ${words} selected for this operation.`;
if (property.startsWith("is_") || property.startsWith("has_"))
return `Whether ${words}.`;
- return `Value for ${words}.`;
+ // A generic sentence would only restate the property name in every schema.
+ return undefined;
};
diff --git a/src/domain/analysisErrorPresentation.ts b/src/domain/analysisErrorPresentation.ts
index 7c71a9b5dc..b4fe4ad64b 100644
--- a/src/domain/analysisErrorPresentation.ts
+++ b/src/domain/analysisErrorPresentation.ts
@@ -191,7 +191,7 @@ export const analysisErrorUserMessage = (error: AnalysisError): string => {
const standardMessage = standardErrorMessage(error._tag);
if (standardMessage !== undefined) return standardMessage;
if (error instanceof ArtifactOperationError)
- return artifactMessage(error.operation, error.reason);
+ return artifactMessage(error.operation, error.reason, error.detail);
if (error instanceof EvidenceReferenceError)
return error.reason === "missing"
? `Evidence ${error.evidenceId} is not retained in this session. Supply complete inline Evidence, re-run its producer, or import its bundle before using this reference.`
@@ -329,13 +329,13 @@ const START_FAILURE_TAGS: ReadonlySet = new Set([
const artifactMessage = (
operation: ArtifactOperationError["operation"],
reason: ArtifactOperationError["reason"],
+ detail?: string,
): string => {
if (reason === "cancelled")
return "Artifact operation was cancelled. Start it again when ready.";
if (reason === "limit")
return "Artifact is too large to process safely. Narrow the requested path or use a smaller artifact.";
- if (reason === "path")
- return "Artifact contains an unsafe or conflicting internal path. Inspect the reported path and correct the artifact before retrying.";
+ if (reason === "path") return artifactPathMessage(detail);
if (reason === "unavailable")
return "Artifact processing is unavailable for the current target or host. Check artifact support and required tools.";
if (reason === "integrity" && operation === "analyze_javascript_application")
@@ -345,6 +345,23 @@ const artifactMessage = (
return "Artifact could not be read or written. Check file access and try again.";
};
+const artifactPathMessage = (detail: string | undefined): string => {
+ // Kept in step with DESTINATION_CASE_COLLISION_PREFIX. Domain does not import
+ // the artifact adapter that produces the detail.
+ if (
+ detail !== undefined &&
+ detail.startsWith("Destination filesystem cannot store both ")
+ )
+ return detail;
+ if (
+ detail !== undefined &&
+ (detail.startsWith("Artifact path is absolute or unsafe:") ||
+ detail.startsWith("Artifact path is not normalized:"))
+ )
+ return "Artifact contains an unsafe internal path. Inspect the reported path and correct the artifact before retrying.";
+ return "Artifact contains a conflicting internal path. Inspect the reported path before retrying.";
+};
+
const evidenceFileMessage = ({
operation,
reason,
diff --git a/src/domain/analysisView/analysisView.test.ts b/src/domain/analysisView/analysisView.test.ts
index e3aabcf143..bc94baf4aa 100644
--- a/src/domain/analysisView/analysisView.test.ts
+++ b/src/domain/analysisView/analysisView.test.ts
@@ -1,6 +1,8 @@
import { expect, it } from "vitest";
import { resolveJavaScriptSourceMapReference } from "../javascript/javascriptSourceMapPaths.js";
+import { functionDossierSchema } from "../hopperValues.js";
+import { ghidraFunctionDossier } from "../ghidraValues.fixture.js";
import { JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE } from "../../contracts/javascript/javascriptRuntimeReconciliationExample.js";
import {
analysisViewBindJavaScriptGraphs,
@@ -48,21 +50,248 @@ const javascriptParent = (
limitations: JAVASCRIPT_APPLICATION_EVIDENCE_EXAMPLE.limitations,
});
-it("accepts caller-selected page sizes and rejects malformed bounds", () => {
- const input = {
- source: { kind: "retained-evidence", evidence_id: `ev_${"a".repeat(64)}` },
- view: { kind: "page", collection: "sections", offset: 0, limit: 500 },
- };
- expect(inspectAnalysisViewInputSchema.safeParse(input).success).toBe(true);
- for (const limit of [0, -1, 0.5, Infinity, Number.MAX_SAFE_INTEGER + 1])
- expect(
- inspectAnalysisViewInputSchema.safeParse({
- ...input,
- view: { ...input.view, limit },
- }).success,
- ).toBe(false);
+const nativeParent = (
+ result: unknown = ghidraFunctionDossier(),
+): AnalysisViewParent => ({
+ evidenceId: "ev_" + "b".repeat(64),
+ operation: "analyze_function",
+ normalizedResult: result,
+ limitations: ["Retained provider limitation"],
+ artifact: { path: "/fixtures/example.exe", sha256: "a".repeat(64) },
});
+it("projects native procedure and bounded assembly and high-pcode with exact parent identity", () => {
+ const parent = nativeParent();
+ const procedure = projectAnalysisView(parent, {
+ kind: "native",
+ facet: "procedure",
+ offset: 0,
+ limit: 64,
+ });
+ if (!procedure.ok) throw procedure.error;
+ expect(procedure.value).toMatchObject({
+ kind: "native",
+ parent_operation: "analyze_function",
+ artifact: parent.artifact,
+ item: { address: "0x401000" },
+ });
+ expect(procedure.value.view_digest).not.toBe(parent.evidenceId.slice(3));
+ const assembly = projectAnalysisView(parent, {
+ kind: "native",
+ facet: "assembly",
+ offset: 0,
+ limit: 1,
+ });
+ if (!assembly.ok) throw assembly.error;
+ expect(assembly.value).toMatchObject({
+ coverage: { examined: 1, total: 2, next_offset: 1 },
+ item: ["0x401000: CALL 0x401020"],
+ });
+ const flow = projectAnalysisView(parent, {
+ kind: "native",
+ facet: "value_flow_operations",
+ offset: 0,
+ limit: 1,
+ });
+ if (!flow.ok) throw flow.error;
+ expect(flow.value).toMatchObject({ item: [{ opcode: "COPY" }] });
+});
+
+it("rejects invalid native parents, unsupported source operations and singleton offsets", () => {
+ const view = {
+ kind: "native",
+ facet: "assembly",
+ offset: 0,
+ limit: 4,
+ } as const;
+ expect(projectAnalysisView(nativeParent({}), view)).toMatchObject({
+ ok: false,
+ error: { _tag: "AnalysisInputError" },
+ });
+ expect(
+ projectAnalysisView({ ...nativeParent(), artifact: null }, view),
+ ).toMatchObject({
+ ok: false,
+ error: { _tag: "AnalysisInputError" },
+ });
+ expect(projectAnalysisView(layoutParent(), view)).toMatchObject({
+ ok: false,
+ error: { _tag: "AnalysisInputError" },
+ });
+ expect(projectAnalysisView(javascriptParent(), view)).toMatchObject({
+ ok: false,
+ error: { _tag: "AnalysisInputError" },
+ });
+ expect(
+ projectAnalysisView(nativeParent(), {
+ kind: "native",
+ facet: "procedure",
+ offset: 1,
+ limit: 2,
+ }),
+ ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } });
+});
+
+it("preserves native API limitations and unavailable residual unknowns", () => {
+ const original = functionDossierSchema.parse(ghidraFunctionDossier());
+ const view = {
+ kind: "native",
+ facet: "procedure",
+ offset: 0,
+ limit: 64,
+ } as const;
+ const unavailable = projectAnalysisView(
+ nativeParent({
+ ...original,
+ native_api: {
+ available: false,
+ reason: "No decompiler",
+ residual_unknowns: ["Calling convention unresolved"],
+ },
+ }),
+ view,
+ );
+ if (!unavailable.ok) throw unavailable.error;
+ expect(unavailable.value.unknowns).toContain("Calling convention unresolved");
+ const available = projectAnalysisView(nativeParent(), view);
+ if (!available.ok) throw available.error;
+ if (!original.native_api?.available) throw new Error("expected API fixture");
+ expect(available.value.limitations).toEqual(
+ expect.arrayContaining(original.native_api.limitations),
+ );
+});
+
+it("pages every retained dossier collection and the native API record", () => {
+ const original = functionDossierSchema.parse(ghidraFunctionDossier());
+ const page = (
+ facet:
+ | "basic_blocks"
+ | "comments"
+ | "unresolved_calls"
+ | "referenced_strings"
+ | "referenced_names",
+ result: unknown = original,
+ ) => {
+ const view = projectAnalysisView(nativeParent(result), {
+ kind: "native",
+ facet,
+ offset: 0,
+ limit: 64,
+ });
+ if (!view.ok) throw view.error;
+ return view.value;
+ };
+ expect(page("basic_blocks")).toMatchObject({
+ item: original.basic_blocks,
+ coverage: { total: original.basic_blocks.length, exhausted: true },
+ });
+ expect(page("referenced_strings")).toMatchObject({
+ item: original.referenced_strings,
+ });
+ expect(
+ page("comments", {
+ ...original,
+ comments: [{ address: "0x401000", kind: "comment", text: "entry" }],
+ }),
+ ).toMatchObject({ item: [{ text: "entry" }] });
+ expect(page("referenced_names")).toMatchObject({ coverage: { total: 0 } });
+ const { unresolved_calls: _omitted, ...withoutUnresolved } = original;
+ expect(
+ page("unresolved_calls", {
+ ...withoutUnresolved,
+ native_value_flow: {
+ available: false,
+ reason: "No high p-code",
+ limitations: [],
+ },
+ }),
+ ).toMatchObject({ item: { available: false, reason: "not recorded" } });
+ const api = projectAnalysisView(nativeParent(), {
+ kind: "native",
+ facet: "native_api",
+ offset: 0,
+ limit: 64,
+ });
+ if (!api.ok) throw api.error;
+ expect(api.value).toMatchObject({
+ item: original.native_api,
+ coverage: { status: "complete-within-view" },
+ });
+});
+
+it("reports the malformed native dossier field", () => {
+ const result = projectAnalysisView(
+ nativeParent({
+ ...functionDossierSchema.parse(ghidraFunctionDossier()),
+ pseudocode: 12,
+ }),
+ {
+ kind: "native",
+ facet: "pseudocode",
+ offset: 0,
+ limit: 64,
+ },
+ );
+ expect(result).toMatchObject({
+ ok: false,
+ error: {
+ issues: expect.arrayContaining([
+ expect.objectContaining({
+ path: ["source", "normalized_result", "pseudocode"],
+ }),
+ ]),
+ },
+ });
+});
+
+it("native pseudocode pages do not split surrogate pairs", () => {
+ const original = functionDossierSchema.parse(ghidraFunctionDossier());
+ const parent = nativeParent({ ...original, pseudocode: "ab😀cd" });
+ expect(
+ projectAnalysisView(parent, {
+ kind: "native",
+ facet: "pseudocode",
+ offset: 3,
+ limit: 2,
+ }),
+ ).toMatchObject({ ok: false, error: { _tag: "AnalysisInputError" } });
+ const safe = projectAnalysisView(parent, {
+ kind: "native",
+ facet: "pseudocode",
+ offset: 2,
+ limit: 3,
+ });
+ if (!safe.ok) throw safe.error;
+ expect(safe.value).toMatchObject({
+ item: { text: "😀c", unit: "utf16-code-units" },
+ coverage: { examined: 3, next_offset: 5 },
+ });
+});
+
+it.each([
+ { kind: "page", collection: "sections", offset: 0, limit: 500 },
+ { kind: "native", facet: "assembly", offset: 0, limit: 500 },
+])(
+ "accepts caller-selected page sizes and rejects malformed bounds: $kind",
+ (view) => {
+ const input = {
+ source: {
+ kind: "retained-evidence",
+ evidence_id: `ev_${"a".repeat(64)}`,
+ },
+ view,
+ };
+ expect(inspectAnalysisViewInputSchema.safeParse(input).success).toBe(true);
+ for (const limit of [0, -1, 0.5, Infinity, Number.MAX_SAFE_INTEGER + 1])
+ expect(
+ inspectAnalysisViewInputSchema.safeParse({
+ ...input,
+ view: { ...input.view, limit },
+ }).success,
+ ).toBe(false);
+ },
+);
+
it("projects layout summary, facet, item, and stable pages", () => {
const parent = layoutParent();
const layout = analysisViewLayoutFixture();
diff --git a/src/domain/analysisView/analysisView.ts b/src/domain/analysisView/analysisView.ts
index efb9f5d44b..b31849b19c 100644
--- a/src/domain/analysisView/analysisView.ts
+++ b/src/domain/analysisView/analysisView.ts
@@ -12,6 +12,7 @@ import { jsonObjectSchema, jsonValueSchema } from "../jsonValue.js";
import { err, ok, type Result } from "../result.js";
import type { AnalysisError } from "../analysisErrorBase.js";
import { binaryLayoutSchema } from "../native/binaryLayout.js";
+import { functionDossierSchema } from "../hopperValues.js";
import {
parseOwnedJavaScriptApplicationAnalysisSteps,
type JavaScriptApplicationAnalysisResult,
@@ -19,6 +20,7 @@ import {
import { analysisInputErrorFromIssues } from "../inputIssueProjection.js";
import { projectBinaryLayoutView } from "./binaryLayoutView.js";
import { projectJavaScriptApplicationView } from "./javascriptApplicationView.js";
+import { projectNativeFunctionView } from "./nativeFunctionView.js";
/** Public name of the selected-view workflow. */
export const INSPECT_ANALYSIS_VIEW_OPERATION = "inspect_analysis_view" as const;
@@ -73,12 +75,41 @@ const pageViewSchema = z.strictObject({
limit: z.number().int().positive(),
});
+/** Bounded native function facets of already authenticated analyze_function Evidence. */
+const nativeViewSchema = z.strictObject({
+ kind: z.literal("native"),
+ facet: z.enum([
+ "procedure",
+ "pseudocode",
+ "assembly",
+ "basic_blocks",
+ "comments",
+ "callers",
+ "callees",
+ "incoming_references",
+ "outgoing_references",
+ "unresolved_calls",
+ "referenced_strings",
+ "referenced_names",
+ "native_api",
+ "value_flow_summary",
+ "value_flow_operations",
+ "value_flow_def_use",
+ "value_flow_effects",
+ "value_flow_parameters",
+ "value_flow_parameter_uses",
+ ]),
+ offset: z.number().int().nonnegative().default(0),
+ limit: z.number().int().positive().default(64),
+});
+
/** Caller-selected projection of already completed analysis Evidence. */
export const analysisViewRequestSchema = z.discriminatedUnion("kind", [
summaryViewSchema,
facetViewSchema,
itemViewSchema,
pageViewSchema,
+ nativeViewSchema,
]);
/** Inline Evidence or an exact same-session retained reference. */
@@ -108,7 +139,9 @@ const coverageSchema = z.strictObject({
});
const artifactSchema = z.strictObject({
- path: z.string().min(1),
+ path: z
+ .string()
+ .describe("Exact recorded artifact path, including an empty string."),
sha256: digestSchema,
});
@@ -150,6 +183,7 @@ const viewResultBase = {
parent_operation: z.enum([
"inspect_binary_layout",
"analyze_javascript_application",
+ "analyze_function",
]),
parent_digest: digestSchema,
view_digest: digestSchema,
@@ -185,6 +219,13 @@ export const analysisViewResultSchema = z.discriminatedUnion("kind", [
items: z.array(jsonValueSchema),
...viewResultBase,
}),
+ z.strictObject({
+ kind: z.literal("native"),
+ view: nativeViewSchema,
+ procedure_address: z.string().nullable(),
+ item: jsonValueSchema,
+ ...viewResultBase,
+ }),
]);
export type AnalysisViewRequest = z.output;
@@ -200,6 +241,7 @@ export interface AnalysisViewParent {
readonly operation: string;
readonly normalizedResult: unknown;
readonly limitations: readonly string[];
+ readonly artifact?: { readonly path: string; readonly sha256: string } | null;
}
/** Construct a typed input failure for one selected-view constraint. */
@@ -216,7 +258,11 @@ export type UnsignedAnalysisView =
>
| Omit, "view_digest">
| Omit, "view_digest">
- | Omit, "view_digest">;
+ | Omit, "view_digest">
+ | Omit<
+ Extract,
+ "view_digest"
+ >;
/** Attach the content digest that must change when projected bytes change. */
export const sealAnalysisView = (
@@ -261,7 +307,7 @@ const unsupportedParent = (
new AnalysisUnsupportedTargetError(
INSPECT_ANALYSIS_VIEW_OPERATION,
parentArtifactPath(parent),
- `Parent Evidence operation ${JSON.stringify(parent.operation)} is not inspect_binary_layout or analyze_javascript_application.`,
+ `Parent Evidence operation ${JSON.stringify(parent.operation)} is not inspect_binary_layout, analyze_javascript_application or analyze_function.`,
);
const parentArtifactPath = (parent: AnalysisViewParent): string => {
@@ -327,5 +373,22 @@ export const projectAnalysisView = (
);
return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
}
+ if (parent.operation === "analyze_function") {
+ const native = functionDossierSchema.safeParse(parent.normalizedResult);
+ if (!native.success)
+ return err(
+ analysisInputErrorFromIssues(
+ INSPECT_ANALYSIS_VIEW_OPERATION,
+ native.error.issues.map((issue) => ({
+ ...issue,
+ path: ["source", "normalized_result", ...issue.path],
+ })),
+ { source: { normalized_result: parent.normalizedResult } },
+ { cause: native.error },
+ ),
+ );
+ const projected = projectNativeFunctionView(parent, native.data, view);
+ return projected.ok ? ok(sealAnalysisView(projected.value)) : projected;
+ }
return err(unsupportedParent(parent));
};
diff --git a/src/domain/analysisView/binaryLayoutView.ts b/src/domain/analysisView/binaryLayoutView.ts
index 5a52904a85..13a9b766f3 100644
--- a/src/domain/analysisView/binaryLayoutView.ts
+++ b/src/domain/analysisView/binaryLayoutView.ts
@@ -152,6 +152,10 @@ export const projectBinaryLayoutView = (
view: AnalysisViewRequest,
): Result => {
const shared = parentFields(parent, layout);
+ if (view.kind === "native")
+ return err(
+ layoutIncompatible("Native views require analyze_function Evidence."),
+ );
if (view.kind === "summary")
return ok({
kind: "summary",
diff --git a/src/domain/analysisView/javascriptApplicationView.ts b/src/domain/analysisView/javascriptApplicationView.ts
index 5216c72b44..d6e994fc94 100644
--- a/src/domain/analysisView/javascriptApplicationView.ts
+++ b/src/domain/analysisView/javascriptApplicationView.ts
@@ -243,6 +243,10 @@ export const projectJavaScriptApplicationView = (
): Result => {
const shared = parentFields(parent, analysis);
const modules = moduleEntries(analysis.graph.nodes);
+ if (view.kind === "native")
+ return err(
+ jsIncompatible("Native views require analyze_function Evidence."),
+ );
if (view.kind === "summary")
return ok({
kind: "summary",
diff --git a/src/domain/analysisView/nativeFunctionView.ts b/src/domain/analysisView/nativeFunctionView.ts
new file mode 100644
index 0000000000..511453763b
--- /dev/null
+++ b/src/domain/analysisView/nativeFunctionView.ts
@@ -0,0 +1,222 @@
+import type { AnalysisError } from "../analysisErrorBase.js";
+import { uniqueSorted } from "../canonicalOrdering.js";
+import type { FunctionDossier } from "../hopperValues.js";
+import { jsonValueSchema } from "../jsonValue.js";
+import { err, ok, type Result } from "../result.js";
+import {
+ analysisViewInputError,
+ type AnalysisViewCoverage,
+ type AnalysisViewParent,
+ type AnalysisViewRequest,
+ type UnsignedAnalysisView,
+} from "./analysisView.js";
+import {
+ completeWithinViewCoverage,
+ pageViewCoverage,
+} from "./analysisViewCoverage.js";
+
+type Artifact = { readonly path: string; readonly sha256: string };
+
+const viewError = (message: string): AnalysisError =>
+ analysisViewInputError([
+ { path: ["view"], reason: "invalid_value", message },
+ ]);
+
+const parentFields = (
+ parent: AnalysisViewParent,
+ dossier: FunctionDossier,
+ artifact: Artifact,
+) => {
+ const flow = dossier.native_value_flow;
+ const api = dossier.native_api;
+ const unknowns: string[] = [];
+ if (flow === null) unknowns.push("native_value_flow was not recorded");
+ else if (!flow.available)
+ unknowns.push("native_value_flow unavailable: " + flow.reason);
+ else if (flow.truncated)
+ unknowns.push(
+ "native_value_flow is truncated; omitted observations remain unknown",
+ );
+ if (api === null) unknowns.push("native_api was not recorded");
+ else if (!api.available)
+ unknowns.push(
+ "native_api unavailable: " + api.reason,
+ ...api.residual_unknowns,
+ );
+ return {
+ parent_evidence_id: parent.evidenceId,
+ parent_operation: "analyze_function" as const,
+ parent_digest: parent.evidenceId.slice(3),
+ procedure_address: dossier.procedure.address,
+ artifact,
+ limitations: uniqueSorted([
+ ...parent.limitations,
+ ...dossier.limitations,
+ ...(flow?.limitations ?? []),
+ ...(api?.available ? api.limitations : []),
+ ]),
+ unknowns: uniqueSorted(unknowns),
+ };
+};
+
+const pseudocodePage = (
+ body: string,
+ offset: number,
+ limit: number,
+): Result<
+ {
+ readonly item: { readonly text: string; readonly unit: "utf16-code-units" };
+ readonly coverage: AnalysisViewCoverage;
+ },
+ AnalysisError
+> => {
+ const start = Math.min(offset, body.length);
+ const high = (n: number) => n >= 0xd800 && n <= 0xdbff;
+ const low = (n: number) => n >= 0xdc00 && n <= 0xdfff;
+ if (
+ start > 0 &&
+ start < body.length &&
+ high(body.charCodeAt(start - 1)) &&
+ low(body.charCodeAt(start))
+ )
+ return err(viewError("Pseudocode offset splits a UTF-16 surrogate pair."));
+ let end = Math.min(start + limit, body.length);
+ if (
+ end > start &&
+ end < body.length &&
+ high(body.charCodeAt(end - 1)) &&
+ low(body.charCodeAt(end))
+ )
+ end--;
+ if (end === start && start < body.length)
+ return err(
+ viewError(
+ "Increase pseudocode limit to include the next complete Unicode character.",
+ ),
+ );
+ return ok({
+ item: { text: body.slice(start, end), unit: "utf16-code-units" },
+ coverage: pageViewCoverage(start, end - start, body.length),
+ });
+};
+
+const nativeRows = (
+ dossier: FunctionDossier,
+ facet: Exclude<
+ Extract["facet"],
+ "procedure" | "pseudocode" | "native_api" | "value_flow_summary"
+ >,
+): readonly unknown[] | undefined => {
+ const sourceFlow = dossier.native_value_flow;
+ const flow = sourceFlow?.available ? sourceFlow : undefined;
+ return {
+ assembly: dossier.assembly,
+ basic_blocks: dossier.basic_blocks,
+ comments: dossier.comments,
+ callers: dossier.callers,
+ callees: dossier.callees,
+ incoming_references: dossier.incoming_references,
+ outgoing_references: dossier.outgoing_references,
+ unresolved_calls: dossier.unresolved_calls,
+ referenced_strings: dossier.referenced_strings,
+ referenced_names: dossier.referenced_names,
+ value_flow_operations: flow?.operations,
+ value_flow_def_use: flow?.def_use,
+ value_flow_effects: flow?.effects,
+ value_flow_parameters: flow?.parameters,
+ value_flow_parameter_uses: flow?.parameter_uses,
+ }[facet];
+};
+
+/** Project bounded native function facets from complete retained Evidence without reanalysis. */
+export const projectNativeFunctionView = (
+ parent: AnalysisViewParent,
+ dossier: FunctionDossier,
+ view: AnalysisViewRequest,
+): Result => {
+ if (view.kind !== "native")
+ return err(viewError("analyze_function Evidence requires a native view."));
+ const artifact = parent.artifact;
+ if (!artifact)
+ return err(
+ analysisViewInputError([
+ {
+ path: ["source", "subject"],
+ reason: "invalid_value",
+ message:
+ "Native function Evidence requires an artifact-bound subject.",
+ },
+ ]),
+ );
+ const common = parentFields(parent, dossier, artifact);
+ const singleton = (item: unknown): UnsignedAnalysisView => ({
+ kind: "native",
+ view,
+ item: jsonValueSchema.parse(item),
+ coverage: completeWithinViewCoverage(1, 1),
+ ...common,
+ });
+ const flow = dossier.native_value_flow;
+ if (
+ view.facet === "procedure" ||
+ view.facet === "native_api" ||
+ view.facet === "value_flow_summary"
+ ) {
+ if (view.offset !== 0)
+ return err(viewError("Singleton native facets require offset 0."));
+ if (view.facet === "procedure") return ok(singleton(dossier.procedure));
+ if (view.facet === "native_api")
+ return ok(
+ singleton(
+ dossier.native_api ?? { available: false, reason: "not recorded" },
+ ),
+ );
+ const status =
+ flow === null
+ ? { available: false, reason: "not recorded" }
+ : !flow.available
+ ? { available: false, reason: flow.reason }
+ : {
+ available: true,
+ provenance: flow.provenance,
+ truncated: flow.truncated,
+ omitted_operations_lower_bound:
+ flow.omitted_operations_lower_bound,
+ known_omitted_inputs: flow.known_omitted_inputs,
+ known_omitted_edges: flow.known_omitted_edges,
+ operations: flow.operations.length,
+ def_use: flow.def_use.length,
+ effects: flow.effects.length,
+ parameters: flow.parameters.length,
+ parameter_uses: flow.parameter_uses.length,
+ };
+ return ok(singleton(status));
+ }
+ if (view.facet === "pseudocode") {
+ const page = pseudocodePage(dossier.pseudocode, view.offset, view.limit);
+ return page.ok
+ ? ok({ kind: "native", view, ...page.value, ...common })
+ : page;
+ }
+ const rows = nativeRows(dossier, view.facet);
+ if (rows === undefined)
+ return ok(
+ singleton({
+ available: false,
+ // Only value-flow facets inherit the value-flow failure reason; an
+ // older dossier simply has no unresolved_calls record.
+ reason:
+ view.facet.startsWith("value_flow_") && flow?.available === false
+ ? flow.reason
+ : "not recorded",
+ }),
+ );
+ const items = rows.slice(view.offset, view.offset + view.limit);
+ return ok({
+ kind: "native",
+ view,
+ item: jsonValueSchema.parse(items),
+ coverage: pageViewCoverage(view.offset, items.length, rows.length),
+ ...common,
+ });
+};
diff --git a/src/domain/android/androidApplication.ts b/src/domain/android/androidApplication.ts
index 745b8e0c49..6744257490 100644
--- a/src/domain/android/androidApplication.ts
+++ b/src/domain/android/androidApplication.ts
@@ -125,6 +125,8 @@ export const projectAndroidApplication = (
"Manifest, resource, signing, and bytecode semantics require a dedicated Android provider; this projection reports exact inventory paths and hashes only.",
"Runtime families are inferred from inventory formats and paths; filename suffixes do not establish valid DEX or JVM class bytes.",
"Bridge candidates are path-based hypotheses, not decoded JNI declarations or observed runtime calls.",
+ "A bridge basis is inferred from the native path and is repeated for every managed component.",
+ "ZIP inventory cannot see the APK Signing Block, so an empty signing array does not mean the APK is unsigned.",
...(bridgeProjection.coverage.status === "partial"
? [
`Bridge candidate pairs exceeded the projection safety budget; ${bridgeProjection.coverage.omitted_candidates} hypotheses are omitted. Component arrays still include every component from the supplied inventory pages.`,
@@ -157,23 +159,12 @@ export const projectAndroidApplication = (
};
const classify = (all: readonly Component[]) => ({
- manifests: all.filter(
- ({ path, format }) =>
- format === "android-manifest" ||
- /(?:^|\/)AndroidManifest\.xml$/u.test(path),
- ),
- resources: all.filter(
- ({ path, format }) =>
- format === "android-resources" || /(?:^|\/)resources\.arsc$/u.test(path),
- ),
- dex: all.filter(
- ({ path, format }) =>
- format === "dex" || /(?:^|\/)[^/]+\.dex$/iu.test(path),
- ),
- jvm_classes: all.filter(
- ({ path, format }) =>
- format === "jvm-class" || /(?:^|\/)[^/]+\.class$/iu.test(path),
+ manifests: all.filter(({ path }) =>
+ /(?:^|\/)AndroidManifest\.xml$/u.test(path),
),
+ resources: all.filter(({ path }) => /(?:^|\/)resources\.arsc$/u.test(path)),
+ dex: all.filter(({ path }) => /(?:^|\/)[^/]+\.dex$/iu.test(path)),
+ jvm_classes: all.filter(({ path }) => /(?:^|\/)[^/]+\.class$/iu.test(path)),
native_libraries: all.filter(
({ path, format }) =>
format === "elf" || /(?:^|\/)lib\/[^/]+\/[^/]+\.so$/iu.test(path),
@@ -189,19 +180,9 @@ const runtimeFamilies = (all: readonly Component[]) => {
const families = new Set<
AndroidApplicationProjectionResult["runtime_families"][number]
>();
- if (
- all.some(
- ({ path, format }) =>
- format === "dex" || path.toLowerCase().endsWith(".dex"),
- )
- )
+ if (all.some(({ path }) => path.toLowerCase().endsWith(".dex")))
families.add("dalvik-art");
- if (
- all.some(
- ({ path, format }) =>
- format === "jvm-class" || path.toLowerCase().endsWith(".class"),
- )
- )
+ if (all.some(({ path }) => path.toLowerCase().endsWith(".class")))
families.add("java-kotlin");
if (all.some(({ format }) => format === "elf")) families.add("native");
if (all.some(({ format }) => format === "javascript-bundle"))
diff --git a/src/domain/apple/interfaceBuilderGraph.test.ts b/src/domain/apple/interfaceBuilderGraph.test.ts
index 13d6f79fb9..bfa2c71de3 100644
--- a/src/domain/apple/interfaceBuilderGraph.test.ts
+++ b/src/domain/apple/interfaceBuilderGraph.test.ts
@@ -509,3 +509,75 @@ describe("AppKit keyed archive connectors", () => {
expect(to).toEqual(expect.arrayContaining(["objc_selector", "unknown"]));
});
});
+
+it("names the archive once per evidence record", () => {
+ const relativePath = "Views/Main.storyboardc/scene.nib/objects.nib";
+ const result = buildInterfaceBuilderAnalysis({
+ targetSha256: hash,
+ toolVersion: "test",
+ documents: [
+ {
+ relativePath,
+ archiveSha256: hash,
+ documentKind: "storyboard_scene",
+ raw: {
+ "com.apple.ibtool.document.objects": {
+ controller: { customClass: "StoreViewController" },
+ next: { customClass: "BuildViewController" },
+ },
+ "com.apple.ibtool.document.connections": {
+ controller: [
+ { type: "segue", identifier: "showBuild", destinationId: "next" },
+ ],
+ },
+ },
+ },
+ ],
+ limits: interfaceBuilderLimitsSchema.parse({}),
+ });
+ const evidence = [...result.graph.nodes, ...result.graph.edges].flatMap(
+ (item) => item.evidence,
+ );
+ expect(new Set(evidence.map(({ artifact_path }) => artifact_path))).toEqual(
+ new Set([relativePath]),
+ );
+ expect(evidence.map(({ description }) => description)).toEqual(
+ expect.arrayContaining([
+ "compiled document",
+ "object controller",
+ "segue showBuild",
+ ]),
+ );
+ expect(
+ evidence.filter(({ description }) => description.includes(relativePath)),
+ ).toEqual([]);
+});
+
+it("states each object's class, label, and ID once", () => {
+ const { objects } = parseInterfaceBuilderRecords({
+ "com.apple.ibtool.document.objects": {
+ "7": {
+ customClass: "NSButton",
+ label: "Start",
+ objectID: "7",
+ nibValues: { NSContents: "Start" },
+ },
+ button: { class: "UIButton", title: "Build", objectID: "authored" },
+ },
+ });
+ expect(objects).toEqual([
+ expect.objectContaining({
+ id: "7",
+ class_name: "NSButton",
+ name: "Start",
+ attributes: { nibValues: { NSContents: "Start" } },
+ }),
+ expect.objectContaining({
+ id: "button",
+ class_name: "UIButton",
+ name: "Build",
+ // Values that differ from the canonical fields stay as authored.
+ attributes: { class: "UIButton", title: "Build", objectID: "authored" },
+ }),
+ ]);
+});
diff --git a/src/domain/apple/interfaceBuilderGraph.ts b/src/domain/apple/interfaceBuilderGraph.ts
index 17e94d584f..bd35130a85 100644
--- a/src/domain/apple/interfaceBuilderGraph.ts
+++ b/src/domain/apple/interfaceBuilderGraph.ts
@@ -72,7 +72,9 @@ export const buildInterfaceBuilderAnalysis = (input: {
const evidenceFor = (description: string) => [
{
kind: "interface_builder_resource" as const,
- description: `${document.relativePath}: ${description}`,
+ // artifact_path names the archive; repeating it here cost a path per
+ // node and edge.
+ description,
location: { address: null, file_offset: null },
artifact_path: document.relativePath,
artifact_sha256: document.archiveSha256,
diff --git a/src/domain/apple/interfaceBuilderKeyedArchive.ts b/src/domain/apple/interfaceBuilderKeyedArchive.ts
index 51b446b52b..c0b7f35077 100644
--- a/src/domain/apple/interfaceBuilderKeyedArchive.ts
+++ b/src/domain/apple/interfaceBuilderKeyedArchive.ts
@@ -71,13 +71,23 @@ export const parseInterfaceBuilderRecords = (
attributes["name"],
className,
);
+ // class_name, name, and id already carry these values; repeating them in
+ // attributes doubled them on every graph node.
+ const carried = new Set();
+ if (attributes["customClass"] === className) carried.add("customClass");
+ if (attributes["label"] === label) carried.add("label");
+ if (attributes["objectID"] === id) carried.add("objectID");
objects.push(
objectNode.parse({
id,
kind: classifyObject(className),
class_name: className,
name: label ?? id,
- attributes: jsonSafeRecord(attributes),
+ attributes: jsonSafeRecord(
+ Object.fromEntries(
+ Object.entries(attributes).filter(([key]) => !carried.has(key)),
+ ),
+ ),
}),
);
}
diff --git a/src/domain/artifactComparison.ts b/src/domain/artifactComparison.ts
index 3a11011823..04bc33ec9f 100644
--- a/src/domain/artifactComparison.ts
+++ b/src/domain/artifactComparison.ts
@@ -350,19 +350,20 @@ const relationsByPath = (
inventory.occurrences.map((item) => [item.occurrence_id, item]),
);
for (const edge of inventory.edges) {
- const path = edge.logical_path ?? ".";
- const values = output.get(path);
const occurrence = occurrences.get(edge.occurrence_id);
+ // An incomplete inventory can omit the occurrence that names the child.
+ if (occurrence?.artifact_id === null || occurrence === undefined) continue;
+ const path = occurrence.logical_path;
+ const values = output.get(path);
const projection: RelationProjection = {
parent_logical_path:
- occurrence?.parent_occurrence_id === null ||
- occurrence?.parent_occurrence_id === undefined
+ occurrence.parent_occurrence_id === null
? null
: (occurrences.get(occurrence.parent_occurrence_id)?.logical_path ??
null),
- child_artifact_id: edge.child_artifact_id,
+ child_artifact_id: occurrence.artifact_id,
relation: edge.relation,
- logical_path: edge.logical_path,
+ logical_path: occurrence.logical_path,
producer: edge.producer,
};
if (values === undefined) output.set(path, [projection]);
diff --git a/src/domain/artifactGraph.ts b/src/domain/artifactGraph.ts
index 34e4af9323..02f612a224 100644
--- a/src/domain/artifactGraph.ts
+++ b/src/domain/artifactGraph.ts
@@ -151,12 +151,14 @@ export const integrityContradictionSchema = z.object({
limitations: z.array(z.string()).min(1),
});
-/** Parent-child derivation relation between two content-addressed artifacts. */
+/**
+ * Parent-child derivation relation between two content-addressed artifacts.
+ * The child artifact and its logical path are those of the edge's occurrence.
+ */
const artifactEdgeSchema = z.object({
edge_id: edgeIdSchema,
ordinal: z.number().int().min(0),
parent_artifact_id: artifactIdSchema,
- child_artifact_id: artifactIdSchema,
relation: z.enum([
"contains",
"extracts",
@@ -167,7 +169,6 @@ const artifactEdgeSchema = z.object({
"derived-from",
]),
occurrence_id: occurrenceIdSchema,
- logical_path: boundedRelativePathSchema.nullable(),
producer: artifactCommandSchema.nullable(),
});
diff --git a/src/domain/artifactIdentity.ts b/src/domain/artifactIdentity.ts
index ad773de332..3c46a3b975 100644
--- a/src/domain/artifactIdentity.ts
+++ b/src/domain/artifactIdentity.ts
@@ -71,14 +71,21 @@ export const artifactContradictionId = (input: {
"Artifact",
)}`;
-/** Identify an edge by its semantic relationship and exact occurrence. */
+/**
+ * Identify an edge by its semantic relationship and exact occurrence. The
+ * child artifact and logical path come from that occurrence.
+ */
export const artifactEdgeId = (
- semantic: Pick<
- ArtifactEdge,
- | "parent_artifact_id"
- | "child_artifact_id"
- | "relation"
- | "occurrence_id"
- | "logical_path"
- >,
-): string => `edge_${digestCanonicalValue(semantic, "Artifact")}`;
+ edge: Pick,
+ occurrence: Pick,
+): string =>
+ `edge_${digestCanonicalValue(
+ {
+ parent_artifact_id: edge.parent_artifact_id,
+ child_artifact_id: occurrence.artifact_id,
+ relation: edge.relation,
+ occurrence_id: edge.occurrence_id,
+ logical_path: occurrence.logical_path,
+ },
+ "Artifact",
+ )}`;
diff --git a/src/domain/artifactInspection.ts b/src/domain/artifactInspection.ts
index f4c197b9c5..19ccb9b8f9 100644
--- a/src/domain/artifactInspection.ts
+++ b/src/domain/artifactInspection.ts
@@ -16,30 +16,12 @@ const artifactIdSchema = prefixedDigestSchema("art");
const inspectionObservationSchema = z.strictObject({
observation_id: prefixedDigestSchema("aio"),
- kind: z.enum(["root-manifest", "artifact", "occurrence", "integrity"]),
+ kind: z.enum(["root-manifest", "integrity"]),
subject: textSchema,
value: jsonValueSchema,
evidence_id: evidenceIdSchema,
});
-const inspectionRelationshipSchema = z.strictObject({
- relationship_id: prefixedDigestSchema("air"),
- relation: z.enum([
- "contains",
- "extracts",
- "slice-of",
- "embeds",
- "loads",
- "maps-source",
- "derived-from",
- ]),
- source_artifact_id: artifactIdSchema,
- target_artifact_id: artifactIdSchema,
- occurrence_id: prefixedDigestSchema("occ"),
- logical_path: textSchema.nullable(),
- evidence_id: evidenceIdSchema,
-});
-
const inspectionHypothesisSchema = z.strictObject({
hypothesis_id: prefixedDigestSchema("aih"),
statement: textSchema,
@@ -97,7 +79,6 @@ export const artifactInspectionResultSchema = z.strictObject({
)
.min(1),
observations: z.array(inspectionObservationSchema),
- derived_relationships: z.array(inspectionRelationshipSchema),
hypotheses: z.array(inspectionHypothesisSchema),
contradictions: z.array(inspectionContradictionSchema),
unexplored_branches: z.array(unexploredBranchSchema),
@@ -107,8 +88,6 @@ export const artifactInspectionResultSchema = z.strictObject({
substeps_completed: z.literal(1),
observations_retained: z.number().int().min(0),
observations_omitted: z.number().int().min(0),
- relationships_retained: z.number().int().min(0),
- relationships_omitted: z.number().int().min(0),
hypotheses_retained: z.number().int().min(0),
hypotheses_omitted: z.number().int().min(0),
unexplored_branches_retained: z.number().int().min(0),
@@ -136,10 +115,6 @@ export const createArtifactInspection = (
inventory,
inventoryEvidence.evidence_id,
);
- const relationships = allRelationships(
- inventory,
- inventoryEvidence.evidence_id,
- );
const hypotheses = allHypotheses(inventory, inventoryEvidence.evidence_id);
const nextProbes = probesFor(inventory);
const branches = allUnexploredBranches(
@@ -149,14 +124,12 @@ export const createArtifactInspection = (
);
const retained = {
observations,
- relationships,
hypotheses,
branches,
probes: nextProbes,
};
const omissions = {
observations: 0,
- relationships: 0,
hypotheses: 0,
branches: 0,
probes: 0,
@@ -184,7 +157,6 @@ export const createArtifactInspection = (
},
],
observations: retained.observations,
- derived_relationships: retained.relationships,
hypotheses: retained.hypotheses,
contradictions: contradictions(inventory, inventoryEvidence.evidence_id),
unexplored_branches: retained.branches,
@@ -194,8 +166,6 @@ export const createArtifactInspection = (
substeps_completed: 1 as const,
observations_retained: retained.observations.length,
observations_omitted: omissions.observations,
- relationships_retained: retained.relationships.length,
- relationships_omitted: omissions.relationships,
hypotheses_retained: retained.hypotheses.length,
hypotheses_omitted: omissions.hypotheses,
unexplored_branches_retained: retained.branches.length,
@@ -224,6 +194,8 @@ const parseInventoryEvidence = (input: unknown): Evidence => {
return evidence;
};
+// Artifact, occurrence, and edge records stay in the nested inventory Evidence;
+// restating each one here would repeat the complete graph in every result.
const allObservations = (
inventory: ArtifactInventoryResult,
evidenceId: string,
@@ -234,12 +206,6 @@ const allObservations = (
inventory.manifest,
evidenceId,
),
- ...inventory.nodes.map((node) =>
- observation("artifact", node.artifact_id, node, evidenceId),
- ),
- ...inventory.occurrences.map((occurrence) =>
- observation("occurrence", occurrence.occurrence_id, occurrence, evidenceId),
- ),
...inventory.integrity_contradictions.map((value) =>
observation("integrity", value.contradiction_id, value, evidenceId),
),
@@ -263,25 +229,6 @@ const observation = (
};
};
-const allRelationships = (
- inventory: ArtifactInventoryResult,
- evidenceId: string,
-): ArtifactInspectionResult["derived_relationships"] =>
- inventory.edges.map((edge) => {
- const semantic = {
- relation: edge.relation,
- source_artifact_id: edge.parent_artifact_id,
- target_artifact_id: edge.child_artifact_id,
- occurrence_id: edge.occurrence_id,
- logical_path: edge.logical_path,
- evidence_id: evidenceId,
- };
- return {
- relationship_id: `air_${digestCanonicalValue(semantic, "Artifact inspection")}`,
- ...semantic,
- };
- });
-
const allHypotheses = (
inventory: ArtifactInventoryResult,
evidenceId: string,
diff --git a/src/domain/artifactInventoryEvidence.ts b/src/domain/artifactInventoryEvidence.ts
index a1c74794ac..33491c7a77 100644
--- a/src/domain/artifactInventoryEvidence.ts
+++ b/src/domain/artifactInventoryEvidence.ts
@@ -211,8 +211,14 @@ const validateCompleteInventory = (inventory: InventorySet): void => {
);
for (const occurrence of inventory.occurrences)
validateOccurrence(occurrence, inventory, nodeIds, occurrenceIds);
+ const occurrencesById = new Map(
+ inventory.occurrences.map((occurrence) => [
+ occurrence.occurrence_id,
+ occurrence,
+ ]),
+ );
for (const edge of inventory.edges)
- validateEdge(edge, nodeIds, occurrenceIds);
+ validateEdge(edge, nodeIds, occurrencesById);
for (const contradiction of inventory.integrityContradictions) {
if (
!nodeIds.has(contradiction.parent_artifact_id) ||
@@ -273,22 +279,17 @@ const validateOccurrence = (
const validateEdge = (
edge: ArtifactInventoryResult["edges"][number],
nodeIds: ReadonlySet,
- occurrenceIds: ReadonlySet,
+ occurrences: ReadonlyMap,
): void => {
+ const occurrence = occurrences.get(edge.occurrence_id);
if (
!nodeIds.has(edge.parent_artifact_id) ||
- !nodeIds.has(edge.child_artifact_id) ||
- !occurrenceIds.has(edge.occurrence_id)
+ occurrence?.artifact_id === null ||
+ occurrence === undefined ||
+ !nodeIds.has(occurrence.artifact_id)
)
throw new TypeError("Artifact edge references a missing graph member");
- const semantic = {
- parent_artifact_id: edge.parent_artifact_id,
- child_artifact_id: edge.child_artifact_id,
- relation: edge.relation,
- occurrence_id: edge.occurrence_id,
- logical_path: edge.logical_path,
- };
- if (edge.edge_id !== artifactEdgeId(semantic))
+ if (edge.edge_id !== artifactEdgeId(edge, occurrence))
throw new TypeError("Artifact edge ID does not match its identity");
};
diff --git a/src/domain/compactStringIdentity.ts b/src/domain/compactStringIdentity.ts
new file mode 100644
index 0000000000..421555a908
--- /dev/null
+++ b/src/domain/compactStringIdentity.ts
@@ -0,0 +1,10 @@
+import { digestCanonicalValue } from "./canonicalDigest.js";
+
+/** Retain short exact string keys or commit their value in a reserved digest namespace. */
+export const compactStringIdentityKey = (value: string): string => {
+ const prefix = "value-sha256:";
+ const digestKey = `${prefix}${digestCanonicalValue(value, "String identity")}`;
+ return value.length <= digestKey.length && !value.startsWith(prefix)
+ ? value
+ : digestKey;
+};
diff --git a/src/domain/inputIssueProjection.ts b/src/domain/inputIssueProjection.ts
index 25798721c9..1d5db83b65 100644
--- a/src/domain/inputIssueProjection.ts
+++ b/src/domain/inputIssueProjection.ts
@@ -35,7 +35,23 @@ const projectIssue = (
typeof part === "string" || typeof part === "number" ? [part] : [],
);
// Branch issue paths are relative to the union; branches often agree.
- if (issue.code === "invalid_union")
+ if (issue.code === "invalid_union") {
+ // Unmatched discriminators and ambiguous unions have no branch issues.
+ // Keep their outer failure instead of flattening it into an empty list.
+ if (issue.errors.length === 0) {
+ const expected =
+ "options" in issue ? issue.options?.filter(isSafeExpected) : undefined;
+ return [
+ {
+ path,
+ reason:
+ valueAtPath(input, path) === undefined
+ ? "missing_argument"
+ : "invalid_value",
+ ...(expected === undefined ? {} : { expected }),
+ },
+ ];
+ }
return uniqueIssues(
issue.errors.flatMap((branch) =>
branch.flatMap((branchIssue) =>
@@ -46,6 +62,7 @@ const projectIssue = (
),
),
);
+ }
if (issue.code === "invalid_key")
return issue.issues.flatMap((keyIssue) =>
projectIssue(
diff --git a/src/domain/javascript/electronStaticAnalysisBrowser.ts b/src/domain/javascript/electronStaticAnalysisBrowser.ts
index f29e7cf8f4..a44baf89a7 100644
--- a/src/domain/javascript/electronStaticAnalysisBrowser.ts
+++ b/src/domain/javascript/electronStaticAnalysisBrowser.ts
@@ -12,6 +12,7 @@ import type {
import {
boundedExpression,
collectContextBridgeMembers,
+ electronCalleeName,
electronStaticValue,
objectProperty,
} from "./electronStaticAnalysisValues.js";
@@ -19,7 +20,6 @@ import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js";
import { addLocatedFinding } from "./javascriptStaticAnalysisFindings.js";
import {
argumentNode,
- calleeName,
range,
staticPath,
staticPathResolutionContext,
@@ -42,7 +42,7 @@ const inspectBrowserWindow = (
node: t.NewExpression,
context: JavaScriptFindingContext,
): void => {
- const name = calleeName(node.callee);
+ const name = electronCalleeName(node.callee, context);
if (name !== "BrowserWindow" && !name.endsWith(".BrowserWindow")) return;
const options = argumentNode(node.arguments[0]);
const collected = collectWindowOptions(context.source, options);
@@ -165,7 +165,7 @@ const inspectContextBridge = (
node: t.CallExpression,
context: JavaScriptFindingContext,
): void => {
- const name = calleeName(node.callee);
+ const name = electronCalleeName(node.callee, context);
const main =
name === "contextBridge.exposeInMainWorld" ||
name.endsWith(".contextBridge.exposeInMainWorld");
@@ -213,7 +213,7 @@ const inspectUtilityProcess = (
node: t.CallExpression,
context: JavaScriptFindingContext,
): void => {
- const name = calleeName(node.callee);
+ const name = electronCalleeName(node.callee, context);
if (name !== "utilityProcess.fork" && !name.endsWith(".utilityProcess.fork"))
return;
const moduleNode = argumentNode(node.arguments[0]);
diff --git a/src/domain/javascript/electronStaticAnalysisIpc.ts b/src/domain/javascript/electronStaticAnalysisIpc.ts
index 9efc2670b8..a560ea1844 100644
--- a/src/domain/javascript/electronStaticAnalysisIpc.ts
+++ b/src/domain/javascript/electronStaticAnalysisIpc.ts
@@ -7,6 +7,7 @@ import type {
} from "./electronStaticAnalysisTypes.js";
import {
boundedExpression,
+ electronCalleeName,
electronStaticValue,
handlerKind,
} from "./electronStaticAnalysisValues.js";
@@ -36,7 +37,7 @@ const inspectIpcCall = (
node: t.CallExpression,
context: JavaScriptFindingContext,
): void => {
- const name = calleeName(node.callee);
+ const name = electronCalleeName(node.callee, context);
const descriptor = ipcDescriptor(name);
if (descriptor === undefined) return;
const channelNode = argumentNode(node.arguments[0]);
diff --git a/src/domain/javascript/electronStaticAnalysisValues.ts b/src/domain/javascript/electronStaticAnalysisValues.ts
index 2a800e6bbd..c2bb6e76a2 100644
--- a/src/domain/javascript/electronStaticAnalysisValues.ts
+++ b/src/domain/javascript/electronStaticAnalysisValues.ts
@@ -6,7 +6,26 @@ import {
readExactJavaScriptLiteral,
semanticStaticPropertyName,
} from "./javascriptAstValues.js";
-import { sourceSlice } from "./javascriptStaticAnalysisHelpers.js";
+import { calleeName, sourceSlice } from "./javascriptStaticAnalysisHelpers.js";
+import type { JavaScriptFindingContext } from "./javascriptStaticAnalysisState.js";
+
+/** Name a callee, spelling a root renamed from Electron as its export. */
+export const electronCalleeName = (
+ callee: t.Node,
+ context: JavaScriptFindingContext,
+): string => {
+ const name = calleeName(callee);
+ let root = callee;
+ while (t.isMemberExpression(root) || t.isOptionalMemberExpression(root)) {
+ if (!t.isNode(root.object)) return name;
+ root = root.object;
+ }
+ if (!t.isIdentifier(root)) return name;
+ const exported = context.electronBindings?.get(root.start ?? -1);
+ return exported === undefined
+ ? name
+ : `${exported}${name.slice(root.name.length)}`;
+};
/** Preserve one literal value or the exact inert expression. */
export const electronStaticValue = (
diff --git a/src/domain/javascript/javascriptApplicationChangeGraph.test.ts b/src/domain/javascript/javascriptApplicationChangeGraph.test.ts
new file mode 100644
index 0000000000..e3c8e13108
--- /dev/null
+++ b/src/domain/javascript/javascriptApplicationChangeGraph.test.ts
@@ -0,0 +1,103 @@
+import { describe, expect, it } from "vitest";
+
+import {
+ createJavaScriptApplicationEdge,
+ createJavaScriptApplicationGraph,
+ type JavaScriptApplicationGraph,
+} from "./javascriptApplicationGraph.js";
+import {
+ artifactEvidence,
+ completeCoverage,
+ fixtureNode,
+} from "./javascriptApplicationGraph.fixture.js";
+import { compareJavaScriptApplicationVersions } from "./javascriptApplicationVersionComparison.js";
+
+const LEFT_SHA = "a".repeat(64);
+const RIGHT_SHA = "b".repeat(64);
+
+const digest = (value: number): string => value.toString(16).padStart(64, "0");
+
+const buildGraph = (
+ artifactSha: string,
+ changed: number,
+): JavaScriptApplicationGraph => {
+ const nodes = ["a", "b", "c", "d"].map((name, index) =>
+ fixtureNode(
+ "javascript-module",
+ {
+ strategy: "canonical-path",
+ stability: "artifact-version",
+ artifact_sha256: artifactSha,
+ path: `${name}.js`,
+ },
+ `${name}.js`,
+ {
+ structural_fingerprint_sha256: digest(index + 1),
+ structural_fingerprint_algorithm: "fixture",
+ source_sha256: digest(index === 1 || index === 3 ? changed : 0),
+ },
+ artifactEvidence(artifactSha, `${name}.js`),
+ ),
+ );
+ const [a, b] = nodes;
+ if (a === undefined || b === undefined)
+ throw new Error("Missing fixture node");
+ return createJavaScriptApplicationGraph({
+ schema: "JavaScriptApplicationGraph",
+ root_node_ids: [a.node_id],
+ nodes,
+ edges: [
+ createJavaScriptApplicationEdge({
+ source_node_id: a.node_id,
+ target_node_id: b.node_id,
+ relation: "imports",
+ properties: {},
+ evidence: artifactEvidence(artifactSha, "a.js"),
+ }),
+ ],
+ coverage: completeCoverage,
+ limitations: [],
+ });
+};
+
+describe("application change graph", () => {
+ it("anchors each changed_from edge to its own right-hand observation", () => {
+ const left = buildGraph(LEFT_SHA, 1);
+ const right = buildGraph(RIGHT_SHA, 2);
+ const result = compareJavaScriptApplicationVersions({
+ left: {
+ evidenceId: `ev_${"1".repeat(64)}`,
+ rootArtifactSha256: LEFT_SHA,
+ graph: left,
+ },
+ right: {
+ evidenceId: `ev_${"2".repeat(64)}`,
+ rootArtifactSha256: RIGHT_SHA,
+ graph: right,
+ },
+ leftNativeEvidence: [],
+ rightNativeEvidence: [],
+ });
+ const changedFrom = result.graph.edges.filter(
+ ({ relation }) => relation === "changed_from",
+ );
+ expect(changedFrom).toHaveLength(2);
+ for (const edge of changedFrom) {
+ const source = right.nodes.find(
+ ({ node_id: id }) => id === edge.source_node_id,
+ );
+ const target = left.nodes.find(
+ ({ node_id: id }) => id === edge.target_node_id,
+ );
+ expect(source?.observations[0]?.label).toBe(
+ target?.observations[0]?.label,
+ );
+ expect(edge.evidence.artifact).toEqual(
+ source?.observations[0]?.evidence.artifact,
+ );
+ expect(edge.evidence.location).toEqual(
+ source?.observations[0]?.evidence.location,
+ );
+ }
+ });
+});
diff --git a/src/domain/javascript/javascriptApplicationChangeGraph.ts b/src/domain/javascript/javascriptApplicationChangeGraph.ts
index 4c80b32045..134fff5200 100644
--- a/src/domain/javascript/javascriptApplicationChangeGraph.ts
+++ b/src/domain/javascript/javascriptApplicationChangeGraph.ts
@@ -53,8 +53,14 @@ export const buildJavaScriptApplicationChangeGraph = (
({ source_node_id: source, target_node_id: target }) =>
retained.has(source) && retained.has(target),
);
+ const leftById = new Map(
+ input.left.nodes.map((node) => [node.node_id, node]),
+ );
+ const rightById = new Map(
+ input.right.nodes.map((node) => [node.node_id, node]),
+ );
const comparisonEdges = input.items.flatMap((item) =>
- changedFromEdge(item, input, retained),
+ changedFromEdge(item, input, retained, leftById, rightById),
);
const candidateEdges = uniqueEdges([...sourceEdges, ...comparisonEdges]);
const rootNodeIds = preferredRoots.filter((nodeId) => retained.has(nodeId));
@@ -104,6 +110,8 @@ const changedFromEdge = (
item: ApplicationVersionComparisonItem,
input: ChangeGraphInput,
retained: ReadonlySet,
+ leftById: ReadonlyMap,
+ rightById: ReadonlyMap,
): ApplicationEdge[] => {
const left = item.left_node_id;
const right = item.right_node_id;
@@ -117,7 +125,12 @@ const changedFromEdge = (
!retained.has(right)
)
return [];
- const evidence = comparisonEvidence(item, input);
+ const evidence = comparisonEvidence(
+ item,
+ input,
+ leftById.get(left),
+ rightById.get(right),
+ );
return [
createJavaScriptApplicationEdge({
source_node_id: right,
@@ -138,13 +151,9 @@ const changedFromEdge = (
const comparisonEvidence = (
item: ApplicationVersionComparisonItem,
input: ChangeGraphInput,
+ left: ApplicationNode | undefined,
+ right: ApplicationNode | undefined,
): ApplicationGraphEvidence => {
- const right = input.right.nodes.find(
- ({ node_id: id }) => id === item.right_node_id,
- );
- const left = input.left.nodes.find(
- ({ node_id: id }) => id === item.left_node_id,
- );
const source =
right?.observations[0]?.evidence ?? left?.observations[0]?.evidence;
const complete =
diff --git a/src/domain/javascript/javascriptApplicationGraph.fixture.ts b/src/domain/javascript/javascriptApplicationGraph.fixture.ts
index 00f7c229ad..3002b822f3 100644
--- a/src/domain/javascript/javascriptApplicationGraph.fixture.ts
+++ b/src/domain/javascript/javascriptApplicationGraph.fixture.ts
@@ -109,7 +109,7 @@ type FixtureNodeArguments = [
evidence: ApplicationGraphEvidence,
];
-const fixtureNode = (
+export const fixtureNode = (
...[kind, identity, label, properties, evidence]: FixtureNodeArguments
): FixtureNode =>
createJavaScriptApplicationNode({
diff --git a/src/domain/javascript/javascriptApplicationGraph.test.ts b/src/domain/javascript/javascriptApplicationGraph.test.ts
index efedf78c06..58f1279284 100644
--- a/src/domain/javascript/javascriptApplicationGraph.test.ts
+++ b/src/domain/javascript/javascriptApplicationGraph.test.ts
@@ -547,6 +547,21 @@ describe("JavaScript Application Graph: source maps and observations", () => {
/Edge identifier/u,
);
});
+
+ it("creates exactly the graph the public schema accepts", () => {
+ const graph = buildSyntheticJavaScriptApplicationGraph();
+ expect(JSON.stringify(graph)).toBe(
+ JSON.stringify(
+ javascriptApplicationGraphSchema.parse(structuredClone(graph)),
+ ),
+ );
+
+ const { graph_id: _graphId, ...semantic } = structuredClone(graph);
+ firstOf(semantic.edges, "edge").edge_id = `jag_edge_${"f".repeat(64)}`;
+ expect(() => createJavaScriptApplicationGraph(semantic)).toThrow(
+ /Edge identifier/u,
+ );
+ });
});
describe("JavaScript Application Graph: duplicate and orphan rejection", () => {
diff --git a/src/domain/javascript/javascriptApplicationGraph.ts b/src/domain/javascript/javascriptApplicationGraph.ts
index ba122e1f06..e16aded759 100644
--- a/src/domain/javascript/javascriptApplicationGraph.ts
+++ b/src/domain/javascript/javascriptApplicationGraph.ts
@@ -304,8 +304,8 @@ const checkNode = (
type GraphRecord = z.infer;
-const checkGraphInvariants = (
- graph: GraphRecord,
+const checkGraphContent = (
+ graph: JavaScriptApplicationGraphInput,
context: z.RefinementCtx,
): void => {
sortedUniqueIssue(
@@ -394,7 +394,13 @@ const checkGraphInvariants = (
context,
);
}
+};
+const checkGraphInvariants = (
+ graph: GraphRecord,
+ context: z.RefinementCtx,
+): void => {
+ checkGraphContent(graph, context);
const { graph_id: identifier, ...semantic } = graph;
if (
identifier !==
@@ -411,6 +417,9 @@ const checkGraphInvariants = (
export const javascriptApplicationGraphSchema =
javascriptApplicationGraphRecordSchema.superRefine(checkGraphInvariants);
+const javascriptApplicationGraphContentSchema =
+ javascriptApplicationGraphInputSchema.superRefine(checkGraphContent);
+
/** Fully validated JavaScript Application Graph. */
export type JavaScriptApplicationGraph = z.infer<
typeof javascriptApplicationGraphSchema
@@ -482,10 +491,11 @@ export const createJavaScriptApplicationGraph = (
},
limitations: uniqueSorted(parsed.limitations),
};
- return javascriptApplicationGraphSchema.parse({
- ...semantic,
- graph_id: `jag_${digestCanonicalValue(semantic, "JavaScript Application Graph")}`,
- });
+ const content = javascriptApplicationGraphContentSchema.parse(semantic);
+ return {
+ ...content,
+ graph_id: `jag_${digestCanonicalValue(content, "JavaScript Application Graph")}`,
+ };
};
const validatedImmutableApplicationGraphs = new WeakSet();
diff --git a/src/domain/javascript/javascriptAstValues.ts b/src/domain/javascript/javascriptAstValues.ts
index f0a0528c06..5479c8d1f3 100644
--- a/src/domain/javascript/javascriptAstValues.ts
+++ b/src/domain/javascript/javascriptAstValues.ts
@@ -76,6 +76,16 @@ export const semanticStaticPropertyKey = (
return !computed && t.isIdentifier(property) ? property.name : null;
};
+/** Read exact consumed object-pattern keys, leaving dynamic keys unknown. */
+export const semanticObjectPatternKeys = (
+ pattern: t.ObjectPattern,
+): readonly string[] =>
+ pattern.properties.flatMap((property) => {
+ if (t.isRestElement(property)) return [];
+ const key = semanticStaticPropertyKey(property.key, property.computed);
+ return key === null ? [] : [key];
+ });
+
/**
* Display an exact JavaScript string as nonempty label text. Graph labels and
* artifact-local keys are nonempty, so the legal empty value is shown as `""`;
diff --git a/src/domain/javascript/javascriptSemanticAnalysis.member_mutations.test.ts b/src/domain/javascript/javascriptSemanticAnalysis.member_mutations.test.ts
index f28436eb1d..3450060cbe 100644
--- a/src/domain/javascript/javascriptSemanticAnalysis.member_mutations.test.ts
+++ b/src/domain/javascript/javascriptSemanticAnalysis.member_mutations.test.ts
@@ -590,3 +590,24 @@ describe("property mutations through TypeScript satisfies aliases", () => {
).toEqual({ status: "literal", value: "TOKEN" });
});
});
+
+describe("property mutation collection on minified bundles", () => {
+ it("records each write to one object without replaying earlier writes", () => {
+ const writes = Array.from(
+ { length: 2000 },
+ (_, index) => `source.p${index} = ${index};`,
+ ).join(" ");
+ const start = performance.now();
+ const value = resultValue(
+ `const source = { token: "TOKEN" }; ${writes} return [source.token, source.p1999];`,
+ );
+ expect(performance.now() - start).toBeLessThan(2000);
+ expect(value).toMatchObject({
+ status: "array",
+ items: [
+ { value: { status: "literal", value: "TOKEN" } },
+ { value: { status: "unknown" } },
+ ],
+ });
+ }, 30000);
+});
diff --git a/src/domain/javascript/javascriptSemanticAnalysis.reference_lifetimes.test.ts b/src/domain/javascript/javascriptSemanticAnalysis.reference_lifetimes.test.ts
index 2795427bd4..73c7897b0f 100644
--- a/src/domain/javascript/javascriptSemanticAnalysis.reference_lifetimes.test.ts
+++ b/src/domain/javascript/javascriptSemanticAnalysis.reference_lifetimes.test.ts
@@ -9,6 +9,163 @@ const resultValue = (body: string) =>
"result",
).returnSites[0]?.value;
+describe("bounded member path propagation", () => {
+ it("bounds cyclic member paths with no observed container origin", () => {
+ const branches = Array.from(
+ { length: 16 },
+ (_, index) => `if (flag === ${index}) node = node.m${index};`,
+ ).join(" else ");
+ const start = performance.now();
+ const callable = onlyCallable(
+ analyzeJavaScriptSemantics(
+ `export function result(seed) { let node=seed; while(node) { ${branches} use(node.state); } return seed.keep; }`,
+ ),
+ "result",
+ );
+ expect(callable.returnSites[0]?.value.status).toBe("unknown");
+ expect(performance.now() - start).toBeLessThan(2000);
+ }, 30000);
+
+ it("preserves Array prototype effects through an otherwise unknown alias", () => {
+ expect(
+ resultValue(
+ "const values=[1]; const prototype=Array.prototype; prototype[Symbol.iterator]=custom; for(const item of values) { mutate(item); } return values[0];",
+ )?.status,
+ ).toBe("unknown");
+ });
+
+ it("bounds loops derived from a scalar member while preserving sibling facts", () => {
+ const branches = Array.from(
+ { length: 16 },
+ (_, index) => `if (flag === ${index}) node = node.m${index};`,
+ ).join(" else ");
+ const start = performance.now();
+ expect(
+ resultValue(
+ `const source={child:1,keep:{value:3}}; let node=source.child; while(node) { ${branches} use(node.state); } return source.keep.value;`,
+ ),
+ ).toEqual({ status: "literal", value: 3 });
+ expect(performance.now() - start).toBeLessThan(2000);
+ }, 30000);
+
+ it.each([
+ ["child:external,", "child", "present"],
+ ["", "missing", "unknown-coverage"],
+ ])(
+ "retains slot presence below an unobserved container: %s",
+ (property, name, presence) => {
+ expect(
+ resultValue(
+ `const source={${property}keep:3}; const alias=source.${name}; alias.deep.value=2; return source;`,
+ ),
+ ).toMatchObject({
+ status: "object",
+ properties: expect.arrayContaining([
+ {
+ name,
+ presence,
+ value: {
+ status: "unknown",
+ reason: "This property may have been mutated.",
+ },
+ },
+ {
+ name: "keep",
+ presence: "present",
+ value: { status: "literal", value: 3 },
+ },
+ ]),
+ });
+ },
+ );
+
+ it.each([
+ ["use(node);", "m0.value", "unknown"],
+ ["node.value = 2;", "m0.value", "unknown"],
+ ["node.value = 2;", "mode", "literal"],
+ ])(
+ "bounds a loop that reassigns one reference through many members: %s seed.%s",
+ (effect, read, status) => {
+ const branches = Array.from(
+ { length: 9 },
+ (_, index) => `if (flag === ${index}) node = node.m${index};`,
+ ).join(" else ");
+ const start = performance.now();
+ const value = resultValue(
+ `const seed = { mode: "initial", m0: { value: 1 } }; let node = seed; while (node) { ${branches} ${effect} } return seed.${read};`,
+ );
+ expect(performance.now() - start).toBeLessThan(2000);
+ expect(value?.status).toBe(status);
+ },
+ 30000,
+ );
+
+ it.each([
+ ["const alias = flag ? owner.child : owner; alias.value = 2;", "value"],
+ ["const alias = flag ? owner : owner.child; alias.value = 2;", "value"],
+ ["const alias = flag ? owner.child : owner; alias[key] = 2;", "value"],
+ [
+ "const alias = flag ? owner.child : owner; alias.child = 2;",
+ "child.value",
+ ],
+ [
+ "let alias = owner; let i = 0; while (i++ < 1) { alias = flag ? alias.child : alias; alias.value = 2; }",
+ "value",
+ ],
+ ])(
+ "keeps a write that reaches a child through a longer path: %s",
+ (body, read) => {
+ expect(
+ resultValue(
+ `const shared = { value: 1, child: { value: 1 } }; const owner = { child: shared }; ${body} return shared.${read};`,
+ )?.status,
+ ).toBe("unknown");
+ },
+ );
+
+ it.each([
+ "mutate(source.left, source.right);",
+ "const alias = flag ? source.left : source.right; mutate(alias);",
+ "const copy = flag ? [source.left] : [source.right]; mutate(copy);",
+ "const alias = flag ? source.left : source.right; alias.value = 2;",
+ ])("keeps members outside the mutated paths: %s", (body) => {
+ expect(
+ resultValue(
+ `const source = { left: { value: 1 }, right: { value: 2 }, keep: { value: 3 } }; ${body} return source.keep.value;`,
+ ),
+ ).toEqual({ status: "literal", value: 3 });
+ });
+});
+
+describe("distinct observable mutation paths", () => {
+ it.each([
+ ["alias.value = 2;", "left"],
+ ["alias.value = 2;", "right"],
+ ["mutate(alias);", "left"],
+ ["mutate(alias);", "right"],
+ ])(
+ "propagates both conditional paths to shared bindings: %s %s",
+ (effect, target) => {
+ expect(
+ resultValue(
+ `const left={value:1}; const right={value:2}; const source={left,right}; const alias=flag?source.left:source.right; ${effect} return ${target}.value;`,
+ )?.status,
+ ).toBe("unknown");
+ },
+ );
+
+ it.each(["mutate(alias);", "alias.value = 2;"])(
+ "preserves sibling facts below a shared prefix: %s",
+ (effect) => {
+ expect(
+ resultValue(
+ `const shared={left:{value:1},right:{value:2},keep:{value:3}}; const source={nested:shared}; const alias=flag?source.nested.left:source.nested.right; ${effect} return shared.keep.value;`,
+ ),
+ ).toEqual({ status: "literal", value: 3 });
+ },
+ );
+});
+
describe("reference lifetimes across calls and shallow copies", () => {
it("bounds dynamic destructuring across shared alias branches", () => {
const declarations = Array.from(
diff --git a/src/domain/javascript/javascriptSemanticAnalysis.ts b/src/domain/javascript/javascriptSemanticAnalysis.ts
index 5a873429e1..4b5ad0f723 100644
--- a/src/domain/javascript/javascriptSemanticAnalysis.ts
+++ b/src/domain/javascript/javascriptSemanticAnalysis.ts
@@ -14,7 +14,10 @@ import {
immutableSemanticBindings,
immutableSemanticScopes,
} from "./javascriptSemanticProjection.js";
-import { semanticStaticPropertyKey } from "./javascriptAstValues.js";
+import {
+ semanticObjectPatternKeys,
+ semanticStaticPropertyKey,
+} from "./javascriptAstValues.js";
import type {
JavaScriptSemanticAnalysisState,
JavaScriptSemanticBindingState,
@@ -29,6 +32,7 @@ import {
semanticVariableScope,
} from "./javascriptSemanticState.js";
import { traverseJavaScriptAst } from "./javascriptSemanticTraversal.js";
+import { semanticRequireOrigin } from "./javascriptSemanticRequireOrigin.js";
import { collectSemanticMemberMutations } from "./javascriptSemanticMemberMutations.js";
import {
collectSemanticReturns,
@@ -282,6 +286,92 @@ const resolveGlobalAliasFact = (
return openGlobalFact(name);
};
+const ELECTRON_MODULE = /^electron(?:\/(?:common|main|renderer|utility))?$/u;
+
+/**
+ * Electron export paths for call and construction roots bound to Electron
+ * under another name, such as esbuild's `import { ipcMain as ipcMain2 }`.
+ * Keys are root identifier offsets; roots spelled as their export are omitted.
+ */
+export const classifyParsedJavaScriptElectronBindings = (
+ file: ParsedJavaScriptSource,
+): ReadonlyMap => {
+ const facts = new Map();
+ const state = createState(file.program);
+ collectDefinitions(file.program, state);
+ const exportsByBinding = new Map();
+ traverseJavaScriptAst(file.program, {
+ enter: (node) => {
+ if (
+ !t.isCallExpression(node) &&
+ !t.isOptionalCallExpression(node) &&
+ !t.isNewExpression(node)
+ )
+ return;
+ const root = calleeRoot(node.callee);
+ if (root === undefined) return;
+ const binding = resolveSemanticBindingState(state, root, root.name);
+ if (binding === undefined) return;
+ let exported = exportsByBinding.get(binding.bindingId);
+ if (exported === undefined) {
+ exported = electronExport(binding, state);
+ exportsByBinding.set(binding.bindingId, exported);
+ }
+ if (exported !== null && exported !== root.name)
+ facts.set(root.start ?? -1, exported);
+ },
+ });
+ return facts;
+};
+
+const calleeRoot = (node: t.Node): t.Identifier | undefined => {
+ while (t.isMemberExpression(node) || t.isOptionalMemberExpression(node)) {
+ if (!t.isNode(node.object)) return undefined;
+ node = node.object;
+ }
+ return t.isIdentifier(node) ? node : undefined;
+};
+
+// Read import and require origins directly; general value evaluation is
+// unnecessary here and costly across large vendor bundles.
+const electronExport = (
+ binding: JavaScriptSemanticBindingState,
+ state: JavaScriptSemanticAnalysisState,
+): string | null => {
+ const origin = bindingOrigin(binding, state);
+ return origin !== undefined &&
+ ELECTRON_MODULE.test(origin.specifier) &&
+ origin.importedPath.length > 0
+ ? origin.importedPath.join(".")
+ : null;
+};
+
+const bindingOrigin = (
+ binding: JavaScriptSemanticBindingState,
+ state: JavaScriptSemanticAnalysisState,
+): JavaScriptModuleOrigin | undefined => {
+ if (binding.directOrigins.length > 0)
+ return binding.directOrigins.length === 1
+ ? binding.directOrigins[0]
+ : undefined;
+ const [initializer] = binding.initializers;
+ // A var/let declaration permits writes without proving one occurred.
+ // Require one unconditional initializer and no actual binding writes.
+ if (
+ initializer === undefined ||
+ binding.initializers.length !== 1 ||
+ binding.definitions.some(({ kind }) => kind === "assignment") ||
+ state.conditionalInitializers.has(initializer.node)
+ )
+ return undefined;
+ const origin = semanticRequireOrigin(initializer.node, state);
+ const projection = initializer.projection;
+ return origin === undefined ||
+ !projection.every((key): key is string => typeof key === "string")
+ ? undefined
+ : { ...origin, importedPath: [...origin.importedPath, ...projection] };
+};
+
const createState = (program: t.Program): JavaScriptSemanticAnalysisState => {
const root: JavaScriptSemanticScopeState = {
scopeId: semanticScopeId("program", program),
@@ -565,9 +655,21 @@ const bindAssignment = (
else if (t.isForOfStatement(node) || t.isForInStatement(node))
for (const pattern of t.isVariableDeclaration(node.left)
? node.left.declarations.map(({ id }) => id)
- : [node.left])
+ : [node.left]) {
for (const identifier of assignedPatternIdentifiers(pattern))
addAssignment(identifier, node, scope, state);
+ if (t.isForOfStatement(node))
+ // Keep the loop value unknown, but preserve references yielded into
+ // each bound slot so writes/escapes can reach their iterable origins.
+ bindPattern({
+ pattern,
+ initializer: node,
+ scope,
+ state,
+ mutable: true,
+ referenceOnly: true,
+ });
+ }
};
const assignedPatternIdentifiers = (
@@ -780,7 +882,11 @@ const bindPattern = (input: BindPatternInput): void => {
}
if (t.isIdentifier(pattern)) {
if (referenceOnly) {
- const binding = scope.bindings.get(pattern.name);
+ const binding = resolveSemanticBindingFromScope(
+ scope,
+ pattern.name,
+ state,
+ );
if (binding !== undefined && initializer !== null)
binding.referenceInitializers.push({
node: initializer,
@@ -864,14 +970,7 @@ const bindPattern = (input: BindPatternInput): void => {
referenceOnly: true,
copyKind: "object-rest",
copyProjectionOffset: projection.length,
- copyExcludedKeys: pattern.properties.flatMap((property) => {
- if (t.isRestElement(property)) return [];
- const key = semanticStaticPropertyKey(
- property.key,
- property.computed,
- );
- return key === null ? [] : [key];
- }),
+ copyExcludedKeys: semanticObjectPatternKeys(pattern),
});
} else {
const name = semanticStaticPropertyKey(property.key, property.computed);
diff --git a/src/domain/javascript/javascriptSemanticCallableResults.ts b/src/domain/javascript/javascriptSemanticCallableResults.ts
new file mode 100644
index 0000000000..d8cfaa4b9c
--- /dev/null
+++ b/src/domain/javascript/javascriptSemanticCallableResults.ts
@@ -0,0 +1,53 @@
+import * as t from "@babel/types";
+
+import { childNodes } from "./javascriptSemanticTraversal.js";
+
+/** One direct return or yield, excluding every nested callable. */
+export interface SemanticCallableResultExpression {
+ readonly node: t.Node | null;
+ readonly site: t.Node;
+ readonly kind: "return" | "yield";
+ readonly delegated: boolean;
+}
+
+/** Collect direct result syntax without executing or evaluating a callable. */
+export const semanticCallableResultExpressions = (
+ callable: t.Node,
+): readonly SemanticCallableResultExpression[] => {
+ if (
+ t.isArrowFunctionExpression(callable) &&
+ !t.isBlockStatement(callable.body)
+ )
+ return [
+ {
+ node: callable.body,
+ site: callable.body,
+ kind: "return",
+ delegated: false,
+ },
+ ];
+ if (!t.isFunction(callable)) return [];
+ const output: SemanticCallableResultExpression[] = [];
+ const pending = childNodes(callable.body).reverse();
+ while (pending.length > 0) {
+ const node = pending.pop();
+ if (node === undefined || t.isFunction(node) || t.isClass(node)) continue;
+ if (t.isReturnStatement(node)) {
+ output.push({
+ node: node.argument ?? null,
+ site: node,
+ kind: "return",
+ delegated: false,
+ });
+ }
+ if (t.isYieldExpression(node))
+ output.push({
+ node: node.argument ?? null,
+ site: node,
+ kind: "yield",
+ delegated: node.delegate,
+ });
+ for (const child of childNodes(node).reverse()) pending.push(child);
+ }
+ return output;
+};
diff --git a/src/domain/javascript/javascriptSemanticGraph.ts b/src/domain/javascript/javascriptSemanticGraph.ts
index e6cf1d5302..9ab2dd485c 100644
--- a/src/domain/javascript/javascriptSemanticGraph.ts
+++ b/src/domain/javascript/javascriptSemanticGraph.ts
@@ -56,6 +56,9 @@ const evidenceContextId = (context: ApplicationGraphEvidenceContext): string =>
/** Graph-scoped owner for canonical provenance shared by semantic facts. */
export class JavaScriptSemanticEvidenceContextRegistry {
readonly #contexts = new Map();
+ // Facts share a few contexts, so key them by canonical text and hash each
+ // distinct context once instead of on every fact.
+ readonly #idsByCanonicalContext = new Map();
/** Intern normalized non-location provenance at fact construction. */
intern(
@@ -63,18 +66,21 @@ export class JavaScriptSemanticEvidenceContextRegistry {
): JavaScriptSemanticEvidenceReference {
const normalized = normalizeEvidence(evidence);
const { location, ...context } = normalized;
+ const canonicalContext = canonicalJson(
+ context,
+ "JavaScript semantic graph",
+ );
+ const internedId = this.#idsByCanonicalContext.get(canonicalContext);
+ if (internedId !== undefined) return { context_id: internedId, location };
const contextId = evidenceContextId(context);
- const row = { ...context, context_id: contextId };
- const existing = this.#contexts.get(contextId);
- if (
- existing !== undefined &&
- canonicalJson(existing, "JavaScript semantic graph") !==
- canonicalJson(row, "JavaScript semantic graph")
- )
+ // This canonical context is new, so a row already stored under its
+ // digest holds different provenance.
+ if (this.#contexts.has(contextId))
throw new TypeError(
"JavaScript semantic evidence context digest collision",
);
- this.#contexts.set(contextId, row);
+ this.#contexts.set(contextId, { ...context, context_id: contextId });
+ this.#idsByCanonicalContext.set(canonicalContext, contextId);
return { context_id: contextId, location };
}
@@ -101,6 +107,7 @@ export class JavaScriptSemanticEvidenceContextRegistry {
/** Release interned contexts after the owning graph is finalized. */
clear(): void {
this.#contexts.clear();
+ this.#idsByCanonicalContext.clear();
}
}
@@ -120,10 +127,6 @@ export const createJavaScriptSemanticGraphNode = (
...parsed,
application_node_ids: uniqueSorted(parsed.application_node_ids),
evidence: evidenceContexts.intern(parsed.evidence),
- identifier_strategy: {
- strategy: "semantic-content-sha256" as const,
- stability: "artifact-version" as const,
- },
};
return javaScriptSemanticNodeSchema.parse({
...semantic,
@@ -140,10 +143,6 @@ export const createJavaScriptSemanticGraphRelation = (
const semantic = {
...parsed,
evidence: evidenceContexts.intern(parsed.evidence),
- identifier_strategy: {
- strategy: "semantic-content-sha256" as const,
- stability: "relationship-exact" as const,
- },
};
return javaScriptSemanticRelationSchema.parse({
...semantic,
diff --git a/src/domain/javascript/javascriptSemanticGraphQuery.test.ts b/src/domain/javascript/javascriptSemanticGraphQuery.test.ts
index ddf3015f08..363cf29ad2 100644
--- a/src/domain/javascript/javascriptSemanticGraphQuery.test.ts
+++ b/src/domain/javascript/javascriptSemanticGraphQuery.test.ts
@@ -1,6 +1,7 @@
import { expect, it } from "vitest";
import { z } from "zod";
+import { digestCanonicalValue } from "../canonicalDigest.js";
import type { ApplicationGraphEvidence } from "./javascriptApplicationEvidenceSchemas.js";
import {
JAVASCRIPT_SEMANTIC_NODE_KINDS,
@@ -615,3 +616,42 @@ it.each(invalidGraphCases)(
expect(issues(() => complete(steps))).toEqual(expected);
},
);
+
+it("interns each distinct evidence context once under its content digest", () => {
+ const registry = new JavaScriptSemanticEvidenceContextRegistry();
+ const elsewhere: ApplicationGraphEvidence = {
+ ...evidence(),
+ location: {
+ available: true,
+ value: {
+ kind: "source-range",
+ source: "other.js",
+ start: { line: 7, column: 2 },
+ end: { line: 7, column: 9 },
+ },
+ },
+ };
+ const first = registry.intern(evidence());
+ const repeated = registry.intern(elsewhere);
+ const reordered = registry.intern({
+ ...evidence("inferred"),
+ limitations: [
+ "Static reachability does not prove runtime execution.",
+ "Static reachability does not prove runtime execution.",
+ ],
+ });
+ const inferred = registry.intern(evidence("inferred"));
+ const { location: _location, ...context } = evidence();
+
+ expect(repeated.context_id).toBe(first.context_id);
+ expect(repeated.location).toEqual(elsewhere.location);
+ expect(reordered.context_id).toBe(inferred.context_id);
+ expect(inferred.context_id).not.toBe(first.context_id);
+ expect(first.context_id).toBe(
+ `jsrg_evidence_${digestCanonicalValue(context, "test")}`,
+ );
+ expect(registry.contexts.map(({ context_id }) => context_id)).toEqual(
+ [first.context_id, inferred.context_id].toSorted(),
+ );
+ expect(registry.resolve(repeated)).toEqual(elsewhere);
+});
diff --git a/src/domain/javascript/javascriptSemanticGraphSchemas.ts b/src/domain/javascript/javascriptSemanticGraphSchemas.ts
index fb03f1a4d2..4048ae698e 100644
--- a/src/domain/javascript/javascriptSemanticGraphSchemas.ts
+++ b/src/domain/javascript/javascriptSemanticGraphSchemas.ts
@@ -233,11 +233,9 @@ export const javaScriptSemanticNodeInputSchema = z.strictObject({
export const javaScriptSemanticNodeSchema =
javaScriptSemanticNodeInputSchema.extend({
evidence: javaScriptSemanticEvidenceReferenceSchema,
- node_id: semanticNodeIdSchema,
- identifier_strategy: z.strictObject({
- strategy: z.literal("semantic-content-sha256"),
- stability: z.literal("artifact-version"),
- }),
+ node_id: semanticNodeIdSchema.describe(
+ "SHA-256 of the node kind and identity; stable for one artifact version.",
+ ),
});
/** One directed semantic relationship before its ID is derived. */
@@ -254,11 +252,9 @@ export const javaScriptSemanticRelationInputSchema = z.strictObject({
export const javaScriptSemanticRelationSchema =
javaScriptSemanticRelationInputSchema.extend({
evidence: javaScriptSemanticEvidenceReferenceSchema,
- relation_id: semanticRelationIdSchema,
- identifier_strategy: z.strictObject({
- strategy: z.literal("semantic-content-sha256"),
- stability: z.literal("relationship-exact"),
- }),
+ relation_id: semanticRelationIdSchema.describe(
+ "SHA-256 of the exact relation content, including its evidence reference.",
+ ),
});
/** Reasons semantic analysis can leave an explicit frontier. */
diff --git a/src/domain/javascript/javascriptSemanticIterationSources.ts b/src/domain/javascript/javascriptSemanticIterationSources.ts
index 1b9e7d7143..ea3708e515 100644
--- a/src/domain/javascript/javascriptSemanticIterationSources.ts
+++ b/src/domain/javascript/javascriptSemanticIterationSources.ts
@@ -1,15 +1,23 @@
import * as t from "@babel/types";
-import { semanticStaticPropertyKey } from "./javascriptAstValues.js";
-import { semanticArrayIndex } from "./javascriptSemanticPropertyPaths.js";
+import {
+ semanticObjectPatternKeys,
+ semanticStaticPropertyKey,
+} from "./javascriptAstValues.js";
+import {
+ semanticArrayIndex,
+ type JavaScriptSemanticPropertyPath,
+} from "./javascriptSemanticPropertyPaths.js";
+import { compareUnicodeCodePoints } from "../unicodeCodePointOrder.js";
interface IterationSourceValue {
readonly node: t.Node;
- readonly projection: readonly (string | number | null)[];
+ readonly projection: JavaScriptSemanticPropertyPath;
}
-/** An implicit iterator receiver, with guards for destructuring defaults. */
+/** An iterator receiver or property reference store, with default guards. */
export interface SemanticIterationSource extends IterationSourceValue {
+ readonly kind: "iteration" | "reference-store";
readonly mutation: t.Node;
readonly fallbackSources?: readonly IterationSourceValue[];
readonly requiredSources?: readonly IterationSourceValue[];
@@ -30,8 +38,10 @@ const add = (
source: IterationSourceValue,
mutation: t.Node,
guards: SourceGuards,
+ kind: SemanticIterationSource["kind"] = "iteration",
): void => {
sources.push({
+ kind,
node: source.node,
projection: source.projection,
mutation,
@@ -66,6 +76,27 @@ const visit = (
guards: SourceGuards = {},
): void => {
const { sources, mutation } = context;
+ if (t.isMemberExpression(pattern)) {
+ // Property stores have no lexical alias binding. Preserve the selected
+ // reference's escape rather than claiming later writes cannot reach it.
+ if (source !== undefined)
+ add(
+ sources,
+ source.restStartIndex === undefined
+ ? source
+ : {
+ node: source.node,
+ projection: [
+ ...source.projection,
+ { excludedKeys: [], startIndex: source.restStartIndex },
+ ],
+ },
+ mutation,
+ guards,
+ "reference-store",
+ );
+ return;
+ }
if (t.isTSParameterProperty(pattern)) {
visit(context, pattern.parameter, source, guards);
return;
@@ -94,8 +125,29 @@ const visit = (
}
if (t.isObjectPattern(pattern)) {
for (const property of pattern.properties) {
- // Object rest creates a fresh object and has no iterable binding pattern.
- if (t.isRestElement(property)) continue;
+ if (t.isRestElement(property)) {
+ // A rest copy owns its scalar slots; only selected child references
+ // escape when that fresh copy is stored through a member target.
+ visit(
+ context,
+ property.argument,
+ source === undefined
+ ? undefined
+ : {
+ node: source.node,
+ projection: [
+ ...source.projection,
+ {
+ excludedKeys: [
+ ...new Set(semanticObjectPatternKeys(pattern)),
+ ].sort(compareUnicodeCodePoints),
+ },
+ ],
+ },
+ guards,
+ );
+ continue;
+ }
visit(
context,
property.value,
@@ -134,7 +186,7 @@ const visit = (
visit(context, pattern.argument, undefined, guards);
};
-/** Describe implicit iteration sources without deciding their runtime values. */
+/** Describe iterator receivers and property reference stores without evaluation. */
export const semanticIterationSources = (
node: t.Node,
parent: t.Node | null,
@@ -143,7 +195,10 @@ export const semanticIterationSources = (
const context = { sources, mutation: node };
if (t.isVariableDeclarator(node) && node.init != null)
visit(context, node.id, { node: node.init, projection: [] });
- else if (t.isAssignmentExpression(node) && node.operator === "=")
+ else if (
+ t.isAssignmentExpression(node) &&
+ ["=", "||=", "&&=", "??="].includes(node.operator)
+ )
visit(context, node.left, { node: node.right, projection: [] });
else if (t.isForOfStatement(node)) {
add(sources, { node: node.right, projection: [] }, node, {});
diff --git a/src/domain/javascript/javascriptSemanticMemberMutations.ts b/src/domain/javascript/javascriptSemanticMemberMutations.ts
index ae887b9c79..4c0dfc91d1 100644
--- a/src/domain/javascript/javascriptSemanticMemberMutations.ts
+++ b/src/domain/javascript/javascriptSemanticMemberMutations.ts
@@ -15,9 +15,12 @@ import {
semanticSlotAtPath,
} from "./javascriptSemanticSlots.js";
import { semanticIterationSources } from "./javascriptSemanticIterationSources.js";
+import { semanticCallableResultExpressions } from "./javascriptSemanticCallableResults.js";
+import { createSemanticReturnedReferences } from "./javascriptSemanticReturnedReferences.js";
import { semanticMutationInitializers } from "./javascriptSemanticMutationInitializers.js";
import {
semanticArrayIndex,
+ SemanticPropertyPathCoverage,
semanticPropertyPathKeyMatches,
type JavaScriptSemanticPropertyPath,
} from "./javascriptSemanticPropertyPaths.js";
@@ -42,8 +45,11 @@ export const collectSemanticMemberMutations = (
if (parent !== null) parents.set(node, parent);
},
});
+ const normalizePath = observableMutationPaths(state);
+ const returnedReferences = createSemanticReturnedReferences(state);
const recordedEffects = new Set();
const selections = new Map();
+ const coveringPaths = new Map();
let arrayIterationUnknown = false;
const pendingReferences: {
readonly initializer: JavaScriptSemanticBindingState["referenceInitializers"][number];
@@ -93,8 +99,10 @@ export const collectSemanticMemberMutations = (
originAt: t.Node | undefined = mutation,
): void => {
const pending = [{ node, path, bindings, originAt }];
- while (pending.length > 0) {
- const current = pending.pop();
+ // Visit shallower alias traversals first so their exact prefixes suppress
+ // longer loop paths before those paths enumerate every assignment order.
+ for (let cursor = 0; cursor < pending.length; cursor++) {
+ const current = pending[cursor];
if (current === undefined) break;
const expression = unwrapJavaScriptExpression(current.node).node;
if (t.isIdentifier(expression)) {
@@ -126,10 +134,13 @@ export const collectSemanticMemberMutations = (
if (current.bindings.has(lifetime)) continue;
// Union compatible copied-slot selections at each binding. Carrying every
// combination of exclusions through branch diamonds is exponential.
- const selectorIndex = current.path.findIndex(
+ // Preserve paths that can change observed slots or Array prototype
+ // effects; canonicalize suffixes that cannot distinguish lattice facts.
+ let path = normalizePath(binding.bindingId, current.path, effect);
+ if (path === null) continue;
+ const selectorIndex = path.findIndex(
(key) => key !== null && typeof key === "object",
);
- let path = current.path;
const selector = path[selectorIndex];
if (selector !== null && typeof selector === "object") {
const selectionIdentity = JSON.stringify([
@@ -157,6 +168,25 @@ export const collectSemanticMemberMutations = (
index === selectorIndex ? { ...selector, excludedKeys } : key,
);
}
+ // Keep an antichain of exact paths. Widening distinct member keys would
+ // also invalidate unrelated siblings that never reach this effect.
+ const pathState = JSON.stringify([
+ binding.bindingId,
+ effect,
+ // Short writes stop at owned literal slots, whereas longer writes
+ // reach shared children. Only equal-depth writes cover one another.
+ effect === "write" ? path.length : null,
+ mutation?.start,
+ mutation?.end,
+ current.originAt?.start,
+ current.originAt?.end,
+ ]);
+ let covering = coveringPaths.get(pathState);
+ if (covering === undefined) {
+ covering = new SemanticPropertyPathCoverage();
+ coveringPaths.set(pathState, covering);
+ }
+ if (!covering.retain(path)) continue;
const identity = JSON.stringify([
binding.bindingId,
effect,
@@ -167,8 +197,11 @@ export const collectSemanticMemberMutations = (
current.originAt?.end,
]);
if (recordedEffects.has(identity)) continue;
+ // A recorded write invalidates the binding to a non-primitive value, so
+ // only its first write can target a primitive. Re-evaluating it would
+ // replay every earlier write and make repeated writes superlinear.
const value =
- effect === "write"
+ effect === "write" && binding.mutatedPaths.length === 0
? evaluateSemanticBinding(binding, state)
: undefined;
const primitiveWrite =
@@ -297,7 +330,7 @@ export const collectSemanticMemberMutations = (
bindings: new Set(),
effect: "escape",
originAt: source.mutation,
- iterationOnly: true,
+ iterationOnly: source.kind === "iteration",
});
if (t.isAssignmentExpression(node)) markTarget(node.left, node);
else if (t.isUpdateExpression(node)) markTarget(node.argument, node);
@@ -310,6 +343,8 @@ export const collectSemanticMemberMutations = (
t.isOptionalCallExpression(node) ||
t.isNewExpression(node)
) {
+ for (const source of returnedReferences(node.callee))
+ markEscaped(source.node, node, source.projection);
for (const argument of node.arguments)
if (t.isSpreadElement(argument))
deferIterable({
@@ -342,6 +377,19 @@ export const collectSemanticMemberMutations = (
const { initializer } = reference;
if (referenceSourcesInactive(initializer, state))
pendingReferences.push(reference);
+ else if (t.isForOfStatement(initializer.node))
+ deferIterable({
+ node: initializer.node.right,
+ projection: [],
+ path: [null, ...reference.path],
+ fallbackPath: [],
+ bindings: reference.bindings,
+ effect: reference.effect,
+ originAt: initializer.node,
+ ...(reference.mutation === undefined
+ ? {}
+ : { mutation: reference.mutation }),
+ });
else if (
initializer.copyKind === "array-rest" &&
initializer.copyProjectionOffset !== undefined
@@ -512,6 +560,11 @@ const referencedValues = (
path: PropertyPath,
effect: ValueEffect,
): readonly ReferencedValue[] => {
+ if (effect === "escape" && t.isFunction(node))
+ return semanticCallableResultExpressions(node).flatMap(
+ ({ node, delegated }) =>
+ node === null ? [] : [{ node, path: delegated ? [null] : [] }],
+ );
// An initializer owns its slots; only deeper writes can affect shared children.
if (t.isObjectExpression(node) || t.isArrayExpression(node)) {
if (effect === "write" && path.length < 2) return [];
@@ -544,6 +597,241 @@ const referencedValues = (
return [];
};
+type MutationObservableOrigin =
+ | t.ObjectExpression
+ | t.ArrayExpression
+ | "ambient"
+ | "array"
+ | "array-prototype"
+ | "opaque";
+
+interface MutationObserverNode {
+ readonly origins: Set;
+ readonly sources: Set;
+ readonly dependents: Set;
+ readonly projections: Map;
+ readonly read?: {
+ readonly source: MutationObserverNode;
+ readonly key: PropertyPath[number];
+ };
+}
+
+const observableMutationPaths = (
+ state: JavaScriptSemanticAnalysisState,
+): ((
+ bindingId: string,
+ path: PropertyPath,
+ effect: ValueEffect,
+) => PropertyPath | null) => {
+ const bindingNodes = new Map();
+ const expressions = new WeakMap();
+ const pendingExpressions: {
+ readonly expression: t.Node;
+ readonly observer: MutationObserverNode;
+ }[] = [];
+ const pending: MutationObserverNode[] = [];
+ const queued = new Set();
+ const schedule = (node: MutationObserverNode): void => {
+ if (queued.has(node)) return;
+ queued.add(node);
+ pending.push(node);
+ };
+ const create = (
+ read?: MutationObserverNode["read"],
+ ): MutationObserverNode => ({
+ origins: new Set(),
+ sources: new Set(),
+ dependents: new Set(),
+ projections: new Map(),
+ ...(read === undefined ? {} : { read }),
+ });
+ const link = (
+ source: MutationObserverNode,
+ target: MutationObserverNode,
+ ): void => {
+ if (target.sources.has(source)) return;
+ target.sources.add(source);
+ source.dependents.add(target);
+ schedule(target);
+ };
+ const expressionNode = (node: t.Node): MutationObserverNode => {
+ const expression = unwrapJavaScriptExpression(node).node;
+ const existing = expressions.get(expression);
+ if (existing !== undefined) return existing;
+ const observer = create();
+ expressions.set(expression, observer);
+ pendingExpressions.push({ expression, observer });
+ return observer;
+ };
+ const project = (
+ source: MutationObserverNode,
+ path: PropertyPath,
+ ): MutationObserverNode => {
+ let observer = source;
+ for (const key of path) {
+ const identity = JSON.stringify(
+ key === null || typeof key === "object" ? key : String(key),
+ );
+ let selected = observer.projections.get(identity);
+ if (selected === undefined) {
+ selected = create({ source: observer, key });
+ observer.projections.set(identity, selected);
+ observer.dependents.add(selected);
+ schedule(selected);
+ }
+ observer = selected;
+ }
+ return observer;
+ };
+ for (const binding of state.bindingsById.values())
+ bindingNodes.set(binding.bindingId, create());
+ for (const binding of state.bindingsById.values()) {
+ const observer = bindingNodes.get(binding.bindingId);
+ if (observer === undefined) continue;
+ for (const initializer of binding.initializers)
+ link(
+ project(expressionNode(initializer.node), initializer.projection),
+ observer,
+ );
+ // Preserve exact paths where copies, defaults or iteration need their
+ // reference-specific projections; an opaque origin prevents pruning.
+ if (binding.referenceInitializers.length > 0)
+ observer.origins.add("opaque");
+ for (const initializer of binding.referenceInitializers) {
+ for (const source of [
+ initializer,
+ ...(initializer.requiredSources ?? []),
+ ...(initializer.fallbackSources ?? []),
+ ])
+ link(expressionNode(source.node), observer);
+ }
+ }
+ const settle = (): void => {
+ while (pendingExpressions.length > 0 || pending.length > 0) {
+ while (pendingExpressions.length > 0) {
+ const selected = pendingExpressions.pop();
+ if (selected === undefined) break;
+ const { expression, observer } = selected;
+ if (t.isObjectExpression(expression) || t.isArrayExpression(expression))
+ observer.origins.add(expression);
+ else if (t.isIdentifier(expression)) {
+ const binding = resolveSemanticBindingState(
+ state,
+ expression,
+ expression.name,
+ );
+ if (binding !== undefined) {
+ const origin = bindingNodes.get(binding.bindingId);
+ if (origin !== undefined) link(origin, observer);
+ } else if (expression.name === "Array") observer.origins.add("array");
+ else if (
+ ["globalThis", "global", "window", "self"].includes(expression.name)
+ )
+ observer.origins.add("ambient");
+ } else if (
+ t.isMemberExpression(expression) ||
+ t.isOptionalMemberExpression(expression)
+ )
+ link(
+ project(expressionNode(expression.object), [
+ semanticStaticPropertyKey(
+ expression.property,
+ expression.computed,
+ ),
+ ]),
+ observer,
+ );
+ else
+ for (const reference of referencedValues(expression, [], "escape"))
+ link(
+ project(expressionNode(reference.node), reference.path),
+ observer,
+ );
+ schedule(observer);
+ }
+ const observer = pending.pop();
+ if (observer === undefined) continue;
+ queued.delete(observer);
+ let changed = false;
+ const add = (origin: MutationObservableOrigin): void => {
+ if (observer.origins.has(origin)) return;
+ observer.origins.add(origin);
+ changed = true;
+ };
+ for (const source of observer.sources)
+ for (const origin of source.origins) add(origin);
+ if (observer.read !== undefined) {
+ const { source, key } = observer.read;
+ if (key === null || typeof key === "object") {
+ // Dynamic keys and copied-slot masks retain their original paths.
+ // Enumerating their exclusion combinations would recreate the growth
+ // that the mutation collector's selection union already prevents.
+ if (source.origins.size > 0) add("opaque");
+ } else
+ for (const origin of source.origins) {
+ if (typeof origin === "string") {
+ if (origin === "array-prototype" || origin === "opaque")
+ add(origin);
+ else if (
+ origin === "ambient" &&
+ semanticPropertyPathKeyMatches(key, "Array")
+ )
+ add("array");
+ else if (
+ origin === "array" &&
+ semanticPropertyPathKeyMatches(key, "prototype")
+ )
+ add("array-prototype");
+ } else {
+ for (const reference of referencedValues(
+ origin,
+ [key],
+ "escape",
+ )) {
+ link(
+ project(expressionNode(reference.node), reference.path),
+ observer,
+ );
+ if (reference.iterableFallbackPath !== undefined)
+ link(
+ project(
+ expressionNode(reference.node),
+ reference.iterableFallbackPath,
+ ),
+ observer,
+ );
+ }
+ }
+ }
+ }
+ if (changed)
+ for (const dependent of observer.dependents) schedule(dependent);
+ }
+ };
+ settle();
+ return (bindingId, path, effect) => {
+ let observer = bindingNodes.get(bindingId);
+ if (observer === undefined) return path;
+ if (path.some((key) => key !== null && typeof key === "object"))
+ return path;
+ if (effect === "escape") {
+ observer = project(observer, path);
+ settle();
+ return observer.origins.size === 0 ? null : path;
+ }
+ if (observer.origins.size === 0) return path.length === 0 ? path : [null];
+ for (let offset = 0; offset < path.length; offset++) {
+ const key = path[offset];
+ if (key === undefined) break;
+ observer = project(observer, [key]);
+ settle();
+ if (observer.origins.size === 0 && offset + 1 < path.length)
+ return [...path.slice(0, offset + 1), null];
+ }
+ return path;
+ };
+};
+
const objectReferencedValues = (
node: t.ObjectExpression,
path: PropertyPath,
@@ -577,11 +865,11 @@ const objectReferencedValues = (
}
const name = semanticStaticPropertyKey(property.key, property.computed);
if (name !== null && overwritten.has(name)) continue;
- if (
- t.isObjectProperty(property) &&
- (name === null || semanticPropertyPathKeyMatches(key ?? null, name))
- )
- references.push({ node: property.value, path: remaining });
+ if (name === null || semanticPropertyPathKeyMatches(key ?? null, name))
+ references.push({
+ node: t.isObjectProperty(property) ? property.value : property,
+ path: remaining,
+ });
// A prototype setter does not replace an own property from a spread.
if (
name !== null &&
diff --git a/src/domain/javascript/javascriptSemanticMutationValues.ts b/src/domain/javascript/javascriptSemanticMutationValues.ts
index dbbc9d4572..bc2213da7b 100644
--- a/src/domain/javascript/javascriptSemanticMutationValues.ts
+++ b/src/domain/javascript/javascriptSemanticMutationValues.ts
@@ -33,49 +33,89 @@ export const invalidateSemanticEscapedPath = (
: { ...value, items: value.items.map(invalidateChild) };
};
-/** Invalidate only slots that an explicit mutation can affect. */
-export const invalidateSemanticMutationPath = (
+/**
+ * Invalidate only slots that explicit mutations can affect, as if applying each
+ * path in order. Paths are grouped per slot so that many writes to one object
+ * cost one pass over its slots rather than one pass per write.
+ */
+export const invalidateSemanticMutationPaths = (
value: JavaScriptSemanticValue,
- path: JavaScriptSemanticPropertyPath,
+ paths: readonly JavaScriptSemanticPropertyPath[],
): JavaScriptSemanticValue => {
- const [key, ...remaining] = path;
- const unknown: JavaScriptSemanticValue = {
- status: "unknown",
- reason: "This property may have been mutated.",
+ if (paths.length === 0) return value;
+ // An unknown key or a replaced value leaves nothing that a later path can
+ // restore, wherever it occurs in the sequence.
+ if (
+ (value.status !== "object" && value.status !== "array") ||
+ paths.some(([key]) => key === undefined || key === null)
+ )
+ return {
+ status: "unknown",
+ reason: "This property may have been mutated.",
+ };
+ // Writing an array length discards every item known before it.
+ const reset =
+ value.status === "array"
+ ? paths.findLastIndex(([key]) => key === "length")
+ : -1;
+ const base: typeof value =
+ reset < 0
+ ? value
+ : { status: "array", items: [], unknownItems: true, omittedItems: null };
+ const slots = base.status === "object" ? base.properties : base.items;
+ const slotsByName = new Map();
+ slots.forEach((property, index) => {
+ const indexes = slotsByName.get(property.name) ?? [];
+ indexes.push(index);
+ slotsByName.set(property.name, indexes);
+ });
+ const slotPaths = new Map();
+ const replacedSlots = new Set();
+ const select = (index: number, remaining: JavaScriptSemanticPropertyPath) => {
+ const selected = slotPaths.get(index) ?? [];
+ selected.push(remaining);
+ slotPaths.set(index, selected);
+ if (remaining.length === 0) replacedSlots.add(index);
};
- if (key === undefined || key === null) return unknown;
- if (value.status === "object" || value.status === "array") {
- if (value.status === "array" && key === "length")
- return {
- status: "array",
- items: [],
- unknownItems: true,
- omittedItems: null,
- };
- const slots = value.status === "object" ? value.properties : value.items;
- const observed = slots.some((property) =>
- semanticPropertyPathKeyMatches(key, property.name),
- );
- const properties = slots.map((property) =>
- semanticPropertyPathKeyMatches(key, property.name)
- ? {
+ // An added slot is already unknown, so later paths that select it are inert.
+ const added = new Set();
+ for (const [key, ...remaining] of paths.slice(reset + 1)) {
+ if (key === undefined || key === null) continue;
+ if (typeof key === "object") {
+ slots.forEach((property, index) => {
+ if (semanticPropertyPathKeyMatches(key, property.name))
+ select(index, remaining);
+ });
+ continue;
+ }
+ const indexes = slotsByName.get(String(key));
+ if (indexes === undefined) added.add(String(key));
+ else for (const index of indexes) select(index, remaining);
+ }
+ const properties: JavaScriptSemanticProperty[] = slots.map(
+ (property, index) => {
+ const selected = slotPaths.get(index);
+ return selected === undefined
+ ? property
+ : {
...property,
- ...(remaining.length === 0
+ ...(replacedSlots.has(index)
? { presence: "unknown-coverage" as const }
: {}),
- value: invalidateSemanticMutationPath(property.value, remaining),
- }
- : property,
- );
- if (!observed && typeof key !== "object")
- properties.push({
- name: String(key),
- value: unknown,
- presence: "unknown-coverage",
- });
- return value.status === "object"
- ? { ...value, properties }
- : { ...value, items: properties };
- }
- return unknown;
+ value: invalidateSemanticMutationPaths(property.value, selected),
+ };
+ },
+ );
+ for (const name of added)
+ properties.push({
+ name,
+ value: {
+ status: "unknown",
+ reason: "This property may have been mutated.",
+ },
+ presence: "unknown-coverage",
+ });
+ return base.status === "object"
+ ? { ...base, properties }
+ : { ...base, items: properties };
};
diff --git a/src/domain/javascript/javascriptSemanticPropertyPaths.ts b/src/domain/javascript/javascriptSemanticPropertyPaths.ts
index 9bee0e9b7a..95cbeef974 100644
--- a/src/domain/javascript/javascriptSemanticPropertyPaths.ts
+++ b/src/domain/javascript/javascriptSemanticPropertyPaths.ts
@@ -35,3 +35,89 @@ export const semanticPropertyPathKeyMatches = (
const index = semanticArrayIndex(name);
return index !== null && index >= key.startIndex;
};
+
+interface PropertyPathCoverageNode {
+ terminal: boolean;
+ readonly exact: Map;
+ readonly selectors: Map<
+ string,
+ {
+ readonly key: JavaScriptSemanticPropertyPath[number];
+ readonly node: PropertyPathCoverageNode;
+ }
+ >;
+}
+
+const coverageNode = (): PropertyPathCoverageNode => ({
+ terminal: false,
+ exact: new Map(),
+ selectors: new Map(),
+});
+
+/** Index exact effect prefixes without widening distinct member paths. */
+export class SemanticPropertyPathCoverage {
+ readonly #root = coverageNode();
+
+ /** Retain a path only when no previously retained prefix covers it. */
+ retain(path: JavaScriptSemanticPropertyPath): boolean {
+ const pending = [{ node: this.#root, offset: 0 }];
+ while (pending.length > 0) {
+ const current = pending.pop();
+ if (current === undefined) break;
+ if (current.node.terminal) return false;
+ const key = path[current.offset];
+ if (key === undefined) continue;
+ if (key !== null && typeof key !== "object") {
+ const exact = current.node.exact.get(String(key));
+ if (exact !== undefined)
+ pending.push({ node: exact, offset: current.offset + 1 });
+ }
+ for (const selected of current.node.selectors.values()) {
+ if (keyCovers(selected.key, key))
+ pending.push({ node: selected.node, offset: current.offset + 1 });
+ }
+ }
+ let node = this.#root;
+ for (const key of path) {
+ if (key !== null && typeof key !== "object") {
+ const name = String(key);
+ let child = node.exact.get(name);
+ if (child === undefined) {
+ child = coverageNode();
+ node.exact.set(name, child);
+ }
+ node = child;
+ } else {
+ const identity = JSON.stringify(key);
+ let selected = node.selectors.get(identity);
+ if (selected === undefined) {
+ selected = { key, node: coverageNode() };
+ node.selectors.set(identity, selected);
+ }
+ node = selected.node;
+ }
+ }
+ node.terminal = true;
+ node.exact.clear();
+ node.selectors.clear();
+ return true;
+ }
+}
+
+const keyCovers = (
+ covering: JavaScriptSemanticPropertyPath[number],
+ key: JavaScriptSemanticPropertyPath[number],
+): boolean => {
+ if (covering === null) return true;
+ if (key === null) return false;
+ if (typeof key !== "object")
+ return typeof covering === "object"
+ ? semanticPropertyPathKeyMatches(covering, String(key))
+ : String(covering) === String(key);
+ return (
+ typeof covering === "object" &&
+ covering.excludedKeys.every((name) => key.excludedKeys.includes(name)) &&
+ (covering.startIndex === undefined ||
+ (key.startIndex !== undefined && covering.startIndex <= key.startIndex))
+ );
+};
diff --git a/src/domain/javascript/javascriptSemanticReturnedReferences.ts b/src/domain/javascript/javascriptSemanticReturnedReferences.ts
new file mode 100644
index 0000000000..998ec06b40
--- /dev/null
+++ b/src/domain/javascript/javascriptSemanticReturnedReferences.ts
@@ -0,0 +1,213 @@
+import * as t from "@babel/types";
+
+import {
+ semanticStaticPropertyKey,
+ unwrapJavaScriptExpression,
+} from "./javascriptAstValues.js";
+import {
+ resolveLocalCallables,
+ type LocalCallableResolution,
+} from "./javascriptSemanticCallResolution.js";
+import { semanticCallableResultExpressions } from "./javascriptSemanticCallableResults.js";
+import {
+ semanticArrayIndex,
+ type JavaScriptSemanticPropertyPath,
+} from "./javascriptSemanticPropertyPaths.js";
+import {
+ resolveSemanticBindingState,
+ type JavaScriptSemanticAnalysisState,
+} from "./javascriptSemanticState.js";
+
+/** A reference potentially handed to a caller or generator consumer. */
+export interface SemanticReturnedReference {
+ readonly node: t.Node;
+ readonly projection: JavaScriptSemanticPropertyPath;
+}
+
+interface Candidate {
+ readonly node: t.Node;
+ readonly path: readonly (string | number | null)[];
+ readonly bindings: ReadonlySet;
+}
+
+/** Resolve local result references conservatively, without claiming call execution. */
+export const createSemanticReturnedReferences = (
+ state: JavaScriptSemanticAnalysisState,
+): ((callee: t.Node) => readonly SemanticReturnedReference[]) => {
+ const callableById = new Map(
+ state.callables.map((callable) => [callable.callableId, callable]),
+ );
+ const bindingCache = new Map();
+ const results = new WeakMap();
+ return (callee) => {
+ const existing = results.get(callee);
+ if (existing !== undefined) return existing;
+ const functions = new Set();
+ const resolution = resolveLocalCallables({
+ node: callee,
+ state,
+ callableById,
+ bindingCache,
+ seenBindings: new Set(),
+ });
+ for (const id of resolution.callableIds) {
+ const node = state.callableNodesById.get(id);
+ if (node !== undefined) functions.add(node);
+ }
+ const pending: Candidate[] = [
+ { node: callee, path: [], bindings: new Set() },
+ ];
+ for (let cursor = 0; cursor < pending.length; cursor++) {
+ const item = pending[cursor];
+ if (item === undefined) break;
+ const node = unwrapJavaScriptExpression(item.node).node;
+ if (t.isIdentifier(node)) {
+ const binding = resolveSemanticBindingState(state, node, node.name);
+ if (binding === undefined || item.bindings.has(binding.bindingId))
+ continue;
+ const bindings = new Set([...item.bindings, binding.bindingId]);
+ for (const initializer of [
+ ...binding.initializers,
+ ...binding.referenceInitializers,
+ ])
+ pending.push({
+ node: initializer.node,
+ path: [...initializer.projection, ...item.path],
+ bindings,
+ });
+ } else if (
+ t.isMemberExpression(node) ||
+ t.isOptionalMemberExpression(node)
+ ) {
+ const path: (string | number | null)[] = [];
+ let receiver: t.Node = node;
+ while (
+ t.isMemberExpression(receiver) ||
+ t.isOptionalMemberExpression(receiver)
+ ) {
+ path.push(
+ semanticStaticPropertyKey(receiver.property, receiver.computed),
+ );
+ receiver = unwrapJavaScriptExpression(receiver.object).node;
+ }
+ pending.push({
+ ...item,
+ node: receiver,
+ path: [...path.reverse(), ...item.path],
+ });
+ } else if (
+ t.isConditionalExpression(node) ||
+ t.isLogicalExpression(node)
+ ) {
+ pending.push(
+ {
+ ...item,
+ node: t.isConditionalExpression(node) ? node.consequent : node.left,
+ },
+ {
+ ...item,
+ node: t.isConditionalExpression(node) ? node.alternate : node.right,
+ },
+ );
+ } else if (t.isSequenceExpression(node)) {
+ const last = node.expressions.at(-1);
+ if (last !== undefined) pending.push({ ...item, node: last });
+ } else if (t.isFunction(node) && item.path.length === 0)
+ functions.add(node);
+ else if (t.isObjectExpression(node))
+ selectObjectTargets(node, item, pending);
+ else if (t.isArrayExpression(node)) {
+ const [key, ...path] = item.path;
+ const index =
+ key === undefined || key === null ? null : semanticArrayIndex(key);
+ let uncertainOffset = false;
+ node.elements.forEach((element, offset) => {
+ if (element === null) return;
+ if (t.isSpreadElement(element)) {
+ uncertainOffset = true;
+ pending.push({
+ ...item,
+ node: element.argument,
+ path: [null, ...path],
+ });
+ } else if (index === null || uncertainOffset || index === offset)
+ pending.push({ ...item, node: element, path });
+ });
+ } else if (t.isNewExpression(node))
+ pending.push({ ...item, node: node.callee });
+ else if (t.isClass(node)) {
+ const [key, ...path] = item.path;
+ for (const member of node.body.body) {
+ if (t.isClassMethod(member)) {
+ if (
+ (item.path.length === 0 && member.kind === "constructor") ||
+ (key !== undefined &&
+ (key === null ||
+ semanticStaticPropertyKey(member.key, member.computed) ===
+ String(key)))
+ )
+ pending.push({ ...item, node: member, path });
+ } else if (
+ t.isClassProperty(member) &&
+ member.value !== null &&
+ member.value !== undefined &&
+ key !== undefined &&
+ (key === null ||
+ semanticStaticPropertyKey(member.key, member.computed) ===
+ String(key))
+ )
+ pending.push({ ...item, node: member.value, path });
+ }
+ if (node.superClass !== null && node.superClass !== undefined)
+ pending.push({ ...item, node: node.superClass });
+ }
+ }
+ const output = [...functions].flatMap((node) => {
+ const callables = t.isClass(node)
+ ? node.body.body.filter(
+ (member) =>
+ t.isClassMethod(member) && member.kind === "constructor",
+ )
+ : [node];
+ return callables.flatMap((callable) =>
+ semanticCallableResultExpressions(callable).flatMap(
+ ({ node, delegated }) =>
+ node === null
+ ? []
+ : [{ node, projection: delegated ? [null] : [] }],
+ ),
+ );
+ });
+ results.set(callee, output);
+ return output;
+ };
+};
+
+const selectObjectTargets = (
+ node: t.ObjectExpression,
+ item: Candidate,
+ pending: Candidate[],
+): void => {
+ const [key, ...path] = item.path;
+ let replaced = false;
+ for (const property of [...node.properties].reverse()) {
+ if (replaced) break;
+ if (t.isSpreadElement(property)) {
+ pending.push({ ...item, node: property.argument });
+ continue;
+ }
+ const name = semanticStaticPropertyKey(property.key, property.computed);
+ if (
+ key !== null &&
+ key !== undefined &&
+ name !== null &&
+ name !== String(key)
+ )
+ continue;
+ if (t.isObjectProperty(property))
+ pending.push({ ...item, node: property.value, path });
+ else pending.push({ ...item, node: property, path });
+ if (key !== null && key !== undefined && name === String(key))
+ replaced = true;
+ }
+};
diff --git a/src/domain/javascript/javascriptSemanticReturns.ts b/src/domain/javascript/javascriptSemanticReturns.ts
index bb931fba51..fb51279bf0 100644
--- a/src/domain/javascript/javascriptSemanticReturns.ts
+++ b/src/domain/javascript/javascriptSemanticReturns.ts
@@ -26,10 +26,8 @@ import {
import { evaluateSemanticExpression } from "./javascriptSemanticValues.js";
import { range } from "./javascriptStaticAnalysisHelpers.js";
import { semanticReturnCoverage } from "./javascriptSemanticCoverage.js";
-import {
- childNodes,
- traverseJavaScriptAst,
-} from "./javascriptSemanticTraversal.js";
+import { traverseJavaScriptAst } from "./javascriptSemanticTraversal.js";
+import { semanticCallableResultExpressions } from "./javascriptSemanticCallableResults.js";
interface ReturnExpression {
readonly node: t.Node | null;
@@ -123,25 +121,10 @@ const callableNodes = (program: t.Program): Map => {
return output;
};
-const directReturnExpressions = (callable: t.Node): ReturnExpression[] => {
- if (
- t.isArrowFunctionExpression(callable) &&
- !t.isBlockStatement(callable.body)
- )
- return [{ node: callable.body, location: range(callable.body) }];
- if (!t.isFunction(callable)) return [];
- const output: ReturnExpression[] = [];
- const visit = (node: t.Node): void => {
- if (t.isFunction(node) || t.isClass(node)) return;
- if (t.isReturnStatement(node)) {
- output.push({ node: node.argument ?? null, location: range(node) });
- return;
- }
- for (const child of childNodes(node)) visit(child);
- };
- for (const child of childNodes(callable.body)) visit(child);
- return output;
-};
+const directReturnExpressions = (callable: t.Node): ReturnExpression[] =>
+ semanticCallableResultExpressions(callable)
+ .filter(({ kind }) => kind === "return")
+ .map(({ node, site }) => ({ node, location: range(site) }));
const callableIdsForBinding = (
binding: JavaScriptSemanticBindingState,
diff --git a/src/domain/javascript/javascriptSemanticValues.ts b/src/domain/javascript/javascriptSemanticValues.ts
index 2f53626708..87f6ad540e 100644
--- a/src/domain/javascript/javascriptSemanticValues.ts
+++ b/src/domain/javascript/javascriptSemanticValues.ts
@@ -10,7 +10,7 @@ import { semanticSlotAtPath } from "./javascriptSemanticSlots.js";
import {
invalidateSemanticEscapedPath,
- invalidateSemanticMutationPath,
+ invalidateSemanticMutationPaths,
} from "./javascriptSemanticMutationValues.js";
import type { JavaScriptBindingProvenance } from "./javascriptSemanticIr.js";
@@ -211,12 +211,11 @@ const evaluateBinding = (
if (isPrimitive(captured)) value = captured;
}
}
- const mutated = binding.mutatedPaths.reduce(
- invalidateSemanticMutationPath,
- value,
- );
- const projected = binding.escapedPaths.reduce(
- (value, escape) =>
+ let projected = invalidateSemanticMutationPaths(value, binding.mutatedPaths);
+ for (const escape of binding.escapedPaths) {
+ // Escapes only invalidate object references; other lattice values survive.
+ if (projected.status !== "object" && projected.status !== "array") break;
+ if (
context.capturePoint !== undefined &&
semanticEscapeFollowsCapture(
binding,
@@ -224,10 +223,10 @@ const evaluateBinding = (
escape.node,
context.state.parentsByNode,
)
- ? value
- : invalidateSemanticEscapedPath(value, escape.path),
- mutated,
- );
+ )
+ continue;
+ projected = invalidateSemanticEscapedPath(projected, escape.path);
+ }
// Point-local memoization also bounds failed speculative captures. Only a
// primitive result is allowed back into ordinary value evaluation.
if (
diff --git a/src/domain/javascript/javascriptStaticAnalysis.ts b/src/domain/javascript/javascriptStaticAnalysis.ts
index af20eaa926..524af811eb 100644
--- a/src/domain/javascript/javascriptStaticAnalysis.ts
+++ b/src/domain/javascript/javascriptStaticAnalysis.ts
@@ -25,9 +25,11 @@ import { finalizeLocatedFindings } from "./javascriptStaticAnalysisFindings.js";
import {
createJavaScriptAnalysisAccumulator,
type JavaScriptAnalysisAccumulator as AnalysisAccumulator,
+ type JavaScriptFindingContext,
} from "./javascriptStaticAnalysisState.js";
import type { JavaScriptStaticAnalysis } from "./javascriptStaticAnalysisTypes.js";
import {
+ classifyParsedJavaScriptElectronBindings,
classifyParsedJavaScriptOpenReceivers,
type JavaScriptOpenReceiverFact,
} from "./javascriptSemanticAnalysis.js";
@@ -57,21 +59,24 @@ export const analyzeParsedJavaScriptStaticSource = (
openReceiverFacts?: ReadonlyMap,
): JavaScriptStaticAnalysis => {
const accumulator = createJavaScriptAnalysisAccumulator(source.length);
- traverseStaticSource(source, file, accumulator, openReceiverFacts);
+ traverseStaticSource(file, {
+ source,
+ accumulator,
+ ...(openReceiverFacts === undefined ? {} : { openReceiverFacts }),
+ electronBindings: classifyParsedJavaScriptElectronBindings(file),
+ });
addSourceMapDirectives(source, file.comments ?? [], accumulator);
return finalizeStaticAnalysis(source, file, accumulator);
};
const traverseStaticSource = (
- source: string,
file: ParsedJavaScriptSource,
- accumulator: AnalysisAccumulator,
- openReceiverFacts?: ReadonlyMap,
+ findings: JavaScriptFindingContext,
): void => {
traverseJavaScriptAst(file, {
enter: (node) => {
- accumulator.visitedNodes += 1;
- inspectNode(source, node, accumulator, openReceiverFacts);
+ findings.accumulator.visitedNodes += 1;
+ inspectNode(node, findings);
return undefined;
},
});
@@ -156,16 +161,10 @@ const finalizeStaticAnalysis = (
};
const inspectNode = (
- source: string,
node: t.Node,
- accumulator: AnalysisAccumulator,
- openReceiverFacts?: ReadonlyMap,
+ findings: JavaScriptFindingContext,
): void => {
- const findings = {
- source,
- accumulator,
- ...(openReceiverFacts === undefined ? {} : { openReceiverFacts }),
- };
+ const { source, accumulator } = findings;
inspectElectronStaticNode(node, findings);
if (t.isCallExpression(node)) {
inspectBundlerRegistration(source, node, accumulator);
diff --git a/src/domain/javascript/javascriptStaticAnalysisState.ts b/src/domain/javascript/javascriptStaticAnalysisState.ts
index 23de6952bf..47e25bccdb 100644
--- a/src/domain/javascript/javascriptStaticAnalysisState.ts
+++ b/src/domain/javascript/javascriptStaticAnalysisState.ts
@@ -65,6 +65,8 @@ export interface JavaScriptFindingContext {
| "local-indexed-db"
| "local-cache-storage"
>;
+ /** Electron export paths for renamed call roots, by identifier offset. */
+ readonly electronBindings?: ReadonlyMap;
}
/** Candidate import, require, worker, or service-worker reference. */
diff --git a/src/domain/process/processScenario.ts b/src/domain/process/processScenario.ts
index 8312e1d2a1..ff9197d158 100644
--- a/src/domain/process/processScenario.ts
+++ b/src/domain/process/processScenario.ts
@@ -108,6 +108,10 @@ export const processComparisonContract = (
...(scenario.normalization.ports
? { port_normalization_version: "contextual-endpoints-v2" }
: {}),
+ ...(scenario.normalization.patterns.length > 0
+ ? { pattern_normalization_version: "literal-replacements-v1" }
+ : {}),
+ rendered_line_format: "trailing-spaces-trimmed-v1",
});
/**
diff --git a/src/domain/zipPackageFormat.ts b/src/domain/zipPackageFormat.ts
index c45109135a..39a8f86a58 100644
--- a/src/domain/zipPackageFormat.ts
+++ b/src/domain/zipPackageFormat.ts
@@ -22,7 +22,8 @@ export const zipPackageFormatForPath = (
const lower = path.toLowerCase();
if (lower.endsWith(".zip")) return "zip";
if (lower.endsWith(".ipa")) return "ipa";
- if (lower.endsWith(".apk") || lower.endsWith(".aab")) return "apk";
+ // An Android App Bundle is a ZIP archive, not a standalone APK.
+ if (lower.endsWith(".apk")) return "apk";
if (lower.endsWith(".msix") || lower.endsWith(".msixbundle")) return "msix";
if (lower.endsWith(".appx") || lower.endsWith(".appxbundle")) return "appx";
return undefined;
diff --git a/src/generatedPackageMetadata.ts b/src/generatedPackageMetadata.ts
index 29177bf02f..468117b055 100644
--- a/src/generatedPackageMetadata.ts
+++ b/src/generatedPackageMetadata.ts
@@ -5,5 +5,5 @@ export const PACKAGE_METADATA = {
serverSdkVersion: "2.3.1",
clientSdkVersion: "2.3.1",
coreSdkVersion: "2.3.1",
- skillVersion: "34",
+ skillVersion: "35",
} as const;
diff --git a/src/ghidra/GhidraAnalysisProfile.ts b/src/ghidra/GhidraAnalysisProfile.ts
index 1af7469fa0..eeb8646a41 100644
--- a/src/ghidra/GhidraAnalysisProfile.ts
+++ b/src/ghidra/GhidraAnalysisProfile.ts
@@ -34,6 +34,12 @@ export const resolveGhidraAnalysisProfile = (
profile: createAnalysisProfile(provider, {
target_kind: target.kind,
target_format: target.format,
+ ...(target.format === "pe"
+ ? {
+ executable_role: target.executableRole ?? null,
+ managed: target.managed ?? null,
+ }
+ : {}),
architecture: target.architecture ?? null,
available_architectures: [
...(target.availableArchitectures ?? []),
diff --git a/src/ghidra/GhidraProvider.behavior.test.ts b/src/ghidra/GhidraProvider.behavior.test.ts
index 7bbbc8c89d..c0518297b3 100644
--- a/src/ghidra/GhidraProvider.behavior.test.ts
+++ b/src/ghidra/GhidraProvider.behavior.test.ts
@@ -198,6 +198,32 @@ describe("Ghidra Mach-O slice support", () => {
});
describe("Ghidra platform support", () => {
+ it("commits the PE role and managed classification to profile identity", async () => {
+ const ghidra = provider({ ...installationHost(), platform: "win32" });
+ const application = peTarget("x86_64");
+ const library = {
+ ...application,
+ executableRole: "shared-library",
+ } as const;
+ const digests = new Set();
+ for (const target of [
+ application,
+ library,
+ { ...library, managed: true },
+ ]) {
+ const resolved = await ghidra.resolveAnalysisProfile(target);
+ if (!resolved.ok) throw resolved.error;
+ const profile = resolved.value.profile;
+ if (profile === null) throw new Error("Expected a PE analysis profile");
+ expect(profile.parameters).toMatchObject({
+ executable_role: target.executableRole,
+ managed: target.managed,
+ });
+ digests.add(profile.digest);
+ }
+ expect(digests.size).toBe(3);
+ });
+
it("keeps Windows annotation mutation unavailable independently of native controls", () => {
const ghidra = provider({ ...installationHost(), platform: "win32" });
expect(
@@ -270,7 +296,7 @@ describe("Ghidra platform support", () => {
expect(
ghidra.inspectTargetSupport({
...nativeApplication,
- executableRole: "shared-library",
+ executableRole: "non-executable",
}),
).toMatchObject({
status: "unsupported",
@@ -295,45 +321,59 @@ describe("Ghidra platform support", () => {
code: "architecture_unsupported",
});
});
- it("admits generated PE32 applications while rejecting unsupported x86 roles", async () => {
+});
+
+describe("Ghidra Windows PE fixture admission", () => {
+ it("admits generated PE32 and PE32+ DLLs and applications while preserving native-only Windows admission", async () => {
const root = fileURLToPath(new URL("../../", import.meta.url));
await promisify(execFile)(process.execPath, [
join(root, "scripts/create-ghidra-windows-fixture.mjs"),
]);
- const path = join(root, "build/fixtures/rea-ghidra-windows-x86.exe");
- const bytes = await readFile(path);
- const metadata = parseExecutableHeader(bytes, "x64");
- expect(metadata).toMatchObject({
- ok: true,
- value: {
- format: "pe",
- architecture: "x86",
- executableRole: "application",
- managed: false,
- },
- });
- if (!metadata.ok) throw new Error(metadata.error);
- if (metadata.value.format !== "pe")
- throw new Error("Expected a PE fixture");
- const target: BinaryTarget = {
- path,
- sha256: createHash("sha256").update(bytes).digest("hex"),
- kind: "executable",
- ...metadata.value,
- };
const ghidra = provider({ ...installationHost(), platform: "win32" });
- expect(ghidra.inspectTargetSupport(target).status).toBe("supported");
- for (const [candidate, code] of [
- [
- { ...target, executableRole: "shared-library" },
- "target_role_unsupported",
- ],
- [{ ...target, managed: true }, "managed_target_unsupported"],
- ] as const)
- expect(ghidra.inspectTargetSupport(candidate)).toMatchObject({
- status: "unsupported",
- code,
- });
+ for (const architecture of ["x86", "x86_64"] as const) {
+ for (const dll of [false, true]) {
+ const path = join(
+ root,
+ "build/fixtures",
+ `rea-ghidra-windows${architecture === "x86" ? "-x86" : ""}.${dll ? "dll" : "exe"}`,
+ );
+ const bytes = await readFile(path);
+ const metadata = parseExecutableHeader(bytes, "x64");
+ const role = dll ? "shared-library" : "application";
+ expect(metadata).toMatchObject({
+ ok: true,
+ value: {
+ format: "pe",
+ architecture,
+ executableRole: role,
+ managed: false,
+ },
+ });
+ if (!metadata.ok) throw new Error(metadata.error);
+ if (metadata.value.format !== "pe")
+ throw new Error("Expected a PE fixture");
+ const target: BinaryTarget = {
+ path,
+ sha256: createHash("sha256").update(bytes).digest("hex"),
+ kind: "executable",
+ ...metadata.value,
+ };
+ expect(ghidra.inspectTargetSupport(target).status).toBe("supported");
+ for (const [candidate, code] of [
+ [
+ { ...target, executableRole: "non-executable" },
+ "target_role_unsupported",
+ ],
+ [{ ...target, managed: true }, "managed_target_unsupported"],
+ [{ ...target, architecture: "arm64" }, "architecture_unsupported"],
+ ] as const) {
+ expect(ghidra.inspectTargetSupport(candidate)).toMatchObject({
+ status: "unsupported",
+ code,
+ });
+ }
+ }
+ }
});
});
@@ -500,24 +540,23 @@ describe("Ghidra client projection", () => {
});
describe("Ghidra result projection", () => {
+ const createElfClient = async (ghidra: GhidraProvider) => {
+ const target = executableTarget("elf", "x86_64");
+ const resolved = await ghidra.resolveAnalysisProfile(target);
+ if (!resolved.ok) throw resolved.error;
+ if (resolved.value.profile === null)
+ throw new Error("Expected a bound Ghidra profile");
+ return ghidra.createClient(target, resolved.value.profile);
+ };
+
it("rejects malformed inventory output before Evidence creation", async () => {
const ghidra = provider(installationHost(), () => ({
start: () => Promise.resolve(ok(sessionInfo())),
callTool: () => Promise.resolve(ok({ items: "not-an-inventory" })),
close: () => Promise.resolve(ok(null)),
}));
- const resolved = await ghidra.resolveAnalysisProfile(
- executableTarget("elf", "x86_64"),
- );
- if (!resolved.ok) throw resolved.error;
- if (resolved.value.profile === null)
- throw new Error("Expected a bound Ghidra profile");
-
- await expect(
- ghidra
- .createClient(executableTarget("elf", "x86_64"), resolved.value.profile)
- .execute("list_procedures", {}),
- ).resolves.toMatchObject({
+ const client = await createElfClient(ghidra);
+ await expect(client.execute("list_procedures", {})).resolves.toMatchObject({
ok: false,
error: { _tag: "AnalysisOutputError" },
});
@@ -546,21 +585,13 @@ describe("Ghidra result projection", () => {
),
close: () => Promise.resolve(ok(null)),
}));
- const resolved = await ghidra.resolveAnalysisProfile(
- executableTarget("elf", "x86_64"),
- );
- if (!resolved.ok) throw resolved.error;
- if (resolved.value.profile === null)
- throw new Error("Expected a bound Ghidra profile");
-
+ const client = await createElfClient(ghidra);
await expect(
- ghidra
- .createClient(
- executableTarget("elf", "x86_64"),
- resolved.value.profile,
- )
- .execute("list_procedures", {}),
- ).resolves.toMatchObject({ ok: false, error: { _tag: tag } });
+ client.execute("list_procedures", {}),
+ ).resolves.toMatchObject({
+ ok: false,
+ error: { _tag: tag },
+ });
},
);
@@ -582,19 +613,12 @@ describe("Ghidra result projection", () => {
),
close: () => Promise.resolve(ok(null)),
}));
- const resolved = await ghidra.resolveAnalysisProfile(
- executableTarget("elf", "x86_64"),
- );
- if (!resolved.ok) throw resolved.error;
- if (resolved.value.profile === null)
- throw new Error("Expected a bound profile");
+ const client = await createElfClient(ghidra);
await expect(
- ghidra
- .createClient(executableTarget("elf", "x86_64"), resolved.value.profile)
- .execute("annotate_native_function", {
- procedure: "0x10100",
- name: "bad name",
- }),
+ client.execute("annotate_native_function", {
+ procedure: "0x10100",
+ name: "bad name",
+ }),
).resolves.toMatchObject({
ok: false,
error: {
@@ -622,16 +646,10 @@ describe("Ghidra result projection", () => {
),
close: () => Promise.resolve(ok(null)),
}));
- const resolved = await ghidra.resolveAnalysisProfile(
- executableTarget("elf", "x86_64"),
- );
- if (!resolved.ok) throw resolved.error;
- if (resolved.value.profile === null)
- throw new Error("Expected a bound Ghidra profile");
-
- const result = await ghidra
- .createClient(executableTarget("elf", "x86_64"), resolved.value.profile)
- .execute("procedure_pseudo_code", { procedure: "main" });
+ const client = await createElfClient(ghidra);
+ const result = await client.execute("procedure_pseudo_code", {
+ procedure: "main",
+ });
expect(result).toMatchObject({
ok: false,
error: { _tag: tag },
diff --git a/src/ghidra/GhidraProvider.ts b/src/ghidra/GhidraProvider.ts
index d657a156e3..d782827d96 100644
--- a/src/ghidra/GhidraProvider.ts
+++ b/src/ghidra/GhidraProvider.ts
@@ -247,12 +247,15 @@ const inspectWindowsP0TargetSupport = (
reason: "Windows Ghidra P0 accepts x86 and x86-64 PE targets only.",
diagnostics,
};
- if (target.executableRole !== "application")
+ if (
+ target.executableRole !== "application" &&
+ target.executableRole !== "shared-library"
+ )
return {
status: "unsupported",
code: "target_role_unsupported",
reason:
- "Windows Ghidra P0 accepts PE applications, not DLL or non-executable images.",
+ "Windows Ghidra P0 accepts PE applications and DLLs, not non-executable or unclassified images.",
diagnostics,
};
if (target.managed !== false)
@@ -260,7 +263,7 @@ const inspectWindowsP0TargetSupport = (
status: "unsupported",
code: "managed_target_unsupported",
reason:
- "Windows Ghidra P0 accepts native PE applications; managed or unclassified PE targets are unsupported.",
+ "Windows Ghidra P0 accepts native PE applications and DLLs; managed or unclassified PE targets are unsupported.",
diagnostics,
};
return {
diff --git a/src/ghidra/GhidraProviderCapabilities.ts b/src/ghidra/GhidraProviderCapabilities.ts
index edbbb11c31..5b9a6ca8ba 100644
--- a/src/ghidra/GhidraProviderCapabilities.ts
+++ b/src/ghidra/GhidraProviderCapabilities.ts
@@ -31,7 +31,7 @@ export const healthLimitations = Object.freeze([
/** Additional limitations applied to the experimental Windows x64 P0 boundary. */
export const windowsP0Limitations = Object.freeze([
- "Windows Ghidra P0 accepts approved native x86 and x86-64 PE applications only; DLL, managed, hostile, sensitive, and mutable-path targets are unsupported.",
+ "Windows Ghidra P0 accepts approved native x86 and x86-64 PE applications and DLLs; managed, hostile, sensitive, and mutable-path targets are unsupported.",
"The Windows bridge uses authenticated IPv4 loopback because Node path-based IPC does not expose Java AF_UNIX sockets; the endpoint file contains no bearer token.",
"Windows sessions require the matching packaged Windows x64 native addon, local NTFS targets and runtimes, and Windows 10 or later. Native handles enforce path admission, private DACLs, and Job Object ownership automatically.",
]);
diff --git a/src/hopper/HopperProvider.ts b/src/hopper/HopperProvider.ts
index 5b1da0ba9b..6c314ddb68 100644
--- a/src/hopper/HopperProvider.ts
+++ b/src/hopper/HopperProvider.ts
@@ -249,7 +249,8 @@ export class HopperProvider implements AnalysisProviderCandidate {
? {
ok: true,
value: createAnalysisExecution(mapped.value, executionProvider, {
- rawResult: result.value,
+ // Only the file-offset mapping differs from Hopper's reply.
+ rawResult: mapped === result ? null : result.value,
...(profile === undefined ? {} : { analysisProfile: profile }),
limitations: [
...(preparedImage === undefined
diff --git a/src/mcpDoctor.ts b/src/mcpDoctor.ts
index 0f5de39fce..62a5882807 100644
--- a/src/mcpDoctor.ts
+++ b/src/mcpDoctor.ts
@@ -6,6 +6,7 @@ import { resolve } from "node:path";
import { CATALOG_IDENTITY } from "./catalogIdentity.js";
import { PROMPT_CONTRACTS } from "./contracts/promptContracts.js";
import { TOOL_CONTRACTS } from "./contracts/toolContracts.js";
+import { err, ok } from "./domain/result.js";
import { PRODUCT_IDENTITY } from "./identity.js";
import { MCP_STARTUP_POLICY } from "./mcpStartupPolicy.js";
@@ -131,11 +132,22 @@ const inspectProductionMcpSession = async (
signal: AbortSignal,
) => {
const request = requestOptions(deadline, signal);
- const [tools, prompts, requestFlow] = await Promise.all([
+ const [tools, prompts] = await Promise.all([
client.listTools(undefined, request),
client.listPrompts(undefined, request),
- client.callTool({ name: "binary_session", arguments: {} }, request),
]);
+ const sessionTool = tools.tools.find(({ name }) => name === "binary_session");
+ const requestFlow = await client
+ .callTool(
+ { name: "binary_session", arguments: {} },
+ {
+ ...requestOptions(deadline, signal),
+ ...(sessionTool === undefined ? {} : { toolDefinition: sessionTool }),
+ },
+ )
+ .then(ok, (cause: unknown) =>
+ err({ kind: "request-flow" as const, detail: errorMessage(cause) }),
+ );
const inventory = {
tools: compareInventory(
TOOL_CONTRACTS.map(({ name }) => name),
@@ -147,7 +159,7 @@ const inspectProductionMcpSession = async (
),
};
const observedIdentity = parseIdentityToolResult(
- requestFlow.structuredContent,
+ requestFlow.ok ? requestFlow.value.structuredContent : undefined,
);
const serverVersion = client.getServerVersion();
const protocolVersion = client.getNegotiatedProtocolVersion();
@@ -173,8 +185,10 @@ const inspectProductionMcpSession = async (
},
{
name: "request-flow",
- ok: requestFlow.isError !== true,
- detail: "binary_session target-free request",
+ ok: requestFlow.ok && requestFlow.value.isError !== true,
+ detail: requestFlow.ok
+ ? "binary_session target-free request"
+ : requestFlow.error.detail,
},
];
return {
@@ -194,7 +208,7 @@ const inspectProductionMcpSession = async (
},
request_flow: {
tool: "binary_session",
- ok: requestFlow.isError !== true,
+ ok: requestFlow.ok && requestFlow.value.isError !== true,
},
checks,
};
diff --git a/src/native/NativeMacOSProvider.ts b/src/native/NativeMacOSProvider.ts
index 12c94814fc..d63db25061 100644
--- a/src/native/NativeMacOSProvider.ts
+++ b/src/native/NativeMacOSProvider.ts
@@ -233,7 +233,6 @@ class NativeMacOSClient implements AnalysisClient {
return observation.ok
? ok(
createAnalysisExecution(observation.value.result, IDENTITY, {
- rawResult: { provenance: observation.value.provenance },
limitations: observation.value.limitations,
locations: observation.value.locations,
}),
@@ -317,7 +316,6 @@ class NativeMacOSClient implements AnalysisClient {
});
return ok({
result: jsonValueSchema.parse(result),
- provenance,
limitations: [],
locations: architectureLocations(result.architectures.items),
});
@@ -351,7 +349,6 @@ class NativeMacOSClient implements AnalysisClient {
});
return ok({
result: jsonValueSchema.parse(result),
- provenance,
limitations: [],
locations: [],
});
@@ -448,7 +445,6 @@ class NativeMacOSClient implements AnalysisClient {
});
return ok({
result: jsonValueSchema.parse(result),
- provenance,
limitations: result.limitations,
locations: [],
});
@@ -611,7 +607,6 @@ class NativeMacOSClient implements AnalysisClient {
});
return ok({
result: jsonValueSchema.parse(result),
- provenance,
limitations: parsed.value.limitations,
locations: [{ kind: "artifact-path", path: plist.value }],
});
@@ -659,7 +654,6 @@ class NativeMacOSClient implements AnalysisClient {
interface NativeObservation {
readonly result: JsonValue;
- readonly provenance: readonly NativeCommandInvocation[];
readonly limitations: readonly string[];
readonly locations: readonly EvidenceLocation[];
}
diff --git a/src/native/NativeMachoInspection.ts b/src/native/NativeMachoInspection.ts
index 417b43fef5..2ef2c95ae9 100644
--- a/src/native/NativeMachoInspection.ts
+++ b/src/native/NativeMachoInspection.ts
@@ -15,7 +15,6 @@ import { parseOtoolLoadCommands } from "./parsers/otool.js";
interface NativeMachoObservation {
readonly result: JsonValue;
- readonly provenance: readonly NativeCommandInvocation[];
readonly limitations: readonly string[];
readonly locations: readonly EvidenceLocation[];
}
@@ -116,7 +115,6 @@ export const inspectNativeMacho = async (
: normalized;
return ok({
result: jsonValueSchema.parse(result),
- provenance: result.provenance,
limitations: result.limitations,
locations: fileOffsetLocations(result, segmentOffset),
});
diff --git a/src/process/capture/ProcessNormalization.test.ts b/src/process/capture/ProcessNormalization.test.ts
index 9631c66f3f..9c9ba1951b 100644
--- a/src/process/capture/ProcessNormalization.test.ts
+++ b/src/process/capture/ProcessNormalization.test.ts
@@ -59,18 +59,68 @@ const scenario = (normalization: {
readonly pids?: boolean;
readonly paths?: boolean;
readonly ports?: boolean;
+ readonly patterns?: readonly {
+ readonly pattern: string;
+ readonly replacement: string;
+ }[];
}) =>
parseProcessScenario({
executable: "/usr/bin/node",
working_directory: "/workspace",
normalization: {
paths: true,
- ...normalization,
time_bucket_ms: 10,
patterns: [{ pattern: "marker", replacement: "selected" }],
+ ...normalization,
},
});
+it("replaces caller-declared patterns literally, without expanding replacement patterns", () => {
+ const declared = scenario({
+ paths: false,
+ patterns: [
+ { pattern: "8080", replacement: "$&$&" },
+ { pattern: "READY", replacement: "$$" },
+ { pattern: "NAME", replacement: "$1" },
+ ],
+ });
+ expect(
+ normalizeProcessText(
+ "listening on 8080; READY; NAME=alpha; tail NAME",
+ declared,
+ "/temporary",
+ rootPid,
+ ),
+ ).toBe("listening on $&$&; $$; $1=alpha; tail $1");
+});
+
+it("preserves empty-pattern boundaries, including empty text", () => {
+ const declared = scenario({
+ patterns: [{ pattern: "", replacement: "$$" }],
+ });
+ expect(normalizeProcessText("ab", declared, "/temporary", rootPid)).toBe(
+ "$$a$$b$$",
+ );
+ expect(normalizeProcessText("", declared, "/temporary", rootPid)).toBe("$$");
+});
+
+it.each(["$`", "$'"])(
+ "retains the selected replacement %s verbatim",
+ (replacement) => {
+ const declared = scenario({
+ patterns: [{ pattern: "marker", replacement }],
+ });
+ expect(
+ normalizeProcessText(
+ "before marker after",
+ declared,
+ "/temporary",
+ rootPid,
+ ),
+ ).toBe(`before ${replacement} after`);
+ },
+);
+
it("preserves compact JSON, counters, line numbers, and ambiguous endpoint spellings", () => {
const input = [
'{"pid":32,"size":108,"height":480,"width":640,"score":20}',
diff --git a/src/process/capture/ProcessNormalization.ts b/src/process/capture/ProcessNormalization.ts
index 24209012cb..9364ec95b0 100644
--- a/src/process/capture/ProcessNormalization.ts
+++ b/src/process/capture/ProcessNormalization.ts
@@ -34,8 +34,13 @@ export const normalizeProcessText = (
normalized = normalizePortTokens(normalized);
if (scenario.normalization.pids)
normalized = normalizePidTokens(normalized, [pid]);
+ // A callback keeps replacement text literal while preserving replaceAll's
+ // empty-pattern boundaries, which the process scenario contract permits.
for (const pattern of scenario.normalization.patterns)
- normalized = normalized.replaceAll(pattern.pattern, pattern.replacement);
+ normalized = normalized.replaceAll(
+ pattern.pattern,
+ () => pattern.replacement,
+ );
return normalized;
};
diff --git a/src/process/capture/TerminalRenderer.test.ts b/src/process/capture/TerminalRenderer.test.ts
new file mode 100644
index 0000000000..e5a19996b1
--- /dev/null
+++ b/src/process/capture/TerminalRenderer.test.ts
@@ -0,0 +1,47 @@
+import { expect, it } from "vitest";
+
+import { TerminalRenderer } from "./TerminalRenderer.js";
+
+const renderFrames = async (
+ data: string,
+ normalize: (value: string) => string = (value) => value,
+) => {
+ const renderer = new TerminalRenderer({
+ columns: 20,
+ rows: 4,
+ scrollback: 10,
+ maxBytes: 100_000,
+ normalize,
+ });
+ renderer.write(data, 0);
+ const frames = await renderer.frames();
+ const retention = renderer.retention();
+ await renderer.dispose();
+ return { frames, retention };
+};
+
+it("omits trailing blank cells from visible lines without losing rows", async () => {
+ const { frames, retention } = await renderFrames(
+ "left right \r\n\r\n indented",
+ );
+ const [frame] = frames;
+ expect(frame?.lines).toEqual(["left right", "", " indented", ""]);
+ // Fixed-width rows remain recoverable from the recorded column count.
+ expect(frame?.lines.map((line) => line.padEnd(frame.columns, " "))).toEqual([
+ "left right ",
+ " ".repeat(20),
+ " indented ",
+ " ".repeat(20),
+ ]);
+ expect(retention.retained_bytes).toBe(
+ Buffer.byteLength(frame?.serialized_state ?? "") +
+ Buffer.byteLength("left right indented"),
+ );
+});
+
+it("trims only after normalization sees the full-width row", async () => {
+ const { frames } = await renderFrames("pid 4242", (value) =>
+ value.replace(/4242 +$/u, ""),
+ );
+ expect(frames[0]?.lines[0]).toBe("pid ");
+});
diff --git a/src/process/capture/TerminalRenderer.ts b/src/process/capture/TerminalRenderer.ts
index e7f1a0fc8a..4322e2781a 100644
--- a/src/process/capture/TerminalRenderer.ts
+++ b/src/process/capture/TerminalRenderer.ts
@@ -14,6 +14,8 @@ const HeadlessPackage =
const SerializePackage =
require("@xterm/addon-serialize") as typeof import("@xterm/addon-serialize");
+const TRAILING_SPACES = / +$/u;
+
interface TerminalRendererOptions {
readonly columns: number;
readonly rows: number;
@@ -106,13 +108,16 @@ export class TerminalRenderer {
const lines: string[] = [];
for (let row = 0; row < this.#terminal.rows; row += 1) {
const line = buffer.getLine(buffer.viewportY + row);
+ // Trailing blank cells carry no information beyond `columns`; dropping
+ // them keeps sparse screens from costing a full row per blank line.
lines.push(
- this.options.normalize(
- (line?.translateToString(false, 0, this.#terminal.cols) ?? "").padEnd(
- this.#terminal.cols,
- " ",
- ),
- ),
+ this.options
+ .normalize(
+ (
+ line?.translateToString(false, 0, this.#terminal.cols) ?? ""
+ ).padEnd(this.#terminal.cols, " "),
+ )
+ .replace(TRAILING_SPACES, ""),
);
}
const serializedState = this.options.normalize(
diff --git a/src/server/EvidenceMcpServer.ts b/src/server/EvidenceMcpServer.ts
index 706ad05772..d5f5e0598f 100644
--- a/src/server/EvidenceMcpServer.ts
+++ b/src/server/EvidenceMcpServer.ts
@@ -5,14 +5,22 @@ import {
isJSONRPCNotification,
isJSONRPCResultResponse,
type Implementation,
+ type Icon,
type JSONRPCMessage,
type McpServerOptions,
+ type RegisteredTool,
type RequestId,
+ type ScopeChallengeHandler,
+ type StandardSchemaWithJSON,
+ type ToolAnnotations,
+ type ToolCallback,
type Transport,
type TransportSendOptions,
} from "@modelcontextprotocol/server";
import { analysisErrorProjectionSchema } from "../contracts/errorSchemas.js";
+import { compactAdvertisedInputSchema } from "../contracts/compactInputSchema.js";
+import { transformAdvertisedInputJsonSchema } from "../contracts/toolSchemaMetadata.js";
import type { EvidenceWriter } from "../application/investigation/InvestigationRecordPort.js";
import { AnalysisResourceConstraintError } from "../domain/analysisErrorCore.js";
import { projectAnalysisError } from "../domain/analysisErrorProjection.js";
@@ -21,6 +29,24 @@ import { jsonObjectSchema } from "../domain/jsonValue.js";
import type { ToolResultDelivery } from "./toolResult.js";
import { encodeToolResult } from "./toolResultEncoding.js";
+/** Compact advertised input schemas within this per-schema byte budget. */
+export interface CompactInputSchemaPresentation {
+ readonly budgetBytes: number;
+}
+
+/** A Zod schema whose standard interface exposes a JSON Schema projection. */
+const hasAdvertisedInputJsonSchema = (value: unknown): boolean => {
+ if (typeof value !== "object" || value === null) return false;
+ const standard = Reflect.get(value, "~standard");
+ if (typeof standard !== "object" || standard === null) return false;
+ const jsonSchema = Reflect.get(standard, "jsonSchema");
+ return (
+ typeof jsonSchema === "object" &&
+ jsonSchema !== null &&
+ Reflect.get(jsonSchema, "input") !== undefined
+ );
+};
+
/** Bind oversized MCP error recovery to this server's Evidence ledger. */
export class EvidenceMcpServer extends McpServer {
constructor(
@@ -30,10 +56,60 @@ export class EvidenceMcpServer extends McpServer {
| EvidenceWriter["recordEvidence"]
| undefined,
readonly delivery: ToolResultDelivery,
+ private readonly compactPresentation:
+ | CompactInputSchemaPresentation
+ | undefined = undefined,
) {
super(info, options);
}
+ /**
+ * Compact the advertised input schema of every registered tool when the
+ * compact profile is active. The registered canonical schema, its parser,
+ * and output schemas are untouched; only the advertised JSON Schema
+ * projection is rendered within the provider's per-schema size budget.
+ * The signatures mirror the SDK's registrations so call sites keep their
+ * inferred handler argument types.
+ */
+ override registerTool<
+ OutputArgs extends StandardSchemaWithJSON,
+ InputArgs extends StandardSchemaWithJSON | undefined = undefined,
+ >(
+ name: string,
+ config: {
+ title?: string;
+ description?: string;
+ inputSchema?: InputArgs;
+ outputSchema?: OutputArgs;
+ annotations?: ToolAnnotations;
+ icons?: Icon[];
+ scopeChallenge?: ScopeChallengeHandler;
+ _meta?: Record;
+ },
+ callback: ToolCallback,
+ ): RegisteredTool {
+ const budget = this.compactPresentation?.budgetBytes;
+ const inputSchema =
+ budget !== undefined &&
+ config !== undefined &&
+ hasAdvertisedInputJsonSchema(config.inputSchema)
+ ? transformAdvertisedInputJsonSchema(
+ config.inputSchema as Parameters<
+ typeof transformAdvertisedInputJsonSchema
+ >[0],
+ (projected) =>
+ compactAdvertisedInputSchema(projected, budget).schema,
+ )
+ : undefined;
+ return super.registerTool(
+ name,
+ (inputSchema === undefined
+ ? config
+ : { ...config, inputSchema }) as never,
+ callback as never,
+ );
+ }
+
/** Preserve the SDK transport lifecycle while retaining oversized errors. */
override connect(transport: Transport): Promise {
return super.connect(
diff --git a/src/server/createServer.ts b/src/server/createServer.ts
index b578e26c50..f5d3884bf2 100644
--- a/src/server/createServer.ts
+++ b/src/server/createServer.ts
@@ -15,6 +15,11 @@ import {
type ToolResultDelivery,
} from "./toolResult.js";
import { parseMcpResponseBudget } from "../config/mcpResponseBudget.js";
+import {
+ COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
+ parseMcpInputSchemaProfile,
+} from "../config/mcpInputSchemaProfile.js";
+import type { CompactInputSchemaPresentation } from "./EvidenceMcpServer.js";
import { isAbsolute } from "node:path";
import type { BinarySessionPort } from "../application/binary/BinarySessionPort.js";
@@ -148,6 +153,7 @@ const installSessionToolAvailability = (
const createMcpServer = (
session: BinarySessionPort | undefined,
delivery: ToolResultDelivery,
+ compactPresentation: CompactInputSchemaPresentation | undefined,
): EvidenceMcpServer =>
new EvidenceMcpServer(
{
@@ -165,6 +171,7 @@ const createMcpServer = (
? undefined
: (evidence) => session.recordEvidence(evidence),
delivery,
+ compactPresentation,
);
/**
@@ -183,7 +190,11 @@ export const createServer = (
const selectedOptions = { ...options, environment, delivery };
const startedAt = new Date().toISOString();
const logger = options.logger ?? silentLogger;
- const server = createMcpServer(session, delivery);
+ const server = createMcpServer(
+ session,
+ delivery,
+ selectCompactInputSchemaPresentation(environment),
+ );
const android =
options.androidAnalysis ?? createAndroidAnalysisProvider(environment);
const availability = installSessionToolAvailability(
@@ -256,6 +267,18 @@ const selectToolResultDelivery = (
return createToolResultDelivery(configured.value);
};
+const selectCompactInputSchemaPresentation = (
+ environment: Readonly,
+): CompactInputSchemaPresentation | undefined => {
+ const configured = parseMcpInputSchemaProfile(
+ environment.REA_MCP_INPUT_SCHEMA_PROFILE,
+ );
+ if (!configured.ok) throw configured.error;
+ return configured.value === "compact"
+ ? { budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES }
+ : undefined;
+};
+
const registerConfiguredAnalysisTools = (
{
server,
diff --git a/tests/acceptance/applications/javascriptEndpointPayload.test.ts b/tests/acceptance/applications/javascriptEndpointPayload.test.ts
new file mode 100644
index 0000000000..6a980aa7ad
--- /dev/null
+++ b/tests/acceptance/applications/javascriptEndpointPayload.test.ts
@@ -0,0 +1,263 @@
+import { Client } from "@modelcontextprotocol/client";
+import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
+import { createHash } from "node:crypto";
+import { writeFile } from "node:fs/promises";
+import { join, resolve } from "node:path";
+import { expect } from "vitest";
+
+import { toolContract } from "../../../src/contracts/toolContracts.js";
+import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
+import { applicationVersionComparisonResultSchema } from "../../../src/domain/javascript/javascriptApplicationVersionComparisonSchemas.js";
+import { applicationFeatureTraceResultSchema } from "../../../src/domain/javascript/javascriptFeatureTraceSchemas.js";
+import { javaScriptSemanticTraceResultSchema } from "../../../src/domain/javascript/javascriptSemanticTraceSchemas.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+cliTest(
+ "keeps full endpoint evidence usable without copying large values into metadata",
+ async ({ cli }) => {
+ const root = await createTestTempDirectory("rea-endpoint-payload-");
+ const endpoint = `https://example.invalid/${"part/ğ".repeat(160_000)}`;
+ const digestLookalike = `value-sha256:${createHash("sha256").update(JSON.stringify(endpoint)).digest("hex")}`;
+ // UTF-8 replacement would conflate these distinct JavaScript strings.
+ const surrogate = "\ud800".repeat(100);
+ const replacement = "\ufffd".repeat(100);
+ const values = [
+ endpoint,
+ digestLookalike,
+ "/ok",
+ "/ok",
+ surrogate,
+ replacement,
+ ];
+ const source = values
+ .map((value) => `fetch(${JSON.stringify(value)});`)
+ .join("\n");
+ const sourcePath = join(root, "main.js");
+ await writeFile(sourcePath, source);
+ const cliResponse = await cli.run({
+ arguments: [
+ "analyze-javascript-application",
+ root,
+ "--artifact-format",
+ "directory",
+ "--json",
+ ],
+ timeoutMs: 60_000,
+ });
+ expect(cliResponse.exitCode).toBe(0);
+ const cliEvidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(cliResponse.json);
+
+ const client = new Client({ name: "endpoint-payload-e2e", version: "1" });
+ const transport = new StdioClientTransport({
+ command: process.execPath,
+ args: [resolve("scripts/rea.mjs"), "mcp"],
+ cwd: process.cwd(),
+ env: { PATH: process.env.PATH ?? "" },
+ stderr: "pipe",
+ });
+ transport.stderr?.on("data", () => undefined);
+ try {
+ await client.connect(transport);
+ const response = await client.callTool({
+ name: "analyze_javascript_application",
+ arguments: { input_path: root, format: "directory" },
+ });
+ expect(response.isError).not.toBe(true);
+ const evidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(response.structuredContent);
+ expect(evidence.evidence_id).toBe(cliEvidence.evidence_id);
+ const text = response.content.find((item) => item.type === "text");
+ if (text?.type !== "text") throw new Error("Missing MCP text result");
+ const textEvidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(JSON.parse(text.text));
+ expect(textEvidence.evidence_id).toBe(evidence.evidence_id);
+ const result = javascriptApplicationAnalysisResultSchema.parse(
+ evidence.normalized_result,
+ );
+ const endpoints = result.graph.nodes.filter(
+ (node) => node.kind === "endpoint",
+ );
+ const nodeFor = (value: string) => {
+ const node = endpoints.find((candidate) =>
+ candidate.observations.some(
+ ({ properties }) => properties.value === value,
+ ),
+ );
+ if (node === undefined) throw new Error("Complete endpoint was lost");
+ return node;
+ };
+ const endpointNode = nodeFor(endpoint);
+ expect(
+ new Set(
+ [endpoint, digestLookalike, surrogate, replacement].map(
+ (value) => nodeFor(value).node_id,
+ ),
+ ).size,
+ ).toBe(4);
+ const repeatedNode = nodeFor("/ok");
+ expect(
+ endpoints.filter((node) =>
+ node.observations.some(
+ ({ properties }) => properties.value === "/ok",
+ ),
+ ),
+ ).toHaveLength(1);
+ expect(repeatedNode.observations).toHaveLength(2);
+ expect(repeatedNode.identity).toMatchObject({ key: "/ok" });
+ expect(
+ repeatedNode.observations.every(({ label }) => label === "/ok"),
+ ).toBe(true);
+ expect(endpointNode.identity).toMatchObject({
+ artifact_sha256: createHash("sha256").update(source).digest("hex"),
+ });
+ expect(endpointNode.observations[0]?.evidence.location).toMatchObject({
+ available: true,
+ value: { kind: "source-range", source: "main.js" },
+ });
+ const requests = result.semantic_graph.nodes.filter(
+ (node) =>
+ node.kind === "request" && node.properties.endpoint === endpoint,
+ );
+ expect(requests).toHaveLength(1);
+
+ await verifyEndpointWorkflows({
+ client,
+ evidenceId: evidence.evidence_id,
+ endpoint,
+ endpointNodeId: endpointNode.node_id,
+ requestNodeIds: requests.map(({ node_id }) => node_id),
+ root,
+ source,
+ sourcePath,
+ });
+ await client.ping();
+ } finally {
+ await client.close();
+ await transport.close();
+ }
+ },
+ 120_000,
+);
+
+interface EndpointWorkflowInput {
+ readonly client: Client;
+ readonly evidenceId: string;
+ readonly endpoint: string;
+ readonly endpointNodeId: string;
+ readonly requestNodeIds: readonly string[];
+ readonly root: string;
+ readonly source: string;
+ readonly sourcePath: string;
+}
+
+const verifyEndpointWorkflows = async ({
+ client,
+ evidenceId,
+ endpoint,
+ endpointNodeId,
+ requestNodeIds,
+ root,
+ source,
+ sourcePath,
+}: EndpointWorkflowInput): Promise => {
+ const application = {
+ kind: "retained-evidence",
+ evidence_id: evidenceId,
+ };
+ const featureResponse = await client.callTool({
+ name: "trace_application_feature",
+ arguments: {
+ application,
+ seed: {
+ kind: "string",
+ value: endpoint,
+ match: "exact",
+ case_sensitive: true,
+ },
+ },
+ });
+ expect(featureResponse.isError).not.toBe(true);
+ const featureEvidence = toolContract(
+ "trace_application_feature",
+ ).outputSchema.parse(featureResponse.structuredContent);
+ const feature = applicationFeatureTraceResultSchema.parse(
+ featureEvidence.normalized_result,
+ );
+ expect(feature.seed_matches.map(({ node_id }) => node_id)).toContain(
+ endpointNodeId,
+ );
+
+ const traceResponse = await client.callTool({
+ name: "trace_javascript_semantics",
+ arguments: {
+ application,
+ query: {
+ seed: { kind: "endpoint", value: endpoint },
+ direction: "backward-provenance",
+ },
+ },
+ });
+ expect(traceResponse.isError).not.toBe(true);
+ const traceEvidence = toolContract(
+ "trace_javascript_semantics",
+ ).outputSchema.parse(traceResponse.structuredContent);
+ const trace = javaScriptSemanticTraceResultSchema.parse(
+ traceEvidence.normalized_result,
+ );
+ expect(trace.seed_node_ids).toEqual(requestNodeIds);
+ expect(
+ trace.nodes.some((node) => node.properties.endpoint === endpoint),
+ ).toBe(true);
+
+ await writeFile(sourcePath, `// next artifact version\n${source}`);
+ const nextResponse = await client.callTool({
+ name: "analyze_javascript_application",
+ arguments: { input_path: root, format: "directory" },
+ });
+ expect(nextResponse.isError).not.toBe(true);
+ const nextEvidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(nextResponse.structuredContent);
+ const next = javascriptApplicationAnalysisResultSchema.parse(
+ nextEvidence.normalized_result,
+ );
+ const nextEndpoint = next.graph.nodes.find(
+ (node) =>
+ node.kind === "endpoint" &&
+ node.observations.some(({ properties }) => properties.value === endpoint),
+ );
+ if (nextEndpoint === undefined) throw new Error("Next endpoint was lost");
+ const comparisonResponse = await client.callTool({
+ name: "compare_application_versions",
+ arguments: {
+ left: application,
+ right: {
+ kind: "retained-evidence",
+ evidence_id: nextEvidence.evidence_id,
+ },
+ },
+ });
+ expect(comparisonResponse.isError).not.toBe(true);
+ const comparisonEvidence = toolContract(
+ "compare_application_versions",
+ ).outputSchema.parse(comparisonResponse.structuredContent);
+ const comparison = applicationVersionComparisonResultSchema.parse(
+ comparisonEvidence.normalized_result,
+ );
+ expect(comparison.items).toContainEqual(
+ expect.objectContaining({
+ node_kind: "endpoint",
+ left_node_id: endpointNodeId,
+ right_node_id: nextEndpoint.node_id,
+ match: expect.objectContaining({
+ status: "matched",
+ basis: "semantic-key",
+ }),
+ }),
+ );
+};
diff --git a/tests/acceptance/applications/javascriptEventPayload.test.ts b/tests/acceptance/applications/javascriptEventPayload.test.ts
new file mode 100644
index 0000000000..c4689250de
--- /dev/null
+++ b/tests/acceptance/applications/javascriptEventPayload.test.ts
@@ -0,0 +1,164 @@
+import { Client } from "@modelcontextprotocol/client";
+import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
+import { createHash } from "node:crypto";
+import { writeFile } from "node:fs/promises";
+import { join, resolve } from "node:path";
+import { expect } from "vitest";
+
+import { toolContract } from "../../../src/contracts/toolContracts.js";
+import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
+import { javaScriptSemanticTraceResultSchema } from "../../../src/domain/javascript/javascriptSemanticTraceSchemas.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+cliTest(
+ "retains large event names and emitter-scoped listener ownership through CLI and stdio MCP",
+ async ({ cli }) => {
+ const root = await createTestTempDirectory("rea-event-payload-");
+ const name = "payload/ğ".repeat(80_000);
+ const lookalike = `value-sha256:${createHash("sha256").update(JSON.stringify(name)).digest("hex")}`;
+ const source = [
+ 'import {EventEmitter} from "node:events";',
+ "const bus = new EventEmitter(); const other = new EventEmitter(); const handler = () => {};",
+ `bus.on(${JSON.stringify(name)}, handler);`,
+ `bus.emit(${JSON.stringify(name)});`,
+ `bus.off(${JSON.stringify(name)}, handler);`,
+ `bus.on(${JSON.stringify(lookalike)}, handler); bus.emit(${JSON.stringify(lookalike)});`,
+ 'bus.on("ok", handler); bus.emit("ok"); other.on("ok", handler); other.emit("ok");',
+ "bus.on(getEventName(), handler);",
+ ].join("\n");
+ await writeFile(join(root, "main.js"), source);
+ const cliResponse = await cli.run({
+ arguments: [
+ "analyze-javascript-application",
+ root,
+ "--artifact-format",
+ "directory",
+ "--json",
+ ],
+ timeoutMs: 60_000,
+ });
+ expect(cliResponse.exitCode).toBe(0);
+ const cliEvidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(cliResponse.json);
+ const client = new Client({ name: "event-payload-e2e", version: "1" });
+ const transport = new StdioClientTransport({
+ command: process.execPath,
+ args: [resolve("scripts/rea.mjs"), "mcp"],
+ cwd: process.cwd(),
+ env: { PATH: process.env.PATH ?? "" },
+ stderr: "pipe",
+ });
+ transport.stderr?.on("data", () => undefined);
+ try {
+ await client.connect(transport);
+ const response = await client.callTool({
+ name: "analyze_javascript_application",
+ arguments: { input_path: root, format: "directory" },
+ });
+ expect(response.isError).not.toBe(true);
+ const evidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(response.structuredContent);
+ expect(evidence.evidence_id).toBe(cliEvidence.evidence_id);
+ const text = response.content.find((item) => item.type === "text");
+ if (text?.type !== "text") throw new Error("Missing MCP text result");
+ expect(
+ toolContract("analyze_javascript_application").outputSchema.parse(
+ JSON.parse(text.text),
+ ).evidence_id,
+ ).toBe(evidence.evidence_id);
+ const graph = javascriptApplicationAnalysisResultSchema.parse(
+ evidence.normalized_result,
+ ).semantic_graph;
+ const events = graph.nodes.filter((node) => node.kind === "event");
+ const selected = events.filter(
+ (node) => node.properties.event_name === name,
+ );
+ expect(selected).toHaveLength(1);
+ const event = selected[0];
+ if (event === undefined) throw new Error("Complete event was lost");
+ const listeners = graph.nodes.filter(
+ (node) =>
+ node.kind === "listener" && node.properties.event_name === name,
+ );
+ expect(new Set(listeners.map((node) => node.properties.method))).toEqual(
+ new Set(["on", "off"]),
+ );
+ const listenerIds = new Set(listeners.map((node) => node.node_id));
+ expect(
+ graph.relations
+ .filter(
+ (relation) =>
+ relation.source_node_id === event.node_id &&
+ listenerIds.has(relation.target_node_id),
+ )
+ .map((relation) => relation.relation),
+ ).toEqual(
+ expect.arrayContaining(["registers-listener", "removes-listener"]),
+ );
+ const lookalikeEvent = events.find(
+ (node) => node.properties.event_name === lookalike,
+ );
+ expect(lookalikeEvent).toBeDefined();
+ expect(lookalikeEvent?.node_id).not.toBe(event.node_id);
+ const shortEvents = events.filter(
+ (node) => node.properties.event_name === "ok",
+ );
+ expect(shortEvents).toHaveLength(2);
+ expect(
+ new Set(shortEvents.map((node) => node.properties.emitter_key)).size,
+ ).toBe(2);
+ expect(
+ shortEvents.every(
+ (node) =>
+ node.label === "ok" && node.identity.role_key.endsWith("\0ok"),
+ ),
+ ).toBe(true);
+ const dynamic = events.find(
+ (node) => node.properties.event_name === null,
+ );
+ expect(dynamic?.label).toBeNull();
+ expect(
+ graph.unknowns.some(
+ (unknown) =>
+ unknown.node_id === dynamic?.node_id && unknown.family === "event",
+ ),
+ ).toBe(true);
+ const traceResponse = await client.callTool({
+ name: "trace_javascript_semantics",
+ arguments: {
+ application: {
+ kind: "retained-evidence",
+ evidence_id: evidence.evidence_id,
+ },
+ query: {
+ seed: { kind: "event", name },
+ direction: "forward-influence",
+ },
+ },
+ });
+ expect(traceResponse.isError).not.toBe(true);
+ const traceEvidence = toolContract(
+ "trace_javascript_semantics",
+ ).outputSchema.parse(traceResponse.structuredContent);
+ const trace = javaScriptSemanticTraceResultSchema.parse(
+ traceEvidence.normalized_result,
+ );
+ expect(new Set(trace.seed_node_ids)).toEqual(
+ new Set([event.node_id, ...listenerIds]),
+ );
+ expect(
+ trace.nodes
+ .filter((node) => listenerIds.has(node.node_id))
+ .map((node) => node.properties.event_name),
+ ).toEqual(listeners.map(() => name));
+ await client.ping();
+ } finally {
+ await client.close();
+ await transport.close();
+ }
+ },
+ 120_000,
+);
diff --git a/tests/acceptance/applications/javascriptIterationAliases.test.ts b/tests/acceptance/applications/javascriptIterationAliases.test.ts
new file mode 100644
index 0000000000..a7f039eb83
--- /dev/null
+++ b/tests/acceptance/applications/javascriptIterationAliases.test.ts
@@ -0,0 +1,76 @@
+import { expect } from "vitest";
+
+import {
+ verifyJavaScriptReturnShapes,
+ type JavaScriptReturnFields,
+} from "../../fixtures/javascriptReturnShapes.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+const source = [
+ "export function inline() { const shared = { x: 1, keep: 7 }; for (const t of [shared]) t.x = 2; return { x: shared.x, keep: shared.keep }; }",
+ "export function named() { const shared = { x: 1 }; const arr = [shared]; for (const t of arr) t.x = 2; return shared.x; }",
+ "export function declaredVar() { const shared = { x: 1 }; for (var t of [shared]) t.x = 2; return shared.x; }",
+ "export function assigned() { const shared = { x: 1 }; let t; for (t of [shared]) t.x = 2; return shared.x; }",
+ "export function assignedDestructured() { const shared = { x: 1 }; let t; for ({child: t} of [{child: shared}]) t.x = 2; return shared.x; }",
+ "export function declaredVarDestructured() { const shared = { x: 1 }; for (var {child: t} of [{child: shared}]) t.x = 2; return shared.x; }",
+ "export function memberTarget() { const shared = { x: 1 }; const holder = {}; for (holder.item of [shared]) holder.item.x = 2; return shared.x; }",
+ "export function memberPattern() { const shared = { x: 1 }; const holder = {}; for ({child: holder.item} of [{child: shared}]) holder.item.x = 2; return shared.x; }",
+ "export function destructured() { const shared = { x: 1 }; for (const {child: t} of [{child: shared}]) t.x = 2; return shared.x; }",
+ "export function arrayDestructured() { const shared = { x: 1 }; for (const [t] of [[shared]]) t.x = 2; return shared.x; }",
+ "export function spread() { const shared = { x: 1 }; const arr = [shared]; for (const t of [...arr]) t.x = 2; return shared.x; }",
+ "export function objectRestChild() { const shared = { x: 1 }; for (const {...t} of [{child: shared}]) t.child.x = 2; return shared.x; }",
+ "export function arrayRestChild() { const shared = { x: 1 }; for (const [...t] of [[shared]]) t[0].x = 2; return shared.x; }",
+ "export function defaultChild() { const shared = { x: 1 }; for (const {child: t = shared} of [{}]) t.x = 2; return shared.x; }",
+ "export function updated() { const shared = { x: 1 }; for (const t of [shared]) t.x++; return shared.x; }",
+ "export function deleted() { const shared = { x: 1 }; for (const t of [shared]) delete t.x; return shared.x; }",
+ "export function escaped() { const shared = { x: 1 }; for (const t of [shared]) consume(t); return shared.x; }",
+ "export async function awaited() { const shared = { x: 1 }; for await (const t of [shared]) t.x = 2; return shared.x; }",
+ "export function indexed() { const shared = { x: 1 }; const arr = [shared]; for (let i = 0; i < arr.length; i++) { const t = arr[i]; t.x = 2; } return shared.x; }",
+ "export function copyWrite() { const shared = { x: 1 }; for (const {...t} of [shared]) t.x = 2; return shared.x; }",
+ "export function noWrite() { const shared = { x: 1 }; for (const t of [shared]) { const read = t.x; } return shared.x; }",
+ "export function empty() { const shared = { x: 1 }; for (const t of []) t.x = 2; return shared.x; }",
+ "export function keys() { const shared = { x: 1 }; for (const key in [shared]) { const read = key; } return shared.x; }",
+ "export function shadowed() { const t = { x: 1 }; const shared = { x: 3 }; for (const t of [shared]) t.x = 2; return t.x; }",
+].join("\n");
+
+const assertReturns = (fields: JavaScriptReturnFields): void => {
+ expect(fields("inline")).toEqual(
+ expect.arrayContaining([
+ expect.objectContaining({ path: "/x", state: "unknown" }),
+ expect.objectContaining({ path: "/keep", state: "literal", value: 7 }),
+ ]),
+ );
+ for (const name of [
+ "named",
+ "declaredVar",
+ "assigned",
+ "assignedDestructured",
+ "declaredVarDestructured",
+ "memberTarget",
+ "memberPattern",
+ "destructured",
+ "arrayDestructured",
+ "spread",
+ "objectRestChild",
+ "arrayRestChild",
+ "defaultChild",
+ "updated",
+ "deleted",
+ "escaped",
+ "awaited",
+ "indexed",
+ ])
+ expect(fields(name), name).toContainEqual(
+ expect.objectContaining({ path: "", state: "unknown" }),
+ );
+ for (const name of ["copyWrite", "noWrite", "empty", "keys", "shadowed"])
+ expect(fields(name), name).toContainEqual(
+ expect.objectContaining({ path: "", state: "literal", value: 1 }),
+ );
+};
+
+cliTest(
+ "preserves yielded-object mutation uncertainty through CLI and stdio MCP",
+ ({ cli }) => verifyJavaScriptReturnShapes(cli, source, assertReturns),
+ 120_000,
+);
diff --git a/tests/acceptance/applications/javascriptReturnedReferences.test.ts b/tests/acceptance/applications/javascriptReturnedReferences.test.ts
new file mode 100644
index 0000000000..bd9a05c761
--- /dev/null
+++ b/tests/acceptance/applications/javascriptReturnedReferences.test.ts
@@ -0,0 +1,100 @@
+import { expect } from "vitest";
+
+import {
+ verifyJavaScriptReturnShapes,
+ type JavaScriptReturnFields,
+} from "../../fixtures/javascriptReturnShapes.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+const writes = {
+ direct: "const get = () => shared; get().x = 2;",
+ captured: "const get = () => shared; const t = get(); t.x = 2;",
+ declaration:
+ "function get() { return shared; } const o = {}; o.a = get(); o.a.x = 2;",
+ immediate: "const o = {}; o.a = (() => shared)(); o.a.x = 2;",
+ method:
+ "const box = {get() { return shared; }}; const o = {}; o.a = box.get(); o.a.x = 2;",
+ methodDirect: "const box = {get() { return shared; }}; box.get().x = 2;",
+ quotedMethod:
+ "const box = {['get/ref']() { return shared; }}; box['get/ref']().x = 2;",
+ alias: "const get = () => shared; const another = get; another().x = 2;",
+ alternative: "const get = true ? () => shared : () => ({x: 0}); get().x = 2;",
+ nestedReturn: "const get = () => ({child: shared}); get().child.x = 2;",
+ arrayReturn: "const get = () => [shared]; get()[0].x = 2;",
+ returnedProperty:
+ "const holder = {inner: shared}; const get = () => holder.inner; get().x = 2;",
+ transitive:
+ "const first = () => shared; const second = () => first(); second().x = 2;",
+ updated: "const get = () => shared; get().x++;",
+ asyncAwait: "const get = async () => shared; const t = await get(); t.x = 2;",
+ generator:
+ "function* get() { yield shared; } for (const t of get()) t.x = 2;",
+ returnedYield:
+ "function* get() { return yield shared; } for (const t of get()) t.x = 2;",
+ delegatedYield:
+ "function* get() { yield* [shared]; } for (const t of get()) t.x = 2;",
+ recursiveWrapper:
+ "let count = 0; const get = () => count++ === 0 ? {child: get()} : shared; get().child.x = 2;",
+ destructuredMethod:
+ "const box = {get() { return shared; }}; const {get} = box; get().x = 2;",
+ arrayCallable: "const methods = [() => shared]; methods[0]().x = 2;",
+ spreadArrayCallable:
+ "const methods = [...[], () => shared]; methods[0]().x = 2;",
+ classMethod:
+ "class Box { get() { return shared; } } const box = new Box(); box.get().x = 2;",
+ staticMethod:
+ "class Box { static get() { return shared; } } Box.get().x = 2;",
+ constructor:
+ "function Box() { return shared; } const t = new Box(); t.x = 2;",
+ callback: "const get = () => shared; [0].forEach(() => { get().x = 2; });",
+};
+const source = [
+ ...Object.entries(writes).map(
+ ([name, body]) =>
+ `export ${name === "asyncAwait" ? "async " : ""}function ${name}(){ const shared = {x: 1}; ${body} return shared.x; }`,
+ ),
+ "export function directSibling(){ const shared = {x: 1}; const parent = {shared, keep: 7}; const get = () => parent.shared; get().x = 2; return {x: shared.x, keep: parent.keep}; }",
+ "export function primitive(){ const n = 1; const get = () => n; const t = get(); return n; }",
+ "export function copy(){ const shared = {x: 1}; const get = () => ({...shared}); get().x = 2; return shared.x; }",
+ "export function methodCopy(){ const shared = {x: 1}; const box = {get(){ return {...shared}; }}; box.get().x = 2; return shared.x; }",
+ "export function arrayCopy(){ const shared = {x: 1}; const get = () => [shared.x]; get()[0] = 2; return shared.x; }",
+ "export function unused(){ const shared = {x: 1}; const get = () => shared; get(); return shared.x; }",
+ "export function nestedCallable(){ const shared = {x: 1}; const get = () => { const inner = () => shared; return {x: 0}; }; get().x = 2; return shared.x; }",
+ "export function unrelated(){ const shared = {x: 1}; const other = {x: 9}; const get = () => shared; get().x = 2; return other.x; }",
+].join("\n");
+
+const assertReturns = (fields: JavaScriptReturnFields): void => {
+ for (const name of Object.keys(writes)) {
+ const field = fields(name).find(({ path }) => path === "");
+ expect(field?.state, name).toMatch(/^(unknown|literal)$/);
+ if (field?.state === "literal") expect(field.value, name).toBe(2);
+ }
+ const changed = fields("directSibling").find(({ path }) => path === "/x");
+ expect(changed?.state).toMatch(/^(unknown|literal)$/);
+ if (changed?.state === "literal") expect(changed.value).toBe(2);
+ expect(fields("directSibling")).toContainEqual(
+ expect.objectContaining({ path: "/keep", state: "literal", value: 7 }),
+ );
+ for (const name of [
+ "primitive",
+ "copy",
+ "methodCopy",
+ "arrayCopy",
+ "nestedCallable",
+ ])
+ expect(fields(name), name).toContainEqual(
+ expect.objectContaining({ path: "", state: "literal", value: 1 }),
+ );
+ const unused = fields("unused").find(({ path }) => path === "");
+ expect(unused?.state).toMatch(/^(unknown|literal)$/);
+ if (unused?.state === "literal") expect(unused.value).toBe(1);
+ expect(fields("unrelated")).toContainEqual(
+ expect.objectContaining({ path: "", state: "literal", value: 9 }),
+ );
+};
+
+cliTest(
+ "preserves mutation uncertainty for objects returned by local calls",
+ ({ cli }) => verifyJavaScriptReturnShapes(cli, source, assertReturns),
+ 120_000,
+);
diff --git a/tests/acceptance/applications/javascriptSlotStores.test.ts b/tests/acceptance/applications/javascriptSlotStores.test.ts
new file mode 100644
index 0000000000..3af7e3eeed
--- /dev/null
+++ b/tests/acceptance/applications/javascriptSlotStores.test.ts
@@ -0,0 +1,98 @@
+import { expect } from "vitest";
+
+import {
+ verifyJavaScriptReturnShapes,
+ type JavaScriptReturnFields,
+} from "../../fixtures/javascriptReturnShapes.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+const stores = {
+ object: "const o = {a: {x: 0}}; o.a = shared; o.a.x = 2;",
+ array: "const arr = [{x: 0}]; arr[0] = shared; arr[0].x = 2;",
+ captured: "const o = {a: {x: 0}}; o.a = shared; const t = o.a; t.x = 2;",
+ nested: "const o = {a: {b: {x: 0}}}; o.a.b = shared; o.a.b.x = 2;",
+ quoted: 'const o = {}; o["a/b~c"] = shared; o["a/b~c"].x = 2;',
+ dynamic: 'const o = {}; const key = "entry"; o[key] = shared; o[key].x = 2;',
+ receiverAlias: "const o = {}; const t = o; t.a = shared; o.a.x = 2;",
+ transitive: "const p = {}; const o = {}; o.a = p; p.b = shared; o.a.b.x = 2;",
+ overwritten: "const o = {}; o.a = {x: 0}; o.a = shared; o.a.x = 2;",
+ replacedReference:
+ "const o = {}; const other = {x: 0}; o.a = other; o.a = shared; o.a.x = 2;",
+ copiedDestination:
+ "const o = {}; o.a = shared; const copy = {...o}; copy.a.x = 2;",
+ iteration: "const arr = []; arr[0] = shared; for (const t of arr) t.x = 2;",
+ nestedArray: "const o = {}; o.a = [shared]; o.a[0].x = 2;",
+ orAssignment: "const o = {a: null}; o.a ||= shared; o.a.x = 2;",
+ nullishAssignment: "const o = {a: null}; o.a ??= shared; o.a.x = 2;",
+ andAssignment: "const o = {a: {x: 0}}; o.a &&= shared; o.a.x = 2;",
+ objectPattern: "const o = {}; ({child: o.a} = {child: shared}); o.a.x = 2;",
+ arrayPattern: "const o = {}; [o.a] = [shared]; o.a.x = 2;",
+ defaultPattern: "const o = {}; ({child: o.a = shared} = {}); o.a.x = 2;",
+ objectRestMember:
+ "const o = {}; ({...o.a} = {child: shared}); o.a.child.x = 2;",
+ arrayRestMember: "const o = {}; [...o.a] = [shared]; o.a[0].x = 2;",
+ iteratedObjectRest:
+ "const o = {}; for ({...o.a} of [{child: shared}]) o.a.child.x = 2;",
+ iteratedArrayRest: "const o = {}; for ([...o.a] of [[shared]]) o.a[0].x = 2;",
+ classField:
+ "class Holder {} const o = new Holder(); o.a = shared; o.a.x = 2;",
+};
+const source = [
+ ...Object.entries(stores).map(
+ ([name, body]) =>
+ `export function ${name}() { const shared = {x: 1}; ${body} return shared.x; }`,
+ ),
+ "export function primitive() { const n = 1; const o = {}; o.a = n; o.a = 2; return n; }",
+ "export function nestedChild() { const shared = {inner: {x: 1}}; const o = {}; o.a = shared; o.a.inner.x = 2; return shared.inner.x; }",
+ "export function readOnlyStore() { const shared = {x: 1}; const o = {}; o.a = shared; const captured = o.a.x; return shared.x; }",
+ "export function copiedScalar() { const shared = {x: 1}; const o = {}; o.a = {...shared}; o.a.x = 2; return shared.x; }",
+ "export function copiedArrayScalar() { const shared = {x: 1}; const o = {}; o.a = [shared.x]; o.a[0] = 2; return shared.x; }",
+ "export function unrelated() { const shared = {x: 1}; const other = {x: 9}; const o = {}; o.a = shared; o.a.x = 2; return other.x; }",
+ "export function copiedChild() { const child = {x: 1}; const parent = {child, keep: 7}; const o = {}; o.a = {...parent}; o.a.child.x = 2; return {child: child.x, keep: parent.keep}; }",
+ "export function objectRestExclusion() { const child = {x: 1}; const skip = {x: 3}; const parent = {child, skip, keep: 7}; const o = {}; let ignored; ({skip: ignored, ...o.a} = parent); o.a.child.x = 2; return {child: child.x, skip: skip.x, keep: parent.keep}; }",
+ "export function arrayRestOffset() { const child = {x: 1}; const skip = {x: 3}; const items = [skip, child]; const o = {}; let ignored; [ignored, ...o.a] = items; o.a[0].x = 2; return {child: child.x, skip: items[0].x}; }",
+].join("\n");
+
+const assertReturns = (fields: JavaScriptReturnFields): void => {
+ for (const name of [...Object.keys(stores), "nestedChild"]) {
+ const field = fields(name).find(({ path }) => path === "");
+ expect(field, name).toBeDefined();
+ expect(field?.state, name).toMatch(/^(unknown|literal)$/);
+ if (field?.state === "literal") expect(field.value, name).toBe(2);
+ }
+ const readOnly = fields("readOnlyStore").find(({ path }) => path === "");
+ expect(readOnly?.state).toMatch(/^(unknown|literal)$/);
+ if (readOnly?.state === "literal") expect(readOnly.value).toBe(1);
+ for (const [name, value] of Object.entries({
+ primitive: 1,
+ copiedScalar: 1,
+ copiedArrayScalar: 1,
+ unrelated: 9,
+ }))
+ expect(fields(name), name).toContainEqual(
+ expect.objectContaining({ path: "", state: "literal", value }),
+ );
+ const child = fields("copiedChild").find(({ path }) => path === "/child");
+ expect(child?.state).toMatch(/^(unknown|literal)$/);
+ if (child?.state === "literal") expect(child.value).toBe(2);
+ expect(fields("copiedChild")).toContainEqual(
+ expect.objectContaining({ path: "/keep", state: "literal", value: 7 }),
+ );
+ for (const name of ["objectRestExclusion", "arrayRestOffset"]) {
+ const child = fields(name).find(({ path }) => path === "/child");
+ expect(child?.state, name).toMatch(/^(unknown|literal)$/);
+ if (child?.state === "literal") expect(child.value, name).toBe(2);
+ expect(fields(name), name).toContainEqual(
+ expect.objectContaining({ path: "/skip", state: "literal", value: 3 }),
+ );
+ }
+ expect(fields("objectRestExclusion")).toContainEqual(
+ expect.objectContaining({ path: "/keep", state: "literal", value: 7 }),
+ );
+};
+
+cliTest(
+ "does not report stale literals after storing references in member slots",
+ ({ cli }) => verifyJavaScriptReturnShapes(cli, source, assertReturns),
+ 120_000,
+);
diff --git a/tests/acceptance/applications/runtime.test.ts b/tests/acceptance/applications/runtime.test.ts
index 1a50c3fc10..bfa75b7219 100644
--- a/tests/acceptance/applications/runtime.test.ts
+++ b/tests/acceptance/applications/runtime.test.ts
@@ -144,6 +144,8 @@ describe("production stdio runtime", () => {
expect(evidence.operation).toBe("current_document");
expect(evidence.provider.id).toBe("hopper");
expect(evidence.normalized_result).toBe(observation.normalized_result);
+ // Hopper's reply is the result; it is not repeated as raw_result.
+ expect(evidence.raw_result).toBeNull();
expect(records).toContainEqual(
expect.objectContaining({
application: "rea",
diff --git a/tests/acceptance/setup/hermesClientSetup.test.ts b/tests/acceptance/setup/hermesClientSetup.test.ts
new file mode 100644
index 0000000000..bd65d735b8
--- /dev/null
+++ b/tests/acceptance/setup/hermesClientSetup.test.ts
@@ -0,0 +1,98 @@
+import { access, mkdir, readFile, writeFile } from "node:fs/promises";
+import { join } from "node:path";
+import { parse } from "yaml";
+import { expect } from "vitest";
+
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+cliTest(
+ "Hermes setup, doctor and uninstall use the expanded profile and its personal skill",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-hermes-cli-");
+ const profile = join(home, "profile");
+ const config = join(profile, "config.yaml");
+ const skill = join(profile, "skills/reverse-engineer-anything/SKILL.md");
+ const original =
+ "# user settings\nmodel: custom\nmcp_servers:\n other:\n command: other-server\n";
+ await mkdir(profile);
+ await writeFile(config, original);
+ const run = (arguments_: readonly string[]) =>
+ cli.run({
+ arguments: arguments_,
+ cwd: home,
+ environment: {
+ HOME: home,
+ USERPROFILE: home,
+ HERMES_HOME: " ${HOME}/profile ",
+ },
+ timeoutMs: 20_000,
+ });
+ const planned = await run([
+ "setup",
+ "--client",
+ "hermes",
+ "--dry-run",
+ "--json",
+ ]);
+ expect(planned.exitCode).toBe(0);
+ expect(planned.json).toMatchObject({
+ status: "planned",
+ plannedActions: expect.arrayContaining([
+ expect.objectContaining({ target: config }),
+ expect.objectContaining({
+ target: join(profile, "skills/reverse-engineer-anything"),
+ }),
+ ]),
+ });
+ expect(await readFile(config, "utf8")).toBe(original);
+ await expect(access(skill)).rejects.toMatchObject({ code: "ENOENT" });
+ const applied = await run([
+ "setup",
+ "--client",
+ "hermes",
+ "--yes",
+ "--json",
+ ]);
+ expect(applied.exitCode).toBe(0);
+ expect(applied.json).toMatchObject({ status: "ready" });
+ expect(parse(await readFile(config, "utf8"))).toMatchObject({
+ model: "custom",
+ mcp_servers: {
+ other: { command: "other-server" },
+ rea: { enabled: true },
+ },
+ });
+ expect(await readFile(config + ".rea.backup", "utf8")).toBe(original);
+ expect(await readFile(skill, "utf8")).toContain(
+ "reverse-engineer-anything",
+ );
+ await expect(
+ access(join(home, ".agents/skills/reverse-engineer-anything/SKILL.md")),
+ ).rejects.toMatchObject({ code: "ENOENT" });
+ const doctor = await run([
+ "doctor",
+ "--client",
+ "hermes",
+ "--skill",
+ "--json",
+ ]);
+ expect(doctor.exitCode).toBe(0);
+ expect(doctor.json).toMatchObject({
+ healthy: true,
+ identity: { skill: { state: "aligned" } },
+ });
+ const repeat = await run([
+ "setup",
+ "--client",
+ "hermes",
+ "--yes",
+ "--json",
+ ]);
+ expect(repeat.json).toMatchObject({ status: "ready", appliedActions: [] });
+ const uninstall = await run(["uninstall", "--json"]);
+ expect(uninstall.exitCode).toBe(0);
+ expect(parse(await readFile(config, "utf8"))).toEqual(parse(original));
+ await expect(access(skill)).rejects.toMatchObject({ code: "ENOENT" });
+ },
+);
diff --git a/tests/acceptance/setup/piClientSetup.test.ts b/tests/acceptance/setup/piClientSetup.test.ts
new file mode 100644
index 0000000000..bfb1d1b0a7
--- /dev/null
+++ b/tests/acceptance/setup/piClientSetup.test.ts
@@ -0,0 +1,311 @@
+import { mkdir, readFile, readdir, writeFile } from "node:fs/promises";
+import { join, resolve } from "node:path";
+
+import { expect } from "vitest";
+
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+cliTest(
+ "Pi public CLI plans, requires approval, applies, diagnoses, repeats, and uninstalls only owned content",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-pi-cli-");
+ const agent = join(home, "custom-agent");
+ const config = join(agent, "mcp.json");
+ const skill = join(
+ home,
+ ".agents/skills/reverse-engineer-anything/SKILL.md",
+ );
+ const original =
+ '{ "mcpServers": { "other": { "command": "other-server" } }, "autoEnableCodemode": false }\n';
+ await mkdir(agent);
+ await writeFile(config, original);
+ const cliPath = resolve("scripts/rea.mjs");
+ const run = (arguments_: readonly string[]) =>
+ cli.run({
+ arguments: arguments_,
+ cwd: home,
+ environment: {
+ HOME: home,
+ USERPROFILE: home,
+ PI_CODING_AGENT_DIR: "custom-agent",
+ OMP_PROFILE: "work",
+ PI_PROFILE: "legacy",
+ PI_CONFIG_DIR: ".omp-other",
+ },
+ timeoutMs: 20_000,
+ });
+
+ const planned = await run([
+ "setup",
+ "--client",
+ "pi",
+ "--dry-run",
+ "--json",
+ ]);
+ expect(planned.exitCode).toBe(0);
+ expect(planned.json).toMatchObject({
+ status: "planned",
+ plannedActions: [
+ {
+ id: "configure_client:pi",
+ label: "Pi",
+ target: join("custom-agent", "mcp.json"),
+ operation: "update",
+ },
+ { id: "install_skill" },
+ ],
+ appliedActions: [],
+ });
+ expect(await readFile(config, "utf8")).toBe(original);
+ await expect(readFile(`${config}.rea.backup`)).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ await expect(readFile(skill)).rejects.toMatchObject({ code: "ENOENT" });
+
+ const unapproved = await run(["setup", "--client", "pi", "--json"]);
+ expect(unapproved.json).toMatchObject({
+ status: "needs_confirmation",
+ appliedActions: [],
+ });
+ expect(await readFile(config, "utf8")).toBe(original);
+
+ const applied = await run(["setup", "--client", "pi", "--yes", "--json"]);
+ expect(applied.exitCode).toBe(0);
+ expect(applied.json).toMatchObject({
+ status: "ready",
+ clients: { pi: { status: "configured" } },
+ appliedActions: ["configured_pi", "installed_skill"],
+ });
+ const configured = await readFile(config, "utf8");
+ expect(JSON.parse(configured)).toMatchObject({
+ mcpServers: {
+ other: { command: "other-server" },
+ rea: {
+ type: "stdio",
+ command: process.platform === "win32" ? process.execPath : cliPath,
+ args: process.platform === "win32" ? [cliPath, "mcp"] : ["mcp"],
+ },
+ },
+ autoEnableCodemode: false,
+ });
+ expect(await readFile(`${config}.rea.backup`, "utf8")).toBe(original);
+ expect(await readFile(skill, "utf8")).toContain(
+ "reverse-engineer-anything",
+ );
+
+ const doctor = await run(["doctor", "--client", "pi", "--json"]);
+ expect(doctor.exitCode).toBe(0);
+ expect(doctor.json).toMatchObject({
+ healthy: true,
+ identity: {
+ registrations: expect.arrayContaining([
+ expect.objectContaining({ client: "pi", state: "aligned" }),
+ ]),
+ },
+ });
+
+ const repeat = await run(["setup", "--client", "pi", "--yes", "--json"]);
+ expect(repeat.exitCode).toBe(0);
+ expect(repeat.json).toMatchObject({
+ status: "ready",
+ plannedActions: [],
+ appliedActions: [],
+ });
+ expect(await readFile(config, "utf8")).toBe(configured);
+ expect(await readFile(`${config}.rea.backup`, "utf8")).toBe(original);
+
+ // Uninstall is global; there is no --client or --yes on this CLI surface.
+ const help = await run(["uninstall", "--help"]);
+ expect(help.exitCode).toBe(0);
+ expect(help.stdout).toContain("--purge-data");
+ const removed = await run(["uninstall", "--json"]);
+ expect(removed.exitCode).toBe(0);
+ expect(JSON.parse(await readFile(config, "utf8"))).toEqual(
+ JSON.parse(original),
+ );
+ expect(await readFile(`${config}.rea.backup`, "utf8")).toBe(original);
+ await expect(readFile(skill)).rejects.toMatchObject({ code: "ENOENT" });
+ },
+);
+
+cliTest(
+ "Pi public CLI creates missing strict-JSON configuration and never repairs a malformed file",
+ async ({ cli }) => {
+ const created = await createTestTempDirectory("rea-pi-cli-strict-");
+ const agent = join(created, "custom-agent");
+ const config = join(agent, "mcp.json");
+ await mkdir(agent);
+ const runIn = (home: string) => (arguments_: readonly string[]) =>
+ cli.run({
+ arguments: arguments_,
+ cwd: home,
+ environment: {
+ HOME: home,
+ USERPROFILE: home,
+ PI_CODING_AGENT_DIR: "custom-agent",
+ },
+ timeoutMs: 20_000,
+ });
+ const run = runIn(created);
+ const setup = await run(["setup", "--client", "pi", "--yes", "--json"]);
+ expect(setup.exitCode).toBe(0);
+ expect(setup.json).toMatchObject({
+ status: "ready",
+ clients: { pi: { status: "configured" } },
+ });
+ const text = await readFile(config, "utf8");
+ expect(() => JSON.parse(text)).not.toThrow();
+ expect(JSON.parse(text)).toMatchObject({
+ mcpServers: { rea: { type: "stdio" } },
+ });
+ const doctor = await run(["doctor", "--client", "pi", "--json"]);
+ expect(doctor.exitCode).toBe(0);
+ expect(doctor.json).toMatchObject({
+ healthy: true,
+ identity: {
+ registrations: expect.arrayContaining([
+ expect.objectContaining({ client: "pi", state: "aligned" }),
+ ]),
+ },
+ });
+ await run(["uninstall", "--json"]);
+ expect(
+ JSON.parse(await readFile(config, "utf8")).mcpServers.rea,
+ ).toBeUndefined();
+
+ const blocked = await createTestTempDirectory("rea-pi-cli-malformed-");
+ const malformedPath = join(blocked, "custom-agent", "mcp.json");
+ await mkdir(join(blocked, "custom-agent"));
+ const malformed = `{ // Pi loads strict JSON only\n"mcpServers": { "other": { "command": "other" } }\n}\n`;
+ await writeFile(malformedPath, malformed);
+ const runMalformed = runIn(blocked);
+ const plan = await runMalformed([
+ "setup",
+ "--client",
+ "pi",
+ "--dry-run",
+ "--json",
+ ]);
+ expect(plan.exitCode).not.toBe(0);
+ expect(plan.json).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ remediation: expect.stringContaining("malformed"),
+ });
+ const applied = await runMalformed([
+ "setup",
+ "--client",
+ "pi",
+ "--yes",
+ "--json",
+ ]);
+ expect(applied.exitCode).not.toBe(0);
+ expect(applied.json).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ });
+ expect(await readFile(malformedPath, "utf8")).toBe(malformed);
+ await expect(readFile(`${malformedPath}.rea.backup`)).rejects.toMatchObject(
+ {
+ code: "ENOENT",
+ },
+ );
+ const doctorMalformed = await runMalformed([
+ "doctor",
+ "--client",
+ "pi",
+ "--json",
+ ]);
+ expect(doctorMalformed.exitCode).not.toBe(0);
+ expect(doctorMalformed.json).toMatchObject({
+ healthy: false,
+ identity: {
+ registrations: expect.arrayContaining([
+ expect.objectContaining({ client: "pi", state: "invalid" }),
+ ]),
+ },
+ });
+ const removed = await runMalformed(["uninstall", "--json"]);
+ expect(removed.exitCode).not.toBe(0);
+ expect(removed.json).toMatchObject({
+ status: "failed",
+ items: expect.arrayContaining([
+ expect.objectContaining({ name: "pi", status: "failed" }),
+ ]),
+ });
+ expect(await readFile(malformedPath, "utf8")).toBe(malformed);
+ },
+);
+
+cliTest(
+ "malformed Pi overrides do not block selected Codex setup or doctor and never fall back for Pi",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-pi-invalid-cli-");
+ const run = (arguments_: readonly string[]) =>
+ cli.run({
+ arguments: arguments_,
+ cwd: home,
+ environment: {
+ HOME: home,
+ USERPROFILE: home,
+ PI_CODING_AGENT_DIR: "file:///agent%2fdir",
+ },
+ timeoutMs: 20_000,
+ });
+ const plan = await run([
+ "setup",
+ "--client",
+ "codex",
+ "--dry-run",
+ "--json",
+ ]);
+ expect(plan.exitCode).toBe(0);
+ expect(plan.json).toMatchObject({ status: "planned", appliedActions: [] });
+ expect(await readdir(home)).toEqual([]);
+
+ const rejected = await run(["setup", "--client", "pi", "--yes", "--json"]);
+ expect(rejected.exitCode).not.toBe(0);
+ expect(rejected.json).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ remediation: expect.stringContaining("PI_CODING_AGENT_DIR"),
+ });
+ expect(await readdir(home)).toEqual([]);
+
+ const applied = await run([
+ "setup",
+ "--client",
+ "codex",
+ "--yes",
+ "--json",
+ ]);
+ expect(applied.exitCode).toBe(0);
+ expect(applied.json).toMatchObject({
+ status: "ready",
+ clients: { codex: { status: "configured" } },
+ });
+ const doctor = await run(["doctor", "--client", "codex", "--json"]);
+ expect(doctor.exitCode).toBe(0);
+ expect(doctor.json).toMatchObject({
+ healthy: true,
+ identity: {
+ registrations: [
+ { client: "codex", state: "aligned" },
+ {
+ client: "pi",
+ state: "invalid",
+ remediation: expect.stringContaining("PI_CODING_AGENT_DIR"),
+ },
+ ],
+ },
+ });
+ const piDoctor = await run(["doctor", "--client", "pi", "--json"]);
+ expect(piDoctor.exitCode).not.toBe(0);
+ expect(piDoctor.json).toMatchObject({ healthy: false });
+ expect((await readdir(home)).sort()).toEqual([".agents", ".codex"]);
+ },
+);
diff --git a/tests/acceptance/setup/qwenProfileSetup.test.ts b/tests/acceptance/setup/qwenProfileSetup.test.ts
new file mode 100644
index 0000000000..9f52ba08ff
--- /dev/null
+++ b/tests/acceptance/setup/qwenProfileSetup.test.ts
@@ -0,0 +1,96 @@
+import { access, mkdir, readFile, writeFile } from "node:fs/promises";
+import { join, resolve } from "node:path";
+import { parse } from "jsonc-parser";
+import { expect } from "vitest";
+
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+cliTest(
+ "keeps Qwen profile registration and shared skill consistent through setup, doctor and uninstall",
+ async ({ cli }) => {
+ const root = await createTestTempDirectory("rea-qwen-profile-");
+ const home = join(root, "account");
+ const profile = join(root, "selected profile");
+ const settings = join(profile, "settings.json");
+ const skill = join(
+ home,
+ ".agents",
+ "skills",
+ "reverse-engineer-anything",
+ "SKILL.md",
+ );
+ const environment = { USERPROFILE: home, QWEN_HOME: profile };
+ const original =
+ '{\n // Keep my configuration.\n "model": {"name": "existing"},\n "mcpServers": {"other": {"command": "other-server"}}\n}\n';
+ await mkdir(profile, { recursive: true });
+ await writeFile(settings, original);
+ const run = (arguments_: readonly string[]) =>
+ cli.run({ arguments: [...arguments_, "--json"], environment });
+
+ const plan = await run(["setup", "--client", "qwen_code", "--dry-run"]);
+ expect(plan.exitCode, plan.stderr).toBe(0);
+ expect(plan.json).toMatchObject({
+ status: "planned",
+ plannedActions: [
+ { kind: "configure_client", target: settings },
+ {
+ kind: "install_skill",
+ target: join(home, ".agents", "skills", "reverse-engineer-anything"),
+ },
+ ],
+ });
+ expect(await readFile(settings, "utf8")).toBe(original);
+ await expect(access(skill)).rejects.toMatchObject({ code: "ENOENT" });
+
+ const installed = await run(["setup", "--client", "qwen_code", "--yes"]);
+ expect(installed.exitCode, installed.stderr).toBe(0);
+ expect(installed.json).toMatchObject({ status: "ready" });
+ expect(await readFile(`${settings}.rea.backup`, "utf8")).toBe(original);
+ const configured = await readFile(settings, "utf8");
+ expect(configured).toContain("// Keep my configuration.");
+ expect(parse(configured)).toMatchObject({
+ model: { name: "existing" },
+ mcpServers: {
+ other: { command: "other-server" },
+ rea: {
+ command:
+ process.platform === "win32"
+ ? process.execPath
+ : resolve("scripts/rea.mjs"),
+ args:
+ process.platform === "win32"
+ ? [resolve("scripts/rea.mjs"), "mcp"]
+ : ["mcp"],
+ },
+ },
+ });
+ expect(await readFile(skill, "utf8")).toContain(
+ "name: reverse-engineer-anything",
+ );
+
+ const doctor = await run(["doctor", "--client", "qwen_code", "--skill"]);
+ expect(doctor.exitCode, doctor.stderr).toBe(0);
+ expect(doctor.json).toMatchObject({ healthy: true });
+ const repeated = await run(["setup", "--client", "qwen_code", "--yes"]);
+ expect(repeated.exitCode, repeated.stderr).toBe(0);
+ expect(repeated.json).toMatchObject({
+ status: "ready",
+ appliedActions: [],
+ });
+ expect(await readFile(settings, "utf8")).toBe(configured);
+
+ const removed = await run(["uninstall"]);
+ expect(removed.exitCode, removed.stderr).toBe(0);
+ expect(removed.json).toMatchObject({ status: "complete" });
+ expect(parse(await readFile(settings, "utf8"))).toEqual({
+ model: { name: "existing" },
+ mcpServers: { other: { command: "other-server" } },
+ });
+ await expect(access(skill)).rejects.toMatchObject({ code: "ENOENT" });
+ await expect(access(join(home, ".qwen"))).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ },
+ 120_000,
+);
diff --git a/tests/acceptance/tools/historicalCaptureResources.test.ts b/tests/acceptance/tools/historicalCaptureResources.test.ts
index 7aea0fcce2..87e509e652 100644
--- a/tests/acceptance/tools/historicalCaptureResources.test.ts
+++ b/tests/acceptance/tools/historicalCaptureResources.test.ts
@@ -83,18 +83,22 @@ it("reports real HAR heap exhaustion through CLI and MCP and preserves server us
});
expect(JSON.stringify(response)).not.toContain("caller-marked");
expect(JSON.stringify(response)).not.toContain("rea doctor");
- await expect(
- execute(process.execPath, [
- cli,
- "inspect-web-network-capture",
- path,
- "har",
- "--json",
- ]),
- ).rejects.toMatchObject({
- code: 1,
- stdout: expect.stringContaining('"code": "resource_constraint"'),
- });
+ const failed: unknown = await execute(process.execPath, [
+ cli,
+ "inspect-web-network-capture",
+ path,
+ "har",
+ "--json",
+ ]).then(
+ () => undefined,
+ (cause: unknown) => cause,
+ );
+ expect(failed).toMatchObject({ code: 1 });
+ const stdout =
+ typeof failed === "object" && failed !== null && "stdout" in failed
+ ? String(failed.stdout)
+ : "";
+ expect(JSON.parse(stdout)).toMatchObject({ code: "resource_constraint" });
const small = join(root, "small.har");
await writeFile(small, capture(2));
await client.ping();
diff --git a/tests/boundary/cli/analysisViewCli.test.ts b/tests/boundary/cli/analysisViewCli.test.ts
index e6e6fb51d7..2abaf700ac 100644
--- a/tests/boundary/cli/analysisViewCli.test.ts
+++ b/tests/boundary/cli/analysisViewCli.test.ts
@@ -7,6 +7,8 @@ import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { cliTest } from "../../support/cli/cliFixture.js";
import { analysisViewLayoutEvidence } from "../../fixtures/analysisView.js";
+import { createEvidence } from "../../../src/domain/evidence.js";
+import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
cliTest(
"projects one section from a JSON file of inline layout Evidence",
async ({ cli }) => {
@@ -43,6 +45,48 @@ cliTest(
},
);
+cliTest(
+ "projects native function pseudocode without requiring Ghidra",
+ async ({ cli }) => {
+ const root = await createTestTempDirectory("rea-native-view-cli-");
+ const parent = createEvidence(
+ { path: "/fixtures/native.exe", format: "pe", sha256: "a".repeat(64) },
+ { id: "ghidra", name: "Ghidra", version: "12.1.4" },
+ {
+ operation: "analyze_function",
+ parameters: { address: "0x401000" },
+ result: ghidraFunctionDossier(),
+ },
+ );
+ const input = join(root, "native-view.json");
+ await writeFile(
+ input,
+ JSON.stringify({
+ source: { kind: "inline", evidence: parent },
+ view: { kind: "native", facet: "pseudocode", offset: 0, limit: 12 },
+ }),
+ );
+ const result = await cli.run({
+ arguments: ["inspect-analysis-view", input, "--json"],
+ environment: { HOME: root, XDG_CONFIG_HOME: root, XDG_CACHE_HOME: root },
+ });
+ expect(result.exitCode).toBe(0);
+ expect(result.json).toMatchObject({
+ operation: "inspect_analysis_view",
+ normalized_result: {
+ kind: "native",
+ parent_evidence_id: parent.evidence_id,
+ procedure_address: "0x401000",
+ item: { text: "int fixture_", unit: "utf16-code-units" },
+ },
+ locations: [
+ { kind: "artifact-path", path: "/fixtures/native.exe" },
+ { kind: "address", address: "0x401000" },
+ ],
+ });
+ },
+);
+
cliTest("rejects malformed JSON before projection", async ({ cli }) => {
const root = await createTestTempDirectory("rea-analysis-view-cli-bad-");
const result = await cli.run({
@@ -80,3 +124,48 @@ cliTest("rejects a request that omits source", async ({ cli }) => {
details: { operation: "inspect-analysis-view" },
});
});
+
+for (const [view, reason] of [
+ [{ kind: "unknown-view" }, "invalid_value"],
+ [{}, "missing_argument"],
+] as const) {
+ cliTest(
+ `reports correction details for ${reason} view kind`,
+ async ({ cli }) => {
+ const root = await createTestTempDirectory("rea-view-variant-");
+ const input = join(root, "view.json");
+ await writeFile(
+ input,
+ JSON.stringify({
+ source: {
+ kind: "inline",
+ evidence: analysisViewLayoutEvidence(),
+ },
+ view,
+ }),
+ );
+ const result = await cli.run({
+ arguments: ["inspect-analysis-view", input, "--json"],
+ environment: {
+ HOME: root,
+ XDG_CONFIG_HOME: root,
+ XDG_CACHE_HOME: root,
+ },
+ });
+ expect(result.exitCode).toBe(1);
+ expect(result.json).toMatchObject({
+ code: "invalid_request",
+ details: {
+ issues: [
+ {
+ path: ["view", "kind"],
+ reason,
+ expected: expect.arrayContaining(["summary", "page"]),
+ },
+ ],
+ },
+ });
+ expect(JSON.stringify(result.json)).not.toContain("unknown-view");
+ },
+ );
+}
diff --git a/tests/boundary/cli/cliStartupImports.test.ts b/tests/boundary/cli/cliStartupImports.test.ts
new file mode 100644
index 0000000000..c084b62122
--- /dev/null
+++ b/tests/boundary/cli/cliStartupImports.test.ts
@@ -0,0 +1,76 @@
+import { execFile } from "node:child_process";
+import { mkdir } from "node:fs/promises";
+import { join } from "node:path";
+import { promisify } from "node:util";
+
+import { describe, expect, it } from "vitest";
+
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+
+const execute = promisify(execFile);
+const MARKER = "REA_TEST_RESOLVED ";
+// Dependencies that only one command uses; startup must not resolve them.
+const DEFERRED_PACKAGES = ["playwright-core", "isomorphic-git"] as const;
+// Report each resolved deferred package on stderr without changing resolution.
+const RESOLUTION_HOOK = `data:text/javascript,${encodeURIComponent(`
+import { registerHooks } from "node:module";
+const packages = ${JSON.stringify(DEFERRED_PACKAGES)};
+registerHooks({
+ resolve(specifier, context, next) {
+ const resolved = next(specifier, context);
+ const name = packages.find((item) =>
+ resolved.url.includes("/node_modules/" + item + "/"),
+ );
+ if (name !== undefined)
+ process.stderr.write(${JSON.stringify(MARKER)} + name + "\\n");
+ return resolved;
+ },
+});
+`)}`;
+
+const resolvedPackages = async (
+ arguments_: readonly string[],
+): Promise> => {
+ const { stderr } = await execute(
+ process.execPath,
+ [`--import=${RESOLUTION_HOOK}`, ...arguments_],
+ { cwd: process.cwd(), maxBuffer: 16 * 1_024 * 1_024 },
+ );
+ return new Set(
+ stderr
+ .split("\n")
+ .filter((line) => line.startsWith(MARKER))
+ .map((line) => line.slice(MARKER.length)),
+ );
+};
+
+const evaluate = (source: string): readonly string[] => [
+ "--input-type=module",
+ "--eval",
+ source,
+];
+
+describe("CLI startup imports", () => {
+ it("detects each deferred package when its command code loads", async () => {
+ await expect(
+ resolvedPackages(
+ evaluate('await import("./dist/composition/browserScenario.js");'),
+ ),
+ ).resolves.toContain("playwright-core");
+ const source = await createTestTempDirectory("rea-startup-imports-");
+ await mkdir(join(source, ".git"));
+ await expect(
+ resolvedPackages(
+ evaluate(
+ `const { readReferenceSourceVcs } = await import("./dist/application/ReferenceSourceVcsAdapter.js"); await readReferenceSourceVcs(${JSON.stringify(source)});`,
+ ),
+ ),
+ ).resolves.toContain("isomorphic-git");
+ });
+
+ it("does not load command-specific packages to register commands", async () => {
+ await expect(
+ resolvedPackages(["scripts/rea.mjs", "--help"]),
+ ).resolves.toEqual(new Set());
+ });
+});
diff --git a/tests/boundary/cli/clientProfiles.test.ts b/tests/boundary/cli/clientProfiles.test.ts
new file mode 100644
index 0000000000..996958986a
--- /dev/null
+++ b/tests/boundary/cli/clientProfiles.test.ts
@@ -0,0 +1,786 @@
+import { access, mkdir, readFile, writeFile, rm } from "node:fs/promises";
+import { dirname, join } from "node:path";
+import { expect } from "vitest";
+import { parse } from "jsonc-parser";
+import { PRODUCT_IDENTITY } from "../../../src/identity.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+const cases = [
+ {
+ client: "gemini_cli",
+ env: { GEMINI_CLI_HOME: "profile" },
+ config: "profile/.gemini/settings.json",
+ skill: "profile/.agents/skills",
+ },
+ {
+ client: "opencode",
+ env: { OPENCODE_CONFIG_DIR: "profile" },
+ config: "profile/opencode.json",
+ skill: ".agents/skills",
+ },
+ {
+ client: "opencode",
+ env: { OPENCODE_CONFIG_DIR: "" },
+ config: ".config/opencode/opencode.json",
+ skill: ".agents/skills",
+ },
+ {
+ client: "codex",
+ env: { CODEX_HOME: "" },
+ config: ".codex/config.toml",
+ skill: ".agents/skills",
+ },
+ {
+ client: "grok_build",
+ env: { GROK_HOME: "" },
+ config: ".grok/config.toml",
+ skill: ".agents/skills",
+ },
+ {
+ client: "omp",
+ env: { PI_CODING_AGENT_DIR: "profile" },
+ config: "profile/mcp.json",
+ skill: ".agents/skills",
+ },
+ {
+ client: "devin",
+ env: { XDG_CONFIG_HOME: "profile" },
+ config: "profile/devin/mcp_config.json",
+ skill: ".agents/skills",
+ },
+ {
+ client: "antigravity",
+ env: {},
+ config: ".gemini/config/mcp_config.json",
+ skill: ".gemini/config/skills",
+ },
+];
+for (const testCase of cases.filter(
+ ({ client }) => process.platform !== "win32" || client !== "devin",
+)) {
+ cliTest(
+ `${testCase.client} lifecycle follows the selected roots (${testCase.config})`,
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-client-profile-");
+ const environment: NodeJS.ProcessEnv = { HOME: home, USERPROFILE: home };
+ for (const [key, value] of Object.entries(testCase.env))
+ environment[key] = value === "profile" ? join(home, value) : value;
+ // OMP's relative override is relative to the caller, not silently ignored.
+ if (testCase.client === "omp")
+ environment.PI_CODING_AGENT_DIR = "profile";
+ const config = join(home, testCase.config);
+ const skillDirectory = join(
+ home,
+ testCase.skill,
+ "reverse-engineer-anything",
+ );
+ const run = (arguments_: readonly string[]) =>
+ cli.run({
+ arguments: arguments_,
+ cwd: home,
+ environment,
+ timeoutMs: 20_000,
+ });
+ const planned = await run([
+ "setup",
+ "--client",
+ testCase.client,
+ "--dry-run",
+ "--json",
+ ]);
+ expect(planned.exitCode).toBe(0);
+ expect(planned.json).toMatchObject({
+ status: "planned",
+ plannedActions: expect.arrayContaining([
+ expect.objectContaining({ target: config }),
+ expect.objectContaining({ target: skillDirectory }),
+ ]),
+ });
+ await expect(access(config)).rejects.toMatchObject({ code: "ENOENT" });
+ const setup = await run([
+ "setup",
+ "--client",
+ testCase.client,
+ "--yes",
+ "--json",
+ ]);
+ expect(setup.exitCode).toBe(0);
+ expect(setup.json).toMatchObject({ status: "ready" });
+ await expect(access(config)).resolves.toBeUndefined();
+ expect(
+ await readFile(join(skillDirectory, "SKILL.md"), "utf8"),
+ ).toContain("reverse-engineer-anything");
+ await expect(access(join(home, "config.toml"))).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ const doctor = await run([
+ "doctor",
+ "--client",
+ testCase.client,
+ "--skill",
+ "--json",
+ ]);
+ expect(doctor.exitCode).toBe(0);
+ expect(doctor.json).toMatchObject({
+ healthy: true,
+ identity: { skill: { state: "aligned" } },
+ });
+ if (testCase.client === "grok_build") {
+ const original = await readFile(config, "utf8");
+ await writeFile(
+ config,
+ original.replace(
+ "[mcp_servers.rea]",
+ "[mcp_servers.rea]\ndisabled = true",
+ ),
+ );
+ expect(
+ (await run(["doctor", "--client", "grok_build", "--json"])).exitCode,
+ ).toBe(0);
+ await writeFile(config, original);
+ }
+ const repeat = await run([
+ "setup",
+ "--client",
+ testCase.client,
+ "--yes",
+ "--json",
+ ]);
+ expect(repeat.json).toMatchObject({
+ status: "ready",
+ appliedActions: [],
+ });
+ const uninstall = await run(["uninstall", "--json"]);
+ expect(uninstall.exitCode).toBe(0);
+ expect(uninstall.json).toMatchObject({ status: "complete" });
+ await expect(
+ access(join(skillDirectory, "SKILL.md")),
+ ).rejects.toMatchObject({ code: "ENOENT" });
+ },
+ );
+}
+
+for (const policy of [{ excluded: ["rea"] }, { allowed: ["other"] }]) {
+ cliTest(
+ `Gemini refuses a registration blocked by ${Object.keys(policy)[0]}`,
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-gemini-policy-");
+ const config = join(home, ".gemini/settings.json");
+ const original = JSON.stringify({
+ mcp: policy,
+ mcpServers: {
+ rea: {
+ command: "npx",
+ args: ["-y", PRODUCT_IDENTITY.registrationPackageSpecifier, "mcp"],
+ },
+ },
+ });
+ await mkdir(dirname(config), { recursive: true });
+ await writeFile(config, original);
+ const run = (arguments_: readonly string[]) =>
+ cli.run({
+ arguments: arguments_,
+ cwd: home,
+ environment: { HOME: home, USERPROFILE: home },
+ timeoutMs: 20_000,
+ });
+ const setup = await run([
+ "setup",
+ "--client",
+ "gemini_cli",
+ "--yes",
+ "--json",
+ ]);
+ expect(setup.exitCode).toBe(1);
+ expect(setup.json).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ remediation: expect.stringContaining("mcp." + Object.keys(policy)[0]),
+ });
+ expect(await readFile(config, "utf8")).toBe(original);
+ await expect(access(config + ".rea.backup")).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ const doctor = await run(["doctor", "--client", "gemini_cli", "--json"]);
+ expect(doctor.exitCode).toBe(1);
+ expect(doctor.json).toMatchObject({
+ healthy: false,
+ identity: {
+ registrations: expect.arrayContaining([
+ expect.objectContaining({
+ client: "gemini_cli",
+ state: "stale",
+ remediation: expect.stringContaining("mcp."),
+ }),
+ ]),
+ },
+ });
+ },
+ );
+}
+
+cliTest(
+ "OpenCode diagnoses merged JSON/JSONC and removes owned entries from both files",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-opencode-layers-");
+ const directory = join(home, ".config/opencode");
+ await mkdir(directory, { recursive: true });
+ const lower = join(directory, "opencode.json");
+ const higher = join(directory, "opencode.jsonc");
+ const originalLower = JSON.stringify({
+ mcp: {
+ rea: {
+ type: "local",
+ command: [
+ "npx",
+ "-y",
+ PRODUCT_IDENTITY.registrationPackageSpecifier,
+ "mcp",
+ ],
+ enabled: true,
+ },
+ other: { type: "local", command: ["other"] },
+ },
+ });
+ const originalHigher =
+ '// keep this comment\n{"mcp":{"rea":{"enabled":false}},"theme":"system"}\n';
+ await writeFile(lower, originalLower);
+ await writeFile(higher, originalHigher);
+ const run = (arguments_: readonly string[]) =>
+ cli.run({
+ arguments: arguments_,
+ cwd: home,
+ environment: { HOME: home, USERPROFILE: home },
+ timeoutMs: 20_000,
+ });
+ const before = await run(["doctor", "--client", "opencode", "--json"]);
+ expect(before.exitCode).toBe(1);
+ expect(before.json).toMatchObject({
+ identity: {
+ registrations: expect.arrayContaining([
+ expect.objectContaining({ client: "opencode", state: "stale" }),
+ ]),
+ },
+ });
+ const setup = await run([
+ "setup",
+ "--client",
+ "opencode",
+ "--yes",
+ "--json",
+ ]);
+ expect(setup.exitCode).toBe(0);
+ expect(setup.json).toMatchObject({ status: "ready" });
+ expect(await readFile(lower, "utf8")).toBe(originalLower);
+ expect(await readFile(higher + ".rea.backup", "utf8")).toBe(originalHigher);
+ expect(await readFile(higher, "utf8")).toContain("// keep this comment");
+ expect(
+ (await run(["doctor", "--client", "opencode", "--json"])).exitCode,
+ ).toBe(0);
+ const uninstall = await run(["uninstall", "--json"]);
+ expect(uninstall.exitCode).toBe(0);
+ const report = uninstall.json as {
+ items?: readonly { name: string; status: string }[];
+ };
+ const opencodeItems = report.items?.filter(
+ ({ name }) => name === "opencode",
+ );
+ expect(opencodeItems).toEqual([
+ expect.objectContaining({ status: "removed" }),
+ ]);
+ expect(parse(await readFile(lower, "utf8"))).toEqual({
+ mcp: { other: { type: "local", command: ["other"] } },
+ });
+ expect(parse(await readFile(higher, "utf8"))).toEqual({
+ mcp: {},
+ theme: "system",
+ });
+ },
+);
+
+cliTest(
+ "malformed lower-priority OpenCode config blocks setup before any writes",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-opencode-invalid-layer-");
+ const directory = join(home, ".config/opencode");
+ await mkdir(directory, { recursive: true });
+ const lower = join(directory, "opencode.json");
+ const higher = join(directory, "opencode.jsonc");
+ await writeFile(lower, "malformed");
+ await writeFile(higher, "{}");
+ const result = await cli.run({
+ arguments: ["setup", "--client", "opencode", "--yes", "--json"],
+ cwd: home,
+ environment: { HOME: home, USERPROFILE: home },
+ });
+ expect(result.exitCode).toBe(1);
+ expect(result.json).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ });
+ expect(await readFile(lower, "utf8")).toBe("malformed");
+ expect(await readFile(higher, "utf8")).toBe("{}");
+ await expect(access(higher + ".rea.backup")).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ },
+);
+
+cliTest(
+ "Gemini system policy blocks a new registration without modifying managed files",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-gemini-managed-");
+ const policy = join(home, "managed.json");
+ const original = JSON.stringify({
+ mcp: { excluded: ["rea"] },
+ unrelated: true,
+ });
+ await writeFile(policy, original);
+ const environment = {
+ HOME: home,
+ USERPROFILE: home,
+ GEMINI_CLI_SYSTEM_SETTINGS_PATH: policy,
+ GEMINI_CLI_SYSTEM_DEFAULTS_PATH: join(home, "defaults.json"),
+ };
+ const setup = await cli.run({
+ arguments: ["setup", "--client", "gemini_cli", "--yes", "--json"],
+ cwd: home,
+ environment,
+ });
+ expect(setup.exitCode).toBe(1);
+ expect(setup.json).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ remediation: expect.stringContaining("mcp.excluded"),
+ });
+ expect(await readFile(policy, "utf8")).toBe(original);
+ await expect(
+ access(join(home, ".gemini/settings.json")),
+ ).rejects.toMatchObject({ code: "ENOENT" });
+ await expect(access(policy + ".rea.backup")).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ },
+);
+
+cliTest(
+ "Gemini intersects system/default/user allowlists and accepts an empty effective allowlist",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-gemini-allowlists-");
+ const system = join(home, "system.json");
+ const defaults = join(home, "defaults.json");
+ const user = join(home, ".gemini/settings.json");
+ await mkdir(dirname(user), { recursive: true });
+ await writeFile(
+ system,
+ JSON.stringify({ mcp: { allowed: ["rea", "other"] } }),
+ );
+ await writeFile(defaults, JSON.stringify({ mcp: { allowed: ["other"] } }));
+ await writeFile(
+ user,
+ JSON.stringify({ mcp: { allowed: ["rea", "other"] } }),
+ );
+ const environment = {
+ HOME: home,
+ USERPROFILE: home,
+ GEMINI_CLI_SYSTEM_SETTINGS_PATH: system,
+ GEMINI_CLI_SYSTEM_DEFAULTS_PATH: defaults,
+ };
+ const run = () =>
+ cli.run({
+ arguments: ["setup", "--client", "gemini_cli", "--yes", "--json"],
+ cwd: home,
+ environment,
+ });
+ const blocked = await run();
+ expect(blocked.exitCode).toBe(1);
+ expect(blocked.json).toMatchObject({
+ status: "needs_human",
+ appliedActions: [],
+ });
+ await writeFile(defaults, JSON.stringify({ mcp: { allowed: ["rea"] } }));
+ const allowed = await run();
+ expect(allowed.exitCode).toBe(0);
+ expect(allowed.json).toMatchObject({ status: "ready" });
+ expect(JSON.parse(await readFile(defaults, "utf8"))).toEqual({
+ mcp: { allowed: ["rea"] },
+ });
+ // The consumer treats an empty consolidated allowlist as unrestricted.
+ await writeFile(defaults, JSON.stringify({ mcp: { allowed: [] } }));
+ const empty = await run();
+ expect(empty.exitCode).toBe(0);
+ expect(empty.json).toMatchObject({ status: "ready", appliedActions: [] });
+ },
+);
+
+cliTest(
+ "OpenCode doctor sees a global registration when the custom directory is absent",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-opencode-absent-custom-");
+ const global = join(home, ".config/opencode/opencode.json");
+ const custom = join(home, "missing-custom");
+ await mkdir(dirname(global), { recursive: true });
+ await writeFile(
+ global,
+ JSON.stringify({
+ mcp: {
+ rea: {
+ type: "local",
+ command: [
+ "npx",
+ "-y",
+ PRODUCT_IDENTITY.registrationPackageSpecifier,
+ "mcp",
+ ],
+ enabled: true,
+ },
+ },
+ }),
+ );
+ const run = (args: readonly string[]) =>
+ cli.run({
+ arguments: args,
+ cwd: home,
+ environment: {
+ HOME: home,
+ USERPROFILE: home,
+ OPENCODE_CONFIG_DIR: custom,
+ },
+ });
+ const doctor = await run(["doctor", "--client", "opencode", "--json"]);
+ expect(doctor.exitCode).toBe(0);
+ expect(doctor.json).toMatchObject({
+ identity: {
+ registrations: expect.arrayContaining([
+ expect.objectContaining({ client: "opencode", state: "aligned" }),
+ ]),
+ },
+ });
+ const planned = await run([
+ "setup",
+ "--client",
+ "opencode",
+ "--dry-run",
+ "--json",
+ ]);
+ expect(planned.json).toMatchObject({
+ clientStates: expect.arrayContaining([
+ expect.objectContaining({
+ client: expect.objectContaining({ name: "opencode" }),
+ detected: true,
+ }),
+ ]),
+ });
+ await expect(access(custom)).rejects.toMatchObject({ code: "ENOENT" });
+ },
+);
+
+cliTest(
+ "OpenCode retains the XDG global layer when a custom directory is selected",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-opencode-global-custom-");
+ const global = join(home, ".config/opencode/opencode.json");
+ const custom = join(home, "custom/opencode.jsonc");
+ await mkdir(dirname(global), { recursive: true });
+ await mkdir(dirname(custom), { recursive: true });
+ const owned = {
+ type: "local",
+ command: [
+ "npx",
+ "-y",
+ PRODUCT_IDENTITY.registrationPackageSpecifier,
+ "mcp",
+ ],
+ enabled: true,
+ };
+ const original = JSON.stringify({
+ mcp: { rea: owned, other: { type: "local", command: ["other"] } },
+ });
+ await writeFile(global, original);
+ await writeFile(custom, "{}");
+ const run = (args: readonly string[]) =>
+ cli.run({
+ arguments: args,
+ cwd: home,
+ environment: {
+ HOME: home,
+ USERPROFILE: home,
+ OPENCODE_CONFIG_DIR: dirname(custom),
+ },
+ });
+ const doctor = await run(["doctor", "--client", "opencode", "--json"]);
+ expect(doctor.json).toMatchObject({
+ identity: {
+ registrations: expect.arrayContaining([
+ expect.objectContaining({ client: "opencode", state: "aligned" }),
+ ]),
+ },
+ });
+ await writeFile(global, "malformed");
+ expect(
+ (await run(["setup", "--client", "opencode", "--yes", "--json"])).json,
+ ).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ remediation: expect.stringContaining(global),
+ });
+ expect(await readFile(custom, "utf8")).toBe("{}");
+ await writeFile(global, original);
+ expect(
+ (await run(["setup", "--client", "opencode", "--yes", "--json"]))
+ .exitCode,
+ ).toBe(0);
+ expect((await run(["uninstall", "--json"])).exitCode).toBe(0);
+ expect(parse(await readFile(global, "utf8"))).toEqual({
+ mcp: { other: { type: "local", command: ["other"] } },
+ });
+ expect(parse(await readFile(custom, "utf8"))).toEqual({ mcp: {} });
+ },
+);
+
+cliTest(
+ "Gemini blocks a conflicting system definition and accepts an identical managed entry",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-gemini-managed-server-");
+ const user = join(home, ".gemini/settings.json");
+ const system = join(home, "system.json");
+ const defaults = join(home, "defaults.json");
+ const environment = {
+ HOME: home,
+ USERPROFILE: home,
+ GEMINI_CLI_SYSTEM_SETTINGS_PATH: system,
+ GEMINI_CLI_SYSTEM_DEFAULTS_PATH: defaults,
+ };
+ const run = (args: readonly string[]) =>
+ cli.run({ arguments: args, cwd: home, environment });
+ const command = ["setup", "--client", "gemini_cli", "--yes", "--json"];
+ // A user registration overrides a default definition, including its entire env object.
+ await writeFile(
+ defaults,
+ JSON.stringify({
+ mcpServers: {
+ rea: { command: "managed-default", env: { EXTRA: "default" } },
+ },
+ }),
+ );
+ expect((await run(command)).exitCode).toBe(0);
+ const original = await readFile(user, "utf8");
+ const managed = JSON.stringify({
+ mcpServers: { rea: { command: "managed-override", args: [] } },
+ unrelated: true,
+ });
+ await writeFile(system, managed);
+ expect((await run(command)).json).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ remediation: expect.stringContaining(system),
+ });
+ expect(
+ (await run(["doctor", "--client", "gemini_cli", "--json"])).json,
+ ).toMatchObject({
+ healthy: false,
+ identity: {
+ registrations: expect.arrayContaining([
+ expect.objectContaining({
+ client: "gemini_cli",
+ state: "stale",
+ remediation: expect.stringContaining("mcpServers.rea"),
+ }),
+ ]),
+ },
+ });
+ expect(await readFile(user, "utf8")).toBe(original);
+ expect(await readFile(system, "utf8")).toBe(managed);
+ const aligned = JSON.stringify({
+ mcpServers: JSON.parse(original).mcpServers,
+ });
+ await writeFile(system, aligned);
+ expect((await run(command)).json).toMatchObject({
+ status: "ready",
+ appliedActions: [],
+ });
+ expect(
+ (await run(["doctor", "--client", "gemini_cli", "--json"])).exitCode,
+ ).toBe(0);
+ expect(await readFile(system, "utf8")).toBe(aligned);
+ await expect(access(system + ".rea.backup")).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ },
+);
+
+cliTest(
+ "Gemini workspace policy follows persisted trust and explicit trust overrides",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-gemini-workspace-policy-");
+ const project = join(home, "project");
+ const workspace = join(project, ".gemini/settings.json");
+ const trust = join(home, "trust.json");
+ await mkdir(dirname(workspace), { recursive: true });
+ const environment: NodeJS.ProcessEnv = {
+ HOME: home,
+ USERPROFILE: home,
+ GEMINI_CLI_SYSTEM_SETTINGS_PATH: join(home, "system.json"),
+ GEMINI_CLI_SYSTEM_DEFAULTS_PATH: join(home, "defaults.json"),
+ GEMINI_CLI_TRUSTED_FOLDERS_PATH: trust,
+ };
+ const run = (args: readonly string[]) =>
+ cli.run({ arguments: args, cwd: project, environment });
+ const command = ["setup", "--client", "gemini_cli", "--yes", "--json"];
+ const original = JSON.stringify({
+ mcp: { excluded: ["rea"] },
+ unrelated: true,
+ });
+ await writeFile(workspace, original);
+ await writeFile(
+ trust,
+ JSON.stringify({ [home]: "TRUST_FOLDER", [project]: "DO_NOT_TRUST" }),
+ );
+ expect((await run(command)).exitCode).toBe(0);
+ await writeFile(
+ trust,
+ JSON.stringify({ [join(project, "child")]: "TRUST_PARENT" }),
+ );
+ expect((await run(command)).json).toMatchObject({
+ status: "needs_human",
+ appliedActions: [],
+ remediation: expect.stringContaining(workspace),
+ });
+ expect(
+ (await run(["doctor", "--client", "gemini_cli", "--json"])).json,
+ ).toMatchObject({
+ healthy: false,
+ identity: {
+ registrations: expect.arrayContaining([
+ expect.objectContaining({
+ client: "gemini_cli",
+ state: "stale",
+ remediation: expect.stringContaining("mcp.excluded"),
+ }),
+ ]),
+ },
+ });
+ environment.GEMINI_CLI_TRUST_WORKSPACE = "false";
+ expect((await run(command)).json).toMatchObject({
+ status: "ready",
+ appliedActions: [],
+ });
+ environment.GEMINI_CLI_TRUST_WORKSPACE = "true";
+ await writeFile(workspace, JSON.stringify({ mcp: { allowed: ["other"] } }));
+ expect((await run(command)).json).toMatchObject({
+ status: "needs_human",
+ remediation: expect.stringContaining("mcp.allowed"),
+ });
+ // Restricted mode takes precedence over an explicit trust opt-in.
+ environment.GEMINI_RESTRICTED_MODE = "true";
+ expect((await run(command)).json).toMatchObject({
+ status: "ready",
+ appliedActions: [],
+ });
+ expect(JSON.parse(await readFile(workspace, "utf8"))).toEqual({
+ mcp: { allowed: ["other"] },
+ });
+ await expect(access(workspace + ".rea.backup")).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ },
+);
+
+cliTest(
+ "Gemini workspace definitions use shallow precedence and remain read-only",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-gemini-workspace-server-");
+ const project = join(home, "project");
+ const workspace = join(project, ".gemini/settings.json");
+ const system = join(home, "system.json");
+ await mkdir(dirname(workspace), { recursive: true });
+ const environment: NodeJS.ProcessEnv = {
+ HOME: home,
+ USERPROFILE: home,
+ GEMINI_CLI_SYSTEM_SETTINGS_PATH: system,
+ GEMINI_CLI_SYSTEM_DEFAULTS_PATH: join(home, "defaults.json"),
+ GEMINI_CLI_TRUST_WORKSPACE: "true",
+ };
+ const run = (args: readonly string[]) =>
+ cli.run({ arguments: args, cwd: project, environment });
+ const command = ["setup", "--client", "gemini_cli", "--yes", "--json"];
+ const original = JSON.stringify({
+ mcpServers: { rea: { command: "project-server", args: [] } },
+ });
+ await writeFile(workspace, original);
+ expect((await run(command)).json).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ remediation: expect.stringContaining(workspace),
+ });
+ await expect(
+ access(join(home, ".gemini/settings.json")),
+ ).rejects.toMatchObject({ code: "ENOENT" });
+ // Obtain this setup invocation's actual command while workspace settings are untrusted.
+ environment.GEMINI_CLI_TRUST_WORKSPACE = "false";
+ expect((await run(command)).exitCode).toBe(0);
+ const user = JSON.parse(
+ await readFile(join(home, ".gemini/settings.json"), "utf8"),
+ );
+ environment.GEMINI_CLI_TRUST_WORKSPACE = "true";
+ // System's entire entry wins over the workspace entry.
+ await writeFile(system, JSON.stringify({ mcpServers: user.mcpServers }));
+ expect((await run(command)).exitCode).toBe(0);
+ expect(
+ (await run(["doctor", "--client", "gemini_cli", "--json"])).exitCode,
+ ).toBe(0);
+ expect(await readFile(workspace, "utf8")).toBe(original);
+ await expect(access(workspace + ".rea.backup")).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ },
+);
+
+cliTest(
+ "Gemini validates trust configuration and honors a disabled folder trust feature",
+ async ({ cli }) => {
+ const home = await createTestTempDirectory("rea-gemini-trust-invalid-");
+ const project = join(home, "project");
+ const workspace = join(project, ".gemini/settings.json");
+ const user = join(home, ".gemini/settings.json");
+ const trust = join(home, ".gemini/trustedFolders.json");
+ await mkdir(dirname(workspace), { recursive: true });
+ await mkdir(dirname(user), { recursive: true });
+ await writeFile(workspace, JSON.stringify({ mcp: { excluded: ["rea"] } }));
+ await writeFile(trust, JSON.stringify({ [project]: "INVALID" }));
+ const run = () =>
+ cli.run({
+ arguments: ["setup", "--client", "gemini_cli", "--yes", "--json"],
+ cwd: project,
+ environment: {
+ HOME: home,
+ USERPROFILE: home,
+ GEMINI_CLI_SYSTEM_SETTINGS_PATH: join(home, "system.json"),
+ GEMINI_CLI_SYSTEM_DEFAULTS_PATH: join(home, "defaults.json"),
+ },
+ });
+ expect((await run()).json).toMatchObject({
+ status: "needs_human",
+ plannedActions: [],
+ appliedActions: [],
+ remediation: expect.stringContaining(trust),
+ });
+ await expect(access(user)).rejects.toMatchObject({ code: "ENOENT" });
+ await rm(trust);
+ await writeFile(
+ user,
+ JSON.stringify({ security: { folderTrust: { enabled: false } } }),
+ );
+ expect((await run()).json).toMatchObject({
+ status: "needs_human",
+ remediation: expect.stringContaining("mcp.excluded"),
+ });
+ },
+);
diff --git a/tests/boundary/cli/jsonOutput.test.ts b/tests/boundary/cli/jsonOutput.test.ts
index 0294073885..fd03f0e920 100644
--- a/tests/boundary/cli/jsonOutput.test.ts
+++ b/tests/boundary/cli/jsonOutput.test.ts
@@ -16,6 +16,7 @@ import { silentLogger } from "../../../src/logger.js";
const collect = async (
value: unknown,
format: "json" | "jsonl" = "json",
+ terminal = true,
): Promise => {
const chunks: Buffer[] = [];
const destination = new Writable({
@@ -28,6 +29,7 @@ const collect = async (
callback();
},
});
+ if (terminal) Object.defineProperty(destination, "isTTY", { value: true });
await writeJsonOutput(value, destination, format);
return Buffer.concat(chunks).toString("utf8");
};
@@ -52,6 +54,16 @@ describe("incremental CLI JSON", () => {
expect(await collect(value)).toBe(`${JSON.stringify(value, null, 2)}\n`);
});
+ it("indents JSON only for a terminal and writes piped JSON compactly", async () => {
+ const value = { nested: [{ value: null }], text: "雪𝟠" };
+ expect(await collect(value, "json", true)).toBe(
+ `${JSON.stringify(value, null, 2)}\n`,
+ );
+ expect(await collect(value, "json", false)).toBe(
+ `${JSON.stringify(value)}\n`,
+ );
+ });
+
it("writes compact JSONL records and preserves an empty record set", async () => {
const value = [{ text: "first\nrecord" }, { text: "雪𝟠" }];
expect(await collect(value, "jsonl")).toBe(
@@ -78,6 +90,7 @@ describe("incremental CLI JSON", () => {
}, 1);
},
});
+ Object.defineProperty(destination, "isTTY", { value: true });
await writeJsonOutput({ leaf }, destination);
expect(largestChunk).toBeLessThan(Buffer.byteLength(leaf));
const actual = Buffer.concat(chunks).toString("utf8");
@@ -212,6 +225,8 @@ describe("streamed CLI result surface", () => {
expect(operation).toBe("inspect_managed_members");
return value;
});
+ // A terminal destination exercises the indented encoding past the limit.
+ Object.defineProperty(destination, "isTTY", { value: true });
const argv = ["inspect-managed-members", "fixture.dll", "--json"];
const output = createStreamedCliJsonOutput(argv, destination);
if (output === undefined) throw new Error("Missing JSON output surface");
diff --git a/tests/boundary/filesystem/artifactInventoryEntryOrder.test.ts b/tests/boundary/filesystem/artifactInventoryEntryOrder.test.ts
index 6d106e3352..4d40227f32 100644
--- a/tests/boundary/filesystem/artifactInventoryEntryOrder.test.ts
+++ b/tests/boundary/filesystem/artifactInventoryEntryOrder.test.ts
@@ -50,10 +50,9 @@ describe("artifact inventory entry order", () => {
const parent = artifactOccurrenceAt(observed, parentPath);
expect(observed.edges).toContainEqual(
expect.objectContaining({
- logical_path: path,
+ occurrence_id: occurrence.occurrence_id,
relation: "contains",
parent_artifact_id: parent.artifact_id,
- child_artifact_id: occurrence.artifact_id,
}),
);
}
@@ -100,7 +99,6 @@ describe("entry-order validation and reader controls", () => {
it.each([
["pkg/data.txt", "pkg/data.txt/"],
["pkg/data.txt/", "pkg/data.txt"],
- ["pkg/data.txt", "PKG/other.txt"],
["pkg/data.txt", "pkg"],
["../escape.txt"],
])("keeps invalid/colliding paths rejected for %j", async (...entries) => {
@@ -112,6 +110,25 @@ describe("entry-order validation and reader controls", () => {
});
});
+ it("keeps case-distinct directory names separate in either entry order", async () => {
+ const root = await createTestTempDirectory("rea-entry-order-case-");
+ const archive = join(root, "case-distinct.zip");
+ const entries = ["pkg/", "pkg/data.txt", "PKG/", "PKG/other.txt"];
+ const parents = {
+ ".": null,
+ pkg: ".",
+ "pkg/data.txt": "pkg",
+ PKG: ".",
+ "PKG/other.txt": "PKG",
+ };
+ for (const ordered of [entries, entries.toReversed()]) {
+ await writeOrderedZip(archive, ordered);
+ expect(artifactParentPaths(await inventoryArtifact(archive))).toEqual(
+ parents,
+ );
+ }
+ });
+
it("preserves ordinary directory and ASAR structure", async () => {
const root = await createTestTempDirectory("rea-entry-order-readers-");
const directory = join(root, "source");
diff --git a/tests/boundary/filesystem/asarInventoryContainment.test.ts b/tests/boundary/filesystem/asarInventoryContainment.test.ts
index 1be8a81739..0f4b6bab9f 100644
--- a/tests/boundary/filesystem/asarInventoryContainment.test.ts
+++ b/tests/boundary/filesystem/asarInventoryContainment.test.ts
@@ -75,10 +75,9 @@ describe("expanded ASAR inventory containment", () => {
const child = artifactOccurrenceAt(nested, path);
expect(nested.edges).toContainEqual(
expect.objectContaining({
- logical_path: path,
+ occurrence_id: child.occurrence_id,
relation: "contains",
parent_artifact_id: artifactOccurrenceAt(nested, parent).artifact_id,
- child_artifact_id: child.artifact_id,
}),
);
if (path.startsWith("resources/app.asar/"))
diff --git a/tests/boundary/filesystem/cliJsonInput.test.ts b/tests/boundary/filesystem/cliJsonInput.test.ts
index 099c95b252..dd4809ea81 100644
--- a/tests/boundary/filesystem/cliJsonInput.test.ts
+++ b/tests/boundary/filesystem/cliJsonInput.test.ts
@@ -9,6 +9,7 @@ import { createCli } from "../../../src/cli.js";
import { parseCliJsonInput } from "../../../src/cliJsonInput.js";
import { readCliJsonFile } from "../../../src/cliJsonFile.js";
import { analysisCliErrorEnvelopeSchema } from "../../../src/contracts/errorSchemas.js";
+import { JSON_BYTE_ORDER_MARK_MESSAGE } from "../../../src/application/Utf8JsonInput.js";
import { readWithoutFifoWriter } from "../../fixtures/fifoInput.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
@@ -160,6 +161,21 @@ describe("CLI JSON streamed input", () => {
}
});
+ it("names a leading byte-order mark in small and streamed files", async () => {
+ const root = await createTestTempDirectory("rea-json-input-bom-");
+ const path = join(root, "input.json");
+ for (const body of ["{}", `${" ".repeat(9 * 1024 * 1024)}{}`]) {
+ await writeFile(path, `\uFEFF${body}`);
+ expect(await parseCliJsonInput(path, "test-input")).toMatchObject({
+ ok: false,
+ error: {
+ input_reason: "invalid-json",
+ details: { issues: [{ message: JSON_BYTE_ORDER_MARK_MESSAGE }] },
+ },
+ });
+ }
+ });
+
it.each([Buffer.from([0xf0, 0x28, 0x8c, 0xbc]), Buffer.from([0xe2, 0x82])])(
"rejects malformed and incomplete UTF-8 at a read boundary (%j)",
async (invalid) => {
diff --git a/tests/boundary/filesystem/clientConfigurationDialects.test.ts b/tests/boundary/filesystem/clientConfigurationDialects.test.ts
index 71d6512ec1..a378cc7582 100644
--- a/tests/boundary/filesystem/clientConfigurationDialects.test.ts
+++ b/tests/boundary/filesystem/clientConfigurationDialects.test.ts
@@ -32,6 +32,7 @@ const getClients = (home: string) =>
"vscode",
"devin",
"omp",
+ "pi",
].includes(name),
);
@@ -44,6 +45,7 @@ describe("additional client configuration dialects", () => {
"vscode",
"devin",
"omp",
+ "pi",
] as const)("registers, reads back, and uninstalls %s", async (name) => {
const home = await createTestTempDirectory("rea-client-dialect-");
const client = getClients(home).find(
@@ -78,7 +80,7 @@ describe("additional client configuration dialects", () => {
command,
enabled: true,
});
- if (name === "vscode" || name === "omp")
+ if (name === "vscode" || name === "omp" || name === "pi")
expect(registration).toMatchObject({
type: "stdio",
command: "npx",
diff --git a/tests/boundary/filesystem/clientLifecycle.test.ts b/tests/boundary/filesystem/clientLifecycle.test.ts
index 88eab99539..62efeddd77 100644
--- a/tests/boundary/filesystem/clientLifecycle.test.ts
+++ b/tests/boundary/filesystem/clientLifecycle.test.ts
@@ -59,9 +59,12 @@ describe("client configuration filesystem lifecycle", () => {
"antigravity",
"copilot_cli",
"commandcode",
+ "qwen_code",
"vscode",
"grok_build",
"omp",
+ "pi",
+ "hermes",
"grok_bot",
]);
expect(
diff --git a/tests/boundary/filesystem/dependencyInstall.test.ts b/tests/boundary/filesystem/dependencyInstall.test.ts
index 66378360f4..806d91b321 100644
--- a/tests/boundary/filesystem/dependencyInstall.test.ts
+++ b/tests/boundary/filesystem/dependencyInstall.test.ts
@@ -169,7 +169,7 @@ const createBuildCacheFixture = async (): Promise => {
"scripts/clean-build-output.mjs",
"scripts/generate-package-metadata.mjs",
"scripts/lib/generated-file.mjs",
- "skill-src/reverse-engineer-anything/SKILL.md",
+ ".agents/skills/reverse-engineer-anything/SKILL.md",
"src/generatedPackageMetadata.ts",
];
await Promise.all(
diff --git a/tests/boundary/filesystem/generatedSkill.test.ts b/tests/boundary/filesystem/generatedSkill.test.ts
index 5df6e3146a..2b7ebf754b 100644
--- a/tests/boundary/filesystem/generatedSkill.test.ts
+++ b/tests/boundary/filesystem/generatedSkill.test.ts
@@ -21,7 +21,7 @@ async function fixture(eol = "\n") {
await mkdir(dirname(join(root, path)), { recursive: true });
await copyFile(join(repositoryRoot, path), join(root, path));
}
- const sourceRoot = join(root, "skill-src/reverse-engineer-anything");
+ const sourceRoot = join(root, ".agents/skills/reverse-engineer-anything");
const outputRoot = join(root, "skills/reverse-engineer-anything");
await mkdir(join(sourceRoot, "references"), { recursive: true });
await mkdir(join(root, "dist"));
diff --git a/tests/boundary/filesystem/hermesClientConfiguration.test.ts b/tests/boundary/filesystem/hermesClientConfiguration.test.ts
new file mode 100644
index 0000000000..e2f2cf74ca
--- /dev/null
+++ b/tests/boundary/filesystem/hermesClientConfiguration.test.ts
@@ -0,0 +1,165 @@
+import { mkdir, readFile, writeFile } from "node:fs/promises";
+import { join } from "node:path";
+
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import { parse as parseYaml } from "yaml";
+
+import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js";
+import { npxRegistrationCommand } from "../../../src/application/ClientRegistrationIdentity.js";
+import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js";
+import { supportedClients } from "../../../src/application/SupportedClients.js";
+import { systemUninstallHost } from "../../../src/application/Uninstall.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { clearClientLocationEnvironment } from "../../fixtures/clientEnvironment.js";
+
+/**
+ * The registration command setup writes on the host running the tests. Windows
+ * wraps it in `cmd.exe /d /c`, so the expected entry follows the host too.
+ */
+const command = npxRegistrationCommand(process.platform);
+
+beforeEach(clearClientLocationEnvironment);
+afterEach(() => vi.unstubAllEnvs());
+
+const hermesPath = (
+ env: Parameters[2],
+ platform: NodeJS.Platform = "linux",
+) =>
+ supportedClients(join("/home/a"), platform, env).find(
+ ({ name }) => name === "hermes",
+ );
+
+describe("Hermes configuration paths", () => {
+ it("resolves config.yaml under the platform Hermes home", () => {
+ expect(hermesPath({})).toEqual({
+ name: "hermes",
+ displayName: "Hermes",
+ configPath: join("/home/a", ".hermes", "config.yaml"),
+ markerPath: join("/home/a", ".hermes"),
+ format: "hermes",
+ });
+ // Windows uses the native data directory, matching Hermes's own resolver.
+ expect(
+ hermesPath({ LOCALAPPDATA: "C:\\Users\\a\\AppData\\Local" }, "win32")
+ ?.configPath,
+ ).toBe(join("C:\\Users\\a\\AppData\\Local", "hermes", "config.yaml"));
+ });
+
+ it("expands and trims Hermes overrides without changing the suffix", () => {
+ expect(hermesPath({ HERMES_HOME: " ~/profile " })?.markerPath).toBe(
+ join("/home/a", "profile"),
+ );
+ expect(
+ hermesPath({
+ HERMES_HOME: "${PROFILE_ROOT}/agent",
+ PROFILE_ROOT: "/custom",
+ })?.markerPath,
+ ).toBe(join("/custom", "agent"));
+ expect(
+ hermesPath({
+ HERMES_HOME: "$PROFILE_ROOT/agent",
+ PROFILE_ROOT: "/custom",
+ })?.markerPath,
+ ).toBe(join("/custom", "agent"));
+ expect(hermesPath({ LOCALAPPDATA: " " }, "win32")?.markerPath).toBe(
+ join("/home/a", "AppData", "Local", "hermes"),
+ );
+ });
+
+ it("honours HERMES_HOME and Hermes's data-directory suffix", () => {
+ expect(
+ hermesPath({ HERMES_HOME: join("/srv", "hermes") })?.configPath,
+ ).toBe(join("/srv", "hermes", "config.yaml"));
+ // A blank override falls through to the platform default.
+ expect(hermesPath({ HERMES_HOME: " " })?.configPath).toBe(
+ join("/home/a", ".hermes", "config.yaml"),
+ );
+ // Hermes appends the suffix to the literal directory name.
+ expect(hermesPath({ HERMES_DATA_DIR_SUFFIX: "-dev" })?.configPath).toBe(
+ join("/home/a", ".hermes-dev", "config.yaml"),
+ );
+ expect(
+ hermesPath(
+ {
+ HERMES_DATA_DIR_SUFFIX: "-dev",
+ LOCALAPPDATA: "C:\\Users\\a\\AppData\\Local",
+ },
+ "win32",
+ )?.configPath,
+ ).toBe(join("C:\\Users\\a\\AppData\\Local", "hermes-dev", "config.yaml"));
+ });
+});
+
+describe("Hermes YAML registration", () => {
+ const hermesClient = async () => {
+ const home = await createTestTempDirectory("rea-hermes-");
+ const client = supportedClients(home, process.platform, {}).find(
+ ({ name }) => name === "hermes",
+ );
+ if (client?.markerPath === undefined)
+ throw new Error("missing Hermes client");
+ await mkdir(client.markerPath, { recursive: true });
+ return { home, client };
+ };
+
+ it("writes mcp_servers, preserves comments, reads back, and uninstalls", async () => {
+ const { home, client } = await hermesClient();
+ await writeFile(
+ client.configPath,
+ `# Hermes configuration
+display:
+ language: en
+mcp_servers:
+ # An independently managed server.
+ other:
+ command: other-server
+`,
+ );
+
+ expect(
+ (await configureClientConfiguration(client, {}, command)).status,
+ ).toBe("configured");
+ const text = await readFile(client.configPath, "utf8");
+ expect(text).toContain("# Hermes configuration");
+ expect(text).toContain("# An independently managed server.");
+ expect(parseYaml(text)).toEqual({
+ display: { language: "en" },
+ mcp_servers: {
+ other: { command: "other-server" },
+ rea: { command: command[0], args: command.slice(1), enabled: true },
+ },
+ });
+
+ expect(await configureClientConfiguration(client, {}, command)).toEqual({
+ status: "unchanged",
+ });
+ expect(
+ (
+ await readClientRegistrationStatuses(home, undefined, {
+ platform: process.platform,
+ environment: {},
+ })
+ ).find(({ client: name }) => name === "hermes"),
+ ).toMatchObject({ state: "aligned", command });
+
+ expect((await systemUninstallHost(home).removeClient(client)).status).toBe(
+ "removed",
+ );
+ expect(parseYaml(await readFile(client.configPath, "utf8"))).toEqual({
+ display: { language: "en" },
+ mcp_servers: { other: { command: "other-server" } },
+ });
+ });
+
+ it("creates config.yaml when Hermes has no configuration yet", async () => {
+ const { client } = await hermesClient();
+ expect(
+ (await configureClientConfiguration(client, {}, command)).status,
+ ).toBe("configured");
+ expect(parseYaml(await readFile(client.configPath, "utf8"))).toEqual({
+ mcp_servers: {
+ rea: { command: command[0], args: command.slice(1), enabled: true },
+ },
+ });
+ });
+});
diff --git a/tests/boundary/filesystem/javascriptApplicationDeepReturns.test.ts b/tests/boundary/filesystem/javascriptApplicationDeepReturns.test.ts
new file mode 100644
index 0000000000..32a2d99af7
--- /dev/null
+++ b/tests/boundary/filesystem/javascriptApplicationDeepReturns.test.ts
@@ -0,0 +1,44 @@
+import { execFile } from "node:child_process";
+import { writeFile } from "node:fs/promises";
+import { join } from "node:path";
+import { promisify } from "node:util";
+
+import { expect, it } from "vitest";
+
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+
+it("analyzes direct returns after deep source without exhausting the default process stack", async () => {
+ const root = await createTestTempDirectory("rea-deep-direct-returns-");
+ await writeFile(
+ join(root, "app.js"),
+ `export function render() { object${".property".repeat(20_000)}; function nested() { return "nested"; } class Inner { method() { return "method"; } } return "ready"; }`,
+ );
+ const service = new URL(
+ "../../../dist/application/javascript/JavaScriptApplicationService.js",
+ import.meta.url,
+ ).href;
+ const script = `
+ import { analyzeJavaScriptApplication } from ${JSON.stringify(service)};
+ const result = await analyzeJavaScriptApplication({ input_path: ${JSON.stringify(root)}, format: "directory" });
+ if (!result.ok) throw result.error;
+ const nodes = result.value.normalized_result.semantic_graph.nodes;
+ console.log(JSON.stringify(nodes.filter(node => node.kind === "return-site").map(node => node.function_node_id)));
+ `;
+ const { stdout, stderr } = await promisify(execFile)(
+ process.execPath,
+ ["--input-type=module", "-e", script],
+ {
+ timeout: 10_000,
+ // Bound the child heap while retaining Node's default stack size.
+ env: {
+ HOME: root,
+ TMPDIR: root,
+ NODE_OPTIONS: "--max-old-space-size=768",
+ },
+ },
+ );
+ expect(stderr).toBe("");
+ const owners: unknown = JSON.parse(stdout);
+ expect(owners).toHaveLength(3);
+ expect(new Set(Array.isArray(owners) ? owners : []).size).toBe(3);
+});
diff --git a/tests/boundary/filesystem/ompClientConfiguration.test.ts b/tests/boundary/filesystem/ompClientConfiguration.test.ts
index 0f5aeb2cc9..7a961e08d6 100644
--- a/tests/boundary/filesystem/ompClientConfiguration.test.ts
+++ b/tests/boundary/filesystem/ompClientConfiguration.test.ts
@@ -38,9 +38,9 @@ describe("OMP configuration paths", () => {
expect(ompPath({ PI_CODING_AGENT_DIR: "/custom/agent" })?.configPath).toBe(
"/custom/agent/mcp.json",
);
- // A relative override names a different file in every working directory.
+ // Match OMP by resolving the override against the caller working directory.
expect(ompPath({ PI_CODING_AGENT_DIR: "agent" })?.configPath).toBe(
- "/home/a/.omp/agent/mcp.json",
+ resolve("agent", "mcp.json"),
);
});
diff --git a/tests/boundary/filesystem/piClientConfiguration.test.ts b/tests/boundary/filesystem/piClientConfiguration.test.ts
new file mode 100644
index 0000000000..bec8a7d086
--- /dev/null
+++ b/tests/boundary/filesystem/piClientConfiguration.test.ts
@@ -0,0 +1,333 @@
+import { mkdir, readFile, writeFile } from "node:fs/promises";
+import { join } from "node:path";
+
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+
+import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js";
+import { parseClientConfiguration } from "../../../src/application/ClientConfigurationDocument.js";
+import {
+ configureClientConfiguration,
+ inspectClientConfiguration,
+} from "../../../src/application/SetupClientConfiguration.js";
+import { detectClients } from "../../../src/application/SetupHost.js";
+import { supportedClients } from "../../../src/application/SupportedClients.js";
+import { systemUninstallHost } from "../../../src/application/Uninstall.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import {
+ clearClientLocationEnvironment,
+ NPX_REGISTRATION_COMMAND,
+} from "../../fixtures/clientEnvironment.js";
+
+beforeEach(clearClientLocationEnvironment);
+afterEach(() => vi.unstubAllEnvs());
+
+const piPath = (
+ home: string,
+ platform: NodeJS.Platform,
+ env: Parameters[2],
+) => supportedClients(home, platform, env).find(({ name }) => name === "pi");
+
+// Catch fallback to OMP paths, loss of relative overrides, and normalization
+// that differs from Pi's public getAgentDir()/normalizePath boundary.
+describe("Pi user configuration paths", () => {
+ it.each([
+ [undefined, "/home/a/.pi/agent"],
+ ["", "/home/a/.pi/agent"],
+ ["/custom/agent", "/custom/agent"],
+ ["~", "/home/a"],
+ ["~/custom/agent", "/home/a/custom/agent"],
+ ["agent", "agent"],
+ ["./agent", "./agent"],
+ [" agent ", " agent "],
+ ["~other/agent", "~other/agent"],
+ ["file:///custom/agent%20dir", "/custom/agent dir"],
+ ["/mnt/c/agent", "/mnt/c/agent"],
+ ])(
+ "uses Pi's agent directory for %s, independently of OMP",
+ (override, expected) => {
+ const client = piPath("/home/a", "linux", {
+ PI_CODING_AGENT_DIR: override,
+ OMP_PROFILE: "work",
+ PI_PROFILE: "legacy",
+ PI_CONFIG_DIR: ".omp-other",
+ });
+ expect(client?.markerPath).toBe(expected);
+ expect(client?.configPath).toBe(join(expected, "mcp.json"));
+ },
+ );
+
+ it.each([
+ [undefined, "C:\\Users\\a\\.pi\\agent"],
+ ["", "C:\\Users\\a\\.pi\\agent"],
+ ["~", "C:\\Users\\a"],
+ ["~/agent", "C:\\Users\\a\\agent"],
+ ["~\\agent", "C:\\Users\\a\\agent"],
+ ["C:\\custom\\agent", "C:\\custom\\agent"],
+ ["/c/custom/agent", "C:\\custom\\agent"],
+ ["/mnt/c/custom/agent", "C:\\custom\\agent"],
+ ["/cygdrive/c/custom/agent", "C:\\custom\\agent"],
+ ["/c", "C:\\"],
+ ["//server/share/agent", "//server/share/agent"],
+ ["\\\\server\\share\\agent", "\\\\server\\share\\agent"],
+ ["agent", "agent"],
+ ["file:///C:/custom/agent%20dir", "C:\\custom\\agent dir"],
+ ["file://server/share/agent", "\\\\server\\share\\agent"],
+ ])("normalizes Windows Pi override %s", (override, expected) => {
+ const client = piPath("C:\\Users\\a", "win32", {
+ PI_CODING_AGENT_DIR: override,
+ });
+ expect(client?.markerPath).toBe(expected);
+ expect(client?.configPath).toBe(
+ `${expected.replace(/[\\/]+$/u, "").replaceAll("/", "\\")}\\mcp.json`,
+ );
+ });
+
+ it("reports malformed file URLs instead of silently choosing the default", async () => {
+ const home = await createTestTempDirectory("rea-pi-invalid-path-");
+ const client = piPath(home, "linux", {
+ PI_CODING_AGENT_DIR: "file:///agent%2fdir",
+ });
+ expect(client).toMatchObject({
+ configPath: "file:///agent%2fdir",
+ configPathError: expect.stringContaining("PI_CODING_AGENT_DIR"),
+ });
+ if (client === undefined) throw new Error("missing Pi client");
+ expect(await configureClientConfiguration(client)).toEqual({
+ status: "failed",
+ reason: "path",
+ });
+ expect(await systemUninstallHost(home).removeClient(client)).toMatchObject({
+ status: "failed",
+ detail: expect.stringContaining("PI_CODING_AGENT_DIR"),
+ });
+ expect(
+ await detectClients(home, "linux", {
+ PI_CODING_AGENT_DIR: "file:///agent%2fdir",
+ }),
+ ).toEqual([]);
+ });
+
+ it("detects the normalized override marker without needing mcp.json", async () => {
+ const home = await createTestTempDirectory("rea-pi-detection-");
+ await mkdir(join(home, "custom-agent"));
+ const detected = await detectClients(home, process.platform, {
+ PI_CODING_AGENT_DIR: "~/custom-agent",
+ });
+ expect(detected.find(({ name }) => name === "pi")?.configPath).toBe(
+ join(home, "custom-agent", "mcp.json"),
+ );
+ });
+});
+
+const piClient = async () => {
+ const home = await createTestTempDirectory("rea-pi-config-");
+ const client = piPath(home, process.platform, {});
+ if (client?.markerPath === undefined) throw new Error("missing Pi client");
+ await mkdir(client.markerPath, { recursive: true });
+ return { home, client };
+};
+
+describe("Pi registration semantics", () => {
+ it("preserves unrelated content and OMP-only lists during setup, repeat, and uninstall", async () => {
+ const { home, client } = await piClient();
+ // Pi loads mcp.json with strict JSON, so the retained unrelated settings
+ // are valid strict JSON rather than the commented JSONC fixture.
+ const original = `{"autoEnableCodemode": false,
+ "mcpServers": { "other": { "command": "other-server" } },
+ "enabledServers": ["other"],
+ "disabledServers": ["rea", "other"]
+}\n`;
+ await writeFile(client.configPath, original);
+ expect(
+ await configureClientConfiguration(client, {}, NPX_REGISTRATION_COMMAND),
+ ).toMatchObject({ status: "configured" });
+ expect(await readFile(`${client.configPath}.rea.backup`, "utf8")).toBe(
+ original,
+ );
+ const configured = await readFile(client.configPath, "utf8");
+ expect(JSON.parse(configured)).toEqual({
+ autoEnableCodemode: false,
+ mcpServers: {
+ other: { command: "other-server" },
+ rea: {
+ type: "stdio",
+ command: "npx",
+ args: NPX_REGISTRATION_COMMAND.slice(1),
+ },
+ },
+ enabledServers: ["other"],
+ disabledServers: ["rea", "other"],
+ });
+ expect(
+ await configureClientConfiguration(client, {}, NPX_REGISTRATION_COMMAND),
+ ).toEqual({ status: "unchanged" });
+ expect(await readFile(client.configPath, "utf8")).toBe(configured);
+ expect(
+ (
+ await readClientRegistrationStatuses(home, undefined, {
+ environment: {},
+ })
+ ).find(({ client: name }) => name === "pi"),
+ ).toMatchObject({ state: "aligned" });
+ expect((await systemUninstallHost(home).removeClient(client)).status).toBe(
+ "removed",
+ );
+ expect(JSON.parse(await readFile(client.configPath, "utf8"))).toEqual(
+ JSON.parse(original),
+ );
+ expect(await readFile(`${client.configPath}.rea.backup`, "utf8")).toBe(
+ original,
+ );
+ });
+
+ it.each([
+ [undefined, true, "aligned"],
+ ["stdio", true, "aligned"],
+ ["http", true, "invalid"],
+ ["streamable-http", true, "invalid"],
+ ["sse", true, "invalid"],
+ [undefined, false, "stale"],
+ ["stdio", false, "stale"],
+ ])(
+ "doctor handles type %s and enabled %s without OMP allowlist overrides",
+ async (type, enabled, state) => {
+ const { home, client } = await piClient();
+ await writeFile(
+ client.configPath,
+ JSON.stringify({
+ mcpServers: {
+ rea: {
+ command: "npx",
+ args: NPX_REGISTRATION_COMMAND.slice(1),
+ type,
+ enabled,
+ },
+ },
+ enabledServers: ["rea"],
+ }),
+ );
+ const statuses = await readClientRegistrationStatuses(home, undefined, {
+ environment: {},
+ });
+ expect(statuses.find(({ client: name }) => name === "pi")).toMatchObject({
+ state,
+ });
+ },
+ );
+});
+
+describe("Pi strict JSON parsing", () => {
+ it.each([
+ [
+ "a comment",
+ `{ // Pi loads strict JSON only\n"mcpServers": { "other": { "command": "other" } }\n}\n`,
+ ],
+ [
+ "a trailing comma",
+ `{"mcpServers": { "other": { "command": "other" }, },}\n`,
+ ],
+ [
+ "a BOM before valid JSON",
+ `\uFEFF{"mcpServers": { "other": { "command": "other" } }}\n`,
+ ],
+ ["empty text", ""],
+ ["whitespace only", "\n \t\r\n"],
+ ["a BOM only", "\uFEFF"],
+ ])(
+ "rejects an existing Pi configuration with %s before any mutation",
+ async (_description, original) => {
+ const { home, client } = await piClient();
+ await writeFile(client.configPath, original);
+ expect(() => parseClientConfiguration(original, "pi")).toThrow();
+ expect(
+ await inspectClientConfiguration(client, {}, NPX_REGISTRATION_COMMAND),
+ ).toMatchObject({ status: "invalid" });
+ expect(
+ await configureClientConfiguration(
+ client,
+ {},
+ NPX_REGISTRATION_COMMAND,
+ ),
+ ).toEqual({ status: "failed", reason: "readback" });
+ expect(await readFile(client.configPath, "utf8")).toBe(original);
+ await expect(
+ readFile(`${client.configPath}.rea.backup`),
+ ).rejects.toMatchObject({ code: "ENOENT" });
+ expect(
+ (
+ await readClientRegistrationStatuses(home, undefined, {
+ environment: {},
+ })
+ ).find(({ client: name }) => name === "pi"),
+ ).toMatchObject({ state: "invalid" });
+ expect(
+ await systemUninstallHost(home).removeClient(client),
+ ).toMatchObject({
+ name: "pi",
+ status: "failed",
+ detail: expect.stringContaining("not valid JSON"),
+ });
+ expect(await readFile(client.configPath, "utf8")).toBe(original);
+ await expect(
+ readFile(`${client.configPath}.rea.backup`),
+ ).rejects.toMatchObject({ code: "ENOENT" });
+ },
+ );
+
+ it("initializes a missing Pi configuration as valid strict JSON without a backup", async () => {
+ const { home, client } = await piClient();
+ await expect(readFile(client.configPath)).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ expect(
+ await inspectClientConfiguration(client, {}, NPX_REGISTRATION_COMMAND),
+ ).toEqual({ status: "create" });
+ expect(
+ await configureClientConfiguration(client, {}, NPX_REGISTRATION_COMMAND),
+ ).toMatchObject({ status: "configured" });
+ const created = await readFile(client.configPath, "utf8");
+ expect(() => JSON.parse(created)).not.toThrow();
+ expect(JSON.parse(created)).toEqual({
+ mcpServers: {
+ rea: {
+ type: "stdio",
+ command: "npx",
+ args: NPX_REGISTRATION_COMMAND.slice(1),
+ },
+ },
+ });
+ await expect(
+ readFile(`${client.configPath}.rea.backup`),
+ ).rejects.toMatchObject({ code: "ENOENT" });
+ expect(
+ await configureClientConfiguration(client, {}, NPX_REGISTRATION_COMMAND),
+ ).toEqual({ status: "unchanged" });
+ expect(await readFile(client.configPath, "utf8")).toBe(created);
+ expect(
+ (
+ await readClientRegistrationStatuses(home, undefined, {
+ environment: {},
+ })
+ ).find(({ client: name }) => name === "pi"),
+ ).toMatchObject({ state: "aligned" });
+ expect(await systemUninstallHost(home).removeClient(client)).toMatchObject({
+ name: "pi",
+ status: "removed",
+ });
+ expect(JSON.parse(await readFile(client.configPath, "utf8"))).toEqual({
+ mcpServers: {},
+ });
+ });
+
+ it("keeps JSONC tolerance for other dialects while Pi rejects the same text", () => {
+ const jsonc = `\uFEFF{\n // A comment Pi rejects but OpenCode keeps.\n "mcp": { "other": { "type": "local" }, },\n}\n`;
+ expect(parseClientConfiguration(jsonc, "opencode").servers).toEqual({
+ other: { type: "local" },
+ });
+ expect(parseClientConfiguration("", "opencode").document).toEqual({});
+ expect(parseClientConfiguration("\uFEFF", "opencode").document).toEqual({});
+ expect(() => parseClientConfiguration(jsonc, "pi")).toThrow();
+ expect(() => parseClientConfiguration("", "pi")).toThrow();
+ expect(() => parseClientConfiguration("\uFEFF", "pi")).toThrow();
+ });
+});
diff --git a/tests/boundary/filesystem/qwenClientConfiguration.test.ts b/tests/boundary/filesystem/qwenClientConfiguration.test.ts
new file mode 100644
index 0000000000..0fae29640f
--- /dev/null
+++ b/tests/boundary/filesystem/qwenClientConfiguration.test.ts
@@ -0,0 +1,183 @@
+import { access, mkdir, readFile, writeFile } from "node:fs/promises";
+import { join, resolve } from "node:path";
+
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+import { parse as parseJsonc } from "jsonc-parser";
+
+import { readClientRegistrationStatuses } from "../../../src/application/ClientRegistrationStatus.js";
+import { configureClientConfiguration } from "../../../src/application/SetupClientConfiguration.js";
+import { systemSetupHost } from "../../../src/application/SetupHost.js";
+import { supportedClients } from "../../../src/application/SupportedClients.js";
+import { systemUninstallHost } from "../../../src/application/Uninstall.js";
+import { PRODUCT_IDENTITY } from "../../../src/identity.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import {
+ clearClientLocationEnvironment,
+ NPX_REGISTRATION_COMMAND,
+} from "../../fixtures/clientEnvironment.js";
+
+const command = NPX_REGISTRATION_COMMAND;
+
+beforeEach(clearClientLocationEnvironment);
+afterEach(() => vi.unstubAllEnvs());
+
+const qwenPath = (
+ home: string,
+ platform: NodeJS.Platform,
+ env: Parameters[2] = {},
+) =>
+ supportedClients(home, platform, env).find(
+ ({ name }) => name === "qwen_code",
+ );
+
+describe("Qwen Code configuration paths", () => {
+ it("resolves the user settings file and marker directory", () => {
+ expect(qwenPath("/home/a", "linux")).toEqual({
+ name: "qwen_code",
+ displayName: "Qwen Code",
+ configPath: join("/home/a", ".qwen", "settings.json"),
+ markerPath: join("/home/a", ".qwen"),
+ format: "json",
+ });
+ expect(qwenPath("/Users/a", "darwin")?.configPath).toBe(
+ join("/Users/a", ".qwen", "settings.json"),
+ );
+ });
+
+ it.each([
+ ["", join("/home/a", ".qwen")],
+ ["~", "/home/a"],
+ ["~/profiles/work", join("/home/a", "profiles", "work")],
+ ["~\\profiles\\work", join("/home/a", "profiles", "work")],
+ ["profiles/work", resolve("profiles/work")],
+ [" profile ", resolve(" profile ")],
+ ["$PROFILE", resolve("$PROFILE")],
+ ])("matches Qwen's QWEN_HOME resolution for %j", (override, directory) => {
+ vi.stubEnv("QWEN_HOME", override);
+ const client = qwenPath("/home/a", process.platform, {
+ QWEN_HOME: override,
+ });
+ expect(
+ supportedClients("/home/a").find(({ name }) => name === "qwen_code"),
+ ).toEqual(client);
+ expect(client).toMatchObject({
+ configPath: join(directory, "settings.json"),
+ markerPath: directory,
+ });
+ });
+});
+
+describe("Qwen Code client lifecycle", () => {
+ const qwenClient = async () => {
+ const home = await createTestTempDirectory("rea-qwen-");
+ const client = qwenPath(home, process.platform, {});
+ if (client?.markerPath === undefined)
+ throw new Error("missing Qwen Code client");
+ await mkdir(client.markerPath, { recursive: true });
+ return { home, client };
+ };
+
+ it("registers rea in mcpServers while preserving unrelated settings and comments", async () => {
+ const { home, client } = await qwenClient();
+ await writeFile(
+ client.configPath,
+ `{
+ // Keep this note.
+ "model": { "name": "qwen3-coder-plus" },
+ "mcpServers": { "other": { "command": "other-server" } }
+}
+`,
+ );
+ expect(
+ (await configureClientConfiguration(client, {}, command)).status,
+ ).toBe("configured");
+ const text = await readFile(client.configPath, "utf8");
+ expect(text).toContain("// Keep this note.");
+ expect(parseJsonc(text)).toEqual({
+ model: { name: "qwen3-coder-plus" },
+ mcpServers: {
+ other: { command: "other-server" },
+ rea: { command: "npx", args: command.slice(1) },
+ },
+ });
+ expect(await configureClientConfiguration(client, {}, command)).toEqual({
+ status: "unchanged",
+ });
+ expect((await systemUninstallHost(home).removeClient(client)).status).toBe(
+ "removed",
+ );
+ const removedText = await readFile(client.configPath, "utf8");
+ const removed = parseJsonc(removedText);
+ expect(removed).toEqual({
+ model: { name: "qwen3-coder-plus" },
+ mcpServers: { other: { command: "other-server" } },
+ });
+ expect(removedText).toContain("// Keep this note.");
+ });
+
+ it("reports a package-runner registration as aligned", async () => {
+ const { home, client } = await qwenClient();
+ expect(
+ (await configureClientConfiguration(client, {}, command)).status,
+ ).toBe("configured");
+ const entry = resolve("scripts/rea.mjs");
+ const status = (
+ await readClientRegistrationStatuses(home, entry, { environment: {} })
+ ).find(({ client: name }) => name === "qwen_code");
+ expect(status).toMatchObject({ state: "aligned" });
+ });
+});
+
+describe("Qwen Code profile setup", () => {
+ it.each([false, true])(
+ "installs and removes the shared skill with a custom home: %s",
+ async (customHome) => {
+ const home = await createTestTempDirectory("rea-qwen-profile-");
+ const profile = join(home, customHome ? "qwen-profile" : ".qwen");
+ const environment = {
+ HOME: home,
+ USERPROFILE: home,
+ ...(customHome ? { QWEN_HOME: profile } : {}),
+ };
+ await mkdir(profile);
+ const setup = systemSetupHost(undefined, environment);
+ const client = (await setup.detectedClients()).find(
+ ({ name }) => name === "qwen_code",
+ );
+ if (client === undefined) throw new Error("missing Qwen Code client");
+ expect(client.configPath).toBe(join(profile, "settings.json"));
+ expect(await setup.configureClient(client, {}, command)).toEqual({
+ status: "configured",
+ });
+ expect(await setup.installSkill([client.name])).toBe("installed");
+ const skill = join(home, ".agents", "skills", PRODUCT_IDENTITY.skillName);
+ expect(await readFile(join(skill, "SKILL.md"), "utf8")).toContain(
+ "reverse-engineer-anything",
+ );
+ expect(await setup.clientNeedsConfigure(client, {}, command)).toBe(false);
+ expect(await setup.skillNeedsInstall([client.name])).toBe(false);
+ expect(await setup.installSkill([client.name])).toBe("unchanged");
+ expect(
+ (
+ await readClientRegistrationStatuses(
+ home,
+ resolve("scripts/rea.mjs"),
+ { environment },
+ )
+ ).find(({ client: name }) => name === "qwen_code"),
+ ).toMatchObject({ state: "aligned" });
+ const uninstall = systemUninstallHost(home, undefined, environment);
+ const registered = (await uninstall.clients()).find(
+ ({ name }) => name === "qwen_code",
+ );
+ expect(registered).toEqual(client);
+ expect((await uninstall.removeClient(client)).status).toBe("removed");
+ expect((await uninstall.removeSkill()).status).toBe("removed");
+ await expect(access(skill)).rejects.toMatchObject({ code: "ENOENT" });
+ if (customHome)
+ await expect(access(join(home, ".qwen"))).rejects.toMatchObject({
+ code: "ENOENT",
+ });
+ },
+ );
+});
diff --git a/tests/boundary/filesystem/setupSkill.test.ts b/tests/boundary/filesystem/setupSkill.test.ts
index 0d1caa93a0..40bbcfd825 100644
--- a/tests/boundary/filesystem/setupSkill.test.ts
+++ b/tests/boundary/filesystem/setupSkill.test.ts
@@ -13,7 +13,6 @@ import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import {
canonicalSkillNeedsInstall,
- claudeCodeSkillsDirectory,
installCanonicalSkill,
} from "../../../src/application/SetupSkill.js";
import {
@@ -56,9 +55,7 @@ describe("canonical skill transaction", () => {
it("installs only the selected Claude Code personal skill", async () => {
const home = await createTestTempDirectory("rea-claude-skill-");
const configDirectory = join(home, "claude-config");
- const skillsDirectory = claudeCodeSkillsDirectory(home, {
- CLAUDE_CONFIG_DIR: configDirectory,
- });
+ const skillsDirectory = join(configDirectory, "skills");
const claudeSkill = join(
skillsDirectory,
"reverse-engineer-anything/SKILL.md",
@@ -84,13 +81,19 @@ describe("canonical skill transaction", () => {
});
expect(
- await canonicalSkillNeedsInstall(home, ["claude_code"], skillsDirectory),
+ await canonicalSkillNeedsInstall(home, ["claude_code"], {
+ CLAUDE_CONFIG_DIR: dirname(skillsDirectory),
+ }),
).toBe(true);
expect(
- await installCanonicalSkill(home, ["claude_code"], skillsDirectory),
+ await installCanonicalSkill(home, ["claude_code"], {
+ CLAUDE_CONFIG_DIR: dirname(skillsDirectory),
+ }),
).toBe("installed");
expect(
- await canonicalSkillNeedsInstall(home, ["claude_code"], skillsDirectory),
+ await canonicalSkillNeedsInstall(home, ["claude_code"], {
+ CLAUDE_CONFIG_DIR: dirname(skillsDirectory),
+ }),
).toBe(false);
await expect(access(claudeSkill)).resolves.toBeUndefined();
await expect(
@@ -112,7 +115,7 @@ describe("canonical skill transaction", () => {
home,
".agents/skills/reverse-engineer-anything/SKILL.md",
);
- const claudeSkills = claudeCodeSkillsDirectory(home);
+ const claudeSkills = join(home, ".claude", "skills");
const claudeSkill = join(
claudeSkills,
"reverse-engineer-anything/SKILL.md",
diff --git a/tests/boundary/mcp/analysisViewMcp.test.ts b/tests/boundary/mcp/analysisViewMcp.test.ts
index 3edeffba78..3c15312aa6 100644
--- a/tests/boundary/mcp/analysisViewMcp.test.ts
+++ b/tests/boundary/mcp/analysisViewMcp.test.ts
@@ -6,7 +6,9 @@ import { z } from "zod";
import { BinaryLayoutService } from "../../../src/application/binaryDiagnostics/BinaryLayoutService.js";
import { toolContract } from "../../../src/contracts/toolContracts.js";
-import { parseEvidence } from "../../../src/domain/evidence.js";
+import { createEvidence, parseEvidence } from "../../../src/domain/evidence.js";
+import { functionDossierSchema } from "../../../src/domain/hopperValues.js";
+import { ghidraFunctionDossier } from "../../../src/domain/ghidraValues.fixture.js";
import { ok } from "../../../src/domain/result.js";
import {
createJavaScriptApplicationGraph,
@@ -109,6 +111,68 @@ it("advertises exact schemas and projects one layout section from retained Evide
);
});
+it("delivers a bounded native dossier view over MCP without a Ghidra provider or lost connection", async () => {
+ const { client, session } = await connect();
+ const original = functionDossierSchema.parse(ghidraFunctionDossier());
+ const parent = createEvidence(
+ {
+ path: "/fixtures/oversized-native.exe",
+ format: "pe",
+ sha256: "b".repeat(64),
+ },
+ { id: "ghidra", name: "Ghidra", version: "12.1.4" },
+ {
+ operation: "analyze_function",
+ parameters: { address: "0x401000" },
+ result: { ...original, pseudocode: "X".repeat(11 * 1024 * 1024) },
+ },
+ );
+ expect(session.recordEvidence(parent).ok).toBe(true);
+ const advertised = (await client.listTools()).tools.find(
+ (tool) => tool.name === "inspect_analysis_view",
+ );
+ if (advertised?.outputSchema === undefined)
+ throw new Error("missing view schemas");
+ const ajv = new Ajv2020({ strict: false, validateFormats: false });
+ const inputSchema: Record = advertised.inputSchema;
+ const outputSchema: Record = advertised.outputSchema;
+ const arguments_ = {
+ source: { kind: "retained-evidence", evidence_id: parent.evidence_id },
+ view: { kind: "native", facet: "pseudocode", offset: 0, limit: 500 },
+ };
+ expect(ajv.validateSchema(inputSchema)).toBe(true);
+ expect(ajv.validateSchema(outputSchema)).toBe(true);
+ expect(
+ ajv.validate(inputSchema, arguments_),
+ JSON.stringify(ajv.errors),
+ ).toBe(true);
+ const response = await client.callTool({
+ name: "inspect_analysis_view",
+ arguments: arguments_,
+ });
+ expect(response.isError).not.toBe(true);
+ expect(
+ ajv.validate(outputSchema, response.structuredContent),
+ JSON.stringify(ajv.errors),
+ ).toBe(true);
+ const parsed = toolContract("inspect_analysis_view").outputSchema.parse(
+ response.structuredContent,
+ );
+ expect(parsed.normalized_result).toMatchObject({
+ kind: "native",
+ parent_evidence_id: parent.evidence_id,
+ procedure_address: "0x401000",
+ coverage: { examined: 500, next_offset: 500, total: 11 * 1024 * 1024 },
+ });
+ expect(parsed.evidence_links).toEqual([parent.evidence_id]);
+ expect(parsed.locations).toEqual([
+ { kind: "artifact-path", path: "/fixtures/oversized-native.exe" },
+ { kind: "address", address: "0x401000" },
+ ]);
+ expect(JSON.stringify(response.structuredContent).length).toBeLessThan(12000);
+ await client.ping();
+});
+
it("retains an oversized selected observation and keeps subsequent views usable", async () => {
const { client, session } = await connect();
const analysis = analysisViewJavaScriptAnalysisWithSource();
@@ -185,6 +249,71 @@ it("retains an oversized selected observation and keeps subsequent views usable"
expect(summary.isError).not.toBe(true);
});
+it("retains an oversized native assembly row and permits a subsequent smaller view", async () => {
+ const { client, session } = await connect();
+ const row = "\0".repeat(1_000_000);
+ const parent = createEvidence(
+ { path: "/fixtures/large-row.exe", format: "pe", sha256: "c".repeat(64) },
+ { id: "ghidra", name: "Ghidra", version: "12.1.4" },
+ {
+ operation: "analyze_function",
+ parameters: {},
+ result: {
+ ...functionDossierSchema.parse(ghidraFunctionDossier()),
+ assembly: [row],
+ },
+ },
+ );
+ expect(session.recordEvidence(parent).ok).toBe(true);
+ const source = { kind: "retained-evidence", evidence_id: parent.evidence_id };
+ const response = await client.callTool({
+ name: "inspect_analysis_view",
+ arguments: {
+ source,
+ view: { kind: "native", facet: "assembly", offset: 0, limit: 1 },
+ },
+ });
+ expect(response.isError).toBe(true);
+ const error = z
+ .object({
+ error: z.object({
+ details: z.object({
+ reported_limits: z.object({
+ evidence_reference: z.object({
+ kind: z.literal("retained-evidence"),
+ evidence_id: z.string(),
+ }),
+ }),
+ }),
+ }),
+ })
+ .parse(parseMcpToolError(response));
+ const retained = session.evidenceById(
+ error.error.details.reported_limits.evidence_reference.evidence_id,
+ );
+ expect(retained?.normalized_result).toMatchObject({
+ item: [row],
+ parent_evidence_id: parent.evidence_id,
+ });
+ const next = await client.callTool({
+ name: "inspect_analysis_view",
+ arguments: {
+ source,
+ view: { kind: "native", facet: "pseudocode", offset: 0, limit: 12 },
+ },
+ });
+ expect(next.isError).not.toBe(true);
+ expect(
+ toolContract("inspect_analysis_view").outputSchema.parse(
+ next.structuredContent,
+ ).normalized_result,
+ ).toMatchObject({
+ item: { text: "int fixture_" },
+ parent_evidence_id: parent.evidence_id,
+ });
+ await client.ping();
+});
+
it("returns a JavaScript summary without graph payloads from a retained reference", async () => {
const { client, session } = await connect();
const parent = analysisViewJavaScriptEvidence();
diff --git a/tests/boundary/mcp/compactInputSchemaMcp.test.ts b/tests/boundary/mcp/compactInputSchemaMcp.test.ts
new file mode 100644
index 0000000000..536b7afeb5
--- /dev/null
+++ b/tests/boundary/mcp/compactInputSchemaMcp.test.ts
@@ -0,0 +1,281 @@
+import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
+import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from "@modelcontextprotocol/server";
+import { Ajv2020 } from "ajv/dist/2020.js";
+import { describe, expect, it } from "vitest";
+
+import { TOOL_CONTRACTS } from "../../../src/contracts/toolContracts.js";
+import { compactAdvertisedInputSchema } from "../../../src/contracts/compactInputSchema.js";
+import {
+ COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
+ parseMcpInputSchemaProfile,
+} from "../../../src/config/mcpInputSchemaProfile.js";
+import { processScenarioSchema } from "../../../src/domain/process/processScenario.js";
+import { EvidenceMcpServer } from "../../../src/server/EvidenceMcpServer.js";
+import { ToolResultDelivery } from "../../../src/server/toolResult.js";
+import { toolRegistrationOptions } from "../../../src/server/toolRegistrationOptions.js";
+
+interface AdvertisedTool {
+ readonly name: string;
+ readonly inputSchema: Record;
+ readonly outputSchema?: Record | undefined;
+ readonly description?: string | undefined;
+}
+
+const serializedBytes = (value: unknown): number =>
+ Buffer.byteLength(JSON.stringify(value), "utf8");
+
+const listToolsThroughTransport = async (
+ server: EvidenceMcpServer,
+): Promise => {
+ const client = new Client({ name: "compact-profile", version: "0" });
+ const [clientTransport, serverTransport] =
+ InMemoryTransport.createLinkedPair();
+ try {
+ await server.connect(serverTransport);
+ await client.connect(clientTransport);
+ return (await client.listTools()).tools;
+ } finally {
+ await Promise.allSettled([client.close(), server.close()]);
+ }
+};
+
+const registerCatalog = (server: EvidenceMcpServer): void => {
+ for (const contract of TOOL_CONTRACTS)
+ server.registerTool(
+ contract.name,
+ toolRegistrationOptions(contract),
+ async () => ({ content: [] }),
+ );
+};
+
+const compactServer = (): EvidenceMcpServer =>
+ new EvidenceMcpServer(
+ { name: "compact-profile", version: "0" },
+ { capabilities: {} },
+ undefined,
+ new ToolResultDelivery(STDIO_DEFAULT_MAX_BUFFER_SIZE),
+ { budgetBytes: COMPACT_INPUT_SCHEMA_BUDGET_BYTES },
+ );
+
+const fullServer = (): EvidenceMcpServer =>
+ new EvidenceMcpServer(
+ { name: "full-profile", version: "0" },
+ { capabilities: {} },
+ undefined,
+ new ToolResultDelivery(STDIO_DEFAULT_MAX_BUFFER_SIZE),
+ );
+
+describe("MCP compact input schema profile", () => {
+ it("advertises every tool input schema within the provider budget", async () => {
+ const server = compactServer();
+ registerCatalog(server);
+ const tools = await listToolsThroughTransport(server);
+ expect(tools.map(({ name }) => name).sort()).toEqual(
+ TOOL_CONTRACTS.map(({ name }) => name).sort(),
+ );
+ const oversized = tools.filter(
+ ({ inputSchema }) =>
+ serializedBytes(inputSchema) > COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
+ );
+ expect(oversized.map(({ name }) => name)).toEqual([]);
+ });
+
+ it("keeps output schemas and tool descriptions identical to the default profile", async () => {
+ const compact = compactServer();
+ const full = fullServer();
+ registerCatalog(compact);
+ registerCatalog(full);
+ const [compactTools, fullTools] = await Promise.all([
+ listToolsThroughTransport(compact),
+ listToolsThroughTransport(full),
+ ]);
+ const fullByName = new Map(fullTools.map((tool) => [tool.name, tool]));
+ expect(compactTools.length).toBe(fullTools.length);
+ for (const tool of compactTools) {
+ const reference = fullByName.get(tool.name);
+ if (reference === undefined) throw new Error(`Missing ${tool.name}`);
+ expect(tool.outputSchema, tool.name).toEqual(reference.outputSchema);
+ expect(tool.description, tool.name).toBe(reference.description);
+ }
+ });
+
+ it("keeps compact schemas valid and accepting canonical examples", async () => {
+ const server = compactServer();
+ registerCatalog(server);
+ const tools = await listToolsThroughTransport(server);
+ const ajv = new Ajv2020({ strict: false, validateFormats: false });
+ const contractsByName = new Map(
+ TOOL_CONTRACTS.map((contract) => [contract.name, contract]),
+ );
+ for (const tool of tools) {
+ expect(
+ ajv.validateSchema(tool.inputSchema),
+ `${tool.name}: ${ajv.errorsText(ajv.errors)}`,
+ ).toBe(true);
+ const contract = contractsByName.get(tool.name);
+ const validate = ajv.compile(tool.inputSchema);
+ for (const example of contract?.examples ?? [])
+ expect(validate(example.input), `${tool.name}: ${example.title}`).toBe(
+ true,
+ );
+ }
+ });
+
+ it("reduces at most the structurally oversized tools", async () => {
+ const full = fullServer();
+ registerCatalog(full);
+ const tools = await listToolsThroughTransport(full);
+ const reduced: string[] = [];
+ for (const tool of tools) {
+ const { presentation } = compactAdvertisedInputSchema(
+ tool.inputSchema,
+ COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
+ );
+ if (presentation === "reduced") reduced.push(tool.name);
+ }
+ expect(reduced.sort()).toEqual([
+ "build_reconstruction_obligation_ledger",
+ "capture_browser_scenario",
+ "compare_application_versions",
+ "compare_web_captures",
+ "evaluate_reconstruction_coverage",
+ "project_managed_application_graph",
+ ]);
+ expect(tools.length).toBe(TOOL_CONTRACTS.length);
+ });
+
+ it("retains parameter types and avoids client-side reference expansion", async () => {
+ const server = compactServer();
+ registerCatalog(server);
+ const tools = await listToolsThroughTransport(server);
+ for (const tool of tools) {
+ // Kimi Code 2.1.1 expands references and defaults description-only
+ // properties to string. Inline schemas with explicit shallow types
+ // avoid both conversions for the structurally oversized tools.
+ expect(JSON.stringify(tool.inputSchema), tool.name).not.toContain(
+ '"$ref":',
+ );
+ }
+ const comparison = tools.find(
+ ({ name }) => name === "compare_managed_members",
+ );
+ expect(comparison?.inputSchema.properties).toMatchObject({
+ left: {
+ properties: {
+ normalized_result: {
+ type: ["null", "boolean", "object", "array", "number", "string"],
+ },
+ },
+ },
+ });
+ const ledger = tools.find(
+ ({ name }) => name === "build_reconstruction_obligation_ledger",
+ );
+ expect(ledger?.inputSchema.properties).toMatchObject({
+ evidence_bundle: { type: "object" },
+ reviewed_obligations: { type: "array" },
+ manifest: { type: "object" },
+ });
+ const coverage = tools.find(
+ ({ name }) => name === "evaluate_reconstruction_coverage",
+ );
+ expect(coverage?.inputSchema.properties).toMatchObject({
+ coverage: { type: "object" },
+ boundary_id: { type: "string" },
+ });
+ });
+
+ it("parses the profile selection the server startup reads", () => {
+ expect(parseMcpInputSchemaProfile("compact")).toEqual({
+ ok: true,
+ value: "compact",
+ });
+ expect(parseMcpInputSchemaProfile("surprise")).not.toHaveProperty("value");
+ });
+});
+
+describe("MCP compact input schema canonical enforcement", () => {
+ it("still rejects canonical-invalid arguments the compact advertisement accepts", async () => {
+ const server = compactServer();
+ const contract = TOOL_CONTRACTS.find(
+ ({ name }) => name === "capture_process_scenario",
+ );
+ if (contract === undefined)
+ throw new Error("Process capture contract was not registered");
+ let handlerCalled = false;
+ server.registerTool(
+ contract.name,
+ toolRegistrationOptions(contract),
+ async () => {
+ handlerCalled = true;
+ return {
+ content: [{ type: "text" as const, text: "handler ran" }],
+ isError: true,
+ };
+ },
+ );
+ const client = new Client({ name: "compact-validation", version: "0" });
+ const [clientTransport, serverTransport] =
+ InMemoryTransport.createLinkedPair();
+ try {
+ await server.connect(serverTransport);
+ await client.connect(clientTransport);
+ const advertised = (await client.listTools()).tools.find(
+ ({ name }) => name === contract.name,
+ );
+ if (advertised === undefined)
+ throw new Error("Process capture tool was not advertised");
+ const ajv = new Ajv2020({ strict: false, validateFormats: false });
+ const validate = ajv.compile(advertised.inputSchema);
+ const valid = { executable: "node", environment: { APP_MODE: "test" } };
+ const reserved = {
+ executable: "node",
+ environment: { REA_PROCESS_RUN_ID: "caller-value" },
+ };
+ expect(validate(valid)).toBe(true);
+ expect(processScenarioSchema.safeParse(valid).success).toBe(true);
+ expect(processScenarioSchema.safeParse(reserved).success).toBe(false);
+ const result = await client.callTool({
+ name: contract.name,
+ arguments: reserved,
+ });
+ expect(result.isError).toBe(true);
+ expect(handlerCalled).toBe(false);
+ } finally {
+ await Promise.allSettled([client.close(), server.close()]);
+ }
+ });
+
+ it("compacts plain standard schemas that are not Zod instances", async () => {
+ // Mirrors registerWebNetworkCaptureTool: a plain standard schema reusing
+ // the advertised projection while owning its own permissive validator.
+ const server = compactServer();
+ const contract = TOOL_CONTRACTS.find(
+ ({ name }) => name === "inspect_web_network_capture",
+ );
+ if (contract === undefined)
+ throw new Error("Network capture contract was not registered");
+ const registration = toolRegistrationOptions(contract);
+ const inputSchema = {
+ "~standard": {
+ ...registration.inputSchema["~standard"],
+ vendor: "rea-application",
+ validate: (value: unknown) => ({ value }),
+ },
+ };
+ server.registerTool(
+ contract.name,
+ { ...registration, inputSchema },
+ async () => ({ content: [] }),
+ );
+ const tools = await listToolsThroughTransport(server);
+ const advertised = tools.find(({ name }) => name === contract.name);
+ if (advertised === undefined)
+ throw new Error("Network capture tool was not advertised");
+ expect(serializedBytes(advertised.inputSchema)).toBeLessThanOrEqual(
+ COMPACT_INPUT_SCHEMA_BUDGET_BYTES,
+ );
+ const ajv = new Ajv2020({ strict: false, validateFormats: false });
+ expect(ajv.validateSchema(advertised.inputSchema)).toBe(true);
+ });
+});
diff --git a/tests/boundary/mcp/javascriptLiteralPayload.test.ts b/tests/boundary/mcp/javascriptLiteralPayload.test.ts
new file mode 100644
index 0000000000..b7fcbb8764
--- /dev/null
+++ b/tests/boundary/mcp/javascriptLiteralPayload.test.ts
@@ -0,0 +1,154 @@
+import { Client } from "@modelcontextprotocol/client";
+import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
+import { createHash } from "node:crypto";
+import { writeFile } from "node:fs/promises";
+import { join, resolve } from "node:path";
+import { expect } from "vitest";
+
+import { toolContract } from "../../../src/contracts/toolContracts.js";
+import { javascriptApplicationAnalysisResultSchema } from "../../../src/domain/javascript/javascriptApplicationAnalysis.js";
+import { javaScriptSemanticTraceResultSchema } from "../../../src/domain/javascript/javascriptSemanticTraceSchemas.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { cliTest } from "../../support/cli/cliFixture.js";
+
+cliTest(
+ "delivers complete large literals through CLI and default stdio MCP without metadata copies",
+ async ({ cli }) => {
+ const root = await createTestTempDirectory("rea-literal-payload-");
+ const value = 'rea_literal_payload_ğ\\"\0'.padEnd(24, "x").repeat(65_536);
+ const source = [
+ `export const payload = ${JSON.stringify(value)};`,
+ 'export const choice = flag ? 1 : "1";',
+ 'export const empty = "";',
+ ].join("\n");
+ await writeFile(join(root, "main.js"), source);
+ const cliResponse = await cli.run({
+ arguments: [
+ "analyze-javascript-application",
+ root,
+ "--artifact-format",
+ "directory",
+ "--json",
+ ],
+ timeoutMs: 60_000,
+ });
+ expect(cliResponse.exitCode).toBe(0);
+ const cliEvidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(cliResponse.json);
+
+ const client = new Client({ name: "literal-payload-e2e", version: "1" });
+ const transport = new StdioClientTransport({
+ command: process.execPath,
+ args: [resolve("scripts/rea.mjs"), "mcp"],
+ cwd: process.cwd(),
+ env: { PATH: process.env.PATH ?? "" },
+ stderr: "pipe",
+ });
+ transport.stderr?.on("data", () => undefined);
+ try {
+ await client.connect(transport);
+ const response = await client.callTool({
+ name: "analyze_javascript_application",
+ arguments: { input_path: root, format: "directory" },
+ });
+ // The old payload-bearing role key/label exceed the default wire budget.
+ expect(response.isError).not.toBe(true);
+ const evidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(response.structuredContent);
+ expect(evidence.evidence_id).toBe(cliEvidence.evidence_id);
+ const text = response.content.find((item) => item.type === "text");
+ if (text?.type !== "text") throw new Error("Missing MCP text result");
+ const textEvidence = toolContract(
+ "analyze_javascript_application",
+ ).outputSchema.parse(JSON.parse(text.text));
+ expect(textEvidence.evidence_id).toBe(evidence.evidence_id);
+ const graph = javascriptApplicationAnalysisResultSchema.parse(
+ evidence.normalized_result,
+ ).semantic_graph;
+ const literal = graph.nodes.find(
+ (node) => node.kind === "literal" && node.properties.value === value,
+ );
+ if (literal === undefined) throw new Error("Complete literal was lost");
+ const sourceSha256 = createHash("sha256").update(source).digest("hex");
+ expect(literal.identity).toMatchObject({
+ module_path: "main.js",
+ artifact_sha256: sourceSha256,
+ });
+ expect(literal.evidence.location).toMatchObject({
+ available: true,
+ value: { kind: "source-range", source: "main.js" },
+ });
+ const choices = graph.nodes.filter(
+ (node) =>
+ node.kind === "literal" &&
+ (node.properties.value === 1 || node.properties.value === "1"),
+ );
+ expect(choices.length).toBe(2);
+ expect(new Set(choices.map((node) => node.node_id)).size).toBe(2);
+ expect(
+ graph.nodes.some(
+ (node) => node.kind === "literal" && node.properties.value === "",
+ ),
+ ).toBe(true);
+
+ const traceResponse = await client.callTool({
+ name: "trace_javascript_semantics",
+ arguments: {
+ application: {
+ kind: "retained-evidence",
+ evidence_id: evidence.evidence_id,
+ },
+ query: {
+ seed: { kind: "semantic-node", node_id: literal.node_id },
+ direction: "backward-provenance",
+ },
+ },
+ });
+ expect(traceResponse.isError).not.toBe(true);
+ const traceEvidence = toolContract(
+ "trace_javascript_semantics",
+ ).outputSchema.parse(traceResponse.structuredContent);
+ const trace = javaScriptSemanticTraceResultSchema.parse(
+ traceEvidence.normalized_result,
+ );
+ expect(
+ trace.nodes.some(
+ (node) =>
+ node.node_id === literal.node_id && node.properties.value === value,
+ ),
+ ).toBe(true);
+ const valueTraceResponse = await client.callTool({
+ name: "trace_javascript_semantics",
+ arguments: {
+ application: {
+ kind: "retained-evidence",
+ evidence_id: evidence.evidence_id,
+ },
+ query: {
+ seed: { kind: "literal", value: "1" },
+ direction: "backward-provenance",
+ },
+ },
+ });
+ expect(valueTraceResponse.isError).not.toBe(true);
+ const valueTraceEvidence = toolContract(
+ "trace_javascript_semantics",
+ ).outputSchema.parse(valueTraceResponse.structuredContent);
+ const valueTrace = javaScriptSemanticTraceResultSchema.parse(
+ valueTraceEvidence.normalized_result,
+ );
+ expect(valueTrace.seed_node_ids).toEqual(
+ choices
+ .filter((node) => node.properties.value === "1")
+ .map((node) => node.node_id),
+ );
+ await client.ping();
+ } finally {
+ await client.close();
+ await transport.close();
+ }
+ },
+ 60_000,
+);
diff --git a/tests/boundary/mcp/mcpDoctor.test.ts b/tests/boundary/mcp/mcpDoctor.test.ts
index 9858e4ff2e..e9b68b36f5 100644
--- a/tests/boundary/mcp/mcpDoctor.test.ts
+++ b/tests/boundary/mcp/mcpDoctor.test.ts
@@ -1,4 +1,6 @@
-import { readFile } from "node:fs/promises";
+import { Client } from "@modelcontextprotocol/client";
+import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
+import { readFile, writeFile } from "node:fs/promises";
import { join, resolve } from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
@@ -43,6 +45,79 @@ describe("production MCP doctor", () => {
);
}, 30_000);
+ it("rejects a schema-invalid probe while retaining the discovered inventory", async () => {
+ const root = await createTestTempDirectory("rea-mcp-doctor-reply-");
+ const snapshotPath = join(root, "reply.json");
+ const environment = { PATH: process.env.PATH ?? "" };
+ const client = new Client({ name: "doctor-reply-capture", version: "1" });
+ const transport = new StdioClientTransport({
+ command: process.execPath,
+ args: [resolve("scripts/rea.mjs"), "mcp"],
+ cwd: process.cwd(),
+ env: environment,
+ stderr: "pipe",
+ });
+ transport.stderr?.on("data", () => undefined);
+ try {
+ await client.connect(transport);
+ const tools = await client.listTools();
+ const prompts = await client.listPrompts();
+ const toolDefinition = tools.tools.find(
+ ({ name }) => name === "binary_session",
+ );
+ if (toolDefinition === undefined)
+ throw new Error("Production binary_session contract is missing");
+ const reply = await client.callTool(
+ { name: "binary_session", arguments: {} },
+ { toolDefinition },
+ );
+ await writeFile(
+ snapshotPath,
+ JSON.stringify({
+ serverInfo: client.getServerVersion(),
+ protocolVersion: client.getNegotiatedProtocolVersion(),
+ tools,
+ prompts,
+ reply,
+ }),
+ );
+ } finally {
+ await client.close();
+ await transport.close();
+ }
+ // Replay real production metadata through a synthetic stdio producer.
+ for (const mode of ["valid", "invalid"]) {
+ const result = await runProductionMcpDoctor({
+ command: process.execPath,
+ args: [
+ resolve("tests/fixtures/mcpDoctorReply.mjs"),
+ snapshotPath,
+ mode,
+ ],
+ cwd: process.cwd(),
+ environment,
+ });
+ expect(result.healthy).toBe(mode === "valid");
+ expect(result.inventory?.tools.observed).toBe(
+ CATALOG_IDENTITY.counts.mcp_tools,
+ );
+ expect(result.inventory?.prompts.observed).toBe(
+ CATALOG_IDENTITY.counts.mcp_prompts,
+ );
+ expect(result.checks).toContainEqual({
+ name: "initialize",
+ ok: true,
+ detail: expect.any(String),
+ });
+ if (mode === "invalid")
+ expect(result.checks).toContainEqual({
+ name: "request-flow",
+ ok: false,
+ detail: expect.stringMatching(/output schema/iu),
+ });
+ }
+ }, 30_000);
+
it("reports exact missing, unexpected, and duplicate inventory names", () => {
expect(compareInventory(["a", "b"], ["b", "c", "c"])).toEqual({
expected: 2,
diff --git a/tests/boundary/mcp/outputValidatorReuse.test.ts b/tests/boundary/mcp/outputValidatorReuse.test.ts
new file mode 100644
index 0000000000..256c29d588
--- /dev/null
+++ b/tests/boundary/mcp/outputValidatorReuse.test.ts
@@ -0,0 +1,108 @@
+import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
+import { McpServer } from "@modelcontextprotocol/server";
+import { expect, it } from "vitest";
+import { z } from "zod";
+
+import { toolContract } from "../../../src/contracts/toolContracts.js";
+import { toolOutputSchemaWithMetadata } from "../../../src/contracts/toolSchemaMetadata.js";
+
+const advertised = <
+ Schema extends z.ZodType>>,
+>(
+ schema: Schema,
+): Schema =>
+ toolOutputSchemaWithMetadata({
+ ...toolContract("binary_session"),
+ outputSchema: schema,
+ });
+
+const connect = async (server: McpServer, client: Client): Promise => {
+ const [clientTransport, serverTransport] =
+ InMemoryTransport.createLinkedPair();
+ await server.connect(serverTransport);
+ await client.connect(clientTransport);
+};
+
+it("validates changed schemas after catalog refreshes and availability changes", async () => {
+ const client = new Client({ name: "output-reuse-test", version: "1" });
+ const server = new McpServer({ name: "output-reuse-fixture", version: "1" });
+ const schema = advertised(z.strictObject({ state: z.literal("ready") }));
+ let state = "ready";
+ const selected = server.registerTool(
+ "selected",
+ { inputSchema: z.strictObject({}), outputSchema: schema },
+ () => ({ content: [], structuredContent: { state } }),
+ );
+ server.registerTool(
+ "equivalent",
+ { inputSchema: z.strictObject({}), outputSchema: schema },
+ () => ({ content: [], structuredContent: { state: "ready" } }),
+ );
+ try {
+ await connect(server, client);
+ await client.listTools();
+ expect(
+ (await client.callTool({ name: "selected" })).structuredContent,
+ ).toEqual({
+ state: "ready",
+ });
+ await client.listTools();
+ await client.callTool({ name: "equivalent" });
+
+ selected.update({
+ outputSchema: advertised(z.strictObject({ state: z.literal("updated") })),
+ });
+ state = "updated";
+ await client.listTools();
+ expect(
+ (await client.callTool({ name: "selected" })).structuredContent,
+ ).toEqual({
+ state: "updated",
+ });
+ selected.disable();
+ const unavailable = await client.listTools();
+ expect(unavailable.tools.some((tool) => tool.name === "selected")).toBe(
+ false,
+ );
+ await expect(client.callTool({ name: "selected" })).rejects.toThrow(
+ /not found|disabled/iu,
+ );
+ await client.callTool({ name: "equivalent" });
+ selected.enable();
+ await client.listTools();
+ await client.callTool({ name: "selected" });
+ } finally {
+ await Promise.allSettled([client.close(), server.close()]);
+ }
+});
+
+it("uses the new server's schema after reconnecting the client", async () => {
+ const client = new Client({ name: "output-reconnect-test", version: "1" });
+ const servers: McpServer[] = [];
+ try {
+ for (const state of ["first", "second"]) {
+ const server = new McpServer({ name: `fixture-${state}`, version: "1" });
+ servers.push(server);
+ server.registerTool(
+ "selected",
+ {
+ inputSchema: z.strictObject({}),
+ outputSchema: advertised(z.strictObject({ state: z.literal(state) })),
+ },
+ () => ({ content: [], structuredContent: { state } }),
+ );
+ await connect(server, client);
+ await client.listTools();
+ expect(
+ (await client.callTool({ name: "selected" })).structuredContent,
+ ).toEqual({ state });
+ await client.close();
+ await server.close();
+ }
+ } finally {
+ await Promise.allSettled([
+ client.close(),
+ ...servers.map((server) => server.close()),
+ ]);
+ }
+});
diff --git a/tests/boundary/mcp/oversizedErrorDeliveryMcp.test.ts b/tests/boundary/mcp/oversizedErrorDeliveryMcp.test.ts
index ba90eafdfe..72b6165621 100644
--- a/tests/boundary/mcp/oversizedErrorDeliveryMcp.test.ts
+++ b/tests/boundary/mcp/oversizedErrorDeliveryMcp.test.ts
@@ -43,12 +43,14 @@ const exerciseDeliveryFailure = async (
recordEvidence,
delivery,
);
- server.registerTool("large_failure", { inputSchema: {} }, async () =>
- oversizedFailure(),
+ server.registerTool(
+ "large_failure",
+ { inputSchema: z.object({}) },
+ async () => oversizedFailure(),
);
server.registerTool(
"healthy",
- { inputSchema: {} },
+ { inputSchema: z.object({}) },
async (): Promise => ({
content: [{ type: "text", text: "still connected" }],
}),
@@ -161,7 +163,7 @@ it("delivers an error whose text fits although structured and text copies would
},
delivery,
);
- server.registerTool("mid_failure", { inputSchema: {} }, async () =>
+ server.registerTool("mid_failure", { inputSchema: z.object({}) }, async () =>
failureWithName(nameBytes),
);
const client = new Client({ name: "mid-sized-error-client", version: "1" });
diff --git a/tests/boundary/mcp/toolSchemaValidity.test.ts b/tests/boundary/mcp/toolSchemaValidity.test.ts
index 1540964fd6..993a5e00f5 100644
--- a/tests/boundary/mcp/toolSchemaValidity.test.ts
+++ b/tests/boundary/mcp/toolSchemaValidity.test.ts
@@ -117,7 +117,7 @@ function expectKnownAuthorityHints(tools: readonly ToolSchemas[]): void {
expect(advertised.get(name), name).toMatchObject({ annotations });
}
-function expectRecursivePropertyDescriptions(
+function expectMeaningfulPropertyDescriptions(
schema: unknown,
path: string,
root: unknown = schema,
@@ -125,7 +125,7 @@ function expectRecursivePropertyDescriptions(
): void {
if (!isRecord(schema)) return;
if (typeof schema.$ref === "string" && !active.has(schema.$ref))
- expectRecursivePropertyDescriptions(
+ expectMeaningfulPropertyDescriptions(
resolveReference(root, schema.$ref),
`${path}.${schema.$ref}`,
root,
@@ -133,10 +133,13 @@ function expectRecursivePropertyDescriptions(
);
if (isRecord(schema.properties)) {
for (const [property, child] of Object.entries(schema.properties)) {
- expect(child, `${path}.${property}`).toMatchObject({
- description: expect.any(String),
- });
- expectRecursivePropertyDescriptions(
+ // Omit a description rather than restate the property name.
+ const description = isRecord(child) ? child.description : undefined;
+ if (description !== undefined)
+ expect(description, `${path}.${property}`).toEqual(
+ expect.not.stringMatching(/^(?:Value for .*)?$/u),
+ );
+ expectMeaningfulPropertyDescriptions(
child,
`${path}.${property}`,
root,
@@ -147,12 +150,12 @@ function expectRecursivePropertyDescriptions(
for (const key of ["items", "additionalProperties"])
if (schema[key] !== undefined)
- expectRecursivePropertyDescriptions(schema[key], path, root, active);
+ expectMeaningfulPropertyDescriptions(schema[key], path, root, active);
for (const key of ["allOf", "anyOf", "oneOf", "prefixItems"]) {
const children = schema[key];
if (Array.isArray(children))
children.forEach((child: unknown, index: number) =>
- expectRecursivePropertyDescriptions(
+ expectMeaningfulPropertyDescriptions(
child,
`${path}.${key}[${index}]`,
root,
@@ -358,7 +361,7 @@ describe("MCP JSON Schema validity", () => {
contract.inputSchema.safeParse(example.input).success,
`${contract.name}: ${example.title}`,
).toBe(true);
- expectRecursivePropertyDescriptions(tool?.inputSchema, contract.name);
+ expectMeaningfulPropertyDescriptions(tool?.inputSchema, contract.name);
}
} finally {
await Promise.allSettled([client.close(), server.close()]);
diff --git a/tests/boundary/process/processCaptureNormalization.test.ts b/tests/boundary/process/processCaptureNormalization.test.ts
index 58cf22c0fd..965dc2af5a 100644
--- a/tests/boundary/process/processCaptureNormalization.test.ts
+++ b/tests/boundary/process/processCaptureNormalization.test.ts
@@ -11,6 +11,7 @@ import {
parseProcessScenario,
} from "../../../src/domain/process/processScenario.js";
import { parseProcessCapture } from "../../../src/domain/process/processCaptureParsing.js";
+import { compareProcessCaptures } from "../../../src/domain/process/processComparison.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
import { itWithCaptureCapability } from "./processCaptureCapability.js";
@@ -64,7 +65,11 @@ itWithCaptureCapability(
).toBe(true);
expect(capture.manifest.comparison_contract).toMatchObject({
port_normalization_version: "contextual-endpoints-v2",
+ rendered_line_format: "trailing-spaces-trimmed-v1",
});
+ expect(capture.manifest.comparison_contract).not.toHaveProperty(
+ "pattern_normalization_version",
+ );
expect(capture.exit).toMatchObject({ code: 0, reason: "exited" });
expect(capture.cleanup).toMatchObject({
owned_process_group: "verified",
@@ -73,6 +78,67 @@ itWithCaptureCapability(
},
);
+itWithCaptureCapability.each([
+ { pattern: "AB", replacement: "$&$&", expected: "$&$&" },
+ {
+ pattern: "AB",
+ replacement: "$$ $` $' $1",
+ expected: "$$ $` $' $1",
+ },
+ { pattern: "", replacement: "-", expected: "-A-B-" },
+])(
+ "preserves literal replacement $replacement and rejects legacy comparison semantics",
+ async ({ pattern, replacement, expected }) => {
+ const root = await createTestTempDirectory("rea-process-literal-");
+ const scenario = parseProcessScenario({
+ executable: process.execPath,
+ arguments: ["-e", 'process.stdout.write("AB")'],
+ working_directory: root,
+ normalization: {
+ paths: false,
+ pids: false,
+ ports: false,
+ patterns: [{ pattern, replacement }],
+ },
+ settle_ms: 100,
+ });
+ const result = await captureProcessScenario(scenario);
+ if (!result.ok) throw result.error;
+ const capture = parseProcessCapture(
+ createProcessCaptureEvidence(scenario, result.value).normalized_result,
+ );
+ expect(capture.frames.map(({ data }) => data).join("")).toBe(expected);
+ expect(
+ capture.frames.map(({ raw_data, data }) => raw_data ?? data).join(""),
+ ).toBe("AB");
+ expect(capture.cleanup).toMatchObject({
+ owned_process_group: "verified",
+ temporary_root: "removed",
+ });
+ expect(compareProcessCaptures(capture, capture).terminal).toBe("unchanged");
+ const legacyContract = { ...capture.manifest.comparison_contract };
+ delete legacyContract.pattern_normalization_version;
+ const legacy = parseProcessCapture({
+ ...capture,
+ manifest: {
+ ...capture.manifest,
+ comparison_contract: legacyContract,
+ comparison_contract_sha256: digestProcessCommitment(legacyContract),
+ },
+ });
+ expect(() => compareProcessCaptures(legacy, capture)).toThrow(
+ expect.objectContaining({
+ issues: [
+ expect.objectContaining({
+ expected: ["pattern_normalization_version"],
+ }),
+ ],
+ }),
+ );
+ },
+ 20_000,
+);
+
itWithLinuxCaptureCapability.each([false, true])(
"honors pids=%s in owned Linux capture samples and committed Evidence",
async (pids) => {
diff --git a/tests/boundary/process/processFilesystemCoverageAdapters.test.ts b/tests/boundary/process/processFilesystemCoverageAdapters.test.ts
index ec7ce5b4c0..ff3b118a73 100644
--- a/tests/boundary/process/processFilesystemCoverageAdapters.test.ts
+++ b/tests/boundary/process/processFilesystemCoverageAdapters.test.ts
@@ -1,4 +1,5 @@
import { execFile } from "node:child_process";
+import { createHash } from "node:crypto";
import { mkdir, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { promisify } from "node:util";
@@ -7,6 +8,7 @@ import { expect, it, onTestFinished } from "vitest";
import { parseEvidence } from "../../../src/domain/evidence.js";
import { compareProcessCaptures } from "../../../src/domain/process/processComparison.js";
import { parseProcessCapture } from "../../../src/domain/process/processCaptureParsing.js";
+import { parseProcessScenario } from "../../../src/domain/process/processScenario.js";
import { createServer } from "../../../src/server/createServer.js";
import { createTestBinarySession } from "../../fixtures/binarySession.js";
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
@@ -58,6 +60,108 @@ const captureViaMcp = async (scenario: Record) => {
return evidence;
};
+itWithCaptureCapability.each(["cli", "mcp"] as const)(
+ "hashes the documented complete report despite omitted terminal output through %s",
+ async (adapter) => {
+ const document = await readFile("docs/process-capture.md", "utf8");
+ const example = document
+ .split("## Hash a report independently of terminal output")[1]
+ ?.match(/```json\n([\s\S]*?)\n```/u)?.[1];
+ if (example === undefined)
+ throw new Error("Missing report capture example");
+ const scenario = parseProcessScenario(JSON.parse(example));
+ const root = await createTestTempDirectory("rea-full-report-adapter-");
+ const evidence = await (adapter === "cli" ? captureViaCli : captureViaMcp)({
+ ...scenario,
+ executable: process.execPath,
+ working_directory: root,
+ filesystem_observation_paths: [join(root, "reports")],
+ });
+ const capture = parseProcessCapture(evidence.normalized_result);
+ const bytes = await readFile(join(root, "reports", "final.json"));
+ expect(JSON.parse(bytes.toString())).toEqual({
+ status: "complete",
+ data: "X".repeat(131072),
+ });
+ expect(bytes.length).toBeGreaterThan(scenario.limits.output_bytes);
+ expect(capture.files_after).toContainEqual(
+ expect.objectContaining({
+ path: "root_0:final.json",
+ size: bytes.length,
+ sha256: createHash("sha256").update(bytes).digest("hex"),
+ }),
+ );
+ expect(capture.truncation_details.filesystem_after).toMatchObject({
+ enumeration_truncated: false,
+ hash_budget_bytes: scenario.limits.file_bytes,
+ hashed_bytes: bytes.length,
+ hash_omissions: [],
+ });
+ expect(capture.truncation_details.raw_terminal.retained_bytes).toBeLessThan(
+ capture.truncation_details.raw_terminal.observed_bytes,
+ );
+ },
+ 20_000,
+);
+
+itWithCaptureCapability.each(["cli", "mcp"] as const)(
+ "accounts for exact cumulative hash budgets independently at each %s checkpoint",
+ async (adapter) => {
+ const runCapture = adapter === "cli" ? captureViaCli : captureViaMcp;
+ for (const shortfall of [0, 1]) {
+ const root = await createTestTempDirectory("rea-report-cumulative-");
+ const periodic = Buffer.from('{"status":"running"}');
+ const final = Buffer.from('{"status":"complete"}');
+ await writeFile(join(root, "periodic.json"), periodic);
+ const budget = periodic.length + final.length - shortfall;
+ const evidence = await runCapture({
+ executable: process.execPath,
+ arguments: [
+ "-e",
+ "require('node:fs').writeFileSync('final.json','{\"status\":\"complete\"}')",
+ ],
+ working_directory: root,
+ filesystem_observation_paths: [root],
+ limits: { output_bytes: 1, file_bytes: budget },
+ });
+ const capture = parseProcessCapture(evidence.normalized_result);
+ expect(capture.truncation_details.filesystem_before).toMatchObject({
+ hashed_bytes: periodic.length,
+ hash_omissions: [],
+ });
+ expect(capture.files_after).toContainEqual(
+ expect.objectContaining({
+ path: "root_0:final.json",
+ sha256: createHash("sha256").update(final).digest("hex"),
+ }),
+ );
+ expect(capture.truncation_details.filesystem_after).toMatchObject({
+ enumeration_truncated: false,
+ hashed_bytes: final.length + (shortfall === 0 ? periodic.length : 0),
+ hash_omissions:
+ shortfall === 0
+ ? []
+ : [
+ {
+ path: "root_0:periodic.json",
+ reason: "file_bytes_budget",
+ remaining_budget_bytes: periodic.length - 1,
+ },
+ ],
+ });
+ expect(
+ capture.files_after.find(({ path }) => path === "root_0:periodic.json")
+ ?.sha256,
+ ).toBe(
+ shortfall === 0
+ ? createHash("sha256").update(periodic).digest("hex")
+ : null,
+ );
+ }
+ },
+ 20_000,
+);
+
itWithCaptureCapability.each(["cli", "mcp"] as const)(
"reports unknown absence instead of deletion in truncated %s capture Evidence",
async (adapter) => {
diff --git a/tests/boundary/providers/native/artifactProvider.archiveInventory.test.ts b/tests/boundary/providers/native/artifactProvider.archiveInventory.test.ts
index 06fe705277..79765e711a 100644
--- a/tests/boundary/providers/native/artifactProvider.archiveInventory.test.ts
+++ b/tests/boundary/providers/native/artifactProvider.archiveInventory.test.ts
@@ -60,6 +60,18 @@ describe("artifact archive inventory", () => {
expect(inspectionEvidence.evidence_links).toEqual(
inspection.evidence_links,
);
+ // The graph is delivered once, in the nested inventory Evidence.
+ expect(
+ artifactInventoryResultSchema.parse(
+ inspection.substeps[0]?.evidence.normalized_result,
+ ).occurrences,
+ ).toHaveLength(3);
+ // Occurrences name every logical path; neither Evidence repeats them.
+ expect(inspectionEvidence.locations).toEqual([]);
+ expect(inspection.substeps[0]?.evidence.locations).toEqual([]);
+ expect(inspection.observations.map(({ kind }) => kind)).toEqual([
+ "root-manifest",
+ ]);
const cancellation = new AbortController();
cancellation.abort();
await expect(
diff --git a/tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts b/tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts
index 368d029dc3..8e8ff883e6 100644
--- a/tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts
+++ b/tests/boundary/providers/native/artifactProvider.archiveSafety.test.ts
@@ -1,4 +1,4 @@
-import { readFile, writeFile } from "node:fs/promises";
+import { lstat, readFile, writeFile } from "node:fs/promises";
import { join } from "node:path";
import { TextReader, Uint8ArrayWriter, ZipWriter } from "@zip.js/zip.js";
@@ -9,6 +9,8 @@ import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js
import { createDirectAnalysis } from "../../../../src/composition/directAnalysis.js";
import {
ArtifactPathRegistry,
+ destinationCaseCollisionMessage,
+ ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION,
normalizeArtifactPath,
} from "../../../../src/artifacts/ArtifactPaths.js";
import { ArtifactProvider } from "../../../../src/artifacts/ArtifactProvider.js";
@@ -18,6 +20,7 @@ import { artifactInventoryResultSchema } from "../../../../src/domain/artifactGr
import type { BinaryTarget } from "../../../../src/domain/binaryTargetTypes.js";
import { parseBinaryTarget } from "../../../../src/application/BinaryTargetResolver.js";
import { parseEvidence } from "../../../../src/domain/evidence.js";
+import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js";
const { runProviderAnalysis } = createDirectAnalysis({});
@@ -95,17 +98,8 @@ describe("artifact archive safety", () => {
});
const registry = new ArtifactPathRegistry();
registry.add("A.js", "file");
- let collisionError: unknown;
- try {
- registry.add("a.js", "file");
- } catch (error: unknown) {
- collisionError = error;
- }
- expect(collisionError).toBeInstanceOf(ArtifactReaderFailure);
- expect(collisionError).toMatchObject({
- reason: "path",
- message: "Artifact path collision: a.js differs only in case from A.js",
- });
+ expect(() => registry.add("a.js", "file")).not.toThrow();
+ expect(() => registry.add("A.js", "file")).toThrow(ArtifactReaderFailure);
for (const [filePath, childPath] of [
["Foo", "foo/bar"],
@@ -113,11 +107,15 @@ describe("artifact archive safety", () => {
] as const) {
const casePrefixRegistry = new ArtifactPathRegistry();
casePrefixRegistry.add(filePath, "file");
- expect(() => casePrefixRegistry.add(childPath, "file")).toThrow(
- ArtifactReaderFailure,
- );
+ expect(() => casePrefixRegistry.add(childPath, "file")).not.toThrow();
}
+ const exactPrefixRegistry = new ArtifactPathRegistry();
+ exactPrefixRegistry.add("Foo", "file");
+ expect(() => exactPrefixRegistry.add("Foo/bar", "file")).toThrow(
+ ArtifactReaderFailure,
+ );
+
const sameDirectoryRegistry = new ArtifactPathRegistry();
sameDirectoryRegistry.add("Foo/one.js", "file");
expect(() => sameDirectoryRegistry.add("Foo/two.js", "file")).not.toThrow();
@@ -126,8 +124,64 @@ describe("artifact archive safety", () => {
caseVariantDirectoryRegistry.add("Foo/one.js", "file");
expect(() =>
caseVariantDirectoryRegistry.add("foo/two.js", "file"),
- ).toThrow(ArtifactReaderFailure);
+ ).not.toThrow();
});
+
+ it("names a destination case collision from the directory listing", () => {
+ expect(
+ destinationCaseCollisionMessage("res/2f.xml", ["2F.xml", "other.xml"]),
+ ).toBe(
+ "Destination filesystem cannot store both res/2f.xml and res/2F.xml; inventory retains both logical names.",
+ );
+ expect(
+ destinationCaseCollisionMessage("res/2f.xml", ["2f.xml", "2F.xml"]),
+ ).toBeUndefined();
+ });
+});
+
+it("extracts case-distinct names only when the destination can preserve them", async () => {
+ const root = await createTestTempDirectory("rea-case-distinct-zip-");
+ await writeFile(join(root, "case-probe"), "lower");
+ await writeFile(join(root, "CASE-PROBE"), "upper");
+ const caseSensitive =
+ (await readFile(join(root, "case-probe"), "utf8")) === "lower";
+ const path = join(root, "fixture.zip");
+ const writer = new ZipWriter(new Uint8ArrayWriter());
+ await writer.add("Main.js", new TextReader("upper"));
+ await writer.add("main.js", new TextReader("lower"));
+ await writeFile(path, await writer.close());
+ const parsed = await parseBinaryTarget(path);
+ if (!parsed.ok) throw parsed.error;
+ const result = await inventory(parsed.value);
+ expect(result.limitations).toContain(
+ ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION,
+ );
+ expect(
+ result.occurrences
+ .filter(({ logical_path }) => logical_path !== ".")
+ .map(({ logical_path }) => logical_path)
+ .sort(),
+ ).toEqual(["Main.js", "main.js"]);
+ const output = join(root, "output");
+ const extracted = await new ArtifactProvider(process.env)
+ .createClient(parsed.value)
+ .execute(
+ "extract_artifact",
+ artifactExtractionExecutionSchema.parse({ output_root: output }),
+ );
+ if (!caseSensitive) {
+ if (extracted.ok) throw new Error("expected a destination case collision");
+ expect(projectAnalysisError(extracted.error)).toMatchObject({
+ message:
+ "Destination filesystem cannot store both main.js and Main.js; inventory retains both logical names.",
+ details: { operation: "extract_artifact", reason: "path" },
+ });
+ await expect(lstat(output)).rejects.toMatchObject({ code: "ENOENT" });
+ return;
+ }
+ if (!extracted.ok) throw extracted.error;
+ expect(await readFile(join(output, "Main.js"), "utf8")).toBe("upper");
+ expect(await readFile(join(output, "main.js"), "utf8")).toBe("lower");
});
const inventory = async (targetValue: BinaryTarget) => {
const result = await new ArtifactProvider(process.env)
diff --git a/tests/boundary/providers/native/artifactProvider.directoryInventory.test.ts b/tests/boundary/providers/native/artifactProvider.directoryInventory.test.ts
index f6e226a1df..31f5915f7e 100644
--- a/tests/boundary/providers/native/artifactProvider.directoryInventory.test.ts
+++ b/tests/boundary/providers/native/artifactProvider.directoryInventory.test.ts
@@ -65,7 +65,7 @@ describe("artifact directory inventory", () => {
expect.objectContaining({
relation: "maps-source",
parent_artifact_id: script?.artifact_id,
- child_artifact_id: sourceMap?.artifact_id,
+ occurrence_id: sourceMap?.occurrence_id,
}),
]),
);
diff --git a/tests/boundary/providers/native/artifactProvider.extraction.test.ts b/tests/boundary/providers/native/artifactProvider.extraction.test.ts
index 9726e5560b..9a538e2b12 100644
--- a/tests/boundary/providers/native/artifactProvider.extraction.test.ts
+++ b/tests/boundary/providers/native/artifactProvider.extraction.test.ts
@@ -65,6 +65,8 @@ describe("artifact extraction", () => {
]),
extraction_manifest: { output_root_alias: "$OUTPUT_ROOT" },
});
+ // artifacts names every extracted path; the Evidence does not repeat them.
+ expect(result.value.locations).toEqual([]);
expect(await readFile(join(output, "assets", "selected.js"), "utf8")).toBe(
"selected();\n",
);
diff --git a/tests/boundary/providers/native/artifactProvider.failureDetail.test.ts b/tests/boundary/providers/native/artifactProvider.failureDetail.test.ts
index dcec03c93b..e2b2fd3859 100644
--- a/tests/boundary/providers/native/artifactProvider.failureDetail.test.ts
+++ b/tests/boundary/providers/native/artifactProvider.failureDetail.test.ts
@@ -6,30 +6,73 @@ import { writeOrderedZip } from "../../../fixtures/artifactEntryOrder.js";
import { createTestTempDirectory } from "../../../fixtures/temporaryDirectory.js";
import { parseBinaryTarget } from "../../../../src/application/BinaryTargetResolver.js";
+import {
+ destinationCaseCollisionMessage,
+ ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION,
+} from "../../../../src/artifacts/ArtifactPaths.js";
import { ArtifactProvider } from "../../../../src/artifacts/ArtifactProvider.js";
+import { artifactInventoryResultSchema } from "../../../../src/domain/artifactGraph.js";
+import { ArtifactOperationError } from "../../../../src/domain/artifactOperationError.js";
import { projectAnalysisError } from "../../../../src/domain/analysisErrorProjection.js";
-it.each([
- [
- ["Main.js", "main.js"],
- "Artifact path collision: main.js differs only in case from Main.js",
- ],
- [["a\\b.js"], 'Artifact path is absolute or unsafe: "a\\\\b.js"'],
-] as const)(
- "reports the failed path constraint for %j",
- async (entries, detail) => {
- const root = await createTestTempDirectory("rea-artifact-detail-");
- const path = join(root, "fixture.zip");
- await writeOrderedZip(path, entries);
- const target = await parseBinaryTarget(path);
- if (!target.ok) throw new Error("expected a ZIP target");
- const result = await new ArtifactProvider(process.env)
- .createClient(target.value)
- .execute("inventory_artifact", {});
- if (result.ok) throw new Error("expected the path constraint to fail");
- expect(projectAnalysisError(result.error)).toMatchObject({
- code: "artifact_operation_failed",
- details: { operation: "inventory_artifact", reason: "path", detail },
- });
- },
-);
+it("keeps logical names that differ only in case and records the limitation", async () => {
+ const root = await createTestTempDirectory("rea-artifact-detail-");
+ const path = join(root, "fixture.zip");
+ await writeOrderedZip(path, ["Main.js", "main.js"]);
+ const target = await parseBinaryTarget(path);
+ if (!target.ok) throw new Error("expected a ZIP target");
+ const result = await new ArtifactProvider(process.env)
+ .createClient(target.value)
+ .execute("inventory_artifact", {});
+ if (!result.ok) throw result.error;
+ const inventory = artifactInventoryResultSchema.parse(result.value.result);
+ expect(inventory.occurrences.map(({ logical_path }) => logical_path)).toEqual(
+ expect.arrayContaining(["Main.js", "main.js"]),
+ );
+ expect(inventory.limitations).toContain(
+ ENGLISH_UNICODE_CASE_INVENTORY_LIMITATION,
+ );
+ for (const logicalPath of ["Main.js", "main.js"]) {
+ expect(
+ inventory.occurrences
+ .find((occurrence) => occurrence.logical_path === logicalPath)
+ ?.limitations.join("\n"),
+ ).toContain("English (en-US) Unicode case folding");
+ }
+});
+
+it("reports an unsafe path as a hostile artifact path", async () => {
+ const root = await createTestTempDirectory("rea-artifact-detail-");
+ const path = join(root, "fixture.zip");
+ await writeOrderedZip(path, ["a\\b.js"]);
+ const target = await parseBinaryTarget(path);
+ if (!target.ok) throw new Error("expected a ZIP target");
+ const result = await new ArtifactProvider(process.env)
+ .createClient(target.value)
+ .execute("inventory_artifact", {});
+ if (result.ok) throw new Error("expected the path constraint to fail");
+ expect(projectAnalysisError(result.error)).toMatchObject({
+ code: "artifact_operation_failed",
+ message:
+ "Artifact contains an unsafe internal path. Inspect the reported path and correct the artifact before retrying.",
+ details: {
+ operation: "inventory_artifact",
+ reason: "path",
+ detail: 'Artifact path is absolute or unsafe: "a\\\\b.js"',
+ },
+ });
+});
+
+it("returns a destination case collision as the user-facing path message", () => {
+ const detail = destinationCaseCollisionMessage("res/2f.xml", ["2F.xml"]);
+ expect(detail).toEqual(expect.any(String));
+ if (detail === undefined) return;
+ expect(
+ projectAnalysisError(
+ new ArtifactOperationError("extract_artifact", "path", undefined, detail),
+ ),
+ ).toMatchObject({
+ message: detail,
+ details: { reason: "path", detail },
+ });
+});
diff --git a/tests/boundary/providers/native/nativeProvider.test.ts b/tests/boundary/providers/native/nativeProvider.test.ts
index d5be920ba1..70a3b06ad6 100644
--- a/tests/boundary/providers/native/nativeProvider.test.ts
+++ b/tests/boundary/providers/native/nativeProvider.test.ts
@@ -552,6 +552,43 @@ describe("native macOS provider inspection", () => {
});
});
+describe("native macOS provider raw results", () => {
+ it("records command provenance once, in the normalized result", async () => {
+ directory = await createTestTempDirectory("rea-native-");
+ const app = join(directory, "Fixture.app");
+ const executable = join(app, "Contents/MacOS/Fixture");
+ await mkdir(join(app, "Contents/MacOS"), { recursive: true });
+ await writeFile(executable, "fixture");
+ await writeFile(join(app, "Contents/Info.plist"), "fixture");
+ const client = new NativeMacOSProvider(
+ {},
+ new FixtureRunner(),
+ "darwin",
+ ).createClient(await nativeMachoTargetForFile(executable, app));
+ const requests: readonly (readonly [
+ Parameters[0],
+ Parameters[1],
+ ])[] = [
+ ["inspect_macho", {}],
+ ["list_architectures", {}],
+ ["inspect_signature", {}],
+ ["inspect_plist", {}],
+ ["demangle_swift", { symbols: ["$s4Test3fooyyF", "plain_symbol"] }],
+ ];
+
+ for (const [operation, parameters] of requests) {
+ const execution = await client.execute(operation, parameters);
+ if (!execution.ok) throw execution.error;
+ expect(execution.value.rawResult, operation).toBeNull();
+ expect(execution.value.result, operation).toMatchObject({
+ provenance: expect.arrayContaining([
+ expect.objectContaining({ tool: expect.any(String) }),
+ ]),
+ });
+ }
+ });
+});
+
describe("native plist defaults for iOS-style bundles", () => {
it("defaults to the Info.plist the bundle program was resolved from", async () => {
directory = await createTestTempDirectory("rea-native-flat-");
diff --git a/tests/composition/android/application.test.ts b/tests/composition/android/application.test.ts
index 082a706b0b..3529dc4e85 100644
--- a/tests/composition/android/application.test.ts
+++ b/tests/composition/android/application.test.ts
@@ -86,6 +86,12 @@ describe("Android application projection", () => {
expect(left.bridge_candidates).toEqual([
expect.objectContaining({ basis: "react-native-convention" }),
]);
+ expect(left.limitations).toEqual(
+ expect.arrayContaining([
+ "A bridge basis is inferred from the native path and is repeated for every managed component.",
+ "ZIP inventory cannot see the APK Signing Block, so an empty signing array does not mean the APK is unsigned.",
+ ]),
+ );
expect(JSON.stringify(left)).not.toContain("opaque signing");
});
@@ -146,6 +152,45 @@ describe("Android application projection", () => {
});
});
+describe("case-distinct APK resource names", () => {
+ it("inventories and projects an APK whose resource names differ only in case", async () => {
+ const root = await createTestTempDirectory("rea-android-case-");
+ const path = join(root, "Shrunk.apk");
+ const writer = new ZipWriter(new Uint8ArrayWriter());
+ await writer.add(
+ "AndroidManifest.xml",
+ new Uint8ArrayReader(Uint8Array.from([3, 0, 8, 0])),
+ );
+ await writer.add(
+ "classes.dex",
+ new Uint8ArrayReader(
+ Uint8Array.from([0x64, 0x65, 0x78, 0x0a, 0x30, 0x33, 0x35, 0]),
+ ),
+ );
+ await writer.add("res/2F.xml", new TextReader("upper"));
+ await writer.add("res/2f.xml", new TextReader("lower"));
+ await writeFile(path, await writer.close());
+
+ const inventory = parseEvidence(
+ await runProviderAnalysis(path, "inventory_artifact", {}),
+ );
+ const result = projectAndroidApplicationEvidence({
+ inventory_evidence: [inventory],
+ });
+ const projection = androidApplicationProjectionResultSchema.parse(
+ requireSuccessfulProjection(result).normalized_result,
+ );
+ expect(projection.root_format).toBe("apk");
+ expect(projection.components.manifests).toHaveLength(1);
+ expect(projection.components.dex).toHaveLength(1);
+ expect(JSON.stringify(inventory)).toContain("res/2F.xml");
+ expect(JSON.stringify(inventory)).toContain("res/2f.xml");
+ expect(JSON.stringify(inventory)).toContain(
+ "English (en-US) Unicode case folding",
+ );
+ });
+});
+
describe("Android bridge candidate coverage", () => {
it("returns every component and bridge candidate from the inventory", async () => {
const root = await createTestTempDirectory("rea-android-complete-");
diff --git a/tests/composition/javascript-applications/electronApplicationAnalysis.test.ts b/tests/composition/javascript-applications/electronApplicationAnalysis.test.ts
index 3484f98a24..dd7bd8809e 100644
--- a/tests/composition/javascript-applications/electronApplicationAnalysis.test.ts
+++ b/tests/composition/javascript-applications/electronApplicationAnalysis.test.ts
@@ -89,6 +89,45 @@ describe("static Electron application analysis", () => {
expect(requestedMembers).toEqual(["*"]);
});
+ it("maps Electron boundaries through bundler-renamed bindings", async () => {
+ const root = await renamedBindingFixtureDirectory();
+
+ const result = await reconstructJavaScriptArtifact({ input_path: root });
+ const graph = parseJavaScriptApplicationGraph(result.graph);
+ const handledChannels = graph.nodes.flatMap(({ kind, observations }) =>
+ kind === "ipc-handler"
+ ? observations.flatMap(({ properties }) =>
+ properties.side === "main" ? [properties.channel] : [],
+ )
+ : [],
+ );
+
+ expect(result.electron_summary).toMatchObject({
+ browser_windows: 1,
+ context_bridge_apis: 1,
+ exposed_api_members: 2,
+ ipc: {
+ main_handlers: 2,
+ paired_renderer_transmissions: 2,
+ unpaired_literal_renderer_transmissions: 0,
+ },
+ utility_processes: 1,
+ });
+ expect(handledChannels.sort()).toEqual(["rea:read", "rea:write"]);
+ });
+
+ it("maps unchanged var and let require aliases while excluding reassigned bindings", async () => {
+ const root = await requireAliasFixtureDirectory();
+
+ const result = await reconstructJavaScriptArtifact({ input_path: root });
+
+ expect(result.electron_summary.ipc).toMatchObject({
+ main_handlers: 2,
+ paired_renderer_transmissions: 2,
+ unpaired_literal_renderer_transmissions: 0,
+ });
+ });
+
it("returns a tagged cancellation without executing application code", async () => {
const root = await fixtureDirectory();
const controller = new AbortController();
@@ -299,6 +338,89 @@ const fixtureDirectory = async (): Promise => {
return root;
};
+const requireAliasFixtureDirectory = async (): Promise => {
+ const root = await createTestTempDirectory("rea-electron-require-aliases-");
+ await Promise.all([
+ writeFile(
+ join(root, "package.json"),
+ JSON.stringify({
+ name: "rea-electron-require-aliases",
+ main: "main.cjs",
+ }),
+ ),
+ writeFile(
+ join(root, "main.cjs"),
+ String.raw`
+var { ipcMain: main } = require("electron");
+let listener = require("electron/main").ipcMain;
+main.handle("rea:read", async () => "value");
+listener.on("rea:write", () => undefined);
+
+let replaced = require("electron").ipcMain;
+replaced = { handle() {} };
+replaced.handle("rea:reassigned", () => undefined);
+
+var assignedLater;
+assignedLater.handle("rea:before-assignment", () => undefined);
+assignedLater = require("electron").ipcMain;
+
+if (globalThis.registerIpc) {
+ var conditional = require("electron").ipcMain;
+}
+conditional.handle("rea:conditional", () => undefined);
+`,
+ ),
+ writeFile(
+ join(root, "preload.cjs"),
+ String.raw`
+const { ipcRenderer } = require("electron");
+ipcRenderer.invoke("rea:read");
+ipcRenderer.send("rea:write", "value");
+`,
+ ),
+ ]);
+ return root;
+};
+
+const renamedBindingFixtureDirectory = async (): Promise => {
+ const root = await createTestTempDirectory("rea-electron-renamed-");
+ await Promise.all([
+ writeFile(
+ join(root, "package.json"),
+ JSON.stringify({ name: "rea-electron-renamed", main: "main.mjs" }),
+ ),
+ // Bundlers rename colliding imports, e.g. esbuild's ipcMain2.
+ writeFile(
+ join(root, "main.mjs"),
+ String.raw`
+import { BrowserWindow as BrowserWindow2, ipcMain as ipcMain2 } from "electron";
+import { ipcMain as ipcMain3, utilityProcess as utility } from "electron/main";
+
+new BrowserWindow2({ webPreferences: { preload: "./preload.cjs" } });
+ipcMain2.handle("rea:read", async () => "value");
+ipcMain3.on("rea:write", () => undefined);
+utility.fork("./worker.js");
+function register(ipcMain2) {
+ ipcMain2.handle("rea:shadowed", async () => "local");
+}
+export { register };
+`,
+ ),
+ writeFile(
+ join(root, "preload.cjs"),
+ String.raw`
+const { contextBridge: bridge, ipcRenderer: ipc } = require("electron");
+bridge.exposeInMainWorld("reaApi", {
+ read: () => ipc.invoke("rea:read"),
+ write: (value) => ipc.send("rea:write", value),
+});
+`,
+ ),
+ writeFile(join(root, "worker.js"), "module.exports = {};"),
+ ]);
+ return root;
+};
+
const computedMemberFixtureDirectory = async (): Promise => {
const root = await createTestTempDirectory("rea-electron-computed-members-");
await mkdir(join(root, "native"));
diff --git a/tests/conformance/contracts/mcpInputSchemaProfile.test.ts b/tests/conformance/contracts/mcpInputSchemaProfile.test.ts
new file mode 100644
index 0000000000..ababc135e8
--- /dev/null
+++ b/tests/conformance/contracts/mcpInputSchemaProfile.test.ts
@@ -0,0 +1,29 @@
+import { describe, expect, it } from "vitest";
+
+import { parseMcpInputSchemaProfile } from "../../../src/config/mcpInputSchemaProfile.js";
+
+describe("MCP input schema profile configuration", () => {
+ it("accepts the full and compact presentations", () => {
+ expect(parseMcpInputSchemaProfile(undefined)).toEqual({
+ ok: true,
+ value: undefined,
+ });
+ expect(parseMcpInputSchemaProfile("full")).toEqual({
+ ok: true,
+ value: "full",
+ });
+ expect(parseMcpInputSchemaProfile("compact")).toEqual({
+ ok: true,
+ value: "compact",
+ });
+ });
+
+ it.each(["", "FULL", "compact ", "minimal", "0"])(
+ "rejects unknown profiles: %j",
+ (value) => {
+ const parsed = parseMcpInputSchemaProfile(value);
+ if (parsed.ok) throw new Error("Expected invalid profile");
+ expect(parsed.error.message).toContain("REA_MCP_INPUT_SCHEMA_PROFILE");
+ },
+ );
+});
diff --git a/tests/conformance/ghidra/cross-format.c b/tests/conformance/ghidra/cross-format.c
index f718b9b540..22efa61597 100644
--- a/tests/conformance/ghidra/cross-format.c
+++ b/tests/conformance/ghidra/cross-format.c
@@ -8,10 +8,15 @@ __attribute__((noinline, used)) int rea_cross_leaf(int value) {
}
__attribute__((noinline, used)) int rea_cross_branch(int value) {
- if (value > 10) {
- return rea_cross_leaf(value);
+ /* A volatile-dependent loop keeps this callee multi-block at every
+ optimization level: newer clang folds a symmetric if/else of tail calls
+ into a conditional negate, and cross-format verification requires a
+ recovered multi-block callee. */
+ int adjusted = value > 10 ? value + 1 : -value;
+ while (adjusted > 100) {
+ adjusted -= rea_cross_global;
}
- return rea_cross_leaf(-value);
+ return rea_cross_leaf(adjusted);
}
__attribute__((noinline, used)) int rea_cross_indirect(
diff --git a/tests/conformance/readmeConsistency.test.ts b/tests/conformance/readmeConsistency.test.ts
index 65faab5a2d..79a1fc518b 100644
--- a/tests/conformance/readmeConsistency.test.ts
+++ b/tests/conformance/readmeConsistency.test.ts
@@ -21,6 +21,7 @@ const translatedReadmes = [
"README_pt-BR.md",
"README_ar.md",
"README_fa.md",
+ "README_id.md",
] as const;
const jsonExamples = (content: string): unknown[] =>
diff --git a/tests/fixtures/analysisView.ts b/tests/fixtures/analysisView.ts
index 0f928a22a2..d3be2b1e3b 100644
--- a/tests/fixtures/analysisView.ts
+++ b/tests/fixtures/analysisView.ts
@@ -99,7 +99,7 @@ export const analysisViewLayoutEvidence = (
operation: "inspect_binary_layout",
parameters: { path: layout.artifact.path },
result: layout,
- rawResult: layout,
+ rawResult: null,
confidence: "observed",
limitations: layout.limitations,
locations: [{ kind: "artifact-path", path: layout.artifact.path }],
diff --git a/tests/fixtures/android/jadx.ts b/tests/fixtures/android/jadx.ts
index 3488647a07..8e8ac48847 100644
--- a/tests/fixtures/android/jadx.ts
+++ b/tests/fixtures/android/jadx.ts
@@ -74,7 +74,11 @@ export const createJadxProtocolFixture = async (
ownsProcessLifetime: true,
cleanup: () => cleanupOwnedProcessGroup(spawned.ownership),
});
- expect((await supervisor.stop()).status).not.toBe("incomplete");
+ // The SDK can close the child transport before process exit settles.
+ // Retry retained ownership without bypassing the supervisor's proof.
+ await expect
+ .poll(() => supervisor.stop(), { timeout: 5_000 })
+ .not.toMatchObject({ status: "incomplete" });
if (options.cwd !== undefined)
await rm(options.cwd, { recursive: true, force: true });
});
diff --git a/tests/fixtures/clientEnvironment.ts b/tests/fixtures/clientEnvironment.ts
index 79961e49d4..91ccb92cfd 100644
--- a/tests/fixtures/clientEnvironment.ts
+++ b/tests/fixtures/clientEnvironment.ts
@@ -17,11 +17,22 @@ export const CLIENT_LOCATION_ENVIRONMENT = [
"CODEX_HOME",
"COPILOT_HOME",
"GROK_HOME",
+ "GEMINI_CLI_HOME",
+ "GEMINI_CLI_TRUST_WORKSPACE",
+ "GEMINI_RESTRICTED_MODE",
+ "GEMINI_CLI_TRUSTED_FOLDERS_PATH",
+ "GEMINI_CLI_SYSTEM_SETTINGS_PATH",
+ "GEMINI_CLI_SYSTEM_DEFAULTS_PATH",
+ "HERMES_DATA_DIR_SUFFIX",
+ "HERMES_HOME",
+ "LOCALAPPDATA",
"OMP_PROFILE",
"OPENCODE_CONFIG",
+ "OPENCODE_CONFIG_DIR",
"PI_CODING_AGENT_DIR",
"PI_CONFIG_DIR",
"PI_PROFILE",
+ "QWEN_HOME",
"SAND_DATA_ROOT",
"XDG_CONFIG_HOME",
] as const;
diff --git a/tests/fixtures/javascriptReturnShapes.ts b/tests/fixtures/javascriptReturnShapes.ts
new file mode 100644
index 0000000000..02c66bc84b
--- /dev/null
+++ b/tests/fixtures/javascriptReturnShapes.ts
@@ -0,0 +1,91 @@
+import { Client } from "@modelcontextprotocol/client";
+import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
+import { writeFile } from "node:fs/promises";
+import { join, resolve } from "node:path";
+import { expect } from "vitest";
+
+import { toolContract } from "../../src/contracts/toolContracts.js";
+import { javascriptApplicationAnalysisResultSchema } from "../../src/domain/javascript/javascriptApplicationAnalysis.js";
+import { projectedExportReturnShapesSchema } from "../../src/domain/javascript/javascriptExportShapeComparisonSchemas.js";
+import { createTestTempDirectory } from "./temporaryDirectory.js";
+import type { TestCli } from "../support/cli/cliFixture.js";
+
+const exportedReturnFields = (value: unknown) => {
+ const analysis = javascriptApplicationAnalysisResultSchema.parse(value);
+ const projections = analysis.graph.nodes
+ .flatMap(({ observations }) => observations)
+ .filter(
+ ({ properties }) => properties.semantic_role === "export-return-shapes",
+ )
+ .map(({ properties }) =>
+ projectedExportReturnShapesSchema.parse(properties),
+ );
+ return (name: string) => {
+ const projection = projections.find(
+ ({ exported_name }) => exported_name === name,
+ );
+ if (projection === undefined) throw new Error(`Missing export ${name}`);
+ expect(projection.return_shape_coverage).toMatchObject({
+ status: "complete",
+ projection_complete: true,
+ });
+ return projection.static_return_shapes.flatMap(({ fields }) => fields);
+ };
+};
+
+/** Select complete return fields from a real JavaScript application result. */
+export type JavaScriptReturnFields = ReturnType;
+
+/** Exercise identical return assertions through the compiled CLI and stdio MCP. */
+export const verifyJavaScriptReturnShapes = async (
+ cli: TestCli,
+ source: string,
+ check: (fields: JavaScriptReturnFields) => void,
+): Promise => {
+ const root = await createTestTempDirectory("rea-return-shapes-");
+ await writeFile(join(root, "app.js"), source);
+ const cliResponse = await cli.run({
+ arguments: [
+ "analyze-javascript-application",
+ root,
+ "--artifact-format",
+ "directory",
+ "--json",
+ ],
+ });
+ expect(cliResponse.exitCode).toBe(0);
+ const contract = toolContract("analyze_javascript_application");
+ const cliEvidence = contract.outputSchema.parse(cliResponse.json);
+ check(exportedReturnFields(cliEvidence.normalized_result));
+ const client = new Client({ name: "return-shapes-e2e", version: "1" });
+ const transport = new StdioClientTransport({
+ command: process.execPath,
+ args: [resolve("scripts/rea.mjs"), "mcp"],
+ cwd: process.cwd(),
+ env: { PATH: process.env.PATH ?? "" },
+ stderr: "pipe",
+ });
+ transport.stderr?.on("data", () => undefined);
+ try {
+ await client.connect(transport);
+ const response = await client.callTool({
+ name: "analyze_javascript_application",
+ arguments: { input_path: root, format: "directory" },
+ });
+ expect(response.isError).not.toBe(true);
+ const evidence = contract.outputSchema.parse(response.structuredContent);
+ expect(evidence.evidence_id).toBe(cliEvidence.evidence_id);
+ check(exportedReturnFields(evidence.normalized_result));
+ const text = response.content.find((item) => item.type === "text");
+ if (text?.type !== "text") throw new Error("Missing MCP text result");
+ check(
+ exportedReturnFields(
+ contract.outputSchema.parse(JSON.parse(text.text)).normalized_result,
+ ),
+ );
+ await client.ping();
+ } finally {
+ await client.close();
+ await transport.close();
+ }
+};
diff --git a/tests/fixtures/mcpDoctorReply.mjs b/tests/fixtures/mcpDoctorReply.mjs
new file mode 100644
index 0000000000..14dad1240e
--- /dev/null
+++ b/tests/fixtures/mcpDoctorReply.mjs
@@ -0,0 +1,48 @@
+import { readFile } from "node:fs/promises";
+import { createInterface } from "node:readline";
+
+const snapshot = JSON.parse(await readFile(process.argv[2], "utf8"));
+const reply = structuredClone(snapshot.reply);
+if (process.argv[3] === "invalid") {
+ reply.structuredContent.schema_violation_control = true;
+ reply.content = reply.content.map((item) =>
+ item.type === "text"
+ ? { ...item, text: JSON.stringify(reply.structuredContent) }
+ : item,
+ );
+}
+
+const input = createInterface({ input: process.stdin });
+input.on("line", (line) => {
+ const request = JSON.parse(line);
+ if (request.id === undefined) return;
+ let result;
+ switch (request.method) {
+ case "initialize":
+ result = {
+ protocolVersion: snapshot.protocolVersion,
+ capabilities: { tools: {}, prompts: {} },
+ serverInfo: snapshot.serverInfo,
+ };
+ break;
+ case "tools/list":
+ result = snapshot.tools;
+ break;
+ case "prompts/list":
+ result = snapshot.prompts;
+ break;
+ case "tools/call":
+ if (request.params.name === "binary_session") result = reply;
+ break;
+ case "ping":
+ result = {};
+ break;
+ }
+ const response =
+ result === undefined
+ ? { error: { code: -32601, message: "Unknown fixture method or tool" } }
+ : { result };
+ process.stdout.write(
+ `${JSON.stringify({ jsonrpc: "2.0", id: request.id, ...response })}\n`,
+ );
+});
diff --git a/tests/process-global/mcpDeliveryEnvironment.test.ts b/tests/process-global/mcpDeliveryEnvironment.test.ts
index 6d8751d5e5..51e371b45a 100644
--- a/tests/process-global/mcpDeliveryEnvironment.test.ts
+++ b/tests/process-global/mcpDeliveryEnvironment.test.ts
@@ -1,6 +1,7 @@
import { Client, InMemoryTransport } from "@modelcontextprotocol/client";
import { STDIO_DEFAULT_MAX_BUFFER_SIZE } from "@modelcontextprotocol/server";
import { afterEach, expect, it, vi } from "vitest";
+import { z } from "zod";
import { createAnalysisExecution } from "../../src/application/AnalysisProvider.js";
import { AnalysisInputError } from "../../src/domain/analysisErrorCore.js";
@@ -73,16 +74,19 @@ it("keeps independent server budgets for successful results and oversized errors
Math.ceil((smallBudget + 65536) / 17),
);
for (const server of [small, large])
- server.registerTool("selected_failure", { inputSchema: {} }, async () =>
- server.delivery.toErrorToolResult(
- new AnalysisInputError("fixture", undefined, [
- {
- path: ["procedure"],
- reason: "invalid_value",
- message: diagnostic,
- },
- ]),
- ),
+ server.registerTool(
+ "selected_failure",
+ { inputSchema: z.object({}) },
+ async () =>
+ server.delivery.toErrorToolResult(
+ new AnalysisInputError("fixture", undefined, [
+ {
+ path: ["procedure"],
+ reason: "invalid_value",
+ message: diagnostic,
+ },
+ ]),
+ ),
);
const [smallClient, largeClient] = await Promise.all([
diff --git a/turbo.json b/turbo.json
index 3925237083..5b93503d78 100644
--- a/turbo.json
+++ b/turbo.json
@@ -39,7 +39,7 @@
"package-lock.json",
"scripts/generate-package-metadata.mjs",
"scripts/lib/generated-file.mjs",
- "skill-src/reverse-engineer-anything/SKILL.md",
+ ".agents/skills/reverse-engineer-anything/SKILL.md",
"src/generatedPackageMetadata.ts"
]
},
@@ -138,7 +138,7 @@
"docs/**",
"!docs/public/product-catalog.json",
"!docs/verification/managed-conformance-*.json",
- "skill-src/**",
+ ".agents/skills/reverse-engineer-anything/**",
"README.md",
{
"mode": "dependencyOutputs",
@@ -170,7 +170,7 @@
"package-lock.json",
"scripts/generate-skill-metadata.mjs",
"scripts/lib/generated-file.mjs",
- "skill-src/**"
+ ".agents/skills/reverse-engineer-anything/**"
],
"outputs": ["skills/**"]
},
diff --git a/website/AGENTS.md b/website/AGENTS.md
index d96ab2b623..e97181ec6c 100644
--- a/website/AGENTS.md
+++ b/website/AGENTS.md
@@ -14,6 +14,11 @@ for asset notes and the source of case-study claims.
case study focused on one question, with further examples in native details.
- Show what REA returns and how the agent uses it. Preserve source attribution,
target identity and verification scope.
+- Blog articles teach methods, decisions and workflows through concrete cases.
+ Lead with the takeaway. Essays can develop a personal argument through
+ experience; figures are optional and should add an explanation the prose
+ needs. Use engine details only when they explain a decision. Label working
+ outlines and mark them `noindex`.
- Keep personal paths, account data, credentials and raw captures outside the
public assets. Use generic inputs and label shortened display paths.
- Follow the same clear writing standard in search titles and descriptions.
diff --git a/website/README.md b/website/README.md
index 2005c633da..93faf5e596 100644
--- a/website/README.md
+++ b/website/README.md
@@ -42,6 +42,8 @@ HTML files. No URL list needs to be maintained.
- `public/examples/dino-lab/index.html`: adjustable-speed mini-game, recovered rule, original-game check and browser analysis steps.
- `public/first-investigation/index.html`: a guided Notes export investigation, from setup to a checked CSV prediction.
- `public/showcase/index.html`: the case-study index.
+- `public/blog/index.html`: articles about reconstruction methods, ports, mods and reverse engineering.
+- `public/blog/touhou-reconstruction/index.html`: a personal essay about agent-driven reconstruction, knowledge that compounds across TH08 and TH095, and the workflow's applicability to PC-98 TH04.
- `public/showcase/aegis/index.html`: Aegis's Android login-code calculation, with an adjustable clock and reference checks.
- `public/showcase/dx-ball/index.html`: sound-pan investigation and project status.
- `public/showcase/notion/index.html`: Notion's Electron clipboard bridge and rich clipboard format.
@@ -121,6 +123,19 @@ and Cloudflare's [managed robots behavior](https://developers.cloudflare.com/bot
## Content
+Blog articles explain methods, decisions and ideas through project experience.
+Lead with a takeaway, then develop the argument through concrete examples.
+An essay can explore a change in perspective without becoming a step-by-step
+guide. Add a figure when it contributes a useful explanation; an opening flow
+is optional. Case studies remain focused on one inspected behavior. Keep engine
+details in supporting references unless they explain a relevant decision.
+
+The Touhou article is an English essay. Its provenance,
+quoted README passage and milestone counting rules are in
+[evidence/touhou-reconstruction-method.md](evidence/touhou-reconstruction-method.md).
+The completed text has normal search metadata and is included in the generated
+sitemap.
+
Keep the copy direct and specific. Explain the task and the result before listing
tool names. Setup commands and runtime requirements should match the released
package. Keep the core learning path inside the site. Source, issue and evidence
@@ -177,11 +192,14 @@ The homepage starts with a short installation prompt and the ordinary setup
command, then explains what reverse engineering is and why someone would use
it. The manual/agent comparison introduces REA’s role before the examples.
Its opening links directly to Showcases and lets experienced readers skip to
-the analysis guides. Case-study previews live on the Showcases page.
+the analysis guides. A Blog link below that shortcut leads to reconstruction
+methods and project notes. Case-study previews live on the Showcases page.
The closing section offers copyable project prompts, from cloning `rea.tools`
to reconstructing a game from its executable. The experienced-reader shortcut
lands directly on the guide links below these prompts.
-The page ends with FAQ, Discord and issue-report links under “Any questions?”.
+“Any questions?” offers FAQ, Discord and issue-report links. The page ends
+with “Join the community”, which links Discord and REA's X account.
+Every content page's header and footer link GitHub, Discord and X.
A right-side table of contents stays visible at widths of 1440px and above.
On narrower screens it becomes a sticky, native disclosure; selecting a link
closes the menu and focuses the destination. The homepage script follows
diff --git a/website/evidence/touhou-reconstruction-method.md b/website/evidence/touhou-reconstruction-method.md
new file mode 100644
index 0000000000..99f3ceb339
--- /dev/null
+++ b/website/evidence/touhou-reconstruction-method.md
@@ -0,0 +1,175 @@
+# Touhou reconstruction method: evidence notes
+
+Checked on 10 October 2026. This document supports the full English essay
+at `public/blog/touhou-reconstruction/index.html`.
+
+## Method sources
+
+The Factory checkpoint is
+[`038b6bed`](https://github.com/N0zoM1z0/touhou-reconstruction-factory/tree/038b6bed51d273c8b3680553b6f43e1b97d19bd1).
+Read its [agent autonomy](https://github.com/N0zoM1z0/touhou-reconstruction-factory/blob/038b6bed51d273c8b3680553b6f43e1b97d19bd1/docs/agent-autonomy.md),
+[semantic workflow](https://github.com/N0zoM1z0/touhou-reconstruction-factory/blob/038b6bed51d273c8b3680553b6f43e1b97d19bd1/docs/semantic-reconstruction.md),
+and [cross-game knowledge](https://github.com/N0zoM1z0/touhou-reconstruction-factory/blob/038b6bed51d273c8b3680553b6f43e1b97d19bd1/docs/knowledge-base.md)
+documents for the underlying method: autonomous investigation, bounded changes,
+empirical feedback, durable checkpoints and reusable lessons.
+
+The industrial-age analogy explains that method. The Factory's
+software architecture also uses the Abstract Factory pattern; the article's
+analogy should focus on the working process rather than that implementation.
+
+REA supplies analysis evidence to an agent. The historical projects also used
+project-specific analysis bridges and tooling. The article should attribute
+historical work to those projects and explain where REA fits in the general
+workflow, without relabeling every historical tool result as REA output.
+
+## Collection and counting
+
+Public history was retrieved with `gh api`, including all pages of each
+repository's default-branch commits, repository metadata and release records.
+Pinned progress documents were read from the corresponding Git objects in the
+local project checkouts. Every milestone SHA below appears in the public
+GitHub history. No local-only revision or uncommitted change supplies a claim.
+
+Elapsed days are the difference between public commit **committer timestamps**,
+in UTC, divided by 86,400 and rounded to two decimal places. The article uses
+rounded calendar intervals. They measure recorded milestone spacing, not
+compute time, person-hours or a controlled speedup benchmark.
+
+TH08's start is the independent continuation at `001bf3e`, not its inherited
+2025 initial commit or its hosting creation date. Its imported GensokyoClub
+history ends at `7ad3792`; preserve that attribution and license. TH095 starts
+with zero confirmed authored functions, while using methods and infrastructure
+from earlier projects. TH04 uses ReC98 as a credited source of existing work.
+
+The denominators change as candidates are classified and boundaries reviewed.
+Read each pinned progress document's definitions. Avoid a cross-game function
+rate or an executable-wide completion percentage.
+
+## Public history snapshot
+
+This is the initial 10 October research snapshot. The later TH04 phase update
+below uses its own pinned checkpoint; it does not change these earlier counts.
+
+| Project | Default-branch commits | Commits whose subject begins `gpt-web:` | Public head |
+| ------- | ---------------------: | --------------------------------------: | ------------------------------------------ |
+| TH08 | 1,117 | 400 | `d2a00f4acb4a13916e07a00bd13f714961714628` |
+| TH095 | 713 | 450 | `6084b36cfa90ac1720f0cba36caec3ee56fc5e1f` |
+| TH04 | 957 | 623 | `33b80522f03565022dbfe4d5c8a88bdc57f566ae` |
+
+TH08 has 696 reachable continuation commits after the imported boundary;
+its total includes the upstream history. Subject prefixes identify a recorded
+workflow convention, not the number of sessions, experiments or verified
+functions. Commit counts remain supporting research rather than article
+headlines.
+
+## Milestone records
+
+### TH08
+
+Start: [001bf3e9](https://github.com/N0zoM1z0/th08/commit/001bf3e9c91cc35b79c7a0e36b3565b86f494362) at 2026-08-13T02:23:48Z.
+
+| Checkpoint | UTC timestamp | Elapsed days | Recorded milestone |
+| -------------------------------------------------------------------------------------------- | -------------------- | -----------: | ---------------------------------------------------------------- |
+| [15caba73](https://github.com/N0zoM1z0/th08/commit/15caba73df6735fc44eac4783bc61b8d098ea830) | 2026-08-19T12:19:33Z | 6.41 | Restore clean authored build baseline |
+| [da07b21c](https://github.com/N0zoM1z0/th08/commit/da07b21c8cb63230142b0a66228d6aac66555f3f) | 2026-08-24T10:43:33Z | 11.35 | Add playable Linux reconstruction port |
+| [529eb1a3](https://github.com/N0zoM1z0/th08/commit/529eb1a3afa05ae36467731eaccb12e3e50e132f) | 2026-08-26T05:12:23Z | 13.12 | Link the playable TH08 Web edition |
+| [5d314e21](https://github.com/N0zoM1z0/th08/commit/5d314e21ec725a0de7a38b7c735f683ad358e79f) | 2026-08-30T10:30:16Z | 17.34 | docs: publish native Linux 64-bit release |
+| [bd1bae44](https://github.com/N0zoM1z0/th08/commit/bd1bae444803adfa1fc865bfe64415e9951945e6) | 2026-08-26T15:59:04Z | 13.57 | semantic: recover GameManager state and exact title registration |
+
+### TH095
+
+Start: [8c31e6f8](https://github.com/N0zoM1z0/th095/commit/8c31e6f892d945423b7f7b439747a2bd915f82d8) at 2026-08-29T07:49:20Z.
+
+| Checkpoint | UTC timestamp | Elapsed days | Recorded milestone |
+| --------------------------------------------------------------------------------------------- | -------------------- | -----------: | ---------------------------------------- |
+| [0fa857fb](https://github.com/N0zoM1z0/th095/commit/0fa857fb2b2ec6237732e4d6be8d0a28069c9330) | 2026-09-07T05:23:52Z | 8.90 | gpt-web: match enemy movement exactly |
+| [74baddfc](https://github.com/N0zoM1z0/th095/commit/74baddfccb712d1311a4d6e7487a63cd1100452e) | 2026-09-08T09:19:45Z | 10.06 | Match final ANM x87 draw functions |
+| [33d46a0e](https://github.com/N0zoM1z0/th095/commit/33d46a0ee48f37060d973125a1a7632a40f8d998) | 2026-09-09T11:15:00Z | 11.14 | build: complete whole-program linkage |
+| [3442dcf2](https://github.com/N0zoM1z0/th095/commit/3442dcf29d9e0b5bef384a49bd0c9ad32a711826) | 2026-09-10T07:07:28Z | 11.97 | Mark the Windows reconstruction playable |
+
+### TH04
+
+Start: [f402cb30](https://github.com/N0zoM1z0/th04/commit/f402cb30b7e795ea4328b6de794e9eefc52c97b3) at 2026-09-06T09:54:05Z.
+
+| Checkpoint | UTC timestamp | Elapsed days | Recorded milestone |
+| -------------------------------------------------------------------------------------------- | -------------------- | -----------: | --------------------------------------------------------------- |
+| [1cd1a257](https://github.com/N0zoM1z0/th04/commit/1cd1a2577a3dbf4c17ac7a939faed5eeb81a6ebf) | 2026-09-26T21:16:40Z | 20.47 | gpt-web: accept MAIN SND_LOAD |
+| [4b6a57f8](https://github.com/N0zoM1z0/th04/commit/4b6a57f8c0625da07c38258b78f5f205aae92805) | 2026-10-02T07:44:53Z | 25.91 | gpt-6.1-sol: build four native products and restore DOS handoff |
+
+## What the milestones establish
+
+- **TH08:** the 19 August progress ledger records 1,107 / 1,107 authored
+ functions with source. The 24 August change introduces a playable Linux
+ reconstruction port; the 30 August GitHub release publishes the 64-bit Linux
+ product. The 26 August exact ledger records 1,106 / 1,107 accepted functions.
+- **TH095:** the initial ledger records zero confirmed/source-present authored
+ functions. The 7 September ledger records 697 source-present mappings;
+ 8 September records 696 exact functions out of 697 confirmed authored
+ functions. The 10 September checkpoint marks the Windows i386 reconstruction
+ playable and documents the exercised startup, gameplay and transition paths.
+ The remaining `Controller::GetInput` comparison is separate from playability.
+- **TH04:** the 26 September ledger records 661 exact functions out of 663
+ reviewed authored candidates across OP, MAIN, MAINE and ZUN. MAIN's byte
+ denominator covers its reviewed authored extents; other artifacts use decoded
+ comparisons. The 2 October checkpoint builds four standalone DOS products.
+ Runtime work and a modern x64 product have their own milestones; a four-product
+ build is not a whole-game runtime result.
+
+TH08's playable ports preceded later historical-platform runtime audits. The
+Factory's current recommended order reflects lessons learned across those
+projects; do not present that later recommendation as a process applied
+perfectly from the beginning.
+
+## Editorial scope
+
+The article follows the author's change in perspective: previous manual RE,
+starting agent-driven work in August 2026, TH08's continuation, TH095's reuse of
+experience, and TH04's application of the method to another architecture. The
+manual-RE background and August starting point are author-provided experience.
+The industrial-era conclusion and the interpretation of the two contribution
+models are the author's argument, supported by the described project record.
+They are not a measured agent-versus-human performance ratio.
+
+Engine APIs, compiler flags, addresses and ledger schemas remain in references.
+The full essay replaces the outline and removes its `noindex` directive; the
+generated sitemap includes the article. The author approved publication on
+10 October 2026, and the review labels were removed from the article and Blog hub.
+
+## GensokyoClub quotation and policy
+
+The public README was read with `gh api` on 10 October 2026. Its latest
+README-changing commit is
+[`e874b98e`](https://github.com/GensokyoClub/th08/blob/e874b98e210b3be1ed21c5e03d54b883eac5069c/README.md#important-notice),
+dated 6 September. The notice announces a decision effective 5 September and
+describes concerns about AI decompilations, ports, attribution and their effect
+on the maintainers. Its stated plan is private development until completion.
+
+The article quotes 23 words from the passage's opening clause, followed by an
+ellipsis; it paraphrases the remaining point about the psychological toll.
+The excerpt was checked against the retrieved original, with whitespace
+normalized for display. The linked notice supplies its context.
+
+The current public head is
+[`f345c7de`](https://github.com/GensokyoClub/th08/tree/f345c7dee07b849f2ff93a0029ab5ede126bf12c).
+Its [contribution policy](https://github.com/GensokyoClub/th08/blob/f345c7dee07b849f2ff93a0029ab5ede126bf12c/CODE_OF_CONDUCT.md)
+excludes PRs created primarily using AI. The article paraphrases this policy
+and discusses the different contribution models. It preserves GensokyoClub's
+credit, describes the author's own manual and agent workflows, and does not
+assign motives beyond what the public notice states.
+
+## TH04's current DOS and 64-bit phases
+
+The latest public main checkpoint checked for this revision is
+[`bb9faca5`](https://github.com/N0zoM1z0/th04/blob/bb9faca54e2eb3c0d6c757c75d59ac29d2cd5e93/docs/RE_HANDOFF.md),
+dated 10 October 2026. Its handoff records the standalone DOS products,
+repaired normal and invincible variants, and the user's manual Windows
+observations of complete Normal routes, endings and saves. It identifies
+resumed x64 work as the current phase on the separate `port/modern-64` branch.
+That branch was publicly present at `1ffc53a61e1e82409adb40c7658d4ee2f3a21916`
+when inspected with `gh`.
+
+The essay's first-person DOS testing statement describes those maintainer
+observations, also confirmed by the author in this revision request. It is
+not attributed to a new automated website test or a fresh runtime replay.
+The 64-bit port is described as in development. The earlier four-product build
+and function-count milestones retain their historical scopes.
diff --git a/website/figures.md b/website/figures.md
index 72a32ac61f..534d8ac24b 100644
--- a/website/figures.md
+++ b/website/figures.md
@@ -94,6 +94,14 @@ APK's selected Java and an explanatory summary.
APK, methods, source attribution and reconstruction checks. Keep the block
formula, default settings, slider outputs and reference cases consistent.
+## Touhou reconstruction essay
+
+The essay opens with a takeaway and develops its argument in prose. It has no
+opening flow figure. Project dates and the attributed README excerpt support
+the narrative; detailed milestone references are available in a native
+disclosure. Their provenance is documented in
+[evidence/touhou-reconstruction-method.md](evidence/touhou-reconstruction-method.md).
+
## Original image generation prompts
## Prompt 1
diff --git a/website/public/assets/styles.css b/website/public/assets/styles.css
index fb7093b9e7..542baf8920 100644
--- a/website/public/assets/styles.css
+++ b/website/public/assets/styles.css
@@ -163,10 +163,12 @@ section[id] {
white-space: nowrap;
}
.navigation a:hover,
-.navigation a[aria-current="page"] {
+.navigation a[aria-current="page"],
+.navigation a[aria-current="true"] {
color: var(--ink);
}
-.navigation a[aria-current="page"] {
+.navigation a[aria-current="page"],
+.navigation a[aria-current="true"] {
text-decoration: underline;
}
.navigation .nav-start {
@@ -178,7 +180,8 @@ section[id] {
gap: 8px;
}
.nav-github,
-.nav-discord {
+.nav-discord,
+.nav-x {
display: inline-flex;
align-items: center;
justify-content: center;
@@ -192,12 +195,14 @@ section[id] {
text-decoration: none;
}
.nav-github:hover,
-.nav-discord:hover {
+.nav-discord:hover,
+.nav-x:hover {
background: var(--soft);
color: var(--ink);
}
.nav-github svg,
-.nav-discord svg {
+.nav-discord svg,
+.nav-x svg {
display: block;
flex-shrink: 0;
}
@@ -675,6 +680,27 @@ pre code {
.article .stats dd {
color: var(--muted);
}
+.blog-takeaway {
+ max-width: 730px;
+ margin-top: 1.8rem;
+ padding-left: 1.25rem;
+ border-left: 2px solid var(--accent);
+}
+.blog-takeaway .eyebrow {
+ color: var(--accent);
+}
+.blog-takeaway p:last-child {
+ margin-bottom: 0;
+}
+.blog-article blockquote {
+ margin: 1.6rem 0;
+ padding-left: 1.25rem;
+ border-left: 2px solid var(--line);
+ font-size: 1.05rem;
+}
+.blog-article blockquote p {
+ color: var(--ink);
+}
.caption {
font-size: 0.875rem;
color: var(--muted);
@@ -1372,6 +1398,9 @@ pre code {
.home-expert-shortcut a {
white-space: nowrap;
}
+.home-expert-shortcut + .home-expert-shortcut {
+ margin-top: 0.5rem;
+}
@media (min-width: 1440px) {
.home-page-nav {
position: fixed;
@@ -2117,6 +2146,9 @@ pre code {
right: calc(1rem + env(safe-area-inset-right, 0px));
bottom: calc(1rem + env(safe-area-inset-bottom, 0px));
}
+ .site-footer {
+ padding-bottom: calc(2rem + 44px + env(safe-area-inset-bottom, 0px));
+ }
.container {
width: calc(100% - 2.5rem);
}
diff --git a/website/public/blog/index.html b/website/public/blog/index.html
new file mode 100644
index 0000000000..90bf245ad8
--- /dev/null
+++ b/website/public/blog/index.html
@@ -0,0 +1,174 @@
+
+
+
+ REA Blog — Thoughts on Reverse Engineering
+ Skip to content
+
+
+
+
Home / Blog
+
Blog.
+
Thoughts on reverse engineering.
+
+
+ Game reconstruction
+
+
The industrial revolution in reverse engineering
+
+ What manual RE and reconstructing TH08, TH095 and TH04 taught me
+ about agents, verification and knowledge that compounds across
+ projects.
+
+
Read the article
+
By N0zoM1z0 · 10 October 2026
+
+
+
+
+
+ ↑ Top
+
+
+
diff --git a/website/public/blog/touhou-reconstruction/index.html b/website/public/blog/touhou-reconstruction/index.html
new file mode 100644
index 0000000000..2afa3f1d93
--- /dev/null
+++ b/website/public/blog/touhou-reconstruction/index.html
@@ -0,0 +1,669 @@
+
+
+
+ Reverse Engineering’s Industrial Revolution — REA
+ Skip to content
+
+
+
+
+ Home / Blog / Touhou
+ reconstruction
+
+
Essay · Touhou reconstruction
+
The industrial revolution in reverse engineering.
+
By N0zoM1z0 · 10 October 2026
+
+
Takeaway
+
+ Agent-driven reverse engineering changes how the work can
+ accumulate.
+ An investigation can become a checked result, a failed experiment
+ can become a reusable test, and one project's experience can become
+ the next project's starting point.
+
+
+ Reconstructing TH08, TH095 and TH04 convinced me that RE is entering
+ its industrial age.
+
+
+
+
+ In this article
+
+
+
+
+ What changed in August 2026
+
+ Before this reconstruction work, I had done a lot of manual
+ reverse engineering. The familiar loop was to inspect
+ instructions, follow references, form an explanation, test it, and
+ revise it. Progress depended on how much of that loop I could
+ carry out myself, and how well I could keep the growing picture in
+ my head.
+
+
+ In August 2026, I began exploring agent-driven RE seriously. I
+ wanted to see how far an agent could go when it had access to the
+ program, ordinary development tools, and feedback from real
+ checks. Touhou reconstruction became the place where I could
+ pursue that question through a substantial project.
+
+
+ The early progress was startling. Tasks that I would have budgeted
+ around my own time and attention could keep moving while the agent
+ investigated, tried source formulations, compiled them, and
+ examined the result. My work increasingly involved choosing the
+ direction, improving the tools and checks, and deciding what the
+ accumulated evidence meant.
+
+
+ That changed which projects seemed reasonable to attempt. A large
+ executable began to look like a collection of investigations we
+ could organize and keep advancing. Recovering source, making it
+ readable, and eventually using it for a port became parts of an
+ achievable project.
+
+
+ The numbers first got my attention. The more interesting question
+ was what made that progress possible—and whether the process could
+ improve as it worked.
+
+
+
+ TH08: building on human work
+
+ TH08 , Imperishable
+ Night, began for me as a continuation of
+ GensokyoClub's reconstruction . Their public work already contained source, names, build
+ knowledge and a contribution history. It gave the continuation a
+ foundation. That foundation matters when discussing everything
+ that followed.
+
+
+ I preserved the imported history through the public 10 August
+ checkpoint. The independent continuation began on 13 August. By 19
+ August, the project's ledger recorded source for all 1,107
+ identified game functions. A playable Linux reconstruction port
+ was committed on 24 August, roughly eleven days after the
+ continuation began. A Web edition followed, then a native Linux
+ 64-bit release on 30 August.
+
+
+ Those dates made the change in pace tangible. What interested me
+ most was that the work extended beyond producing plausible C++. It
+ included comparisons, build repairs, recovery of meaningful names
+ and types, and the work needed to turn reconstructed pieces into a
+ program people could run.
+
+
+ This also exposed the importance of feedback. A function can look
+ correct and compile correctly while its place in the whole program
+ is wrong. Two functions can accidentally use separate copies of
+ state that should be shared. A port can hide a problem that
+ appears when the historical build runs. Each failure adds
+ something useful if we can reproduce it and keep the check.
+
+
+ Over time, more of the project lived outside any one conversation:
+ source, evidence, comparison scripts, regression tests, and
+ focused notes about what to do next. A fresh agent could inspect
+ that state and continue.
+ The repository was becoming the project's working
+ memory.
+
+
+ A successful batch gave us both recovered code and a better
+ environment for the next batch.
+
+
+
+ Two different ideas of reconstruction
+
+ There is a real disagreement around this work, and GensokyoClub's
+ public README expresses it directly. Its notice announces that
+ further development will happen privately until completion. One
+ passage reads:
+
+
+
+ The rise of grifters (AI decompilations and ports) in this space
+ taking from our work paints a bad image for future decompilation
+ efforts…
+
+
+
+ The notice also describes the psychological toll on the
+ maintainers. Their contribution policy excludes pull requests
+ produced primarily with AI. These are people putting their free
+ time into difficult work, and their published work helped make my
+ continuation possible. That effort deserves respect. The
+ disagreement I want to discuss concerns how reconstruction is done
+ and how contributions are evaluated.
+
+
+ In the manual RE workflow I knew, expertise and execution were
+ closely connected. The person developing an understanding of a
+ function also carried out much of the investigation and
+ reconstruction. Personal knowledge, careful authorship and trust
+ in contributors were central to the project.
+
+
+ Manual projects already preserve knowledge in code, build tools
+ and tests. What changed for me was having an agent use that
+ accumulated knowledge to investigate, implement and test the next
+ change.
+
+
+ My continuation takes an agent-led approach. Agents carry out the
+ new engineering; the human sets direction and release decisions.
+ Understanding has to be retained in source and evidence, while
+ comparisons, builds and runtime experiments test the results. A
+ contribution can be useful even when an agent performed most of
+ the work, provided its reasoning and result can be examined.
+
+
+ This creates a difficult transition for communities built around
+ the earlier model. Open source makes existing work available for
+ continuation, and agents can change the pace of that continuation
+ dramatically. Years of careful work may become the starting point
+ for something that moves much faster. Recognition, provenance and
+ the standards for accepting new work become more important in that
+ situation.
+
+
+ The industrial analogy helps me make sense of the disagreement. A
+ craft contains skill, judgment and accumulated knowledge. When
+ machinery changes how execution happens, those skills find new
+ roles in designing the process, detecting defects and deciding
+ what should be produced. Different communities can reasonably
+ choose different ways to organize that work.
+
+
+ My choice is to develop this continuation openly, preserve its
+ sources and history, and make the new work reviewable. The
+ technical question I want to pursue is how far agent-driven
+ reconstruction can go with strong evidence and a process that
+ keeps improving.
+
+
+ Source: GensokyoClub's
+ README notice , checked on 10 October 2026, and its
+ contribution policy . The quotation is a shortened excerpt. TH08's
+ credits and provenance
+ record the continuation boundary.
+
+
+
+ TH095: experience starts compounding
+
+ TH095 , Shoot the
+ Bullet, made the accumulated experience much easier to see. Its
+ new repository began on 29 August with zero confirmed game
+ functions in the reconstruction ledger. It already had a way to
+ work: identify the target, bring up analysis and comparison tools,
+ keep tasks manageable, preserve evidence, and leave checkpoints
+ another session could use.
+
+
+ By 7 September, all 697 identified game functions had source. By 8
+ September, 696 of them were accepted as exact comparisons. The
+ whole program linked on 9 September, and the Windows i386
+ reconstruction was marked playable on 10 September, about twelve
+ days after initialization.
+
+
+ I found this more exciting than the first project's speed. TH095
+ began with the process and experience developed during TH08 and
+ the other reconstruction work. We could approach a new target with
+ better questions, better checks and fewer assumptions left
+ implicit.
+
+
+ For example, matching individual functions had taught us to pay
+ attention to the assembled program early. Does the source build
+ together? Do different parts of the game use the same state? Does
+ the reconstructed executable survive the transitions we need to
+ exercise? Those questions became part of how the next project was
+ organized.
+
+
+ This is what I mean by knowledge compounding. A lesson that stays
+ in my head helps while I am present. A lesson retained as a test,
+ a script or a precise evidence note can help every later session.
+ A failure becomes valuable when the next agent can recognize it
+ without repeating the whole investigation.
+
+
+ The method becomes part of the starting material for the next
+ game.
+ As that material improves, we can spend more attention on what is
+ new about the target.
+
+
+ This compounds across people as well as projects. Someone joining
+ later can rerun a check, inspect the reason for a source choice,
+ or continue from a recorded unknown. The project can make use of
+ experience without requiring everyone to reconstruct its entire
+ history from scratch.
+
+
+
+ TH04: the workflow survives a different architecture
+
+ TH04 , Lotus Land
+ Story, took the work into the PC-98 DOS era. We moved from 32-bit
+ Windows games to a 16-bit environment with four cooperating
+ programs and hardware behavior that had to be understood on its
+ own terms. Existing
+ ReC98 work provided
+ valuable knowledge and source material here too.
+
+
+ The DOS reconstruction is now working, including complete Normal
+ routes, endings and saves from my manual testing. The current work
+ is a native 64-bit port. These are distinct phases: first
+ establish a working reconstruction on the original platform, then
+ carry its behavior into a new environment.
+
+
+ TH04's architecture changed the evidence we needed and the tools
+ used to collect it. It did not require inventing a different way
+ to organize the work. We could still isolate a question, inspect
+ the original behavior, reconstruct it, compare the result and
+ retain the lesson.
+
+
+ The difficult parts became concrete experiments. Which part of
+ startup establishes a piece of state? What happens when gameplay
+ hands control to an ending? Which hardware effect must the
+ reconstructed code preserve? With observations and a useful check,
+ an agent could work through these questions in the same iterative
+ way as it did on the Windows titles.
+
+
+ That is why TH04 matters to the larger argument. The process
+ generalized across a substantial change in platform. Architecture
+ determined the subject of the investigation; it did not become a
+ fundamental barrier to the workflow.
+
+
+ The 64-bit port now gives that work another use. We can revisit
+ the recovered behavior, separate it from old platform services and
+ test new implementations against what we learned. The
+ reconstruction supplies both source and a growing body of
+ knowledge about how the game should behave.
+
+
+ Project state as of 10 October 2026:
+ DOS reconstruction and manual testing
+ ·
+ 64-bit port . The port remains in development.
+
+
+
+ What makes this an industrial shift
+
+ These projects changed my view of the bottleneck. As more
+ investigation and implementation can be carried out by agents, the
+ working environment becomes increasingly important. The agent
+ needs access to the target, tools it can compose, feedback it can
+ act on, and memory that survives its current session.
+
+
+ REA fits into that picture by making reverse-engineering tools
+ available through interfaces an agent can use. It can inspect
+ code, follow relationships and return evidence for the next
+ question. Reconstruction projects add their own compilers,
+ comparisons and runtime checks. Together, these give an agent a
+ way to move from an explanation to an experiment with an
+ observable result.
+
+
+ Fast execution makes good feedback more valuable. A bad assumption
+ can produce a large amount of plausible source. A useful check can
+ reject that assumption early and point to the next investigation.
+ Improving a check improves many later decisions, which is why
+ tooling and verification deserve as much attention as the
+ recovered code.
+
+
+ The work also needs continuity. A conversation can end while the
+ campaign continues. Its result should leave the repository in a
+ state another agent can understand: what changed, what was tested,
+ what remains uncertain, and where to continue. This lets a large
+ reconstruction proceed through many small experiments without
+ relying on one uninterrupted session.
+
+
+ The
+ Touhou Reconstruction Factory
+ grew out of these experiences. It gives the projects shared ways
+ to retain evidence, run checks and carry useful lessons between
+ games. The point is to make the next investigation easier to
+ execute, easier to judge and easier to resume.
+
+
+ This is the industrial shift I see: expertise increasingly lives
+ in a working process that can be reused and improved. Human
+ judgment shapes the process. Agents supply much of the execution.
+ Verified results and accumulated lessons make the process more
+ capable over time.
+
+
+
+ The projects we can now consider
+
+ The pace matters because it changes the cost of beginning. When I
+ imagined doing every investigation manually, I also imagined
+ committing my own attention to each one. A sufficiently large
+ project could be interesting and still remain something I would
+ never realistically attempt.
+
+
+ An agent with access, feedback and project memory changes that
+ calculation. Recovering a game, making its source readable,
+ building a port, or exploring a mod can become a sustained
+ engineering effort with a practical way to keep going. The work
+ still involves difficult questions, but there is now a process for
+ giving those questions sustained attention.
+
+
+ TH08 showed me what an agent-led continuation could achieve on top
+ of human work. TH095 showed how the experience could carry
+ forward. TH04 showed that the method could cross into a very
+ different platform. Together, they made the industrial analogy
+ feel concrete.
+
+
+ I now look at an unfamiliar program and ask:
+ what access, feedback and accumulated knowledge would let an
+ agent work on this reliably?
+
+
+ That question opens up projects I would previously have left
+ alone. Each one can add code, checks and experience that make the
+ next more approachable. That is the future of reverse engineering
+ I want to keep exploring.
+
+
+
+
+ Project milestones and sources
+
+ The dates describe recorded project checkpoints, checked against
+ public GitHub history on 10 October 2026. Elapsed time is
+ calendar time between commits. Source presence, exact
+ comparisons, a build and a runtime result each name a different
+ milestone.
+
+
+
+
+ More articles ·
+ Explore a TH04 investigation
+
+
+
+
+
+
+ ↑ TopReverse Engineer Anything
- Showcases Showcases
+ Blog Get started Guides FAQ
@@ -107,6 +108,25 @@
/>
+
+
+
+
@@ -383,7 +403,8 @@
Source MIT license Community
+ >
Join the community Follow us on X
diff --git a/website/public/faq/index.html b/website/public/faq/index.html
index 60fe1183db..3ab51b9d39 100644
--- a/website/public/faq/index.html
+++ b/website/public/faq/index.html
@@ -63,6 +63,7 @@
>
Showcases
+ Blog
Get started
Guides
FAQ
@@ -107,6 +108,25 @@
/>
+
+
+
+
@@ -482,7 +502,8 @@
Source MIT license Community
+ >
Join the community Follow us on X
diff --git a/website/public/first-investigation/index.html b/website/public/first-investigation/index.html
index 349ece913b..0db427248a 100644
--- a/website/public/first-investigation/index.html
+++ b/website/public/first-investigation/index.html
@@ -62,7 +62,8 @@
>
Reverse Engineer Anything
- Showcases Showcases
+ Blog Get started Guides FAQ
@@ -106,6 +107,25 @@
/>
+
+
+
+
@@ -420,7 +440,8 @@ JS
Source MIT license Community
+ >
Join the community Follow us on X
diff --git a/website/public/get-started/index.html b/website/public/get-started/index.html
index b86987e9ed..cc1070c924 100644
--- a/website/public/get-started/index.html
+++ b/website/public/get-started/index.html
@@ -62,7 +62,8 @@
>
+ 
+
+**[Situs Web](https://rea.tools/) · [Panduan](https://rea.tools/guides/) · [Contoh Penggunaan](https://rea.tools/showcase/)**
+
+[Mulai Cepat](#mulai-cepat) · [Cara Kerja REA](#cara-kerja-rea) · [Hal yang Dapat Anda Analisis](#hal-yang-dapat-anda-analisis) · [Contoh Penggunaan](#contoh-penggunaan) · [FAQ](#faq) · [Dokumentasi](#dokumentasi)
+
+
+
+