feat(environment): compare an environment with probe captures of its application (#5699)
* chore: restack environment fidelity onto main Keep probe comparison, paging, and tied-create behavior on main after #5600. Capture parsing keeps field-path errors and records whether a probe kept request and response bodies. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(environment): keep served collection order with the fidelity checks The tied-create collection order lives with the checks that match encoded parents. The write simulation reads field checks from that attribution, and taint uses the same segment match. Co-authored-by: morluto <morluto@users.noreply.github.com> * fix(environment): translate an object's keys only when it is keyed by learned ids Differ.object mapped every key through the probe's id map, so a field named like a learned id was compared with another field, and a real rename could pass as a mapped key. Keys now translate only for an object keyed by learned ids as learning reads one: a key both sides share needs every key evidently an id, and otherwise each entry is an entity or each key evidently an id. Any other object compares field by field. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: morluto <morluto@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
This commit is contained in:
40 files changed
+4360
-117
No files matched your search
@@ -380,7 +380,7 @@ data-plane operations and no children. Served
|
||||
operations and one intake extension instead, and the catalog holds the read
|
||||
tools and only the write tools whose effect the capture confirmed: a read tool
|
||||
is described as answering from the state where the state backs it, and a
|
||||
served write tool by the effect it applies to the state and the body members it
|
||||
served write tool by the effect it applies to the state and the quoted body members it
|
||||
accepts.
|
||||
|
||||
## Serve from the state
|
||||
@@ -721,6 +721,69 @@ state is unavailable, and that neither is emulation and served
|
||||
content is untrusted data. It states that the recorded content belongs to the
|
||||
captured application and must not be published without the rights to it.
|
||||
|
||||
## Measure fidelity against the application
|
||||
|
||||
An environment agrees with its own capture by construction. What matters to an agent is how it answers the calls the
|
||||
capture did not make, what it shows after a write, and how it fails. `rea environment-compare` measures that against the
|
||||
application itself:
|
||||
|
||||
```sh
|
||||
rea environment-compare manifest.json --evidence capture.json \
|
||||
--probe probe-crud.json --probe probe-reads.json --json
|
||||
```
|
||||
|
||||
Each `--probe` is capture Evidence (`inspect-web-network-capture` of a HAR, or `capture-browser-scenario`) of one call
|
||||
sequence made against the live application, seeded as it was for the capture and reset before the sequence. The command
|
||||
makes no network call: whoever records the probes drives the application with its own credentials. For each probe it
|
||||
resets the environment's state, takes the requests in the order they were made (record order when any lacks a start time), matches each to the manifest tool of the
|
||||
same method whose path template fits with the most literal segments (a format-rule parameter must fit its format once
|
||||
percent-decoded). A path argument is the raw segment, which the environment decodes once.
|
||||
It answers the call as the served environment would, and compares that answer with the application's.
|
||||
|
||||
A re-seeded application holds the same entities under other ids, so the comparison maps them. Objects in the two answers
|
||||
to one call pair by a natural key, a string field unique on each side that both share, such as a name or a slug; their
|
||||
identifier fields (`id`, `uuid`, `guid`, and names ending in `_id`, `Id`, `ID`, `_uuid` or `Uuid`), and those of two single
|
||||
answers such as a create's, map the live value to the environment's, including a create that returns only the new id.
|
||||
Ids are learned only from an answer the environment
|
||||
gave that call itself, from its state or a recording of that very call, request line and body, never from another call's
|
||||
recording replayed in its place and never from a recording whose body the probe omitted. The answer's own `id`, `uuid` or `guid` also maps within the collection the call names, since
|
||||
two collections can reuse a live id, and an equal pair stays in that collection. A live id in a later request's path, query or body is translated before the
|
||||
environment answers, a path parameter by the collection the live path names (a nested collection keeps its parent values, each decoded once, and a literal route segment stays), except a query value that pages a list, such as `limit`
|
||||
or `page`. Each call is checked against the served tool's input schema first, as the MCP server checks it. After mapping, two clock values compare by format only (ISO-8601 strings, and epoch numbers under
|
||||
time-named fields), and two different values each side generated on its own compare equal: both UUIDs, or one length of
|
||||
eight or more with the same character classes and a digit. The report counts each kind of normalised value.
|
||||
|
||||
A step agrees when the statuses and the normalised bodies are equal. The report gives agreement overall, per probe and
|
||||
per tool, `agreement_ignoring_order` for steps that differ only in item order, and each step's differences, at most 20,
|
||||
with a JSON pointer and both values cut short:
|
||||
|
||||
| Category | Meaning |
|
||||
| --------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| `status` | The statuses differ |
|
||||
| `missing-field` | The application's object has a member the environment's lacks |
|
||||
| `extra-field` | The environment's object has a member the application's lacks |
|
||||
| `value` | Values differ, or one is an object or array and the other is not |
|
||||
| `length` | Arrays differ in length, or not every item pairs |
|
||||
| `order` | Paired items come in a different order |
|
||||
| `clock-format` | Both are clocks, in different formats |
|
||||
| `refused` | The environment refused the call |
|
||||
| `not-served` | The request's tool is one the environment does not offer, such as a withheld write |
|
||||
| `no-tool` | No manifest tool matches the request |
|
||||
| `unmapped-id` | The application answered with success and the environment did not, for a path id the comparison never mapped |
|
||||
|
||||
What a probe did not observe is not counted as the application's behaviour. A record without a usable request line,
|
||||
such as a redacted URL or a transaction the capture cannot attribute, is not replayed, nor is a request to another
|
||||
origin than the application's (`other-origin`: the probe's origin with the most JSON-like records, as the manifest
|
||||
chooses its own), and a request with no recorded
|
||||
answer is made, since it may have changed the application, but not compared: each probe lists them under `unobserved` with the reason, and the summary counts them. A live
|
||||
body the probe did not record compares by status only and is counted as `unobserved_bodies`, unless the answer never
|
||||
carries one (204, 205, 304, `HEAD`). A browser probe captured without `capture.network.response_body` is refused, as
|
||||
is one that makes POST, PUT or PATCH calls without `capture.network.request_body`.
|
||||
|
||||
Agreement covers only the calls the probes make, after the mapping the answers allow. Values compared by form, clocks
|
||||
and generated values, can still differ in content, and an equal answer after a write shows the state matched for that
|
||||
read, not that the write had no other effect.
|
||||
|
||||
## Compatibility verification
|
||||
|
||||
`npm run verify:agentenv:environment` is the hosted `real-agentenv` lane
|
||||
|
||||
+16
@@ -514,6 +514,22 @@ of the serve command's stopped document. See
|
||||
[agent environments](agent-environments.md#export-a-bundle) for the folder
|
||||
layout and what the smoke task checks.
|
||||
|
||||
## Compare an environment with its application
|
||||
|
||||
`rea environment-compare <manifest.json> --evidence <capture.json> --probe <probe.json> [--probe <probe.json> ...]`
|
||||
compares the environment the pair serves with probe captures of the application it came from. Each probe is capture
|
||||
Evidence of one call sequence recorded against the application after a reset; the environment's state is reset before
|
||||
each probe, every request to the application's origin is answered as the served environment answers it, and the
|
||||
answer is compared with the application's after ids are mapped and clocks and generated values are normalised. The command reads files only and
|
||||
makes no network call. It prints the report and exits `0`, or prints a structured `invalid_request` error and exits `1`
|
||||
for the same pair refusals as `environment-serve`, a probe that is not capture Evidence, or a browser probe captured
|
||||
without response bodies, or without the request bodies its writes need. See
|
||||
[agent environments](agent-environments.md#measure-fidelity-against-the-application) for the report.
|
||||
|
||||
```sh
|
||||
rea environment-compare manifest.json --evidence capture.json --probe probe.json --json
|
||||
```
|
||||
|
||||
## Output and exit status
|
||||
|
||||
`rea mcp doctor --json` checks production MCP startup and reports child logs in
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
# Plan
|
||||
|
||||
1. Export the capture parser from `environmentSource.ts` so a probe loads like a capture.
|
||||
2. `environmentCompareMatch.ts`: a probe request to a tool and its arguments.
|
||||
3. `environmentCompareIds.ts`: the id map, natural-key pairing, learning from two answers, translating arguments.
|
||||
4. `environmentCompareDiff.ts`: normalisation and categorised differences.
|
||||
5. `environmentCompare.ts`: per probe, reset the state, answer each request, compare and learn; the report.
|
||||
6. `rea environment-compare` in `src/cli/environmentCommands.ts`, documented in `docs/agent-environments.md` and
|
||||
`docs/cli.md`.
|
||||
7. Tests: matching, pairing, normalisation and categories as unit tests; the command over a fixture capture and probes.
|
||||
8. Verification on VMs: six applications brought up and seeded as for their capture, reset from a snapshot per sequence,
|
||||
driven by a script that records each sequence as a HAR; baseline reports per environment.
|
||||
@@ -0,0 +1,86 @@
|
||||
# Compare an environment with the application it came from
|
||||
|
||||
Issue: https://github.com/morluto/rea/issues/4933
|
||||
|
||||
## Outcome
|
||||
|
||||
An environment is only as useful as it is faithful: an agent trained or graded on it should meet the behaviour of the
|
||||
application it was captured from. REA checks an environment only against its own capture, so nothing says how far it
|
||||
drifts from the application on calls the capture did not make, after writes, or on errors.
|
||||
|
||||
`rea environment-compare` measures that. It takes an environment (a manifest and its capture Evidence) and one or more
|
||||
**probe captures**, each a recording of one call sequence made against the live application, and reports how many of the
|
||||
probe's calls the environment answers as the application did, which tools disagree and how.
|
||||
|
||||
The command only reads Evidence: it makes no network call. Whoever records the probes drives the application with its own
|
||||
credentials and resets it between sequences.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Inputs: the manifest Evidence, its capture Evidence (`--evidence`), and one or more probe capture Evidence files
|
||||
(`--probe`, repeatable), each `inspect-web-network-capture` or `capture-browser-scenario` Evidence of one sequence.
|
||||
- Each probe is one sequence run against a freshly reset application. Before each probe the environment's state is reset,
|
||||
so sequences are independent on both sides.
|
||||
- Requests are taken in the probe's order (start time, then ordinal; ordinal alone when any record lacks a start
|
||||
time, as state extraction orders them). A request is matched to the manifest tool of the
|
||||
same method whose path template it fits with the most literal segments; format-rule parameters must fit their format
|
||||
once percent-decoded, as replay reads them.
|
||||
Query values the tool takes and a JSON body become the tool's arguments. A path argument is the raw segment: the
|
||||
environment decodes it once, so a segment encoded twice is not read as a second decoding. Replay matches that raw
|
||||
argument against a recorded segment that equals it, equals its one decoding, or decodes once to it.
|
||||
- A record without a usable request line (a redacted URL, a transaction the capture cannot attribute, an unparseable
|
||||
URL or no method) is not replayed, nor is a request to another origin than the probe's application (`other-origin`),
|
||||
which is the origin with the most JSON-like records, as the manifest chooses its own. A request with no recorded
|
||||
answer is made, since it may have changed the application, but not compared. Each is listed per probe with its
|
||||
reason and left out of agreement.
|
||||
- A request no tool matches is reported `no-tool`; a matched tool the environment does not offer, such as a write it
|
||||
withholds, is reported `not-served`. Neither is answered.
|
||||
- Ids differ between a re-seeded application and the capture. The comparison maps them:
|
||||
- seeded entities pair across the two answers by a natural key: a field present on every element of both arrays whose
|
||||
string values are unique on each side and shared between them;
|
||||
- identifier fields (`id`, `uuid`, `guid`, and names ending in `_id`, `Id`, `ID`, `_uuid` or `Uuid`) of paired objects,
|
||||
and of the two answers to the same call, map the live value to the environment's value; a create's two answers map
|
||||
the new ids, including a response that is only that id at the write plan's identity location. The `id`, `uuid` or
|
||||
`guid` of the answer's own entities (at the read view's entity or list location, a write's response location, or
|
||||
else the root) also maps within the collection the call names, since two collections can reuse a live id for
|
||||
different entities. An equal pair is kept in that collection, so another collection's mapping of the same live id
|
||||
does not replace it. A collection's map is a bijection: the first pairing of either side wins. Outside identifier
|
||||
fields, a string maps when the environment's state holds it as an id, and the keys of two objects keyed by ids map
|
||||
when their values pair (keys that partly overlap only when they are digits, UUIDs or their entries' identities, and
|
||||
every value pairs);
|
||||
- ids are learned only from an answer the environment gave this call, from its state or from a recording of exactly
|
||||
this call, its whole request line and its request body: a recording of another call that replay answers with
|
||||
instead holds other entities. A tool that takes a body does not match a recording whose body the probe did not send;
|
||||
- path, query and body values equal to a mapped live id are translated before the environment answers, a path
|
||||
parameter by the collection it names, except query values that page a list (`limit`, `offset`, `page` and the
|
||||
like), which count items rather than name one. A literal route segment is not translated. A nested collection is
|
||||
the live path's concrete parent values, each parameter decoded once, so two parents do not share one child map and
|
||||
spellings replay treats as equal name one. A second encoding stays a different parent. Learning and translation use
|
||||
that same live path.
|
||||
A path value the map does not know, and the environment's state does not hold, is reported `unmapped-id`.
|
||||
- A call is checked against the served tool's input schema first, as the MCP server checks it; a call it refuses is
|
||||
reported `refused`. A browser probe keeps its request bodies; one captured without response bodies is refused, since
|
||||
only statuses would compare, and so is one that makes POST, PUT or PATCH calls without recorded request bodies, since
|
||||
those calls would be made empty.
|
||||
- Answers compare after normalisation: empty response text equals no body (a JSON empty string stays one); mapped ids
|
||||
compare equal, in scalar sets as well, an entity's own identity within the call's collection and any other id by the
|
||||
probe's mapping, and an entity's own identity both sides hold verbatim differs when its collection's mapping gives it
|
||||
another environment value (a reference keeps raw equality, as the probe's mapping cannot tell which collection it
|
||||
names); an object's members compare under the environment's mapped keys only when it is keyed by learned ids as
|
||||
learning reads one, and any other object compares field by field; two clock values (ISO-8601 strings, or epoch numbers under
|
||||
time-named fields) compare by format only; two different generated values (both UUIDs, or both of one length and the
|
||||
same character classes with a digit) that neither side maps compare equal and are counted. A body over the replay
|
||||
limit, text or binary, compares by status only, as does a live body the probe did not record, unless the answer has
|
||||
none by definition (204, 205, 304, or an answer to `HEAD`). Both are counted.
|
||||
- A step agrees when the statuses are equal and the normalised bodies are. Differences are categorised: `status`,
|
||||
`missing-field` (the application had it), `extra-field`, `value`, `length`, `order`, `clock-format`, `refused` (the
|
||||
environment refused the call), `not-served`, `no-tool` and `unmapped-id`, each with a JSON pointer and short values.
|
||||
- The report gives agreement overall, per probe and per tool, agreement ignoring item order beside it, the counts per
|
||||
category, the ids mapped and the values normalised, the records left uncompared, and each step's differences (at most
|
||||
20 per step).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No network access, no credentials, no reset of the application: probes are recorded outside REA.
|
||||
- No generation of call sequences; a later change generates them where the environment disagrees.
|
||||
- No change to how environments serve.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Tasks
|
||||
|
||||
- [x] Record the outcome, requirements and non-goals.
|
||||
- [x] Load probes as captures.
|
||||
- [x] Match probe requests to tools.
|
||||
- [x] Map ids and translate arguments.
|
||||
- [x] Normalise and categorise differences.
|
||||
- [x] Compare per probe and report.
|
||||
- [x] Add the command and its documentation.
|
||||
- [x] Add unit and command tests.
|
||||
- [x] Run the gates.
|
||||
- [x] Measure six environments against their live applications and record the results.
|
||||
@@ -0,0 +1,146 @@
|
||||
# Verification
|
||||
|
||||
Verified commit: 235b22f2 on the write-plan branch at 7056f5ff. Executed on macOS with Node v22.23.1 (the repository
|
||||
pins 24.18.0 in `.nvmrc`, which was not installed); the live applications ran on six disposable Linux VMs.
|
||||
|
||||
## Gates
|
||||
|
||||
- `npm run build:cached`, `npm run check:fast` (typecheck and lint), `npm run format:check`, `npx knip`,
|
||||
`npm run scan:todos`, `npm run docs:check`, `npm run verify:test-discovery` and `npm run mcp-catalog:check`: passed.
|
||||
- `npm run jscpd`: 3.46% duplicated lines over 3609 files, under the 3.5% gate.
|
||||
- `npx vitest run --project domain --project services`: 7487 passed, 5 pre-existing skips.
|
||||
- `npm run test:prepare`, then `npm run test:focused --` the environment-compare, export and serve CLI boundaries, the
|
||||
serve MCP boundary and the tool schema validity boundary: 5 files, 30 passed.
|
||||
|
||||
New tests: matching, id mapping and translation, pairing guards, normalisation and every category as unit tests
|
||||
(`environmentCompareIds`, `environmentCompareDiff`, `environmentCompare`, 19 tests), and the command through the built
|
||||
CLI (`environmentCompareCli`, 2 tests): a probe of the books application re-seeded under other ids agrees on every step
|
||||
with 2 ids mapped, and a probe that is not capture Evidence is refused with its index.
|
||||
|
||||
## Six environments against their live applications
|
||||
|
||||
Six published environments were compared with the application each was captured from: NetBox 4.7.3, Komga 1.29.1,
|
||||
Mealie 3.28.0, n8n 1.123.85, Shlink 5.1.7 and self-hosted Sentry 26.9.0, the same images, brought up and seeded by the
|
||||
same scripts as for their captures, on loopback. Each was snapshotted after seeding and restored from that snapshot
|
||||
before every sequence (7 to 80 s), so every sequence met the same application.
|
||||
|
||||
A driver outside REA ran three kinds of sequence per application, each starting with the capture's read-only prefix:
|
||||
the capture re-run in order; per served write type, a list, create, read, update, read, delete, read and list, then a
|
||||
second create with changed free text; and a read of every seeded entity through its item tool. It translated the
|
||||
environment's ids to the application's as it learned them, signed in with the application's own credentials, and wrote
|
||||
each sequence as a HAR without headers or cookies, which `inspect-web-network-capture` turned into probe Evidence. The
|
||||
environments' manifests use `prefix-variance`, so the comparison ran on a local build of this branch with that rule's
|
||||
branch merged.
|
||||
|
||||
| Environment | Steps | Agreement | Ignoring order | Against its own capture |
|
||||
| ----------- | ----: | --------: | -------------: | ----------------------: |
|
||||
| NetBox | 370 | 99.2% | 99.2% | 100% |
|
||||
| Komga | 356 | 93.8% | 93.8% | 93.1% |
|
||||
| Mealie | 301 | 96.0% | 97.0% | 97.1% |
|
||||
| Shlink | 133 | 89.5% | 92.5% | 92.4% |
|
||||
| n8n | 202 | 74.8% | 87.6% | 87.5% |
|
||||
| Sentry | 230 | 23.9% | 46.5% | 84.7% |
|
||||
|
||||
The last column compares each environment with its own capture as the probe: what it cannot reproduce even of the
|
||||
session it was built from. The disagreements fall into four groups.
|
||||
|
||||
- Environment gaps, the ones to fix:
|
||||
- write tools the environment withholds, and the reads after them: readlists and collections (Komga), recipes (Mealie), short URLs
|
||||
(Shlink), workflow and tag writes (n8n);
|
||||
- preconditions: n8n refuses to delete a workflow that is not archived and the environment deletes it;
|
||||
- lists: a create does not appear in a list that keeps its recorded members (Komga), a filtered list is answered
|
||||
without the filter (Shlink), a replayed search ignores its body (Komga);
|
||||
- a domain-scoped short URL answered without its `domain` parameter, where Shlink answers 404;
|
||||
- state rendering: a project field two recorded views disagree on is rendered with one value, and an organisation
|
||||
read whose shape depends on a query parameter is answered the same either way (Sentry);
|
||||
- a tag read refused for want of the fields its item view always carried (Mealie);
|
||||
- time: per-hour event counts move with the current time, while the environment's clock is frozen at the capture
|
||||
(Sentry);
|
||||
- a field the capture's scrubbing replaced, answered as `"<scrubbed>"` (Sentry).
|
||||
- The application differing from itself between seedings: Sentry returns permission lists in no fixed order and
|
||||
reseeding changed release first-event times, feature lists and profile flags; n8n's executions and NetBox's running
|
||||
worker count differ from the captured instance.
|
||||
- The probe observing itself: Komga and Sentry report the driver's own sign-in user agent.
|
||||
- Driver gaps: ids the driver never mapped (Mealie units, Sentry events by hash) were sent as the environment's ids.
|
||||
|
||||
## What the runs fixed in the comparison
|
||||
|
||||
Each live run found a way the comparison itself was wrong, fixed with a test before the next run: requests are matched
|
||||
to tools in the environment's terms, so a tool whose path holds a seeded id literally matches; ids are learned outside
|
||||
identifier fields only from strings, after NetBox's worker count of 1 mapped a live 0 to entity 1; items paired only by
|
||||
position teach ids only when no key is shared and the lengths agree, after a filtered Shlink list mapped one short URL to
|
||||
another; objects keyed by ids pair their entries (n8n data tables); reordered scalar sets differ in order only and epoch
|
||||
numbers in arrays are clocks (Sentry); and bodies over the replay limit compare by status.
|
||||
|
||||
## After the review round
|
||||
|
||||
The branch merged the write-plan branch at b6c3b168 (four serving and gate fixes its review found), then changed the
|
||||
comparison in nine places: ids are learned only from answers to the call itself and also per collection, paging values
|
||||
are never translated, calls go through the served tools' input schemas, browser probes keep their request bodies, empty
|
||||
bodies and large binary bodies compare as the server answers them, scalar sets compare after mapping, and id-keyed
|
||||
objects pair when some keys overlap. Each new test fails on the previous head. The gates above passed again (domain and
|
||||
services 7504 passed with 5 pre-existing skips; the focused boundaries 80; conformance; `jscpd` 3.46%). Re-run on the
|
||||
same probes with this branch and the template rule's merged, the six environments compare exactly as in the table above,
|
||||
step for step, with the same ids mapped and the same values normalised.
|
||||
|
||||
A second round, on Greptile's review of those changes, learned ids from a replay only when the whole recorded line and
|
||||
request body match the call, compared an entity's own identity within the call's collection and other ids by the probe's
|
||||
mapping, read overlapping keys as ids only when they evidently are ids, and kept a JSON empty string apart from empty
|
||||
text. Its four tests fail on the previous head; the six environments again compare exactly as in the table.
|
||||
|
||||
A third round located the answer's own entities where the served environment does, at the read view's entity or list
|
||||
pointer or the write's response pointer, so an entity inside an envelope keeps its collection's mapping. Its test fails
|
||||
on the previous head; the gates passed again (domain and services 7509, boundaries 80, conformance, `jscpd` 3.46%) and
|
||||
the six environments again compare exactly as in the table.
|
||||
|
||||
A fourth round, on Greptile's next review, kept what a probe did not observe out of the agreement figures. Records
|
||||
without a usable request line, and requests without a recorded answer, are listed per probe with the reason and left
|
||||
uncompared. A live body the probe did not record compares by status only and is counted. A browser probe captured
|
||||
without response bodies is refused. A format-rule path parameter fits by its decoded value, as replay reads it. The
|
||||
branch also merged the write-plan branch at 1d1c4787, whose gate no longer confirms a write by a read of a state that a
|
||||
later write replaced. Each new test fails on the previous head. The gates passed again (domain and services
|
||||
7647, 5 pre-existing skips; the compare, export, serve and analysis-view boundaries 69; conformance 279; `jscpd` 3.44%).
|
||||
|
||||
On the same probes, the six environments have no unobserved records or bodies, and five compare exactly as before.
|
||||
Komga now withholds its collection create and patch. The only read the gate could check them by is the 404 after the
|
||||
collection is deleted, because the read after the patch is the patch's own observation. Its agreement falls from
|
||||
96.9% to 93.8%: the three creates and two patches the probes make are not served, and the six reads and deletes of
|
||||
those collections that follow fail. Against its own capture it falls from 96.6% to 93.1%. The table shows the new
|
||||
figures.
|
||||
|
||||
A fifth round merged the write-plan branch at dc5e2aad, which weighs an update's derived changes when a later write
|
||||
replaces it and resets collection spellings on reset or load. It also makes a request whose answer the probe lost,
|
||||
still uncompared, so that a later read sees its effect: a lost delete followed by a 404 now agrees. Its test fails on
|
||||
the previous head. The six environments compare as in the table.
|
||||
|
||||
A sixth round merged the write-plan branch at fe8ab584, whose gate erases the fields later writes and recorded
|
||||
outcomes overwrite from earlier write effects. The six environments again compare exactly as in the table.
|
||||
|
||||
A seventh round merged the write-plan branch at 3580d85c and main d8a9673f. That brought the gate's fix for overwritten
|
||||
document writes, which Greptile raised here against the write-plan code, among others. It also fixed two findings of
|
||||
its own:
|
||||
|
||||
- a probe with any undated record keeps its recorded order, as state extraction does;
|
||||
- a browser probe that makes POST, PUT or PATCH calls without recorded request bodies is refused.
|
||||
|
||||
Both tests fail on the previous head. The six environments compare exactly as in the table, on the probes and against
|
||||
their own captures.
|
||||
|
||||
An eighth round merged the write-plan branch at 43d98ea9 and 264ce410. They bring the gate's fixes for three findings
|
||||
Greptile raised here against the write-plan code: a passing read that shows nothing of a write leaves it pending, an
|
||||
update or delete finds its entity as an item read does, and a built document plan writes later calls at its own
|
||||
object. It also fixed one finding of its own: a request to another origin than the probe's application is neither
|
||||
made nor compared, so another server's `DELETE /api/books/2` no longer deletes the environment's book. Its test fails
|
||||
on the previous head. The six environments compare exactly as in the table, on the probes and against their own
|
||||
captures, with no unobserved records.
|
||||
|
||||
It then merged the write-plan branch at 0ae64400 and 6257de93: an update's identity member never takes a rule, a page
|
||||
size of zero bounds nothing, and the paging rules, with this branch's `isPagingParameter`, move to
|
||||
`environmentStatePaging.ts`. The six environments again compare exactly as in the table.
|
||||
|
||||
A ninth round merged the write-plan branch at 466ed8b4, which fixes two findings Greptile raised here against its
|
||||
code (entity fields that look like pointers match by name; a zero page size bounds nothing only where a recording
|
||||
listed items under it) among others. The six environments compare exactly as in the table, on the probes and against
|
||||
their own captures. It then merged 7cef2fb2, whose fixes settle three more findings Greptile raised here against that branch's code:
|
||||
zero page sizes erasing another bound, partial write responses taken for removals, and interference timed by a tool's
|
||||
call elsewhere. The six environments again compare exactly as in the table.
|
||||
@@ -137,8 +137,9 @@ response, with the entity re-rendered when the response carries it.
|
||||
**Documents.** A document write merges the body's members (replaces them, for
|
||||
`PUT`) into the document object that holds most of them: the root or one
|
||||
member. Later calls write at the exemplar's object. Document reads answer only on their recorded line, so a write is
|
||||
served only at the path the state holds the document at, and the gate treats a
|
||||
recorded write at another path as an unmodelled write. A recorded write whose
|
||||
served only at the path the state holds the document at, each segment verbatim
|
||||
or either spelling once percent-decoded, and the gate treats a recorded write
|
||||
at another path as an unmodelled write. A recorded write whose
|
||||
body serving would refuse (a member of another JSON type than the document
|
||||
holds) is `unexplained-change`.
|
||||
|
||||
@@ -174,7 +175,10 @@ entity (below): its call cannot be checked, so it adds nothing to a built plan.
|
||||
A later create whose generated
|
||||
values take another form than serving gives them (a sequence value out of
|
||||
order, a clock in another format, a non-UUID unique value) makes the plan
|
||||
`unexplained-change`; integer identities are left to the id mapping. An update or
|
||||
`unexplained-change`; integer identities are left to the id mapping. A later
|
||||
create outside the collection a tied plan permits is unmodelled, as serving
|
||||
refuses that call, so a read of what it recorded does not confirm the plan. The
|
||||
gate picks that collection among equal spellings in serving's order (below). An update or
|
||||
delete names its entity as an item read does (below). A later update that does
|
||||
not answer with its entity is observed only by a read whose view shows every
|
||||
clock the plan learned, since serving stamps them; without one it is unmodelled.
|
||||
@@ -205,12 +209,12 @@ of a write.
|
||||
from the state:
|
||||
|
||||
- an item read of a state-backed item view, of the entity serving would name:
|
||||
in a held collection the read's path matches, each segment verbatim or once
|
||||
percent-decoded and verbatim spellings first, the id percent-decoded once,
|
||||
then as written;
|
||||
in a held collection the read's path matches, each segment verbatim or either
|
||||
spelling once percent-decoded and verbatim spellings first, the id
|
||||
percent-decoded once, then as written;
|
||||
- a list or document read on one of its view's checked request lines,
|
||||
chosen as replay chooses the line for the read's call: each path parameter
|
||||
the read's value verbatim or once percent-decoded, preferring verbatim
|
||||
the read's value verbatim or either spelling once percent-decoded, preferring verbatim
|
||||
spellings, then recorded order, and the query values decoded, in any order.
|
||||
A read with no query on a whole list falls back, as serving does, on a
|
||||
checked line that only pages it. A line whose query repeats a name with
|
||||
@@ -221,7 +225,10 @@ from the state:
|
||||
of zero is no bound on a line whose recording listed items under it, while its
|
||||
other sizes and range still bound the page) of a
|
||||
whole list checks nothing once the collection holds more entities than the
|
||||
page, which serving then answers from its recording.
|
||||
page, which serving then answers from its recording. A line without proof of
|
||||
whole membership lists its recorded members still held, unless a learned
|
||||
query evaluates the read over the state. An empty later page therefore does
|
||||
not contradict a create outside that page.
|
||||
- A read made while a modelled write to its entity or collection was still
|
||||
running, starting before that write ended, checks nothing: it may show the
|
||||
state before or after the write. Nor does such a read show an unknown entity
|
||||
@@ -336,7 +343,8 @@ entity; a collection keeps its spelling after its last entity is deleted. A
|
||||
reset or load knows the spellings of the Evidence's state, of the state it
|
||||
installs and of the type's checked list lines (so a collection the capture
|
||||
listed empty keeps its spelling), never those of a state it replaced. When two
|
||||
spellings fit a call equally, the one holding entities wins. `data/add` refuses a document that `data/get` would write back past the
|
||||
spellings fit a call equally, the one holding entities wins, the earlier first
|
||||
entity in state order among held ones and the earlier known among emptied ones. `data/add` refuses a document that `data/get` would write back past the
|
||||
32 MiB intake limit once it adds the members a document may omit, such as
|
||||
`types` and the state Evidence id. A derived state `data/get` would write
|
||||
past that limit is not served: the environment serves replay and reports the
|
||||
@@ -365,11 +373,15 @@ answers with the tool's latest recorded 400, 404 or 410 response, or 2xx JSON
|
||||
them, so the read gate's `membership: "all"` also covers a line that only pages
|
||||
its list (`limit`, `offset`, `page`, `per_page`, `perpage`, `page_size`,
|
||||
`pagesize`, `size`, `start`, `_start`, `_end`) when its recording listed the
|
||||
whole collection. A list call with no query argument answers from such a line
|
||||
of an `all` view of the collection its path arguments name, with
|
||||
whole collection at a proved zero start. Positive `page` values do not prove
|
||||
the beginning, because their base is unknown. Whole membership is per line:
|
||||
`all_record_ordinals` can name whole lines within a recorded-membership view.
|
||||
A learned query can filter, search, sort and page the state; otherwise a later
|
||||
page lists its recorded members still held. A list call with no query argument
|
||||
answers from a whole line of the collection its path arguments name, with
|
||||
`replay.exact_match` false; a collection with no such line answers from its
|
||||
recording. A call that asks for a page
|
||||
still answers from its recording.
|
||||
recording. A call that asks for a page evaluates a learned query when one
|
||||
covers its arguments; otherwise it keeps the recorded page members.
|
||||
|
||||
**Provenance.** Write answers carry `state: {source: "state", view: "create" |
|
||||
"update" | "delete", type, collection, id}`, or `{source: "state", view:
|
||||
|
||||
@@ -0,0 +1,723 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import {
|
||||
analysisViewCaptureEvidence,
|
||||
environmentManifestCaptureFixture,
|
||||
timedCaptureEvidence,
|
||||
type TimedCaptureRow,
|
||||
} from "../../../tests/fixtures/analysisView.js";
|
||||
import {
|
||||
environmentServeSource,
|
||||
environmentStateServeFixture,
|
||||
} from "../../../tests/fixtures/environmentServe.js";
|
||||
import {
|
||||
BOOK_NOT_FOUND,
|
||||
BOOK_ROWS,
|
||||
FIRST_BOOK,
|
||||
SECOND_BOOK,
|
||||
book,
|
||||
renamedBook,
|
||||
} from "../../../tests/fixtures/environmentStateWrites.js";
|
||||
import {
|
||||
scenarioCaptureEvidence,
|
||||
scenarioCaptureFixture,
|
||||
scenarioHeaderOnlyCaptureEvidence,
|
||||
} from "../../../tests/fixtures/browserScenarioCapture.js";
|
||||
import { compareEnvironment } from "./environmentCompare.js";
|
||||
import {
|
||||
environmentArguments,
|
||||
IdMap,
|
||||
translatePath,
|
||||
} from "./environmentCompareIds.js";
|
||||
import {
|
||||
collectionScope,
|
||||
matchProbeRequest,
|
||||
parameterScopes,
|
||||
} from "./environmentCompareMatch.js";
|
||||
import type { EnvironmentManifestTool } from "./environmentSource.js";
|
||||
import { isCut } from "./environmentCompareProbe.js";
|
||||
import { parseCaptureEvidence } from "./environmentSource.js";
|
||||
|
||||
const fixture = environmentStateServeFixture(timedCaptureEvidence(BOOK_ROWS));
|
||||
const source = environmentServeSource(fixture);
|
||||
|
||||
/** The books application re-seeded: the same books under other ids and creation times. */
|
||||
const LIVE_FIRST = book(101, "First Book", "2026-10-20T09:00:00Z");
|
||||
const LIVE_SECOND = book(102, "Second Book", "2026-10-20T10:00:00Z");
|
||||
const LIVE_FOURTH = book(103, "Fourth Book", "2026-10-20T11:00:00Z");
|
||||
const LIVE_RENAMED = renamedBook(LIVE_FIRST, "Renamed Book", 5);
|
||||
const ANN = { id: 1, name: "Ann" };
|
||||
const BOB = { id: 2, name: "Bob" };
|
||||
|
||||
const compare = (...probes: (readonly TimedCaptureRow[])[]) => {
|
||||
const report = compareEnvironment({
|
||||
source,
|
||||
captureValue: fixture.capture,
|
||||
probes: probes.map((rows, index) => ({
|
||||
label: `probe-${String(index + 1)}`,
|
||||
value: timedCaptureEvidence(rows),
|
||||
})),
|
||||
operation: "environment-compare",
|
||||
});
|
||||
if (!report.ok) throw report.error;
|
||||
return report.value;
|
||||
};
|
||||
|
||||
describe("matchProbeRequest", () => {
|
||||
it("prefers literal segments, requires format-rule parameters to fit, and maps query and body arguments", () => {
|
||||
const item = matchProbeRequest(source.manifest, {
|
||||
method: "GET",
|
||||
path: "/api/books/7",
|
||||
query: "?expand=all",
|
||||
body: undefined,
|
||||
});
|
||||
expect(item?.tool.path_template).toBe("/api/books/{numeric}");
|
||||
expect(item?.args).toEqual({ id: "7" });
|
||||
expect(item?.ignoredQuery).toEqual(["expand"]);
|
||||
const create = matchProbeRequest(source.manifest, {
|
||||
method: "POST",
|
||||
path: "/api/books",
|
||||
query: null,
|
||||
body: { name: "X" },
|
||||
});
|
||||
expect(create?.args).toEqual({ body: { name: "X" } });
|
||||
expect(
|
||||
matchProbeRequest(source.manifest, {
|
||||
method: "GET",
|
||||
path: "/api/books/x",
|
||||
query: null,
|
||||
body: undefined,
|
||||
}),
|
||||
).toBeUndefined();
|
||||
});
|
||||
|
||||
it("fits a format-rule parameter by its decoded value and passes the raw segment", () => {
|
||||
const item = matchProbeRequest(source.manifest, {
|
||||
method: "GET",
|
||||
path: "/api/books/%31",
|
||||
query: null,
|
||||
body: undefined,
|
||||
});
|
||||
expect(item?.tool.path_template).toBe("/api/books/{numeric}");
|
||||
// The environment decodes this once, as replay decodes a recorded segment once.
|
||||
expect(item?.args).toEqual({ id: "%31" });
|
||||
});
|
||||
});
|
||||
|
||||
const nestedUsers = {
|
||||
path_template: "/api/orgs/{numeric}/users/{numeric}",
|
||||
params: [
|
||||
{ index: 2, name: "org_id", rule: "numeric" },
|
||||
{ index: 4, name: "id", rule: "numeric" },
|
||||
],
|
||||
} as EnvironmentManifestTool;
|
||||
|
||||
describe("collections under different parents", () => {
|
||||
it("names each parent's collection from the live path, and does not rename a literal segment", () => {
|
||||
const orgOne = ["api", "orgs", "1", "users", "110"];
|
||||
const orgTwo = ["api", "orgs", "2", "users", "110"];
|
||||
expect(collectionScope(nestedUsers, orgOne)).toBe("/api/orgs/1/users");
|
||||
expect(collectionScope(nestedUsers, orgTwo)).toBe("/api/orgs/2/users");
|
||||
expect(
|
||||
collectionScope(nestedUsers, ["api", "orgs", "", "users", "110"]),
|
||||
).toBe("/api/orgs/{}/users");
|
||||
const map = new IdMap();
|
||||
map.learn(110, 10, "/api/orgs/1/users");
|
||||
map.learn(110, 20, "/api/orgs/2/users");
|
||||
map.learn("users", "stored-tag");
|
||||
const scopes = parameterScopes(nestedUsers, orgOne);
|
||||
// Template indexes of api, orgs and users; the two parameters are not literals.
|
||||
const literals = new Set([0, 1, 3]);
|
||||
expect(
|
||||
translatePath(
|
||||
"/api/orgs/1/users/110",
|
||||
map,
|
||||
(index) => scopes.get(index),
|
||||
(index) => literals.has(index),
|
||||
),
|
||||
).toBe("/api/orgs/1/users/10");
|
||||
expect(map.env(110, "/api/orgs/2/users")).toBe(20);
|
||||
});
|
||||
|
||||
it("shares a child map when the same parent is spelled with one encoding", () => {
|
||||
const encoded = ["api", "orgs", "%32", "users", "101"];
|
||||
const plain = ["api", "orgs", "2", "users", "101"];
|
||||
const twice = ["api", "orgs", "%2532", "users", "101"];
|
||||
expect(collectionScope(nestedUsers, encoded)).toBe("/api/orgs/2/users");
|
||||
expect(collectionScope(nestedUsers, plain)).toBe(
|
||||
collectionScope(nestedUsers, encoded),
|
||||
);
|
||||
// Decoding again would make this parent the same as `2`. Encoding the once-decoded segment keeps it distinct.
|
||||
expect(collectionScope(nestedUsers, twice)).toBe("/api/orgs/%2532/users");
|
||||
const map = new IdMap();
|
||||
map.learn(101, 10, "/api/orgs/1/users");
|
||||
map.learn(101, 20, collectionScope(nestedUsers, encoded));
|
||||
const literals = new Set([0, 1, 3]);
|
||||
const translate = (segments: readonly string[], path: string) => {
|
||||
const scopes = parameterScopes(nestedUsers, segments);
|
||||
return translatePath(
|
||||
path,
|
||||
map,
|
||||
(index) => scopes.get(index),
|
||||
(index) => literals.has(index),
|
||||
);
|
||||
};
|
||||
// Org 1 taught 101 → 10 globally. The child learned under `%32` is the one `/api/orgs/2/users/101` names.
|
||||
expect(translate(plain, "/api/orgs/2/users/101")).toBe(
|
||||
"/api/orgs/2/users/20",
|
||||
);
|
||||
expect(translate(twice, "/api/orgs/%2532/users/101")).toBe(
|
||||
"/api/orgs/%2532/users/10",
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps child ids apart when a decoded parent contains slashes", () => {
|
||||
const nested = {
|
||||
path_template:
|
||||
"/api/groups/uid/{uid}/folders/uid/{uid_2}/users/{numeric}",
|
||||
params: [
|
||||
{ index: 3, name: "uid", rule: "after-uid-literal" },
|
||||
{ index: 6, name: "uid_2", rule: "after-uid-literal" },
|
||||
{ index: 8, name: "id", rule: "numeric" },
|
||||
],
|
||||
} as EnvironmentManifestTool;
|
||||
const first = [
|
||||
"api",
|
||||
"groups",
|
||||
"uid",
|
||||
"a%2Ffolders%2Fuid%2Fb",
|
||||
"folders",
|
||||
"uid",
|
||||
"c",
|
||||
"users",
|
||||
"101",
|
||||
];
|
||||
const second = [
|
||||
"api",
|
||||
"groups",
|
||||
"uid",
|
||||
"a",
|
||||
"folders",
|
||||
"uid",
|
||||
"b%2Ffolders%2Fuid%2Fc",
|
||||
"users",
|
||||
"101",
|
||||
];
|
||||
const firstScope =
|
||||
"/api/groups/uid/a%2Ffolders%2Fuid%2Fb/folders/uid/c/users";
|
||||
const secondScope =
|
||||
"/api/groups/uid/a/folders/uid/b%2Ffolders%2Fuid%2Fc/users";
|
||||
expect(collectionScope(nested, first)).toBe(firstScope);
|
||||
expect(collectionScope(nested, second)).toBe(secondScope);
|
||||
const map = new IdMap();
|
||||
map.learn(101, 10, collectionScope(nested, first));
|
||||
map.learn(101, 20, collectionScope(nested, second));
|
||||
const literals = new Set([0, 1, 2, 4, 5, 7]);
|
||||
const translate = (segments: readonly string[], path: string) => {
|
||||
const scopes = parameterScopes(nested, segments);
|
||||
return translatePath(
|
||||
path,
|
||||
map,
|
||||
(index) => scopes.get(index),
|
||||
(index) => literals.has(index),
|
||||
);
|
||||
};
|
||||
expect(translate(first, `/${first.join("/")}`)).toBe(
|
||||
"/api/groups/uid/a%2Ffolders%2Fuid%2Fb/folders/uid/c/users/10",
|
||||
);
|
||||
expect(translate(second, `/${second.join("/")}`)).toBe(
|
||||
"/api/groups/uid/a/folders/uid/b%2Ffolders%2Fuid%2Fc/users/20",
|
||||
);
|
||||
expect(map.env(101, secondScope)).toBe(20);
|
||||
});
|
||||
});
|
||||
|
||||
describe("compareEnvironment", () => {
|
||||
it("agrees with a re-seeded application once seeded and created ids are mapped", () => {
|
||||
const report = compare([
|
||||
["GET", "/api/books", [LIVE_FIRST, LIVE_SECOND]],
|
||||
["GET", "/api/books/101", LIVE_FIRST],
|
||||
["POST", "/api/books", LIVE_FOURTH, 201, { name: "Fourth Book" }],
|
||||
["GET", "/api/books/103", LIVE_FOURTH],
|
||||
["PATCH", "/api/books/101", LIVE_RENAMED, 200, { name: "Renamed Book" }],
|
||||
["GET", "/api/books/101", LIVE_RENAMED],
|
||||
["DELETE", "/api/books/103", "", 204],
|
||||
["GET", "/api/books/103", BOOK_NOT_FOUND, 404],
|
||||
["GET", "/api/books", [LIVE_RENAMED, LIVE_SECOND]],
|
||||
]);
|
||||
expect(report.environment.state).toBe("served");
|
||||
expect(report.probes[0]?.results.filter((step) => !step.equal)).toEqual([]);
|
||||
expect(report.summary).toMatchObject({
|
||||
probes: 1,
|
||||
steps: 9,
|
||||
equal: 9,
|
||||
agreement: 1,
|
||||
});
|
||||
expect(report.probes[0]?.ids_mapped).toBe(3);
|
||||
expect(report.summary.normalised.ids).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("resets the state per probe and categorises every kind of disagreement", () => {
|
||||
const report = compare(
|
||||
[
|
||||
["GET", "/api/books", [LIVE_FIRST, LIVE_SECOND]],
|
||||
["DELETE", "/api/books/101", { detail: "Book is referenced." }, 409],
|
||||
["GET", "/api/authors", []],
|
||||
["GET", "/api/books/555", LIVE_FIRST],
|
||||
],
|
||||
[["GET", "/api/books", [LIVE_SECOND, LIVE_FIRST, LIVE_FOURTH]]],
|
||||
);
|
||||
const [first, second] = report.probes;
|
||||
expect(first?.results.map((step) => step.categories)).toEqual([
|
||||
[],
|
||||
["status", "value"],
|
||||
["no-tool"],
|
||||
["unmapped-id", "status", "missing-field", "extra-field"],
|
||||
]);
|
||||
expect(second?.results[0]?.categories).toEqual(["length", "order"]);
|
||||
expect(second?.results[0]?.equal_ignoring_order).toBe(false);
|
||||
expect(report.summary.agreement_ignoring_order).toBe(
|
||||
report.summary.agreement,
|
||||
);
|
||||
expect(report.summary.categories).toMatchObject({
|
||||
status: 2,
|
||||
"no-tool": 1,
|
||||
"unmapped-id": 1,
|
||||
length: 1,
|
||||
order: 1,
|
||||
});
|
||||
expect(report.tools.map((tool) => tool.tool)).toContain(
|
||||
"delete_books_by_id",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("what the comparison translates and learns", () => {
|
||||
/** The books capture with a paged, filtered list and reports the state does not model, so other reports replay this one. */
|
||||
const extra = environmentStateServeFixture(
|
||||
timedCaptureEvidence([
|
||||
...BOOK_ROWS,
|
||||
["GET", "/api/books?limit=2&author=7", [FIRST_BOOK, SECOND_BOOK]],
|
||||
["GET", "/api/reports/1", [{ owner_id: 7 }]],
|
||||
["GET", "/api/authors", [ANN, BOB]],
|
||||
["GET", "/api/authors/1", ANN],
|
||||
["GET", "/api/authors/2", BOB],
|
||||
["GET", "/api/ping", ""],
|
||||
["POST", "/api/books/search", [FIRST_BOOK], 200, { q: "First" }],
|
||||
["POST", "/api/tags/search", [{ id: 1 }], 200, { q: "first" }],
|
||||
["GET", "/api/motd", '""'],
|
||||
]),
|
||||
);
|
||||
const extraSource = environmentServeSource(extra);
|
||||
const compareExtra = (rows: readonly TimedCaptureRow[]) => {
|
||||
const report = compareEnvironment({
|
||||
source: extraSource,
|
||||
captureValue: extra.capture,
|
||||
probes: [{ label: "probe", value: timedCaptureEvidence(rows) }],
|
||||
operation: "environment-compare",
|
||||
});
|
||||
if (!report.ok) throw report.error;
|
||||
return report.value.probes[0]!;
|
||||
};
|
||||
|
||||
it("translates a path id by the collection it names when two collections reuse a live id", () => {
|
||||
// Live book 101 is the environment's book 1, while live author 101 is its author 2.
|
||||
const probe = compareExtra([
|
||||
["GET", "/api/books", [LIVE_FIRST, LIVE_SECOND]],
|
||||
[
|
||||
"GET",
|
||||
"/api/authors",
|
||||
[
|
||||
{ ...ANN, id: 102 },
|
||||
{ ...BOB, id: 101 },
|
||||
],
|
||||
],
|
||||
["GET", "/api/authors/101", { ...BOB, id: 101 }],
|
||||
]);
|
||||
expect(probe.results.map((step) => step.equal)).toEqual([true, true, true]);
|
||||
});
|
||||
|
||||
it("refuses arguments the served tool's input schema refuses, as the server would", () => {
|
||||
// A search replays its recording whatever it is sent, but the server refuses a body that is not an object.
|
||||
const [step] = compareExtra([
|
||||
["POST", "/api/books/search", [FIRST_BOOK], 200, ["First"]],
|
||||
]).results;
|
||||
expect(step?.categories).toEqual(["refused"]);
|
||||
expect(step?.differences[0]?.env).toContain("Invalid arguments");
|
||||
});
|
||||
|
||||
it("agrees on an empty response however each side spells it, and keeps a JSON empty string", () => {
|
||||
expect(compareExtra([["GET", "/api/ping", ""]]).results[0]?.equal).toBe(
|
||||
true,
|
||||
);
|
||||
expect(compareExtra([["GET", "/api/motd", '""']]).results[0]?.equal).toBe(
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
it("learns ids from a replayed recording only of the same call, body included", () => {
|
||||
expect(
|
||||
compareExtra([
|
||||
["POST", "/api/tags/search", [{ id: 102 }], 200, { q: "second" }],
|
||||
]).ids_mapped,
|
||||
).toBe(0);
|
||||
expect(
|
||||
compareExtra([
|
||||
["POST", "/api/tags/search", [{ id: 102 }], 200, { q: "first" }],
|
||||
]).ids_mapped,
|
||||
).toBe(1);
|
||||
// A body the probe did not send is not the body the recording sent.
|
||||
expect(
|
||||
compareExtra([["POST", "/api/tags/search", [{ id: 102 }], 200]])
|
||||
.ids_mapped,
|
||||
).toBe(0);
|
||||
});
|
||||
|
||||
it("compares a large binary body only by status, as it does large text", () => {
|
||||
expect(isCut({ kind: "base64", encoded: "AAAA", whole: false })).toBe(true);
|
||||
expect(isCut({ kind: "base64", encoded: "AAAA", whole: true })).toBe(false);
|
||||
expect(isCut({ kind: "cut", text: "a" })).toBe(true);
|
||||
});
|
||||
|
||||
it("translates mapped ids in query values except those that page a list", () => {
|
||||
const call = matchProbeRequest(extraSource.manifest, {
|
||||
method: "GET",
|
||||
path: "/api/books",
|
||||
query: "?limit=101&author=101",
|
||||
body: undefined,
|
||||
});
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1);
|
||||
expect(call?.pagingArguments).toEqual(["limit"]);
|
||||
expect(call && environmentArguments(call, map)).toEqual({
|
||||
limit: "101",
|
||||
author: "1",
|
||||
});
|
||||
});
|
||||
|
||||
it("learns no ids from another call's recording replayed in place of an answer", () => {
|
||||
const report = compareEnvironment({
|
||||
source: extraSource,
|
||||
captureValue: extra.capture,
|
||||
probes: [
|
||||
{
|
||||
label: "reports",
|
||||
value: timedCaptureEvidence([
|
||||
["GET", "/api/reports/2", [{ owner_id: 107 }]],
|
||||
]),
|
||||
},
|
||||
],
|
||||
operation: "environment-compare",
|
||||
});
|
||||
if (!report.ok) throw report.error;
|
||||
const [probe] = report.value.probes;
|
||||
expect(probe?.ids_mapped).toBe(0);
|
||||
expect(probe?.results[0]?.differences).toEqual([
|
||||
{ category: "value", pointer: "/0/owner_id", live: "107", env: "7" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
const compareRows = (
|
||||
recorded: readonly TimedCaptureRow[],
|
||||
probe: readonly TimedCaptureRow[],
|
||||
) => {
|
||||
const capture = timedCaptureEvidence(recorded);
|
||||
const built = environmentStateServeFixture(capture);
|
||||
const report = compareEnvironment({
|
||||
source: environmentServeSource(built),
|
||||
captureValue: capture,
|
||||
probes: [{ label: "probe", value: timedCaptureEvidence(probe) }],
|
||||
operation: "environment-compare",
|
||||
});
|
||||
if (!report.ok) throw report.error;
|
||||
return report.value.probes[0]!;
|
||||
};
|
||||
|
||||
describe("ids the comparison used to misread", () => {
|
||||
it("reads a twice-encoded path id once, so book%2531 is not book1", () => {
|
||||
const probe = compareRows(
|
||||
[
|
||||
["GET", "/api/files/book1", { id: "book1", name: "One" }],
|
||||
["GET", "/api/files/book%2531", { id: "book%31", name: "Encoded" }],
|
||||
["GET", "/api/files/other", { id: "other", name: "Other" }],
|
||||
],
|
||||
[["GET", "/api/files/book%2531", { id: "book%31", name: "Encoded" }]],
|
||||
);
|
||||
expect(probe.results[0]?.equal).toBe(true);
|
||||
expect(probe.results[0]?.categories).not.toContain("value");
|
||||
});
|
||||
|
||||
it("learns a create that answers with only the new id", () => {
|
||||
const item = (id: number, name: string) => ({ id, name });
|
||||
const third = item(3, "Third");
|
||||
const probe = compareRows(
|
||||
[
|
||||
["GET", "/api/items", [item(1, "First"), item(2, "Second")]],
|
||||
["GET", "/api/items/1", item(1, "First")],
|
||||
["GET", "/api/items/2", item(2, "Second")],
|
||||
["POST", "/api/items", { created: 3 }, 201, { name: "Third" }],
|
||||
["GET", "/api/items/3", third],
|
||||
["GET", "/api/items", [item(1, "First"), item(2, "Second"), third]],
|
||||
],
|
||||
[
|
||||
["GET", "/api/items", [item(101, "First"), item(102, "Second")]],
|
||||
["POST", "/api/items", { created: 103 }, 201, { name: "Third" }],
|
||||
["GET", "/api/items/103", item(103, "Third")],
|
||||
],
|
||||
);
|
||||
expect(probe.results.map((step) => step.equal)).toEqual([true, true, true]);
|
||||
expect(probe.ids_mapped).toBeGreaterThan(0);
|
||||
const scalar = compareRows(
|
||||
[
|
||||
["GET", "/api/items", [item(1, "First"), item(2, "Second")]],
|
||||
["GET", "/api/items/1", item(1, "First")],
|
||||
["GET", "/api/items/2", item(2, "Second")],
|
||||
["POST", "/api/items", 3, 201, { name: "Third" }],
|
||||
["GET", "/api/items/3", third],
|
||||
["GET", "/api/items", [item(1, "First"), item(2, "Second"), third]],
|
||||
],
|
||||
[
|
||||
["GET", "/api/items", [item(101, "First"), item(102, "Second")]],
|
||||
["POST", "/api/items", 103, 201, { name: "Third" }],
|
||||
["GET", "/api/items/103", item(103, "Third")],
|
||||
],
|
||||
);
|
||||
expect(scalar.results.map((step) => step.equal)).toEqual([
|
||||
true,
|
||||
true,
|
||||
true,
|
||||
]);
|
||||
});
|
||||
|
||||
it("compares a create id within its collection when another collection already mapped that live id", () => {
|
||||
const item = (id: number, name: string) => ({ id, name });
|
||||
const recorded = (
|
||||
created: number | { readonly created: number },
|
||||
): TimedCaptureRow[] => [
|
||||
["GET", "/api/books", [item(1, "Book")]],
|
||||
["GET", "/api/books/1", item(1, "Book")],
|
||||
["GET", "/api/authors", [item(2, "Ann")]],
|
||||
["GET", "/api/authors/2", item(2, "Ann")],
|
||||
["POST", "/api/authors", created, 201, { name: "Bob" }],
|
||||
["GET", "/api/authors/3", item(3, "Bob")],
|
||||
["GET", "/api/authors", [item(2, "Ann"), item(3, "Bob")]],
|
||||
];
|
||||
const probe = (
|
||||
created: number | { readonly created: number },
|
||||
): TimedCaptureRow[] => [
|
||||
["GET", "/api/books", [item(103, "Book")]],
|
||||
["POST", "/api/authors", created, 201, { name: "Bob" }],
|
||||
["GET", "/api/authors/103", item(103, "Bob")],
|
||||
];
|
||||
expect(
|
||||
compareRows(recorded(3), probe(103)).results.map((step) => step.equal),
|
||||
).toEqual([true, true, true]);
|
||||
expect(
|
||||
compareRows(
|
||||
recorded({ created: 3 }),
|
||||
probe({ created: 103 }),
|
||||
).results.map((step) => step.equal),
|
||||
).toEqual([true, true, true]);
|
||||
});
|
||||
|
||||
it("keeps each parent's child ids and does not rename a literal route segment", () => {
|
||||
const users = (id: number, name: string) => ({ id, name });
|
||||
const probe = compareRows(
|
||||
[
|
||||
["GET", "/api/orgs/1/users", [users(10, "Ann"), users(11, "Bea")]],
|
||||
["GET", "/api/orgs/2/users", [users(20, "Cy"), users(21, "Di")]],
|
||||
["GET", "/api/orgs/1/users/10", users(10, "Ann")],
|
||||
["GET", "/api/orgs/2/users/20", users(20, "Cy")],
|
||||
["GET", "/api/tags", [{ id: "stored-tag", name: "Books" }]],
|
||||
["GET", "/api/tags/stored-tag", { id: "stored-tag", name: "Books" }],
|
||||
["GET", "/api/books", [users(1, "First")]],
|
||||
],
|
||||
[
|
||||
["GET", "/api/orgs/1/users", [users(110, "Ann"), users(111, "Bea")]],
|
||||
["GET", "/api/orgs/2/users", [users(110, "Cy"), users(111, "Di")]],
|
||||
["GET", "/api/orgs/1/users/110", users(110, "Ann")],
|
||||
["GET", "/api/orgs/2/users/110", users(110, "Cy")],
|
||||
["GET", "/api/tags", [{ id: "books", name: "Books" }]],
|
||||
["GET", "/api/books", [users(1, "First")]],
|
||||
],
|
||||
);
|
||||
expect(probe.results.map((step) => step.categories)).toEqual([
|
||||
[],
|
||||
[],
|
||||
[],
|
||||
[],
|
||||
[],
|
||||
[],
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("browser probes", () => {
|
||||
it("keeps a browser probe's request bodies, which are its call arguments", () => {
|
||||
const post = (purpose?: "manifest") => {
|
||||
const parsed = parseCaptureEvidence(
|
||||
"environment-compare",
|
||||
scenarioCaptureEvidence(),
|
||||
purpose,
|
||||
);
|
||||
if (!parsed.ok) throw parsed.error;
|
||||
return parsed.value.records.find((record) =>
|
||||
JSON.stringify(record.reported).includes('"method":"POST"'),
|
||||
)?.reported;
|
||||
};
|
||||
expect(JSON.stringify(post("manifest"))).toContain("postData");
|
||||
expect(JSON.stringify(post())).not.toContain("postData");
|
||||
});
|
||||
});
|
||||
|
||||
describe("what a probe did not observe", () => {
|
||||
const at = (second: number): string =>
|
||||
new Date(
|
||||
Date.parse("2026-10-20T09:00:00.000Z") + second * 1000,
|
||||
).toISOString();
|
||||
const exchange = (
|
||||
second: number,
|
||||
method: string,
|
||||
url: string | null,
|
||||
status: number,
|
||||
) => ({
|
||||
startedDateTime: at(second),
|
||||
request: { method, url },
|
||||
response: { status, content: { mimeType: "application/json" } },
|
||||
});
|
||||
const run = (value: unknown) =>
|
||||
compareEnvironment({
|
||||
source,
|
||||
captureValue: fixture.capture,
|
||||
probes: [{ label: "probe-1", value }],
|
||||
operation: "environment-compare",
|
||||
});
|
||||
|
||||
it("compares an unrecorded body by status only and lists the records it could not compare", () => {
|
||||
const report = run(
|
||||
analysisViewCaptureEvidence(
|
||||
environmentManifestCaptureFixture([
|
||||
exchange(0, "GET", "http://app.example/api/books/1", 200),
|
||||
exchange(1, "GET", null, 200),
|
||||
{ startedDateTime: at(2) },
|
||||
exchange(3, "GET", "http://app.example/api/books", 0),
|
||||
// A 204 carries no body, so its missing body is known to be empty.
|
||||
exchange(4, "DELETE", "http://app.example/api/books/2", 204),
|
||||
]),
|
||||
),
|
||||
);
|
||||
if (!report.ok) throw report.error;
|
||||
expect(report.value.probes[0]?.unobserved).toEqual([
|
||||
{ ordinal: 1, reason: "url-redacted" },
|
||||
{ ordinal: 2, reason: "request-unavailable" },
|
||||
{ ordinal: 3, reason: "no-response" },
|
||||
]);
|
||||
expect(report.value.summary).toMatchObject({
|
||||
steps: 2,
|
||||
equal: 2,
|
||||
unobserved: 3,
|
||||
});
|
||||
expect(report.value.summary.normalised.unobserved_bodies).toBe(1);
|
||||
});
|
||||
|
||||
it("makes a request whose answer was lost, so later reads see its effect", () => {
|
||||
const report = run(
|
||||
analysisViewCaptureEvidence(
|
||||
environmentManifestCaptureFixture([
|
||||
exchange(0, "DELETE", "http://app.example/api/books/2", 0),
|
||||
{
|
||||
...exchange(1, "GET", "http://app.example/api/books/2", 404),
|
||||
response: {
|
||||
status: 404,
|
||||
content: {
|
||||
mimeType: "application/json",
|
||||
text: JSON.stringify(BOOK_NOT_FOUND),
|
||||
},
|
||||
},
|
||||
},
|
||||
]),
|
||||
),
|
||||
);
|
||||
if (!report.ok) throw report.error;
|
||||
expect(report.value.probes[0]?.unobserved).toEqual([
|
||||
{ ordinal: 0, reason: "no-response" },
|
||||
]);
|
||||
expect(report.value.summary).toMatchObject({ steps: 1, equal: 1 });
|
||||
});
|
||||
|
||||
it("makes no request to another origin, so it cannot change the environment", () => {
|
||||
const answered = (second: number, url: string, body: JsonValue) => ({
|
||||
...exchange(second, "GET", url, 200),
|
||||
response: {
|
||||
status: 200,
|
||||
content: { mimeType: "application/json", text: JSON.stringify(body) },
|
||||
},
|
||||
});
|
||||
const report = run(
|
||||
analysisViewCaptureEvidence(
|
||||
environmentManifestCaptureFixture([
|
||||
// The probe also called another server's API, at a path the application's tools match.
|
||||
exchange(0, "DELETE", "http://cdn.example/api/books/2", 204),
|
||||
answered(1, "http://app.example/api/books/1", FIRST_BOOK),
|
||||
answered(2, "http://app.example/api/books/2", SECOND_BOOK),
|
||||
]),
|
||||
),
|
||||
);
|
||||
if (!report.ok) throw report.error;
|
||||
expect(report.value.probes[0]?.unobserved).toEqual([
|
||||
{ ordinal: 0, reason: "other-origin" },
|
||||
]);
|
||||
expect(report.value.summary).toMatchObject({ steps: 2, equal: 2 });
|
||||
});
|
||||
|
||||
it("keeps the recorded order when a record has no start time, as state extraction does", () => {
|
||||
const { startedDateTime: _undated, ...read } = {
|
||||
...exchange(1, "GET", "http://app.example/api/books/2", 404),
|
||||
response: {
|
||||
status: 404,
|
||||
content: {
|
||||
mimeType: "application/json",
|
||||
text: JSON.stringify(BOOK_NOT_FOUND),
|
||||
},
|
||||
},
|
||||
};
|
||||
const report = run(
|
||||
analysisViewCaptureEvidence(
|
||||
environmentManifestCaptureFixture([
|
||||
exchange(0, "DELETE", "http://app.example/api/books/2", 204),
|
||||
read,
|
||||
]),
|
||||
),
|
||||
);
|
||||
if (!report.ok) throw report.error;
|
||||
expect(report.value.summary).toMatchObject({ steps: 2, equal: 2 });
|
||||
});
|
||||
|
||||
it("refuses a browser probe that makes writes without recorded request bodies", () => {
|
||||
const report = run(
|
||||
scenarioCaptureEvidence(
|
||||
scenarioCaptureFixture(undefined, {
|
||||
request_body: false,
|
||||
response_body: true,
|
||||
header_values: true,
|
||||
}),
|
||||
),
|
||||
);
|
||||
expect(report.ok).toBe(false);
|
||||
if (report.ok) return;
|
||||
expect(JSON.stringify(report.error)).toContain(
|
||||
"needs capture.network.request_body selected",
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses a browser probe captured without response bodies", () => {
|
||||
const report = run(scenarioHeaderOnlyCaptureEvidence());
|
||||
expect(report.ok).toBe(false);
|
||||
if (report.ok) return;
|
||||
expect(JSON.stringify(report.error)).toContain(
|
||||
"probe-1: a browser probe needs capture.network.response_body selected",
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,508 @@
|
||||
import { decodedSegment } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import { fromJsonSchema } from "@modelcontextprotocol/server";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import { err, ok, type Result } from "../../domain/result.js";
|
||||
import {
|
||||
compareAnswers,
|
||||
DIFFERENCE_CATEGORIES,
|
||||
STEP_DIFFERENCE_LIMIT,
|
||||
type Answer,
|
||||
type Difference,
|
||||
type DifferenceCategory,
|
||||
type Normalised,
|
||||
} from "./environmentCompareDiff.js";
|
||||
import {
|
||||
environmentArguments,
|
||||
IdMap,
|
||||
learnIds,
|
||||
translatePath,
|
||||
} from "./environmentCompareIds.js";
|
||||
import {
|
||||
collectionScope,
|
||||
literalPositions,
|
||||
matchProbeRequest,
|
||||
parameterScopes,
|
||||
type ProbeToolCall,
|
||||
} from "./environmentCompareMatch.js";
|
||||
import {
|
||||
probeRequests,
|
||||
requestBody,
|
||||
sendsBodies,
|
||||
type ProbeStep,
|
||||
type UnobservedRecord,
|
||||
} from "./environmentCompareProbe.js";
|
||||
import {
|
||||
inputError,
|
||||
parseCaptureEvidence,
|
||||
replayedBody,
|
||||
type EnvironmentManifestTool,
|
||||
type EnvironmentSource,
|
||||
} from "./environmentSource.js";
|
||||
import {
|
||||
selectsRecordedLine,
|
||||
type ReplayArguments,
|
||||
type ReplayResult,
|
||||
} from "./environmentReplay.js";
|
||||
import { prepareServedState, servedCatalog } from "./environmentState.js";
|
||||
import type { StateAnswer } from "./environmentStateAnswer.js";
|
||||
import { EnvironmentStateStore } from "./environmentStateStore.js";
|
||||
import {
|
||||
environmentToolAnswer,
|
||||
type EnvironmentToolAnswer,
|
||||
} from "./environmentToolAnswer.js";
|
||||
import {
|
||||
environmentToolArguments,
|
||||
type EnvironmentToolDefinition,
|
||||
} from "./environmentTools.js";
|
||||
import { same } from "../../domain/analysisView/environmentStateRules.js";
|
||||
|
||||
/** One probe capture file: its label for the report and its parsed JSON. */
|
||||
export interface CompareProbeInput {
|
||||
readonly label: string;
|
||||
readonly value: unknown;
|
||||
}
|
||||
|
||||
/** One probe request, how the environment answered it, and how that differs from the application. */
|
||||
export interface CompareStep {
|
||||
readonly ordinal: number;
|
||||
readonly method: string;
|
||||
readonly path: string;
|
||||
readonly tool: string | null;
|
||||
readonly live_status: number | null;
|
||||
readonly env_status: number | null;
|
||||
readonly equal: boolean;
|
||||
/** Equal but for item order, which an application that returns sets in no fixed order cannot reproduce either. */
|
||||
readonly equal_ignoring_order: boolean;
|
||||
readonly categories: DifferenceCategory[];
|
||||
readonly differences: Difference[];
|
||||
readonly ignored_query: string[];
|
||||
}
|
||||
|
||||
export interface CompareProbeReport {
|
||||
readonly label: string;
|
||||
readonly evidence_id: string;
|
||||
readonly steps: number;
|
||||
readonly equal: number;
|
||||
readonly ids_mapped: number;
|
||||
readonly id_conflicts: number;
|
||||
readonly normalised: Normalised;
|
||||
readonly results: CompareStep[];
|
||||
/** Records left uncompared, in probe order, each with its reason. */
|
||||
readonly unobserved: UnobservedRecord[];
|
||||
}
|
||||
|
||||
export interface CompareReport {
|
||||
readonly status: "complete";
|
||||
readonly environment: {
|
||||
readonly name: string;
|
||||
readonly manifest_evidence_id: string;
|
||||
readonly evidence_id: string;
|
||||
readonly state: "served" | "replay-only";
|
||||
};
|
||||
readonly summary: {
|
||||
readonly probes: number;
|
||||
readonly steps: number;
|
||||
readonly equal: number;
|
||||
readonly agreement: number | null;
|
||||
readonly agreement_ignoring_order: number | null;
|
||||
/** Probe records left uncompared, outside the steps agreement counts. */
|
||||
readonly unobserved: number;
|
||||
readonly categories: Record<DifferenceCategory, number>;
|
||||
readonly normalised: Normalised;
|
||||
};
|
||||
readonly tools: { tool: string; steps: number; equal: number }[];
|
||||
readonly probes: CompareProbeReport[];
|
||||
}
|
||||
|
||||
const NORMALISED_KEYS = [
|
||||
"ids",
|
||||
"clocks",
|
||||
"generated",
|
||||
"large_bodies",
|
||||
"unobserved_bodies",
|
||||
] as const;
|
||||
const emptyNormalised = (): Normalised => ({
|
||||
ids: 0,
|
||||
clocks: 0,
|
||||
generated: 0,
|
||||
large_bodies: 0,
|
||||
unobserved_bodies: 0,
|
||||
});
|
||||
|
||||
const isSuccess = (status: number | null): boolean =>
|
||||
status !== null && status >= 200 && status < 300;
|
||||
|
||||
/** Where a call's answer holds its own entities, and the collection they belong to: a read view's entity or list pointer, a write's response pointer, else the root. A write that answers with only the new id learns that scalar. */
|
||||
const entityOf = (
|
||||
run: ProbeRun,
|
||||
tool: EnvironmentManifestTool,
|
||||
segments: readonly string[],
|
||||
): {
|
||||
readonly scope: string;
|
||||
readonly at: string;
|
||||
readonly identity: boolean;
|
||||
} => {
|
||||
const view = run.store?.state.views.get(tool.name);
|
||||
const plan = run.store?.state.writes.get(tool.name);
|
||||
const response = plan?.response;
|
||||
const scope = collectionScope(tool, segments);
|
||||
if (view?.shape === "item" || view?.shape === "list")
|
||||
return { scope, at: view.pointer, identity: false };
|
||||
const identity = response?.shape === "identity";
|
||||
const at = identity
|
||||
? (response.pointer ?? "")
|
||||
: (response?.pointer ?? plan?.observation?.pointer ?? "");
|
||||
return { scope, at, identity };
|
||||
};
|
||||
|
||||
const fromState = (answer: ReplayResult): boolean =>
|
||||
(answer as Partial<StateAnswer>).state?.source === "state";
|
||||
|
||||
/** The environment's body as the application's is read: a replayed recording of empty text, unlike a JSON empty string, is no body. */
|
||||
const envBody = (run: ProbeRun, answer: ReplayResult): JsonValue => {
|
||||
if (answer.body !== "" || fromState(answer)) return answer.body;
|
||||
const exchange = run.source.exchanges.get(answer.replay.record_ordinal);
|
||||
return exchange !== undefined && replayedBody(exchange).kind === "json"
|
||||
? answer.body
|
||||
: null;
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether the environment answered this call itself: from its state, or from a recording of exactly this call, every
|
||||
* recorded query value supplied and the body it sent, rather than replaying another call's recording.
|
||||
*/
|
||||
const answersCall = (
|
||||
run: ProbeRun,
|
||||
call: ProbeToolCall,
|
||||
args: Readonly<Record<string, JsonValue>>,
|
||||
answer: ReplayResult,
|
||||
): boolean => {
|
||||
if (fromState(answer)) return true;
|
||||
const exchange = run.source.exchanges.get(answer.replay.record_ordinal);
|
||||
if (
|
||||
!answer.replay.exact_match ||
|
||||
exchange === undefined ||
|
||||
!selectsRecordedLine(call.tool, exchange, args)
|
||||
)
|
||||
return false;
|
||||
const body = environmentToolArguments(call.tool).body;
|
||||
// A tool with no body parameter matches; one that takes a body matches only the body this call sent, including none.
|
||||
if (body === undefined) return true;
|
||||
return same(run.recordedBodies.get(exchange.ordinal), args[body]);
|
||||
};
|
||||
|
||||
/** A served tool's input check: the first problem with a call's arguments, or undefined. */
|
||||
type InputCheck = (args: ReplayArguments) => string | undefined;
|
||||
|
||||
/** The input schema check the MCP server applies before a served tool answers, so a call it refuses is not compared as answered. */
|
||||
const inputCheck = (definition: EnvironmentToolDefinition): InputCheck => {
|
||||
const schema = fromJsonSchema<ReplayArguments>(definition.inputSchema)[
|
||||
"~standard"
|
||||
];
|
||||
return (args) => {
|
||||
const checked = schema.validate(args);
|
||||
return checked instanceof Promise
|
||||
? undefined
|
||||
: checked.issues?.[0]?.message;
|
||||
};
|
||||
};
|
||||
|
||||
interface ProbeRun {
|
||||
readonly source: EnvironmentSource;
|
||||
/** The request body of each capture record, by ordinal, that a replayed answer's call is compared with. */
|
||||
readonly recordedBodies: ReadonlyMap<number, JsonValue | undefined>;
|
||||
readonly answer: EnvironmentToolAnswer;
|
||||
readonly store: EnvironmentStateStore | undefined;
|
||||
/** The tools the environment offers, by name, with their input checks. */
|
||||
readonly served: ReadonlyMap<string, InputCheck>;
|
||||
readonly map: IdMap;
|
||||
readonly normalised: Normalised;
|
||||
}
|
||||
|
||||
const heldIds = (
|
||||
store: EnvironmentStateStore | undefined,
|
||||
): ReadonlySet<string> =>
|
||||
new Set(
|
||||
store?.snapshot().rea_state.entities.map((entity) => entity.id) ?? [],
|
||||
);
|
||||
|
||||
const stepOf = (
|
||||
base: Pick<CompareStep, "ordinal" | "method" | "path" | "live_status">,
|
||||
call: ProbeToolCall | undefined,
|
||||
outcome: { env_status: number | null; differences: Difference[] },
|
||||
): CompareStep => {
|
||||
const categories = [...new Set(outcome.differences.map((d) => d.category))];
|
||||
return {
|
||||
...base,
|
||||
tool: call?.tool.name ?? null,
|
||||
env_status: outcome.env_status,
|
||||
equal: outcome.differences.length === 0,
|
||||
equal_ignoring_order: outcome.differences.every(
|
||||
(difference) => difference.category === "order",
|
||||
),
|
||||
categories,
|
||||
differences: outcome.differences.slice(0, STEP_DIFFERENCE_LIMIT),
|
||||
ignored_query: [...(call?.ignoredQuery ?? [])],
|
||||
};
|
||||
};
|
||||
|
||||
/** Answer one probe request as the environment would and compare it with the application's answer; one without an answer is only made. */
|
||||
const compareStep = (
|
||||
run: ProbeRun,
|
||||
manifest: EnvironmentSource["manifest"],
|
||||
probe: ProbeStep,
|
||||
): CompareStep | undefined => {
|
||||
const { request, live } = probe;
|
||||
const base = {
|
||||
ordinal: probe.ordinal,
|
||||
method: request.method,
|
||||
path: request.path,
|
||||
live_status: live?.status ?? null,
|
||||
};
|
||||
const report = (
|
||||
call: ProbeToolCall | undefined,
|
||||
outcome: { env_status: number | null; differences: Difference[] },
|
||||
): CompareStep | undefined =>
|
||||
live === undefined ? undefined : stepOf(base, call, outcome);
|
||||
// Matched in the environment's terms, so a tool whose path holds a seeded id literally still matches. Scopes and
|
||||
// literals come from the live path: translating a parent id first would name a different collection than learning did.
|
||||
const liveSegments = request.path.slice(1).split("/");
|
||||
const fitted = matchProbeRequest(manifest, request);
|
||||
const scopes =
|
||||
fitted === undefined
|
||||
? undefined
|
||||
: parameterScopes(fitted.tool, liveSegments);
|
||||
const literals =
|
||||
fitted === undefined ? undefined : literalPositions(fitted.tool);
|
||||
const call = matchProbeRequest(manifest, {
|
||||
...request,
|
||||
path: translatePath(
|
||||
request.path,
|
||||
run.map,
|
||||
(index) => scopes?.get(index),
|
||||
(index) => literals?.has(index) === true,
|
||||
),
|
||||
});
|
||||
if (call === undefined)
|
||||
return report(call, {
|
||||
env_status: null,
|
||||
differences: [{ category: "no-tool", pointer: "" }],
|
||||
});
|
||||
const check = run.served.get(call.tool.name);
|
||||
if (check === undefined)
|
||||
return report(call, {
|
||||
env_status: null,
|
||||
differences: [{ category: "not-served", pointer: "" }],
|
||||
});
|
||||
const held = heldIds(run.store);
|
||||
const unknownPath = call.pathArguments.some((name) => {
|
||||
const value = call.args[name];
|
||||
if (typeof value !== "string") return false;
|
||||
const id = decodedSegment(value);
|
||||
return !run.map.hasEnv(id) && !held.has(id);
|
||||
});
|
||||
const refusedWith = (message: string): CompareStep | undefined =>
|
||||
report(call, {
|
||||
env_status: null,
|
||||
differences: [
|
||||
{ category: "refused", pointer: "", env: message.slice(0, 200) },
|
||||
],
|
||||
});
|
||||
const args = environmentArguments(call, run.map);
|
||||
const problem = check(args);
|
||||
if (problem !== undefined)
|
||||
return refusedWith(`Invalid arguments: ${problem}`);
|
||||
const answered = run.answer(call.tool, args);
|
||||
if (!answered.ok) return refusedWith(answered.error.message);
|
||||
if (live === undefined) return undefined;
|
||||
const env: Answer = {
|
||||
status: answered.value.status,
|
||||
body: envBody(run, answered.value),
|
||||
};
|
||||
const entity = entityOf(run, call.tool, liveSegments);
|
||||
// Another call's recording holds other entities, whose ids would map wrongly and, the first pairing winning, for good.
|
||||
if (answersCall(run, call, args, answered.value))
|
||||
learnIds(live.body, env.body, run.map, {
|
||||
heldIds: heldIds(run.store),
|
||||
scope: entity.scope,
|
||||
entityAt: entity.at,
|
||||
identity: entity.identity,
|
||||
});
|
||||
const compared = compareAnswers(live, env, run.map, entity);
|
||||
for (const key of NORMALISED_KEYS)
|
||||
run.normalised[key] += compared.normalised[key];
|
||||
const differences = [...compared.differences];
|
||||
if (unknownPath && isSuccess(live.status) && !isSuccess(env.status))
|
||||
differences.unshift({ category: "unmapped-id", pointer: "" });
|
||||
return stepOf(base, call, { env_status: env.status, differences });
|
||||
};
|
||||
|
||||
const probeReport = (
|
||||
probe: { readonly label: string; readonly evidenceId: string },
|
||||
run: ProbeRun,
|
||||
results: CompareStep[],
|
||||
unobserved: UnobservedRecord[],
|
||||
): CompareProbeReport => ({
|
||||
label: probe.label,
|
||||
evidence_id: probe.evidenceId,
|
||||
steps: results.length,
|
||||
equal: results.filter((step) => step.equal).length,
|
||||
ids_mapped: run.map.size,
|
||||
id_conflicts: run.map.conflicts,
|
||||
normalised: run.normalised,
|
||||
results,
|
||||
unobserved,
|
||||
});
|
||||
|
||||
const summaryOf = (
|
||||
probes: readonly CompareProbeReport[],
|
||||
): CompareReport["summary"] => {
|
||||
const steps = probes.flatMap((probe) => probe.results);
|
||||
const categories = Object.fromEntries(
|
||||
DIFFERENCE_CATEGORIES.map((category) => [category, 0]),
|
||||
) as Record<DifferenceCategory, number>;
|
||||
for (const step of steps)
|
||||
for (const category of step.categories) categories[category] += 1;
|
||||
const equal = steps.filter((step) => step.equal).length;
|
||||
const ratio = (count: number): number | null =>
|
||||
steps.length === 0
|
||||
? null
|
||||
: Math.round((count / steps.length) * 1000) / 1000;
|
||||
const normalised = emptyNormalised();
|
||||
for (const probe of probes)
|
||||
for (const key of NORMALISED_KEYS) normalised[key] += probe.normalised[key];
|
||||
return {
|
||||
probes: probes.length,
|
||||
steps: steps.length,
|
||||
equal,
|
||||
agreement: ratio(equal),
|
||||
agreement_ignoring_order: ratio(
|
||||
steps.filter((step) => step.equal_ignoring_order).length,
|
||||
),
|
||||
unobserved: probes.reduce((sum, probe) => sum + probe.unobserved.length, 0),
|
||||
categories,
|
||||
normalised,
|
||||
};
|
||||
};
|
||||
|
||||
const toolsOf = (
|
||||
probes: readonly CompareProbeReport[],
|
||||
): CompareReport["tools"] => {
|
||||
const tools = new Map<
|
||||
string,
|
||||
{ tool: string; steps: number; equal: number }
|
||||
>();
|
||||
for (const step of probes.flatMap((probe) => probe.results)) {
|
||||
if (step.tool === null) continue;
|
||||
const entry = tools.get(step.tool) ?? {
|
||||
tool: step.tool,
|
||||
steps: 0,
|
||||
equal: 0,
|
||||
};
|
||||
entry.steps += 1;
|
||||
if (step.equal) entry.equal += 1;
|
||||
tools.set(step.tool, entry);
|
||||
}
|
||||
return [...tools.values()].sort(
|
||||
(left, right) =>
|
||||
left.equal / left.steps - right.equal / right.steps ||
|
||||
(left.tool < right.tool ? -1 : 1),
|
||||
);
|
||||
};
|
||||
|
||||
/** Compare an environment with probe captures of the application: each probe from a reset state, every request answered and compared. */
|
||||
export const compareEnvironment = (input: {
|
||||
readonly source: EnvironmentSource;
|
||||
readonly captureValue: unknown;
|
||||
readonly probes: readonly CompareProbeInput[];
|
||||
readonly operation: string;
|
||||
}): Result<CompareReport, AnalysisError> => {
|
||||
const { source, operation } = input;
|
||||
const state = prepareServedState(source, input.captureValue, {
|
||||
operation,
|
||||
}).state;
|
||||
const catalog = servedCatalog(source, state, source.manifest.name, operation);
|
||||
if (!catalog.ok) return catalog;
|
||||
const served = new Map(
|
||||
catalog.value.tools.map((definition) => [
|
||||
definition.name,
|
||||
inputCheck(definition),
|
||||
]),
|
||||
);
|
||||
const recorded = parseCaptureEvidence(
|
||||
operation,
|
||||
input.captureValue,
|
||||
"manifest",
|
||||
);
|
||||
const recordedBodies = new Map(
|
||||
recorded.ok
|
||||
? recorded.value.records.map((record) => [
|
||||
record.ordinal,
|
||||
requestBody(record),
|
||||
])
|
||||
: [],
|
||||
);
|
||||
const probes: CompareProbeReport[] = [];
|
||||
for (const [index, probe] of input.probes.entries()) {
|
||||
// A probe's request bodies are its call arguments, which a replay projection of a browser capture drops.
|
||||
const capture = parseCaptureEvidence(operation, probe.value, "manifest");
|
||||
// Without response bodies only statuses would compare, and without request bodies writes would be made empty.
|
||||
const problem = !capture.ok
|
||||
? capture.error.message
|
||||
: !capture.value.responseBodies
|
||||
? "a browser probe needs capture.network.response_body selected, or only statuses would compare."
|
||||
: !capture.value.requestBodies && sendsBodies(capture.value.records)
|
||||
? "a browser probe that makes POST, PUT or PATCH calls needs capture.network.request_body selected, or those calls would be made without their bodies."
|
||||
: undefined;
|
||||
if (!capture.ok || problem !== undefined)
|
||||
return err(
|
||||
inputError(
|
||||
operation,
|
||||
[
|
||||
{
|
||||
path: ["probe", index],
|
||||
reason: "invalid_value",
|
||||
message: `${probe.label}: ${problem}`,
|
||||
},
|
||||
],
|
||||
"A probe file cannot be compared.",
|
||||
),
|
||||
);
|
||||
const store =
|
||||
state === undefined ? undefined : new EnvironmentStateStore(state);
|
||||
const run: ProbeRun = {
|
||||
source,
|
||||
recordedBodies,
|
||||
answer: environmentToolAnswer(source, store),
|
||||
store,
|
||||
served,
|
||||
map: new IdMap(),
|
||||
normalised: emptyNormalised(),
|
||||
};
|
||||
const { steps, unobserved } = probeRequests(capture.value.records);
|
||||
const results = steps.flatMap(
|
||||
(step) => compareStep(run, source.manifest, step) ?? [],
|
||||
);
|
||||
probes.push(
|
||||
probeReport(
|
||||
{ label: probe.label, evidenceId: capture.value.evidenceId },
|
||||
run,
|
||||
results,
|
||||
unobserved,
|
||||
),
|
||||
);
|
||||
}
|
||||
return ok({
|
||||
status: "complete",
|
||||
environment: {
|
||||
name: source.manifest.name,
|
||||
manifest_evidence_id: source.manifestEvidenceId,
|
||||
evidence_id: source.evidenceId,
|
||||
state: state === undefined ? "replay-only" : "served",
|
||||
},
|
||||
summary: summaryOf(probes),
|
||||
tools: toolsOf(probes),
|
||||
probes,
|
||||
});
|
||||
};
|
||||
@@ -0,0 +1,307 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import { compareAnswers } from "./environmentCompareDiff.js";
|
||||
import { IdMap, learnIds } from "./environmentCompareIds.js";
|
||||
|
||||
const compare = (
|
||||
live: unknown,
|
||||
env: unknown,
|
||||
map = new IdMap(),
|
||||
status: [number, number] = [200, 200],
|
||||
) =>
|
||||
compareAnswers(
|
||||
{ status: status[0], body: live as never },
|
||||
{ status: status[1], body: env as never },
|
||||
map,
|
||||
);
|
||||
|
||||
describe("compareAnswers", () => {
|
||||
it("agrees on mapped ids, clocks of one format and generated values, and counts each", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1);
|
||||
const result = compare(
|
||||
{
|
||||
id: 101,
|
||||
created_at: "2026-10-20T10:00:00Z",
|
||||
token: "3f2a9c8d7e",
|
||||
uuid: "6f1c2b9e-2a3d-4f5e-8a9b-0c1d2e3f4a5b",
|
||||
},
|
||||
{
|
||||
id: 1,
|
||||
created_at: "2026-10-17T18:00:03Z",
|
||||
token: "9b8e7d6c5a",
|
||||
uuid: "0a1b2c3d-4e5f-4a6b-9c7d-8e9f0a1b2c3d",
|
||||
},
|
||||
map,
|
||||
);
|
||||
expect(result.differences).toEqual([]);
|
||||
expect(result.normalised).toEqual({
|
||||
ids: 1,
|
||||
clocks: 1,
|
||||
generated: 2,
|
||||
large_bodies: 0,
|
||||
unobserved_bodies: 0,
|
||||
});
|
||||
});
|
||||
|
||||
it("categorises status, missing and extra fields, values and clock formats", () => {
|
||||
const result = compare(
|
||||
{ name: "A", gone: 1, at: "2026-10-20T10:00:00.123+00:00" },
|
||||
{ name: "B", added: true, at: "2026-10-20T10:00:00Z" },
|
||||
new IdMap(),
|
||||
[409, 204],
|
||||
);
|
||||
expect(result.differences.map((d) => [d.category, d.pointer])).toEqual([
|
||||
["status", ""],
|
||||
["value", "/name"],
|
||||
["missing-field", "/gone"],
|
||||
["clock-format", "/at"],
|
||||
["extra-field", "/added"],
|
||||
]);
|
||||
});
|
||||
|
||||
it("reports array length and order and compares paired items", () => {
|
||||
const result = compare(
|
||||
[
|
||||
{ name: "B", n: 2 },
|
||||
{ name: "A", n: 1 },
|
||||
{ name: "C", n: 3 },
|
||||
],
|
||||
[
|
||||
{ name: "A", n: 1 },
|
||||
{ name: "B", n: 5 },
|
||||
],
|
||||
);
|
||||
expect(result.differences.map((d) => [d.category, d.pointer])).toEqual([
|
||||
["length", ""],
|
||||
["order", ""],
|
||||
["value", "/0/n"],
|
||||
]);
|
||||
});
|
||||
|
||||
it("treats an epoch number as a clock only under a time-named field", () => {
|
||||
expect(
|
||||
compare({ updated: 1760000000 }, { updated: 1760000999 }).differences,
|
||||
).toEqual([]);
|
||||
expect(
|
||||
compare({ size: 1760000000 }, { size: 1760000999 }).differences.map(
|
||||
(d) => d.category,
|
||||
),
|
||||
).toEqual(["value"]);
|
||||
});
|
||||
|
||||
it("compares an object keyed by mapped ids under the environment's keys", () => {
|
||||
const map = new IdMap();
|
||||
map.learn("liveTable", "envTable");
|
||||
expect(
|
||||
compare(
|
||||
{ tables: { liveTable: { name: "T" } } },
|
||||
{ tables: { envTable: { name: "T" } } },
|
||||
map,
|
||||
).differences,
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("compares ordinary fields by name, even one named like a learned id", () => {
|
||||
const map = new IdMap();
|
||||
map.learn("books", "tags");
|
||||
expect(
|
||||
compare({ books: 1, tags: 2 }, { books: 1, tags: 2 }, map).differences,
|
||||
).toEqual([]);
|
||||
// A real rename shows, rather than being taken for a mapped key.
|
||||
expect(
|
||||
compare({ books: 1 }, { tags: 1 }, map).differences.map((d) => [
|
||||
d.category,
|
||||
d.pointer,
|
||||
]),
|
||||
).toEqual([
|
||||
["missing-field", "/books"],
|
||||
["extra-field", "/tags"],
|
||||
]);
|
||||
});
|
||||
|
||||
it("still compares an object keyed by ids under the environment's keys when some ids are the same on both sides", () => {
|
||||
const map = new IdMap();
|
||||
map.learn("101", "2");
|
||||
expect(
|
||||
compare(
|
||||
{ counts: { "1": 3, "101": 5 } },
|
||||
{ counts: { "1": 3, "2": 5 } },
|
||||
map,
|
||||
).differences,
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("reports a reordered set of scalars as order alone and reads epoch array items as clocks", () => {
|
||||
expect(
|
||||
compare(
|
||||
{ access: ["a", "b", "c"] },
|
||||
{ access: ["c", "a", "b"] },
|
||||
).differences.map((d) => [d.category, d.pointer]),
|
||||
).toEqual([["order", "/access"]]);
|
||||
expect(
|
||||
compare({ stats: [[1792450800, 3]] }, { stats: [[1792285200, 3]] })
|
||||
.differences,
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("compares a live body over the replay limit by status only and counts it", () => {
|
||||
const result = compareAnswers(
|
||||
{ status: 200, body: '{"data":[', cut: true },
|
||||
{ status: 200, body: null },
|
||||
new IdMap(),
|
||||
);
|
||||
expect(result.differences).toEqual([]);
|
||||
expect(result.normalised.large_bodies).toBe(1);
|
||||
});
|
||||
|
||||
it("compares a live body the probe did not record by status only and counts it", () => {
|
||||
const unrecorded = { status: 200, body: null, unobserved: true };
|
||||
const result = compareAnswers(
|
||||
unrecorded,
|
||||
{ status: 200, body: { id: 1 } },
|
||||
new IdMap(),
|
||||
);
|
||||
expect(result.differences).toEqual([]);
|
||||
expect(result.normalised.unobserved_bodies).toBe(1);
|
||||
expect(
|
||||
compareAnswers(unrecorded, { status: 404, body: null }, new IdMap())
|
||||
.differences,
|
||||
).toEqual([{ category: "status", pointer: "", live: "200", env: "404" }]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("scalar sets of mapped ids", () => {
|
||||
it("differ only in order once live ids are mapped", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1);
|
||||
map.learn(102, 2);
|
||||
const compared = compareAnswers(
|
||||
{ status: 200, body: { ids: [101, 102] } },
|
||||
{ status: 200, body: { ids: [2, 1] } },
|
||||
map,
|
||||
);
|
||||
expect(compared.differences).toEqual([
|
||||
{ category: "order", pointer: "/ids" },
|
||||
]);
|
||||
expect(compared.normalised.ids).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe("ids two collections reuse", () => {
|
||||
it("compares an entity's own id within the call's collection and any other id by the probe's mapping", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1, "/api/books");
|
||||
map.learn(101, 2, "/api/authors");
|
||||
const answer = (body: JsonValue) => ({ status: 200, body });
|
||||
expect(
|
||||
compareAnswers(answer({ id: 101 }), answer({ id: 2 }), map, {
|
||||
scope: "/api/authors",
|
||||
}).differences,
|
||||
).toEqual([]);
|
||||
expect(
|
||||
compareAnswers(answer({ book_id: 101 }), answer({ book_id: 2 }), map)
|
||||
.differences,
|
||||
).toEqual([
|
||||
{ category: "value", pointer: "/book_id", live: "101", env: "2" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("identity responses two collections reuse", () => {
|
||||
it("compares a create's bare id and created member within the call's collection", () => {
|
||||
const books = new IdMap();
|
||||
books.learn(103, 1, "/api/books");
|
||||
learnIds(103, 3, books, {
|
||||
heldIds: new Set(),
|
||||
scope: "/api/authors",
|
||||
identity: true,
|
||||
});
|
||||
const created = new IdMap();
|
||||
created.learn(103, 1, "/api/books");
|
||||
learnIds({ created: 103 }, { created: 3 }, created, {
|
||||
heldIds: new Set(),
|
||||
scope: "/api/authors",
|
||||
entityAt: "/created",
|
||||
identity: true,
|
||||
});
|
||||
const answer = (body: JsonValue) => ({ status: 201, body });
|
||||
expect(books.env(103)).toBe(1);
|
||||
expect(books.env(103, "/api/authors")).toBe(3);
|
||||
expect(created.env(103, "/api/authors")).toBe(3);
|
||||
expect(
|
||||
compareAnswers(answer(103), answer(3), books, {
|
||||
scope: "/api/authors",
|
||||
identity: true,
|
||||
}).differences,
|
||||
).toEqual([]);
|
||||
expect(
|
||||
compareAnswers(
|
||||
answer({ created: 103 }),
|
||||
answer({ created: 3 }),
|
||||
created,
|
||||
{
|
||||
scope: "/api/authors",
|
||||
at: "/created",
|
||||
identity: true,
|
||||
},
|
||||
).differences,
|
||||
).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("wrapped answers", () => {
|
||||
it("compares the identity of an entity inside an envelope within the call's collection", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1, "/api/books");
|
||||
learnIds(
|
||||
{ data: [{ id: 101, name: "Bob" }] },
|
||||
{ data: [{ id: 2, name: "Bob" }] },
|
||||
map,
|
||||
{ heldIds: new Set(), scope: "/api/authors", entityAt: "/data" },
|
||||
);
|
||||
expect(map.env(101, "/api/authors")).toBe(2);
|
||||
expect(
|
||||
compareAnswers(
|
||||
{ status: 200, body: { data: { id: 101, name: "Bob" } } },
|
||||
{ status: 200, body: { data: { id: 2, name: "Bob" } } },
|
||||
map,
|
||||
{ scope: "/api/authors", at: "/data" },
|
||||
).differences,
|
||||
).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("raw ids a mapping gives to other entities", () => {
|
||||
const answer = (body: JsonValue) => ({ status: 200, body });
|
||||
|
||||
it("reports an entity's own identity equal on both sides that its collection's mapping gives to another entity", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1, "/api/books");
|
||||
map.learn(102, 101, "/api/books");
|
||||
const books = { scope: "/api/books" };
|
||||
expect(
|
||||
compareAnswers(answer({ id: 101 }), answer({ id: 101 }), map, books)
|
||||
.differences,
|
||||
).toEqual([{ category: "value", pointer: "/id", live: "101", env: "101" }]);
|
||||
expect(
|
||||
compareAnswers(answer({ id: 102 }), answer({ id: 101 }), map, books)
|
||||
.differences,
|
||||
).toEqual([]);
|
||||
});
|
||||
|
||||
it("leaves an equal reference equal, as the probe's mapping cannot tell which collection it names", () => {
|
||||
const map = new IdMap();
|
||||
// Another collection's create reassigned live 2, while the seeded project 2 is the same on both sides.
|
||||
map.learn(2, 3);
|
||||
map.learn(5, 2);
|
||||
expect(
|
||||
compareAnswers(
|
||||
answer({ project_id: "2" }),
|
||||
answer({ project_id: "2" }),
|
||||
map,
|
||||
).differences,
|
||||
).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,355 @@
|
||||
import type { JsonObject } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import {
|
||||
isUuid,
|
||||
readClock,
|
||||
type ClockFormat,
|
||||
} from "../../domain/analysisView/environmentStateWriteValues.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import {
|
||||
IDENTITY_FIELD,
|
||||
keyedByLearnedIds,
|
||||
pairElements,
|
||||
translateValue,
|
||||
type IdMap,
|
||||
} from "./environmentCompareIds.js";
|
||||
|
||||
export const DIFFERENCE_CATEGORIES = [
|
||||
"status",
|
||||
"missing-field",
|
||||
"extra-field",
|
||||
"value",
|
||||
"length",
|
||||
"order",
|
||||
"clock-format",
|
||||
"refused",
|
||||
"not-served",
|
||||
"no-tool",
|
||||
"unmapped-id",
|
||||
] as const;
|
||||
export type DifferenceCategory = (typeof DIFFERENCE_CATEGORIES)[number];
|
||||
|
||||
/** One way an environment answer differs from the live one: where, and both values cut short. */
|
||||
export interface Difference {
|
||||
readonly category: DifferenceCategory;
|
||||
readonly pointer: string;
|
||||
readonly live?: string;
|
||||
readonly env?: string;
|
||||
}
|
||||
|
||||
/** Values the comparison treated as equal although they differ: mapped ids, clocks of one format, generated values. */
|
||||
export interface Normalised {
|
||||
ids: number;
|
||||
clocks: number;
|
||||
generated: number;
|
||||
/** Live bodies over the replay limit, compared by status only. */
|
||||
large_bodies: number;
|
||||
/** Live bodies the probe did not record, compared by status only. */
|
||||
unobserved_bodies: number;
|
||||
}
|
||||
|
||||
/** At most this many differences are kept per step. */
|
||||
export const STEP_DIFFERENCE_LIMIT = 20;
|
||||
const VALUE_CHARS = 120;
|
||||
const TIME_FIELD = /(at|time|date|timestamp|created|updated|modified)$/iu;
|
||||
|
||||
const isObject = (value: JsonValue | undefined): value is JsonObject =>
|
||||
typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
|
||||
const shown = (value: JsonValue | undefined): string => {
|
||||
const text = value === undefined ? "<absent>" : JSON.stringify(value);
|
||||
return text.length > VALUE_CHARS ? `${text.slice(0, VALUE_CHARS)}…` : text;
|
||||
};
|
||||
|
||||
const escapePointer = (key: string): string =>
|
||||
key.replaceAll("~", "~0").replaceAll("/", "~1");
|
||||
|
||||
const formatKey = (format: ClockFormat): string =>
|
||||
format.kind === "epoch"
|
||||
? `epoch-${format.unit}`
|
||||
: `iso${format.separator}${String(format.fraction)}${format.zone === "" ? "" : format.zone === "Z" ? "Z" : "offset"}`;
|
||||
|
||||
/** The format of a clock value: an ISO-8601 string, or an epoch number under a time-named field or as an array item. */
|
||||
const clockFormat = (
|
||||
value: JsonValue,
|
||||
key: string | undefined,
|
||||
): string | undefined => {
|
||||
if (typeof value === "number" && key !== undefined && !TIME_FIELD.test(key))
|
||||
return undefined;
|
||||
const clock = readClock(value);
|
||||
return clock === undefined ? undefined : formatKey(clock.format);
|
||||
};
|
||||
|
||||
const characterClasses = (text: string): string =>
|
||||
[
|
||||
...new Set(
|
||||
text
|
||||
.replaceAll(/\d/gu, "0")
|
||||
.replaceAll(/[a-z]/gu, "a")
|
||||
.replaceAll(/[A-Z]/gu, "A"),
|
||||
),
|
||||
]
|
||||
.sort()
|
||||
.join("");
|
||||
|
||||
/** Two values each side generated on its own: both UUIDs, or one length of eight or more with the same character classes and a digit. */
|
||||
const generatedPair = (live: string, env: string): boolean =>
|
||||
(isUuid(live) && isUuid(env)) ||
|
||||
(live.length === env.length &&
|
||||
live.length >= 8 &&
|
||||
/\d/u.test(live) &&
|
||||
/\d/u.test(env) &&
|
||||
characterClasses(live) === characterClasses(env));
|
||||
|
||||
class Differ {
|
||||
readonly differences: Difference[] = [];
|
||||
readonly normalised: Normalised = {
|
||||
ids: 0,
|
||||
clocks: 0,
|
||||
generated: 0,
|
||||
large_bodies: 0,
|
||||
unobserved_bodies: 0,
|
||||
};
|
||||
|
||||
/** The answer's own entities: where they sit (the object, or the array of them, at `at`) and the collection the call names, whose own mapping their identities compare by. `identity` marks a write that answers with only the new id, at `at`. */
|
||||
constructor(
|
||||
readonly map: IdMap,
|
||||
readonly entity: {
|
||||
readonly scope: string | undefined;
|
||||
readonly at: string;
|
||||
readonly identity: boolean;
|
||||
},
|
||||
) {}
|
||||
|
||||
add(difference: Difference): void {
|
||||
this.differences.push(difference);
|
||||
}
|
||||
|
||||
/** Whether a value compares within the call's collection, as an entity's own identity or a create's bare id does; other ids compare by the probe's mapping. */
|
||||
#inCollection(
|
||||
pointer: string,
|
||||
at: { readonly key?: string; readonly own?: boolean },
|
||||
): boolean {
|
||||
return (
|
||||
(this.entity.identity && pointer === this.entity.at) ||
|
||||
(at.own === true && at.key !== undefined && IDENTITY_FIELD.test(at.key))
|
||||
);
|
||||
}
|
||||
|
||||
/** Whether an identity both sides hold verbatim is one its collection's mapping gives to another environment entity; a reference's collection is unknown, so it is not judged. */
|
||||
#reassigned(
|
||||
value: JsonValue,
|
||||
pointer: string,
|
||||
at: { readonly key?: string; readonly own?: boolean },
|
||||
): boolean {
|
||||
if (typeof value !== "string" && typeof value !== "number") return false;
|
||||
if (!this.#inCollection(pointer, at)) return false;
|
||||
const mapped = this.map.env(value, this.entity.scope);
|
||||
return mapped !== undefined && String(mapped) !== String(value);
|
||||
}
|
||||
|
||||
scalar(
|
||||
live: JsonValue,
|
||||
env: JsonValue,
|
||||
pointer: string,
|
||||
at: { readonly key?: string; readonly own?: boolean } = {},
|
||||
): void {
|
||||
if (live === env) {
|
||||
if (this.#reassigned(live, pointer, at))
|
||||
this.add({
|
||||
category: "value",
|
||||
pointer,
|
||||
live: shown(live),
|
||||
env: shown(env),
|
||||
});
|
||||
return;
|
||||
}
|
||||
const { key } = at;
|
||||
if (
|
||||
(typeof live === "string" || typeof live === "number") &&
|
||||
(typeof env === "string" || typeof env === "number")
|
||||
) {
|
||||
const mapped = this.map.env(
|
||||
live,
|
||||
this.#inCollection(pointer, at) ? this.entity.scope : undefined,
|
||||
);
|
||||
if (mapped !== undefined && String(mapped) === String(env)) {
|
||||
this.normalised.ids += 1;
|
||||
return;
|
||||
}
|
||||
const liveClock = clockFormat(live, key);
|
||||
const envClock = clockFormat(env, key);
|
||||
if (liveClock !== undefined && envClock !== undefined) {
|
||||
if (liveClock === envClock) this.normalised.clocks += 1;
|
||||
else
|
||||
this.add({
|
||||
category: "clock-format",
|
||||
pointer,
|
||||
live: shown(live),
|
||||
env: shown(env),
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (
|
||||
typeof live === "string" &&
|
||||
typeof env === "string" &&
|
||||
!this.map.hasLive(live) &&
|
||||
!this.map.hasEnv(env) &&
|
||||
generatedPair(live, env)
|
||||
) {
|
||||
this.normalised.generated += 1;
|
||||
return;
|
||||
}
|
||||
}
|
||||
this.add({
|
||||
category: "value",
|
||||
pointer,
|
||||
live: shown(live),
|
||||
env: shown(env),
|
||||
});
|
||||
}
|
||||
|
||||
array(live: JsonValue[], env: JsonValue[], pointer: string): void {
|
||||
const own = pointer === this.entity.at;
|
||||
if (this.sameItems(live, env, pointer)) return;
|
||||
const pairs = pairElements(live, env);
|
||||
if (live.length !== env.length || pairs.length !== live.length)
|
||||
this.add({
|
||||
category: "length",
|
||||
pointer,
|
||||
live: `${String(live.length)} items`,
|
||||
env: `${String(env.length)} items (${String(pairs.length)} paired)`,
|
||||
});
|
||||
const envOrder = pairs.map(([, right]) => right);
|
||||
if (
|
||||
envOrder.some((right, index) => index > 0 && right < envOrder[index - 1]!)
|
||||
)
|
||||
this.add({ category: "order", pointer });
|
||||
for (const [left, right] of pairs)
|
||||
this.value(live[left]!, env[right]!, `${pointer}/${String(left)}`, {
|
||||
own,
|
||||
});
|
||||
}
|
||||
|
||||
/** Arrays of scalars holding the same items in another order differ only in order, as sets an application returns unordered do. */
|
||||
sameItems(live: JsonValue[], env: JsonValue[], pointer: string): boolean {
|
||||
const scalar = (item: JsonValue): boolean =>
|
||||
item === null || typeof item !== "object";
|
||||
if (live.length !== env.length || !live.every(scalar) || !env.every(scalar))
|
||||
return false;
|
||||
// Live ids compare in the environment's terms, as scalar values do.
|
||||
const mapped = live.map((item) => translateValue(item, this.map));
|
||||
const key = (items: JsonValue[]): string =>
|
||||
JSON.stringify(items.map((item) => JSON.stringify(item)).sort());
|
||||
if (key(mapped) !== key(env)) return false;
|
||||
this.normalised.ids += mapped.filter(
|
||||
(item, index) => item !== live[index],
|
||||
).length;
|
||||
if (JSON.stringify(mapped) !== JSON.stringify(env))
|
||||
this.add({ category: "order", pointer });
|
||||
return true;
|
||||
}
|
||||
|
||||
object(
|
||||
live: JsonObject,
|
||||
env: JsonObject,
|
||||
pointer: string,
|
||||
own = false,
|
||||
): void {
|
||||
const self = own || pointer === this.entity.at;
|
||||
// An object keyed by ids compares each member under the environment's key; any other compares field by field.
|
||||
const keyed = keyedByLearnedIds(live, env, this.map);
|
||||
const envKey = (key: string): string => {
|
||||
const mapped = keyed ? this.map.env(key) : undefined;
|
||||
return mapped !== undefined && Object.hasOwn(env, String(mapped))
|
||||
? String(mapped)
|
||||
: key;
|
||||
};
|
||||
const matched = new Set<string>();
|
||||
for (const [key, value] of Object.entries(live)) {
|
||||
const child = `${pointer}/${escapePointer(key)}`;
|
||||
const other = envKey(key);
|
||||
if (!Object.hasOwn(env, other))
|
||||
this.add({
|
||||
category: "missing-field",
|
||||
pointer: child,
|
||||
live: shown(value),
|
||||
});
|
||||
else {
|
||||
matched.add(other);
|
||||
this.value(value, env[other]!, child, { key, own: self });
|
||||
}
|
||||
}
|
||||
for (const [key, value] of Object.entries(env))
|
||||
if (!matched.has(key))
|
||||
this.add({
|
||||
category: "extra-field",
|
||||
pointer: `${pointer}/${escapePointer(key)}`,
|
||||
env: shown(value),
|
||||
});
|
||||
}
|
||||
|
||||
/** `own` marks an item of the entity array, or with `key` a field of an own entity; a field's own children are not. */
|
||||
value(
|
||||
live: JsonValue,
|
||||
env: JsonValue,
|
||||
pointer: string,
|
||||
at: { readonly key?: string; readonly own?: boolean } = {},
|
||||
): void {
|
||||
if (Array.isArray(live) && Array.isArray(env))
|
||||
this.array(live, env, pointer);
|
||||
else if (isObject(live) && isObject(env))
|
||||
this.object(live, env, pointer, at.own === true && at.key === undefined);
|
||||
else if (
|
||||
Array.isArray(live) ||
|
||||
Array.isArray(env) ||
|
||||
isObject(live) ||
|
||||
isObject(env)
|
||||
)
|
||||
this.add({
|
||||
category: "value",
|
||||
pointer,
|
||||
live: shown(live),
|
||||
env: shown(env),
|
||||
});
|
||||
else this.scalar(live, env, pointer, at);
|
||||
}
|
||||
}
|
||||
|
||||
/** One answer: its status, and its JSON body, its text, or null without one. */
|
||||
export interface Answer {
|
||||
readonly status: number | null;
|
||||
readonly body: JsonValue;
|
||||
/** Set when the body was over the replay limit, so only a prefix is known. */
|
||||
readonly cut?: boolean;
|
||||
/** Set when the probe did not record the body, so only the status is known. */
|
||||
readonly unobserved?: boolean;
|
||||
}
|
||||
|
||||
/** Compare a live answer with the environment's after normalisation. */
|
||||
export const compareAnswers = (
|
||||
live: Answer,
|
||||
env: Answer,
|
||||
map: IdMap,
|
||||
entity: {
|
||||
readonly scope?: string;
|
||||
readonly at?: string;
|
||||
readonly identity?: boolean;
|
||||
} = {},
|
||||
): { readonly differences: Difference[]; readonly normalised: Normalised } => {
|
||||
const differ = new Differ(map, {
|
||||
scope: entity.scope,
|
||||
at: entity.at ?? "",
|
||||
identity: entity.identity === true,
|
||||
});
|
||||
if (live.status !== env.status)
|
||||
differ.add({
|
||||
category: "status",
|
||||
pointer: "",
|
||||
live: String(live.status),
|
||||
env: String(env.status),
|
||||
});
|
||||
if (live.cut === true) differ.normalised.large_bodies += 1;
|
||||
else if (live.unobserved === true) differ.normalised.unobserved_bodies += 1;
|
||||
else differ.value(live.body, env.body, "");
|
||||
return { differences: differ.differences, normalised: differ.normalised };
|
||||
};
|
||||
@@ -0,0 +1,251 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
IdMap,
|
||||
learnIds,
|
||||
pairElements,
|
||||
translatePath,
|
||||
translateValue,
|
||||
} from "./environmentCompareIds.js";
|
||||
|
||||
describe("IdMap", () => {
|
||||
it("keeps the first pairing of either side and counts conflicting ones", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1);
|
||||
map.learn("101", 1);
|
||||
map.learn(101, 2);
|
||||
map.learn(102, 1);
|
||||
expect(map.env(101)).toBe(1);
|
||||
expect(map.env("101")).toBe(1);
|
||||
expect(map.hasEnv(1)).toBe(true);
|
||||
expect(map.size).toBe(1);
|
||||
expect(map.conflicts).toBe(2);
|
||||
});
|
||||
|
||||
it("ignores a pair of equal values", () => {
|
||||
const map = new IdMap();
|
||||
map.learn("same", "same");
|
||||
expect(map.size).toBe(0);
|
||||
});
|
||||
|
||||
it("keeps an equal id in its collection and rejects a second live id for one environment id there", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1, "/api/books");
|
||||
map.learn(101, 101, "/api/authors");
|
||||
map.learn(102, 1, "/api/books");
|
||||
expect(map.env(101, "/api/authors")).toBe(101);
|
||||
expect(map.env(101, "/api/books")).toBe(1);
|
||||
expect(map.env(102, "/api/books")).toBeUndefined();
|
||||
expect(map.env(102)).toBeUndefined();
|
||||
expect(map.conflicts).toBe(1);
|
||||
expect(map.size).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("pairElements", () => {
|
||||
it("pairs objects by the shared unique string field that pairs the most", () => {
|
||||
const live = [
|
||||
{ id: 102, name: "Second", kind: "book" },
|
||||
{ id: 101, name: "First", kind: "book" },
|
||||
];
|
||||
const env = [
|
||||
{ id: 1, name: "First", kind: "book" },
|
||||
{ id: 2, name: "Second", kind: "book" },
|
||||
{ id: 3, name: "Third", kind: "book" },
|
||||
];
|
||||
expect(pairElements(live, env)).toEqual([
|
||||
[0, 1],
|
||||
[1, 0],
|
||||
]);
|
||||
});
|
||||
|
||||
it("pairs by position without a shared key or for scalar items", () => {
|
||||
expect(pairElements([{ n: 1 }, { n: 2 }], [{ n: 3 }])).toEqual([[0, 0]]);
|
||||
expect(pairElements(["a", "b"], ["c", "d"])).toEqual([
|
||||
[0, 0],
|
||||
[1, 1],
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("learnIds", () => {
|
||||
it("maps identifier fields of paired list items and of two single answers", () => {
|
||||
const map = new IdMap();
|
||||
learnIds(
|
||||
[
|
||||
{ id: 102, name: "Second", owner_id: "u-9" },
|
||||
{ id: 101, name: "First", owner_id: "u-9" },
|
||||
],
|
||||
[
|
||||
{ id: 1, name: "First", owner_id: "u-1" },
|
||||
{ id: 2, name: "Second", owner_id: "u-1" },
|
||||
],
|
||||
map,
|
||||
{ heldIds: new Set() },
|
||||
);
|
||||
expect([map.env(101), map.env(102), map.env("u-9")]).toEqual([1, 2, "u-1"]);
|
||||
learnIds(
|
||||
{ data: { uid: "x9", ref: "r-1" } },
|
||||
{ data: { uid: "e3", ref: "r-2" } },
|
||||
map,
|
||||
{ heldIds: new Set(["r-2"]) },
|
||||
);
|
||||
expect(map.env("r-1")).toBe("r-2");
|
||||
expect(map.env("x9")).toBeUndefined();
|
||||
learnIds({ workers: 0 }, { workers: 1 }, map, { heldIds: new Set(["1"]) });
|
||||
expect(map.env(0)).toBeUndefined();
|
||||
});
|
||||
|
||||
it("learns nothing from items that share a key but no value, or differ in count without one", () => {
|
||||
const map = new IdMap();
|
||||
learnIds([{ code: "kelp", id: 1 }], [{ code: "menu", id: 2 }], map, {
|
||||
heldIds: new Set(["menu"]),
|
||||
});
|
||||
learnIds(
|
||||
[{ n: 1, id: 3 }],
|
||||
[
|
||||
{ n: 1, id: 4 },
|
||||
{ n: 2, id: 5 },
|
||||
],
|
||||
map,
|
||||
{ heldIds: new Set() },
|
||||
);
|
||||
expect(map.size).toBe(0);
|
||||
learnIds([{ n: 1, id: 3 }], [{ n: 1, id: 4 }], map, { heldIds: new Set() });
|
||||
expect(map.env(3)).toBe(4);
|
||||
});
|
||||
|
||||
it("learns a scalar identity response at its pointer and ignores any other root scalar", () => {
|
||||
const map = new IdMap();
|
||||
learnIds(103, 3, map, {
|
||||
heldIds: new Set(),
|
||||
scope: "/api/books",
|
||||
identity: true,
|
||||
});
|
||||
learnIds({ created: 104 }, { created: 4 }, map, {
|
||||
heldIds: new Set(),
|
||||
scope: "/api/books",
|
||||
entityAt: "/created",
|
||||
identity: true,
|
||||
});
|
||||
learnIds(7, 8, map, { heldIds: new Set(), scope: "/api/books" });
|
||||
expect(map.env(103, "/api/books")).toBe(3);
|
||||
expect(map.env(104)).toBe(4);
|
||||
expect(map.env(7)).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("learnIds over objects keyed by ids", () => {
|
||||
it("pairs entries by a natural key in their values and maps the keys", () => {
|
||||
const map = new IdMap();
|
||||
learnIds(
|
||||
{
|
||||
data: {
|
||||
liveA: { id: "liveA", name: "Lamp Oil" },
|
||||
liveB: { id: "liveB", name: "Quarry" },
|
||||
},
|
||||
},
|
||||
{
|
||||
data: {
|
||||
envB: { id: "envB", name: "Quarry" },
|
||||
envA: { id: "envA", name: "Lamp Oil" },
|
||||
},
|
||||
},
|
||||
map,
|
||||
{ heldIds: new Set() },
|
||||
);
|
||||
expect([map.env("liveA"), map.env("liveB")]).toEqual(["envA", "envB"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("translateValue", () => {
|
||||
it("replaces mapped live identifiers at any depth in the original value's type", () => {
|
||||
const map = new IdMap();
|
||||
map.learn(101, 1);
|
||||
map.learn("abc", "xyz");
|
||||
expect(
|
||||
translateValue({ ids: [101, "101", "abc"], name: "abc def", n: 5 }, map),
|
||||
).toEqual({ ids: [1, "1", "xyz"], name: "abc def", n: 5 });
|
||||
});
|
||||
});
|
||||
|
||||
describe("translatePath", () => {
|
||||
it("replaces segments that decode to a mapped live id and re-encodes them", () => {
|
||||
const map = new IdMap();
|
||||
map.learn("live@1", "env@1");
|
||||
map.learn(7, 3);
|
||||
expect(translatePath("/api/releases/live%401/deploys/7/", map)).toBe(
|
||||
"/api/releases/env%401/deploys/3/",
|
||||
);
|
||||
expect(translatePath("/api/%E0%A4%A/x", map)).toBe("/api/%E0%A4%A/x");
|
||||
});
|
||||
|
||||
it("leaves a literal route segment in place when that text is a mapped id", () => {
|
||||
const map = new IdMap();
|
||||
map.learn("books", "stored-tag");
|
||||
map.learn(1, 9);
|
||||
expect(
|
||||
translatePath(
|
||||
"/api/books/1",
|
||||
map,
|
||||
() => undefined,
|
||||
(index) => index < 2,
|
||||
),
|
||||
).toBe("/api/books/9");
|
||||
});
|
||||
});
|
||||
|
||||
describe("ids two collections reuse", () => {
|
||||
it("keeps each collection's pairing of a live id and translates a path id by its collection", () => {
|
||||
const map = new IdMap();
|
||||
learnIds([{ id: 101, name: "First" }], [{ id: 1, name: "First" }], map, {
|
||||
heldIds: new Set(),
|
||||
scope: "/api/books",
|
||||
});
|
||||
learnIds([{ id: 101, name: "Bob" }], [{ id: 2, name: "Bob" }], map, {
|
||||
heldIds: new Set(),
|
||||
scope: "/api/authors",
|
||||
});
|
||||
expect(map.env(101, "/api/authors")).toBe(2);
|
||||
expect(map.env(101)).toBe(1);
|
||||
expect(
|
||||
translatePath("/api/authors/101", map, (index) =>
|
||||
index === 2 ? "/api/authors" : undefined,
|
||||
),
|
||||
).toBe("/api/authors/2");
|
||||
});
|
||||
});
|
||||
|
||||
describe("objects keyed by ids that overlap", () => {
|
||||
it("pairs entries by a natural key when some re-seeded keys are reused, and only when every entry pairs", () => {
|
||||
const map = new IdMap();
|
||||
learnIds(
|
||||
{ "2": { name: "A" }, "3": { name: "B" } },
|
||||
{ "1": { name: "A" }, "2": { name: "B" } },
|
||||
map,
|
||||
{ heldIds: new Set() },
|
||||
);
|
||||
expect([map.env("2"), map.env("3")]).toEqual(["1", "2"]);
|
||||
const partial = new IdMap();
|
||||
learnIds(
|
||||
{ "2": { name: "A" }, "3": { name: "C" } },
|
||||
{ "1": { name: "A" }, "2": { name: "B" } },
|
||||
partial,
|
||||
{ heldIds: new Set() },
|
||||
);
|
||||
expect(partial.size).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe("ordinary objects whose fields overlap", () => {
|
||||
it("does not read their fields as ids, even when every value pairs", () => {
|
||||
const map = new IdMap();
|
||||
learnIds(
|
||||
{ current: { name: "A" }, previous: { name: "B" } },
|
||||
{ current: { name: "B" }, next: { name: "A" } },
|
||||
map,
|
||||
{ heldIds: new Set() },
|
||||
);
|
||||
expect(map.size).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,424 @@
|
||||
import type { JsonObject } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import { isUuid } from "../../domain/analysisView/environmentStateWriteValues.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import type { ProbeToolCall } from "./environmentCompareMatch.js";
|
||||
|
||||
/** Field names that hold an identifier: `id`, `uuid`, `guid`, or names ending in `id` or `_id`. */
|
||||
export const ID_FIELD = /^(id|uuid|guid)$|(_id|Id|ID|_uuid|Uuid)$/u;
|
||||
|
||||
const isObject = (value: JsonValue | undefined): value is JsonObject =>
|
||||
typeof value === "object" && value !== null && !Array.isArray(value);
|
||||
|
||||
const isScalarId = (value: JsonValue | undefined): value is string | number =>
|
||||
(typeof value === "string" && value !== "") ||
|
||||
(typeof value === "number" && Number.isFinite(value));
|
||||
|
||||
/** Entity identity fields, whose values an answer's own collection scopes. */
|
||||
export const IDENTITY_FIELD = /^(id|uuid|guid)$/u;
|
||||
|
||||
/**
|
||||
* Live identifier values and the environment's values they stand for, in both directions: one map for the whole probe,
|
||||
* and one per collection for the ids of its own entities, since two collections can reuse a live id for entities the
|
||||
* environment holds under different ids.
|
||||
*/
|
||||
export class IdMap {
|
||||
readonly #toEnv = new Map<string, string | number>();
|
||||
readonly #toLive = new Map<string, string | number>();
|
||||
readonly #scoped = new Map<string, Map<string, string | number>>();
|
||||
readonly #envValues = new Set<string>();
|
||||
#conflicts = 0;
|
||||
|
||||
/**
|
||||
* Pair a live value with an environment value. The first pairing of either side wins. A collection keeps its own
|
||||
* first pairing, including a pair of equal values, and one environment value stands for one live value there.
|
||||
*/
|
||||
learn(live: string | number, env: string | number, scope?: string): void {
|
||||
const liveKey = String(live);
|
||||
const envKey = String(env);
|
||||
if (scope !== undefined && !this.#pairScoped(scope, liveKey, env, envKey))
|
||||
return;
|
||||
if (liveKey === envKey) return;
|
||||
const known = this.#toEnv.get(liveKey);
|
||||
if (known !== undefined) {
|
||||
if (String(known) !== envKey) this.#conflicts += 1;
|
||||
return;
|
||||
}
|
||||
if (this.#toLive.has(envKey)) {
|
||||
this.#conflicts += 1;
|
||||
return;
|
||||
}
|
||||
this.#toEnv.set(liveKey, env);
|
||||
this.#toLive.set(envKey, live);
|
||||
this.#envValues.add(envKey);
|
||||
}
|
||||
|
||||
/** Store one collection's pairing. False when it conflicts with a pairing that collection already kept. */
|
||||
#pairScoped(
|
||||
scope: string,
|
||||
liveKey: string,
|
||||
env: string | number,
|
||||
envKey: string,
|
||||
): boolean {
|
||||
const scoped =
|
||||
this.#scoped.get(scope) ?? new Map<string, string | number>();
|
||||
const known = scoped.get(liveKey);
|
||||
if (known !== undefined) {
|
||||
if (String(known) !== envKey) this.#conflicts += 1;
|
||||
return false;
|
||||
}
|
||||
for (const value of scoped.values()) {
|
||||
if (String(value) === envKey) {
|
||||
this.#conflicts += 1;
|
||||
return false;
|
||||
}
|
||||
}
|
||||
scoped.set(liveKey, env);
|
||||
this.#scoped.set(scope, scoped);
|
||||
this.#envValues.add(envKey);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** The environment's value for a live one: the collection's own when a scope is given and it knows one, else the probe's. */
|
||||
env(live: string | number, scope?: string): string | number | undefined {
|
||||
const liveKey = String(live);
|
||||
return (
|
||||
(scope === undefined
|
||||
? undefined
|
||||
: this.#scoped.get(scope)?.get(liveKey)) ?? this.#toEnv.get(liveKey)
|
||||
);
|
||||
}
|
||||
|
||||
hasLive(value: string | number): boolean {
|
||||
const key = String(value);
|
||||
return (
|
||||
this.#toEnv.has(key) ||
|
||||
[...this.#scoped.values()].some((scoped) => scoped.has(key))
|
||||
);
|
||||
}
|
||||
|
||||
hasEnv(value: string | number): boolean {
|
||||
return this.#envValues.has(String(value));
|
||||
}
|
||||
|
||||
get size(): number {
|
||||
return this.#toEnv.size;
|
||||
}
|
||||
|
||||
get conflicts(): number {
|
||||
return this.#conflicts;
|
||||
}
|
||||
}
|
||||
|
||||
/** A field every object element has as a non-empty string, unique on its side; not an identifier, which differs across sides. */
|
||||
const keyCandidates = (items: readonly JsonObject[]): Set<string> => {
|
||||
const candidates = new Set<string>();
|
||||
const first = items[0];
|
||||
if (first === undefined) return candidates;
|
||||
for (const key of Object.keys(first)) {
|
||||
if (ID_FIELD.test(key)) continue;
|
||||
const values = items.map((item) => item[key]);
|
||||
if (
|
||||
values.every((value) => typeof value === "string" && value !== "") &&
|
||||
new Set(values).size === values.length
|
||||
)
|
||||
candidates.add(key);
|
||||
}
|
||||
return candidates;
|
||||
};
|
||||
|
||||
/** How two arrays pair: by a natural key, by position because no key is shared, or by position although a shared key paired nothing. */
|
||||
interface Pairing {
|
||||
readonly pairs: Array<readonly [number, number]>;
|
||||
readonly by: "key" | "position" | "unmatched";
|
||||
}
|
||||
|
||||
const pairing = (
|
||||
live: readonly JsonValue[],
|
||||
env: readonly JsonValue[],
|
||||
): Pairing => {
|
||||
const byPosition = (by: Pairing["by"]): Pairing => ({
|
||||
pairs: Array.from(
|
||||
{ length: Math.min(live.length, env.length) },
|
||||
(_, index) => [index, index] as const,
|
||||
),
|
||||
by,
|
||||
});
|
||||
if (!live.every(isObject) || !env.every(isObject))
|
||||
return byPosition("position");
|
||||
const liveItems = live as readonly JsonObject[];
|
||||
const envItems = env as readonly JsonObject[];
|
||||
const envKeys = keyCandidates(envItems);
|
||||
const shared = [...keyCandidates(liveItems)].filter((key) =>
|
||||
envKeys.has(key),
|
||||
);
|
||||
let best: Array<readonly [number, number]> = [];
|
||||
for (const key of shared.sort()) {
|
||||
const envIndex = new Map(envItems.map((item, index) => [item[key], index]));
|
||||
const pairs: Array<readonly [number, number]> = [];
|
||||
for (const [index, item] of liveItems.entries()) {
|
||||
const match = envIndex.get(item[key]);
|
||||
if (match !== undefined) pairs.push([index, match]);
|
||||
}
|
||||
if (pairs.length > best.length) best = pairs;
|
||||
}
|
||||
if (best.length > 0) return { pairs: best, by: "key" };
|
||||
return byPosition(shared.length > 0 ? "unmatched" : "position");
|
||||
};
|
||||
|
||||
/**
|
||||
* Pair two arrays' object elements by the natural key that pairs the most, as [live index, env index]; arrays of other
|
||||
* values, or with no shared key, pair by position up to the shorter length.
|
||||
*/
|
||||
export const pairElements = (
|
||||
live: readonly JsonValue[],
|
||||
env: readonly JsonValue[],
|
||||
): Array<readonly [number, number]> => pairing(live, env).pairs;
|
||||
|
||||
/** Whether an object's keys are evidently identifiers: digits or UUIDs, or each the identity of the entry it keys. */
|
||||
const keyedByIds = (
|
||||
entries: readonly (readonly [string, JsonValue])[],
|
||||
): boolean =>
|
||||
entries.every(
|
||||
([key, value]) =>
|
||||
/^\d+$/u.test(key) ||
|
||||
isUuid(key) ||
|
||||
(isObject(value) &&
|
||||
Object.entries(value).some(
|
||||
([field, held]) =>
|
||||
IDENTITY_FIELD.test(field) &&
|
||||
isScalarId(held) &&
|
||||
String(held) === key,
|
||||
)),
|
||||
);
|
||||
|
||||
/** Whether an answer's object is keyed by learned ids, read as learning reads one: a key both sides share needs every key evidently an id, and otherwise each entry is an entity or each key evidently an id. */
|
||||
export const keyedByLearnedIds = (
|
||||
live: JsonObject,
|
||||
env: JsonObject,
|
||||
map: IdMap,
|
||||
): boolean => {
|
||||
const entries = Object.entries(live);
|
||||
if (!entries.some(([key]) => map.hasLive(key))) return false;
|
||||
return entries.some(([key]) => Object.hasOwn(env, key))
|
||||
? keyedByIds(entries)
|
||||
: entries.every(([, value]) => isObject(value)) || keyedByIds(entries);
|
||||
};
|
||||
|
||||
/**
|
||||
* Two objects keyed by ids, whose object values pair by a natural key: each pair maps the live key to the environment's
|
||||
* and is learned from in turn. Returns whether the objects were read that way.
|
||||
*/
|
||||
const learnKeyedEntries = (
|
||||
live: JsonObject,
|
||||
env: JsonObject,
|
||||
map: IdMap,
|
||||
heldIds: ReadonlySet<string>,
|
||||
): boolean => {
|
||||
const liveEntries = Object.entries(live);
|
||||
const envEntries = Object.entries(env);
|
||||
const shared = liveEntries.filter(([key]) => Object.hasOwn(env, key)).length;
|
||||
if (
|
||||
liveEntries.length === 0 ||
|
||||
envEntries.length === 0 ||
|
||||
shared === liveEntries.length ||
|
||||
!liveEntries.every(([, value]) => isObject(value)) ||
|
||||
!envEntries.every(([, value]) => isObject(value))
|
||||
)
|
||||
return false;
|
||||
const paired = pairing(
|
||||
liveEntries.map(([, value]) => value),
|
||||
envEntries.map(([, value]) => value),
|
||||
);
|
||||
// Re-seeded ids can reuse a key for another entity, but ordinary fields overlap too: keys that overlap are read as ids
|
||||
// only when they evidently are ids and every entry pairs.
|
||||
if (
|
||||
paired.by !== "key" ||
|
||||
(shared > 0 &&
|
||||
(paired.pairs.length !== liveEntries.length ||
|
||||
!keyedByIds(liveEntries) ||
|
||||
!keyedByIds(envEntries)))
|
||||
)
|
||||
return false;
|
||||
for (const [left, right] of paired.pairs) {
|
||||
const [liveKey, liveValue] = liveEntries[left]!;
|
||||
const [envKey, envValue] = envEntries[right]!;
|
||||
map.learn(liveKey, envKey);
|
||||
learnIds(liveValue, envValue, map, { heldIds });
|
||||
}
|
||||
return true;
|
||||
};
|
||||
|
||||
/** What learning reads besides the two values: the map it adds to, the ids the state holds, and where the answer's own entities sit. */
|
||||
interface LearnContext {
|
||||
readonly map: IdMap;
|
||||
readonly heldIds: ReadonlySet<string>;
|
||||
readonly scope: string | undefined;
|
||||
/** JSON pointer of the answer's own entity, or of the array holding its entities. */
|
||||
readonly entityAt: string;
|
||||
/** The write answers with the new id alone, at `entityAt`, rather than an entity object. */
|
||||
readonly identity: boolean;
|
||||
}
|
||||
|
||||
/** Two values under one key as a pair of ids, when they are: an identifier field, or outside one only a string the state holds as an id, since a counter that equals an id is not one. */
|
||||
const idPair = (
|
||||
key: string,
|
||||
value: JsonValue,
|
||||
other: JsonValue | undefined,
|
||||
heldIds: ReadonlySet<string>,
|
||||
): readonly [string | number, string | number] | undefined =>
|
||||
isScalarId(value) &&
|
||||
isScalarId(other) &&
|
||||
(ID_FIELD.test(key) ||
|
||||
(typeof value === "string" &&
|
||||
typeof other === "string" &&
|
||||
heldIds.has(other)))
|
||||
? [value, other]
|
||||
: undefined;
|
||||
|
||||
const childPointer = (at: string, key: string | number): string =>
|
||||
`${at}/${String(key).replaceAll("~", "~0").replaceAll("/", "~1")}`;
|
||||
|
||||
const learnAt = (
|
||||
live: JsonValue | undefined,
|
||||
env: JsonValue | undefined,
|
||||
context: LearnContext,
|
||||
place: { readonly at: string; readonly own: boolean },
|
||||
): void => {
|
||||
if (Array.isArray(live) && Array.isArray(env)) {
|
||||
// Items paired only by position stand for the same entities only when nothing could pair them by a key and the
|
||||
// arrays are the same length; otherwise a filtered or reordered list would teach wrong ids.
|
||||
const paired = pairing(live, env);
|
||||
if (
|
||||
paired.by === "unmatched" ||
|
||||
(paired.by === "position" && live.length !== env.length)
|
||||
)
|
||||
return;
|
||||
for (const [left, right] of paired.pairs)
|
||||
learnAt(live[left], env[right], context, {
|
||||
at: childPointer(place.at, left),
|
||||
own: place.at === context.entityAt,
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (!isObject(live) || !isObject(env)) {
|
||||
if (
|
||||
context.identity &&
|
||||
place.at === context.entityAt &&
|
||||
isScalarId(live) &&
|
||||
isScalarId(env)
|
||||
)
|
||||
context.map.learn(live, env, context.scope);
|
||||
return;
|
||||
}
|
||||
if (learnKeyedEntries(live, env, context.map, context.heldIds)) return;
|
||||
const own = place.own || place.at === context.entityAt;
|
||||
for (const [key, value] of Object.entries(live)) {
|
||||
if (!Object.hasOwn(env, key)) continue;
|
||||
const other = env[key];
|
||||
const pair = idPair(key, value, other, context.heldIds);
|
||||
if (pair !== undefined)
|
||||
context.map.learn(
|
||||
...pair,
|
||||
own && IDENTITY_FIELD.test(key) ? context.scope : undefined,
|
||||
);
|
||||
else
|
||||
learnAt(value, other, context, {
|
||||
at: childPointer(place.at, key),
|
||||
own: false,
|
||||
});
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Learn identifier pairs from the live and environment answers to one call: identifier fields, and strings the state holds
|
||||
* as ids, of objects that pair, at any depth. The identity fields of the answer's own entities, the object or the items of
|
||||
* the array at `entityAt` (the root by default), are also learned under `scope`, the collection the call names. A write
|
||||
* whose response is only the new id (`identity`) learns that scalar at `entityAt`.
|
||||
*/
|
||||
export const learnIds = (
|
||||
live: JsonValue | undefined,
|
||||
env: JsonValue | undefined,
|
||||
map: IdMap,
|
||||
known: {
|
||||
/** Ids the environment's state holds, which mark a string outside identifier fields as an id. */
|
||||
readonly heldIds: ReadonlySet<string>;
|
||||
readonly scope?: string;
|
||||
readonly entityAt?: string;
|
||||
readonly identity?: boolean;
|
||||
},
|
||||
): void =>
|
||||
learnAt(
|
||||
live,
|
||||
env,
|
||||
{
|
||||
map,
|
||||
heldIds: known.heldIds,
|
||||
scope: known.scope,
|
||||
entityAt: known.entityAt ?? "",
|
||||
identity: known.identity === true,
|
||||
},
|
||||
{ at: "", own: false },
|
||||
);
|
||||
|
||||
/** A value with every string or number equal to a mapped live identifier replaced by the environment's value, in its own type. */
|
||||
export const translateValue = (value: JsonValue, map: IdMap): JsonValue => {
|
||||
if (Array.isArray(value))
|
||||
return value.map((item) => translateValue(item, map));
|
||||
if (isObject(value))
|
||||
return Object.fromEntries(
|
||||
Object.entries(value).map(([key, item]) => [
|
||||
key,
|
||||
translateValue(item, map),
|
||||
]),
|
||||
);
|
||||
if (!isScalarId(value)) return value;
|
||||
const env = map.env(value);
|
||||
if (env === undefined) return value;
|
||||
return typeof value === "string"
|
||||
? String(env)
|
||||
: typeof env === "number"
|
||||
? env
|
||||
: value;
|
||||
};
|
||||
|
||||
/** The arguments the environment is called with: path values as matched, already in its terms, paging values as sent, and every other value translated. */
|
||||
export const environmentArguments = (
|
||||
call: ProbeToolCall,
|
||||
map: IdMap,
|
||||
): Record<string, JsonValue> =>
|
||||
Object.fromEntries(
|
||||
Object.entries(call.args).map(([name, value]) => [
|
||||
name,
|
||||
call.pathArguments.includes(name) || call.pagingArguments.includes(name)
|
||||
? value
|
||||
: translateValue(value, map),
|
||||
]),
|
||||
);
|
||||
|
||||
/**
|
||||
* A request path with every segment that decodes to a mapped live identifier replaced by the environment's value,
|
||||
* encoded. `scopeAt` names the collection of the segment at a position after the leading slash, whose own mapping wins.
|
||||
* `literalAt` keeps a template's literal segments, so a mapped id cannot rename the route itself.
|
||||
*/
|
||||
export const translatePath = (
|
||||
path: string,
|
||||
map: IdMap,
|
||||
scopeAt: (index: number) => string | undefined = () => undefined,
|
||||
literalAt: (index: number) => boolean = () => false,
|
||||
): string =>
|
||||
path
|
||||
.split("/")
|
||||
.map((segment, index) => {
|
||||
const position = index - 1;
|
||||
if (literalAt(position)) return segment;
|
||||
let decoded = segment;
|
||||
try {
|
||||
decoded = decodeURIComponent(segment);
|
||||
} catch {
|
||||
return segment;
|
||||
}
|
||||
const env =
|
||||
decoded === "" ? undefined : map.env(decoded, scopeAt(position));
|
||||
return env === undefined ? segment : encodeURIComponent(String(env));
|
||||
})
|
||||
.join("/");
|
||||
@@ -0,0 +1,178 @@
|
||||
import { decodedSegment } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import { isPagingParameter } from "../../domain/analysisView/environmentStatePaging.js";
|
||||
import { segmentRule } from "../../domain/analysisView/webNetworkRouteTemplates.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import type {
|
||||
EnvironmentManifest,
|
||||
EnvironmentManifestTool,
|
||||
} from "./environmentSource.js";
|
||||
import { environmentToolArguments } from "./environmentTools.js";
|
||||
|
||||
const FORMAT_RULES: ReadonlySet<string> = new Set([
|
||||
"numeric",
|
||||
"uuid",
|
||||
"hex",
|
||||
"iso-date",
|
||||
]);
|
||||
|
||||
/** One request of a probe capture, as the comparison replays it. */
|
||||
export interface ProbeRequest {
|
||||
readonly method: string;
|
||||
readonly path: string;
|
||||
/** WHATWG query spelling including `?`, or null. */
|
||||
readonly query: string | null;
|
||||
readonly body: JsonValue | undefined;
|
||||
}
|
||||
|
||||
/** A probe request as a call of one manifest tool. */
|
||||
export interface ProbeToolCall {
|
||||
readonly tool: EnvironmentManifestTool;
|
||||
readonly args: Record<string, JsonValue>;
|
||||
/** Argument names of the path values, in path order. */
|
||||
readonly pathArguments: readonly string[];
|
||||
/** Argument names of query values that page a list: counts, never ids. */
|
||||
readonly pagingArguments: readonly string[];
|
||||
/** Query names the tool does not take; the call drops them. */
|
||||
readonly ignoredQuery: readonly string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* A tool's path segments up to `end`. A parameter stays `{}` unless `segments` gives the live value at that position,
|
||||
* decoded once and encoded again, so two parents of one template do not share a child collection, spellings replay
|
||||
* treats as equal name one, and a decoded slash stays inside its segment. An empty parent stays `{}`. A second
|
||||
* encoding stays distinct: decoding again would join it with the value.
|
||||
*/
|
||||
const collectionAt = (
|
||||
tool: EnvironmentManifestTool,
|
||||
end: number,
|
||||
segments?: readonly string[],
|
||||
): string => {
|
||||
const params = new Set(tool.params.map((param) => param.index));
|
||||
return `/${tool.path_template
|
||||
.slice(1)
|
||||
.split("/")
|
||||
.slice(0, end)
|
||||
.map((segment, index) => {
|
||||
if (!params.has(index)) return segment;
|
||||
const value = segments?.[index];
|
||||
return value === undefined || value === ""
|
||||
? "{}"
|
||||
: encodeURIComponent(decodedSegment(value));
|
||||
})
|
||||
.join("/")}`;
|
||||
};
|
||||
|
||||
/** The collection a call of the tool names: its path, less the trailing parameter of an item route. */
|
||||
export const collectionScope = (
|
||||
tool: EnvironmentManifestTool,
|
||||
segments?: readonly string[],
|
||||
): string => {
|
||||
const length = tool.path_template.slice(1).split("/").length;
|
||||
return collectionAt(
|
||||
tool,
|
||||
tool.params.some((param) => param.index === length - 1)
|
||||
? length - 1
|
||||
: length,
|
||||
segments,
|
||||
);
|
||||
};
|
||||
|
||||
/** The collection each path parameter names an entity of: the path before it, by the parameter's position. */
|
||||
export const parameterScopes = (
|
||||
tool: EnvironmentManifestTool,
|
||||
segments?: readonly string[],
|
||||
): ReadonlyMap<number, string> =>
|
||||
new Map(
|
||||
tool.params.map((param) => [
|
||||
param.index,
|
||||
collectionAt(tool, param.index, segments),
|
||||
]),
|
||||
);
|
||||
|
||||
/** Template positions that are literals, so a mapped id is not written over the route. */
|
||||
export const literalPositions = (
|
||||
tool: EnvironmentManifestTool,
|
||||
): ReadonlySet<number> => {
|
||||
const params = new Set(tool.params.map((param) => param.index));
|
||||
return new Set(
|
||||
tool.path_template
|
||||
.slice(1)
|
||||
.split("/")
|
||||
.flatMap((_, index) => (params.has(index) ? [] : [index])),
|
||||
);
|
||||
};
|
||||
|
||||
interface Fit {
|
||||
readonly tool: EnvironmentManifestTool;
|
||||
readonly literals: number;
|
||||
}
|
||||
|
||||
/** How a path fits a tool's template: the literal segments it matches, or undefined when it does not fit. */
|
||||
const fitOf = (
|
||||
tool: EnvironmentManifestTool,
|
||||
segments: readonly string[],
|
||||
): Fit | undefined => {
|
||||
const template = tool.path_template.slice(1).split("/");
|
||||
if (template.length !== segments.length) return undefined;
|
||||
const params = new Map(tool.params.map((param) => [param.index, param]));
|
||||
let literals = 0;
|
||||
for (const [index, part] of template.entries()) {
|
||||
const segment = segments[index]!;
|
||||
const param = params.get(index);
|
||||
if (param === undefined) {
|
||||
if (part !== segment && part !== decodedSegment(segment))
|
||||
return undefined;
|
||||
literals += 1;
|
||||
} else if (
|
||||
// Replay matches a path value decoded, so its format is the decoded value's.
|
||||
FORMAT_RULES.has(param.rule) &&
|
||||
segmentRule(decodedSegment(segment)) !== param.rule
|
||||
)
|
||||
return undefined;
|
||||
else if (segment === "") return undefined;
|
||||
}
|
||||
return { tool, literals };
|
||||
};
|
||||
|
||||
/** The tool of the request's method whose template fits the path with the most literal segments; the manifest's order breaks ties. */
|
||||
export const matchProbeRequest = (
|
||||
manifest: EnvironmentManifest,
|
||||
request: ProbeRequest,
|
||||
): ProbeToolCall | undefined => {
|
||||
const segments = request.path.slice(1).split("/");
|
||||
let best: Fit | undefined;
|
||||
for (const tool of manifest.tools) {
|
||||
if (tool.method !== request.method) continue;
|
||||
const fit = fitOf(tool, segments);
|
||||
if (
|
||||
fit !== undefined &&
|
||||
(best === undefined || fit.literals > best.literals)
|
||||
)
|
||||
best = fit;
|
||||
}
|
||||
if (best === undefined) return undefined;
|
||||
const { tool } = best;
|
||||
const names = environmentToolArguments(tool);
|
||||
const args: Record<string, JsonValue> = {};
|
||||
const pathArguments: string[] = [];
|
||||
for (const param of tool.params) {
|
||||
const name = names.path.get(param.name) ?? param.name;
|
||||
// The environment decodes a path id once. Passing the raw segment keeps `book%2531` from becoming `book1`.
|
||||
args[name] = segments[param.index]!;
|
||||
pathArguments.push(name);
|
||||
}
|
||||
const ignoredQuery: string[] = [];
|
||||
const pagingArguments: string[] = [];
|
||||
if (request.query !== null)
|
||||
for (const [key, value] of new URLSearchParams(request.query.slice(1))) {
|
||||
const name = names.query.get(key);
|
||||
if (name === undefined) ignoredQuery.push(key);
|
||||
else if (!Object.hasOwn(args, name)) {
|
||||
args[name] = value;
|
||||
if (isPagingParameter(key)) pagingArguments.push(name);
|
||||
}
|
||||
}
|
||||
if (names.body !== undefined && request.body !== undefined)
|
||||
args[names.body] = request.body;
|
||||
return { tool, args, pathArguments, pagingArguments, ignoredQuery };
|
||||
};
|
||||
@@ -0,0 +1,177 @@
|
||||
import { z } from "zod";
|
||||
import { isJsonLike } from "../../domain/analysisView/EnvironmentManifestAccumulator.js";
|
||||
import {
|
||||
primaryOrigin,
|
||||
type OriginStats,
|
||||
} from "../../domain/analysisView/environmentManifestOrigin.js";
|
||||
import {
|
||||
mediaTypeEssence,
|
||||
readRecordedRequestLine,
|
||||
type RecordedRequestLine,
|
||||
type RequestLineExclusion,
|
||||
} from "../../domain/analysisView/webNetworkRouteTemplates.js";
|
||||
import { parseExactJson } from "../../domain/exactJsonNumber.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import type { WebNetworkCaptureRecord } from "../../domain/webNetworkCapture.js";
|
||||
import type { Answer } from "./environmentCompareDiff.js";
|
||||
import type { ProbeRequest } from "./environmentCompareMatch.js";
|
||||
import {
|
||||
recordedExchange,
|
||||
replayedBody,
|
||||
type ReplayedBody,
|
||||
} from "./environmentSource.js";
|
||||
|
||||
/** A probe record left uncompared: without a usable request line, or at another origin than the application's, it is not replayed either; without an answer it is replayed, so the state follows, but nothing is compared. */
|
||||
export interface UnobservedRecord {
|
||||
readonly ordinal: number;
|
||||
readonly reason: RequestLineExclusion | "other-origin" | "no-response";
|
||||
}
|
||||
|
||||
/** One probe request with the application's answer, absent when the probe recorded none. */
|
||||
export interface ProbeStep {
|
||||
readonly ordinal: number;
|
||||
readonly request: ProbeRequest;
|
||||
readonly live?: Answer;
|
||||
}
|
||||
|
||||
/** Statuses whose answers carry no body, so a missing body is known to be empty. */
|
||||
const BODYLESS_STATUSES: ReadonlySet<number> = new Set([204, 205, 304]);
|
||||
|
||||
const startedSchema = z.object({ startedDateTime: z.string() });
|
||||
const postDataSchema = z.object({
|
||||
request: z.object({
|
||||
postData: z.object({ mimeType: z.string(), text: z.string() }),
|
||||
}),
|
||||
});
|
||||
|
||||
const methodSchema = z.object({ request: z.object({ method: z.string() }) });
|
||||
|
||||
const startedMs = (record: WebNetworkCaptureRecord): number | undefined => {
|
||||
const started = startedSchema.safeParse(record.reported);
|
||||
const ms = started.success ? Date.parse(started.data.startedDateTime) : NaN;
|
||||
return Number.isFinite(ms) ? ms : undefined;
|
||||
};
|
||||
|
||||
/** Whether a probe makes calls that carry a request body: a POST, PUT or PATCH. */
|
||||
export const sendsBodies = (
|
||||
records: readonly WebNetworkCaptureRecord[],
|
||||
): boolean =>
|
||||
records.some((record) => {
|
||||
const parsed = methodSchema.safeParse(record.reported);
|
||||
return (
|
||||
parsed.success &&
|
||||
["POST", "PUT", "PATCH"].includes(parsed.data.request.method)
|
||||
);
|
||||
});
|
||||
|
||||
/** The JSON a request body holds, its text when it is not JSON, or undefined without one. */
|
||||
export const requestBody = (
|
||||
record: WebNetworkCaptureRecord,
|
||||
): JsonValue | undefined => {
|
||||
const parsed = postDataSchema.safeParse(record.reported);
|
||||
if (!parsed.success) return undefined;
|
||||
const { mimeType, text } = parsed.data.request.postData;
|
||||
return isJsonLike(mediaTypeEssence(mimeType))
|
||||
? (parseExactJson(text) ?? text)
|
||||
: text;
|
||||
};
|
||||
|
||||
/** A body over the replay limit, text or bytes: only a prefix is known, so only the status compares. */
|
||||
export const isCut = (body: ReplayedBody): boolean =>
|
||||
body.kind === "cut" || (body.kind === "base64" && !body.whole);
|
||||
|
||||
/** The application's answer as an environment answer carries it: JSON, text, or null when it is empty; a body the probe did not record is marked. */
|
||||
const liveAnswer = (
|
||||
method: string,
|
||||
status: number,
|
||||
body: ReplayedBody,
|
||||
): Answer => {
|
||||
if (body.kind === "none")
|
||||
return method === "HEAD" || BODYLESS_STATUSES.has(status)
|
||||
? { status, body: null }
|
||||
: { status, body: null, unobserved: true };
|
||||
if (body.kind === "json") return { status, body: body.value };
|
||||
if (body.kind === "base64")
|
||||
return { status, body: body.encoded, cut: isCut(body) };
|
||||
return {
|
||||
status,
|
||||
body: body.text === "" ? null : body.text,
|
||||
cut: isCut(body),
|
||||
};
|
||||
};
|
||||
|
||||
/** The origin a probe calls the application at, chosen as the manifest chooses its own. */
|
||||
const probeOrigin = (
|
||||
lines: ReadonlyMap<number, RecordedRequestLine | null>,
|
||||
): string | null => {
|
||||
const origins = new Map<string, OriginStats>();
|
||||
for (const line of lines.values()) {
|
||||
if (line?.kind !== "line") continue;
|
||||
const stats = origins.get(line.origin) ?? { records: 0, jsonLike: 0 };
|
||||
stats.records += 1;
|
||||
if (isJsonLike(mediaTypeEssence(line.rawMime))) stats.jsonLike += 1;
|
||||
origins.set(line.origin, stats);
|
||||
}
|
||||
return primaryOrigin(origins);
|
||||
};
|
||||
|
||||
/** The probe's requests in the order they were made, each with the application's answer, and the records left uncompared. */
|
||||
export const probeRequests = (
|
||||
records: readonly WebNetworkCaptureRecord[],
|
||||
): { steps: ProbeStep[]; unobserved: UnobservedRecord[] } => {
|
||||
const steps: ProbeStep[] = [];
|
||||
const unobserved: UnobservedRecord[] = [];
|
||||
// Without a start time for every record the probe keeps its recorded order, as state extraction does.
|
||||
const starts = new Map(
|
||||
records.map((record) => [record.ordinal, startedMs(record)]),
|
||||
);
|
||||
const timed = [...starts.values()].every((ms) => ms !== undefined);
|
||||
const ordered = [...records].sort(
|
||||
(left, right) =>
|
||||
(timed ? starts.get(left.ordinal)! - starts.get(right.ordinal)! : 0) ||
|
||||
left.ordinal - right.ordinal,
|
||||
);
|
||||
const lines = new Map(
|
||||
records.map((record) => [
|
||||
record.ordinal,
|
||||
readRecordedRequestLine(record.reported, true, () => true),
|
||||
]),
|
||||
);
|
||||
// Another origin's request goes to some other server, so it neither calls nor changes the environment.
|
||||
const origin = probeOrigin(lines);
|
||||
for (const record of ordered) {
|
||||
const line = lines.get(record.ordinal);
|
||||
const exchange = recordedExchange(record);
|
||||
const { ordinal } = record;
|
||||
if (line?.kind === "line" && line.origin !== origin)
|
||||
unobserved.push({ ordinal, reason: "other-origin" });
|
||||
else if (line?.kind !== "line" || exchange === undefined)
|
||||
unobserved.push({
|
||||
ordinal,
|
||||
reason: line?.kind === "excluded" ? line.reason : "request-unavailable",
|
||||
});
|
||||
else {
|
||||
const request = {
|
||||
method: line.method,
|
||||
path: exchange.path,
|
||||
query: exchange.query,
|
||||
body: requestBody(record),
|
||||
};
|
||||
// A request whose answer was lost may still have changed the application, so it is made but not compared.
|
||||
if (exchange.status === null) {
|
||||
unobserved.push({ ordinal, reason: "no-response" });
|
||||
steps.push({ ordinal, request });
|
||||
} else
|
||||
steps.push({
|
||||
ordinal,
|
||||
request,
|
||||
live: liveAnswer(
|
||||
line.method,
|
||||
exchange.status,
|
||||
replayedBody(exchange),
|
||||
),
|
||||
});
|
||||
}
|
||||
}
|
||||
return { steps, unobserved };
|
||||
};
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
type RecordedExchange,
|
||||
} from "./environmentSource.js";
|
||||
import {
|
||||
pathValueMatches,
|
||||
replayEnvironmentTool,
|
||||
selectReplayExchange,
|
||||
} from "./environmentReplay.js";
|
||||
@@ -114,6 +115,16 @@ describe("replay argument mapping and query admission", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("pathValueMatches", () => {
|
||||
it("matches one decode of either side and not a second decode of both", () => {
|
||||
expect(pathValueMatches("2", "%32")).toBe(true);
|
||||
expect(pathValueMatches("%32", "2")).toBe(true);
|
||||
expect(pathValueMatches("book1", "book%2531")).toBe(false);
|
||||
expect(pathValueMatches("book%2531", "book1")).toBe(false);
|
||||
expect(pathValueMatches("book%31", "book%2531")).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("replayEnvironmentTool selection", () => {
|
||||
it("replays the record whose path segments equal the supplied parameters as an exact match", () => {
|
||||
const result = replay("get_users_by_id", { id: "2" });
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { decodedSegment } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import type { AnalysisError } from "../../domain/analysisErrorBase.js";
|
||||
import { AnalysisInputError } from "../../domain/analysisErrorCore.js";
|
||||
import { pathSegmentMatches } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import type { JsonValue } from "../../domain/jsonValue.js";
|
||||
import {
|
||||
iterateUrlQueryParameters,
|
||||
@@ -48,9 +48,9 @@ interface MatchScore {
|
||||
readonly exact: boolean;
|
||||
}
|
||||
|
||||
/** Replay's path rule: a supplied value matches a recorded segment equal to it verbatim or after percent-decoding. */
|
||||
/** Replay matches verbatim segments or either spelling decoded once. */
|
||||
export const pathValueMatches = (segment: string, value: string): boolean =>
|
||||
segment === value || decodedSegment(segment) === value;
|
||||
pathSegmentMatches(segment, value);
|
||||
|
||||
/** Count how many supplied path and query values the recorded request line satisfies. */
|
||||
const scoreExchange = (
|
||||
|
||||
@@ -171,7 +171,8 @@ const schemaInputError = (
|
||||
})),
|
||||
);
|
||||
|
||||
const recordedExchange = (
|
||||
/** One capture record reduced to its request line and retained response body, or undefined without a request line. */
|
||||
export const recordedExchange = (
|
||||
record: WebNetworkCaptureRecord,
|
||||
): RecordedExchange | undefined => {
|
||||
const line = readRecordedRequestLine(record.reported, true, () => true);
|
||||
@@ -261,6 +262,10 @@ interface CaptureFacts {
|
||||
readonly records: readonly WebNetworkCaptureRecord[];
|
||||
readonly captureKind: EnvironmentCaptureKind;
|
||||
readonly captureSha256: string | null;
|
||||
/** Whether the capture recorded response bodies: a browser scenario does only when they were selected. */
|
||||
readonly responseBodies: boolean;
|
||||
/** Whether the capture recorded request bodies, likewise. */
|
||||
readonly requestBodies: boolean;
|
||||
}
|
||||
|
||||
/** Issue paths of a capture payload refusal start here. */
|
||||
@@ -271,6 +276,7 @@ const parseScenarioCapture = (
|
||||
operation: string,
|
||||
normalized: unknown,
|
||||
parameters: unknown,
|
||||
purpose: "manifest" | "replay",
|
||||
): Result<CaptureFacts, AnalysisError> => {
|
||||
const scenario = browserScenarioCaptureSchema.safeParse(normalized);
|
||||
if (!scenario.success)
|
||||
@@ -289,9 +295,11 @@ const parseScenarioCapture = (
|
||||
.object({ scenario_sha256: z.string() })
|
||||
.safeParse(parameters);
|
||||
return ok({
|
||||
records: scenarioCaptureRecords(scenario.data, "replay").records,
|
||||
records: scenarioCaptureRecords(scenario.data, purpose).records,
|
||||
captureKind: "browser-scenario",
|
||||
captureSha256: definition.success ? definition.data.scenario_sha256 : null,
|
||||
responseBodies: scenario.data.scenario.network_content.response_body,
|
||||
requestBodies: scenario.data.scenario.network_content.request_body,
|
||||
});
|
||||
};
|
||||
|
||||
@@ -340,17 +348,21 @@ const parseHarCapture = (
|
||||
records: capture.data.records,
|
||||
captureKind: "web-network-capture",
|
||||
captureSha256: capture.data.artifact.sha256,
|
||||
responseBodies: true,
|
||||
requestBodies: true,
|
||||
});
|
||||
};
|
||||
|
||||
/**
|
||||
* Authenticate the evidence file and require complete capture Evidence of
|
||||
* either kind. Only an operation no capture schema owns blames the operation;
|
||||
* a recognized operation's payload failures name the payload fields.
|
||||
* a recognized operation's payload failures name the payload fields. A browser
|
||||
* capture's request bodies are kept only for the manifest purpose.
|
||||
*/
|
||||
const parseCaptureEvidence = (
|
||||
export const parseCaptureEvidence = (
|
||||
operation: string,
|
||||
value: unknown,
|
||||
purpose: "manifest" | "replay" = "replay",
|
||||
): Result<CaptureFacts & { readonly evidenceId: string }, AnalysisError> => {
|
||||
const evidence = parseEvidenceFile(operation, "evidence", value);
|
||||
if (!evidence.ok) return evidence;
|
||||
@@ -360,7 +372,12 @@ const parseCaptureEvidence = (
|
||||
);
|
||||
const capture =
|
||||
evidence.value.operation === "capture_browser_scenario"
|
||||
? parseScenarioCapture(operation, normalized, evidence.value.parameters)
|
||||
? parseScenarioCapture(
|
||||
operation,
|
||||
normalized,
|
||||
evidence.value.parameters,
|
||||
purpose,
|
||||
)
|
||||
: evidence.value.operation === "inspect_web_network_capture"
|
||||
? parseHarCapture(operation, normalized)
|
||||
: err(
|
||||
|
||||
@@ -62,6 +62,51 @@ const refusal = (
|
||||
return projectAnalysisError(result.error);
|
||||
};
|
||||
|
||||
/** A list with one proved whole line keeps an entity added to an empty later page off that page. */
|
||||
const expectAddedEntityStaysOffPage = (
|
||||
query: string,
|
||||
args: Readonly<Record<string, unknown>>,
|
||||
membership: "recorded" | "query" = "query",
|
||||
): void => {
|
||||
const alpha = { id: 1, name: "Alpha" };
|
||||
const serving = servingFrom([
|
||||
["GET", "/api/orgs/1/books", [alpha]],
|
||||
["GET", "/api/orgs/1/books/1", alpha],
|
||||
["GET", `/api/orgs/2/books?${query}`, []],
|
||||
]);
|
||||
expect(serving.store.state.views.get("get_orgs_books_by_id")).toMatchObject({
|
||||
shape: "list",
|
||||
membership: "recorded",
|
||||
});
|
||||
load(serving, (entities) => [
|
||||
...entities,
|
||||
{
|
||||
type: "/api/orgs/{numeric}/books",
|
||||
collection: "/api/orgs/2/books",
|
||||
id: "9",
|
||||
fields: { id: 9, name: "Added" },
|
||||
},
|
||||
]);
|
||||
expect(
|
||||
answer(serving, "get_orgs_books_by_id", { id: "2", ...args }),
|
||||
).toMatchObject({
|
||||
body: [],
|
||||
state: { source: "state", membership, listed: 0 },
|
||||
});
|
||||
// Without a proved whole line here, the argument-less call keeps recorded members.
|
||||
expect(answer(serving, "get_orgs_books_by_id", { id: "2" })).toMatchObject({
|
||||
body: [],
|
||||
state:
|
||||
membership === "query"
|
||||
? { source: "state", membership: "recorded", listed: 0 }
|
||||
: { source: "recording", reason: "unrecorded-request" },
|
||||
});
|
||||
};
|
||||
|
||||
/** A list with one proved whole line keeps an entity added to an empty start page off that page. */
|
||||
const expectAddedEntityStaysOffStartPage = (name: string): void =>
|
||||
expectAddedEntityStaysOffPage(`${name}=1`, { [name]: "1" });
|
||||
|
||||
/** Replace the served state with the initial one changed by `edit`. */
|
||||
const load = (
|
||||
serving: ReturnType<typeof servingOf>,
|
||||
@@ -182,6 +227,21 @@ describe("item answers", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("loaded later pages", () => {
|
||||
it.each(["offset", "start", "_start"])(
|
||||
"skips an entity added under an empty ?%s=1 page without whole-collection evidence",
|
||||
expectAddedEntityStaysOffStartPage,
|
||||
);
|
||||
|
||||
it("does not list an entity added under an empty ?page=2&per_page=1 page", () => {
|
||||
expectAddedEntityStaysOffPage(
|
||||
"page=2&per_page=1",
|
||||
{ page: "2", per_page: "1" },
|
||||
"recorded",
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("list answers", () => {
|
||||
it("lists every entity of the collection in state order when membership is all", () => {
|
||||
const serving = servingOf();
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { stateBodyJson } from "../../domain/analysisView/EnvironmentStateAccumulator.js";
|
||||
import {
|
||||
collectionMatch,
|
||||
collectionPathOf,
|
||||
decodedSegment,
|
||||
allLines,
|
||||
@@ -13,6 +14,7 @@ import {
|
||||
type JsonObject,
|
||||
type RecordedViewReason,
|
||||
type StateEntity,
|
||||
type SuppliedSegment,
|
||||
type StateItemView,
|
||||
type StateListView,
|
||||
type StateView,
|
||||
@@ -48,10 +50,8 @@ import {
|
||||
callQuery,
|
||||
callSegments,
|
||||
checkedLine,
|
||||
collectionMatch,
|
||||
queryLine,
|
||||
unfilteredLine,
|
||||
type CallSegment,
|
||||
} from "./environmentStateCallLines.js";
|
||||
import type { EnvironmentStateStore } from "./environmentStateStore.js";
|
||||
|
||||
@@ -182,7 +182,7 @@ export const stateAnswer = (
|
||||
|
||||
/** An item call's collection segments and identity value, kept as supplied rather than joined and re-split. */
|
||||
export interface ItemCall {
|
||||
readonly segments: readonly CallSegment[];
|
||||
readonly segments: readonly SuppliedSegment[];
|
||||
readonly id: string;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
import {
|
||||
allLines,
|
||||
collectionPathOf,
|
||||
segmentsOf,
|
||||
collectionMatch,
|
||||
type SuppliedSegment,
|
||||
type StateListView,
|
||||
} from "../../domain/analysisView/environmentStateRules.js";
|
||||
import { pagesOnly } from "../../domain/analysisView/environmentStatePaging.js";
|
||||
import {
|
||||
pathValueMatches,
|
||||
selectReplayExchange,
|
||||
selectsRecordedLine,
|
||||
type ReplayArguments,
|
||||
@@ -18,20 +18,14 @@ import type {
|
||||
} from "./environmentSource.js";
|
||||
import { environmentToolArguments } from "./environmentTools.js";
|
||||
|
||||
/** One collection segment an item call names: a template literal, or a supplied path value. */
|
||||
export interface CallSegment {
|
||||
readonly text: string;
|
||||
readonly supplied: boolean;
|
||||
}
|
||||
|
||||
/** A call's path segments: template literals and supplied values, or undefined when a path argument is missing. */
|
||||
export const callSegments = (
|
||||
tool: EnvironmentManifestTool,
|
||||
args: ReplayArguments,
|
||||
): CallSegment[] | undefined => {
|
||||
): SuppliedSegment[] | undefined => {
|
||||
const names = environmentToolArguments(tool).path;
|
||||
const params = new Map(tool.params.map((param) => [param.index, param]));
|
||||
const segments: CallSegment[] = [];
|
||||
const segments: SuppliedSegment[] = [];
|
||||
const template = tool.path_template
|
||||
.slice(tool.path_template.indexOf("/") + 1)
|
||||
.split("/");
|
||||
@@ -48,23 +42,6 @@ export const callSegments = (
|
||||
return segments;
|
||||
};
|
||||
|
||||
/** How many supplied segments a held collection matches verbatim, when it matches the call under replay's path rule. */
|
||||
export const collectionMatch = (
|
||||
collection: string,
|
||||
segments: readonly CallSegment[],
|
||||
): number | undefined => {
|
||||
const held = segmentsOf(collection);
|
||||
if (held.length !== segments.length) return undefined;
|
||||
let verbatim = 0;
|
||||
for (const [index, { text, supplied }] of segments.entries()) {
|
||||
const segment = held[index]!;
|
||||
if (supplied ? !pathValueMatches(segment, text) : segment !== text)
|
||||
return undefined;
|
||||
if (supplied && segment === text) verbatim += 1;
|
||||
}
|
||||
return verbatim;
|
||||
};
|
||||
|
||||
/** The checked recorded line a list or document call names exactly, chosen among them as replay chooses; later or filtered recordings never qualify. */
|
||||
export const checkedLine = (
|
||||
source: EnvironmentSource,
|
||||
|
||||
@@ -1,6 +1,9 @@
|
||||
import { collectionPathOf } from "../../domain/analysisView/environmentStateRules.js";
|
||||
import {
|
||||
collectionPathOf,
|
||||
selectedCollection,
|
||||
} from "../../domain/analysisView/environmentStateRules.js";
|
||||
import type { StateServing } from "./environmentStateAnswer.js";
|
||||
import { callSegments, collectionMatch } from "./environmentStateCallLines.js";
|
||||
import { callSegments } from "./environmentStateCallLines.js";
|
||||
import type { ReplayArguments } from "./environmentReplay.js";
|
||||
import type { EnvironmentManifestTool } from "./environmentSource.js";
|
||||
|
||||
@@ -31,17 +34,5 @@ export const createdCollection = (
|
||||
if (segments === undefined || segments.some(({ text }) => text.length === 0))
|
||||
return undefined;
|
||||
// The spelling the call supplies verbatim wins, as item calls choose their entity, then the first known: held ones lead.
|
||||
let held: string | undefined;
|
||||
let verbatim = -1;
|
||||
for (const collection of creatableCollectionsOf(serving, type)) {
|
||||
const matched = collectionMatch(collection, segments);
|
||||
if (matched !== undefined && matched > verbatim) {
|
||||
held = collection;
|
||||
verbatim = matched;
|
||||
}
|
||||
}
|
||||
return (
|
||||
held ??
|
||||
`/${segments.map(({ text, supplied }) => (supplied ? encodeURIComponent(text) : text)).join("/")}`
|
||||
);
|
||||
return selectedCollection(segments, creatableCollectionsOf(serving, type));
|
||||
};
|
||||
@@ -22,8 +22,10 @@ import {
|
||||
} from "../../../tests/fixtures/environmentStateWrites.js";
|
||||
import type { TimedCaptureRow } from "../../../tests/fixtures/analysisView.js";
|
||||
import { seededUuid } from "../../domain/analysisView/environmentStateWriteValues.js";
|
||||
import type { StateWritePlan } from "../../domain/analysisView/environmentStateWriteRules.js";
|
||||
import type { ReplayArguments } from "./environmentReplay.js";
|
||||
import { servedCatalog } from "./environmentState.js";
|
||||
import { describeWrite } from "./environmentStateWriteDescription.js";
|
||||
import {
|
||||
environmentToolArguments,
|
||||
environmentToolDefinitions,
|
||||
@@ -474,6 +476,41 @@ describe("served catalog", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("quotes body member names and omits one that exceeds the sample budget", () => {
|
||||
const plan: StateWritePlan = {
|
||||
tool: "post_books",
|
||||
effect: "create",
|
||||
target: "/api/books",
|
||||
served: true,
|
||||
reason: null,
|
||||
exemplar_ordinal: null,
|
||||
observation: null,
|
||||
response: null,
|
||||
body: {
|
||||
accepted: ["name, title", "note\nline"],
|
||||
fixed: [],
|
||||
required: [],
|
||||
identity: null,
|
||||
kinds: [],
|
||||
},
|
||||
rules: [],
|
||||
collection: null,
|
||||
insert: null,
|
||||
merge: null,
|
||||
checked_record_ordinals: [],
|
||||
contradicted_by: null,
|
||||
};
|
||||
expect(describeWrite(plan)).toContain(
|
||||
'body members "name, title", "note\\nline"',
|
||||
);
|
||||
const long = describeWrite({
|
||||
...plan,
|
||||
body: { ...plan.body!, accepted: ["m".repeat(2000)] },
|
||||
});
|
||||
expect(long).not.toContain("m".repeat(2000));
|
||||
expect(Buffer.byteLength(long ?? "", "utf8")).toBeLessThan(1200);
|
||||
});
|
||||
|
||||
it("quotes the member names a write's description lists and samples a long one", () => {
|
||||
const long = "n".repeat(90_000);
|
||||
const item = (id: number, name: string) => ({
|
||||
@@ -1105,6 +1142,67 @@ describe("paged lines", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("does not list an entity a confirmed create adds on an empty ?page=2&per_page=1 page", () => {
|
||||
const alpha = { id: 1, name: "Alpha", org: "1" };
|
||||
const beta = { id: 2, name: "Beta", org: "2" };
|
||||
const books = servedFrom([
|
||||
["GET", "/api/orgs/1/books", [alpha]],
|
||||
["GET", "/api/orgs/1/books/1", alpha],
|
||||
["GET", "/api/orgs/2/books?page=2&per_page=1", []],
|
||||
["POST", "/api/orgs/2/books", beta, 201, { name: "Beta" }],
|
||||
["GET", "/api/orgs/2/books/2", beta],
|
||||
]);
|
||||
const created = books.answer("post_orgs_books_by_id", {
|
||||
id: "2",
|
||||
body: { name: "Gamma" },
|
||||
}).body as Book;
|
||||
expect(created).toMatchObject({ name: "Gamma" });
|
||||
expect(books.answer("get_orgs_books_by_id", { id: "2" })).toMatchObject({
|
||||
body: [],
|
||||
state: { source: "recording", reason: "unrecorded-request" },
|
||||
});
|
||||
expect(
|
||||
books.answer("get_orgs_books_by_id", {
|
||||
id: "2",
|
||||
page: "2",
|
||||
per_page: "1",
|
||||
}),
|
||||
).toMatchObject({
|
||||
body: [],
|
||||
state: { source: "state", membership: "recorded", listed: 0 },
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["offset", "start", "_start"])(
|
||||
"does not list an entity a confirmed create adds on an empty ?%s=1 page",
|
||||
(name) => {
|
||||
const alpha = { id: 1, name: "Alpha", org: "1" };
|
||||
const beta = { id: 2, name: "Beta", org: "2" };
|
||||
const books = servedFrom([
|
||||
["GET", "/api/orgs/1/books", [alpha]],
|
||||
["GET", "/api/orgs/1/books/1", alpha],
|
||||
["GET", `/api/orgs/2/books?${name}=1`, []],
|
||||
["POST", "/api/orgs/2/books", beta, 201, { name: "Beta" }],
|
||||
["GET", "/api/orgs/2/books/2", beta],
|
||||
]);
|
||||
const created = books.answer("post_orgs_books_by_id", {
|
||||
id: "2",
|
||||
body: { name: "Gamma" },
|
||||
}).body as Book;
|
||||
expect(created).toMatchObject({ name: "Gamma" });
|
||||
expect(books.answer("get_orgs_books_by_id", { id: "2" })).toMatchObject({
|
||||
body: [],
|
||||
state: { source: "state", membership: "recorded", listed: 0 },
|
||||
});
|
||||
expect(
|
||||
books.answer("get_orgs_books_by_id", { id: "2", [name]: "1" }),
|
||||
).toMatchObject({
|
||||
body: [],
|
||||
state: { source: "state", membership: "query", listed: 0 },
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("answers a zero limit a whole line recorded empty from that recording once a create fills its collection", () => {
|
||||
const alpha = { id: 1, name: "Alpha", org: "1" };
|
||||
const beta = { id: 2, name: "Beta", org: "2" };
|
||||
|
||||
@@ -26,6 +26,7 @@ import {
|
||||
spelledLike,
|
||||
} from "../../domain/analysisView/environmentStateWriteValues.js";
|
||||
import {
|
||||
collectionMatch,
|
||||
isObject,
|
||||
scalarText,
|
||||
type JsonObject,
|
||||
@@ -40,7 +41,7 @@ import {
|
||||
type ReplayArguments,
|
||||
} from "./environmentReplay.js";
|
||||
import type { EnvironmentManifestTool } from "./environmentSource.js";
|
||||
import { callSegments, collectionMatch } from "./environmentStateCallLines.js";
|
||||
import { callSegments } from "./environmentStateCallLines.js";
|
||||
import {
|
||||
itemCall,
|
||||
missingAnswer,
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { basename } from "node:path";
|
||||
import { absoluteCallerPath } from "../filesystem/callerPath.js";
|
||||
import type { EnvironmentExportResult } from "../application/environment/environmentExport.js";
|
||||
import type { EnvironmentSource } from "../application/environment/environmentSource.js";
|
||||
@@ -300,6 +301,77 @@ const evidenceOption = z
|
||||
"Parent capture Evidence JSON path (inspect-web-network-capture or capture-browser-scenario output) the manifest cites",
|
||||
);
|
||||
|
||||
interface CompareArguments {
|
||||
readonly manifest: string;
|
||||
readonly evidence: string;
|
||||
readonly probes: readonly string[];
|
||||
}
|
||||
|
||||
/** Read the environment and every probe, then compare; a refusal is the command's output. */
|
||||
const compareEnvironmentCommand = async (
|
||||
input: CompareArguments,
|
||||
): Promise<JsonValue> => {
|
||||
const read = (path: string) =>
|
||||
parseCliJsonInput(path, CLI_COMMANDS.environmentCompare, undefined, {
|
||||
snapshot: true,
|
||||
});
|
||||
const manifest = await read(input.manifest);
|
||||
if (!manifest.ok) return manifest.error;
|
||||
const evidence = await read(input.evidence);
|
||||
if (!evidence.ok) return evidence.error;
|
||||
const probes = [];
|
||||
for (const path of input.probes) {
|
||||
const probe = await read(path);
|
||||
if (!probe.ok) return probe.error;
|
||||
probes.push({ label: basename(path), value: probe.value });
|
||||
}
|
||||
const { loadEnvironmentSource } =
|
||||
await import("../application/environment/environmentSource.js");
|
||||
const source = loadEnvironmentSource(
|
||||
manifest.value,
|
||||
evidence.value,
|
||||
CLI_COMMANDS.environmentCompare,
|
||||
);
|
||||
if (!source.ok) return cliError(source.error);
|
||||
const { compareEnvironment } =
|
||||
await import("../application/environment/environmentCompare.js");
|
||||
const report = compareEnvironment({
|
||||
source: source.value,
|
||||
captureValue: evidence.value,
|
||||
probes,
|
||||
operation: CLI_COMMANDS.environmentCompare,
|
||||
});
|
||||
return report.ok
|
||||
? (report.value as unknown as JsonValue)
|
||||
: cliError(report.error);
|
||||
};
|
||||
|
||||
/** Register the comparison of an environment with probe captures of its application. */
|
||||
const registerCompareCommand = (cli: CliInstance, logger: Logger): void => {
|
||||
cli.command(CLI_COMMANDS.environmentCompare, {
|
||||
description:
|
||||
"Compare an environment with probe captures of the application it was captured from: each probe is one call sequence recorded against a freshly reset application, answered by the environment from a reset state, with agreement reported per step, tool and probe",
|
||||
args: z.object({ manifest: manifestArgument }),
|
||||
options: z.object({
|
||||
evidence: evidenceOption,
|
||||
probe: z
|
||||
.array(z.string())
|
||||
.min(1)
|
||||
.describe(
|
||||
"Probe capture Evidence JSON path (inspect-web-network-capture or capture-browser-scenario output) of one call sequence made against a freshly reset application; repeat once per sequence",
|
||||
),
|
||||
}),
|
||||
run: ({ args, options }) =>
|
||||
logCliCommand(logger, CLI_COMMANDS.environmentCompare, () =>
|
||||
compareEnvironmentCommand({
|
||||
manifest: args.manifest,
|
||||
evidence: options.evidence,
|
||||
probes: options.probe,
|
||||
}),
|
||||
),
|
||||
});
|
||||
};
|
||||
|
||||
/** Register the one-shot bundle export command. */
|
||||
const registerExportCommand = (cli: CliInstance, logger: Logger): void => {
|
||||
cli.command(CLI_COMMANDS.environmentExport, {
|
||||
@@ -348,6 +420,7 @@ export const registerEnvironmentCommands = (
|
||||
environment: Readonly<NodeJS.ProcessEnv>,
|
||||
): void => {
|
||||
registerExportCommand(cli, logger);
|
||||
registerCompareCommand(cli, logger);
|
||||
cli.command(CLI_COMMANDS.environmentServe, {
|
||||
description:
|
||||
"Serve an environment manifest as an agent environment over streamable HTTP until SIGINT or SIGTERM: read tools, and write tools whose effect the capture confirmed, answer from the state the capture observed, derived at startup, or every tool replays retained capture records with --replay-only or when the state is unavailable",
|
||||
|
||||
@@ -136,6 +136,7 @@ export const CLI_COMMANDS = Object.freeze({
|
||||
evidenceExport: "evidence-export",
|
||||
environmentServe: "environment-serve",
|
||||
environmentExport: "environment-export",
|
||||
environmentCompare: "environment-compare",
|
||||
compare: "compare",
|
||||
captureProcess: "capture-process",
|
||||
compareProcessCaptures: "compare-process-captures",
|
||||
|
||||
@@ -20,6 +20,10 @@ import {
|
||||
toolName,
|
||||
toolNameRegistry,
|
||||
} from "./environmentManifestNames.js";
|
||||
import {
|
||||
primaryOrigin,
|
||||
type OriginStats,
|
||||
} from "./environmentManifestOrigin.js";
|
||||
import {
|
||||
applyTemplateRules,
|
||||
SEGMENT_LIMITED_RULES,
|
||||
@@ -189,7 +193,7 @@ const hasParam = (parts: readonly MutablePart[]): boolean =>
|
||||
export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumulator {
|
||||
readonly #groups = new Map<string, GroupState>();
|
||||
readonly #records: RecordLine[] = [];
|
||||
readonly #origins = new Map<string, { records: number; jsonLike: number }>();
|
||||
readonly #origins = new Map<string, OriginStats>();
|
||||
readonly #rules: ReadonlySet<TemplateRule>;
|
||||
#unrepresented = 0;
|
||||
#unrepresentableBodyKey = false;
|
||||
@@ -385,24 +389,6 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumula
|
||||
return ok(undefined);
|
||||
}
|
||||
|
||||
/** The origin with the most JSON-like records, then most records, then lowest code points. */
|
||||
#primaryOrigin(): string | null {
|
||||
let best: { origin: string; records: number; jsonLike: number } | null =
|
||||
null;
|
||||
for (const [origin, stats] of this.#origins) {
|
||||
if (
|
||||
best === null ||
|
||||
stats.jsonLike > best.jsonLike ||
|
||||
(stats.jsonLike === best.jsonLike &&
|
||||
(stats.records > best.records ||
|
||||
(stats.records === best.records &&
|
||||
compare(origin, best.origin) < 0)))
|
||||
)
|
||||
best = { origin, ...stats };
|
||||
}
|
||||
return best?.origin ?? null;
|
||||
}
|
||||
|
||||
#tools(groups: readonly ToolGroup[]): Tool[] {
|
||||
const registry = toolNameRegistry();
|
||||
return applyTemplateRules(groups, this.#rules)
|
||||
@@ -464,7 +450,7 @@ export class EnvironmentManifestAccumulator extends WebNetworkProjectionAccumula
|
||||
|
||||
/** Project the complete manifest; tools, records and exclusions are sorted deterministically. */
|
||||
finish(): Result<ManifestFields, AnalysisError> {
|
||||
const primary = this.#primaryOrigin();
|
||||
const primary = primaryOrigin(this.#origins);
|
||||
const toolGroups: GroupState[] = [];
|
||||
const exclusions = new Map<string, Exclusion & { templated: boolean }>();
|
||||
for (const group of this.#groups.values()) {
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { checkedLines, servedLine } from "./environmentStateCheckedLines.js";
|
||||
import { describe, expect, it } from "vitest";
|
||||
|
||||
import {
|
||||
@@ -39,6 +40,52 @@ const gated = (rows: readonly TimedCaptureRow[], tool: string) => {
|
||||
|
||||
const RENAMED = renamedBook(FIRST_BOOK, "Renamed Book", 3);
|
||||
|
||||
/** A book whose constant names organisation 1, created at exchange `seconds`. */
|
||||
const tiedToOrgOne = (id: number, name: string, seconds: number): Book => ({
|
||||
...book(id, name, bookClock(seconds)),
|
||||
org: "/orgs/1",
|
||||
});
|
||||
|
||||
/**
|
||||
* Two organisations, a create tied to the first by a constant that names it, then the same constant
|
||||
* recorded under the second. `confirmOwn` also reads the created book back from the first.
|
||||
*/
|
||||
const tiedCreateRows = (confirmOwn: boolean): TimedCaptureRow[] => {
|
||||
const alpha = { ...book(1, "Alpha", "2026-10-01T09:00:00Z"), org: "/orgs/1" };
|
||||
const beta = { ...book(10, "Beta", "2026-10-01T10:00:00Z"), org: "/orgs/2" };
|
||||
const made = tiedToOrgOne(2, "Made", 4);
|
||||
const other = tiedToOrgOne(11, "Other", confirmOwn ? 6 : 5);
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/1/books", [alpha]],
|
||||
["GET", "/api/orgs/2/books", [beta]],
|
||||
["GET", "/api/orgs/1/books/1", alpha],
|
||||
["GET", "/api/orgs/2/books/10", beta],
|
||||
["POST", "/api/orgs/1/books", made, 201, { name: "Made" }],
|
||||
];
|
||||
if (confirmOwn) rows.push(["GET", "/api/orgs/1/books/2", made]);
|
||||
rows.push(
|
||||
["POST", "/api/orgs/2/books", other, 201, { name: "Other" }],
|
||||
["GET", "/api/orgs/2/books/11", other],
|
||||
);
|
||||
return rows;
|
||||
};
|
||||
|
||||
/** A book whose constant names organisation `acme1`, created at exchange `seconds`. */
|
||||
const tiedToAcme = (id: number, name: string, seconds: number): Book => ({
|
||||
...book(id, name, bookClock(seconds)),
|
||||
team: "/orgs/uid/acme1",
|
||||
});
|
||||
|
||||
/** The create plan the gate gives a capture of `rows` when the segment after `uid` is a parameter. */
|
||||
const acmeCreate = (rows: readonly TimedCaptureRow[]) => {
|
||||
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows), [
|
||||
"after-uid-literal",
|
||||
]);
|
||||
const plan = writes.find((candidate) => candidate.effect === "create");
|
||||
if (plan === undefined) throw new Error("The gate planned no create");
|
||||
return plan;
|
||||
};
|
||||
|
||||
describe("reads compared as the served environment answers them", () => {
|
||||
/** Item reads in a `{data, revision}` envelope, a rename, then a read recorded at `revision`. */
|
||||
const renamedInEnvelope = (revision: string): TimedCaptureRow[] => [
|
||||
@@ -955,6 +1002,27 @@ describe("what a read shows of a write", () => {
|
||||
checked_record_ordinals: [3],
|
||||
});
|
||||
});
|
||||
|
||||
it("confirms a document write whose path encodes a parent the plain read holds", () => {
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/uid/acme1/settings", { theme: "light", lang: "en" }],
|
||||
[
|
||||
"PATCH",
|
||||
"/api/orgs/uid/%61cme1/settings",
|
||||
{ theme: "dark", lang: "en" },
|
||||
200,
|
||||
{ theme: "dark" },
|
||||
],
|
||||
["GET", "/api/orgs/uid/acme1/settings", { theme: "dark", lang: "en" }],
|
||||
];
|
||||
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows), [
|
||||
"after-uid-literal",
|
||||
]);
|
||||
expect(writes.find((plan) => plan.effect === "document")).toMatchObject({
|
||||
served: true,
|
||||
checked_record_ordinals: [2],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("document writes later writes overwrite", () => {
|
||||
@@ -1163,6 +1231,102 @@ describe("reads named as serving names them", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("encoded parent spellings", () => {
|
||||
it("names the plainly held entity from a path that encodes its parent", () => {
|
||||
const type = "/api/orgs/uid/{uid}/books";
|
||||
const collection = "/api/orgs/uid/acme1/books";
|
||||
const state = new SimulatedState(
|
||||
[
|
||||
{
|
||||
type,
|
||||
collection,
|
||||
id: "1",
|
||||
fields: { id: 1, name: "Alpha" },
|
||||
view_fields: [],
|
||||
record_ordinals: [],
|
||||
},
|
||||
],
|
||||
[],
|
||||
);
|
||||
expect(
|
||||
itemTarget(state, type, { path: "/api/orgs/uid/%61cme1/books/1" }),
|
||||
).toEqual({ collection, id: "1" });
|
||||
});
|
||||
|
||||
it("answers an encoded read from the plain checked line", () => {
|
||||
const tool = {
|
||||
name: "get_orgs_books_by_uid",
|
||||
method: "GET",
|
||||
path_template: "/api/orgs/uid/{uid}/books",
|
||||
record_ordinals: [0],
|
||||
};
|
||||
const record = {
|
||||
ordinal: 0,
|
||||
method: "GET",
|
||||
path: "/api/orgs/uid/acme1/books",
|
||||
query: undefined,
|
||||
status: 200,
|
||||
start: 0,
|
||||
end: 0,
|
||||
};
|
||||
const lines = checkedLines({
|
||||
tools: [tool],
|
||||
records: [record],
|
||||
views: new Map([
|
||||
[
|
||||
tool.name,
|
||||
{
|
||||
tool: tool.name,
|
||||
type: "/api/orgs/uid/{uid}/books",
|
||||
shape: "list",
|
||||
pointer: "",
|
||||
fields: ["id"],
|
||||
required_fields: ["id"],
|
||||
checked_record_ordinals: [0],
|
||||
membership: "all",
|
||||
counters: [],
|
||||
},
|
||||
],
|
||||
]),
|
||||
});
|
||||
expect(
|
||||
servedLine(lines.get(tool.name), {
|
||||
...record,
|
||||
ordinal: 1,
|
||||
path: "/api/orgs/uid/%61cme1/books",
|
||||
}),
|
||||
).toEqual({
|
||||
ordinal: 0,
|
||||
collection: "/api/orgs/uid/acme1/books",
|
||||
named: true,
|
||||
});
|
||||
});
|
||||
|
||||
it("confirms an update whose path encodes a parent the plain reads hold", () => {
|
||||
const alpha = { id: 1, org: "acme1", name: "Alpha" };
|
||||
const renamed = { id: 1, org: "acme1", name: "Renamed" };
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/uid/acme1/books", [alpha]],
|
||||
["GET", "/api/orgs/uid/acme1/books/1", alpha],
|
||||
[
|
||||
"PATCH",
|
||||
"/api/orgs/uid/%61cme1/books/1",
|
||||
renamed,
|
||||
200,
|
||||
{ name: "Renamed" },
|
||||
],
|
||||
["GET", "/api/orgs/uid/acme1/books/1", renamed],
|
||||
];
|
||||
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows), [
|
||||
"after-uid-literal",
|
||||
]);
|
||||
expect(writes.find((plan) => plan.effect === "update")).toMatchObject({
|
||||
served: true,
|
||||
checked_record_ordinals: [3],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("writes named as serving names them", () => {
|
||||
const orgBook = (
|
||||
id: number,
|
||||
@@ -2370,6 +2534,45 @@ describe("what later calls must keep consistent", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("does not let a later empty ?page=2 page contradict a create", () => {
|
||||
const alpha = { id: 1, name: "Alpha", org: "1" };
|
||||
const beta = { id: 2, name: "Beta", org: "2" };
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/1/books", [alpha]],
|
||||
["GET", "/api/orgs/1/books/1", alpha],
|
||||
["GET", "/api/orgs/2/books?page=2", []],
|
||||
["POST", "/api/orgs/2/books", beta, 201, { name: "Beta" }],
|
||||
["GET", "/api/orgs/2/books/2", beta],
|
||||
["GET", "/api/orgs/2/books?page=2", []],
|
||||
];
|
||||
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows));
|
||||
expect(writes.find((plan) => plan.effect === "create")).toMatchObject({
|
||||
served: true,
|
||||
checked_record_ordinals: [4],
|
||||
});
|
||||
});
|
||||
|
||||
it.each(["offset", "start", "_start"])(
|
||||
"does not let a later empty ?%s=1 page contradict a create",
|
||||
(name) => {
|
||||
const alpha = { id: 1, name: "Alpha", org: "1" };
|
||||
const beta = { id: 2, name: "Beta", org: "2" };
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/1/books", [alpha]],
|
||||
["GET", "/api/orgs/1/books/1", alpha],
|
||||
["GET", `/api/orgs/2/books?${name}=1`, []],
|
||||
["POST", "/api/orgs/2/books", beta, 201, { name: "Beta" }],
|
||||
["GET", "/api/orgs/2/books/2", beta],
|
||||
["GET", `/api/orgs/2/books?${name}=1`, []],
|
||||
];
|
||||
const { writes } = environmentStateViewOf(timedCaptureEvidence(rows));
|
||||
expect(writes.find((plan) => plan.effect === "create")).toMatchObject({
|
||||
served: true,
|
||||
checked_record_ordinals: [4],
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("checks nothing by a whole line recorded empty under a zero limit once a create fills its collection", () => {
|
||||
const alpha = { id: 1, name: "Alpha", org: "1" };
|
||||
const beta = { id: 2, name: "Beta", org: "2" };
|
||||
@@ -2390,6 +2593,145 @@ describe("what later calls must keep consistent", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("creates tied to one collection", () => {
|
||||
it("does not let a later create in another collection confirm the plan", () => {
|
||||
expect(gated(tiedCreateRows(false), "post_orgs_books_by_id")).toMatchObject(
|
||||
{
|
||||
served: false,
|
||||
reason: "unconfirmed",
|
||||
collection: "/api/orgs/1/books",
|
||||
checked_record_ordinals: [],
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
it("still confirms the plan from a read of the collection it is tied to", () => {
|
||||
expect(gated(tiedCreateRows(true), "post_orgs_books_by_id")).toMatchObject({
|
||||
served: true,
|
||||
collection: "/api/orgs/1/books",
|
||||
checked_record_ordinals: [5],
|
||||
});
|
||||
});
|
||||
|
||||
const acmeRows = (
|
||||
laterPath: string,
|
||||
heldEncoded: boolean,
|
||||
): TimedCaptureRow[] => {
|
||||
const alpha = {
|
||||
...book(1, "Alpha", "2026-10-01T09:00:00Z"),
|
||||
team: "/orgs/uid/acme1",
|
||||
};
|
||||
const made = tiedToAcme(2, "Made", 2);
|
||||
const other = tiedToAcme(3, "Other", heldEncoded ? 5 : 3);
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", "/api/orgs/uid/acme1/books", [alpha]],
|
||||
["GET", "/api/orgs/uid/acme1/books/1", alpha],
|
||||
];
|
||||
if (heldEncoded) {
|
||||
const encoded = {
|
||||
...book(10, "Encoded", "2026-10-01T10:00:00Z"),
|
||||
team: "/orgs/uid/acme1",
|
||||
};
|
||||
rows.push(
|
||||
["GET", "/api/orgs/uid/%61cme1/books", [encoded]],
|
||||
["GET", "/api/orgs/uid/%61cme1/books/10", encoded],
|
||||
);
|
||||
}
|
||||
rows.push(
|
||||
["POST", "/api/orgs/uid/acme1/books", made, 201, { name: "Made" }],
|
||||
["POST", laterPath, other, 201, { name: "Other" }],
|
||||
["GET", "/api/orgs/uid/acme1/books/3", other],
|
||||
);
|
||||
return rows;
|
||||
};
|
||||
|
||||
it("confirms a tied create whose later call spells the held collection encoded", () => {
|
||||
expect(
|
||||
acmeCreate(acmeRows("/api/orgs/uid/%61cme1/books", false)),
|
||||
).toMatchObject({
|
||||
served: true,
|
||||
collection: "/api/orgs/uid/acme1/books",
|
||||
checked_record_ordinals: [4],
|
||||
});
|
||||
});
|
||||
|
||||
it("does not treat an encoded spelling as the tied collection when that spelling is held itself", () => {
|
||||
// Serving keeps the verbatim encoded collection, outside the plain tie. The plain read of the
|
||||
// entity that create did not add is related to the exemplar, so the plan is a side effect.
|
||||
expect(
|
||||
acmeCreate(acmeRows("/api/orgs/uid/%61cme1/books", true)),
|
||||
).toMatchObject({
|
||||
served: false,
|
||||
reason: "side-effect",
|
||||
collection: "/api/orgs/uid/acme1/books",
|
||||
checked_record_ordinals: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("chooses among emptied spellings of a tied collection in the order serving remembers them", () => {
|
||||
const first = "/api/orgs/uid/%61cme1/books";
|
||||
const second = "/api/orgs/uid/a%63me1/books";
|
||||
const alpha = {
|
||||
...book(1, "Alpha", "2026-10-01T09:00:00Z"),
|
||||
team: "/orgs/uid/acme1",
|
||||
};
|
||||
const beta = {
|
||||
...book(10, "Beta", "2026-10-01T10:00:00Z"),
|
||||
team: "/orgs/uid/acme1",
|
||||
};
|
||||
const made = tiedToAcme(2, "Made", 4);
|
||||
const other = tiedToAcme(3, "Other", 8);
|
||||
// The second spelling empties before the first, but serving still offers the first spelling first.
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", first, [alpha]],
|
||||
["GET", second, [beta]],
|
||||
["GET", `${first}/1`, alpha],
|
||||
["GET", `${second}/10`, beta],
|
||||
["POST", second, made, 201, { name: "Made" }],
|
||||
["DELETE", `${second}/10`, "", 204],
|
||||
["DELETE", `${second}/2`, "", 204],
|
||||
["DELETE", `${first}/1`, "", 204],
|
||||
["POST", "/api/orgs/uid/acme1/books", other, 201, { name: "Other" }],
|
||||
["GET", "/api/orgs/uid/acme1/books/3", other],
|
||||
];
|
||||
expect(acmeCreate(rows)).toMatchObject({
|
||||
collection: second,
|
||||
checked_record_ordinals: [],
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps a tied collection ahead of another spelling once its first entity is gone, as serving orders them", () => {
|
||||
const first = "/api/orgs/uid/%61cme1/books";
|
||||
const second = "/api/orgs/uid/a%63me1/books";
|
||||
const alpha = {
|
||||
...book(1, "Alpha", "2026-10-01T09:00:00Z"),
|
||||
team: "/orgs/uid/acme1",
|
||||
};
|
||||
const beta = {
|
||||
...book(10, "Beta", "2026-10-01T10:00:00Z"),
|
||||
team: "/orgs/uid/acme1",
|
||||
};
|
||||
const made = tiedToAcme(2, "Made", 4);
|
||||
const other = tiedToAcme(3, "Other", 6);
|
||||
// Serving places the created book beside its collection, so the first spelling still leads after its first book goes.
|
||||
const rows: TimedCaptureRow[] = [
|
||||
["GET", first, [alpha]],
|
||||
["GET", second, [beta]],
|
||||
["GET", `${first}/1`, alpha],
|
||||
["GET", `${second}/10`, beta],
|
||||
["POST", first, made, 201, { name: "Made" }],
|
||||
["DELETE", `${first}/1`, "", 204],
|
||||
["POST", "/api/orgs/uid/acme1/books", other, 201, { name: "Other" }],
|
||||
["GET", "/api/orgs/uid/acme1/books/3", other],
|
||||
];
|
||||
expect(acmeCreate(rows)).toMatchObject({
|
||||
served: true,
|
||||
collection: first,
|
||||
checked_record_ordinals: [7],
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("path fields of a create", () => {
|
||||
const held = "/api/orgs/uid/%2561cme1/books";
|
||||
/** A create held under a doubly encoded parent, then one whose call spells the parent once encoded, read back with `org`. */
|
||||
@@ -2428,3 +2770,46 @@ describe("path fields of a create", () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("path fields of a create tied to its collection", () => {
|
||||
const held = "/api/orgs/uid/%61cme1/books";
|
||||
|
||||
/** A tied create in the held encoded collection, then one spelled double-encoded whose parent the application stored as `org`. */
|
||||
const parentRows = (org: string): TimedCaptureRow[] => {
|
||||
const alpha = {
|
||||
...book(1, "Alpha", "2026-10-01T09:00:00Z"),
|
||||
team: "/orgs/uid/acme1",
|
||||
org: "acme1",
|
||||
};
|
||||
const made = { ...tiedToAcme(2, "Made", 2), org: "acme1" };
|
||||
const other = { ...tiedToAcme(3, "Other", 3), org };
|
||||
return [
|
||||
["GET", held, [alpha]],
|
||||
["GET", `${held}/1`, alpha],
|
||||
["POST", held, made, 201, { name: "Made" }],
|
||||
["POST", "/api/orgs/uid/%2561cme1/books", other, 201, { name: "Other" }],
|
||||
["GET", `${held}/3`, other],
|
||||
];
|
||||
};
|
||||
|
||||
it("contradicts a later create whose read shows a parent serving would not write", () => {
|
||||
// Serving selects the held collection and decodes the parent from it, not from the call's spelling.
|
||||
expect(acmeCreate(parentRows("%61cme1"))).toMatchObject({
|
||||
served: false,
|
||||
collection: held,
|
||||
rules: expect.arrayContaining([
|
||||
expect.objectContaining({ field: "org", rule: "path" }),
|
||||
]),
|
||||
checked_record_ordinals: [],
|
||||
contradicted_by: 4,
|
||||
});
|
||||
});
|
||||
|
||||
it("confirms a later create whose read shows the parent decoded from the selected collection", () => {
|
||||
expect(acmeCreate(parentRows("acme1"))).toMatchObject({
|
||||
served: true,
|
||||
collection: held,
|
||||
checked_record_ordinals: [4],
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,25 @@
|
||||
import { compareUnicodeCodePoints as compare } from "../unicodeCodePointOrder.js";
|
||||
|
||||
/** Records and JSON-like records seen at one origin. */
|
||||
export interface OriginStats {
|
||||
records: number;
|
||||
jsonLike: number;
|
||||
}
|
||||
|
||||
/** The origin with the most JSON-like records, then most records, then lowest code points: the application a capture records. */
|
||||
export const primaryOrigin = (
|
||||
origins: ReadonlyMap<string, Readonly<OriginStats>>,
|
||||
): string | null => {
|
||||
let best: ({ origin: string } & OriginStats) | null = null;
|
||||
for (const [origin, stats] of origins) {
|
||||
if (
|
||||
best === null ||
|
||||
stats.jsonLike > best.jsonLike ||
|
||||
(stats.jsonLike === best.jsonLike &&
|
||||
(stats.records > best.records ||
|
||||
(stats.records === best.records && compare(origin, best.origin) < 0)))
|
||||
)
|
||||
best = { origin, ...stats };
|
||||
}
|
||||
return best?.origin ?? null;
|
||||
};
|
||||
@@ -8,7 +8,7 @@ import {
|
||||
} from "../recordedTransportUrl.js";
|
||||
import {
|
||||
allLines,
|
||||
decodedSegment,
|
||||
pathSegmentMatches,
|
||||
isParam,
|
||||
listsAll,
|
||||
segmentsOf,
|
||||
@@ -186,7 +186,11 @@ const bestLine = (
|
||||
if (lines.params.has(index)) verbatim += 1;
|
||||
return true;
|
||||
}
|
||||
return lines.params.has(index) && decodedSegment(segment) === read[index];
|
||||
return (
|
||||
lines.params.has(index) &&
|
||||
read[index] !== undefined &&
|
||||
pathSegmentMatches(segment, read[index])
|
||||
);
|
||||
});
|
||||
if (!matches) continue;
|
||||
const matched = queryMatches(values, query);
|
||||
|
||||
@@ -18,12 +18,14 @@ const PAGING_PARAMETERS = new Set([
|
||||
"_end",
|
||||
]);
|
||||
|
||||
/** Whether a query parameter pages or bounds a list rather than filtering it. */
|
||||
export const isPagingParameter = (name: string): boolean =>
|
||||
PAGING_PARAMETERS.has(name.toLowerCase());
|
||||
|
||||
/** Whether a query only pages its list: no parameters, or paging parameters alone. */
|
||||
export const pagesOnly = (query: string | undefined): boolean =>
|
||||
query === undefined ||
|
||||
[...parseUrlQueryParameterNames(query)].every((name) =>
|
||||
PAGING_PARAMETERS.has(name.toLowerCase()),
|
||||
);
|
||||
[...parseUrlQueryParameterNames(query)].every(isPagingParameter);
|
||||
|
||||
const PAGE_POSITION_PARAMETERS = new Set(["offset", "start", "_start", "page"]);
|
||||
|
||||
|
||||
@@ -214,6 +214,60 @@ export const decodedSegment = (segment: string): string => {
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether two path spellings name one segment: they are equal, or either decodes once to the other.
|
||||
* Decoding both would make `book%2531` match `book1`.
|
||||
*/
|
||||
export const pathSegmentMatches = (segment: string, value: string): boolean =>
|
||||
segment === value ||
|
||||
decodedSegment(segment) === value ||
|
||||
decodedSegment(value) === segment;
|
||||
|
||||
/** One segment of a call: a template literal, or a supplied path value. */
|
||||
export interface SuppliedSegment {
|
||||
readonly text: string;
|
||||
readonly supplied: boolean;
|
||||
}
|
||||
|
||||
/** How many supplied segments a held collection matches verbatim, when it matches the call under the path rule. */
|
||||
export const collectionMatch = (
|
||||
collection: string,
|
||||
segments: readonly SuppliedSegment[],
|
||||
): number | undefined => {
|
||||
const held = segmentsOf(collection);
|
||||
if (held.length !== segments.length) return undefined;
|
||||
let verbatim = 0;
|
||||
for (const [index, { text, supplied }] of segments.entries()) {
|
||||
const segment = held[index]!;
|
||||
if (supplied ? !pathSegmentMatches(segment, text) : segment !== text)
|
||||
return undefined;
|
||||
if (supplied && segment === text) verbatim += 1;
|
||||
}
|
||||
return verbatim;
|
||||
};
|
||||
|
||||
/** The collection a create writes: the candidate with the most verbatim supplied segments, the earliest on a tie, else the call with each supplied segment encoded. */
|
||||
export const selectedCollection = (
|
||||
segments: readonly SuppliedSegment[],
|
||||
candidates: readonly string[],
|
||||
): string => {
|
||||
let chosen: string | undefined;
|
||||
let verbatim = -1;
|
||||
for (const collection of candidates) {
|
||||
const matched = collectionMatch(collection, segments);
|
||||
if (matched !== undefined && matched > verbatim) {
|
||||
chosen = collection;
|
||||
verbatim = matched;
|
||||
}
|
||||
}
|
||||
return (
|
||||
chosen ??
|
||||
`/${segments
|
||||
.map(({ text, supplied }) => (supplied ? encodeURIComponent(text) : text))
|
||||
.join("/")}`
|
||||
);
|
||||
};
|
||||
|
||||
/** The decoded final path segment, which an item request names its entity by. */
|
||||
export const lastSegmentValue = (path: string): string =>
|
||||
decodedSegment(segmentsOf(path).at(-1) ?? "");
|
||||
|
||||
@@ -1009,6 +1009,78 @@ it("accepts stable membership across another root's write and changes confined b
|
||||
expect(state.entities.map((entity) => entity.id)).toEqual(["1"]);
|
||||
});
|
||||
|
||||
it.each(["offset", "start", "_start"])(
|
||||
"does not let an empty ?%s=1 page prove membership all",
|
||||
(name) => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
["GET", "/api/orgs/1/books/1", { id: 1, name: "Alpha" }],
|
||||
["GET", `/api/orgs/2/books?${name}=1`, []],
|
||||
]),
|
||||
);
|
||||
expect(viewOf(state, "get_orgs_books_by_id")).toMatchObject({
|
||||
shape: "list",
|
||||
membership: "recorded",
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
it("does not let an empty ?page=2 page prove membership all", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
["GET", "/api/orgs/1/books/1", { id: 1, name: "Alpha" }],
|
||||
["GET", "/api/orgs/2/books?page=2", []],
|
||||
]),
|
||||
);
|
||||
expect(viewOf(state, "get_orgs_books_by_id")).toMatchObject({
|
||||
shape: "list",
|
||||
membership: "recorded",
|
||||
});
|
||||
});
|
||||
|
||||
it("does not let ?page=2&per_page=1 prove membership all when it lists the collection", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
[
|
||||
"GET",
|
||||
"/api/orgs/1/books?page=2&per_page=1",
|
||||
[{ id: 1, name: "Alpha" }],
|
||||
],
|
||||
["GET", "/api/orgs/1/books/1", { id: 1, name: "Alpha" }],
|
||||
]),
|
||||
);
|
||||
expect(viewOf(state, "get_orgs_books_by_id")).toMatchObject({
|
||||
shape: "list",
|
||||
membership: "recorded",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps page 1 membership recorded because the page base is unproven", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
["GET", "/api/orgs/1/books?page=1", [{ id: 1, name: "Alpha" }]],
|
||||
["GET", "/api/orgs/1/books/1", { id: 1, name: "Alpha" }],
|
||||
]),
|
||||
);
|
||||
expect(viewOf(state, "get_orgs_books_by_id")).toMatchObject({
|
||||
shape: "list",
|
||||
membership: "recorded",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps membership all when a zero offset lists the collection", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
["GET", "/api/orgs/1/books?offset=0", [{ id: 1, name: "Alpha" }]],
|
||||
["GET", "/api/orgs/1/books/1", { id: 1, name: "Alpha" }],
|
||||
]),
|
||||
);
|
||||
expect(viewOf(state, "get_orgs_books_by_id")).toMatchObject({
|
||||
shape: "list",
|
||||
membership: "all",
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps membership checks specific to the full request line", () => {
|
||||
const state = stateOf(
|
||||
captureOf([
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { ServedOrder } from "./environmentStateWriteOrder.js";
|
||||
import type { JsonValue } from "../jsonValue.js";
|
||||
import { compositeKey } from "../unicodeCodePointOrder.js";
|
||||
import {
|
||||
@@ -7,6 +8,7 @@ import {
|
||||
listItems,
|
||||
listsAll,
|
||||
missingRequiredFields,
|
||||
pathSegmentMatches,
|
||||
renderStateEntity,
|
||||
renderStateList,
|
||||
replaceAt,
|
||||
@@ -47,12 +49,18 @@ export class SimulatedState {
|
||||
readonly #collections = new Map<string, string[]>();
|
||||
readonly #documents = new Map<string, JsonValue>();
|
||||
readonly #removed: RemovedTarget[] = [];
|
||||
readonly #served = new ServedOrder();
|
||||
|
||||
constructor(
|
||||
entities: readonly StateEntity[],
|
||||
documents: readonly StateDocument[],
|
||||
) {
|
||||
for (const entity of entities) this.put(entity);
|
||||
this.#served.reset(
|
||||
entities.map(({ type, collection, id }) =>
|
||||
entityTarget(type, collection, id),
|
||||
),
|
||||
);
|
||||
for (const document of documents)
|
||||
this.#documents.set(document.tool, document.value);
|
||||
}
|
||||
@@ -72,9 +80,13 @@ export class SimulatedState {
|
||||
}
|
||||
|
||||
ofType(type: string): StateEntity[] {
|
||||
return [...this.#entities.values()].filter(
|
||||
(entity) => entity.type === type,
|
||||
);
|
||||
return this.#served.keys
|
||||
.map((key) => this.#entities.get(key)!)
|
||||
.filter((entity) => entity.type === type);
|
||||
}
|
||||
|
||||
knownCollections(type: string): readonly string[] {
|
||||
return this.#served.known(type);
|
||||
}
|
||||
|
||||
/** Replace an entity in place, or add it at `index` of its collection (the end when omitted). */
|
||||
@@ -85,13 +97,17 @@ export class SimulatedState {
|
||||
if (known) return;
|
||||
const collection = collectionTarget(entity.type, entity.collection);
|
||||
const order = this.#collections.get(collection) ?? [];
|
||||
order.splice(index ?? order.length, 0, key);
|
||||
const at = index ?? order.length;
|
||||
this.#served.place(key, order, at);
|
||||
order.splice(at, 0, key);
|
||||
this.#collections.set(collection, order);
|
||||
this.#served.remember(entity.type, entity.collection);
|
||||
}
|
||||
|
||||
remove(entity: StateEntity): void {
|
||||
const key = entityTarget(entity.type, entity.collection, entity.id);
|
||||
this.#entities.delete(key);
|
||||
this.#served.remove(key);
|
||||
const collection = collectionTarget(entity.type, entity.collection);
|
||||
const order = this.#collections.get(collection) ?? [];
|
||||
this.#collections.set(
|
||||
@@ -200,16 +216,18 @@ interface ReadBodies {
|
||||
}
|
||||
|
||||
/** An item read: the answer the held entity renders into the served envelope must equal the recording; a recorded 404 or 410 agrees with none held. */
|
||||
/** How many of a held collection's segments match a path's verbatim, or undefined when one matches neither verbatim nor once percent-decoded. */
|
||||
/** How many of a held collection's segments match a path's verbatim, or undefined when one matches neither verbatim nor either spelling once percent-decoded. */
|
||||
const segmentMatch = (
|
||||
held: readonly string[],
|
||||
read: readonly string[],
|
||||
): number | undefined => {
|
||||
if (held.length !== read.length) return undefined;
|
||||
let verbatim = 0;
|
||||
for (const [index, segment] of held.entries())
|
||||
if (segment === read[index]) verbatim += 1;
|
||||
else if (decodedSegment(segment) !== read[index]) return undefined;
|
||||
for (const [index, segment] of held.entries()) {
|
||||
const value = read[index]!;
|
||||
if (segment === value) verbatim += 1;
|
||||
else if (!pathSegmentMatches(segment, value)) return undefined;
|
||||
}
|
||||
return verbatim;
|
||||
};
|
||||
|
||||
|
||||
@@ -1,17 +1,22 @@
|
||||
import type { JsonValue } from "../jsonValue.js";
|
||||
import { ok } from "../result.js";
|
||||
import {
|
||||
collectionPathOf,
|
||||
isObject,
|
||||
isParam,
|
||||
isStateWrite,
|
||||
normalizedPath,
|
||||
scalarText,
|
||||
segmentsOf,
|
||||
selectedCollection,
|
||||
type JsonObject,
|
||||
type StateEntity,
|
||||
type SuppliedSegment,
|
||||
} from "./environmentStateRules.js";
|
||||
import { createdFields } from "./environmentStateWriteApply.js";
|
||||
import {
|
||||
collectionTarget,
|
||||
entityTarget,
|
||||
heldCollection,
|
||||
} from "./environmentStateWriteChecks.js";
|
||||
import {
|
||||
built,
|
||||
@@ -231,6 +236,64 @@ const takeSequences = (
|
||||
}
|
||||
};
|
||||
|
||||
/** Segments of a recorded path as a create call supplies them: template literals, and each parameter's recorded spelling. */
|
||||
const recordedSegments = (
|
||||
template: string,
|
||||
path: string,
|
||||
): SuppliedSegment[] | undefined => {
|
||||
const pattern = segmentsOf(template);
|
||||
const concrete = segmentsOf(path);
|
||||
if (pattern.length !== concrete.length) return undefined;
|
||||
const segments: SuppliedSegment[] = [];
|
||||
for (const [index, part] of pattern.entries()) {
|
||||
const value = concrete[index]!;
|
||||
if (!isParam(part)) {
|
||||
if (part !== value) return undefined;
|
||||
segments.push({ text: part, supplied: false });
|
||||
continue;
|
||||
}
|
||||
if (value.length === 0) return undefined;
|
||||
segments.push({ text: value, supplied: true });
|
||||
}
|
||||
return segments;
|
||||
};
|
||||
|
||||
/** Collections a create may join, in serving's order: held ones, then emptied ones as first known, then checked list lines. */
|
||||
const createCandidates = (context: EffectContext, type: string): string[] => {
|
||||
const candidates: string[] = [];
|
||||
const seen = new Set<string>();
|
||||
const add = (collection: string): void => {
|
||||
if (seen.has(collection)) return;
|
||||
seen.add(collection);
|
||||
candidates.push(collection);
|
||||
};
|
||||
for (const entity of context.state.ofType(type)) add(entity.collection);
|
||||
for (const collection of context.state.knownCollections(type))
|
||||
add(collection);
|
||||
for (const record of context.records) {
|
||||
const view = context.viewOf(record.ordinal);
|
||||
if (
|
||||
view?.shape === "list" &&
|
||||
view.type === type &&
|
||||
view.checked_record_ordinals.includes(record.ordinal)
|
||||
)
|
||||
add(collectionPathOf(record, false));
|
||||
}
|
||||
return candidates;
|
||||
};
|
||||
|
||||
/** The collection serving would write this recorded create into, when the path instantiates the tool. */
|
||||
const servedCollection = (
|
||||
context: EffectContext,
|
||||
step: WriteStep,
|
||||
type: string,
|
||||
): string | undefined => {
|
||||
const segments = recordedSegments(step.tool.path_template, step.record.path);
|
||||
return segments === undefined
|
||||
? undefined
|
||||
: selectedCollection(segments, createCandidates(context, type));
|
||||
};
|
||||
|
||||
/** Add a created entity to the state at the plan's insert position, or last before the plan has one. */
|
||||
const insertCreated = (
|
||||
context: EffectContext,
|
||||
@@ -258,8 +321,15 @@ export const applyCreate = (
|
||||
): EffectResult => {
|
||||
const type = draft.plan.target!;
|
||||
const idField = context.idField(type);
|
||||
// The held collection serving would pick for this path, as an item read's names its entity.
|
||||
const collection = heldCollection(context.state, type, step.record.path);
|
||||
const spelled = normalizedPath(step.record.path);
|
||||
// Serving writes a create into the collection it chooses for the path, and a tied plan refuses any other, so a create there is not counted.
|
||||
const collection = servedCollection(context, step, type) ?? spelled;
|
||||
if (
|
||||
built(draft) &&
|
||||
draft.plan.collection !== null &&
|
||||
draft.plan.collection !== collection
|
||||
)
|
||||
return UNKNOWN;
|
||||
const body = requestObject(context, step.record);
|
||||
const observed = createdObservation(context, step, {
|
||||
type,
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import type { JsonValue } from "../jsonValue.js";
|
||||
import {
|
||||
at as valueAt,
|
||||
decodedSegment,
|
||||
isObject,
|
||||
pathSegmentMatches,
|
||||
pointerTo,
|
||||
same,
|
||||
segmentsOf,
|
||||
@@ -27,16 +27,14 @@ import {
|
||||
} from "./environmentStateWriteEffects.js";
|
||||
import { documentPointerOf } from "./environmentStateWriteExplain.js";
|
||||
|
||||
/** Whether a write's path names the held document's, a segment verbatim or once percent-decoded, as serving matches a call. */
|
||||
/** Whether a write's path names the held document's: each segment verbatim, or either spelling once percent-decoded, as serving matches a supplied value. */
|
||||
const namesDocument = (path: string, held: string): boolean => {
|
||||
const written = segmentsOf(path);
|
||||
const recorded = segmentsOf(held);
|
||||
return (
|
||||
written.length === recorded.length &&
|
||||
recorded.every(
|
||||
(segment, index) =>
|
||||
segment === written[index] ||
|
||||
decodedSegment(segment) === written[index],
|
||||
recorded.every((segment, index) =>
|
||||
pathSegmentMatches(segment, written[index]!),
|
||||
)
|
||||
);
|
||||
};
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
/** Serving's order of the state's entity keys, which orders a type's collections by their first entity, and each type's collections in the order serving first knew them, which emptying does not change. */
|
||||
export class ServedOrder {
|
||||
#keys: string[] = [];
|
||||
readonly #known = new Map<string, Set<string>>();
|
||||
|
||||
/** The keys in state order. */
|
||||
get keys(): readonly string[] {
|
||||
return this.#keys;
|
||||
}
|
||||
|
||||
/** Hold the Evidence's entities in their own order. */
|
||||
reset(keys: readonly string[]): void {
|
||||
this.#keys = [...keys];
|
||||
}
|
||||
|
||||
/** Add a key at `at` of its collection's `members`, beside them as serving places it, or last when the collection is empty. */
|
||||
place(key: string, members: readonly string[], at: number): void {
|
||||
const anchor =
|
||||
at < members.length
|
||||
? this.#keys.indexOf(members[at]!)
|
||||
: members.length > 0
|
||||
? this.#keys.indexOf(members.at(-1)!) + 1
|
||||
: this.#keys.length;
|
||||
this.#keys.splice(anchor, 0, key);
|
||||
}
|
||||
|
||||
remove(key: string): void {
|
||||
this.#keys = this.#keys.filter((member) => member !== key);
|
||||
}
|
||||
|
||||
remember(type: string, collection: string): void {
|
||||
const collections = this.#known.get(type) ?? new Set<string>();
|
||||
this.#known.set(type, collections.add(collection));
|
||||
}
|
||||
|
||||
/** Every collection of a type the state has held, in the order serving remembers them. */
|
||||
known(type: string): readonly string[] {
|
||||
return [...(this.#known.get(type) ?? [])];
|
||||
}
|
||||
}
|
||||
@@ -1,7 +1,7 @@
|
||||
import {
|
||||
decodedSegment,
|
||||
isParam,
|
||||
normalizedPath,
|
||||
pathSegmentMatches,
|
||||
segmentsOf,
|
||||
} from "./environmentStateRules.js";
|
||||
import {
|
||||
@@ -66,15 +66,14 @@ export class WriteTaint {
|
||||
return this.#targets.has(target);
|
||||
}
|
||||
|
||||
/** Whether an unmodelled write above a collection path left everything under it unknown, each collection segment matching the write's verbatim or once percent-decoded. */
|
||||
/** Whether an unmodelled write above a collection path left everything under it unknown, each collection segment matching the write's verbatim or either once percent-decoded. */
|
||||
under(collection: string): boolean {
|
||||
const concrete = segmentsOf(collection);
|
||||
return this.#prefixes.some((prefix) =>
|
||||
segmentsOf(prefix).every(
|
||||
(segment, index) =>
|
||||
isParam(segment) ||
|
||||
segment === concrete[index] ||
|
||||
decodedSegment(concrete[index] ?? "") === segment,
|
||||
pathSegmentMatches(concrete[index] ?? "", segment),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -125,6 +125,3 @@ export const inspectApkSignatureInputSchema = z
|
||||
"view requires summary delivery; omit it for complete Evidence",
|
||||
});
|
||||
});
|
||||
export type InspectApkSignatureInput = z.infer<
|
||||
typeof inspectApkSignatureInputSchema
|
||||
>;
|
||||
@@ -0,0 +1,94 @@
|
||||
import { writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
|
||||
import { describe, expect } from "vitest";
|
||||
|
||||
import { timedCaptureEvidence } from "../../fixtures/analysisView.js";
|
||||
import { environmentStateServeFixture } from "../../fixtures/environmentServe.js";
|
||||
import {
|
||||
BOOK_NOT_FOUND,
|
||||
BOOK_ROWS,
|
||||
book,
|
||||
} from "../../fixtures/environmentStateWrites.js";
|
||||
import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
|
||||
import { cliTest } from "../../support/cli/cliFixture.js";
|
||||
|
||||
/** The books environment and a probe of the application re-seeded under other ids, written as Evidence files. */
|
||||
const writeCompareFiles = async (root: string) => {
|
||||
const fixture = environmentStateServeFixture(timedCaptureEvidence(BOOK_ROWS));
|
||||
const first = book(101, "First Book", "2026-10-20T09:00:00Z");
|
||||
const second = book(102, "Second Book", "2026-10-20T10:00:00Z");
|
||||
const probe = timedCaptureEvidence([
|
||||
["GET", "/api/books", [first, second]],
|
||||
["DELETE", "/api/books/102", "", 204],
|
||||
["GET", "/api/books/102", BOOK_NOT_FOUND, 404],
|
||||
]);
|
||||
const paths = {
|
||||
manifest: join(root, "manifest.json"),
|
||||
capture: join(root, "capture.json"),
|
||||
probe: join(root, "probe.json"),
|
||||
};
|
||||
await writeFile(paths.manifest, JSON.stringify(fixture.manifest));
|
||||
await writeFile(paths.capture, JSON.stringify(fixture.capture));
|
||||
await writeFile(paths.probe, JSON.stringify(probe));
|
||||
return paths;
|
||||
};
|
||||
|
||||
describe("rea environment-compare", () => {
|
||||
cliTest(
|
||||
"compares the served environment with a probe of the re-seeded application and prints the report",
|
||||
async ({ cli }) => {
|
||||
const files = await writeCompareFiles(
|
||||
await createTestTempDirectory("rea-environment-compare-cli-"),
|
||||
);
|
||||
const result = await cli.run({
|
||||
arguments: [
|
||||
"environment-compare",
|
||||
files.manifest,
|
||||
"--evidence",
|
||||
files.capture,
|
||||
"--probe",
|
||||
files.probe,
|
||||
"--json",
|
||||
],
|
||||
timeoutMs: 30_000,
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.json).toMatchObject({
|
||||
status: "complete",
|
||||
environment: { state: "served" },
|
||||
summary: { probes: 1, steps: 3, equal: 3, agreement: 1 },
|
||||
probes: [{ label: "probe.json", ids_mapped: 2 }],
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
cliTest(
|
||||
"refuses a probe that is not capture Evidence and names it",
|
||||
async ({ cli }) => {
|
||||
const files = await writeCompareFiles(
|
||||
await createTestTempDirectory("rea-environment-compare-refusal-cli-"),
|
||||
);
|
||||
const result = await cli.run({
|
||||
arguments: [
|
||||
"environment-compare",
|
||||
files.manifest,
|
||||
"--evidence",
|
||||
files.capture,
|
||||
"--probe",
|
||||
files.manifest,
|
||||
"--json",
|
||||
],
|
||||
timeoutMs: 30_000,
|
||||
});
|
||||
expect(result.exitCode).toBe(1);
|
||||
expect(result.json).toMatchObject({
|
||||
code: "invalid_request",
|
||||
details: {
|
||||
operation: "environment-compare",
|
||||
issues: [{ path: ["probe", 0] }],
|
||||
},
|
||||
});
|
||||
},
|
||||
);
|
||||
});
|
||||
Reference in new issue
Block a user