> ## Documentation Index > Fetch the complete documentation index at: https://docs.moonbounce.ai/llms.txt > Use this file to discover all available pages before exploring further. # Suggestions and previews > How feedback becomes a proposed change to your spec, how a preview tests a change against past cases, and how an accepted change reaches production. > [!NOTE] > > The examples on this page use Northpine, a fictional outdoor-gear shop whose `refund-exception` decision grants, escalates or denies refund requests outside its standard return policy. [How Moonbounce works](https://docs.moonbounce.ai/learn/how-it-works.md#the-running-example) introduces it. The [Review](https://docs.moonbounce.ai/learn/review.md) page showed how reviews and your feedback settle the right outcome for past cases. This page follows those answers into a **suggestion**, a proposed change to your spec, and into a **preview**, which replays a change against past cases so you can decide whether to take it. ## The gap reviews find Northpine's spec has no feature about shipping. After a few days of feedback and reviews, a pattern shows up: tickets where the carrier delivered late keep landing on `escalate`, and the owner keeps saying `grant`. Ticket C is typical: a tent that arrived two weeks late, used once, \$189, 39 days after delivery. It scored 0.540, between the escalate cut (0.38) and the grant cut (0.60). The owner's reason: "The carrier delay ate their return window; that's our failure." No answer to the five existing features can say "the carrier was late". The loop has to notice that a feature is missing. ## What a suggestion is A suggestion is one proposed change to your spec. Today it is one of two kinds: * `feature_rewrite`: new wording for one existing feature (its `prompt`, `criteria` or `guidance`). * `new_feature`: one new boolean feature, added as a term of the `combine` that feeds the output (here `grant_score`). It is made against a specific version of your spec, its **base** (structure and weights hashes), and it carries the change as a **diff** against that base, the full **candidate** spec, and the **evidence** from a preview of the candidate. A suggestion never changes your cut points, and it never writes to your code: you or your coding agent apply it, and it ships when you merge. ## How the suggestion writer picks a change The suggestion writer starts from the cases it is allowed to see: your seed examples and reviewed cases outside the holdout (see [Evidence labels](#evidence-labels)). It replays the base on them and looks at each case whose outcome is wrong. ### Which feature is to blame For each wrong case it asks, feature by feature: if this answer were firmly the opposite (a yes of 0.5 or more becomes 0, anything lower becomes 1), would the case land on the outcome it should have? Each feature that passes this test is a suspect, and the case's blame is split evenly among its suspects. Two kinds of correction change the test. A feature claim that disagrees with the engine (yours, or a reviewer marking a feature wrong) names that feature directly, and no flipping is done. A feature claim that agrees with the engine confirms that answer, so that feature is never a suspect. Ticket C shows why this matters. Flip `outside_window` from 0.95 to 0 and C scores 0.730, which is `grant`. Flip `defective` from 0.05 to 1 and it also scores 0.730. Without more information, blame would split between those two features. But both answers were right: the request was outside the window, and nothing was defective. The owner said so in their corrected feedback on ticket C, from [Feedback](https://docs.moonbounce.ai/learn/feedback.md#corrections): ```json theme={null} { "about": { "evaluation_id": "evl_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e0f" }, "claims": [ { "kind": "outcome", "value": "grant" }, { "kind": "feature", "feature": "outside_window", "value": true }, { "kind": "feature", "feature": "defective", "value": false } ], "source": "owner", "reason": "The carrier delay ate their return window; that's our failure.", "supersedes": "fbk_0199f2c9-3b4c-7d5e-9f60-718293a4b5c6" } ``` With those two answers confirmed, the remaining flips all move C down (to 0.410, 0.344 and 0.346), never to `grant`. No single feature explains C. A case like that is a **missing criterion**: the spec lacks a feature it needs. ### Rewrite or add Across all the wrong cases, the writer compares the most-blamed feature's total blame with the number of missing-criterion cases. If the blame is at least as large, it proposes a rewrite of that feature; otherwise it proposes a new feature. The owner confirmed the same two answers on each carrier-delay ticket they judged, so those are all missing-criterion cases. They outnumber the blame on any single feature (a correction like ticket D's in [Review](https://docs.moonbounce.ai/learn/review.md) blames `item_used` for one case), so the writer adds a feature. A rewrite also sees missing-criterion cases and their reasons as context, not as failures attributed to the rewritten feature. Blamed cases get priority within the shared cap of 20 wrong cases. A language model drafts the words: new wording for the blamed feature’s prompt, or a name and prompt for the new feature. It sees the base spec and the text of the wrong and right cases it may see, with their reasons and corrections. Moonbounce, not the model, builds the diff, checks that the candidate is a valid spec with a new structure hash, and replays each draft on the seed and training cases. The draft that fixes the most cases net of breaks, if any fixes more than it breaks, gets a full preview that includes the held-out cases, and it is proposed only if that preview passes the [propose rule](#the-propose-rule). ## The Northpine suggestion `GET /v1/suggestions/{suggestion_id}` returns the suggestion. Abbreviated: the full response also has `candidate_spec`, each version's `engine_profile`, the generator that wrote it, `status_at`, and the evidence counted by feedback source (`by_source`), and `changed_items` lists all five fixes, not only ticket C's. ```json theme={null} { "id": "sug_01a17b2c-4d5e-7f60-8a91-b2c3d4e5f607", "decision_id": "refund-exception", "kind": "new_feature", "target": "structure", "summary": "Adds a feature for late or lost deliveries, which the owner treated as Northpine's failure and granted even outside the window.", "environment": "production", "status": "proposed", "base": { "structure": "sha256:c21908531c5bb1e922591c1b6d2d9e238205a6ee4c87cddff4674003e880b313", "weights": "sha256:d1340418a598e440b1b7680807d525414e7b2ad6801caab117d123f731e62eba" }, "candidate": { "structure": "sha256:9e2ee8b452b25c5023a874d7bcb1ce91c93e9c604454887e058f4bd9f3b09e2e", "weights": "sha256:9ba920828d06d7f00d2a5e2b5635f5af2f3be457f039835afb31168e6d3d95a5", "weights_pin": "default" }, "requires": [], "diff": [ { "op": "add", "path": "/nodes/carrier_delay", "after": { "type": "boolean", "prompt": "Did the carrier deliver the order late, or lose it?" } }, { "op": "add", "path": "/nodes/grant_score/terms/-", "after": { "node": "carrier_delay", "effect": "raises" } } ], "evidence": { "label": "holdout", "frozen_at": "2026-10-27T15:02:11.482Z", "by_split": { "seed": { "n": 12, "fixes": 1, "breaks": 0, "base_correct": 9, "candidate_correct": 10 }, "train": { "n": 10, "fixes": 3, "breaks": 0, "base_correct": 5, "candidate_correct": 8 }, "holdout": { "n": 1, "fixes": 1, "breaks": 0, "base_correct": 0, "candidate_correct": 1, "holdout_uses": 1, "owner_decided": 1 } }, "noise_rate": 0, "action_mix": { "before": { "deny": 0.22, "escalate": 0.62, "grant": 0.16, "abstain": 0 }, "after": { "deny": 0.26, "escalate": 0.44, "grant": 0.3, "abstain": 0 }, "on": "newest 50 of 50 primary production evaluations, last 7 days", "n": 50 }, "changed_items": [ { "evaluation_id": "evl_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e0f", "feedback_id": "fbk_0199f2d1-7e8f-7a90-b1c2-d3e4f5a6b7c8", "split": "train", "change": "fix", "before": { "outcome": "escalate", "score": 0.54, "features": { "outside_window": 0.95, "item_used": 0.35, "defective": 0.05, "repeat_refunder": 0.02, "high_value": 0.03 } }, "after": { "outcome": "grant", "score": 0.6116666666666666, "features": { "outside_window": 0.95, "item_used": 0.35, "defective": 0.05, "repeat_refunder": 0.02, "high_value": 0.03, "carrier_delay": 0.97 } } } ], "credited_feedback": ["fbk_0199f2d1-7e8f-7a90-b1c2-d3e4f5a6b7c8"] }, "preview_id": "run_01a17b2c-4c90-7b11-9e35-c7a2d6f0e148", "created_at": "2026-10-27T15:02:14.906Z" } ``` The diff is a list of edits addressed by JSON pointers into the spec, all under `/nodes/`. `add` creates a node or appends a term (`/terms/-`); `replace` and `remove` carry a `before`, the value they expect to find, so an edit made against a spec that has since changed fails instead of overwriting your work. A rewrite looks like `{"op": "replace", "path": "/nodes/item_used/prompt", "before": "...", "after": "..."}`. `environment` is the environment of the key that created the suggestion. The whole loop runs in one environment: a suggestion learns from that environment's cases, and only another key in the same environment can read it. Only a production suggestion can be accepted, so the developer's agent asks with a production key. `requires` lists suggestions that must land first; it is empty today. `credited_feedback` lists the feedback the writer saw that decided a case the candidate fixed. The evidence fields are explained under [Previews](#previews). ### Weights for a new feature The candidate's `weights_pin` is the weights pin to write in your decision file, and `candidate.weights` is what that pin resolves to. Northpine pins `"default"`, so the suggestion keeps `"default"`, and `"default"` resolves to the candidate spec's own default weights. Those are recomputed for six terms: each weight is now about 0.167 instead of 0.2, and the bias is 4/6 (about 0.667) instead of 0.8. With `carrier_delay` answered 0.97 for C, 0.02 for A and 0.01 for B: | Ticket | Base score | Candidate score | Outcome at Northpine's cut points | | - | - | - | - | | A: zipper ripped | 0.766 | 0.642 | grant, still grant | | B: kayak, changed mind | 0.362 | 0.303 | deny, still deny | | C: tent, carrier late | 0.540 | 0.612 | escalate, now grant | C crosses the grant cut at 0.60. The other fixes are the same story: the tent seed example, two other carrier-delay tickets in training, and one held-out case. Look at A too. Under default weights every term counts equally, so a sixth feature that is "no" for most tickets lowers their scores by about a sixth: A falls from 0.766 to 0.642. It's still `grant`, but closer to the line. A change like this can break cases that sit just above a cut, which is why the preview counts breaks and replays recent traffic in the action mix. If your file pins a weights hash instead, a rewrite keeps that hash. A new feature under a pinned hash would get no weight at all, so the suggestion ships a new pin: carried weights, which keep every existing weight and the bias and give the new term the average size of the others. A pin equal to your spec's default weights hash counts as `"default"`. ## Previews `POST /v1/decisions/{decision_id}/previews` replays a candidate spec and its base over the decision's past cases and reports what would change. It takes the spec as it is in your code, the weights pin as written, and your cut points; with a decision file, `jq` builds the body: ```bash theme={null} curl -sS --fail-with-body -X POST "$MOONBOUNCE_API_URL/v1/decisions/refund-exception/previews" \ -H "Authorization: Bearer $MOONBOUNCE_API_KEY" \ -H 'Content-Type: application/json' \ -H "Idempotency-Key: $(uuidgen)" \ -d "$(jq '{spec, weights, outcomes, suggestion_id: "sug_01a17b2c-4d5e-7f60-8a91-b2c3d4e5f607"}' moonbounce/refund-exception.json)" ``` What it compares: * **The base.** The suggestion's base when you send `suggestion_id`; else the `base` you send (`{"structure", "weights"}`); else the version serving primary traffic in your key's environment; else the latest version seeded there. With none, `base` is `null` and nothing changes. * **The cases.** Past evaluations in your key's environment with stored content whose right outcome was decided by you, by reviewers or by a seed example, and is one of your request's outcomes. Your app's own feedback never decide a case. Each item group counts once, newest first; contested cases are left out. * **The cut points.** Both versions are cut at the cut points you send, never at the ones each evaluation used, so the only difference between before and after is the spec and weights. * **A fair baseline.** A feature the candidate changed is asked again for the base in the same preview, so both sides carry the engine's current noise. `noise_rate` is how often that fresh base answer gave a different outcome from the stored one: the noise floor. It is 0 when no feature changed, as for a pure addition. Because the changed feature is asked afresh each time, its `before` answer and score can move a little from one preview to the next. Features the candidate leaves alone replay cached answers when there are any; a test key's preview keeps no cache between runs, so there every feature is asked afresh. The response has `base` and `candidate` (each `{structure, weights, engine_profile}`), `hashes_match` (with `suggestion_id`: whether your spec and pin hash to that suggestion's candidate), `evidence`, and `cost` (engine calls, cache hits, input tokens and `usd_micros`, in millionths of a dollar). In `evidence`: * `by_split` counts, per set of cases, `n`, `fixes` (wrong before, right after), `breaks` (right before, wrong after), and how many each version got right. A case the engines failed on temporarily is left out; a version that fails a case permanently counts as wrong on it. * `action_mix` replays a sample of recent primary traffic in your key's environment (the `on` field says which) under both versions and gives the share of each outcome. Northpine's grant share would rise from 16% to 30% as carrier-delay tickets stop escalating, and its deny share would edge up from 22% to 26% as other scores drop a little. * `changed_items` lists every fix and break by ids, scores and answers, never content, and burns any held-out case it lists. A preview doesn't register the candidate. To evaluate by its structure hash, register the spec first with `POST /v1/decisions/{decision_id}/structures`, or send the spec inline on one evaluate call. A preview writes a record of the run and nothing else: no evaluation, no traffic, and never an acceptance. ### Changing cut points in the same change A suggestion leaves your cut points alone, but a preview takes whatever cut points you send, so you can test a structure change and a cut-point change together. Preview Northpine's candidate with the grant cut at 0.65 instead of 0.60, and both versions are cut there: A (0.766 before, 0.642 after) breaks from `grant` to `escalate`, and C (0.612 after) stays on `escalate`. That preview would show the new feature doesn't pay off at a stricter cut, and the action mix at those cut points shows what recent traffic would get. Cut points mint no version, so the hashes still match the suggestion. ## Evidence labels Every preview gets a label for the strongest set of cases it was tested on. | Label | Tested on | How much to trust it | | - | - | - | | Starting examples | Your seed examples only | Weakest: the writer learned from them, so it shows the change is consistent with your examples, not that it generalizes | | Training cases | Reviewed cases the writer could see | Real traffic, but still cases the change was made to fit | | Held out | Reviewed cases the writer never saw | Strongest available: a test the change could not be tuned to | About 30% of reviewed cases are held out. An item's side is fixed by the first feedback it gets (near-duplicate items share one side): a seed example is always a seed, your app's own feedback put it in training, and feedback from you or a reviewer puts it in the holdout about 30% of the time, decided by a hash, never by the outcome or the answers. Held-out content never reaches the suggestion writer. Two caveats sit in the numbers. `owner_decided` counts held-out cases you decided yourself: never seen by the writer is not never seen by you. And `holdout_uses` counts the previews that have scored the holdout; a holdout reused many times is less of an independent test. ### Shown cases become training A held-out case listed in a preview's `changed_items` has been shown to you or your agent, so it is **burned**: it counts as a training case from then on. Previewing the Northpine suggestion again after it was proposed gives: ```json theme={null} { "label": "training_cases", "by_split": { "seed": { "n": 12, "fixes": 1, "breaks": 0, "base_correct": 9, "candidate_correct": 10 }, "train": { "n": 11, "fixes": 4, "breaks": 0, "base_correct": 5, "candidate_correct": 9 } } } ``` The one held-out case it fixed was listed in the proposal's evidence, so it moved to training, and with no held-out cases left the label drops. That is expected. The evidence stored on the suggestion is the observation made when it was proposed, and it doesn't change; quote that label in your pull request beside the new preview's numbers. Every preview that lists held-out cases burns them, so repeated previews use up the holdout. A further label, Validated (tested on reviews that arrive after the change was frozen), is not available yet. ### The propose rule A suggestion is proposed only when its candidate fixes more cases than it breaks on its own label's set: the seed examples for Starting examples, the training cases for Training cases. On Held out it must also beat the noise floor: fixes minus breaks must be greater than `noise_rate` times the number of held-out cases. Northpine's suggestion passed on one held-out fix, no breaks and a noise floor of 0. When the best candidate fails the rule, nothing is proposed and no held-out case is burned. ## Applying a suggestion Your coding agent applies it. Over MCP ([Connect your agent](https://docs.moonbounce.ai/connect-your-agent.md)), `apply_suggestion` with the suggestion id returns step-by-step instructions, the diff and the candidate spec. The agent: **1. Finds the decision file** Usually `moonbounce/refund-exception.json`; the spec may also be inline in code. **2. Checks the base** If the local spec's structure hash equals the suggestion's base, it replaces the spec with the candidate. If not, it applies only the diff entries whose `before` still matches, and tells you which it skipped. **3. Sets the weights pin** To `weights_pin`: here it stays `"default"`, which already resolves to the candidate's weights. Writing the candidate's weights hash instead fails with `WEIGHTS_UNKNOWN` until that structure has been registered. **4. Runs the preview** With `suggestion_id`. `hashes_match` must be `{"structure": true, "weights": true}` when every entry applied. **5. Opens a pull request** On a branch, with your own git credentials, carrying the summary, the label the suggestion was proposed with, and the preview's numbers. It then marks the suggestion `pulled`. You review and merge. Nothing reaches production except through your merge: Moonbounce never writes to your repository. ### Statuses and on-demand suggestions A suggestion is `proposed` when made. `POST /v1/suggestions/{suggestion_id}/status` with `{"status": "pulled"}` marks it picked up; `{"status": "rejected", "reason": "..."}` turns it down (`reason` only goes with `rejected`). Proposed can become pulled, and proposed or pulled can become rejected; repeating the current status changes nothing, and anything else is 409. `accepted` is never set by you. `GET /v1/decisions/{decision_id}/suggestions` lists a decision's suggestions newest first, filtered by `status` and paged with `limit` (up to 100) and `cursor`. A suggestion read (get or list) carries `stale: { "reason": "...", "cases": 2 }` when feedback on cases it was evaluated on changed after its evidence was read: a reviewer changed or withdrew an answer, you corrected or judged one again, another reviewer answered one, or new feedback marked one's input as broken. Its evidence is never rescored; preview it again for current numbers. `POST /v1/decisions/{decision_id}/suggestions` asks for one now. The body may be empty: the base is then the version serving primary traffic in your key's environment (else the latest version seeded there), at the cut points its latest evaluation recorded. You can send `spec` and `weights` to start from your local version (the spec must already be registered, or it is 404 `VERSION_UNKNOWN`), `outcomes` to set the cut points (400 `OUTCOMES_INVALID` when none are known), and `kind` to generate that kind even when blame points to the other. A forced `new_feature` uses missing-criterion cases when there are any, otherwise all visible wrong cases within the prompt cap. A forced `feature_rewrite` rewrites the top suspect, or returns no suggestion when there is no suspect. The replay and propose rules still decide whether the candidate is proposed. The answer is `{"suggestion": ..., "reason": ...}`: * `suggestion: null` with a `reason` when there is nothing worth proposing yet: no cases to learn from, the spec already gets them right, there is no suspect to rewrite or no combine for a new feature to join, or no candidate passed the propose rule. * The existing suggestion, instead of a duplicate, when the same candidate was proposed before. * 404 when the decision has no evaluations or seeds to learn from. * 503 `ENGINE_UNAVAILABLE` with `Retry-After` when no suggestion writer is configured, the writer failed, or the engines could not score the cases. Retry with the same `Idempotency-Key`. ## Acceptance A suggestion is **accepted** when its candidate pair (structure hash and weights hash) first serves primary production traffic, `POST /v1/evaluate` on a production key, after the suggestion was made. Previews, seeds and test keys never count. A pair that already served production before the suggestion existed is a revert, not an acceptance. Only proposed or pulled suggestions made with a production key can be accepted; a test key's suggestions never are. It doesn't matter how the change reached your code. Moonbounce records acceptance when you next read the suggestion, the list or the loop status with a production key. `GET /v1/decisions/{decision_id}/loop` shows where the loop stands: ```json theme={null} { "decision_id": "refund-exception", "accepted": [ { "suggestion_id": "sug_01a17b2c-4d5e-7f60-8a91-b2c3d4e5f607", "evaluation_id": "evl_01a17c40-1a2b-7c3d-8e4f-5a6b7c8d9e0f", "accepted_at": "2026-10-28T14:11:52.317Z", "evidence_label": "holdout", "holdout_win": true } ], "reviews": 24, "random_lane": { "n": 8, "agree": 5, "nonresponse": 0.2, "changed_after_reveal": 0 }, "spot_check": { "before": { "n": 8, "agree": 5 }, "after": { "n": 0, "agree": 0 } }, "stop_check": { "reviews": 24, "threshold": 40, "any_holdout_win": true } } ``` `accepted` carries the label each suggestion was proposed with and whether that evidence was a held-out win. `reviews` counts live feedback from you and reviewers. `random_lane` and `spot_check` report the blind spot checks from [Review](https://docs.moonbounce.ai/learn/review.md): how often they agreed with Moonbounce, on the version that served production before the first acceptance and on accepted versions. They read each reviewer's first, blind answer; `changed_after_reveal` counts the spot checks whose answer was changed afterwards. `stop_check` is a prompt to rethink: after `threshold` reviews with no suggestion that won on held-out cases, more reviews of the same decision are unlikely to help. ```mermaid theme={null} sequenceDiagram participant You participant Agent as Your coding agent participant MB as Moonbounce participant App as Your app You->>MB: Reviews and feedback Agent->>MB: generate_suggestion MB-->>Agent: Suggestion: diff, candidate hashes, evidence Agent->>MB: apply_suggestion MB-->>Agent: Steps, diff, candidate spec Agent->>Agent: Edit the decision file Agent->>MB: preview with suggestion_id MB-->>Agent: Evidence, hashes_match Agent->>You: Pull request You->>App: Merge and deploy App->>MB: evaluate (production key) MB->>MB: Candidate hashes serve production: accepted ``` ## Next [Node types and patterns](https://docs.moonbounce.ai/learn/node-types.md) lists every node type, including the ones coming soon, and shows how grouping `carrier_delay` with `defective` would have helped ticket C while diluting ticket A less. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.