> ## Documentation Index > Fetch the complete documentation index at: https://docs.moonbounce.ai/llms.txt > Use this file to discover all available pages before exploring further. # Feedback > Tell Moonbounce what was true about an evaluation, and what each kind of feedback counts for. > [!NOTE] > > The examples on this page use Northpine, a fictional outdoor-gear shop whose `refund-exception` decision grants, escalates or denies refund requests outside its standard return policy. [How Moonbounce works](https://docs.moonbounce.ai/learn/how-it-works.md#the-running-example) introduces it. An evaluation is Moonbounce's answer for an item ([Evaluate content](https://docs.moonbounce.ai/learn/evaluations.md)). Feedback goes the other way: it says what the right answer was. Moonbounce stores every evaluation as it was and never rewrites it, so feedback sits beside the evaluation. Previews test changes against it and suggestions learn from it. This page follows the owner's feedback on Northpine's ticket C, then covers each part of a submission, who can send it, and what it counts for. ## The smallest submission Ticket C is the tent that arrived two weeks late. It scored 0.540 and landed on `escalate`. Northpine's owner knows it should have been `grant`, so they say so: ```bash theme={null} curl -sS --fail-with-body -X POST "$MOONBOUNCE_API_URL/v1/feedback" \ -H "Authorization: Bearer $MOONBOUNCE_API_KEY" \ -H 'Content-Type: application/json' \ -H "Idempotency-Key: $(uuidgen)" \ -d '{ "about": { "evaluation_id": "evl_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e0f" }, "claims": [{ "kind": "outcome", "value": "grant" }], "source": "owner", "reason": "The carrier delay ate their return window; that'\''s our failure." }' ``` The API answers `201` with the submission's id and the evaluation it was bound to: ```json theme={null} { "id": "fbk_0199f2c9-3b4c-7d5e-9f60-718293a4b5c6", "evaluation_id": "evl_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e0f", "item_id": "t-001", "claims": 1, "resolved_from_item": false } ``` A submission has four parts. `about` says which evaluation you're judging. `claims` holds one or more statements about it. `source` says who is speaking. `reason` says why, in your own words, and can be up to 4,000 characters. The reason matters more than it looks: when Moonbounce writes a suggestion, it reads the reasons on the cases it learns from. This reason is how the loop finds out that late deliveries are missing from the spec. ## What the feedback is about `about` takes one of two forms: * `{"evaluation_id": "evl_..."}` names the exact answer you're judging. Prefer it whenever you kept the id from the evaluate response. * `{"decision_id": "refund-exception", "item_id": "t-001"}` means the item's latest primary evaluation in your key's environment, at the moment the feedback arrives. A primary evaluation is one your app made through `POST /v1/evaluate`. Seed examples never count, so feedback by item can't land on a seed: for an item that has only seed evaluations, the `404` says so and names the seed's `evaluation_id`, which you can address directly (a `decisions:feedback`-only key gets the plain message). The response sets `resolved_from_item: true` and tells you which `evaluation_id` it picked. An evaluation in another environment is invisible to your key. Both forms answer `404` when they find nothing. A `verdict` claim (below) needs `evaluation_id`: it judges one specific answer, so a lookup by item is refused with a `400` on `about`. ## Claims Each claim is one statement about the evaluation, and its `kind` says which. Moonbounce checks every claim against the evaluation's own spec and cut points, the ones that were sent with that call. ### `outcome`: the right outcome ```json theme={null} { "kind": "outcome", "value": "grant" } ``` This is the claim to send most often. It names the outcome the item should have had, and that outcome must be one of the evaluation's own: `below` or one of its cuts. For Northpine that is `deny`, `escalate` or `grant`. Case matters, and a wrong name gets a `400` that lists the outcomes that exist: ```json theme={null} { "error": { "type": "invalid_request", "message": "\"Grant\" is not one of this evaluation's outcomes (deny, escalate, grant)", "param": "claims.0.value" } } ``` ### `target`: whether the target statement is true ```json theme={null} { "kind": "target", "value": true } ``` The target is the statement the score estimates. For Northpine it is "The refund exception should be granted." A `target` claim says whether that statement holds for this item. It needs a spec that declares a `target`. ### Middle outcomes say nothing about the target Outcome and target claims are linked, but only at the ends of the scale. Northpine's outcomes run from `deny` (below 0.38) to `grant` (from 0.60): * `grant` is the highest outcome, so it implies the target is true: the refund should be granted. * `deny` is the lowest outcome, so it implies the target is false. * `escalate` sits in the middle and implies nothing. "A person should look at this" doesn't say whether the refund should end up granted. A submission whose `outcome` and `target` contradict each other is refused. For example, `deny` with `target: true` gets `400` naming the later of the two claims (`outcome "deny" implies the target is false, but the target claim says true`). `escalate` with either target value is accepted. ### `verdict`: approve or reject our answer ```json theme={null} { "kind": "verdict", "value": "reject" } ``` `approve` means the outcome Moonbounce returned was right. `reject` means it was wrong. A verdict needs `about.evaluation_id` and an evaluation whose outcome was actually shown to someone. That means a primary evaluation that decided: it has a score and an outcome. An evaluation that abstained (it has no score) only carries a fallback outcome, and a seed evaluation was never shown, so both refuse verdicts. When a decision has only two outcomes, `reject` tells Moonbounce the right answer. With three or more, it only says "not this one". A rejected `escalate` on ticket C doesn't say whether the answer was `grant` or `deny`. Previews can only replay a case whose agreed answer names an outcome, so if you know the right outcome, send an `outcome` claim instead. ### `feature`: the right value of one feature ```json theme={null} { "kind": "feature", "feature": "outside_window", "value": true } ``` A feature claim gives the right value of one feature, either correcting the engine or confirming it. The `feature` must name a boolean feature of the evaluation's spec, and a `400` lists the ones that exist. Both uses matter on ticket C. The engine answered 0.95 for `outside_window` and 0.05 for `defective`, and both answers are right: the request is past the window, and nothing is broken. Confirming them (`outside_window: true`, `defective: false`) tells a suggestion that neither feature’s answer was wrong, so the spec must be missing one. Without the confirmations, a suggestion could blame either of them, since firmly flipping either one would turn ticket C into `grant`. A correction looks the same, with the value the engine got wrong: had the engine answered 0.8 for `item_used` on a tent that was never set up, you would send `item_used: false`. A feature claim refines a case but doesn't decide one. It tells a suggestion which features were right or wrong in a case that already has an agreed outcome. On its own it sets no right answer for the item. A feature claim is stored against the feature as it was evaluated, identified by the feature's node hash (see [Versions and hashes](https://docs.moonbounce.ai/learn/versions.md)). Renaming the feature keeps the claim. Changing the prompt's wording, its own guidance or criteria, or the spec's `guidance` gives the feature a new hash, and the claim stops applying: an answer judged against old wording says nothing about new wording. ### `input_broken`: the content can't be judged ```json theme={null} { "kind": "input_broken", "problem": "truncated" } ``` Send this when the content itself was the problem. `problem` is one of `truncated`, `wrong_language`, `bad_extraction`, `unreadable` or `other`. Previews and suggestions set the evaluation aside, whatever else was said about it. ### `action`: what your app did ```json theme={null} { "kind": "action", "value": "auto_approved", "logic_version": "refunds@4.2" } ``` An action claim records what your app actually did with the outcome, in your own words (up to 128 characters), with an optional `logic_version` for your own code. It describes what happened, not what was right, so it never sets an answer. Use it to keep Moonbounce's record of the item next to what your system did. ## Rules for one submission A submission holds 1 to 65 claims. Within one submission: * at most one `outcome`, one `target` and one `verdict` claim, and one claim per feature; * an `outcome` and a `target` must not contradict each other at the ends of the scale. A claim that breaks a rule, or that the evaluation can't take, gets a `400` whose `param` names it (`claims.` or a field inside it) and whose message says what is allowed. Nothing in a refused submission is stored. You can also set `unsure: true` when the person judging isn't confident. It is stored with the submission and included in the export. ## Sources `source` says who is speaking. That decides what the feedback counts for. | Source | Sent through | Key scope | Decides a case | | - | - | - | - | | `owner` | `POST /v1/feedback` | `decisions:develop` | Yes, above everything else | | reviewer | The Review page, or the review API | None on the Review page; `decisions:develop` for the API | Yes, by vote | | `import` (seeds) | `POST /v1/decisions/{id}/seeds` | `decisions:develop` | Only when no person has judged it | | `app_native` | `POST /v1/feedback` | `decisions:feedback` | No | | `app_resolution` | `POST /v1/feedback` | `decisions:feedback` | No | A key with `decisions:develop` can send every source. A key that is missing the scope a source needs gets `403 INSUFFICIENT_SCOPE`. * **`owner`** is the decision owner's judgment: yours, or that of someone you trust to decide. All owner feedback is one voice, whoever sends it. The latest owner submission with an `outcome`, `target` or `verdict` claim decides the case and replaces earlier owner judgments. Owner feedback can't name an `actor` (`403`), because the owner is you. * **Reviewers** answer on the [Review](https://docs.moonbounce.ai/learn/review.md) page or through the review API. Each reviewer gets one vote, and a tie leaves the case contested, which previews leave out. * **Seeds** are the labeled starting examples described below. They decide a case only when no owner or reviewer has judged it. * **`app_native`** is your app's own feedback about the outcome it acted on, for example a user approved or reverted it. * **`app_resolution`** is how your own process finally settled the case, for example an appeal. The owner outranks reviewers, and reviewers outrank seeds. The two `app_` sources are weak feedback: they never decide a case. They can name an `actor`, your own id for whoever produced the feedback, which is recorded under your key's namespace. ### What weak feedback still does `app_native` and `app_resolution` never decide a case's right answer, but they aren't ignored: * **They can fix a case's split.** The first feedback a case gets decides whether that case can ever be held out to test suggestions (see [Train and holdout](#train-and-holdout)). If a native verdict arrives before your review, the case goes to training for good. * **Their reasons reach the suggestion writer.** When another source has decided a case, the reasons from every live submission on it, weak ones included, are shown alongside it. * **`input_broken` from any source sets the evaluation aside.** ## What decides a case For each evaluation, Moonbounce works out one agreed answer from its live feedback. The owner decides if they have judged it. Otherwise the reviewers' majority decides, and the seed decides only if nobody else has. Previews replay a case only when it is a primary or seed evaluation, its content is stored, and its agreed answer names one of the preview's outcomes. A case with only a `target` claim, or only a rejected verdict among three outcomes, is counted as left out and not replayed. ## Train and holdout Moonbounce keeps some cases back so it can test a suggestion on cases the suggestion writer never saw. Each case is assigned once, at its first feedback, and keeps that assignment: * owner or reviewer first: about 70% go to **train** and 30% to **holdout**, chosen by a hash, so the same case always lands on the same side; * `app_native` or `app_resolution` first: **train**; * a seed: **seed**, used for fitting only. A case is an item together with any items whose text is the same once case, digits and id-like tokens are ignored, so a templated message sent twice with different order numbers is one case, not two. Cases are per environment: the same item or text in test and in production is two cases, each with its own split, so test feedback never fixes or burns a production case. A held-out case becomes **burned**, which means training from then on, when a preview lists it individually, when it merges with a near-duplicate case, or when its text arrives later as a seed example. After any of those, the case can no longer be trusted as a test of something that never saw it. ## Corrections Send a correction as a new submission that names the one it replaces. Here Northpine's owner resends ticket C's feedback with the two feature confirmations described above: ```json theme={null} { "about": { "evaluation_id": "evl_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e0f" }, "claims": [ { "kind": "outcome", "value": "grant" }, { "kind": "feature", "feature": "outside_window", "value": true }, { "kind": "feature", "feature": "defective", "value": false } ], "source": "owner", "reason": "The carrier delay ate their return window; that's our failure.", "supersedes": "fbk_0199f2c9-3b4c-7d5e-9f60-718293a4b5c6" } ``` The response carries a new id, `fbk_0199f2d1-7e8f-7a90-b1c2-d3e4f5a6b7c8`. From then on that submission is ticket C's owner feedback, and the first one shows `superseded_by` with the new id. `supersedes` must name a submission by the same actor, on the same evaluation, that nothing has replaced yet. Anything else is a `409`. For feedback sent with an API key, the same actor means the same key, plus the same `actor` value when you named one. Only the latest submission in a chain counts. The earlier one stays readable and shows `superseded_by`. Replacing owner, reviewer or seed feedback needs `decisions:develop`. A `decisions:feedback` key can replace only its own `app_native` and `app_resolution` submissions. Nobody replaces someone else's judgment. A different person adds their own submission, and the sources above decide which one counts. ## Reading feedback back `GET /v1/feedback/{feedback_id}` returns any submission in your key's environment as stored: one from `POST /v1/feedback`, a seed example's, or a review answer. It shows the claims as they were sent, plus `source`, `channel` (the surface it came through, set by the server), `actor`, `supersedes`, `superseded_by` and the case's `split`. A reviewer's answer on a spot check also carries `after_reveal`: `false` on the blind answer, `true` on a later change of it ([Review](https://docs.moonbounce.ai/learn/review.md#change-your-answer)), except while its judgment is withheld. A replaced answer keeps the `reason_pending` it had: `true` there means its reason never arrived before it was replaced, so read the latest answer in the chain. Ticket C's case landed in train, so its feedback reads back in full. A submission on a held-out case reads back without its judgment, even to its author. Here is the owner's feedback on another Northpine ticket, one that landed in holdout: ```json theme={null} { "id": "fbk_0199f3a2-5c6d-7e8f-a091-b2c3d4e5f607", "decision_id": "refund-exception", "evaluation_id": "evl_0199f3a0-2b3c-7d4e-8f50-6a7b8c9d0e1f", "item_id": "t-017", "resolved_from_item": false, "source": "owner", "channel": "api", "actor": "key:", "withheld": "holdout", "claim_count": 1, "split": "holdout", "created_at": "2026-10-27T15:04:11.000Z" } ``` `withheld: "holdout"` means the claims, reason and `unsure` flag are kept back. A later preview tests a suggestion on held-out cases, and that test means something only if neither the suggestion writer nor the agent asking for it has read their answers. You still see that the submission was recorded and how many claims it holds. The judgment comes back once the case is burned. `withheld: "unassigned"` is the same for an item that has no split yet. ## Seeds A new decision has nothing to test a change against. Seeds fix that: about a dozen labeled examples you send once, so previews have cases to replay from day one. `POST /v1/decisions/{decision_id}/seeds` takes the version (an inline `spec`, or the `structure` hash of a registered one, as on evaluate), `weights`, `outcomes` and the examples. Here are three of Northpine's twelve, sent to `POST /v1/decisions/refund-exception/seeds`: ```json theme={null} { "structure": "sha256:c21908531c5bb1e922591c1b6d2d9e238205a6ee4c87cddff4674003e880b313", "weights": "default", "outcomes": { "below": "deny", "cuts": [{ "at": 0.38, "outcome": "escalate" }, { "at": 0.6, "outcome": "grant" }] }, "examples": [ { "item_id": "seed-01", "subject": [{ "type": "text", "name": "customer_message", "text": "Order $189, delivered 39 days ago, no refunds in the last year. The tent showed up two weeks late (stuck at the regional hub), I used it once on the trip it was for. Can I still return it?" }], "claims": [{ "kind": "outcome", "value": "grant" }], "reason": "The carrier delay ate their return window; that's our failure." }, { "item_id": "seed-02", "subject": [{ "type": "text", "name": "customer_message", "text": "Order $129, delivered 42 days ago, no refunds in the last year. The zipper on my sleeping bag ripped out the first night." }], "claims": [{ "kind": "outcome", "value": "grant" }], "reason": "Defect under normal use." }, { "item_id": "seed-03", "subject": [{ "type": "text", "name": "customer_message", "text": "Order $899, delivered 61 days ago, no refunds in the last year. I just changed my mind about the kayak, it has been in the garage for two months." }], "claims": [{ "kind": "outcome", "value": "deny" }], "reason": "Change of mind, well outside the window." } ] } ``` The hash is the structure hash of Northpine's spec. Sending `"spec"` with the contents of `moonbounce/refund-exception.json` instead works too, and registers it. The decision comes from the path, so the body has no `decision_id`. Each example takes the same `subject`, `context`, `summary` and `metadata` fields as an evaluate request, plus `claims`, `reason` and `unsure`. Examples can't carry a `verdict`, since nothing was shown yet. Outcome claims need `outcomes` in the request, because a claim is checked against the cut points. Each example runs through the real engine as an evaluation in `seed` mode. The response lists each one with its `evaluation_id`, `feedback_id`, `outcome`, `score` and `agreed`: true when every checked outcome, target and feature claim agrees (feature answers use p ≥ 0.5). A single feature conflict makes it false. The top-level `agreed` counts results whose `agreed` is true. Seed-01 would come back `escalate` with `agreed: false`. That's the carrier-delay gap showing up before any traffic. `agreed` is `null` when nothing could be checked: the evaluation abstained, or the example carries only a `target` claim and landed on a middle outcome. `conflicts` lists each feature claim the engine's answer disagrees with. Each result also carries `cost: {engine_calls, cache_hits, input_tokens, usd_micros}` with the same fields as a preview. `engine_calls` counts upstream engine calls on evaluate and seeds; one call can answer several features (Jev compiles them). On previews it counts features sent to engines instead. The summary's `cost` sums those fields across the batch, using each evaluation's recorded usage (including its rounded money amount). `usd_micros` is millionths of a US dollar. Only previews use the feature cache today; evaluate and seeds report `cache_hits: 0`. `getEvaluation` returns the same per-example cost when a usage row exists; otherwise it omits `cost`. Fresh seed responses always include both levels of cost. A replay of a snapshot stored before costs were reported omits cost from both the summary and every result until those snapshots expire, within 24 hours after deployment; replayed cost is the original spend, not a new charge. Suggestion generation's own cost isn't in per-request `cost`; the usage report shows it under `other_spend`. Things to know about seeds: * **All or nothing.** Every example's content and claims are checked before any engine call, and a `400` names the example (`examples....`). If any example's engine call fails, the batch is refused with `502 ENGINE_FAILED`, `details.items` names every failed example, and nothing is saved. A batch can hold up to 200 examples. A batch too big to evaluate in one request is a `422` telling you to send smaller batches. * **Never traffic.** Seed evaluations are real evaluations, but feedback by item never lands on them, the review queue never offers them, and they never count as a version serving traffic. * **The weakest evidence.** Seed cases are used for fitting and are never held out. A preview tested only on seeds carries the Starting examples label, the weakest there is ([Suggestions and previews](https://docs.moonbounce.ai/learn/improving.md)). * **One key per batch.** One `Idempotency-Key` covers the whole batch, so sending it again replays the first result without calling an engine. ## Exporting feedback `GET /v1/decisions/{decision_id}/feedback` returns the raw feedback on a decision, oldest first and paged with `cursor`. Each submission comes with the evaluation it judged and its split. It needs `decisions:develop`, is HTTP only (not an MCP tool), and never lists held-out feedback: those submissions are only counted in `withheld` on the first page. ## Retries `POST /v1/feedback` and the seeds endpoint honor `Idempotency-Key` the way evaluate does: retry with the same key and you get the first result with `Idempotent-Replayed: true`, never a second submission. Who sent it is part of the request: for an API key, that is the key itself plus any `actor` you name. So the same `Idempotency-Key` and body sent through a different API key, a rotated one included, or with a different `actor`, is a `422` rather than a replay. ## Next Sending feedback for every item doesn't scale. [Review](https://docs.moonbounce.ai/learn/review.md) shows how Moonbounce picks the few items a day worth your judgment, including the blind spot checks that measure how often it agrees with you. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.