> ## Documentation Index > Fetch the complete documentation index at: https://docs.moonbounce.ai/llms.txt > Use this file to discover all available pages before exploring further. # Run a version over many items > Evaluates up to 200 items under one version, each exactly as one evaluate would, and keeps the results under a `run_id` for loss_report and sweep_cut. The body is flat, as import_examples' is: the structure inline (`questions`, `attributes`, `aggregations`, `result`) or a registered one's `structure_hash`, never both; then `items` or `cases`, `dry_run` and `include`. The decision is the path's. **Items:** each `{item_id, state, attribute_values?, expected?, label_source?}`, `item_id` unique in the run. Each is stored as an evaluation, or with `dry_run: true` is a dry run that stores only usage; each result carries its `id` (`evl_` or `dry_`). Score versions with dry runs: a stored run's evaluations are real traffic. `expected` (an outcome) and `label_source` (`human`, `synthetic` or `unknown`) are the item's label, kept with the run for loss_report, never used to evaluate and never recorded as feedback. Every item's content is checked before any model call: content the structure refuses is 400 CONTENT_INVALID naming `items.`. An item whose model calls fail is that item's result (status `failed`, with an `error` whose `code` says why and `retryable` whether sending it again may help); the call fails only on an invalid request, auth, admission or a store error. An outage of any kind (unavailable, rate limited, timeout, auth) is `retryable`; only the content's own failure is not. **Stored cases:** send `cases` in place of `items` to evaluate the decision's labeled cases (the ones list_cases lists; this needs the `decisions:develop` scope): `{evaluation_ids}`, or list_cases' filters (`outcomes`, `structure_hash`, `label_source`, `since`, `until`) with `limit` (default 200) and `cursor` (`cases.next_cursor` from the previous run). They are always dry runs: each case is replayed as a preview replays it, reusing the answer it stored for a question that hasn't changed (same model) and asking the rest; `refresh` names questions to ask afresh. Each result carries the case's `evaluation_id`, and the run keeps its label, label source and answer corrections, so a preview equals a case run under the base (refreshing the questions the candidate changed or dropped), one under the candidate, and loss_report over the two, when both runs set `cases.outcomes` to the outcomes either version can produce and cover the same cases (a preview judges the newest 500: page both runs through them while no label changes; a page is not a snapshot), and both versions can read every case's stored content. The run reports `version` and `model` as evaluate does. It is kept until `expires_at`: pass its `run_id` to loss_report to judge it, so the results never pass through your context; a stored run's evaluations stay after it expires, and it can't be read back by id. `include` adds each result's `answers`, `attributes`, or `evaluation` (everything one evaluate returns for that item). Each item that succeeded is billed as one evaluate or dry run would be; a failed item's spend is recorded and shown as its `cost`. A run has a 60 s budget and takes no `model`; items it could not start fail with `NOT_STARTED`, and a case it ran out of time for with `ENGINE_UNAVAILABLE` (both retryable): send them in a new run with a new Idempotency-Key. 201 with the run; an Idempotency-Key replays the first response without spending again. ## OpenAPI ````yaml /openapi.json post /v1/decisions/{decision_id}/runs openapi: 3.1.0 info: title: Moonbounce Decisions API version: 0.0.0 description: >- Moonbounce decisions: evaluate an item's state against a decision structure (System One questions, attributes you supply and aggregations over both), record feedback and import labeled examples, review a ranked queue, and improve the structure through suggestions that ship as code diffs, each with a preview (backtest) and an evidence label. **Authentication.** Send an API key as `Authorization: Bearer ` on every operation except `GET /health` and `POST /v1/validate`. Call the API from a server, never a browser: a key must not reach one, and the API sends no CORS headers. A key belongs to one organization and one environment: production, or test for a key with the `env:test` permission. Test traffic never counts as production traffic. **Requests.** JSON bodies (`Content-Type: application/json`) of at most 1 MB, parsed strictly: duplicate keys, lone surrogates, integers outside ±(2^53 - 1), `__proto__` keys, U+0000 and nesting deeper than 64 are a 400. **Retries.** Send an `Idempotency-Key` on every POST under /v1 that takes one (all but `POST /v1/validate` and `POST /v1/review/items/fill`, whose retry is another fill within the day's budget). Keep it for the request's retries. The same key with the same request returns the first stored result, marked `Idempotent-Replayed: true`, for 24 hours (410 if that result was since deleted); with another request it is 422 IDEMPOTENCY_KEY_REUSED. Keys are scoped to the organization and environment across every operation, so use a fresh key per request. A request that failed before its result was stored runs again when retried. 429 and 503 under /v1 may carry `Retry-After` (seconds); a daily QUOTA_EXCEEDED cap instead names its reset time. **Errors.** Every error under /v1 is `{"error": {"type", "message", "reason"?, "param"?, "details"?}}` (ErrorBody); `GET /health` answers its own shape. Every response carries a `Request-Id` header. Rate limits apply per key, and per client IP for requests without a valid key; a 429 RATE_LIMITED carries `Retry-After`. servers: - url: https://api.moonbounce.ai description: Production security: [] paths: /v1/decisions/{decision_id}/runs: post: tags: - evaluate summary: Run a version over many items description: >- Evaluates up to 200 items under one version, each exactly as one evaluate would, and keeps the results under a `run_id` for loss_report and sweep_cut. The body is flat, as import_examples' is: the structure inline (`questions`, `attributes`, `aggregations`, `result`) or a registered one's `structure_hash`, never both; then `items` or `cases`, `dry_run` and `include`. The decision is the path's. **Items:** each `{item_id, state, attribute_values?, expected?, label_source?}`, `item_id` unique in the run. Each is stored as an evaluation, or with `dry_run: true` is a dry run that stores only usage; each result carries its `id` (`evl_` or `dry_`). Score versions with dry runs: a stored run's evaluations are real traffic. `expected` (an outcome) and `label_source` (`human`, `synthetic` or `unknown`) are the item's label, kept with the run for loss_report, never used to evaluate and never recorded as feedback. Every item's content is checked before any model call: content the structure refuses is 400 CONTENT_INVALID naming `items.`. An item whose model calls fail is that item's result (status `failed`, with an `error` whose `code` says why and `retryable` whether sending it again may help); the call fails only on an invalid request, auth, admission or a store error. An outage of any kind (unavailable, rate limited, timeout, auth) is `retryable`; only the content's own failure is not. **Stored cases:** send `cases` in place of `items` to evaluate the decision's labeled cases (the ones list_cases lists; this needs the `decisions:develop` scope): `{evaluation_ids}`, or list_cases' filters (`outcomes`, `structure_hash`, `label_source`, `since`, `until`) with `limit` (default 200) and `cursor` (`cases.next_cursor` from the previous run). They are always dry runs: each case is replayed as a preview replays it, reusing the answer it stored for a question that hasn't changed (same model) and asking the rest; `refresh` names questions to ask afresh. Each result carries the case's `evaluation_id`, and the run keeps its label, label source and answer corrections, so a preview equals a case run under the base (refreshing the questions the candidate changed or dropped), one under the candidate, and loss_report over the two, when both runs set `cases.outcomes` to the outcomes either version can produce and cover the same cases (a preview judges the newest 500: page both runs through them while no label changes; a page is not a snapshot), and both versions can read every case's stored content. The run reports `version` and `model` as evaluate does. It is kept until `expires_at`: pass its `run_id` to loss_report to judge it, so the results never pass through your context; a stored run's evaluations stay after it expires, and it can't be read back by id. `include` adds each result's `answers`, `attributes`, or `evaluation` (everything one evaluate returns for that item). Each item that succeeded is billed as one evaluate or dry run would be; a failed item's spend is recorded and shown as its `cost`. A run has a 60 s budget and takes no `model`; items it could not start fail with `NOT_STARTED`, and a case it ran out of time for with `ENGINE_UNAVAILABLE` (both retryable): send them in a new run with a new Idempotency-Key. 201 with the run; an Idempotency-Key replays the first response without spending again. operationId: createRun parameters: - schema: $ref: '#/components/schemas/DecisionId' required: true description: >- A name you choose for this decision, such as `refund-exception`: a lowercase letter, then up to 127 of `a-z 0-9 _ . -`. Unique in your organization and shared by its test and production keys. The first registration or evaluation under a new one creates the decision. name: decision_id in: path - schema: type: string minLength: 1 maxLength: 255 pattern: ^[\x20-\x7e]+$ description: >- Makes a retry safe. For 24 hours, the same key with the same request returns the first stored result (with Idempotent-Replayed: true); the same key with a different request is a 422 IDEMPOTENCY_KEY_REUSED. Keys are scoped to the organization and environment across every operation, so use a fresh key per request. A request that failed before its result was stored runs again when retried with its key. required: false description: >- Makes a retry safe. For 24 hours, the same key with the same request returns the first stored result (with Idempotent-Replayed: true); the same key with a different request is a 422 IDEMPOTENCY_KEY_REUSED. Keys are scoped to the organization and environment across every operation, so use a fresh key per request. A request that failed before its result was stored runs again when retried with its key. name: idempotency-key in: header requestBody: required: true content: application/json: examples: labeled_items: summary: >- Score the registered version on two labeled tickets with dry runs, for loss_report value: structure_hash: >- sha256:bdad6e5b1b879209305f1555b43ec62666a4d74a925280c66e54cc4dc07bb1ae dry_run: true items: - item_id: t-001 state: message: >- My tent showed up two weeks late, I used it once on the trip it was for, and now it's day 39. Can I still return it? attribute_values: days_since_delivery: 39 expected: escalate label_source: human - item_id: t-002 state: message: >- The zipper broke the first time I opened the bag. It arrived four days ago. attribute_values: days_since_delivery: 4 expected: grant label_source: human stored_cases: summary: >- Replay the decision's human-labeled cases under a candidate structure, inline value: questions: item_used: type: noul instructions: >- Has the customer used the item beyond trying it on or setting it up once? reads: - state.message defective: type: noul instructions: >- Does the customer describe a defect, damage on arrival, or the item failing under normal use? reads: - state.message attributes: days_since_delivery: type: number minimum: 0 aggregations: grant_score: type: weighted_mean inputs: - name: questions.item_used invert: true - name: questions.defective handling: type: rules rules: - when: name: attributes.days_since_delivery gt: 30 outcome: escalate - when: name: aggregations.grant_score gte: 0.72 outcome: grant - when: name: aggregations.grant_score lt: 0.38 outcome: deny else: escalate result: aggregations.handling cases: label_source: human limit: 200 schema: $ref: '#/components/schemas/CreateRunRequest' responses: '201': description: The run headers: Idempotent-Replayed: schema: type: string enum: - 'true' description: >- Present and `true` when this is the stored result of an earlier request with the same Idempotency-Key. required: false description: >- Present and `true` when this is the stored result of an earlier request with the same Idempotency-Key. content: application/json: examples: labeled_items: summary: >- Two labeled tickets dry-run on version 1, kept for loss_report value: run_id: evr_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e20 expires_at: '2026-11-07T10:00:00.000Z' dry_run: true environment: test decision_id: refund-exception version: structure_hash: >- sha256:bdad6e5b1b879209305f1555b43ec62666a4d74a925280c66e54cc4dc07bb1ae number: 1 model: id: jev-1.14.0 resolved_model: jev-1.14.0 revision: 1 profile: >- defective=jev:jev-1.14.0@jev@0.1.0,item_used=jev:jev-1.14.0@jev@0.1.0 counts: total: 2 decided: 2 computed: 0 abstained: 0 failed: 0 labeled: 2 cost: model_calls: 2 questions: 4 input_tokens: 1040 output_tokens: 8 usd_micros: 42 results: - item_id: t-001 id: dry_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e21 status: decided outcome: escalate cost: model_calls: 1 questions: 2 input_tokens: 520 output_tokens: 4 usd_micros: 21 - item_id: t-002 id: dry_0199f2c4-1a2b-7c3d-8e4f-5a6b7c8d9e22 status: decided outcome: grant cost: model_calls: 1 questions: 2 input_tokens: 520 output_tokens: 4 usd_micros: 21 schema: $ref: '#/components/schemas/Run' '400': description: 'Invalid request: `param` names the field at fault when one applies' content: application/json: schema: $ref: '#/components/schemas/ErrorBody' '401': description: Missing or invalid API key (reason KEY_INVALID) content: application/json: schema: $ref: '#/components/schemas/ErrorBody' '403': description: The key may not do this content: application/json: schema: $ref: '#/components/schemas/ErrorBody' '404': description: Not found, or not in this key's organization and environment content: application/json: schema: $ref: '#/components/schemas/ErrorBody' '409': description: Conflict with the current state content: application/json: schema: $ref: '#/components/schemas/ErrorBody' '410': description: >- The Idempotency-Key's first result was deleted, so it cannot be replayed content: application/json: schema: $ref: '#/components/schemas/ErrorBody' '413': description: The HTTP body exceeds 1 MB (reason BODY_TOO_LARGE) content: application/json: schema: $ref: '#/components/schemas/ErrorBody' '422': description: >- The request is well formed but cannot be processed (IDEMPOTENCY_KEY_REUSED requires a fresh key; REQUEST_TOO_LARGE requires a smaller request) content: application/json: schema: $ref: '#/components/schemas/ErrorBody' '429': description: >- RATE_LIMITED: retry after Retry-After seconds; QUOTA_EXCEEDED: details.reset_at gives the daily reset time (no Retry-After) headers: Retry-After: schema: type: string description: Seconds to wait before retrying, when waiting helps. required: false description: Seconds to wait before retrying, when waiting helps. content: application/json: schema: $ref: '#/components/schemas/ErrorBody' '503': description: >- Temporarily unavailable, with Retry-After (for example reason AUTH_UNAVAILABLE, ENGINE_UNAVAILABLE or STORE_UNAVAILABLE): retry with the same Idempotency-Key headers: Retry-After: schema: type: string description: Seconds to wait before retrying, when waiting helps. required: true description: Seconds to wait before retrying, when waiting helps. content: application/json: schema: $ref: '#/components/schemas/ErrorBody' security: - bearerAuth: - decisions:evaluate components: schemas: DecisionId: type: string pattern: ^[a-z][a-z0-9_.-]{0,127}$ description: >- A name you choose for this decision, such as `refund-exception`: a lowercase letter, then up to 127 of `a-z 0-9 _ . -`. Unique in your organization and shared by its test and production keys. The first registration or evaluation under a new one creates the decision. CreateRunRequest: anyOf: - type: object properties: questions: type: object additionalProperties: $ref: '#/components/schemas/Question' description: System One questions, keyed by name. attributes: type: object additionalProperties: $ref: '#/components/schemas/Attribute' description: >- Values your app supplies with each call in `attribute_values`. The model never sees them. aggregations: type: object additionalProperties: $ref: '#/components/schemas/Aggregation' result: type: string minLength: 1 maxLength: 80 description: The final feature's path, such as `aggregations.band`. examples: - questions.asks_for_refund - aggregations.band items: type: array items: $ref: '#/components/schemas/RunItem' minItems: 1 maxItems: 200 description: >- The items to evaluate under the same version, each exactly as one evaluate would. Send `items` or `cases`. cases: $ref: '#/components/schemas/RunCases' refresh: type: array items: $ref: '#/components/schemas/Name' minItems: 1 description: >- With `cases`: questions that get fresh answers instead of the ones the cases stored (as a preview refreshes the base questions a candidate changed or dropped). include: type: array items: type: string enum: - answers - attributes - evaluation description: >- Add to each result its `answers`, its `attributes` (as resolved, defaults applied), or its `evaluation`: everything evaluate returns for that item on its own. Default: none. dry_run: type: boolean description: >- `true`: every item is a dry run, storing no evaluation (only usage, and the run's results for a day). Absent or `false`: each item is stored as an evaluation, real traffic. A run of `cases` is always a dry run. structure_hash: not: {} description: >- Must be absent: send the structure inline or by its hash, not both. x-moonbounce-absent: true required: - questions - result additionalProperties: false - type: object properties: items: type: array items: $ref: '#/components/schemas/RunItem' minItems: 1 maxItems: 200 description: >- The items to evaluate under the same version, each exactly as one evaluate would. Send `items` or `cases`. cases: $ref: '#/components/schemas/RunCases' refresh: type: array items: $ref: '#/components/schemas/Name' minItems: 1 description: >- With `cases`: questions that get fresh answers instead of the ones the cases stored (as a preview refreshes the base questions a candidate changed or dropped). include: type: array items: type: string enum: - answers - attributes - evaluation description: >- Add to each result its `answers`, its `attributes` (as resolved, defaults applied), or its `evaluation`: everything evaluate returns for that item on its own. Default: none. dry_run: type: boolean description: >- `true`: every item is a dry run, storing no evaluation (only usage, and the run's results for a day). Absent or `false`: each item is stored as an evaluation, real traffic. A run of `cases` is always a dry run. questions: not: {} description: >- Must be absent: send the structure inline or by its hash, not both. x-moonbounce-absent: true attributes: not: {} description: >- Must be absent: send the structure inline or by its hash, not both. x-moonbounce-absent: true aggregations: not: {} description: >- Must be absent: send the structure inline or by its hash, not both. x-moonbounce-absent: true result: not: {} description: >- Must be absent: send the structure inline or by its hash, not both. x-moonbounce-absent: true structure_hash: $ref: '#/components/schemas/StructureHash' required: - structure_hash additionalProperties: false Run: type: object properties: run_id: type: string pattern: >- ^evr_[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$ description: >- Pass it to loss_report; kept until `expires_at`. A stored run's evaluations stay after it expires. expires_at: $ref: '#/components/schemas/TimestampOutput' dry_run: type: boolean description: >- Whether the items were dry runs (nothing stored) or stored as evaluations. environment: $ref: '#/components/schemas/Environment' decision_id: $ref: '#/components/schemas/DecisionIdOutput' version: $ref: '#/components/schemas/EvaluatedVersion' model: $ref: '#/components/schemas/EvaluatedModel' counts: type: object properties: total: type: integer minimum: 0 decided: type: integer minimum: 0 computed: type: integer minimum: 0 abstained: type: integer minimum: 0 failed: type: integer minimum: 0 labeled: type: integer minimum: 0 description: Items that carried `expected`. required: - total - decided - computed - abstained - failed - labeled additionalProperties: true x-moonbounce-sdk-known-fields: true cost: $ref: '#/components/schemas/Cost' results: type: array items: $ref: '#/components/schemas/RunItemResult' description: 'One per item, in item order (cases: newest label first).' cases: type: object properties: total: type: integer minimum: 0 description: >- Cases that match the filters, across every page; with `evaluation_ids`, those found. excluded: $ref: '#/components/schemas/CaseExcluded' next_cursor: anyOf: - type: string - type: 'null' description: Pass back in `cases.cursor` for the next page. required: - total - excluded - next_cursor additionalProperties: true description: 'With `cases`: which cases the run took.' x-moonbounce-sdk-known-fields: true required: - run_id - expires_at - dry_run - environment - decision_id - version - model - counts - cost - results additionalProperties: true x-moonbounce-sdk-known-fields: true ErrorBody: type: object properties: error: type: object properties: type: $ref: '#/components/schemas/ErrorType' reason: type: string pattern: ^[A-Z][A-Z0-9_]+$ description: >- Set where a caller can act on it, one of the reasons in the ErrorReason schema. New reasons may be added: treat an unknown one by the status and `type`. message: type: string description: For people; don't parse it. param: type: string description: >- The request field at fault, as a dotted path (`attribute_values.region`, `claims.1.value`). details: type: object properties: issues: type: array items: $ref: '#/components/schemas/ErrorIssue' additionalProperties: {} required: - type - message additionalProperties: true x-moonbounce-sdk-known-fields: true required: - error additionalProperties: true x-moonbounce-sdk-known-fields: true Question: oneOf: - $ref: '#/components/schemas/NoulQuestion' discriminator: propertyName: type mapping: noul: '#/components/schemas/NoulQuestion' Attribute: oneOf: - $ref: '#/components/schemas/StringAttribute' - $ref: '#/components/schemas/NumberAttribute' - $ref: '#/components/schemas/BooleanAttribute' discriminator: propertyName: type mapping: string: '#/components/schemas/StringAttribute' number: '#/components/schemas/NumberAttribute' boolean: '#/components/schemas/BooleanAttribute' Aggregation: oneOf: - $ref: '#/components/schemas/WeightedMean' - $ref: '#/components/schemas/Logistic' - $ref: '#/components/schemas/Min' - $ref: '#/components/schemas/Max' - $ref: '#/components/schemas/Product' - $ref: '#/components/schemas/NoisyOr' - $ref: '#/components/schemas/All' - $ref: '#/components/schemas/Any' - $ref: '#/components/schemas/Bands' - $ref: '#/components/schemas/Rules' - $ref: '#/components/schemas/Match' discriminator: propertyName: type mapping: weighted_mean: '#/components/schemas/WeightedMean' logistic: '#/components/schemas/Logistic' min: '#/components/schemas/Min' max: '#/components/schemas/Max' product: '#/components/schemas/Product' noisy_or: '#/components/schemas/NoisyOr' all: '#/components/schemas/All' any: '#/components/schemas/Any' bands: '#/components/schemas/Bands' rules: '#/components/schemas/Rules' match: '#/components/schemas/Match' RunItem: type: object properties: item_id: type: string minLength: 1 maxLength: 256 description: >- Your id for the item, unique in the run: its result and loss rows carry it. state: description: The content the questions evaluate, as on evaluate. attribute_values: type: object additionalProperties: anyOf: - type: string maxLength: 200 - type: number - type: boolean description: The item's attribute values, as on evaluate. expected: type: string minLength: 1 maxLength: 200 description: >- The outcome the item should get. Carried with the run for loss_report; evaluation ignores it. label_source: type: string enum: - human - synthetic - unknown description: Where `expected` came from. Default `unknown`. required: - item_id - state additionalProperties: false RunCases: anyOf: - type: object properties: evaluation_ids: type: array items: $ref: '#/components/schemas/EvlId' minItems: 1 maxItems: 200 description: Labeled cases by id (list_cases gives them). required: - evaluation_ids additionalProperties: false - type: object properties: outcomes: type: array items: $ref: '#/components/schemas/Outcome' minItems: 1 description: Only cases labeled one of these outcomes. structure_hash: type: string pattern: ^sha256:[0-9a-f]{64}$ description: Only cases that ran under this version. label_source: $ref: '#/components/schemas/LabelSource' since: type: string format: date-time description: Only cases whose evaluation ran at or after this time. until: type: string format: date-time description: Only cases whose evaluation ran before this time. limit: type: integer minimum: 1 maximum: 200 description: Cases per run. Default 200. cursor: type: string minLength: 1 maxLength: 1024 description: '`cases.next_cursor` from the previous run.' additionalProperties: false description: >- The decision's labeled cases to evaluate, in place of `items`: by `evaluation_ids`, or by the filters list_cases takes (the same cases, newest label first, `limit` per run; pass `cases.next_cursor` back for the next page). Always dry runs. Each case is replayed: a question unchanged since the case ran reuses the answer it stored (the same model), the rest are asked. Its label, label source and answer corrections go with the run for loss_report. Name: type: string pattern: ^[a-z][a-z0-9_]{0,63}$ StructureHash: type: string pattern: ^sha256:[0-9a-f]{64}$ TimestampOutput: type: string format: date-time Environment: type: string enum: - production - test description: >- The environment of the key that made the call: `production` or `test`. Evaluations, feedback, reviews and suggestions are separate per environment; decisions and their versions belong to the whole organization. DecisionIdOutput: type: string pattern: ^[a-z][a-z0-9_.-]{0,127}$ description: >- A name you choose for this decision, such as `refund-exception`: a lowercase letter, then up to 127 of `a-z 0-9 _ . -`. Unique in your organization and shared by its test and production keys. The first registration or evaluation under a new one creates the decision. EvaluatedVersion: type: object properties: structure_hash: $ref: '#/components/schemas/StructureHashOutput' number: type: - integer - 'null' minimum: 1 description: >- The version's number under the decision (1, 2, 3...). Null when the structure isn't registered: a dry run or a preview of a structure nobody registered. The hash is the identity. required: - structure_hash - number additionalProperties: true description: The version that ran. x-moonbounce-sdk-known-fields: true EvaluatedModel: type: object properties: id: type: string description: >- The model the decision's setting names, as `GET .../model` reports it and `PUT .../model` takes it. resolved_model: type: string description: The versioned model that answered. revision: type: - integer - 'null' minimum: 1 description: >- The model setting's revision it ran on. Null when no setting was stored in the environment, so the deployment default applied. profile: type: string description: >- Every question's model and connector, as one opaque string. Compare it for equality (two runs are comparable when their profiles match); don't parse it. required: - id - resolved_model - revision - profile additionalProperties: true description: The model that answered. x-moonbounce-sdk-known-fields: true Cost: type: object properties: model_calls: type: integer minimum: 0 description: >- Upstream model calls, retries included. One call can answer several questions (Jev compiles them into one). questions: type: integer minimum: 0 description: >- Questions sent to a model. An answer from the cache, or one a stored case kept, is not counted. cache_hits: type: integer minimum: 0 description: >- Questions answered from the answer cache, when any were: previews and case runs always report it, evaluate and imports only when the cache served an answer. input_tokens: type: integer minimum: 0 output_tokens: type: integer minimum: 0 usd_micros: type: integer minimum: 0 description: What the model calls cost, in millionths of a US dollar. required: - model_calls - questions - input_tokens - output_tokens - usd_micros additionalProperties: true x-moonbounce-sdk-known-fields: true RunItemResult: type: object properties: item_id: type: string minLength: 1 maxLength: 256 description: The item's id; a stored case's `evaluation_id`. evaluation_id: type: string pattern: >- ^evl_[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$ description: 'A stored case: the evaluation replayed.' id: type: string description: >- The item's evaluation id, as one evaluate gives it: `evl_` when stored, `dry_` on a dry run (it names only the usage). Absent on a failed item. status: $ref: '#/components/schemas/RunItemStatusOutput' outcome: $ref: '#/components/schemas/OutcomeOutput' value: type: number via: type: string missing: type: array items: type: string description: Questions on the result's path with no answer. answers: type: object additionalProperties: type: - number - 'null' description: 'With `include: ["answers"]`.' attributes: type: object additionalProperties: anyOf: - type: string maxLength: 200 - type: number - type: boolean description: >- With `include: ["attributes"]`, when the decision keeps content (absent when its `store_state` is false). cost: type: object properties: model_calls: type: integer minimum: 0 description: >- Upstream model calls, retries included. One call can answer several questions (Jev compiles them into one). questions: type: integer minimum: 0 description: >- Questions sent to a model. An answer from the cache, or one a stored case kept, is not counted. cache_hits: type: integer minimum: 0 description: >- Questions answered from the answer cache, when any were: previews and case runs always report it, evaluate and imports only when the cache served an answer. input_tokens: type: integer minimum: 0 output_tokens: type: integer minimum: 0 usd_micros: type: integer minimum: 0 description: What the model calls cost, in millionths of a US dollar. required: - model_calls - questions - input_tokens - output_tokens - usd_micros additionalProperties: true description: >- What this item's model calls cost (money): on a failed item, what they spent before failing; absent when none ran. x-moonbounce-sdk-known-fields: true evaluation: anyOf: - type: object properties: environment: $ref: '#/components/schemas/Environment' id: $ref: '#/components/schemas/EvlIdOutput' dry_run: not: {} description: 'Must be absent: only a dry run sets `dry_run`.' x-moonbounce-absent: true answers: type: object additionalProperties: $ref: '#/components/schemas/NoulAnswer' description: Each question's answer, as System One returns it. aggregations: type: object additionalProperties: $ref: '#/components/schemas/AggregationResult' description: >- Every aggregation's result. The result's own aggregation carries `via` when it is bands, rules or match. moonbounce: type: object properties: decision_id: $ref: '#/components/schemas/DecisionIdOutput' item_id: type: string metadata: $ref: '#/components/schemas/MetadataOutput' summary: type: string required: - decision_id - item_id additionalProperties: true description: >- What you sent in `moonbounce`, echoed. What ran is in `version` and `model`. x-moonbounce-sdk-known-fields: true version: $ref: '#/components/schemas/EvaluatedVersion' model: $ref: '#/components/schemas/EvaluatedModel' cost: $ref: '#/components/schemas/EvaluateCost' status: type: string enum: - computed value: type: number outcome: not: {} description: >- Must be absent: a computed result is a number, with no outcome. x-moonbounce-absent: true missing: type: array items: $ref: '#/components/schemas/FeatureRefOutput' description: >- The questions the result depends on (through `result` and every aggregation on its path) that have no answer, abstained or failed, as `questions.x` paths in authored question order. Absent when none is missing; reported for every status. A caller who must not act on partial evidence checks that it is absent, for example `status === "decided" && !missing`. required: - environment - id - answers - aggregations - moonbounce - version - model - cost - status - value additionalProperties: true x-moonbounce-sdk-known-fields: true - type: object properties: environment: $ref: '#/components/schemas/Environment' id: $ref: '#/components/schemas/EvlIdOutput' dry_run: not: {} description: 'Must be absent: only a dry run sets `dry_run`.' x-moonbounce-absent: true answers: type: object additionalProperties: $ref: '#/components/schemas/NoulAnswer' description: Each question's answer, as System One returns it. aggregations: type: object additionalProperties: $ref: '#/components/schemas/AggregationResult' description: >- Every aggregation's result. The result's own aggregation carries `via` when it is bands, rules or match. moonbounce: type: object properties: decision_id: $ref: '#/components/schemas/DecisionIdOutput' item_id: type: string metadata: $ref: '#/components/schemas/MetadataOutput' summary: type: string required: - decision_id - item_id additionalProperties: true description: >- What you sent in `moonbounce`, echoed. What ran is in `version` and `model`. x-moonbounce-sdk-known-fields: true version: $ref: '#/components/schemas/EvaluatedVersion' model: $ref: '#/components/schemas/EvaluatedModel' cost: $ref: '#/components/schemas/EvaluateCost' status: type: string enum: - decided value: type: number outcome: $ref: '#/components/schemas/OutcomeOutput' missing: type: array items: $ref: '#/components/schemas/FeatureRefOutput' description: >- The questions the result depends on (through `result` and every aggregation on its path) that have no answer, abstained or failed, as `questions.x` paths in authored question order. Absent when none is missing; reported for every status. A caller who must not act on partial evidence checks that it is absent, for example `status === "decided" && !missing`. required: - environment - id - answers - aggregations - moonbounce - version - model - cost - status - outcome additionalProperties: true x-moonbounce-sdk-known-fields: true - type: object properties: environment: $ref: '#/components/schemas/Environment' id: $ref: '#/components/schemas/EvlIdOutput' dry_run: not: {} description: 'Must be absent: only a dry run sets `dry_run`.' x-moonbounce-absent: true answers: type: object additionalProperties: $ref: '#/components/schemas/NoulAnswer' description: Each question's answer, as System One returns it. aggregations: type: object additionalProperties: $ref: '#/components/schemas/AggregationResult' description: >- Every aggregation's result. The result's own aggregation carries `via` when it is bands, rules or match. moonbounce: type: object properties: decision_id: $ref: '#/components/schemas/DecisionIdOutput' item_id: type: string metadata: $ref: '#/components/schemas/MetadataOutput' summary: type: string required: - decision_id - item_id additionalProperties: true description: >- What you sent in `moonbounce`, echoed. What ran is in `version` and `model`. x-moonbounce-sdk-known-fields: true version: $ref: '#/components/schemas/EvaluatedVersion' model: $ref: '#/components/schemas/EvaluatedModel' cost: $ref: '#/components/schemas/EvaluateCost' status: type: string enum: - abstained value: not: {} description: 'Must be absent: an abstained result has no value.' x-moonbounce-absent: true outcome: $ref: '#/components/schemas/OutcomeOutput' missing: type: array items: $ref: '#/components/schemas/FeatureRefOutput' description: >- The questions the result depends on (through `result` and every aggregation on its path) that have no answer, abstained or failed, as `questions.x` paths in authored question order. Absent when none is missing; reported for every status. A caller who must not act on partial evidence checks that it is absent, for example `status === "decided" && !missing`. required: - environment - id - answers - aggregations - moonbounce - version - model - cost - status additionalProperties: true x-moonbounce-sdk-known-fields: true - type: object properties: environment: $ref: '#/components/schemas/Environment' id: $ref: '#/components/schemas/DryId' dry_run: type: boolean enum: - true answers: type: object additionalProperties: $ref: '#/components/schemas/NoulAnswer' description: Each question's answer, as System One returns it. aggregations: type: object additionalProperties: $ref: '#/components/schemas/AggregationResult' description: >- Every aggregation's result. The result's own aggregation carries `via` when it is bands, rules or match. moonbounce: type: object properties: decision_id: $ref: '#/components/schemas/DecisionIdOutput' item_id: type: string metadata: $ref: '#/components/schemas/MetadataOutput' summary: type: string required: - decision_id - item_id additionalProperties: true description: >- What you sent in `moonbounce`, echoed. What ran is in `version` and `model`. x-moonbounce-sdk-known-fields: true version: $ref: '#/components/schemas/EvaluatedVersion' model: $ref: '#/components/schemas/EvaluatedModel' cost: $ref: '#/components/schemas/EvaluateCost' status: type: string enum: - computed value: type: number outcome: not: {} description: >- Must be absent: a computed result is a number, with no outcome. x-moonbounce-absent: true missing: type: array items: $ref: '#/components/schemas/FeatureRefOutput' description: >- The questions the result depends on (through `result` and every aggregation on its path) that have no answer, abstained or failed, as `questions.x` paths in authored question order. Absent when none is missing; reported for every status. A caller who must not act on partial evidence checks that it is absent, for example `status === "decided" && !missing`. required: - environment - id - dry_run - answers - aggregations - moonbounce - version - model - cost - status - value additionalProperties: true x-moonbounce-sdk-known-fields: true - type: object properties: environment: $ref: '#/components/schemas/Environment' id: $ref: '#/components/schemas/DryId' dry_run: type: boolean enum: - true answers: type: object additionalProperties: $ref: '#/components/schemas/NoulAnswer' description: Each question's answer, as System One returns it. aggregations: type: object additionalProperties: $ref: '#/components/schemas/AggregationResult' description: >- Every aggregation's result. The result's own aggregation carries `via` when it is bands, rules or match. moonbounce: type: object properties: decision_id: $ref: '#/components/schemas/DecisionIdOutput' item_id: type: string metadata: $ref: '#/components/schemas/MetadataOutput' summary: type: string required: - decision_id - item_id additionalProperties: true description: >- What you sent in `moonbounce`, echoed. What ran is in `version` and `model`. x-moonbounce-sdk-known-fields: true version: $ref: '#/components/schemas/EvaluatedVersion' model: $ref: '#/components/schemas/EvaluatedModel' cost: $ref: '#/components/schemas/EvaluateCost' status: type: string enum: - decided value: type: number outcome: $ref: '#/components/schemas/OutcomeOutput' missing: type: array items: $ref: '#/components/schemas/FeatureRefOutput' description: >- The questions the result depends on (through `result` and every aggregation on its path) that have no answer, abstained or failed, as `questions.x` paths in authored question order. Absent when none is missing; reported for every status. A caller who must not act on partial evidence checks that it is absent, for example `status === "decided" && !missing`. required: - environment - id - dry_run - answers - aggregations - moonbounce - version - model - cost - status - outcome additionalProperties: true x-moonbounce-sdk-known-fields: true - type: object properties: environment: $ref: '#/components/schemas/Environment' id: $ref: '#/components/schemas/DryId' dry_run: type: boolean enum: - true answers: type: object additionalProperties: $ref: '#/components/schemas/NoulAnswer' description: Each question's answer, as System One returns it. aggregations: type: object additionalProperties: $ref: '#/components/schemas/AggregationResult' description: >- Every aggregation's result. The result's own aggregation carries `via` when it is bands, rules or match. moonbounce: type: object properties: decision_id: $ref: '#/components/schemas/DecisionIdOutput' item_id: type: string metadata: $ref: '#/components/schemas/MetadataOutput' summary: type: string required: - decision_id - item_id additionalProperties: true description: >- What you sent in `moonbounce`, echoed. What ran is in `version` and `model`. x-moonbounce-sdk-known-fields: true version: $ref: '#/components/schemas/EvaluatedVersion' model: $ref: '#/components/schemas/EvaluatedModel' cost: $ref: '#/components/schemas/EvaluateCost' status: type: string enum: - abstained value: not: {} description: 'Must be absent: an abstained result has no value.' x-moonbounce-absent: true outcome: $ref: '#/components/schemas/OutcomeOutput' missing: type: array items: $ref: '#/components/schemas/FeatureRefOutput' description: >- The questions the result depends on (through `result` and every aggregation on its path) that have no answer, abstained or failed, as `questions.x` paths in authored question order. Absent when none is missing; reported for every status. A caller who must not act on partial evidence checks that it is absent, for example `status === "decided" && !missing`. required: - environment - id - dry_run - answers - aggregations - moonbounce - version - model - cost - status additionalProperties: true x-moonbounce-sdk-known-fields: true description: >- With `include: ["evaluation"]`: what evaluate returns for this item on its own (absent on a failed item). error: type: object properties: code: type: string description: >- The reason, such as `ENGINE_FAILED`, `ENGINE_UNAVAILABLE` or `NOT_STARTED`. message: type: string retryable: type: boolean description: Sending the item again may work. required: - code - message - retryable additionalProperties: true description: Why it failed; only with status `failed`. x-moonbounce-sdk-known-fields: true required: - item_id - status additionalProperties: true x-moonbounce-sdk-known-fields: true CaseExcluded: type: object properties: withdrawn: type: integer minimum: 1 retired: type: integer minimum: 1 excluded: type: integer minimum: 1 contested: type: integer minimum: 1 no_consensus: type: integer minimum: 1 no_outcome: type: integer minimum: 1 outcome_not_in_result: type: integer minimum: 1 no_deciding_ballot: type: integer minimum: 1 attribute_inputs_unavailable: type: integer minimum: 1 duplicate_group: type: integer minimum: 1 description: >- Evaluations with feedback that are not cases, by reason: `withdrawn` (all feedback withdrawn), `retired` (feedback older than the last policy change), `excluded` (flagged input_broken), `contested` (reviewers disagree), `no_consensus` (no feedback that decides an outcome), `no_outcome` (feedback rules outcomes out without naming one), `outcome_not_in_result` (expected outcome not in `outcomes`), `no_deciding_ballot`, `attribute_inputs_unavailable` (stored before attribute inputs were kept, with a default that can't be read back), `duplicate_group` (another evaluation in its group counts instead: the same item, or near-identical content with the same label). ErrorType: type: string enum: - invalid_request - authentication - permission_denied - not_found - conflict - rate_limited - engine_failed - unavailable - internal ErrorIssue: type: object properties: param: type: string code: type: string pattern: ^[A-Z][A-Z0-9_]+$ description: >- Stable, extensible issue code; unknown codes still have a parameter and human-readable message. message: type: string required: - param - code - message additionalProperties: true x-moonbounce-sdk-known-fields: true NoulQuestion: type: object properties: type: type: string enum: - noul instructions: $ref: '#/components/schemas/Instructions' criteria: type: object properties: 'true': type: string minLength: 1 maxLength: 4000 'false': type: string minLength: 1 maxLength: 4000 additionalProperties: false reads: type: array items: $ref: '#/components/schemas/StatePath' minItems: 1 maxItems: 64 description: >- Paths into an object `state` this question reads, such as `state.message` or `state.order.customer.tier`. The question sees just those values, nested as in the state. Without it the question sees the whole state, as in System One. Every path must exist in the call's `state`; no entry may repeat or contain another. required: - type - instructions additionalProperties: false description: P(true) for a statement about the state. StringAttribute: type: object properties: type: type: string enum: - string enum: type: array items: type: string minLength: 1 maxLength: 200 minItems: 1 maxItems: 255 default: type: string source: type: string minLength: 1 maxLength: 200 description: >- The model and version behind this attribute, such as `toxicity-classifier-v3`. Hashed. from: type: string minLength: 1 maxLength: 256 description: >- A state path (as in `reads`) evaluate fills this attribute from, in place of `attribute_values`: the value at that path, validated by this declaration with no coercion. A missing or null path takes `default`, else it is a 400. A value for it in `attribute_values` is a 400. Hashed as written. examples: - state.message - state.order.order_total_usd required: - type - enum additionalProperties: false NumberAttribute: type: object properties: type: type: string enum: - number minimum: type: number maximum: type: number default: type: number source: type: string minLength: 1 maxLength: 200 description: >- The model and version behind this attribute, such as `toxicity-classifier-v3`. Hashed. from: type: string minLength: 1 maxLength: 256 description: >- A state path (as in `reads`) evaluate fills this attribute from, in place of `attribute_values`: the value at that path, validated by this declaration with no coercion. A missing or null path takes `default`, else it is a 400. A value for it in `attribute_values` is a 400. Hashed as written. examples: - state.message - state.order.order_total_usd required: - type additionalProperties: false BooleanAttribute: type: object properties: type: type: string enum: - boolean default: type: boolean source: type: string minLength: 1 maxLength: 200 description: >- The model and version behind this attribute, such as `toxicity-classifier-v3`. Hashed. from: type: string minLength: 1 maxLength: 256 description: >- A state path (as in `reads`) evaluate fills this attribute from, in place of `attribute_values`: the value at that path, validated by this declaration with no coercion. A missing or null path takes `default`, else it is a 400. A value for it in `attribute_values` is a 400. Hashed as written. examples: - state.message - state.order.order_total_usd required: - type additionalProperties: false WeightedMean: type: object properties: type: type: string enum: - weighted_mean inputs: type: array items: type: object properties: name: $ref: '#/components/schemas/FeatureRef' weight: type: number exclusiveMinimum: 0 invert: type: boolean required: - name additionalProperties: false minItems: 1 maxItems: 64 required: - type - inputs additionalProperties: false description: 'Σ wᵢ·xᵢ / Σ wᵢ: a weighted average from 0 to 1, not a probability.' Logistic: type: object properties: type: type: string enum: - logistic bias: type: number inputs: type: array items: type: object properties: name: $ref: '#/components/schemas/FeatureRef' weight: type: number required: - name - weight additionalProperties: false minItems: 1 maxItems: 64 required: - type - bias - inputs additionalProperties: false description: σ(bias + Σ wᵢ·xᵢ), with signed weights on raw input values. Min: type: object properties: type: type: string enum: - min inputs: type: array items: type: object properties: name: $ref: '#/components/schemas/FeatureRef' invert: type: boolean required: - name additionalProperties: false minItems: 1 maxItems: 64 required: - type - inputs additionalProperties: false description: 'min xᵢ: all of these, when they overlap.' Max: type: object properties: type: type: string enum: - max inputs: type: array items: type: object properties: name: $ref: '#/components/schemas/FeatureRef' invert: type: boolean required: - name additionalProperties: false minItems: 1 maxItems: 64 required: - type - inputs additionalProperties: false description: 'max xᵢ: any of these, when they overlap.' Product: type: object properties: type: type: string enum: - product inputs: type: array items: type: object properties: name: $ref: '#/components/schemas/FeatureRef' invert: type: boolean required: - name additionalProperties: false minItems: 1 maxItems: 64 required: - type - inputs additionalProperties: false description: 'Π xᵢ: all of these, when they are independent.' NoisyOr: type: object properties: type: type: string enum: - noisy_or inputs: type: array items: type: object properties: name: $ref: '#/components/schemas/FeatureRef' invert: type: boolean required: - name additionalProperties: false minItems: 1 maxItems: 64 required: - type - inputs additionalProperties: false description: '1 − Π(1 − xᵢ): any of these, when they are independent.' All: type: object properties: type: type: string enum: - all inputs: type: array items: type: object properties: name: $ref: '#/components/schemas/FeatureRef' gte: type: number gt: type: number lte: type: number lt: type: number required: - name additionalProperties: false minItems: 1 maxItems: 64 required: - type - inputs additionalProperties: false description: 1 when every input passes its threshold (default ≥ 0.5), else 0. Any: type: object properties: type: type: string enum: - any inputs: type: array items: type: object properties: name: $ref: '#/components/schemas/FeatureRef' gte: type: number gt: type: number lte: type: number lt: type: number required: - name additionalProperties: false minItems: 1 maxItems: 64 required: - type - inputs additionalProperties: false description: 1 when any input passes its threshold (default ≥ 0.5), else 0. Bands: type: object properties: type: type: string enum: - bands input: $ref: '#/components/schemas/FeatureRef' below: $ref: '#/components/schemas/Name' cuts: type: array items: type: object properties: at: type: number outcome: $ref: '#/components/schemas/Name' required: - at - outcome additionalProperties: false minItems: 1 maxItems: 9 abstain: $ref: '#/components/schemas/Name' required: - type - input - below - cuts additionalProperties: false description: >- Names an outcome for each range of one feature's value. `at` means ≥; `below` is under the first cut. Rules: type: object properties: type: type: string enum: - rules rules: type: array items: type: object properties: when: $ref: '#/components/schemas/Condition' outcome: $ref: '#/components/schemas/Name' required: - when - outcome additionalProperties: false minItems: 1 maxItems: 64 else: $ref: '#/components/schemas/Name' required: - type - rules - else additionalProperties: false description: >- The first rule whose condition is true names its `outcome`; otherwise `else`. Match: type: object properties: type: type: string enum: - match input: $ref: '#/components/schemas/FeatureRef' cases: type: object additionalProperties: $ref: '#/components/schemas/Name' description: >- Maps labels to outcomes. "__proto__" is not supported as a case key; use a rules is-condition for that label. else: $ref: '#/components/schemas/Name' abstain: $ref: '#/components/schemas/Name' required: - type - input - cases additionalProperties: false description: Maps a label straight to an outcome. EvlId: type: string pattern: >- ^evl_[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$ Outcome: type: string minLength: 1 maxLength: 200 description: >- An outcome name, or the exact enum label when a string attribute is the result. LabelSource: type: string enum: - human - synthetic - unknown description: >- Where the label came from: `human`, `synthetic` (an agent made it) or `unknown`. Only `human` counts toward the no-new-severe-mistake rule. StructureHashOutput: type: string pattern: ^sha256:[0-9a-f]{64}$ RunItemStatusOutput: type: string enum: - decided - computed - abstained - failed OutcomeOutput: type: string minLength: 1 maxLength: 200 description: >- An outcome name, or the exact enum label when a string attribute is the result. EvlIdOutput: type: string pattern: >- ^evl_[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$ NoulAnswer: anyOf: - type: object properties: type: type: string enum: - noul noul: type: number minimum: 0 maximum: 1 required: - type - noul additionalProperties: true x-moonbounce-sdk-known-fields: true - type: object properties: type: type: string enum: - noul abstained: type: boolean enum: - true required: - type - abstained additionalProperties: true x-moonbounce-sdk-known-fields: true - type: object properties: type: type: string enum: - noul failed: type: boolean enum: - true required: - type - failed additionalProperties: true x-moonbounce-sdk-known-fields: true description: P(true), or why there is no answer. AggregationResult: type: object properties: type: type: string value: type: number outcome: $ref: '#/components/schemas/OutcomeOutput' via: type: string description: >- A zero-based rule index, `cases.