Skip to Content

Eval Types

CallerPermissions

Evaluated permissions for the current caller on this artifact type.

FieldTypeRequiredDescription
can_createbooleanYesWhether the caller can create new artifacts
can_draftbooleanYesWhether the caller can create/update drafts
can_duplicatebooleanYesWhether the caller can duplicate artifacts
has_accessbooleanNoWhether the caller can view this entity
can_editbooleanNoWhether the caller can edit this entity
can_deletebooleanNoWhether the caller can delete this entity
disabled_reasonstringNoHuman-readable reason if editing is disabled

ColumnInfo

FieldTypeRequiredDescription
namestringYesColumn name
typestringYesColumn data type
nullablebooleanYesWhether the column is nullable

CreateEvalItem

Single eval item for create — no eval_id.

Required fields (name): provide ID or value.

FieldTypeRequiredDescription
idstringNoOptional pre-assigned UUID
resource_idstringNoOptional preset UUID for the resource snapshot
name_idstringNoREQUIRED FOR CREATE (or pass name): UUID of an existing name resource
namestringNoREQUIRED FOR CREATE (or pass name_id): display name text — creates a new name resource if name_id is not provided
description_idstringNoDescription resource UUID
descriptionstringNoDescription value for resolution
flag_idsstring[]NoFlag option UUIDs
department_idsstring[]NoDepartment UUIDs
departmentsstring[]NoDepartment names for resolution
model_idsstring[]NoModel UUIDs
model_flag_idsstring[]NoModel flag UUIDs
model_rubric_idsstring[]NoModel rubric UUIDs
model_position_idsstring[]NoModel position UUIDs
activebooleanNoDenormalized eval_active flag state; resolved to a flag_ids entry server-side

DeleteEvalResult

Per-item result within a bulk delete response.

FieldTypeRequiredDescription
successbooleanYesWhether the operation succeeded
eval_idstringNoEval UUID (None only for soft-skipped rows)
messagestringYesHuman-readable result message

DocsApiResponse

FieldTypeRequiredDescription
listPageMetaItemYes
detailPageMetaItemYes
newPageMetaItemYes

DocsResponse

FieldTypeRequiredDescription
namestringYesResource or entry name
typestringYesResource or entry type identifier
descriptionstringYesHuman-readable description
materialized_viewMvInfoNoMaterialized view metadata
tablesTableInfo[]YesRelated database tables
operationsOperationInfo[]YesAvailable operations

EvalDepartmentResource

FieldTypeRequiredDescription
department_idstringNoDepartment identifier
namestringNoDisplay name
descriptionstringNoDescription text
generatedbooleanNoWhether this was AI-generated
suggestedbooleanNoWhether this is a suggested option
selectedbooleanNoWhether this is currently selected
pendingbooleanNoWhether this selection is pending acceptance

EvalDescriptionResource

FieldTypeRequiredDescription
idstringNoUnique identifier
descriptionstringNoDescription text
generatedbooleanNoWhether this was AI-generated
suggestedbooleanNoWhether this is a suggested option
selectedbooleanNoWhether this is currently selected
pendingbooleanNoWhether this selection is pending acceptance

EvalFieldError

Per-field error from value resolution.

FieldTypeRequiredDescription
fieldstringYesField name that has the error
messagestringYesHuman-readable error message

EvalFlagResource

Flag option row — one per (name, type, value) entry in flags_resource.

FieldTypeRequiredDescription
idstringNoFlag resource identifier
namestringNoFlag display name
typestringNoFlag type (e.g. ‘eval_active’)
valuebooleanNoUnderlying bool value of this option
descriptionstringNoFlag description text
icon_idstringNoIcon identifier for the flag
iconstringNoResolved SVG markup (hydrated from icons_resource)
generatedbooleanNoWhether the flag was AI-generated
suggestedbooleanNoWhether this item is suggested
selectedbooleanNoWhether this item is selected
pendingbooleanNoWhether this item is pending acceptance

EvalModelFlagOptionResource

Cross-product option row: one per (model_id, flag_type, value) tuple. The ModelFlags picker groups these by (model_id, type) and renders a Switch per group; toggling picks the row whose value matches the new state and sends its flag_id (or denormalized {model_id,type,value}) up through the draft endpoint.

FieldTypeRequiredDescription
model_idstringNoModel identifier
flag_idstringNoFlag resource identifier (flags_resource row)
typestringNoFlag type, e.g. ‘model_active’
valuebooleanNoUnderlying flag value for this option
namestringNoDisplay name from the flags_resource row
descriptionstringNoDescription text from the flags_resource row
iconstringNoIcon SVG markup hydrated from icons_resource

EvalModelFlagResource

FieldTypeRequiredDescription
idstringNoModel-flag resource identifier
model_idstringNoAssociated model identifier
flag_idstringNoAssociated flag identifier
typestringNoFlag type (e.g. ‘model_active’) of the linked flags_resource row
valuebooleanNoUnderlying bool value of the linked flags_resource row
namestringNoDisplay name
descriptionstringNoDescription text
iconstringNoIcon identifier
generatedbooleanNoWhether this was AI-generated
suggestedbooleanNoWhether this is a suggested option
selectedbooleanNoWhether this is currently selected
pendingbooleanNoWhether this selection is pending acceptance

EvalModelFlagValue

Denormalized per-(model, type) selection. Clients send these to update model-flag selections without needing the underlying flag_id; the server resolves (type, value) -> flag_id and upserts the model_flags_resource junction row.

FieldTypeRequiredDescription
model_idstringYesTarget model identifier
typestringYesFlag type, e.g. ‘model_active’
valuebooleanYesDesired flag value

EvalModelPositionResource

FieldTypeRequiredDescription
idstringNoModel-position resource identifier
model_idstringNoAssociated model identifier
valueinteger | numberNoAssociated position value
generatedbooleanNoWhether this was AI-generated
suggestedbooleanNoWhether this is a suggested option
selectedbooleanNoWhether this is currently selected
pendingbooleanNoWhether this selection is pending acceptance

EvalModelResource

FieldTypeRequiredDescription
idstringNoModel resource identifier
namestringNoDisplay name
descriptionstringNoDescription text
modality_idsstring[]NoAssociated modality identifiers
generatedbooleanNoWhether this was AI-generated
suggestedbooleanNoWhether this is a suggested option
selectedbooleanNoWhether this is currently selected
pendingbooleanNoWhether this selection is pending acceptance

EvalModelRubricResource

FieldTypeRequiredDescription
idstringNoModel-rubric resource identifier
model_idstringNoAssociated model identifier
rubric_idstringNoAssociated rubric identifier
generatedbooleanNoWhether this was AI-generated
suggestedbooleanNoWhether this is a suggested option
selectedbooleanNoWhether this is currently selected
pendingbooleanNoWhether this selection is pending acceptance

EvalNameResource

FieldTypeRequiredDescription
idstringNoUnique identifier
namestringNoDisplay name
generatedbooleanNoWhether this was AI-generated
suggestedbooleanNoWhether this is a suggested option
selectedbooleanNoWhether this is currently selected
pendingbooleanNoWhether this selection is pending acceptance

EvalResultItem

Per-item result within a bulk create/update response.

FieldTypeRequiredDescription
successbooleanYesWhether the operation succeeded
eval_idstringNoEval UUID
messagestringYesHuman-readable result message
errorsEvalFieldError[]NoList of per-field errors

EvalRubricResource

Top-level rubric catalog entry. The ModelRubrics picker renders one of these per option (alongside a “no rubric” clear state).

FieldTypeRequiredDescription
idstringNoRubric resource UUID
namestringNoRubric display name
descriptionstringNoRubric description

EvalSetup

Run-level eval scaffold — first-class on the generate response.

Audit’s **output spread carries this onto <artifact>.generate.completed. Null when no rubric-bearing agent participated.

FieldTypeRequiredDescription
test_idstringYes
invocationsInvocationSlot[]Yes

GenerateConfig

Developer configuration — all optional with sensible defaults.

FieldTypeRequiredDescription
operationsstring[]No
dangerousbooleanNo
paramsobjectNo
group_idstringNo

GenerationsEvalListItem

Single generation group in the eval generations response.

FieldTypeRequiredDescription
group_idstringYesUUID of the generation group
session_idstringNoUUID of the parent session
group_namestringNoName of the generation group
created_atstringNoTimestamp of the generation

GetEvalDraftResponse

FieldTypeRequiredDescription
idstringYesUUID of the draft
created_atstringYesCreation timestamp
generatedbooleanYesWhether this was AI-generated
mcpbooleanYesWhether MCP tooling was used
activebooleanYesWhether this draft is active
session_idstringYesAssociated session UUID
namestringNoImmutable draft label set at create time
department_idsstring[]YesAssociated department UUIDs
description_idsstring[]YesAssociated description UUIDs
flag_idsstring[]YesAssociated flag UUIDs
model_idsstring[]YesAssociated model UUIDs
name_idsstring[]YesAssociated name UUIDs
profile_idsstring[]YesAssociated profile UUIDs
rubric_idsstring[]YesAssociated rubric UUIDs
model_flag_idsstring[]NoAssociated model flag UUIDs
model_position_idsstring[]NoAssociated model position UUIDs
model_rubric_idsstring[]NoAssociated model rubric UUIDs
pending_department_idsstring[]NoAssociated department UUIDs stored as inactive pending links
pending_description_idsstring[]NoAssociated description UUIDs stored as inactive pending links
pending_flag_idsstring[]NoAssociated flag UUIDs stored as inactive pending links
pending_model_idsstring[]NoAssociated model UUIDs stored as inactive pending links
pending_name_idsstring[]NoAssociated name UUIDs stored as inactive pending links
pending_rubric_idsstring[]NoAssociated rubric UUIDs stored as inactive pending links
pending_model_flag_idsstring[]NoPending model flag UUIDs
pending_model_position_idsstring[]NoPending model position UUIDs
pending_model_rubric_idsstring[]NoPending model rubric UUIDs

GroupCall

Tool call referenced by a message.

FieldTypeRequiredDescription
idstringYes
tool_namestringNo
template_namestringNo
toolobjectNo
ledger_statusstringNo
ledger_operationstringNo
ledger_artifactstringNo
ledger_artifact_idstringNo

GroupMessage

Message within a run.

FieldTypeRequiredDescription
idstringYes
rolestringYes
created_atstringNo
text_idsstring[]No
audio_idsstring[]No
image_idsstring[]No
video_idsstring[]No
file_idsstring[]No
call_idsstring[]No
callsGroupCall[]No
reasoningbooleanNoTrue when this row is a chain-of-thought trace persisted alongside the assistant answer (rendered as a collapsed accordion).
in_contextbooleanNoWhether this message is included in the LLM context for the next generation. Mirrors the dedup pass that builds chat history (see in_context_reason).
in_context_reasonstringNoWhy this message is in/out of LLM context. ‘kept’ = included; ‘deduped_read’ = older read-only call to a tool that has a fresher result later in the group; future values may include ‘trimmed_top_n’.

GroupResource

Lightweight \{id, name\} for cross-referencing run-level ids (model_id / agent_id / profile_id) against human-readable names on the analytics panel. Names come from the canonical get_models / get_agents / get_profiles black boxes.

FieldTypeRequiredDescription
idstringYes
namestringNo

GroupRun

Run within a group, with its messages.

Carries token / cost / model / agent / profile attribution so the analytics view can render per-run cost + actor info without a parallel detail shape. profile_id is the authoring profile (human user), agent_id is the LLM-side actor, model_id is the model used by that agent. All optional — runs predating these columns or with unresolved attributions surface None.

FieldTypeRequiredDescription
idstringYes
created_atstringNo
input_tokensintegerNo
output_tokensintegerNo
cached_input_tokensintegerNo
costnumberNo
model_idstringNo
agent_idstringNo
profile_idstringNo
previous_context_start_indexintegerNoIndex in messages where the current run’s own messages begin; earlier rows are previous-context replay. None when the run has no previous context attached.
messagesGroupMessage[]No

ImportField

Field descriptor for CSV import column mapping.

FieldTypeRequiredDescription
keystringYes
labelstringYes
requiredbooleanNo
multibooleanNo
typestringNo
examplestringNo
descriptionstringNo

InvocationSlot

One agent’s slot in a multi-agent generation pool.

Populated by setup_generation_test when an agent carries a rubric. The client uses these IDs to drive the eval workflow: review the candidate’s output, optionally fire a grader against its invocation_id, and promote/reject by call_id via the existing idempotency_key + accept pattern.

FieldTypeRequiredDescription
invocation_idstringYes
agent_idstringYes
rubric_idstringNo

ListEvalApiEval

Eval type for list endpoint with computed permissions.

FieldTypeRequiredDescription
idstringNoEval artifact UUID (canonical id; mirrors eval_id)
eval_idstringNoEval UUID
namestringNoEval name
descriptionstringNoEval description
department_idsstring[]NoAssociated department IDs
model_idsstring[]NoModel resource UUIDs assigned to this eval
rubric_idsstring[]NoRubric artifact UUIDs assigned to this eval (via model_rubrics_resource)
is_inactivebooleanNoWhether the eval is inactive
is_dynamicbooleanNoWhether the eval uses dynamic mode
use_groupsbooleanNoWhether the eval uses groups
pending_statusstringNoLatest soft_calls_mv status: ‘pending’ / ‘accepted’ / ‘rejected’
pending_operationstringNoOperation type (‘create'
pending_call_idstringNocall_id (idempotency key for ack) of the pending op
num_runsintegerNoNumber of eval runs
num_groupsintegerNoNumber of eval groups
can_editbooleanNoWhether the current user can edit
can_duplicatebooleanNoWhether the current user can duplicate
can_deletebooleanNoWhether the current user can delete
updated_atstringNoLast updated timestamp

ListFilterOption

Standardized option for list endpoint filter sections.

FieldTypeRequiredDescription
idstringNoUnique identifier for this filter option
namestringNoDisplay name
countintegerNoNumber of matching records
hex_codestringNoHex color code for display
valuestringNoInternal value
typestringNoOption type discriminator

ListFilterSection

Filter section with options and echoed request state.

FieldTypeRequiredDescription
optionsListFilterOption[]NoAvailable filter options
selected_idsstring[]NoCurrently selected filter option IDs
searchstringNoActive search text for filtering

MvInfo

FieldTypeRequiredDescription
namestringYesMaterialized view name
definitionstringYesSQL definition of the view
columnsColumnInfo[]YesList of columns in the view

OperationInfo

FieldTypeRequiredDescription
namestringYesOperation name
descriptionstringYesHuman-readable description of the operation
paramsParamInfo[]YesList of operation parameters
returnsobjectNoReturn type schema

OperationPrompts

Starter prompts keyed by operation name.

Each key is an operation (e.g. “create”, “search”, “draft”, “export”) and the value is a list of starter prompts for that operation. The client picks from the operations the caller has permission for and rotates through them.

FieldTypeRequiredDescription
promptsobjectNoMap of operation name to starter prompts

PageMetaItem

FieldTypeRequiredDescription
titlestringYes
descriptionstringYes

ParamInfo

FieldTypeRequiredDescription
namestringYesParameter name
typestringYesParameter data type
requiredbooleanYesWhether the parameter is required
defaultanyNoDefault value if not required

ProducedMedia

One asset produced by a generation run.

resource_id is the canonical id the per-artifact download tools accept (e.g. Scenario_Image_Download(image_id=resource_id) for modality="image"). It maps to images_resource.id / videos_resource.id / audios_resource.id depending on the modality.

FieldTypeRequiredDescription
modality"image" | "video" | "audio"Yes
resource_idstringYes
upload_idstringYes
mime_typestringNo
file_sizeintegerNo

ProfileSummary

Caller identity derived from JWT — who you are on this page.

Superset of the old six-field version: now carries everything the client needs so that /\{artifact\}/context fully replaces /profiles/context and the extra getLayoutContextData round-trip can be dropped.

FieldTypeRequiredDescription
namestringYesDisplay name of the authenticated user
rolestringYesRole name (e.g. ‘Super Administrator’)
role_levelintegerYesRole hierarchy level (0 = highest privilege)
department_idsstring[]YesDepartments the user belongs to
artifact_accessstring[]YesArtifact types this role can access (sidebar visibility)
role_permissionsany[][]YesFull (artifact, operation) permission tuples for granular page gating
is_activebooleanYesWhether the user’s profile is active
idstringYesProfile UUID (SocketProvider, ProfileProvider)
themeThemeBundleNoResolved theme: hex primitives + derived oklch tokens + score thresholds
session_idstringNoCurrent session UUID
is_emulationbooleanNoWhether user is in emulation mode (ProfileProvider)
role_resourcesQGetProfileContextV4RoleResource[]NoAll role resources for emulation display (ProfileProvider)
scoped_rolesstring[]NoRoles the user can emulate (ProfileProvider)
activebooleanNoAlias for is_active (ProfileProvider uses this name)

QGetProfileContextV4RoleResource

FieldTypeRequiredDescription
rolestringNo
namestringNo
descriptionstringNo
icon_valuestringNo
color_hexstringNo

RunStatus

Outcome of one watched run.

FieldTypeRequiredDescription
run_idstringYes
status"pending" | "completed" | "failed"Yes
modalitystringNoModality of generated output (image, video, text…)
resource_idsstring[]NoResource UUIDs produced by this run (e.g. images_resource ids)
errorstringNo

TableInfo

FieldTypeRequiredDescription
namestringYesTable name
columnsColumnInfo[]YesList of columns in the table

ThemeBundle

Full theme payload for a page bootstrap.

Riding along on every /\{artifact\}/context response via ProfileSummary.theme. Layers:

  • primitives / dark_primitives — hex inputs the settings editor reads/writes (light + dark palettes).
  • tokens / dark_tokens — oklch tokens the client paints with. ThemeStyle emits two <style> blocks: one scoped to :root:not(.dark) (light) and one to :root.dark (dark).
  • thresholds — numeric score thresholds for analytics components. Empty-in → empty-out per token: missing values fall through to the matching globals.css default.
FieldTypeRequiredDescription
primitivesThemePrimitivesNoHex inputs from the setting (light palette, for the theme editor)
tokensThemeTokensNoDerived oklch tokens for light mode (SSR CSS-var injection)
dark_primitivesThemePrimitivesNoHex inputs from the setting (dark palette, for the theme editor)
dark_tokensThemeTokensNoDerived oklch tokens for dark mode (SSR CSS-var injection)
thresholdsThresholdsNoScore thresholds resolved from the setting

ThemePrimitives

40 optional fields. The 17 essentials drive the rest; the other 23 are overrides for fine-tuning when derivation isn’t what you want.

Empty primitive → empty token → client falls back to globals.css.

FieldTypeRequiredDescription
backgroundstringNo
primarystringNo
accentstringNo
cardstringNo
sidebarstringNo
muted_foregroundstringNo
ringstringNo
borderstringNo
destructivestringNo
successstringNo
warningstringNo
infostringNo
chart1stringNo
chart2stringNo
chart3stringNo
chart4stringNo
chart5stringNo
foregroundstringNo
card_foregroundstringNo
popoverstringNo
popover_foregroundstringNo
primary_foregroundstringNo
secondarystringNo
secondary_foregroundstringNo
mutedstringNo
accent_foregroundstringNo
destructive_foregroundstringNo
dangerstringNo
danger_foregroundstringNo
inputstringNo
success_foregroundstringNo
warning_foregroundstringNo
info_foregroundstringNo
sidebar_foregroundstringNo
sidebar_primarystringNo
sidebar_primary_foregroundstringNo
sidebar_accentstringNo
sidebar_accent_foregroundstringNo
sidebar_borderstringNo
sidebar_ringstringNo

ThemeTokens

40 fully-resolved CSS variable values (snake_case 1:1 with vars).

FieldTypeRequiredDescription
backgroundstringNo
foregroundstringNo
cardstringNo
card_foregroundstringNo
popoverstringNo
popover_foregroundstringNo
primarystringNo
primary_foregroundstringNo
secondarystringNo
secondary_foregroundstringNo
mutedstringNo
muted_foregroundstringNo
accentstringNo
accent_foregroundstringNo
destructivestringNo
destructive_foregroundstringNo
dangerstringNo
danger_foregroundstringNo
borderstringNo
inputstringNo
ringstringNo
successstringNo
success_foregroundstringNo
warningstringNo
warning_foregroundstringNo
infostringNo
info_foregroundstringNo
chart1stringNo
chart2stringNo
chart3stringNo
chart4stringNo
chart5stringNo
sidebarstringNo
sidebar_foregroundstringNo
sidebar_primarystringNo
sidebar_primary_foregroundstringNo
sidebar_accentstringNo
sidebar_accent_foregroundstringNo
sidebar_borderstringNo
sidebar_ringstringNo

Thresholds

Numeric score thresholds resolved from the active setting.

Server pre-buckets dashboard metrics into success | warning | danger | neutral already, so most components don’t need these values. Surface them for chart reference lines, tooltips, and any client-side bucketing.

FieldTypeRequiredDescription
successintegerYesScore >= this counts as success
warningintegerYesScore >= this counts as warning
dangerintegerYesScore < success threshold but >= this counts as danger; below is neutral/no-data

UpdateEvalItem

Single eval item for update — eval_id required, all fields optional.

Only provided fields are updated (partial update).

FieldTypeRequiredDescription
idstringYesEval UUID to update
name_idstringNoName resource UUID
namestringNoName value for resolution
description_idstringNoDescription resource UUID
descriptionstringNoDescription value for resolution
flag_idsstring[]NoFlag option UUIDs
department_idsstring[]NoDepartment UUIDs
departmentsstring[]NoDepartment names for resolution
model_idsstring[]NoModel UUIDs
model_flag_idsstring[]NoModel flag UUIDs
model_rubric_idsstring[]NoModel rubric UUIDs
model_position_idsstring[]NoModel position UUIDs
activebooleanNoDenormalized eval_active flag state; resolved to a flag_ids entry server-side

UpdateEvalPatch

Shared patch for bulk-update-all-matching mode.

Inherits every field from UpdateEvalItem and just relaxes id to optional — the bulk impl stamps the resolved id onto a clone of the patch per matched row, so any client-supplied id is ignored. Sparse semantics: only fields the client sets are written.

FieldTypeRequiredDescription
idstringNoIgnored — bulk impl stamps the resolved eval id per matched row
name_idstringNoName resource UUID
namestringNoName value for resolution
description_idstringNoDescription resource UUID
descriptionstringNoDescription value for resolution
flag_idsstring[]NoFlag option UUIDs
department_idsstring[]NoDepartment UUIDs
departmentsstring[]NoDepartment names for resolution
model_idsstring[]NoModel UUIDs
model_flag_idsstring[]NoModel flag UUIDs
model_rubric_idsstring[]NoModel rubric UUIDs
model_position_idsstring[]NoModel position UUIDs
activebooleanNoDenormalized eval_active flag state; resolved to a flag_ids entry server-side

app__infra__eval__types__DraftFormState

Server-authoritative form state returned after draft save.

FieldTypeRequiredDescription
name_idstringNoSelected name resource UUID
namestringNoEchoed selected name value
description_idstringNoSelected description resource UUID
descriptionstringNoEchoed selected description value
flag_idsstring[]NoSelected flag option UUIDs
activebooleanNoEchoed eval_active flag state
department_idsstring[]NoSelected department UUIDs
model_idsstring[]NoSelected model UUIDs
model_flag_idsstring[]NoSelected model flag UUIDs
model_flag_valuesEvalModelFlagValue[]NoDenormalized (model_id, type, value) echo derived from model_flag_ids
model_position_idsstring[]NoSelected model position UUIDs
model_rubric_idsstring[]NoSelected model rubric UUIDs
pending_idsstring[]NoPending resource identifiers

app__infra__eval__types__SectionFilter

FieldTypeRequiredDescription
searchstringNoFilter options by search text
limitintegerNoMax options to return
selectedbooleanNoOnly return selected items
suggestedbooleanNoOnly return suggested items
includebooleanNoInclude this section in response (default true)
parameter_idsstring[]NoReserved for compatibility with shared filter parsing

Last updated on