Create Batch Inference Job
Create a batch inference job that runs a model over an input dataset and writes results to an output dataset, with an optional system prompt.
Authorizations
Body
Optional job-level system prompt. When set, it is injected as a leading system message into every input row that does NOT already begin with a system message (a row's own leading system message takes precedence). This lets callers avoid repeating a large, static system prompt on every row of the input dataset, shrinking the upload. Because the injected prefix is byte-identical across rows, prompt caching still applies.
The precision with which the model should be served. If PRECISION_UNSPECIFIED, a default will be chosen based on the model.
PRECISION_UNSPECIFIED, FP16, FP8, FP8_MM, FP8_AR, FP8_MM_KV_ATTN, FP8_KV, FP8_MM_V2, FP8_V2, FP8_MM_KV_ATTN_V2, NF4, FP4, BF16, FP4_BLOCKSCALED_MM, FP4_MX_MOE The customer-requested wall-clock run window for the job: how long it may run before it is expired. This is the single public input that controls the job's lifetime. The server bounds it to [12h, 72h]; if unset it defaults to 24h. The resulting concrete deadline is surfaced as expire_time (= create_time + the bounded window) and the job is expired once that deadline passes. A duration (relative) is used rather than an absolute timestamp because the client does not know create_time at submit time. Customer-visible input.
Response
A successful response.
The time when the batch inference job will expire (stop running); any completed requests will have been written to the output dataset by then.
This is the job's effective execution deadline, derived by the server as create_time + the bounded run window (see max_job_duration). It is exposed so customers can read back the concrete deadline without recomputing it client-side. OUTPUT_ONLY: it is always computed server-side and any client-supplied value is ignored (previously this was a SUPERUSER_ONLY input that overlapped with max_job_duration; the two are now unified as one public input (duration) + one public derived deadline (this timestamp)).
JobState represents the state an asynchronous job can be in.
JOB_STATE_UNSPECIFIED, JOB_STATE_CREATING, JOB_STATE_RUNNING, JOB_STATE_COMPLETED, JOB_STATE_FAILED, JOB_STATE_CANCELLED, JOB_STATE_DELETING, JOB_STATE_WRITING_RESULTS, JOB_STATE_VALIDATING, JOB_STATE_DELETING_CLEANING_UP, JOB_STATE_PENDING, JOB_STATE_EXPIRED, JOB_STATE_RE_QUEUEING, JOB_STATE_CREATING_INPUT_DATASET, JOB_STATE_IDLE, JOB_STATE_CANCELLING, JOB_STATE_EARLY_STOPPED, JOB_STATE_PAUSED, JOB_STATE_DELETED, JOB_STATE_ARCHIVED Optional job-level system prompt. When set, it is injected as a leading system message into every input row that does NOT already begin with a system message (a row's own leading system message takes precedence). This lets callers avoid repeating a large, static system prompt on every row of the input dataset, shrinking the upload. Because the injected prefix is byte-identical across rows, prompt caching still applies.
The precision with which the model should be served. If PRECISION_UNSPECIFIED, a default will be chosen based on the model.
PRECISION_UNSPECIFIED, FP16, FP8, FP8_MM, FP8_AR, FP8_MM_KV_ATTN, FP8_KV, FP8_MM_V2, FP8_V2, FP8_MM_KV_ATTN_V2, NF4, FP4, BF16, FP4_BLOCKSCALED_MM, FP4_MX_MOE The customer-requested wall-clock run window for the job: how long it may run before it is expired. This is the single public input that controls the job's lifetime. The server bounds it to [12h, 72h]; if unset it defaults to 24h. The resulting concrete deadline is surfaced as expire_time (= create_time + the bounded window) and the job is expired once that deadline passes. A duration (relative) is used rather than an absolute timestamp because the client does not know create_time at submit time. Customer-visible input.
True only while a job that has ALREADY started running is briefly
re-acquiring capacity after a mid-run preemption/stockout (i.e. it
regressed from RUNNING back to an internal PENDING/CREATING phase and is
waiting to resume). This is intentionally a transient sub-status annotation
on a job whose customer-facing state stays RUNNING — NOT a distinct
state value: the job is still running (progress is saved, it auto-resumes)
so introducing a new terminal-or-not enum value would force every state
consumer (SDK/CLI/internal maps/billing) to special-case "still running."
It drives the customer-facing "Briefly paused — waiting on capacity" card.
It must NOT be set during first-time provisioning before the job has ever
run, and is cleared once the job returns to RUNNING or reaches a terminal
state. So "state=RUNNING, waiting_on_capacity=true" means: running, momentarily
paused while it re-acquires capacity.