Skip to main content
A batch on a LoRA adapter is a normal batch with two differences: the adapter lives in your data plane’s resources bucket, and each request names it in model. Impala serves the batch with the base model’s serving setup and loads your adapter on top of it.
BYOC data planes need an Impala release that includes LoRA batch support. If your data plane hasn’t been upgraded yet, contact your Impala account team.

Upload the adapter

Export the adapter in PEFT format, trained on the same base model you run the batch on. The adapter folder must meet these requirements:
  • It contains adapter_config.json and adapter_model.safetensors at its top level, and nothing else.
  • Rank is 16 or less.
  • It’s plain LoRA, with no DoRA and no bias terms.
Copy the folder to loras/<adapter-name>/ in your data plane’s resources bucket. The bucket name is the Terraform output s3_resources_bucket_name and ends in -resources.
Adapter names use letters, digits, ., _ and -, and start with a letter or digit. Upload the adapter once, to the resources bucket in your main region. Impala copies it to every region where the batch runs.

Name the adapter in model

Set model to ft:<base-model>:<adapter-name>. A batch runs on a single adapter. The adapter is loaded once when the batch starts and isn’t swapped per request, so every line in the input file must use the same model value. You can’t mix adapters, or an adapter and its base model, in one batch. To run several adapters, submit one batch per adapter.

Submit the batch

Upload the file and create the batch as you would for a base model. completion_window must be "unlimited".
Poll the batch and download results the same way as any other batch. See Run a batch.
Impala doesn’t validate the adapter. A missing or invalid adapter doesn’t fail the batch. Instead, the batch keeps waiting because no serving setup can load the adapter. Check the adapter path and files before you submit.

Billing

A fine-tuned batch is billed at its base model’s rate. Its tokens count against your monthly token allowance at the base model’s size. Usage is still reported under the full ft:<base-model>:<adapter-name> name, so each fine-tune appears as its own line. See Pricing.