model. Impala serves the batch with the base model’s serving setup and loads your adapter on top of it.
BYOC data planes need an Impala release that includes LoRA batch support. If your data plane hasn’t been upgraded yet, contact your Impala account team.
Upload the adapter
Export the adapter in PEFT format, trained on the same base model you run the batch on. The adapter folder must meet these requirements:- It contains
adapter_config.jsonandadapter_model.safetensorsat its top level, and nothing else. - Rank is 16 or less.
- It’s plain LoRA, with no DoRA and no bias terms.
loras/<adapter-name>/ in your data plane’s resources bucket. The bucket name is the Terraform output s3_resources_bucket_name and ends in -resources.
., _ and -, and start with a letter or digit.
Upload the adapter once, to the resources bucket in your main region. Impala copies it to every region where the batch runs.
Name the adapter in model
Set model to ft:<base-model>:<adapter-name>.
A batch runs on a single adapter. The adapter is loaded once when the batch starts and isn’t swapped per request, so every line in the input file must use the same model value. You can’t mix adapters, or an adapter and its base model, in one batch. To run several adapters, submit one batch per adapter.
Submit the batch
Upload the file and create the batch as you would for a base model.completion_window must be "unlimited".
Billing
A fine-tuned batch is billed at its base model’s rate. Its tokens count against your monthly token allowance at the base model’s size. Usage is still reported under the fullft:<base-model>:<adapter-name> name, so each fine-tune appears as its own line. See Pricing.
