> ## Documentation Index
> Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# ModelBundle
A ModelBundle combines one or more model and profile pairs into a single named, deployable unit. Each entry in `spec.modelConfigs` references a [Model](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model) and a [ModelProfile](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-profile) by name and can specify per-model batching and model settings.
`ModelBundle` resources use `apiVersion: sambanova.ai/v1alpha1`.
A bundle is optional. To serve a single model, pair it with a profile and deploy it directly. Create a bundle when you want to serve several models as one unit, configure speculative decoding, or validate and share a named configuration.
## Structure
The following example shows a multi-model bundle:
```yaml theme={}
apiVersion: sambanova.ai/v1alpha1
kind: ModelBundle
metadata:
name: mb-gpt120-llama70sd8-llama8
spec:
modelConfigs:
- model: gpt-oss-120b:1
profile: gpt-oss-fp8-dyt
batchingConfig:
8k:
batch_sizes: [2]
32k:
batch_sizes: [2]
- model: meta-llama-3-3-70b-instruct:1
profile: llama-3p1-70b-sd
modelSettings:
swappable: true
- model: meta-llama-3-1-8b-instruct:1
profile: llama-3p1-8b
modelSettings:
routable: false
specDecodingPairs:
- draft: meta-llama-3-1-8b-instruct
target: meta-llama-3-3-70b-instruct
```
The example above includes three models, with Meta-Llama-3.1-8B-Instruct serving as the non-routable draft model for Meta-Llama-3.3-70B-Instruct.
**ModelBundle Top-Level Fields**
| Field | Required | Description |
| ------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------ |
| `spec.modelConfigs` | Yes | List of model configurations in the bundle, with one entry per model. See [Model configurations](#model-configurations). |
| `spec.specDecodingPairs` | No | Speculative decoding pairs. See the speculative decoding parameters below. |
| `spec.adapters` | No | LoRA adapter definitions used by the model configurations, keyed by adapter name. |
| `spec.skip_legalizer` | No | Skips legalizer validation for the bundle. |
## Model configurations
Each entry in `spec.modelConfigs` pairs one model with one profile:
| Field | Required | Description |
| ----------------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model` | Yes | `Model` reference in `[:][:]` format. The architecture segment can be omitted when the model has a single architecture, and the version can be omitted to use the latest version. |
| `profile` | Conditional | Name of the `ModelProfile` this model uses. Set exactly one of `profile` or `profileDefinition`. |
| `profileDefinition` | Conditional | Inline profile specification, used when a model's PEFs span multiple standard profiles. Set exactly one of `profile` or `profileDefinition`. |
| `batchingConfig` | No | Per-bundle override of the profile's batching. Either a string naming one of the profile's `batchingConfigs` entries, such as `all` or `recommended`, or an inline tier map in which a tier can be set to `"*"` to select every batch size the profile provides for it. See [Overriding a batching configuration](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-profile#overriding-a-batching-configuration). |
| `modelSettings.swappable` | No | Whether the model is swappable. See [Swappable models](#swappable-models). |
| `modelSettings.routable` | No | Whether the model is directly routable by clients. Omit to inherit the default, which is routable. **Set to `false` for speculative decoding draft models.** |
| `modelSettings.checkpointOverrides` | No | Checkpoint selection applied in place of the `model` reference, using `checkpoint` for a model-level checkpoint and `expertOverrides` for per-expert checkpoints. |
| `modelSettings.adapters` | No | Mapping from serving name to a LoRA adapter binding, referencing an adapter defined in `spec.adapters`. |
| `modelSettings.properties` | No | Additional configuration fields applied across all of the model's experts. |
Batching is resolved in the following order of precedence: the model configuration's `batchingConfig`, then the profile's `recommended` configuration, then its `all` configuration, then a configuration generated from the PEF grouping. See [Resolution order](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-profile#resolution-order).
## Swappable models
The `swappable` setting controls whether a model can be evicted from HBM in favor of another model on the same endpoint. When `swappable` is set to `true`, the entire model is removed from HBM if a different model needs to run on that endpoint. This includes the model's checkpoint weights, its profile metadata, and its PEFs.
Set this field to `true` for models that can be evicted in order to free capacity for another model, and leave it unset or set it to `false` for models that must remain resident.
## Speculative decoding
In speculative decoding, a smaller draft model proposes tokens and a larger target model verifies them. The pairing is declared at the bundle level in `spec.specDecodingPairs`, and the draft model is included as its own non-routable entry in `spec.modelConfigs`.
For the field reference, requirements, and the full workflow including custom draft checkpoints, see [Deploying with speculative decoding](/docs/en/sambastack/service-administration/performance/deploy-with-speculative-decoding).
## Legalizer results
The legalizer validates a configuration's resource requirements before deployment. It runs automatically when you apply a ModelBundle, and its results are published under `status.legalizerInfo`. The bundle status also reports the effective batching configuration that was resolved for each model.
```bash theme={}
kubectl describe modelbundle
```
```bash theme={}
kubectl -n describe modelbundle.sambanova.ai
```
Example status for a bundle that passed with warnings:
```text theme={}
Status:
Batching Configs:
gemma-4-31b-it:1:
128k:
batch_sizes:
2
32k:
batch_sizes:
2
4
6
8
minimax-m2-7:1:
192k:
batch_sizes:
2
8k:
batch_sizes:
2
4
6
8
Conditions:
Last Transition Time: 2026-07-28T22:15:18.967900+00:00
Message: ModelBundle is Valid
Observed Generation: 2
Reason: ValidationSucceeded
Status: True
Type: Valid
Legalizer Info:
Errors:
Status: Legalizer passed
Utilization:
Ddr: 0.0614
hbm_resident: 0.0
Host: 0.0304
Warnings:
gemma-4-31b-it-vision-ckpt: checkpoint sharing not enabled for gemma-4-31B-it-128k-vision-emb and gemma-4-31B-it-32k-vision-emb
Observed Generation: 2
```
| Field | Description |
| ----------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `status.conditions[].reason` | `ValidationSucceeded` or `ValidationFailed`. The accompanying `message` reads `ModelBundle is Valid` on success, or contains the error detail on failure. |
| `status.batchingConfigs` | Effective batching configuration per model, keyed by the model reference and then by sequence length tier. This reflects the configuration actually resolved from the model configuration, the profile's default, or the PEF grouping. |
| `status.legalizerInfo.status` | Human-readable legalizer result: `Legalizer passed`, `Legalizer failed`, or `Legalizer was skipped`. Absent if legalizer output could not be processed. |
| `status.legalizerInfo.errors` | List of validation errors from the legalizer. Empty when the legalizer passed. |
| `status.legalizerInfo.warnings` | List of validation warnings, each in `: ` form. Warnings do not block deployment. |
| `status.legalizerInfo.utilization.ddr` | DDR memory utilization as a decimal fraction (for example, `0.0614` represents approximately 6%) |
| `status.legalizerInfo.utilization.hbm_resident` | HBM resident memory utilization as a decimal fraction. Values above `1.0` indicate over-allocation. |
| `status.legalizerInfo.utilization.host` | Host memory utilization as a decimal fraction (for example, `0.0304` represents approximately 3%) |
A bundle can pass the legalizer with warnings present, as in the example above, and still report `ValidationSucceeded`. Review warnings before deploying, but they do not prevent deployment.
The `utilization` fields show `N/A` when `skip_legalizer: true` is set on a bundle that has been legalized at least once. If the bundle has never been legalized, for example because `skip_legalizer: true` was set from initial deployment, `legalizerInfo` is absent entirely. The `utilization` field may also be absent if the legalizer output could not be parsed.
## Related resources
Author, apply, and validate a bundle of your own
Serves a bundle on the cluster