Create a Custom Model Bundle on SambaStack

Create a bundle to serve several models as one unit, to configure speculative decoding, or to validate and share a named configuration. A bundle is a reusable, named ModelBundle resource that combines one or more model and profile pairs. Creating a bundle does not serve it. Once the bundle passes validation, deploy it with a ModelDeployment as described in Deploy a bundle.

A bundle is optional. To serve a single model, pair it with a profile and deploy it directly. See Deploy a single model.

Prerequisites

Before authoring a bundle, complete the quickstart that applies to you:

Quickstart - Hosted

System set up for hosted SambaStack

Quickstart - On-prem

System set up for on-prem SambaStack

Additionally:

Review the ModelBundle field reference, including model configurations and legalizer results.
Identify the models and profiles to include, and confirm each pair is compatible. See Identify available models and profiles.
Review Supported models and bundles for the available model checkpoints.
Review Deploying with speculative decoding if you are configuring speculative decoding pairs.

Create the bundle

Create the YAML file

Create a YAML file for your ModelBundle:

apiVersion: sambanova.ai/v1alpha1
kind: ModelBundle
metadata:
  name: mb-gpt120
spec:
  modelConfigs:
  - model: gpt-oss-120b:1
    profile: gpt-oss-fp8-dyt
    batchingConfig:
      8k:
        batch_sizes: [2]
      32k:
        batch_sizes: [2]
      64k:
        batch_sizes: [2]
      128k:
        batch_sizes: [2]

For multi-model bundles with speculative decoding, see the ModelBundle structure example.

Apply the ModelBundle

Hosted
On Premise

kubectl apply -f <modelbundle-file>.yaml

kubectl -n <namespace> apply -f <modelbundle-file>.yaml

Verify legalizer validation

The legalizer automatically runs when you apply the bundle and validates whether the configuration fits in RDU memory.

Hosted
On Premise

kubectl describe modelbundle <bundle-name>

kubectl -n <namespace> describe modelbundle.sambanova.ai <bundle-name>

Successful validation
Failed validation

Status:
  Conditions:
    Last Transition Time:  2026-06-22T21:11:05.689262+00:00
    Message:               ModelBundle is Valid
    Observed Generation:   1
    Reason:                ValidationSucceeded
    Status:                True
    Type:                  Valid

Status:
  Conditions:
    Last Transition Time:  2026-06-22T21:11:54.975311+00:00
    Message:               <error-details>
    Reason:                ValidationFailed
    Status:                False
    Type:                  Valid

The Message field contains error details, including legalizer errors if any.

To read the legalizer result directly, including memory utilization and any warnings:

kubectl get modelbundle <bundle-name> -o jsonpath='{.status.legalizerInfo}'

For the full status structure and field descriptions, see Legalizer results.

Do not proceed to deployment until the bundle shows ValidationSucceeded.

Troubleshooting

A bundle is validated by the legalizer before it can be deployed. For the errors and warnings it reports and how to resolve them, see Legalizer validation failures.

Get started

Models

Features

Build

Resources

Create a custom bundle

Prerequisites

Quickstart - Hosted

Quickstart - On-prem

Create the bundle

Troubleshooting

Next steps

Deploy a bundle

ModelBundle

​Prerequisites

Quickstart - Hosted

Quickstart - On-prem

​Create the bundle

​Troubleshooting

​Next steps

Deploy a bundle

ModelBundle

Prerequisites

Create the bundle

Troubleshooting

Next steps