> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy a bundle

This page describes how to serve a bundle by referencing it from a `ModelDeployment`. It applies both to the bundles that SambaNova provides and to bundles you authored yourself.

To author your own bundle first, see [Create a custom bundle](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/deploying-models-and-bundles/create-a-custom-bundle.md). To serve a single model without a bundle, see [Deploy a single model](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-a-single-model.md).

## Prerequisites

Before deploying, complete the quickstart that applies to you:

> [card] [Quickstart - Hosted](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/getting-started/hosted.md)
>
> System set up for hosted SambaStack

> [card] [Quickstart - On-prem](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/getting-started/on-prem.md)
>
> System set up for on-prem SambaStack

Additionally:

* Review the [ModelDeployment](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/custom-resources/model-deployment.md) field reference.
* Confirm the bundle you intend to deploy has passed validation. See [Legalizer results](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/custom-resources/model-bundle.md#legalizer-results).
* Review [Supported models and bundles](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/supported-models-and-bundles.md) for the bundles SambaNova provides.

## Deploy the bundle

**1. Discover the available bundles**

List the bundles present in the cluster and note the name of the one you want to deploy:

> [tab] Hosted
>
> ```bash theme={}
> kubectl get modelbundles
> ```

> [tab] On Premise
>
> ```bash theme={}
> kubectl -n <namespace> get modelbundles.sambanova.ai
> ```

Output:

```text theme={}
mb-gpt120                      17h
mb-gpt120-llama70sd8-llama8    17h
ds-v32-gemma-4-31b             3h
```

Before deploying, confirm the bundle passed validation, as described in [Legalizer results](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/custom-resources/model-bundle.md#legalizer-results).

**2. Create a ModelDeployment**

Insert the bundle name into `spec.bundle`:

```yaml theme={}
apiVersion: sambanova.ai/v1alpha1
kind: ModelDeployment
metadata:
  name: md-gpt120
spec:
  bundle: mb-gpt120
  groups:
  - minReplicas: 1
    name: default
    qosList:
    - free
  owner: no-reply@sambanova.ai
  secretNames:
  - sambanova-artifact-reader
```

**3. Apply the ModelDeployment**

> [tab] Hosted
>
> ```bash theme={}
> kubectl apply -f <modeldeployment-file>.yaml
> ```

> [tab] On Premise
>
> ```bash theme={}
> kubectl -n <namespace> apply -f <modeldeployment-file>.yaml
> ```

**4. Monitor deployment status**

> [tab] Hosted
>
> ```bash theme={}
> kubectl get modeldeployments
> kubectl describe modeldeployment <deployment-name>
> ```

> [tab] On Premise
>
> ```bash theme={}
> kubectl -n <namespace> get modeldeployments.sambanova.ai
> kubectl -n <namespace> describe modeldeployment.sambanova.ai <deployment-name>
> ```

## Verify the deployment

Pods reporting ready means the serving containers started, not that the model answers requests. Confirm the serving name, then send one inference request.

### Resolve the serving name

The name you send to the API is not the Kubernetes resource name. Three names are involved:

| Name | Where it lives | Used for |
| - | - | - |
| `metadata.name` on a Model | The `Model` resource | `kubectl` commands, and `modelConfigs[].model` references in bundles and deployments |
| `spec.name` on a Model | The `Model` resource | The `model` field of an inference request |
| `spec.aliases` on a Model | The `Model` resource | Additional names that route to the same model |

Read the serving name from the Model you deployed:

> [tab] Hosted
>
> ```bash theme={}
> kubectl get model <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
> ```

> [tab] On Premise
>
> ```bash theme={}
> kubectl -n <namespace> get model.sambanova.ai <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
> ```

Then confirm the gateway is serving it. Each `id` in the response is a servable name:

```bash theme={}
curl https://<your-api-domain>/v1/models \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY"
```

> [!NOTE]
>
> A model set to `modelSettings.routable: false`, such as a speculative decoding draft model, is deliberately absent from this list and cannot be addressed directly.

### Send a request

Use the serving name in the `model` field:

```bash theme={}
curl https://<your-api-domain>/v1/chat/completions \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Say hi in one sentence."}
    ],
    "max_tokens": 128,
    "stream": false
  }'
```

A successful response returns the serving name in `model` and a populated `choices[0].message`:

```json theme={}
{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "Hi there, it's a sunny day today!",
        "role": "assistant"
      }
    }
  ],
  "created": 1770927918,
  "id": "c754b468-05e1-4fe4-8a28-f51ecc74f4f2",
  "model": "gpt-oss-120b",
  "object": "chat.completion",
  "system_fingerprint": "fastcoe",
  "usage": {
    "completion_tokens": 121,
    "prompt_tokens": 91,
    "total_tokens": 212,
    "stop_reason": "stop",
    "time_to_first_token": 0.05286884307861328,
    "total_latency": 0.29257988929748535
  }
}
```

The `usage` object carries additional throughput and latency fields; for the full body, and for the API domain to use in each environment, see [Quickstart - Hosted](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/getting-started/hosted.md) or [Quickstart - On-prem](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/getting-started/on-prem.md).

> [!WARNING]
>
> A `model not found` error with pods running usually means the request used `metadata.name` instead of `spec.name`. An inference error on a model that loads and lists usually means the checkpoint and the profile disagree on `model_arch`. See [Deployment failures](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting.md#deployment-failures).

## Switch bundles

A ModelDeployment's bundle reference cannot be changed in place. To serve a different bundle, delete the existing deployment and apply a new one that references the new bundle.

**1. Delete the existing ModelDeployment**

> [tab] Hosted
>
> ```bash theme={}
> kubectl delete modeldeployment <deployment-name>
> ```

> [tab] On Premise
>
> ```bash theme={}
> kubectl -n <namespace> delete modeldeployment <deployment-name>
> ```

**2. Apply a ModelDeployment for the new bundle**

Author a new ModelDeployment whose `spec.bundle` names the bundle you want to serve, then apply it as described in [Deploy the bundle](#deploy-the-bundle).

```yaml theme={}
apiVersion: sambanova.ai/v1alpha1
kind: ModelDeployment
metadata:
  name: md-qwen3-32b-whisper
spec:
  bundle: mb-qwen3-32b-whisper
  groups:
  - minReplicas: 1
    name: default
    qosList:
    - free
  owner: no-reply@sambanova.ai
  secretNames:
  - sambanova-artifact-reader
```

To request new bundles, contact SambaNova support.

## Deploy multiple bundles

To serve more than one bundle at a time, apply a separate ModelDeployment for each bundle.

> [!WARNING]
>
> SambaStack supports only one bundle per node. When deploying multiple bundles, assign each bundle to separate nodes to avoid resource conflicts. Use `spec.nodeSelector` on each ModelDeployment to control placement.

Verify that the pods reflect the deployed bundles:

> [tab] Hosted
>
> ```bash theme={}
> kubectl get pods
> ```

> [tab] On Premise
>
> ```bash theme={}
> kubectl -n <namespace> get pods
> ```

## Update or remove a deployment or bundle

> [tab] Update a configuration
>
> **1. Modify the YAML file**
>
> Edit the ModelBundle, ModelProfile, or ModelDeployment YAML file with your changes.
>
> **2. Reapply the configuration**
>
> > [tab] Hosted
> >
> > ```bash theme={}
> > kubectl apply -f <modified-file>.yaml
> > ```
>
> > [tab] On Premise
> >
> > ```bash theme={}
> > kubectl -n <namespace> apply -f <modified-file>.yaml
> > ```
>
> The legalizer automatically revalidates the changes. Changing a profile or a model triggers a reconcile of every bundle and deployment that references it.

> [tab] Remove a configuration
>
> **1. Delete the ModelDeployment**
>
> > [tab] Hosted
> >
> > ```bash theme={}
> > kubectl delete modeldeployment <deployment-name>
> > ```
>
> > [tab] On Premise
> >
> > ```bash theme={}
> > kubectl -n <namespace> delete modeldeployment <deployment-name>
> > ```
>
> **2. Delete the ModelBundle (if one was used)**
>
> > [tab] Hosted
> >
> > ```bash theme={}
> > kubectl delete modelbundle <bundle-name>
> > ```
>
> > [tab] On Premise
> >
> > ```bash theme={}
> > kubectl -n <namespace> delete modelbundle <bundle-name>
> > ```
>
> **3. Delete the ModelProfile (optional)**
>
> Delete a profile only if you authored it for a custom architecture. Profiles provided with the SambaStack installation are shared across models and should be left in place.
>
> > [tab] Hosted
> >
> > ```bash theme={}
> > kubectl delete modelprofile <profile-name>
> > ```
>
> > [tab] On Premise
> >
> > ```bash theme={}
> > kubectl -n <namespace> delete modelprofile <profile-name>
> > ```

## Troubleshooting

For the failures the operator reports once it starts creating serving pods, and for the failures the legalizer reports before that, see [Troubleshooting deployments](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting.md).

## Related documentation

> [card] [Supported models and bundles](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/supported-models-and-bundles.md)
>
> Catalogue of models and bundles available for deployment

> [card] [ModelDeployment](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/custom-resources/model-deployment.md)
>
> Full field reference for the deployment resource

> [card] [Deploying custom checkpoints](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-custom-checkpoints.md)
>
> Deploy your own custom or fine-tuned checkpoints

> [card] [Migrating bundle configuration](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/deploying-models-and-bundles/sambastack-v2-bundle-migration.md)
>
> Migrate bundles built with the deprecated custom resources