> ## Documentation Index
> Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy a bundle

This page describes how to serve a bundle by referencing it from a `ModelDeployment`. It applies both to the bundles that SambaNova provides and to bundles you authored yourself.

To author your own bundle first, see [Create a custom bundle](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/create-a-custom-bundle). To serve a single model without a bundle, see [Deploy a single model](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-a-single-model).

## Prerequisites

Before deploying, complete the quickstart that applies to you:

<CardGroup cols={2}>
  <Card icon="list-check" href="/docs/en/sambastack/getting-started/hosted" title="Quickstart - Hosted">
    System set up for hosted SambaStack
  </Card>

  <Card icon="list-check" href="/docs/en/sambastack/getting-started/on-prem" title="Quickstart - On-prem">
    System set up for on-prem SambaStack
  </Card>
</CardGroup>

Additionally:

* Review the [ModelDeployment](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-deployment) field reference.
* Confirm the bundle you intend to deploy has passed validation. See [Legalizer results](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-bundle#legalizer-results).
* Review [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles) for the bundles SambaNova provides.

## Deploy the bundle

<Steps>
  <Step title="Discover the available bundles">
    List the bundles present in the cluster and note the name of the one you want to deploy:

    <Tabs>
      <Tab title="Hosted">
        ```bash theme={}
        kubectl get modelbundles
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={}
        kubectl -n <namespace> get modelbundles.sambanova.ai
        ```
      </Tab>
    </Tabs>

    Output:

    ```text theme={}
    mb-gpt120                      17h
    mb-gpt120-llama70sd8-llama8    17h
    ds-v32-gemma-4-31b             3h
    ```

    Before deploying, confirm the bundle passed validation, as described in [Legalizer results](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-bundle#legalizer-results).
  </Step>

  <Step title="Create a ModelDeployment">
    Insert the bundle name into `spec.bundle`:

    ```yaml theme={}
    apiVersion: sambanova.ai/v1alpha1
    kind: ModelDeployment
    metadata:
      name: md-gpt120
    spec:
      bundle: mb-gpt120
      groups:
      - minReplicas: 1
        name: default
        qosList:
        - free
      owner: no-reply@sambanova.ai
      secretNames:
      - sambanova-artifact-reader
    ```
  </Step>

  <Step title="Apply the ModelDeployment">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={}
        kubectl apply -f <modeldeployment-file>.yaml
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={}
        kubectl -n <namespace> apply -f <modeldeployment-file>.yaml
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Monitor deployment status">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={}
        kubectl get modeldeployments
        kubectl describe modeldeployment <deployment-name>
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={}
        kubectl -n <namespace> get modeldeployments.sambanova.ai
        kubectl -n <namespace> describe modeldeployment.sambanova.ai <deployment-name>
        ```
      </Tab>
    </Tabs>
  </Step>
</Steps>

## Verify the deployment

Pods reporting ready means the serving containers started, not that the model answers requests. Confirm the serving name, then send one inference request.

### Resolve the serving name

The name you send to the API is not the Kubernetes resource name. Three names are involved:

| Name                       | Where it lives       | Used for                                                                             |
| -------------------------- | -------------------- | ------------------------------------------------------------------------------------ |
| `metadata.name` on a Model | The `Model` resource | `kubectl` commands, and `modelConfigs[].model` references in bundles and deployments |
| `spec.name` on a Model     | The `Model` resource | The `model` field of an inference request                                            |
| `spec.aliases` on a Model  | The `Model` resource | Additional names that route to the same model                                        |

Read the serving name from the Model you deployed:

<Tabs>
  <Tab title="Hosted">
    ```bash theme={}
    kubectl get model <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
    ```
  </Tab>

  <Tab title="On Premise">
    ```bash theme={}
    kubectl -n <namespace> get model.sambanova.ai <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
    ```
  </Tab>
</Tabs>

Then confirm the gateway is serving it. Each `id` in the response is a servable name:

```bash theme={}
curl https://<your-api-domain>/v1/models \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY"
```

<Note>
  A model set to `modelSettings.routable: false`, such as a speculative decoding draft model, is deliberately absent from this list and cannot be addressed directly.
</Note>

### Send a request

Use the serving name in the `model` field:

```bash theme={}
curl https://<your-api-domain>/v1/chat/completions \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Say hi in one sentence."}
    ],
    "max_tokens": 128,
    "stream": false
  }'
```

A successful response returns the serving name in `model` and a populated `choices[0].message`:

```json theme={}
{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "Hi there, it's a sunny day today!",
        "role": "assistant"
      }
    }
  ],
  "created": 1770927918,
  "id": "c754b468-05e1-4fe4-8a28-f51ecc74f4f2",
  "model": "gpt-oss-120b",
  "object": "chat.completion",
  "system_fingerprint": "fastcoe",
  "usage": {
    "completion_tokens": 121,
    "prompt_tokens": 91,
    "total_tokens": 212,
    "stop_reason": "stop",
    "time_to_first_token": 0.05286884307861328,
    "total_latency": 0.29257988929748535
  }
}
```

The `usage` object carries additional throughput and latency fields; for the full body, and for the API domain to use in each environment, see [Quickstart - Hosted](/docs/en/sambastack/getting-started/hosted) or [Quickstart - On-prem](/docs/en/sambastack/getting-started/on-prem).

<Warning>
  A `model not found` error with pods running usually means the request used `metadata.name` instead of `spec.name`. An inference error on a model that loads and lists usually means the checkpoint and the profile disagree on `model_arch`. See [Deployment failures](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting#deployment-failures).
</Warning>

## Switch bundles

A ModelDeployment's bundle reference cannot be changed in place. To serve a different bundle, delete the existing deployment and apply a new one that references the new bundle.

<Steps>
  <Step title="Delete the existing ModelDeployment">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={}
        kubectl delete modeldeployment <deployment-name>
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={}
        kubectl -n <namespace> delete modeldeployment <deployment-name>
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Apply a ModelDeployment for the new bundle">
    Author a new ModelDeployment whose `spec.bundle` names the bundle you want to serve, then apply it as described in [Deploy the bundle](#deploy-the-bundle).

    ```yaml theme={}
    apiVersion: sambanova.ai/v1alpha1
    kind: ModelDeployment
    metadata:
      name: md-qwen3-32b-whisper
    spec:
      bundle: mb-qwen3-32b-whisper
      groups:
      - minReplicas: 1
        name: default
        qosList:
        - free
      owner: no-reply@sambanova.ai
      secretNames:
      - sambanova-artifact-reader
    ```
  </Step>
</Steps>

To request new bundles, contact SambaNova support.

## Deploy multiple bundles

To serve more than one bundle at a time, apply a separate ModelDeployment for each bundle.

<Warning>
  SambaStack supports only one bundle per node. When deploying multiple bundles, assign each bundle to separate nodes to avoid resource conflicts. Use `spec.nodeSelector` on each ModelDeployment to control placement.
</Warning>

Verify that the pods reflect the deployed bundles:

<Tabs>
  <Tab title="Hosted">
    ```bash theme={}
    kubectl get pods
    ```
  </Tab>

  <Tab title="On Premise">
    ```bash theme={}
    kubectl -n <namespace> get pods
    ```
  </Tab>
</Tabs>

## Update or remove a deployment or bundle

<Tabs>
  <Tab title="Update a configuration">
    <Steps>
      <Step title="Modify the YAML file">
        Edit the ModelBundle, ModelProfile, or ModelDeployment YAML file with your changes.
      </Step>

      <Step title="Reapply the configuration">
        <Tabs>
          <Tab title="Hosted">
            ```bash theme={}
            kubectl apply -f <modified-file>.yaml
            ```
          </Tab>

          <Tab title="On Premise">
            ```bash theme={}
            kubectl -n <namespace> apply -f <modified-file>.yaml
            ```
          </Tab>
        </Tabs>

        The legalizer automatically revalidates the changes. Changing a profile or a model triggers a reconcile of every bundle and deployment that references it.
      </Step>
    </Steps>
  </Tab>

  <Tab title="Remove a configuration">
    <Steps>
      <Step title="Delete the ModelDeployment">
        <Tabs>
          <Tab title="Hosted">
            ```bash theme={}
            kubectl delete modeldeployment <deployment-name>
            ```
          </Tab>

          <Tab title="On Premise">
            ```bash theme={}
            kubectl -n <namespace> delete modeldeployment <deployment-name>
            ```
          </Tab>
        </Tabs>
      </Step>

      <Step title="Delete the ModelBundle (if one was used)">
        <Tabs>
          <Tab title="Hosted">
            ```bash theme={}
            kubectl delete modelbundle <bundle-name>
            ```
          </Tab>

          <Tab title="On Premise">
            ```bash theme={}
            kubectl -n <namespace> delete modelbundle <bundle-name>
            ```
          </Tab>
        </Tabs>
      </Step>

      <Step title="Delete the ModelProfile (optional)">
        Delete a profile only if you authored it for a custom architecture. Profiles provided with the SambaStack installation are shared across models and should be left in place.

        <Tabs>
          <Tab title="Hosted">
            ```bash theme={}
            kubectl delete modelprofile <profile-name>
            ```
          </Tab>

          <Tab title="On Premise">
            ```bash theme={}
            kubectl -n <namespace> delete modelprofile <profile-name>
            ```
          </Tab>
        </Tabs>
      </Step>
    </Steps>
  </Tab>
</Tabs>

## Troubleshooting

For the failures the operator reports once it starts creating serving pods, and for the failures the legalizer reports before that, see [Troubleshooting deployments](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting).

## Related documentation

<CardGroup cols={2}>
  <Card icon="cube" href="/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles" title="Supported models and bundles">
    Catalogue of models and bundles available for deployment
  </Card>

  <Card icon="server" href="/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-deployment" title="ModelDeployment">
    Full field reference for the deployment resource
  </Card>

  <Card icon="compress" href="/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-custom-checkpoints" title="Deploying custom checkpoints">
    Deploy your own custom or fine-tuned checkpoints
  </Card>

  <Card icon="rocket" href="/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/sambastack-v2-bundle-migration" title="Migrating bundle configuration">
    Migrate bundles built with the deprecated custom resources
  </Card>
</CardGroup>