> ## Documentation Index
> Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy a single model

To serve one model, pair it with a profile and deploy it directly with a `ModelDeployment`. No bundle is created.

Create a bundle instead when you want to serve several models as one unit, configure speculative decoding, or validate and share a named configuration. See [Create a custom bundle](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/create-a-custom-bundle).

## Prerequisites

Before deploying, complete the quickstart that applies to you:

<CardGroup cols={2}>
  <Card icon="list-check" href="/docs/en/sambastack/getting-started/hosted" title="Quickstart - Hosted">
    System set up for hosted SambaStack
  </Card>

  <Card icon="list-check" href="/docs/en/sambastack/getting-started/on-prem" title="Quickstart - On-prem">
    System set up for on-prem SambaStack
  </Card>
</CardGroup>

Additionally:

* Review the [deployment overview](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/overview) for how models, profiles, and deployments fit together.
* Identify the model and profile you want to pair, and confirm they are compatible. See [Identify available models and profiles](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/overview#identify-available-models-and-profiles).
* Check [Supported models and bundles](/docs/en/sambastack/service-administration/model-deployment/supported-models-and-bundles) for the available model checkpoints.

## Deploy the model

<Steps>
  <Step title="Create the YAML file">
    Create a YAML file that declares the model and profile inline under `spec.models`:

    ```yaml theme={}
    apiVersion: sambanova.ai/v1alpha1
    kind: ModelDeployment
    metadata:
      name: md-gpt120
    spec:
      models:
        modelConfigs:
        - model: gpt-oss-120b:1
          profile: gpt-oss-fp8-dyt
      groups:
      - minReplicas: 1
        name: default
        qosList:
        - free
      owner: no-reply@sambanova.ai
      secretNames:
      - sambanova-artifact-reader
      engineConfig:
        startupTimeout: 7200
    ```

    A checkpoint path is not required, because the checkpoint is resolved from the referenced `Model`.

    `startupTimeout` is in seconds. The `7200` here gives the engine two hours to download and load the model before its startup probe fails, well above the 45-minute default. See [Startup timeouts](/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-deployment#startup-timeouts).
  </Step>

  <Step title="Apply the ModelDeployment">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={}
        kubectl apply -f <modeldeployment-file>.yaml
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={}
        kubectl -n <namespace> apply -f <modeldeployment-file>.yaml
        ```
      </Tab>
    </Tabs>
  </Step>

  <Step title="Monitor deployment status">
    <Tabs>
      <Tab title="Hosted">
        ```bash theme={}
        kubectl get modeldeployments
        kubectl describe modeldeployment <deployment-name>
        ```
      </Tab>

      <Tab title="On Premise">
        ```bash theme={}
        kubectl -n <namespace> get modeldeployments.sambanova.ai
        kubectl -n <namespace> describe modeldeployment.sambanova.ai <deployment-name>
        ```
      </Tab>
    </Tabs>

    The deployment is complete once all of its serving pods report a ready status.
  </Step>
</Steps>

## Verify the deployment

Pods reporting ready means the serving containers started, not that the model answers requests. Confirm the serving name, then send one inference request.

### Resolve the serving name

The name you send to the API is not the Kubernetes resource name. Three names are involved:

| Name                       | Where it lives       | Used for                                                                             |
| -------------------------- | -------------------- | ------------------------------------------------------------------------------------ |
| `metadata.name` on a Model | The `Model` resource | `kubectl` commands, and `modelConfigs[].model` references in bundles and deployments |
| `spec.name` on a Model     | The `Model` resource | The `model` field of an inference request                                            |
| `spec.aliases` on a Model  | The `Model` resource | Additional names that route to the same model                                        |

Read the serving name from the Model you deployed:

<Tabs>
  <Tab title="Hosted">
    ```bash theme={}
    kubectl get model <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
    ```
  </Tab>

  <Tab title="On Premise">
    ```bash theme={}
    kubectl -n <namespace> get model.sambanova.ai <model-resource-name> -o jsonpath='{.spec.name}{"\n"}'
    ```
  </Tab>
</Tabs>

Then confirm the gateway is serving it. Each `id` in the response is a servable name:

```bash theme={}
curl https://<your-api-domain>/v1/models \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY"
```

<Note>
  A model set to `modelSettings.routable: false`, such as a speculative decoding draft model, is deliberately absent from this list and cannot be addressed directly.
</Note>

### Send a request

Use the serving name in the `model` field:

```bash theme={}
curl https://<your-api-domain>/v1/chat/completions \
  -H "Authorization: Bearer $SAMBANOVA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-oss-120b",
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Say hi in one sentence."}
    ],
    "max_tokens": 128,
    "stream": false
  }'
```

A successful response returns the serving name in `model` and a populated `choices[0].message`:

```json theme={}
{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "Hi there, it's a sunny day today!",
        "role": "assistant"
      }
    }
  ],
  "created": 1770927918,
  "id": "c754b468-05e1-4fe4-8a28-f51ecc74f4f2",
  "model": "gpt-oss-120b",
  "object": "chat.completion",
  "system_fingerprint": "fastcoe",
  "usage": {
    "completion_tokens": 121,
    "prompt_tokens": 91,
    "total_tokens": 212,
    "stop_reason": "stop",
    "time_to_first_token": 0.05286884307861328,
    "total_latency": 0.29257988929748535
  }
}
```

The `usage` object carries additional throughput and latency fields; for the full body, and for the API domain to use in each environment, see [Quickstart - Hosted](/docs/en/sambastack/getting-started/hosted) or [Quickstart - On-prem](/docs/en/sambastack/getting-started/on-prem).

<Warning>
  A `model not found` error with pods running usually means the request used `metadata.name` instead of `spec.name`. An inference error on a model that loads and lists usually means the checkpoint and the profile disagree on `model_arch`. See [Deployment failures](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting#deployment-failures).
</Warning>

## Update or remove the deployment

To change a deployment's configuration, edit its YAML file and reapply it. The legalizer automatically revalidates the changes.

<Tabs>
  <Tab title="Hosted">
    ```bash theme={}
    kubectl apply -f <modified-file>.yaml
    ```
  </Tab>

  <Tab title="On Premise">
    ```bash theme={}
    kubectl -n <namespace> apply -f <modified-file>.yaml
    ```
  </Tab>
</Tabs>

To remove the deployment:

<Tabs>
  <Tab title="Hosted">
    ```bash theme={}
    kubectl delete modeldeployment <deployment-name>
    ```
  </Tab>

  <Tab title="On Premise">
    ```bash theme={}
    kubectl -n <namespace> delete modeldeployment <deployment-name>
    ```
  </Tab>
</Tabs>

## Troubleshooting

A deployment created from an inline `spec.models` specification is legalized in place, so both classes of failure can apply to it. See [Troubleshooting deployments](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting) for [legalizer validation failures](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting#legalizer-validation-failures) and [deployment failures](/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/troubleshooting#deployment-failures).

## Related documentation

<CardGroup cols={2}>
  <Card icon="server" href="/docs/en/sambastack/service-administration/model-deployment/custom-resources/model-deployment" title="ModelDeployment">
    Full field reference for the deployment resource
  </Card>

  <Card icon="layer-group" href="/docs/en/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-a-bundle" title="Deploy a bundle">
    Serve several models together as one unit
  </Card>
</CardGroup>