> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sambanova.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Supported models and bundles

SambaStack supports a variety of models that can be deployed to both on-prem and hosted environments. Contact your system administrator to determine which models are available on your deployment. For definitions of the **Preview** and **Stable** designations, how models move between stages, the notice you receive before a model is retired, and the deprecation history, see [SambaStack model lifecycle](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/model-lifecycle.md).

## All supported models

Search or filter to find a model, then select it to jump to its full configuration.

All providers

All modalities

— select one to jump to its full
configuration below

## Custom checkpoints

All decoder-only text generation models support custom checkpoints, except the models in the following table. To list only the models that support them, select **Custom checkpoints** in the filter above. To convert a checkpoint, see [Checkpoint conversion tool](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/checkpoint-conversion-tool.md).

| Model | Reason |
| - | - |
| `Llama-4-Maverick-17B-128E-Instruct` | Multimodal, not decoder-only text |
| `MiniMax-M3` | Multimodal, not decoder-only text |
| `MiniMax-M3.1-Flash` | Multimodal, not decoder-only text |
| `gemma-3-27b-it` | Multimodal, not decoder-only text |
| `gemma-3-12b-it` | Multimodal, not decoder-only text |
| `gemma-4-31B-it` | Multimodal, not decoder-only text |
| `Qwen3-32B` | No conversion package available yet |
| `GLM-5.3` | No conversion package available yet |
| `Whisper-Large-v3` | Not a text generation model |
| `Qwen3-TTS-Talker` | Not a text generation model |
| `Qwen3-TTS-Vocoder` | Not a text generation model |
| `E5-Mistral-7B-Instruct` | Not a text generation model |

## Finding models on your cluster

You can run the following command to discover available models in your cluster:

```shellscript theme={}
kubectl -n <namespace> get models
```

> [accordion] kubectl get models does not return the name you send to the API
>
> The command lists Kubernetes resource names (`metadata.name`). Inference requests must use the serving name (`spec.name`), which is what the **Model ID** headings below use.
>
> ```yaml theme={}
> apiVersion: sambanova.ai/v1alpha1
> kind: Model
> metadata:
>   name: gemma-4-31b-it    # Kubernetes resource name – what kubectl get models returns
> spec:
>   name: gemma-4-31B-it    # serving name – use this in API requests
> ```
>
> The two often differ only in case or punctuation, so the mismatch is easy to miss. In the example above they differ by a single character.
>
> To read the serving name of a deployed model, run:
>
> ```shellscript theme={}
> kubectl -n <namespace> describe model <resource-name>
> ```
>
> A model can also define `spec.aliases`, additional names that route to the same model. For the full `Model` field reference, see [Deploy custom checkpoints](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deploy-custom-checkpoints.md).
>
> Text to speech is one case where the names diverge: address `/v1/audio/speech` as `qwen3-tts`, not by the talker or vocoder resource names. See [Text to speech](https://docs.sambanova.ai/docs/en/features/speech.md).

## Recommended model bundles

In SambaStack, a bundle is a packaged deployment that groups one or more models together with their associated configurations, such as batch size and sequence length. A single model can also be deployed on its own by pairing it with a model profile, without creating a bundle.

For example, deploying the `Meta‑Llama‑3.3‑70B` model with a batch size of 4 and a sequence length of 16K tokens constitutes a single configuration. A bundle, however, can contain multiple such configurations, either for the same model or for different models.

SambaNova’s RDU technology enables several models and configurations to be loaded simultaneously in a single deployment. This allows you to switch instantly between models and between batch‑/sequence‑size profiles as needed. In contrast to traditional GPU systems, where deployments are typically single‑model and static, SambaStack supports multi‑model, multi‑configuration bundles. This approach delivers higher efficiency, greater flexibility, and increased throughput while preserving low latency.

You can run the following command to discover available bundles in your cluster:

```shellscript theme={}
kubectl -n <namespace> get modelbundles
```

The table below lists the recommended bundles for the models currently available in SambaStack. Each entry pairs a model with its recommended deployment bundle.

> [!NOTE]
>
> If the bundles listed below do not satisfy your inference requirements, you can create [custom bundles](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/deploying-models-and-bundles/create-a-custom-bundle.md) that combine any mix of models and configurations so long as they fit in DDR memory.

### Deployment options

When deploying models in SambaStack, administrators can select from various context length and batch size combinations.

* Smaller batch sizes provide higher token throughput (tokens/second).
* Larger batch sizes provide better concurrency for multiple users.

SambaStack supports two deployment configurations – high-interactivity and high-throughput. See [High-throughput deployment](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/model-deployment/deploying-models-and-bundles/deployment-configurations.md) for when to use each.

### Suggested bundles per model

For each model, this section lists the suggested bundle for typical use and any alternative bundles that trade off context length, batch size, or modality support. See [Bundle configurations](#bundle-configurations) below for the seq length/batch size details of each bundle.

| Provider | Model | Suggested bundle | Alternative bundles |
| - | - | - | - |
| Meta | `Meta-Llama-3.3-70B-Instruct` | `70b-3dot3-ss-4-8-16-32-64-128k` | `70b-3dot3-ss-full-whisper`, `us-agentic-rag-1-1`, `e5-mistral-70b-64k-128k` |
| Meta | `Meta-Llama-3.1-8B-Instruct` | `us-agentic-rag-1-1` | `qwen3-32b-llama405b-s-m` |
| Meta | `Meta-Llama-3.1-405B-Instruct` | `qwen3-32b-llama405b-s-m` | — |
| Meta | `Llama-4-Maverick-17B-128E-Instruct` | `llama-4-medium-8-16-32-64-128k` | `llama-4-medium-ss-16k-bs24`, `us-agentic-rag-1-1` |
| MiniMax | `MiniMax-M3` | `minimax-m3-32-64-128-256-512k-1m` (up to 1M context) | `minimax-m3-512k` (up to 512K context), `minimax-m3-32k`, `minimax-m3-acb-prefix-caching` (adds [prompt caching](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/performance/prompt-caching.md), up to 1M context; text, image, and video input), `minimax-m3-32k-acb-prefix-caching`, `minimax-m3-256k-acb-prefix-caching`, `minimax-m3-512k-acb-prefix-caching`, `minimax-m3-1m-acb-prefix-caching` (prompt caching capped at 32K, 256K, 512K, or 1M context; text, image, and video input) |
| MiniMax | `MiniMax-M3.1-Flash`<sup>PREVIEW</sup> | `minimax-m3-1-flash-acb-prefix-caching` (adds [prompt caching](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/performance/prompt-caching.md), up to 1M context) | `minimax-m3-1-flash-32k-acb-prefix-caching` (prompt caching capped at 32K context) |
| MiniMax | `MiniMax-M2.7` | `dyt-minimax-m2p7-32-64-160-192k-pc` (adds [prompt caching](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/performance/prompt-caching.md)) | `dyt-minimax-m2p7-32k-pc`, `dyt-minimax-m2p7-32-64-160-192k`, `dyt-minimax-m2p7-32-160-192k`, `dyt-minimax-m2p7-32k-v2` |
| MiniMax | `MiniMax-M2.5` | `dyt-minimax-m2p5-32-160k` | `dyt-minimax-m2p5-32k` |
| Mistral AI | `Mistral-Large-3-675B-Instruct-2512`<sup>PREVIEW</sup> | `mistral-large-3-fp8-8-16-32k` | `mistral-large-3-fp8-8k` |
| DeepSeek | `DeepSeek-R1-0528` | `deepseek-4in1-fp8-128k` (higher context length) | `deepseek-r1-v3-fp8-8k`, `deepseek-r1-v31-fp8-8k` |
| DeepSeek | `DeepSeek-V3-0324` | `deepseek-4in1-fp8-128k` (higher context length) | `deepseek-r1-v3-fp8-8k`, `deepseek-v3-v31-fp8-8k`, `deepseek-v3-v3termi-fp8-8k` |
| DeepSeek | `DeepSeek-V3.1` | `deepseek-4in1-fp8-128k` (higher context length) | `deepseek-r1-v31-fp8-8k`, `deepseek-v3-v31-fp8-8k` |
| DeepSeek | `DeepSeek-V3.1-Terminus` | `deepseek-4in1-fp8-128k` (higher context length) | `deepseek-v3-v3termi-fp8-8k` |
| OpenAI | `gpt-oss-120b` | `us-agentic-rag-1-1` | `cd-dyt-gpt-oss-120b-8-32-64-128k`, `gpt-gemma-whisper-mistral` |
| OpenAI | `gpt-oss-20b`<sup>PREVIEW</sup> | `dyt-gpt-oss-20b-32-64-128k` | — |
| OpenAI | `Whisper-Large-v3` | `qwen3-32b-whisper-e5-mistral` | `70b-3dot3-ss-full-whisper` (known issue: does not load within default startup time), `gpt-gemma-whisper-mistral` |
| Google | `gemma-3-27b-it`<sup>PREVIEW</sup> | `gemma3-27b-32-128k` | — |
| Google | `gemma-3-12b-it`<sup>PREVIEW</sup> | `gemma3-v3` | `gpt-gemma-whisper-mistral` |
| Google | `gemma-4-31B-it` | `gemma-4-31b-32-128-256k` (adds 256K context; text, image, and video support), `gemma-4-31b-32-128k` (up to 128K context; text, image, and video support, higher throughput). Both bundles support constrained decoding across all of their sequence lengths. Choose between them on context length and throughput, not capability. | `gemma-4-31b-mtp-cd-8-32-64-128k` (Preview; multi-token prediction, up to 128K context), `gemma-4-31b-acb-prefix-caching` (adds [prompt caching](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/performance/prompt-caching.md), up to 256K context; text, image, and video support), `gemma-4-31b-acb-prefix-caching-32k` (adds [prompt caching](https://docs.sambanova.ai/docs/en/v2.3.2/sambastack/service-administration/performance/prompt-caching.md), up to 32K context, up to 64 concurrent requests; text, image, and video support), `dyt-cd-gpt-oss-gemma4-allseq-e5-whisper` (Preview; multi-token prediction, up to 128K context; shared with `gpt-oss-120b`, `E5-Mistral-7B-Instruct`, and `Whisper-Large-v3`) |
| Alibaba Cloud | `Qwen3-235B-A22B-Instruct-2507` | `dyt-qwen3-235b-32-128k` | — |
| Alibaba Cloud | `Qwen3-32B` | `qwen3-32b-whisper-e5-mistral` | `qwen3-32b-llama405b-s-m` |
| Alibaba Cloud | `Qwen3-TTS-Talker`<sup>PREVIEW</sup> | `qwen3-tts-talker` | — |
| Alibaba Cloud | `Qwen3-TTS-Vocoder`<sup>PREVIEW</sup> | `qwen3-tts-vocoder` | — |
| Zhipu AI | `GLM-5.3`<sup>PREVIEW</sup> | `glm-5-3-acb-prefix-caching` (prompt caching, up to 1M context) | — |
| Other | `E5-Mistral-7B-Instruct` | `us-agentic-rag-1-1` | `e5-mistral-70b-64k-128k`, `qwen3-32b-whisper-e5-mistral`, `gpt-gemma-whisper-mistral` |

### Bundle configurations

The table below lists the configuration details for each bundle.

| Model name | Bundle name | Bundle description | Bundle configuration |
| - | :- | :- | :- |
| `DeepSeek-R1-0528` / `DeepSeek-V3.1` | deepseek-r1-v31-fp8-8k | <ul><li>Medium context length with low batch size</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`DeepSeek-R1-0528`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li><li>`DeepSeek-V3.1`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li></ul></details> |
| `DeepSeek-R1-0528` / `DeepSeek-V3-0324` | deepseek-r1-v3-fp8-8k | <ul><li>Medium context length with low batch size</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`DeepSeek-R1-0528`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li><li>`DeepSeek-V3-0324`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li></ul></details> |
| `DeepSeek-V3-0324` / `DeepSeek-V3.1` | deepseek-v3-v31-fp8-8k | <ul><li>Medium context length with low batch size</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`DeepSeek-V3-0324`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li><li>`DeepSeek-V3.1`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li></ul></details> |
| `DeepSeek-V3-0324` / `DeepSeek-V3.1-Terminus` | deepseek-v3-v3termi-fp8-8k | <ul><li>Medium context length with low batch size</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`DeepSeek-V3-0324`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li><li>`DeepSeek-V3.1-Terminus`<ul><li>Seq Length: 8K, BS: 1, 4</li></ul></li></ul></details> |
| `DeepSeek-R1-0528` / `DeepSeek-V3-0324` / `DeepSeek-V3.1` / `DeepSeek-V3.1-Terminus` | deepseek-4in1-fp8-128k | <ul><li>Large context length with single batch size</li><li>Four DeepSeek models in one bundle</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`DeepSeek-R1-0528`<ul><li>Seq Length: 128K, BS: 1</li></ul></li><li>`DeepSeek-V3-0324`<ul><li>Seq Length: 128K, BS: 1</li></ul></li><li>`DeepSeek-V3.1`<ul><li>Seq Length: 128K, BS: 1</li></ul></li><li>`DeepSeek-V3.1-Terminus`<ul><li>Seq Length: 128K, BS: 1</li></ul></li></ul></details> |
| `E5-Mistral-7B-Instruct` / `Meta-Llama-3.1-8B-Instruct` / `Llama-4-Maverick-17B-128E-Instruct` / `Meta-Llama-3.3-70B-Instruct` / `gpt-oss-120b` | us-agentic-rag-1-1 | <ul><li>Small to medium context length with varied batch size</li><li>Speculative decoding supported for `Meta-Llama-3.3-70B`</li></ul> | <details><summary>View</summary><ul><li>`gpt-oss-120b`<ul><li>Seq Length: 32K, BS: 4</li><li>Seq Length: 64K, BS: 2</li><li>Seq Length: 128K, BS: 2</li></ul></li><li>`Llama-4-Maverick-17B-128E-Instruct`<ul><li>Seq Length: 8K, BS: 1</li><li>Seq Length: 16K, BS: 1</li></ul></li><li>`Meta-Llama-3.3-70B` (Target)/ `Meta-Llama-3.2-1B` (Draft)<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 1, 4, 8</li><li>Seq Length: 16K, BS: 1, 4</li><li>Seq Length: 32K, BS: 1, 4</li><li>Seq Length: 64K, BS: 1</li><li>Seq Length: 128K, BS: 1</li></ul></li><li>`Meta-Llama-3.1-8B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 16, 32</li><li>Seq Length: 8K, BS: 1, 4, 16, 32</li><li>Seq Length: 16K, BS: 1, 4, 8</li></ul></li><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li></ul></li></ul></details> |
| `E5-Mistral-7B-Instruct` / `Meta-Llama-3.3-70B` | e5-mistral-70b-64k-128k | <ul><li>Large context length with low batch size</li><li>Speculative decoding supported for `Meta-Llama-3.3-70B`</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li></ul></li><li>`Meta-Llama-3.3-70B` (Target)/ `Meta-Llama-3.2-1B` (Draft)<ul><li>Seq Length: 64K, BS: 1</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul></details> |
| `gemma-3-27b-it` | gemma3-27b-32-128k | Homogeneous bundle containing `gemma-3-27b-it` configurations. | <details><summary>View</summary><ul><li>`gemma-3-27b-it`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li></ul></details> |
| `gemma-3-12b-it` | gemma3-v3 | <ul><li>Homogeneous bundle containing `gemma-3-12b-it` configurations.</li><li>Large context length with medium batch size</li></ul> | <details><summary>View</summary><ul><li>`gemma-3-12b-it`<ul><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li></ul></details> |
| `gemma-4-31B-it` | gemma-4-31b-32-128k | Homogeneous bundle with constrained decoding for `gemma-4-31B-it`. Supports text, image, and video input. | <details><summary>View</summary><ul><li>`gemma-4-31B-it`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li></ul></details> |
| `gemma-4-31B-it` | `gemma-4-31b-32-128-256k` | <ul><li>Homogeneous bundle with constrained decoding for `gemma-4-31B-it`. Adds context support up to 256K. Supports text, image, and video input at all sequence lengths.</li></ul> | <details><summary>View</summary><ul><li>`gemma-4-31B-it`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2, 4, 6, 8</li><li>Seq Length: 256K, BS: 2</li></ul></li></ul></details> |
| `gemma-4-31B-it` | `gemma-4-31b-acb-prefix-caching` | <ul><li>Homogeneous bundle for `gemma-4-31B-it` with prompt caching, continuous batching, multi-token prediction, and constrained decoding. Up to 256K context.</li><li>Supports text, image, and video input. Requests that contain an image or video, including in an earlier turn, are not cached.</li><li>Accepts video only as a base64 `data:` URL. If you deploy this bundle with another `gemma-4-31B-it` bundle, send all video as a `data:` URL: SambaStack sends each request to a bundle by model name only, so any video request can go to this bundle.</li></ul> | <details><summary>View</summary><ul><li>`gemma-4-31B-it`<ul><li>Seq Length: 256K, BS: 2</li></ul></li></ul></details> |
| `gemma-4-31B-it` | `gemma-4-31b-acb-prefix-caching-32k` | <ul><li>Homogeneous bundle for `gemma-4-31B-it` with prompt caching, continuous batching, multi-token prediction, and constrained decoding. Up to 32K context.</li><li>Serves up to 64 concurrent requests, against 8 on `gemma-4-31b-acb-prefix-caching`.</li><li>Supports text, image, and video input, with the same caching and video rules as `gemma-4-31b-acb-prefix-caching`.</li></ul> | <details><summary>View</summary><ul><li>`gemma-4-31B-it`<ul><li>Seq Length: 32K, BS: 2</li></ul></li></ul></details> |
| `gemma-4-31B-it`<sup>PREVIEW</sup> | `gemma-4-31b-mtp-cd-8-32-64-128k` | <ul><li>Homogeneous bundle with multi-token prediction and constrained decoding for `gemma-4-31B-it`. Up to 128K context; text, image, and video input at all sequence lengths.</li><li>Multi-token prediction raises decode throughput; the model predicts several tokens per step and verifies them in the same pass.</li></ul> | <details><summary>View</summary><ul><li>`gemma-4-31B-it`<ul><li>Seq Length: 8K, BS: 2, 4, 6, 8</li><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li></ul></details> |
| `GLM-5.3`<sup>PREVIEW</sup> | `glm-5-3-acb-prefix-caching` | <ul><li>Homogeneous bundle for `GLM-5.3` with prompt caching, continuous batching, multi-token prediction, and constrained decoding. Up to 1M context.</li><li>Text input only.</li></ul> | <details><summary>View</summary><ul><li>`GLM-5.3`<ul><li>Seq Length: 1M, BS: 4</li></ul></li></ul></details> |
| `gpt-oss-120b` | `cd-dyt-gpt-oss-120b-8-32-64-128k` † | <ul><li>Homogeneous bundle with constrained decoding for `gpt-oss-120b`.</li><li>Covers 8K through 128K context.</li><li>Includes structured output (logit-masking) support.</li></ul> | <details><summary>View</summary><ul><li>`cd-dyt-gpt-oss-120b-8-32-64-128k`<ul><li>Seq Length: 8K, BS: 2, 4, 6, 8</li><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4</li><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details> |
| `gpt-oss-20b` | `dyt-gpt-oss-20b-32-64-128k` † | Homogeneous bundle with constrained decoding for `gpt-oss-20b`. | <details><summary>View</summary><ul><li>`dyt-gpt-oss-20b-32-64-128k`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4</li><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details> |
| `E5-Mistral-7B-Instruct` / `Whisper-Large-v3` / `gemma-3-12b-it` / `gpt-oss-120b` | gpt-gemma-whisper-mistral | <ul><li>Small to medium context length combining embeddings, transcription, image understanding, and a tool-calling agent</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1</li></ul></li><li>`Whisper-Large-v3`<ul><li>BS: 1</li></ul></li><li>`gemma-3-12b-it`<ul><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li><li>`gpt-oss-120b`<ul><li>Seq Length: 8K, BS: 2, 4, 6, 8</li></ul></li></ul></details> |
| `E5-Mistral-7B-Instruct` / `Whisper-Large-v3` / `gemma-4-31B-it`<sup>PREVIEW</sup> / `gpt-oss-120b` | `dyt-cd-gpt-oss-gemma4-allseq-e5-whisper` | <ul><li>Embeddings, transcription, `gemma-4-31B-it` with multi-token prediction and constrained decoding, and `gpt-oss-120b` with constrained decoding in one bundle.</li><li>`gemma-4-31B-it` uses the same compiled model as `gemma-4-31b-mtp-cd-8-32-64-128k`, at 32K and 128K context.</li></ul> | <details><summary>View</summary>**Models:**<ul><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 32</li></ul></li><li>`Whisper-Large-v3`<ul><li>BS: 32</li></ul></li><li>`gemma-4-31B-it`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2, 4, 6, 8</li></ul></li><li>`gpt-oss-120b`<ul><li>Seq Length: 8K, BS: 2, 4, 6, 8</li><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4</li><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details> |
| `Llama-4-Maverick-17B-128E-Instruct` | llama-4-medium-8-16-32-64-128k | <ul><li>Homogeneous bundles containing `Llama-4-Maverick-17B-128E-Instruct` configurations.</li><li>Small to large context length with low batch</li></ul> | <details><summary>View</summary><ul><li>`Llama-4-Maverick-17B-128E-Instruct`<ul><li>Seq Length: 8K, BS: 1</li><li>Seq Length: 16K, BS: 1</li><li>Seq Length: 32K, BS: 1</li><li>Seq Length: 64K, BS: 1</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul></details> |
| `Llama-4-Maverick-17B-128E-Instruct` | llama-4-medium-ss-16k-bs24 | Homogeneous bundle containing `Llama-4-Maverick-17B-128E-Instruct` configurations; medium context length with higher batch size. | <details><summary>View</summary><ul><li>`Llama-4-Maverick-17B-128E-Instruct`<ul><li>Seq Length: 16K, BS: 2, 4</li></ul></li></ul></details> |
| `Meta-Llama-3.3-70B-Instruct` | 70b-3dot3-ss-4-8-16-32-64-128k | <ul><li>Medium to large context length with low batch size</li></ul> | <details><summary>View</summary>**Target Models:**<ul><li>`Meta-Llama-3.3-70B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul>**Draft Models:**<ul><li>`Meta-Llama-3.2-1B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4; private: true</li><li>Seq Length: 128K, BS: 1; private: true</li></ul></li></ul></details> |
| `Meta-Llama-3.3-70B-Instruct` / `Whisper-Large-v3` | 70b-3dot3-ss-full-whisper | <ul><li>Full context-length range with low to medium batch size, plus Whisper transcription</li><li>Speculative decoding supported for `Meta-Llama-3.3-70B-Instruct`</li></ul> | <details><summary>View</summary>**Target Models:**<ul><li>`Meta-Llama-3.3-70B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4</li><li>Seq Length: 128K, BS: 1</li></ul></li><li>`Whisper-Large-v3`<ul><li>BS: 1, 16, 32</li></ul></li></ul>**Draft Models:**<ul><li>`Meta-Llama-3.2-1B-Instruct`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 8K, BS: 2, 4, 8, 16, 32</li><li>Seq Length: 16K, BS: 1, 2, 4</li><li>Seq Length: 32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4</li><li>Seq Length: 128K, BS: 1</li></ul></li></ul></details> |
| `MiniMax-M2.5` | <ul><li>dyt-minimax-m2p5-32k</li><li>dyt-minimax-m2p5-32-160k</li></ul> | <ul><li>Homogeneous bundles containing `MiniMax-M2.5` configurations.</li><li>dyt-minimax-m2p5-32k is better for medium sequence lengths and high batching.</li><li>dyt-minimax-m2p5-32-160k is better for higher sequence lengths and low batching</li></ul> | <details><summary>View</summary><ul><li>dyt-minimax-m2p5-32k<ul><li>Seq Length: 4K-32K, BS: 2, 4, 6, 8</li></ul></li></ul><ul><li>dyt-minimax-m2p5-32-160k<ul><li>Seq Length: 32K, BS: 2</li><li>Seq Length: 160K, BS: 2</li></ul></li></ul></details> |
| `MiniMax-M2.7` | dyt-minimax-m2p7-32k-v2 | Homogeneous bundle containing `MiniMax-M2.7` configurations; medium context length with high batching. | <details><summary>View</summary><ul><li>`MiniMax-M2.7`<ul><li>Seq Length: 8K-32K, BS: 2, 4, 6, 8</li></ul></li></ul></details> |
| `MiniMax-M2.7` | dyt-minimax-m2p7-32-160-192k | Homogeneous bundle containing `MiniMax-M2.7` configurations; better for higher sequence lengths and low batching. | <details><summary>View</summary><ul><li>`MiniMax-M2.7`<ul><li>Seq Length: 8K-32K, BS: 2, 4, 6, 8</li><li>Seq Length: 160K-192K, BS: 2</li></ul></li></ul></details> |
| `MiniMax-M2.7` | dyt-minimax-m2p7-32-64-160-192k-pc | Homogeneous bundle with prompt caching for `MiniMax-M2.7`. | <details><summary>View</summary><ul><li>`MiniMax-M2.7`<ul><li>Seq Length: 8K-32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2, 4</li><li>Seq Length: 160K-192K, BS: 2</li></ul></li></ul></details> |
| `MiniMax-M2.7` | dyt-minimax-m2p7-32-64-160-192k | <ul><li>Homogeneous bundle containing `MiniMax-M2.7` configurations; same context range as `dyt-minimax-m2p7-32-64-160-192k-pc` without prompt caching.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M2.7`<ul><li>Seq Length: 8K-32K, BS: 2, 4, 6, 8</li><li>Seq Length: 64K, BS: 2</li><li>Seq Length: 160K-192K, BS: 2</li></ul></li></ul></details> |
| `MiniMax-M2.7` | dyt-minimax-m2p7-32k-pc | Homogeneous bundle with prompt caching for `MiniMax-M2.7`; medium context length with high batching. | <details><summary>View</summary><ul><li>`MiniMax-M2.7`<ul><li>Seq Length: 8K-32K, BS: 2, 4, 6, 8</li></ul></li></ul></details> |
| `MiniMax-M3` | minimax-m3-32k | Homogeneous bundle containing `MiniMax-M3` configurations; medium context length. Supports text, image, and video input. | <details><summary>View</summary><ul><li>`MiniMax-M3`<ul><li>Seq Length: 8K-32K, BS: 1, 2, 4</li></ul></li></ul></details> |
| `MiniMax-M3` | minimax-m3-512k | <ul><li>Homogeneous bundle containing `MiniMax-M3` configurations.</li><li>Up to 512K context; text, image, and video input at all sequence lengths.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M3`<ul><li>Seq Length: 8K-32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4</li><li>Seq Length: 128K, BS: 1, 2, 4</li><li>Seq Length: 256K, BS: 1, 2, 4</li><li>Seq Length: 512K, BS: 1</li></ul></li></ul></details> |
| `MiniMax-M3` | minimax-m3-32-64-128-256-512k-1m | <ul><li>Homogeneous bundle containing `MiniMax-M3` configurations.</li><li>Up to 1M context; text, image, and video input at all sequence lengths.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M3`<ul><li>Seq Length: 8K-32K, BS: 1, 2, 4</li><li>Seq Length: 64K, BS: 1, 2, 4</li><li>Seq Length: 128K, BS: 1, 2, 4</li><li>Seq Length: 256K, BS: 1, 2, 4</li><li>Seq Length: 512K, BS: 1</li><li>Seq Length: 1M, BS: 1</li></ul></li></ul></details> |
| `MiniMax-M3` | `minimax-m3-acb-prefix-caching` | <ul><li>Homogeneous bundle for `MiniMax-M3` with prompt caching, continuous batching, and constrained decoding. Up to 1M context.</li><li>Text, image, and video input. Requests that contain an image or video, including in an earlier turn, are not cached.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M3`<ul><li>Seq Length: 1M, BS: 1</li></ul></li></ul></details> |
| `MiniMax-M3` | `minimax-m3-32k-acb-prefix-caching` | <ul><li>Same as `minimax-m3-acb-prefix-caching`, with context capped at 32K.</li><li>Text, image, and video input. Requests that contain an image or video, including in an earlier turn, are not cached.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M3`<ul><li>Seq Length: 32K, BS: 1</li></ul></li></ul></details> |
| `MiniMax-M3` | `minimax-m3-256k-acb-prefix-caching` | <ul><li>Same as `minimax-m3-acb-prefix-caching`, with context capped at 256K.</li><li>Text, image, and video input. Requests that contain an image or video, including in an earlier turn, are not cached.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M3`<ul><li>Seq Length: 256K, BS: 1</li></ul></li></ul></details> |
| `MiniMax-M3` | `minimax-m3-512k-acb-prefix-caching` | <ul><li>Same as `minimax-m3-acb-prefix-caching`, with context capped at 512K.</li><li>Text, image, and video input. Requests that contain an image or video, including in an earlier turn, are not cached.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M3`<ul><li>Seq Length: 512K, BS: 1</li></ul></li></ul></details> |
| `MiniMax-M3` | `minimax-m3-1m-acb-prefix-caching` | <ul><li>Same as `minimax-m3-acb-prefix-caching`, with context capped at 1M.</li><li>Text, image, and video input. Requests that contain an image or video, including in an earlier turn, are not cached.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M3`<ul><li>Seq Length: 1M, BS: 1</li></ul></li></ul></details> |
| `MiniMax-M3.1-Flash`<sup>PREVIEW</sup> | `minimax-m3-1-flash-acb-prefix-caching` | <ul><li>Homogeneous bundle for `MiniMax-M3.1-Flash` with prompt caching, continuous batching, and constrained decoding. Up to 1M context.</li><li>Text, image, and video input. Requests that contain an image or video, including in an earlier turn, are not cached.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M3.1-Flash`<ul><li>Seq Length: 1M, BS: 1</li></ul></li></ul></details> |
| `MiniMax-M3.1-Flash`<sup>PREVIEW</sup> | `minimax-m3-1-flash-32k-acb-prefix-caching` | <ul><li>Same as `minimax-m3-1-flash-acb-prefix-caching`, with context capped at 32K.</li><li>Text, image, and video input. Requests that contain an image or video, including in an earlier turn, are not cached.</li></ul> | <details><summary>View</summary><ul><li>`MiniMax-M3.1-Flash`<ul><li>Seq Length: 32K, BS: 1</li></ul></li></ul></details> |
| `Mistral-Large-3-675B-Instruct-2512`<sup>PREVIEW</sup> | mistral-large-3-fp8-8k | <ul><li>Homogeneous bundle containing `Mistral-Large-3-675B-Instruct-2512` configurations.</li><li>Preview model – text-only.</li></ul> | <details><summary>View</summary><ul><li>`Mistral-Large-3-675B-Instruct-2512`<ul><li>Seq Length: 8K, BS: 1</li></ul></li></ul></details> |
| `Mistral-Large-3-675B-Instruct-2512`<sup>PREVIEW</sup> | mistral-large-3-fp8-8-16-32k | <ul><li>Homogeneous bundle containing `Mistral-Large-3-675B-Instruct-2512` configurations.</li><li>Wider context range than `mistral-large-3-fp8-8k`. Preview model – text-only.</li></ul> | <details><summary>View</summary><ul><li>`Mistral-Large-3-675B-Instruct-2512`<ul><li>Seq Length: 8K, BS: 1, 4</li><li>Seq Length: 16K, BS: 1, 2</li><li>Seq Length: 32K, BS: 1</li></ul></li></ul></details> |
| `Qwen3-235B-A22B-Instruct-2507` | dyt-qwen3-235b-32-128k | Homogeneous bundle containing `Qwen3-235B-A22B-Instruct-2507` configurations. | <details><summary>View</summary><ul><li>`Qwen3-235B-A22B-Instruct-2507`<ul><li>Seq Length: 32K, BS: 2, 4, 6, 8</li><li>Seq Length: 128K, BS: 2</li></ul></li></ul></details> |
| `Whisper-Large-v3` / `Qwen3-32B` / `E5-Mistral-7B-Instruct` | qwen3-32b-whisper-e5-mistral | <ul><li>Small to medium context length with varied batch size</li></ul> | <details><summary>View</summary><ul><li>`E5-Mistral-7B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 4, 8, 16, 32</li></ul></li><li>`Qwen3-32B`<ul><li>Seq Length: 8K, BS: 1, 4</li><li>Seq Length: 16K, BS: 1</li><li>Seq Length: 32K, BS: 1, 2</li></ul></li><li>`Whisper-Large-v3`<ul><li>BS: 1, 16, 32</li></ul></li></ul></details> |
| `Qwen3-32B` / `Meta-Llama-3.1-405B-Instruct` | qwen3-32b-llama405b-s-m | <ul><li>Small to medium context length</li><li>Speculative decoding supported for `Meta-Llama-3.1-405B-Instruct`</li></ul> | <details><summary>View</summary>**Target Models:**<ul><li>`Meta-Llama-3.1-405B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 2, 4</li><li>Seq Length: 8K, BS: 1</li><li>Seq Length: 16K, BS: 1</li></ul></li></ul>**Draft Models:**<ul><li>`Meta-Llama-3.1-8B-Instruct`<ul><li>Seq Length: 16K, BS: 1</li></ul></li><li>`Meta-Llama-3.2-3B-Instruct`<ul><li>Seq Length: 4K, BS: 1, 2, 4</li><li>Seq Length: 8K, BS: 1</li></ul></li></ul>**Routable Models:**<ul><li>`Qwen3-32B`<ul><li>Seq Length: 8K, BS: 1, 4</li><li>Seq Length: 16K, BS: 1</li><li>Seq Length: 32K, BS: 1</li></ul></li></ul></details> |
| `Qwen3-TTS-Talker`<sup>PREVIEW</sup> | qwen3-tts-talker | Homogeneous bundle containing `Qwen3-TTS-Talker` configurations. | <details><summary>View</summary><ul><li>`Qwen3-TTS-Talker`<ul><li>Seq Length: 4K, BS: 2, 4, 8, 16</li></ul></li></ul></details> |
| `Qwen3-TTS-Vocoder`<sup>PREVIEW</sup> | qwen3-tts-vocoder | Homogeneous bundle containing `Qwen3-TTS-Vocoder` configurations. | <details><summary>View</summary><ul><li>`Qwen3-TTS-Vocoder`<ul><li>Codes length: 5, 8, 10, 16, BS: 16</li></ul></li></ul></details> |

† `cd-dyt-gpt-oss-120b-8-32-64-128k` supports sequence lengths from 8K–128K, and `dyt-gpt-oss-20b-32-64-128k` supports 32K–128K. If you require shorter context lengths (4K–16K) for either model, contact your SambaNova representative.

## Model details

Full configuration for each model, grouped by provider.

### Meta

#### Meta-Llama-3.3-70B-Instruct

<details>
  <summary>
    Meta

    Meta-Llama-3.3-70B-Instruct

    Task agent, Tool-calling agent, Text to SQL/Cipher
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct)
</details>

#### Meta-Llama-3.1-8B-Instruct

<details>
  <summary>
    Meta

    Meta-Llama-3.1-8B-Instruct

    Gateway agent, Validation agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)
</details>

#### Meta-Llama-3.1-70B-Instruct

<details>
  <summary>
    Meta

    Meta-Llama-3.1-70B-Instruct

    Task agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/meta-llama/Llama-3.1-70B-Instruct)
</details>

#### Meta-Llama-3.1-405B-Instruct

<details>
  <summary>
    Meta

    Meta-Llama-3.1-405B-Instruct

    Task agent, Tool-calling agent, Code generation
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct)
</details>

#### Llama-4-Maverick-17B-128E-Instruct

<details>
  <summary>
    Meta

    Llama-4-Maverick-17B-128E-Instruct

    Image understanding, Task agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct)
</details>

### MiniMax

#### MiniMax-M3

<details>
  <summary>
    MiniMax

    MiniMax-M3

    Coding agent, Long-context agentic tasks, Image and video understanding
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/MiniMaxAI/MiniMax-M3)
</details>

#### MiniMax-M3.1-Flash

<details>
  <summary>
    MiniMax

    MiniMax-M3.1-Flash

    Coding agent, Agentic workflows, Image and video understanding
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash)
</details>

**Reasoning.** `MiniMax-M3.1-Flash` always reasons; reasoning cannot be turned off. It accepts the `reasoning_effort` values `low`, `medium`, `high`, `xhigh`, and `max`. The default is `max`.

#### MiniMax-M2.7

<details>
  <summary>
    MiniMax

    MiniMax-M2.7

    Coding agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/MiniMaxAI/MiniMax-M2.7)
</details>

#### MiniMax-M2.5

<details>
  <summary>
    MiniMax

    MiniMax-M2.5

    Coding agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)
</details>

### Mistral AI

#### Mistral-Large-3-675B-Instruct-2512

<details>
  <summary>
    Mistral AI

    Mistral-Large-3-675B-Instruct-2512

    Multilingual instruction following, Task agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/mistralai/Mistral-Large-3-675B-Instruct-2512)
</details>

### DeepSeek

#### DeepSeek-R1-0528

<details>
  <summary>
    DeepSeek

    DeepSeek-R1-0528

    Complex reasoning
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528)
</details>

#### DeepSeek-R1-Distill-Llama-70B

<details>
  <summary>
    DeepSeek

    DeepSeek-R1-Distill-Llama-70B

    Complex reasoning
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B)
</details>

#### DeepSeek-V3-0324

<details>
  <summary>
    DeepSeek

    DeepSeek-V3-0324

    Main/planner agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324)
</details>

#### DeepSeek-V3.1

<details>
  <summary>
    DeepSeek

    DeepSeek-V3.1

    Main/planner agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-V3.1)
</details>

#### DeepSeek-V3.2

<details>
  <summary>
    DeepSeek

    DeepSeek-V3.2

    Main/planner agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-V3.2)
</details>

#### DeepSeek-V3.1-Terminus

<details>
  <summary>
    DeepSeek

    DeepSeek-V3.1-Terminus

    Main/planner agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/deepseek-ai/DeepSeek-V3.1-Terminus)
</details>

### OpenAI

#### gpt-oss-120b

<details>
  <summary>
    OpenAI

    gpt-oss-120b

    Main/planner agent, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/openai/gpt-oss-120b)
</details>

#### gpt-oss-20b

<details>
  <summary>
    OpenAI

    gpt-oss-20b

    Main/planner agent, Tool-calling agent, Reasoning
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/openai/gpt-oss-20b)
</details>

#### Whisper-Large-v3

<details>
  <summary>
    OpenAI

    Whisper-Large-v3

    Automatic speech recognition (ASR), Audio transcription
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/openai/whisper-large-v3)
</details>

### Google

#### gemma-3-27b-it

<details>
  <summary>
    Google

    gemma-3-27b-it

    Image understanding, Task agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/google/gemma-3-27b-it)
</details>

#### gemma-3-12b-it

<details>
  <summary>
    Google

    gemma-3-12b-it

    Image understanding, Task agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/google/gemma-3-12b-it)
</details>

#### gemma-4-31B-it

<details>
  <summary>
    Google

    gemma-4-31B-it

    Image and video understanding, Task agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/google/gemma-4-31B-it)
</details>

**Thinking mode.** Thinking is off by default. To turn it on with Chat completions (`/v1/chat/completions`), set `chat_template_kwargs` to `{"enable_thinking": true}`. The reasoning is returned in the `reasoning` field, separate from the answer. The `reasoning_effort` parameter is accepted but has no effect on this model. On the Responses API, `reasoning.effort: "none"` is rejected with HTTP 400; to keep thinking off there, leave `reasoning.effort` out of the request.

**Known limitations.**

* If a tool result references an ID that matches no tool call in the conversation, the request is not rejected with a 4xx error. On `gemma-4-31b-32-128-256k` and `gemma-4-31b-mtp-cd-8-32-64-128k` it fails with HTTP 500; on `gemma-4-31b-acb-prefix-caching` it is accepted and answered. Make sure each tool result uses the ID of the tool call it answers.
* On `gemma-4-31b-32-128-256k` and `gemma-4-31b-mtp-cd-8-32-64-128k`, a base64 MP4 video can be rejected with HTTP 400 `invalid_video` even when the file is valid.

### Alibaba Cloud

#### Qwen3-235B-A22B-Instruct-2507

<details>
  <summary>
    Alibaba Cloud

    Qwen3-235B-A22B-Instruct-2507

    Agentic planner, Multilingual instruction following
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507)
</details>

#### Qwen3-32B

<details>
  <summary>
    Alibaba Cloud

    Qwen3-32B

    Task agent, Multilingual instruction following
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/Qwen/Qwen3-32B)
</details>

#### Qwen3-TTS-Talker

<details>
  <summary>
    Alibaba Cloud

    Qwen3-TTS-Talker

    Text-to-speech
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/collections/Qwen/qwen3-tts)
</details>

#### Qwen3-TTS-Vocoder

<details>
  <summary>
    Alibaba Cloud

    Qwen3-TTS-Vocoder

    Audio synthesis
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/collections/Qwen/qwen3-tts)
</details>

### Zhipu AI

#### GLM-5.3

<details>
  <summary>
    Zhipu AI

    GLM-5.3

    Long-context reasoning, Tool-calling agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/zai-org/GLM-5.3)
</details>

**Reasoning effort.** `GLM-5.3` accepts the `reasoning_effort` values `low`, `high`, and `max`. The default is `max`. Any other value, including `medium`, is served as `max`.

**Known limitations.**

* Reasoning cannot be turned off.
* The usage object does not report `completion_tokens_details.reasoning_tokens`.

### Tokyotech-llm

#### Llama-3.3-Swallow-70B-Instruct-v0.4

<details>
  <summary>
    Tokyotech-llm

    Llama-3.3-Swallow-70B-Instruct-v0.4

    Japanese instruction following, Task agent
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Supported

  [View model card on Hugging Face ↗](https://huggingface.co/tokyotech-llm/Llama-3.3-Swallow-70B-Instruct-v0.4)
</details>

### Other

#### E5-Mistral-7B-Instruct

<details>
  <summary>
    Other

    E5-Mistral-7B-Instruct

    Vector storage and retrieval (RAG)
  </summary>

  Modalities

  Deployment options

  Custom checkpoints (BYOC)
  Not supported

  [View model card on Hugging Face ↗](https://huggingface.co/intfloat/e5-mistral-7b-instruct)
</details>