> ## Documentation Index > Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt > Use this file to discover all available pages before exploring further. # ModelProfile A ModelProfile defines *how* a model architecture can be run: which [PEFs](/docs/en/sambastack/service-administration/model-deployment/custom-resources/pef) it uses, which features it supports, and the batch sizes available at each sequence length tier. Profiles are reusable. One profile applies to every model and checkpoint that shares its architecture, including custom checkpoints for fine-tuned models. The profiles for supported architectures are provided with the SambaStack installation, so in most cases you reference an existing profile by name rather than authoring one. `ModelProfile` resources use `apiVersion: sambanova.ai/v1alpha1`. ## Structure ```yaml theme={} apiVersion: sambanova.ai/v1alpha1 kind: ModelProfile metadata: name: gemma4-31b-dyt-cd spec: model_arch: gemma4-31b features: - constrained_decoding batchingConfigs: all: 8k: batch_sizes: [2, 4, 6, 8] promptBatchingBuckets: [4096, 8192] 32k: batch_sizes: [2, 4, 6, 8] promptBatchingBuckets: [4096, 32768] 128k: batch_sizes: [2, 4, 6, 8] promptBatchingBuckets: [4096, 32768, 65536, 131072] recommended: 8k: batch_sizes: [2, 4, 6, 8] promptBatchingBuckets: [4096, 8192] 32k: batch_sizes: [2, 4, 6, 8] promptBatchingBuckets: [4096, 32768] 128k: batch_sizes: [2] promptBatchingBuckets: [4096, 32768, 65536, 131072] pefs: - gemma4-31b-ss262144-bs2-dyt-1-cd:2 - gemma4-31b-ss131072-bs8-dyt-1-cd:1 secretNames: - sambanova-artifact-reader ``` **ModelProfile Top-Level Fields** | Field | Required | Description | | ----------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `spec.pefs` | Yes | Ordered list of PEF custom resource references, each in `[:]` format. The list must be non-empty and must not contain duplicate PEF names. If the version suffix is omitted, the latest version is used. | | `spec.model_arch` | Yes | Model architecture targeted by this profile. All PEFs in a profile share the same `model_arch`, and this field makes that value visible without inspecting the individual `Pef` resources. | | `spec.features` | Yes | Features supported by this profile, derived from its PEFs. Known values are `continuous_batching`, `prompt_caching`, `speculative_decoding`, `speculative_prefill`, `constrained_decoding`, and `multi_token_prediction`. An empty list means no optional features. | | `spec.batchingConfigs` | Yes | Named batching configurations, keyed by configuration name and then by sequence length tier. See [Batching configurations](#batching-configurations). If omitted, the operator generates one from the PEF grouping. | | `spec.secretNames` | Yes | List of Kubernetes secrets required in order to download the PEFs. Must match secrets configured in your environment. | | `spec.skip_legalizer` | No | Skips legalizer validation for this profile. | | `status.batchingConfig` | N/A | Read-only. The single resolved default configuration for this profile: `recommended` if present, otherwise `all`, otherwise one generated by the operator. Published for visibility only — deployment always resolves through the bundle. | `spec.batchingConfigs` replaces the earlier `spec.defaultBatchingConfig` field. The old field held one tier map directly; the new field holds a map of *named* tier maps. A profile written against the old field needs its tier map nested under a configuration name, conventionally `all`. ## Batching configurations `spec.batchingConfigs` is a two-level map: a configuration name, then a sequence length tier, then the settings for that tier. Two configuration names are meaningful to the operator: | Name | Description | | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `all` | Every batch size the profile's PEFs support at every tier. Always present on the profiles SambaNova provides. | | `recommended` | A curated, narrower subset — typically fewer batch sizes at the longest tiers, where the widest fan-out costs the most memory. Present only when it differs from `all`. | Any other name is accepted and can be selected by name from a bundle, but the operator treats `recommended` and `all` specially when resolving the default. Common tier keys are `8k`, `16k`, `32k`, `64k`, and `128k`. PEFs below 1k use a bare integer key, for example `448`, and vocoder PEFs use a codes-length key in the form `t`, for example `10t`. ```yaml theme={} spec: batchingConfigs: all: 128k: batch_sizes: [1] promptBatchingBuckets: [4096, 32768, 131072] 32k: batch_sizes: [1, 2, 4] 8k: batch_sizes: [1, 2, 4, 8] 4k: batch_sizes: [1, 2, 4, 8] is_default: true ``` | Parameter | Required | Description | | ----------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `batch_sizes` | Yes | Batch sizes served at this tier. In a profile's own configuration, an explicit list of integers, for example `[1, 2, 4]`. In an override, this can also be the string `"*"`. See [Overriding a batching configuration](#overriding-a-batching-configuration). | | `promptBatchingBuckets` | No | Prompt lengths this tier advertises to the gateway for batching-queue routing. When left unset, the operator generates the buckets from the PEFs the tier actually references. | | `is_default` | No | If `true`, this tier is served as the `default` expert. At most one tier per configuration may set this. | | Additional keys | No | Any other key set on a tier is passed through unchanged to the expert built from it. | Including multiple batch sizes for each tier allows the inference engine to select the smallest and fastest configuration based on current workload. ## Overriding a batching configuration A profile establishes the batch sizes available at each tier, so its own configurations always list them explicitly. They never use `"*"`. You override a profile's batching for a single model through `modelConfigs[].batchingConfig` in a ModelBundle, or through the inline `spec.models` of a ModelDeployment. An override takes one of two forms. **Select one of the profile's configurations by name.** Set `batchingConfig` to a string naming an entry in the profile's `batchingConfigs`: ```yaml theme={} spec: modelConfigs: - model: gemma-4-31b-it:1 profile: gemma4-31b-dyt-cd batchingConfig: all # use the profile's full fan-out instead of its recommended subset ``` Naming a configuration the profile does not define is a validation error on the owning ModelBundle or ModelDeployment. **Or spell out the tiers inline.** Only in an inline override can `batch_sizes` be set to `"*"`, which selects every batch size the profile provides for that tier: ```yaml theme={} spec: modelConfigs: - model: gpt-oss-120b:1 profile: gpt-oss-fp8-dyt batchingConfig: 128k: batch_sizes: "*" # every batch size the profile defines for 128k 32k: batch_sizes: [2, 4] # a subset of what the profile defines for 32k ``` An override can only narrow or restate what the profile already provides. It cannot introduce a batch size the profile does not contain. You only need to spell out what you are actually changing: any tier that omits `promptBatchingBuckets` has them backfilled from the PEFs that tier references. ### Resolution order The batching configuration used for a model is resolved in this order, first match winning: 1. The model configuration's `batchingConfig`, whether a name or an inline map. 2. The profile's `recommended` configuration, if it defines one. 3. The profile's `all` configuration. 4. A configuration generated by the operator from the PEF grouping. ## Related resources The compiled executables a profile references The checkpoints a profile is paired with Where a profile's batching configuration can be overridden How profiles relate to the other resources