SambaStack Pef Custom Resource - SambaNova Documentation

A PEF (Processor Executable Format) is the compiled artifact for one model configuration. A Pef resource registers one such executable together with the metadata describing what it supports, including its model architecture, batch size, and maximum sequence length. A ModelProfile selects which PEF, sequence size, and batch size are used, so in a bundle or a deployment you reference a profile rather than an individual PEF. Author Pef resources only when introducing a model architecture that SambaStack does not yet support. Pef resources use apiVersion: sambanova.ai/v1alpha1.

Structure

apiVersion: sambanova.ai/v1alpha1
kind: Pef
metadata:
  name: llama-4-maverick-ss131072-bs1
spec:
  metadata:
    model_arch: llama-4-maverick
    batch_size: 1
    max_seq_length: 131072
    max_completion_tokens: 131072
    job_type: infer
    num_rdus: 16
    rdu_arch: SN40L-16
    task_name: mllama
    continuous_batching: false
    use_context_cache: false
    constrained_decoding: false
    is_spec_prefill: false
  versions:
    "1":
      source: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/model.pef
      cached_path: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/model.pef
      ckpt_sharing_uuid: c5f61b8d9a1f41ed02b7d5d6098f6abc
      pef_status: stable
      num_tokens_at_a_time: 20
      prefill_graphs:
      - 131072
      vision_embedding_pef: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/vision.pef
      vision_embedding_pef_cached_path: gs://<SAMBASTACK_ARTIFACTS_BUCKET>/path/to/vision.pef
  owner: no-reply@sambanova.ai
  secretName: sambanova-artifact-reader

Identity and shape

Field	Required	Description
`spec.metadata.model_arch`	Yes	Architecture of this PEF, which determines the checkpoints that the PEF is compatible with.
`spec.metadata.batch_size`	Yes	Batch size that the PEF was compiled for.
`spec.metadata.max_seq_length`	Yes	Maximum sequence length that the PEF supports.
`spec.metadata.max_completion_tokens`	Yes	Maximum number of completion tokens the PEF supports. Must equal `max_seq_length`.
`spec.metadata.job_type`	No	Whether the PEF is compiled for inference or training, for example `infer`.
`spec.metadata.num_rdus`, `spec.metadata.rdu_arch`	No	Number of RDUs and the RDU architecture that the PEF was compiled for, for example `SN40L-16`.
`spec.metadata.task_name`	No	Task in the app driver this PEF uses. Defaults to the text generation task when unset.
`spec.metadata.mapping`, `spec.metadata.mixed_precision_mode`	No	Compilation settings recorded for traceability: the mapping used, and the mixed precision mode.
`spec.metadata.vocab_size`	No	Vocabulary size the PEF was compiled for.
`spec.metadata.pef_version`	No	Version of the PEF protocol this PEF was compiled against.

Feature support flags

These values determine the features list of the profiles built from this PEF.

Field	Required	Description
`spec.metadata.continuous_batching`	No	Whether the PEF supports continuous batching. Defaults to `false`.
`spec.metadata.use_context_cache`	No	Whether the PEF supports and uses prompt caching. Defaults to `false`.
`spec.metadata.constrained_decoding`	No	Whether the PEF supports constrained decoding. Defaults to `false`.
`spec.metadata.is_spec_prefill`	No	Whether the PEF can be used as a speculator for speculative prefill. Defaults to `false`.
`spec.metadata.cache_batch_size`, `spec.metadata.decode_batch_size`	No	Cache and decode batch sizes used for continuous batching.
`spec.metadata.k`	No	The `k` value used for a speculative decoding target model PEF, under the two-expert draft/target scheme.
`spec.metadata.mtp_weight_category`	No	Weight category the PEF’s fused multi-token prediction drafter was compiled with, for example `GEMMA4_MTP_WEIGHT`. Set only on single-PEF MTP PEFs, whose drafter weights are overlaid from a separate drafter checkpoint at load time. Distinct from `k`.
`spec.metadata.dynamic_dims`	No	Dynamic dimensions available in this PEF, such as `batch_size` and the sequence length dimensions.
`spec.metadata.vision_dynamic_dims`	No	Dynamic dimensions available for the PEF’s vision component, for multimodal models.

Versions

Each key under spec.versions is a version string, and its value describes one build of the PEF.

Field	Required	Description
`spec.versions.<version>.source`	Yes	Location to download this version of the PEF from.
`spec.versions.<version>.cached_path`	Yes	Path to this PEF as cached in the legalizer database. Path substitution does not affect this value.
`spec.versions.<version>.ckpt_sharing_uuid`	Yes	Identifier that defines checkpoint-sharing relationships. PEFs with the same identifier can share a checkpoint.
`spec.versions.<version>.pef_status`	Yes	Lifecycle status of this PEF version. Defaults to `preview`. See PEF and checkpoint lifecycle status.
`spec.versions.<version>.prefill_graphs`	No	Discrete prefill lengths compiled into this version. Combined with `metadata.dynamic_dims` to compute the prompt sizes the version advertises for gateway batching-queue routing, which surface as `promptBatchingBuckets` on a profile. Left unset when it cannot be derived.
`spec.versions.<version>.num_tokens_at_a_time`	No	Number of tokens generated on each call to the PEF.
`spec.versions.<version>.copy_pef`	No	Source path for the copy PEF.
`spec.versions.<version>.vision_embedding_pef`, `spec.versions.<version>.vision_embedding_pef_cached_path`	No	Source and cached paths for the companion vision embedding PEF, used by multimodal models.
`spec.versions.<version>.pef_name_override`, `spec.versions.<version>.copy_pef_name_override`, `spec.versions.<version>.vision_embedding_pef_name_override`	No	Name overrides retained for backward compatibility.

Ownership and access

Field	Required	Description
`spec.owner`	Yes	Owner of the PEF.
`spec.secretName`	Yes	Name of the single secret required in order to download the artifact.

spec.metadata.seq_lengths no longer exists. The sequence lengths a PEF serves are now derived from metadata.max_seq_length together with metadata.dynamic_dims and the version’s prefill_graphs.

PEF and checkpoint lifecycle status

SambaStack assigns a pef_status field to PEF CR versions and a checkpoint_status field to model CR checkpoint versions to indicate their support lifecycle. Understanding these statuses helps you make informed decisions when selecting PEF or checkpoint versions. PEF and checkpoint version status values Each version entry in a PEF CR includes a pef_status field. Model CR checkpoint versions use checkpoint_status. Both share the same set of values:

Status	Description
`preview`	Not fully tested or supported. May have unknown reliability or performance issues, or limited functionality (for example, partial function calling support). Not recommended for production workloads.
`stable`	Fully supported and tested.
`deprecated`	Has known reliability or performance issues. Still available for a limited transition period (up to 3 months from the deprecation announcement) to allow migration to a stable version.
`removed`	No longer usable. The version entry is retained in the PEF CR or model CR for traceability and auditability, but the path may no longer exist, causing deployment to fail if referenced.

Example PEF CR versions with status

versions:
  '1':
    source: gs://ext-sambastack-artifacts-prod-0/path/to/pef_v1.pef
    pef_status: deprecated
  '2':
    source: gs://ext-sambastack-artifacts-prod-0/path/to/pef_v2.pef
    pef_status: stable

To check version statuses, run kubectl describe pef <pef-name> or kubectl describe model <model-name> and review the pef_status or checkpoint_status field in the Versions section.

ModelProfile

Groups feature-compatible PEFs for one model architecture

Deploying models and bundles

How PEFs, profiles, models, bundles, and deployments fit together

Getting Started

Service Administration

Hardware Administration

Reference Architecture

Resources

PEF

Structure

Identity and shape

Feature support flags

Versions

Ownership and access

PEF and checkpoint lifecycle status

ModelProfile

Deploying models and bundles

​Structure

​Identity and shape

​Feature support flags

​Versions

​Ownership and access

​PEF and checkpoint lifecycle status

​Related resources

ModelProfile

Deploying models and bundles

Structure

Identity and shape

Feature support flags

Versions

Ownership and access

PEF and checkpoint lifecycle status

Related resources