Build with Instructor - SambaNova Integration Guide

Instructor enables structured output generation with SambaNova models. It allows large language models (LLMs) to produce responses in predefined formats–such as JSON, XML, or custom data schemas–ensuring consistency and making the output easier to parse and integrate into downstream systems. This functionality is particularly valuable for APIs, automation pipelines, and AI-driven applications that require reliable and predictable outputs.

Prerequisites

A SambaCloud account
A SambaNova API key
Python 3.10 or higher
Pydantic v2. Instructor requires pydantic>=2.8, so Pydantic v1 syntax such as class Config and .dict() does not work in these examples
A model that supports the mode you choose. See Choose a mode
Familiarity with Pydantic is helpful but not required

Instructor’s package metadata claims Python 3.9 support, but it does not work on 3.9. On 3.9 the install succeeds and instructor.from_openai() then fails with TypeError: Unable to evaluate type annotation 'str | Path', because Instructor’s own modules use | union syntax that Pydantic evaluates at runtime. macOS ships 3.9 and exposes it as python3 rather than python. Check with python3 --version and install a supported version with pyenv, uv, or brew install python@3.12 if needed.

Installation

Create and activate a virtual environment, then install Instructor:

python3 -m venv .venv
source .venv/bin/activate
python --version
pip install instructor

Instructor depends on openai and pydantic, so no extra is required. Then set your API key. The examples on this page read it from the SAMBANOVA_API_KEY environment variable.

export SAMBANOVA_API_KEY="your-sambanova-api-key"

The examples on this page were verified against instructor 1.15.4, openai 2.48.0, and pydantic 2.13.4. Instructor moved some import paths in 1.11, so if you pin an older release, check the Instructor documentation for the equivalent names.

Basic usage

The following code demonstrates how to use the SambaCloud API with Instructor to generate structured output from the Meta-Llama-3.3-70B-Instruct model. A User schema is defined using Pydantic, requiring the model to return a response with a name (string) and age (integer). Instructor handles the response validation and parsing, resulting in a structured Python object.

from openai import OpenAI
import instructor
import os
from pydantic import BaseModel

client = instructor.from_openai(
    OpenAI(
        base_url="https://api.sambanova.ai/v1",
        api_key=os.environ["SAMBANOVA_API_KEY"]
    )
)

class User(BaseModel):
    name: str
    age: int

user = client.chat.completions.create(
    model="Meta-Llama-3.3-70B-Instruct",
    messages=[
        {"role": "user", "content": "Ivan is 28"},
    ],
    response_model=User,
)
print(user)
# > User(name='Ivan', age=28)

Choose a mode

instructor.from_openai() accepts a mode argument that controls how Instructor asks the model for structured data. The example above does not pass one, so it uses the default, Mode.TOOLS. The mode you pick must match what your chosen model supports, and picking the wrong one is the most common cause of failures on this page.

Mode	How Instructor asks for the data	Model requirement
`Mode.TOOLS` (default)	Sends your Pydantic schema as a tool definition and reads the tool call back	Function calling
`Mode.JSON_SCHEMA`	Sends `response_format={"type": "json_schema", ...}` with your schema	JSON schema support
`Mode.JSON`	Sends `response_format={"type": "json_object"}` and puts your schema in a system message	JSON mode
`Mode.MD_JSON`	Asks the model in plain text to return JSON in a Markdown code block, then parses it out	Any chat model

Meta-Llama-3.3-70B-Instruct, used throughout this page, supports function calling, so the default Mode.TOOLS works without any extra configuration. For the full list of SambaCloud models that support function calling, JSON schema, and JSON mode, see Function calling and JSON mode. For the current model IDs and context lengths, see SambaCloud models. To pick a mode explicitly, import Mode and pass it:

import instructor
from instructor import Mode
from openai import OpenAI
import os

client = instructor.from_openai(
    OpenAI(
        base_url="https://api.sambanova.ai/v1",
        api_key=os.environ["SAMBANOVA_API_KEY"]
    ),
    mode=Mode.JSON,
)

If you switch to a model that does not support function calling while leaving the default Mode.TOOLS in place, the request either returns an API error or the model answers in prose instead of calling the tool. Instructor then retries until it runs out of attempts and raises InstructorRetryException. Use Mode.JSON or Mode.MD_JSON with those models instead. Of the models currently offered on SambaCloud, gemma-4-31B-it is not listed as supporting function calling.

Async usage

This code also uses SambaCloud API with Instructor to enforce structured output. The result is fetched asynchronously and printed, outputting User(name='Ivan', age=28).

from openai import AsyncOpenAI
import instructor
import asyncio
import os
from pydantic import BaseModel

client = instructor.from_openai(
    AsyncOpenAI(
        base_url="https://api.sambanova.ai/v1",
        api_key=os.environ["SAMBANOVA_API_KEY"]
    )
)

class User(BaseModel):
    name: str
    age: int

async def get_user():
    user = await client.chat.completions.create(
        model="Meta-Llama-3.3-70B-Instruct",
        messages=[
            {"role": "user", "content": "Ivan is 28"},
        ],
        response_model=User,
    )
    return user

user = asyncio.run(get_user())
print(user)
# > User(name='Ivan', age=28)

asyncio.run() fails with RuntimeError: asyncio.run() cannot be called from a running event loop inside Jupyter or any notebook, because the notebook already runs an event loop. Use user = await get_user() in a notebook cell instead.

Extract a nested schema

Real extraction tasks rarely fit a two-field model. The example below pulls a support ticket out of a raw email, and shows the parts that make Instructor reliable in production:

Field(description=...) tells the model what each field means. This is the single most effective way to raise extraction accuracy.
An Enum constrains a field to a fixed set of values.
Optional[str] with a default lets the model omit a value it cannot find, instead of inventing one.
A field_validator enforces a rule Pydantic types cannot express. When it raises, Instructor sends the error back to the model and retries.

import instructor
import os
from enum import Enum
from openai import OpenAI
from pydantic import BaseModel, Field, field_validator
from typing import Optional

client = instructor.from_openai(
    OpenAI(
        base_url="https://api.sambanova.ai/v1",
        api_key=os.environ["SAMBANOVA_API_KEY"]
    )
)

class Priority(str, Enum):
    low = "low"
    medium = "medium"
    high = "high"

class Customer(BaseModel):
    name: str = Field(description="Full name of the person who wrote the message.")
    email: Optional[str] = Field(
        default=None, description="Email address, or null if not stated."
    )

class SupportTicket(BaseModel):
    customer: Customer
    priority: Priority = Field(description="How urgent the request is.")
    summary: str = Field(description="One sentence describing the problem.")
    tags: list[str] = Field(
        default_factory=list, description="Up to three short topic tags."
    )

    @field_validator("tags")
    @classmethod
    def at_most_three_tags(cls, value: list[str]) -> list[str]:
        if len(value) > 3:
            raise ValueError("Return at most three tags.")
        return value

email = """
From: Dana Whitfield <dana.w@example.com>
Our production dashboard has been down since 3am and the whole team is blocked.
"""

ticket = client.chat.completions.create(
    model="Meta-Llama-3.3-70B-Instruct",
    messages=[{"role": "user", "content": email}],
    response_model=SupportTicket,
    max_retries=3,
)
print(ticket.model_dump_json(indent=2))

Handle validation failures

When the model returns something that does not satisfy your schema, Instructor does not hand you the broken output. It appends the validation error to the conversation and asks the model again, up to max_retries times. max_retries defaults to 3 on client.chat.completions.create(). If every attempt fails, Instructor raises InstructorRetryException. Catch it to inspect what the model actually produced instead of guessing:

from instructor.core import InstructorRetryException

try:
    ticket = client.chat.completions.create(
        model="Meta-Llama-3.3-70B-Instruct",
        messages=[{"role": "user", "content": email}],
        response_model=SupportTicket,
        max_retries=3,
    )
except InstructorRetryException as exc:
    print(f"Gave up after {exc.n_attempts} attempts")
    print(f"Total token usage: {exc.total_usage}")
    for attempt in exc.failed_attempts:
        print(f"Attempt {attempt.attempt_number} failed: {attempt.exception}")
    print(f"Last raw completion: {exc.last_completion}")

Import exceptions from instructor.core. The older instructor.exceptions path still works but emits a DeprecationWarning as of Instructor 1.11.

When retries do not converge, work through these in order:

Add or sharpen Field(description=...) on the fields that failed. Ambiguous field names are the most common cause.
Loosen the schema. Make a field Optional if the source text genuinely may not contain it. Forcing the model to fill a field it cannot find guarantees retries.
Raise max_retries. Useful when failures are intermittent rather than structural.
Switch models. A larger model handles deeply nested schemas more reliably. See SambaCloud models.
Switch modes. If the model does not support function calling, Mode.TOOLS never succeeds no matter how many times it retries. See Choose a mode.

Next steps

Explore an example notebook explaining how to create an email classification tool.
Compare Instructor against the native SambaCloud approach in Function calling and JSON mode.
Review the available model IDs in SambaCloud models.

Troubleshooting

InstructorRetryException: max retries exceeded

The model’s output failed Pydantic validation on every attempt. Instructor retries automatically (max_retries defaults to 3) and raises this once the final attempt fails, so the underlying validation error appears further down the traceback. Add Field(description=...) to your schema so the model knows what each field expects, raise max_retries, or try a more capable model. See Handle validation failures for how to inspect each failed attempt.

The model answers in prose instead of returning an object

The mode does not match the model. instructor.from_openai() defaults to Mode.TOOLS, which requires the model to support function calling. If it does not, the model replies conversationally, validation fails every time, and you get InstructorRetryException. Switch to Mode.JSON or Mode.MD_JSON, or choose a model that supports function calling. See Choose a mode.

KeyError: SAMBANOVA_API_KEY

Export the key before running: export SAMBANOVA_API_KEY="your-key". Verify with echo $SAMBANOVA_API_KEY.

ModuleNotFoundError: No module named 'instructor'

Install with: pip install instructor. If it was already installed, the virtual environment is not active: run source .venv/bin/activate and confirm the prompt shows (.venv).

TypeError: Unable to evaluate type annotation 'str | Path'

You are on Python 3.9. Instructor installs on 3.9, but its own modules use | union syntax that requires 3.10, and Pydantic evaluates those annotations at runtime. import instructor succeeds, so the failure appears at instructor.from_openai() rather than at install time. Recreate the environment with Python 3.10 or higher.

ValidationError: Input should be 'low', 'medium' or 'high'

The model returned a value outside your Enum. Add a Field(description=...) that spells out the allowed values and when to use each one. Instructor retries automatically, so this only surfaces as an error if every attempt fails.

Overview

Developer tools

Frameworks

Interfaces and low-code

Data and infrastructure

Multimodal

Evaluation and monitoring

Instructor integration guide

Prerequisites

Installation

Basic usage

Choose a mode

Async usage

Extract a nested schema

Handle validation failures

Next steps

Troubleshooting

​Prerequisites

​Installation

​Basic usage

​Choose a mode

​Async usage

​Extract a nested schema

​Handle validation failures

​Next steps

​Troubleshooting

Prerequisites

Installation

Basic usage

Choose a mode

Async usage

Extract a nested schema

Handle validation failures

Next steps

Troubleshooting