Build with LangChain - SambaNova Integration Guide

This article walks you through using LangChain to interact with SambaNova models: from setup to instantiation, invocation, streaming, and chaining. LangChain is a software framework for developing applications powered by LLMs (large language models).

Prerequisites

Before you begin, ensure you have:

A SambaCloud account and API key. See API keys and URLs.
Python 3.10 or later. Version 1.x of langchain-sambanova requires Python 3.10 or later. On Python 3.9, pip installs the much older 0.2.0 release instead of reporting an error.

Setup

Do the following to access ChatSambaNova models:

Create a SambaCloud account and get an API key.
Run the command below to install the langchain-sambanova integration package.

pip install "langchain-sambanova>=1.1,<2"

Keep the version bounds. Release 1.0.0 removed the ChatSambaNovaCloud and ChatSambaStudio classes, so this package does ship breaking major releases. An unbounded pip install langchain-sambanova can pick up a future major that breaks the examples on this page.

Confirm which version you installed:

python -c "import langchain_sambanova; print(langchain_sambanova.__version__)"

You should see an output similar to the one below.

# Expected output
1.1.1

Credentials

Register the API key you received from cloud.sambanova.ai as an environment variable. ChatSambaNova reads SAMBANOVA_API_KEY on its own, so you never have to pass the key in code.

export SAMBANOVA_API_KEY="your-sambanova-api-key"

To set the key from inside a Python session or a notebook, prompt for it at runtime instead of hardcoding it in your source:

import getpass
import os

os.environ["SAMBANOVA_API_KEY"] = getpass.getpass("SambaNova API key: ")

SambaCloud users only need the API key. Requests default to https://api.sambanova.ai/v1. SambaStack users also set SAMBANOVA_API_BASE to their deployment URL, or pass base_url to ChatSambaNova.

Instantiation

Now you can instantiate a ChatSambaNova model object and generate chat completions, as shown in the example below. The following creates a ChatSambaNova model object configured for Meta’s Llama 3.3 70B:

from langchain_sambanova import ChatSambaNova

llm = ChatSambaNova(
    model="Meta-Llama-3.3-70B-Instruct",
    max_tokens=1024,
    temperature=0.7,
    top_p=0.01,
)

Pass any model ID that SambaCloud currently serves. The catalog changes over time, so confirm the ID against SambaCloud models or list it directly with curl https://api.sambanova.ai/v1/models.

Invocation

Pass a system prompt and a user message to the model:

messages = [
    (
        "system",
        "You are a helpful assistant that translates English to French. "
        "Translate the user sentence.",
    ),
    ("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
print(ai_msg.content)

You should see an output similar to the one below.

# Expected output
J'adore la programmation.

Streaming

To display tokens as the model generates them, call stream() instead of invoke() and print the content of each chunk. Each chunk is an AIMessageChunk, and concatenating their content values reproduces the full response.

for chunk in llm.stream(messages):
    print(chunk.content, end="", flush=True)

You should see an output similar to the one below, printed incrementally rather than all at once.

# Expected output
J'adore la programmation.

Chaining

Chain the model with a prompt template to handle dynamic language translation:

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate(
    [
        (
            "system",
            "You are a helpful assistant that translates {input_language} "
            "to {output_language}.",
        ),
        ("human", "{input}"),
    ]
)

chain = prompt | llm
response = chain.invoke(
    {
        "input_language": "English",
        "output_language": "German",
        "input": "I love programming.",
    }
)
print(response.content)

You should see an output similar to the one below.

# Expected output
Ich liebe das Programmieren.

Tool calling

Bind Python functions to the model with bind_tools. LangChain converts each function’s signature and docstring into a JSON Schema and sends it in the request’s tools field, so the docstring is what tells the model when to call it.

from langchain_core.tools import tool


@tool
def get_weather(city: str) -> str:
    """Return the current weather for a city."""
    return f"It is sunny in {city}."


llm_with_tools = llm.bind_tools([get_weather])
response = llm_with_tools.invoke("What is the weather in Palo Alto?")
print(response.tool_calls)

You should see an output similar to the one below.

# Expected output
[{'name': 'get_weather', 'args': {'city': 'Palo Alto'}, 'id': 'call_1', 'type': 'tool_call'}]

The model returns the call it wants made, not the result. Your code runs the function and passes the output back:

from langchain_core.messages import HumanMessage

messages = [HumanMessage("What is the weather in Palo Alto?")]
messages.append(response)

for call in response.tool_calls:
    result = get_weather.invoke(call)
    messages.append(result)

print(llm_with_tools.invoke(messages).content)

response.tool_calls is empty when the model answers directly instead of calling a tool, so branch on it rather than assuming a call happened.

Tool calling requires a model that supports it. See Function calling for the current list. Meta-Llama-3.3-70B-Instruct is supported; gemma-4-31B-it is not, and on an unsupported model the request returns a normal text answer with no error to indicate the tools were ignored.

Structured output

Use with_structured_output to get a validated Pydantic object instead of text. This does not use tool calling. It sends your schema as response_format with type: json_schema and parses the reply into your model, so it depends on structured output support rather than on function calling.

from typing import Literal, Optional

from pydantic import BaseModel, Field


class Ticket(BaseModel):
    """A support ticket extracted from a message."""

    title: str = Field(description="One-line summary")
    priority: Literal["low", "medium", "high"] = Field(description="Urgency")
    owner: Optional[str] = Field(default=None, description="Assignee if named")


structured_llm = llm.with_structured_output(Ticket)
ticket = structured_llm.invoke("Ada needs the quarterly review finished, it is urgent.")
print(ticket)

You should see an output similar to the one below.

# Expected output
title='Quarterly review' priority='high' owner='Ada'

The result is a real Ticket instance, so ticket.priority is typed and any value outside the Literal is rejected before your code sees it. Field descriptions are sent as part of the schema, so they are worth writing carefully.

Troubleshooting

SambaNovaError: The api_key client option must be set

The variable is not set in the environment that runs your script. Export it as shown in Credentials, or pass api_key="..." to ChatSambaNova.

AuthenticationError: Error code: 401 invalid_api_key

The key is wrong, revoked, or carries a stray quote or newline. Generate a new one at cloud.sambanova.ai/apis.

TypeError: object.__init__() takes exactly one argument with ChatSambaNovaCloud

ChatSambaNovaCloud was removed in langchain-sambanova 1.0.0 and is now an empty deprecated stub that accepts no arguments. Use ChatSambaNova instead, as shown in Instantiation.

ImportError: cannot import name 'ChatSambaNova' from 'langchain_sambanova'

You are on 0.1.x, which only shipped ChatSambaNovaCloud. Upgrade with pip install -U "langchain-sambanova>=1.1,<2".

pip installs 0.2.0 even though you asked for the latest release

You are running Python 3.9 or earlier. Version 1.x requires Python 3.10 or later. Upgrade Python, then reinstall.

SAMBANOVA_URL appears to be ignored

Version 1.x reads the base URL from SAMBANOVA_API_BASE. SAMBANOVA_URL was the 0.1.x variable and no longer has any effect.

RateLimitError: Error code: 429 rate limit exceeded

You exceeded the requests allowed for your tier. Back off and retry, or reduce concurrency. See Rate limits for the limits that apply to you.

OutputParserException: Invalid json output from with_structured_output

The reply was not valid JSON for your schema, which happens on a model without structured output support. Confirm the model on Function calling, and simplify deeply nested schemas.

response.tool_calls is empty and the model answers in prose

The model either chose not to call the tool or does not support tool calling. Check it against Function calling, and make the tool’s docstring state plainly when it should be used.

Additional resources

View the following resources for more information:

ChatSambaNova document
SambaNova-LangChain document
LangChain official document
End-to-end code examples using LangChain and SambaNova
SambaNovaEmbeddings document. Embedding models run on SambaStack, not SambaCloud. See Implement embeddings features.

Overview

Developer tools

Frameworks

Interfaces and low-code

Data and infrastructure

Multimodal

Evaluation and monitoring

LangChain integration guide

Prerequisites

Setup

Credentials

Instantiation

Invocation

Streaming

Chaining

Tool calling

Structured output

Troubleshooting

Additional resources

​Prerequisites

​Setup

​Credentials

​Instantiation

​Invocation

​Streaming

​Chaining

​Tool calling

​Structured output

​Troubleshooting

​Additional resources

Prerequisites

Setup

Credentials

Instantiation

Invocation

Streaming

Chaining

Tool calling

Structured output

Troubleshooting

Additional resources