Skip to main content
Haystack is an open-source framework for building production-ready LLM pipelines. This guide shows you how to integrate SambaNova models into Haystack to get up and running quickly.

​
Prerequisites

Before you begin, ensure you have:
  • A SambaCloud account with an active API key
  • Python 3.10 or higher. The haystack-ai 2.31 release pinned below requires Python 3.10 or later, so 3.9 cannot install it.
  • A model ID from the SambaCloud model list. This guide uses Meta-Llama-3.3-70B-Instruct.
macOS ships with Python 3.9 and exposes it as python3 rather than python. Check your version with python3 --version. If it is below 3.10, install a supported version with pyenv, uv, or brew install python@3.12 before continuing.

​
Installation and setup

Follow the steps below to set up and install Haystack:
1

Create a virtual environment

Confirm the environment uses a supported version before installing:
2

Install Haystack

Haystack 3.0 removed the OpenAIGenerator component that the example notebook imports, so the version is pinned to the 2.x line. Note that OpenAIGenerator is already deprecated on 2.x and emits a warning on every run; Haystack recommends OpenAIChatGenerator for new code.The notebook’s webpage example converts HTML with HTMLToDocument, which requires trafilatura. It is not a dependency of haystack-ai and is not available through an extra, so install it alongside.
3

Set your API key

Export your key as an environment variable so your code can read it without hard-coding it:
This name matters. Both the snippet below and the example notebook read SAMBANOVA_API_KEY, so a different name produces a missing-key error.

​
Make your first call

Confirm the setup end to end before opening the notebook. OpenAIGenerator talks to SambaNova through the OpenAI-compatible endpoint, so it needs the base URL and your key. Save the following as first_call.py:
Run it with python first_call.py. A working setup prints one sentence of model output plus a deprecation warning, similar to this:
The exact sentence varies between runs because the model generates it. The FutureWarning comes from haystack-ai 2.31 itself, is expected on the pinned version, and does not affect the result. If you see no warning, you are on an earlier 2.x release, which is also fine.

​
Example use cases

  • Answer questions from a webpage — fetch a URL, convert the HTML to text with HTMLToDocument, and pipe it through an OpenAIGenerator to answer user questions grounded in that page’s content.
  • Build Retrieval-Augmented Generation (RAG) pipelines — index your documents with InMemoryBM25Retriever and let the model answer questions from the retrieved passages. No embedding model or second API key required.
  • Orchestrate modular LLM components with SambaNova models — chain Haystack components (converters, retrievers, generators, rankers) into a single pipeline and swap in any SambaCloud model ID.

​
Example notebooks

Both examples below live in a single notebook, haystack_sambanova_examples.ipynb:
  • Webpage Q&A: build a pipeline that answers questions from a webpage. This is the example that needs trafilatura.
  • RAG: implement Retrieval-Augmented Generation with Haystack and SambaCloud. It retrieves with InMemoryBM25Retriever, which is keyword-based, so the example needs no embedding model and no second provider key.
The notebook’s first cell runs !pip install haystack-ai==2.8.0. Skip that cell. You already installed a supported version in the previous section, and 2.8.0 both downgrades your environment and declares python <3.13, so it fails outright on Python 3.13 or later.Run the notebook top to bottom as well. The RAG cell calls Secret.from_env_var but does not import it, so running that cell in a fresh kernel raises NameError: name 'Secret' is not defined. Add from haystack.utils import Secret if you run it on its own.

​
Troubleshooting

macOS does not provide a python executable, only python3. Use python3 -m venv .venv to create the environment. After you activate it with source .venv/bin/activate, python works as expected inside the environment.
Haystack 3.0 or later is installed. The component was removed in 3.0, and the example notebook imports it. Reinstall with the pinned range: pip install "haystack-ai>=2.31,<3".
Expected on the 2.x line, and safe to ignore for the notebook. The warning fires on every run. For new code, Haystack recommends OpenAIChatGenerator, which also accepts string inputs.
trafilatura is missing. Haystack’s HTML converter depends on it, but it is not installed by haystack-ai and is not available through an extra, so it has to be installed alongside: pip install trafilatura.
The notebook reads os.environ["SAMBANOVA_API_KEY"]. Older versions of this guide told you to export api_key, which the notebook never reads. Export SAMBANOVA_API_KEY instead.
Your key is missing, wrong, or not exported into the shell that runs the code. Confirm it with echo $SAMBANOVA_API_KEY, and remember that export applies only to the current shell. Haystack reads the variable when the component is constructed, so restart your Python process or notebook kernel after exporting.
The model value is not a model you can call. Model IDs are case-sensitive and must match the SambaCloud model list exactly. Check the deprecations page if a model ID that used to work has stopped working.
You ran the notebook’s RAG cell without the earlier cells. That cell uses Secret but does not import it. Run the notebook from the top, or add from haystack.utils import Secret to the cell.

​
Additional resources