SambaRack Monitoring Reference Dashboards

SambaNova provides three example Grafana dashboards, one per exporter. They are a starting point, not a supported product interface. Import them to confirm the metrics answer your questions and to see how the queries are written, then copy the panels you want into the dashboards your team already runs.

Dashboard	UID	Exporter	Answers
SambaRack - Rack Power	`sambarack-power`	PDU exporter	How much power does each rack draw?
SambaRack - Fan and Temperature	`sambarack-thermal`	XRDU exporter	How hot are the XRDUs and how fast are the fans?
SambaRack - RDU Utilization and Health	`sambarack-rdu`	SambaNova exporter	How busy are the RDUs, and are there faults or errors?

There is one dashboard per exporter because that is the boundary you install along. An exporter you did not enable leaves no dashboard full of No data panels. Each dashboard has the same rows in the same order: is it alive, what is it doing, what is wrong.

The chart deploys the dashboards by default (dashboards.enabled: true). To import them through the Grafana UI instead, get the JSON files from the chart’s dashboards/ folder:

helm pull oci://<sambanova-registry>/sambanova-hardware-monitoring --version 0.2.4 --untar

What each dashboard shows

Rack power (`sambarack-power`)

Variables: datasource, rack.

Row	Panels	What it tells you
Fleet	PDUs reachable; Total draw; Measurements unavailable; PDUs down	Whether monitoring works, and the total power. The last two stay uncolored at zero.
Power per rack	Power draw by rack; Draw by PDU; Down PDUs	Draw per rack over time, each PDU’s current draw, and which PDUs are not answering
Device (collapsed)	PDU inventory	Model, firmware, and serial of each PDU

Fan and temperature (`sambarack-thermal`)

Variables: datasource, rack, xrdu. rack is single-select: every panel shows one rack, so the per-sensor charts stay readable as the fleet grows. xrdu lists only the selected rack’s XRDUs.

Row	Panels	What it tells you
Fleet	BMCs reachable; Sensors reporting	Whether monitoring works for the rack
Temperature	Temperature by sensor	Heat over time, per sensor, for the selected rack
Fan	Fan speed by sensor	Fan speed over time, for the selected rack

RDU utilization and health (`sambarack-rdu`)

Variables: datasource, rack, node. The node list narrows to the selected racks.

Row	Panels	What it tells you
Fleet	Nodes reporting; Utilization (avg over range); New faults; Tiles offline	Whether nodes are up, how busy they were over the selected time range, and whether anything broke in the time range
Utilization	Utilization by rack	How busy each rack is, as mean tile utilization (100 - idle)
Health	Faults and errors per hour by node; Tiles offline by node	Which node changed recently
Platform (collapsed)	Platform inventory and firmware	Rack, uptime, and firmware versions per node

Panels aggregate to node and rack only; there are no per-tile or per-XRDU series. A non-zero error count is normal, so the dashboard shows errors only as a per-hour rate.

Customize the dashboards

Some panels depend on your configuration rather than on an exporter, and stay empty without it.

Panel	Needs
Power draw by rack	A `rack` label on each PDU target. The exporter cannot tell which rack a PDU is in. Without it, the panel shows one unlabeled series; the total is still correct.
Every panel on the fan and temperature dashboard	A `rack` label on each XRDU target. The `rack` variable lists nothing without it.
Every panel on the RDU dashboard filtered by rack	`sambanova-exporter.serviceMonitor.rackMap`, which supplies the `rack` label

pdu-exporter:
  targets:
    - address: <pdu-ip>
      name: <rack-id>-pdu1
      labels:
        rack: "<rack-id>"

Each dashboard’s data source is a template variable, so nothing is tied to a data source ID from another environment. The variable pre-selects a data source named Prometheus. If yours has a different name, choose it from the list at the top of the dashboard. When you build your own panels, four conventions from these dashboards are worth keeping:

Convention	Reason
Filter power on `level="total"`	The PDU reports the same watts at several levels. An unfiltered sum roughly triples the real figure. See PDU exporter labels.
Leave gap-filling (`spanNulls`) off	The exporters withhold a reading the device has lost rather than publishing it stale or as zero, so a gap is a signal. See PDU exporter behavior.
Never sum tile percentages	`sambanova_tile_exec_percent` reports one series per tile. Use the mean per node or rack. See Tile collector.
Filter out `-1`	Tile and throughput metrics report `-1.0` for a component that was not present or did not respond. Add `!= -1` to every aggregation. See Sentinel value.

Install the dashboards

Method	Use it when
Helm chart	You install the monitoring chart anyway
ConfigMaps by hand	You want dashboards without touching the chart release
Grafana UI	You want a one-off look, or Grafana has no dashboard sidecar

The first two require a Grafana dashboard sidecar. Without one, the ConfigMaps are created and nothing reads them.

Install with the Helm chart

The chart installs the dashboards by default. If you set dashboards.enabled to false, turn it back on. --reuse-values keeps your existing exporter and target configuration.

helm upgrade --install hwmon \
  oci://<sambanova-registry>/sambanova-hardware-monitoring \
  --version 0.2.4 --namespace monitoring --reuse-values \
  --set dashboards.enabled=true

In an air-gapped install, upgrade from the local chart file instead. See Install monitoring. Check that the sidecar selects on the label the chart applies:

kubectl get deploy -n monitoring -l app.kubernetes.io/name=grafana \
  -o jsonpath='{.items[*].spec.template.spec.containers[?(@.name=="grafana-sc-dashboard")].env[?(@.name=="LABEL")].value}'

kube-prometheus-stack uses grafana_dashboard, which is the chart default. If yours differs, set dashboards.sidecarLabel.key.

A wrong sidecar label is silent. The ConfigMap is created and the dashboard never appears.

To add your own dashboard alongside the bundled ones:

helm upgrade --install hwmon \
  oci://<sambanova-registry>/sambanova-hardware-monitoring \
  --version 0.2.4 -n monitoring --reuse-values \
  --set dashboards.enabled=true \
  --set-file 'dashboards.extra.my-dashboard\.json=./my-dashboard.json'

In an air-gapped install, use ./sambanova-hardware-monitoring-0.2.4.tgz in place of the oci:// chart reference. Escape the dot in the file name. Without the backslash, the schema rejects the value with got object, want string. Repeat the flag for each dashboard. Set dashboards.bundled: [] to ship only your own. An extra entry with the same file name as a bundled dashboard replaces it.

Install as ConfigMaps by hand

Create one ConfigMap per dashboard, each with the sidecar’s discovery label. Do not combine them: a ConfigMap is capped at 1 MiB, and the sidecar reloads a changed ConfigMap as a whole.

cd sambanova-hardware-monitoring/dashboards

for f in sambarack-power sambarack-thermal sambarack-rdu; do
  kubectl -n monitoring create configmap "$f" \
    --from-file="$f.json=$f.json" \
    --dry-run=client -o yaml \
  | kubectl label -f- --local --dry-run=client -o yaml \
      grafana_dashboard=1 app=sambanova-hardware-monitoring \
  | kubectl apply -f -
done

The ConfigMap key must end in .json. Grafana ignores anything else. Confirm the sidecar loaded them:

kubectl -n monitoring get cm -l grafana_dashboard=1
kubectl -n monitoring logs deploy/<grafana-deployment> -c grafana-sc-dashboard | tail -20

The sidecar logs one line per file it writes. No line means the label selector did not match. To update a dashboard, rerun the same command. To remove them, delete the three ConfigMaps.

Upload in the Grafana UI

In Grafana, go to Dashboards, then New, then Import. Upload the JSON file and choose your Prometheus data source when prompted. You do not need to edit the file.

Find the dashboards

Open Grafana at the URL your organization uses and search for sambarack. On a default kube-prometheus-stack the dashboards land in the General folder, not a SambaRack folder. The folder setting takes effect only if the sidecar runs with a folder annotation option, and the default install does not set one.

Known limits

Limit	Detail
RDU hosts are identified by the `node` label, not by hostname	The ServiceMonitor adds `node`; the exporter itself publishes no hostname. `rack` is present only where `rackMap` is set.
The bundled dashboards do not combine exporters	To build a fleet view, set `rack` on all three: `sambanova-exporter.serviceMonitor.rackMap` for RDU series, and the target `labels` for PDU and XRDU series.
Alerts are not included	See the behavior tables on PDU exporter and XRDU exporter for the signals worth alerting on.

Get started

Models

Features

Build

Resources

Reference dashboards

What each dashboard shows

Rack power (`sambarack-power`)

Fan and temperature (`sambarack-thermal`)

RDU utilization and health (`sambarack-rdu`)

Customize the dashboards

Install the dashboards

Install with the Helm chart

Install as ConfigMaps by hand

Upload in the Grafana UI

Find the dashboards

Known limits

Next steps

​What each dashboard shows

​Rack power (sambarack-power)

​Fan and temperature (sambarack-thermal)

​RDU utilization and health (sambarack-rdu)

​Customize the dashboards

​Install the dashboards

​Install with the Helm chart

​Install as ConfigMaps by hand

​Upload in the Grafana UI

​Find the dashboards

​Known limits

​Next steps

What each dashboard shows

Rack power (`sambarack-power`)

Fan and temperature (`sambarack-thermal`)

RDU utilization and health (`sambarack-rdu`)

Customize the dashboards

Install the dashboards

Install with the Helm chart

Install as ConfigMaps by hand

Upload in the Grafana UI

Find the dashboards

Known limits

Next steps