> ## Documentation Index
> Fetch the complete documentation index at: https://sambanova-systems.mintlify.site/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Device administration

This guide covers checking and updating firmware on devices within SambaRack. Devices are individual components attached to racks and nodes, including BMCs, switches, PDUs, and serial terminals.

## Commands

### Check firmware version

Check firmware versions on devices:

```bash theme={null}
snctl device firmware-check \
  (--rack-id <rack_id> | --group <group_name>) \
  --type <device_type> \
  [--index <index>]
```

**Target selection:**

One of `--rack-id` or `--group` is required (mutually exclusive):

| Parameter              | Description                               |
| ---------------------- | ----------------------------------------- |
| `--rack-id <rack_id>`  | Check devices in a specific rack          |
| `--group <group_name>` | Check devices in all racks within a group |

**Device types:**

| Type            | Description                     |
| --------------- | ------------------------------- |
| `access_switch` | Access switch                   |
| `data_switch`   | Data switch                     |
| `scs`           | System Control Switch           |
| `bios`          | BIOS                            |
| `bmc`           | Baseboard Management Controller |
| `card`          | Card/Adapter                    |
| `pdu`           | Power Distribution Unit         |
| `xrdu`          | XRDU                            |
| `all`           | All device types                |

#### Single rack

**Example – check BMC firmware on a single rack:**

```bash theme={null}
snctl device firmware-check --rack-id 001 --type bmc
```

**Example output:**

```text theme={null}
             Firmware Version Information
┏━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Rack     ┃ Component ┃ Status ┃ CURR     ┃ POR      ┃
┡━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ Rack-001 │ BMC       │ PASS   │ 12.61.21 │ 12.61.21 │
└──────────┴───────────┴────────┴──────────┴──────────┘
```

**Example – check all firmware on a rack:**

```bash theme={null}
snctl device firmware-check --rack-id 001 --type all
```

**Example output:**

```text theme={null}
                                        Firmware Version Information
┏━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Rack     ┃ Component     ┃ Status ┃ CURR                        ┃ POR         ┃ OBMC_CURR ┃ OBMC_POR ┃ RDUC_CURR ┃ RDUC_POR ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━┩
│ Rack-001 │ ACCESS_SWITCH │ FAIL   │                             │             │           │          │           │          │
│ Rack-001 │ BIOS          │ PASS   │ R24_P1                      │ R24_P1      │           │          │           │          │
│ Rack-001 │ BMC           │ PASS   │ 12.61.21                    │ 12.61.21    │           │          │           │          │
│ Rack-001 │ CARD          │ PASS   │ snhni0 -> 228.1.111.0  PASS │ 228.1.111.0 │           │          │           │          │
│          │               │        │ snni0 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni2 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni4 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni6 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni8 -> 228.1.111.0  PASS  │             │           │          │           │          │
│          │               │        │ snni10 -> 228.1.111.0  PASS │             │           │          │           │          │
│          │               │        │ snni12 -> 228.1.111.0  PASS │             │           │          │           │          │
│          │               │        │ snni14 -> 228.1.111.0  PASS │             │           │          │           │          │
│ Rack-001 │ DATA_SWITCH   │ FAIL   │                             │             │           │          │           │          │
│ Rack-001 │ PDU_PDU_1     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ PDU_PDU_2     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ PDU_PDU_3     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ PDU_PDU_4     │ FAIL   │ 5.10.8                      │ 6.3.0       │           │          │           │          │
│ Rack-001 │ SCS           │ FAIL   │ 9.4.0.1R1                   │ 9.5.0.0R15  │           │          │           │          │
│ Rack-001 │ XRDU_XRDU_0   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_1   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_2   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_3   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_4   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_5   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_6   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
│ Rack-001 │ XRDU_XRDU_7   │ PASS   │                             │             │ 5.1.4-0   │ 5.1.4-0  │ 5.1.4     │ 5.1.4    │
└──────────┴───────────────┴────────┴─────────────────────────────┴─────────────┴───────────┴──────────┴───────────┴──────────┘
```

#### Group

**Example – check BMC firmware across all racks in a group:**

```bash theme={null}
snctl device firmware-check --group g1 --type bmc
```

**Example output** (one row per rack):

```text theme={null}
             Firmware Version Information
┏━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┓
┃ Rack     ┃ Component ┃ Status ┃ CURR     ┃ POR      ┃
┡━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━┩
│ Rack-001 │ BMC       │ PASS   │ 12.61.21 │ 12.61.21 │
│ Rack-002 │ BMC       │ FAIL   │ 12.85.05 │ 12.61.21 │
└──────────┴───────────┴────────┴──────────┴──────────┘
```

<Note>
  If devices are unreachable or configured incorrectly, the firmware check fails for those devices with a `FAIL` status, as shown in the example output above. For card devices, the `CURR` column lists each network interface with its firmware version and a per-interface `PASS`/`FAIL` comparison. The overall `CARD` status is `PASS` only when every interface matches the POR version.
</Note>

### Update firmware

Update firmware on devices within a rack or group:

```bash theme={null}
snctl device firmware-update \
  (--rack-id <rack_id> | --group <group_name>) \
  --type <device_type> \
  [--index <index_or_list>] \
  [--file <path> ...] \
  [--force]
```

**Target selection:**

One of `--rack-id` or `--group` is required (mutually exclusive):

| Parameter              | Description                                |
| ---------------------- | ------------------------------------------ |
| `--rack-id <rack_id>`  | Update devices in a specific rack          |
| `--group <group_name>` | Update devices in all racks within a group |

**Device types:**

| Type   | Description                     |
| ------ | ------------------------------- |
| `bios` | BIOS                            |
| `bmc`  | Baseboard Management Controller |
| `card` | Card/Adapter                    |
| `xrdu` | XRDU                            |

<Note>
  Other device types (`access_switch`, `data_switch`, `pdu`, `scs`) are not currently supported for firmware update and will be rejected by the CLI.
</Note>

**Optional parameters:**

| Parameter                 | Description                                                                                                                                                                                                                                                                                                                                                    |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--index <index_or_list>` | Device index to update. For XRDU: omit to target all XRDUs, pass a single index (0–7), or a comma-separated list (e.g. `0,2,5`). Multiple indices run in parallel within the rack.                                                                                                                                                                             |
| `--file <path>`           | Local firmware file. For XRDU, pass one combined archive whose name starts with `xrdu` (e.g. `xrdu-combined-5.1.4.tgz`), or two split archives whose names start with `obmc` and `rduc` respectively. Repeat the flag for two files: `--file obmc-5.1.4.tgz --file rduc-5.1.4.tgz`. If omitted, the firmware is downloaded from the configured artifact store. |
| `--force`                 | Skip checksum validation                                                                                                                                                                                                                                                                                                                                       |

#### Single rack

**Example – update BMC firmware:**

```bash theme={null}
snctl device firmware-update --rack-id 001 --type bmc
```

**Example – update all XRDUs:**

```bash theme={null}
snctl device firmware-update --rack-id 001 --type xrdu
```

**Example – update specific XRDUs:**

```bash theme={null}
snctl device firmware-update --rack-id 001 --type xrdu --index 0,2,5
```

**Example – update XRDU firmware from local files:**

```bash theme={null}
snctl device firmware-update --rack-id 001 --type xrdu --index 3 \
  --file /path/to/obmc-5.1.4.tgz --file /path/to/rduc-5.1.4.tgz --force
```

#### Group

**Example – update firmware across all racks in a group:**

```bash theme={null}
snctl device firmware-update --group g1 --type bmc
```

When you use `--group`, `snctl` processes racks sequentially (one rack at a time) while running individual device updates in parallel within each rack. This bounds concurrency – for example, a group with 10 racks and 8 XRDUs each won't launch 80 simultaneous jobs.

During the update, `snctl` prints:

* Per-component `PASS`/`FAIL` lines as each job completes
* Per-rack checkpoints after each rack finishes, including `PASS`/`FAIL` counts and log file paths for any failures
* A final **Firmware Update Results** table summarizing the entire run

If one or more devices fail, the command exits non-zero and prints the stdout, stderr, and instruction output log paths for each failed job.

## Order of execution

When updating multiple components, there is no strict dependency between them. However, if upgrading more than one type in the same maintenance window, follow this order:

**THOR CARD → BIOS → BMC → XRDU**

| Device    | Pre-condition            | Post-upgrade operations     | Expected duration                                 |
| --------- | ------------------------ | --------------------------- | ------------------------------------------------- |
| THOR CARD | Host must be powered on  | Power cycle, firmware check | \~25 min (\~5 min upgrade + \~20 min power cycle) |
| BIOS      | Host must be powered on  | Power cycle, firmware check | \~25 min (\~5 min upgrade + \~20 min power cycle) |
| BMC       | None                     | Power cycle, firmware check | \~25 min (\~5 min upgrade + \~20 min power cycle) |
| XRDU      | Host must be powered off | Power on, firmware check    | \~30 min (\~20 min upgrade + \~10 min power on)   |

### 1. THOR CARD

**Pre-condition:** Host must be powered on.

```bash theme={null}
# Verify host is powered on
snctl node powerstate --node <node_id> --type host

# Power on host if needed
snctl node poweron --node <node_id> --type host
```

**Update:**

```bash theme={null}
snctl device firmware-update --rack-id <rack_id> --type card
```

**Post-upgrade:**

```bash theme={null}
snctl node powercycle --node <node_id>
snctl device firmware-check --rack-id <rack_id> --type card
```

### 2. BIOS

**Pre-condition:** Host must be powered on.

```bash theme={null}
# Verify host is powered on
snctl node powerstate --node <node_id> --type host

# Power on host if needed
snctl node poweron --node <node_id> --type host
```

**Update:**

```bash theme={null}
snctl device firmware-update --rack-id <rack_id> --type bios
```

**Post-upgrade:**

```bash theme={null}
snctl node powercycle --node <node_id> --type node
snctl device firmware-check --rack-id <rack_id> --type bios
```

### 3. BMC

**Update:**

```bash theme={null}
snctl device firmware-update --rack-id <rack_id> --type bmc
```

**Post-upgrade:**

```bash theme={null}
snctl node powercycle --node <node_id> --type node
snctl device firmware-check --rack-id <rack_id> --type bmc
```

### 4. XRDU

**Pre-condition:** Host must be powered off before updating XRDU firmware.

```bash theme={null}
# Verify host is powered off
snctl node powerstate --node <node_id> --type host

# Power off host if needed
snctl node poweroff --node <node_id> --type host
```

**Update:**

```bash theme={null}
snctl device firmware-update --rack-id <rack_id> --type xrdu
```

**Post-upgrade:**

```bash theme={null}
snctl node poweron --node <node_id> --type node
snctl device firmware-check --rack-id <rack_id> --type xrdu
```

## Output columns

The firmware check output includes the following columns:

| Column     | Description                                                                                                                       |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------- |
| Rack       | Rack identifier                                                                                                                   |
| Component  | Device type and index                                                                                                             |
| Status     | `PASS` if firmware matches expected version, `FAIL` otherwise                                                                     |
| CURR       | Current firmware version. For card devices, lists each network interface with its version and per-interface `PASS`/`FAIL` result. |
| POR        | Plan of Record (expected) firmware version                                                                                        |
| OBMC\_CURR | Current OpenBMC firmware version (XRDU only)                                                                                      |
| OBMC\_POR  | Expected OpenBMC firmware version (XRDU only)                                                                                     |
| RDUC\_CURR | Current RDUC firmware version (XRDU only)                                                                                         |
| RDUC\_POR  | Expected RDUC firmware version (XRDU only)                                                                                        |