Switch ConnectX mode from Infiniband to Ethernet

Posted on

New server which has a Mellanox/nVidia ConnectX-6 NIC. After installing Ubuntu 24.04.1 on the server the NIC was per default set into Infiniband mode:

lspci | grep Mellanox
01:00.0 Infiniband controller: Mellanox Technologies MT28908 Family [ConnectX-6]

To make it be usable as a Ethernet adapter I had to do the following steps.

First install mstflint, which gives us access to a config tool:

apt install mstflint

Then configure the NIC to enter Ethernet mode:

mstconfig -d 01:00.0 s LINK_TYPE_P1=ETH

Device #1:
----------

Device type:        ConnectX6
Name:               MCX653105A-ECA_Ax
Description:        ConnectX-6 VPI adapter card; 100Gb/s (HDR100; EDR IB and 100GbE); single-port QSFP56; PCIe3.0 x16; tall bracket; ROHS R6
Device:             01:00.0

Configurations:                                     Next Boot       New
        LINK_TYPE_P1                                IB(1)                ETH(2)

 Apply new Configuration? (y/n) [n] : y
Applying... Done!
-I- Please reboot machine to load new configurations.

Reboot the server and now the card is identified as a Ethernet controller:

lspci | grep Mellanox
01:00.0 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]

A new device should also be shown when doing ip link, if you run ethtool -i on it you'll see it's using the proper mlx driver.