This is a continuation of the previous post. This time I wanted to test direct Infiniband connection on Linux.
Setup is the same:
- Two retired developer's desktops (built in 2008): AMD Opteron 2216 @2.4MHz, 8G DDR2.
- A pair of Mellanox Infinihost III adapter MHGA28-XTC
- CentOS Linux 6.3 (Minimal Install in my case)
- MLNX_OFED_LINUX-1.5.3-4.0.42-rhel6.3-x86_64.iso OFED driver (Still available on mellanox.com)
Linux setup is pretty straightforward but in my opinion more involved than on Windows. Main problem was old age of these cars. In order to avoid rebuilding OFED drivers for these cards I used old version of CentOS (6.3). I've tried 2.x version but got MFE_OLD_DEVICE_TYPE error. Besides I wanted to test SDP in Java 7 and this protocol seems to be no longer available in OFED 2.x +.
Bottom line: for these old Infinihost III-family cards use older OFED driver (1.5.3). If you don't want to rebuild the driver, use Linux distro/version specified by the driver (there are quite a few).
I found that the following two resources most useful for this project: A and B. There is no reason to repeat these steps here. Connection verification and testing using OFED utilities is similar to Windows version.
Configuring simple Java Socket application to use SDP worked like a charm. See Oracle's tutorial.
