Running Hama Over InfiniBand

Create an SDP configuration file

Each BSP peer (task) will have its own port number for communicating with others. For example, if you set set bsp.peer.port to 61000 and set bsp.tasks.maximum to 5, BSP peers will use the port range from 61000 to 61005.

To enable SDP^[1] support for BSP peers, create a configuration file like the following sample:

[edward@192.168.10.1 ~]# cat {$HAMA_HOME}/conf/sdp.conf
# Use SDP when binding to
bind 192.168.10.1 61000-61010

# Use SDP when connecting to
connect 192.168.10.0/24 61000-61010

NOTE: This configuration file must reside on every node.

Set the system property that specifies the location of the configuration file

And then, set the location of the configuration file by adding -Dcom.sun.sdp.conf to the bsp.child.java.opts property as following:

  <property>
    <name>bsp.child.java.opts</name>
    <value>-Dcom.sun.sdp.conf={$HAMA_HOME}/conf/sdp.conf -Xmx1024m</value>
  </property>

If everything is OK, you will see the following logs.

CONNECT to 192.168.10.2:61005 (socket converted to SDP protocol)
CONNECT to 192.168.10.5:61002 (socket converted to SDP protocol)
CONNECT to 192.168.10.4:61003 (socket converted to SDP protocol)
CONNECT to 192.168.10.1:61006 (socket converted to SDP protocol)
...

1. http://en.wikipedia.org/wiki/Sockets_Direct_Protocol