Data movement functions

C function: void shmem_get (long *target, long *source, int nlong, int node)

shmem_get() copies nlong 32-bit words from the remote buffer source on node node to the local buffer target. It is an individual operation. Note that:

• This function is blocking, meaning that it will not return until the all of the data has been received into the target buffer.
• Data is not sent from the remote node atomically (i.e. if one node is modifying a region of memory while another node is shmem_get()ing it, target will wind up with some combination of old and new data).

C function: void shmem_cget (long *target, long *source, int nlong, int node, int *pcount)

This function is similar to shmem_get(), but does not wait for the remote data to arrive before returning. The advantage is that this allows multiple get operations (or a get operation and any code that does not require the use of target) to proceed concurrently. As soon as the data in target becomes valid, the value pointed to by *pcount is incremented.

shmem_cget() is generally used in conjunction with shmem_cwait() and is an individual function.

C function: void shmem_put (long *target, long *source, int nlong, int node)

shmem_put() copies nlong 32-bit words from the local buffer source to the remote buffer target on node node. It is an individual operation. Note that:

• The function returns when the operation has completed at the local node; not the remote node. The data may not yet be stored at the remote node when shmem_put() returns.
• Put operations will always complete in program order when they have the same destination node. There are no guarantees on completion order when the put operations are to different nodes.
• Data is not received at the remote node atomically with respect to other put operations (i.e. if two nodes shmem_put() different data to the same region of memory on a third node, the third node will wind up with some combination of the other nodes' message).

C function: long shmem_swap (long *target, long swap_value, int node)

shmem_swap() atomically sets the value at address target on node node to swap_value and returns the value previously stored there. This function is an individual operation.

C function: void shmem_broadcast (long *target, long *source, int nlong, int PE_root, int PE_start, int logPE_stride, int PE_size, long *pSync)

shmem_broadcast() is a collective operation which broadcasts nlong 32-bit words from the root node (PE_root) to the other nodes in the active set. Recall that the active set is defined by the start node (PE_start), the number of nodes (PE_size), and the stride (logPE_stride). The stride is specified as log, base 2, of the actual stride. The value of PE_root is relative to the active set, so to broadcast from the first node in the active set (PE_start), PE_root should be zero, not PE_start.

Note that source is not copied to target on the root node. Also, all nodes in the active set must call the function with the same arguments, including the address for the target and pSync buffers. Furthermore, only the processors in the active set should call the function.

pSync is a synchronization buffer and it must be initialized before it is first used. It should be of size _SHMEM_BCAST_SYNC_SIZE, and each element must be set to _SHMEM_SYNC_VALUE. Because it must be initialized before any node might try to access it, there should be a barrier between when it is set and when any nodes might use it for the first time:

static long pSync[_SHMEM_BCAST_SYNC_SIZE];
                                   .
                                   .
                                   .
for (i=0; i<_SHMEM_BCAST_SYNC_SIZE; i++)
  pSync[i] = _SHMEM_SYNC_VALUE;

barrier();

shmem_broadcast (..., pSync);

Here is an example use of active sets, where node 1 broadcasts a value to all of the odd nodes:

shmem_broadcast(target, source, nlong, 0, 1, 1, numnodes/2, pSync);

Go to the first, previous, next, last section, table of contents.