The Distributed Addressing Callback

The da_address procedure computes the address of an array element.

proc da_address {type_id address indices replication} {
    # Each element lives in only one place, so we return a null
    # result if asked for other places for it.
    if {$replication != 0} {
        return ""
    }
    
    global _da_info _da_nprocs
    
    set typeinfo $_da_info($type_id)
    set bound [read_store \
                    [expr $address + [lindex $typeinfo 4]] int]
    
    set distributed_index [lindex $indices 0]
    set other_index [lindex $indices 1]
    set node [expr $distributed_index%$_da_nprocs]
    set local_index [expr $distributed_index/$_da_nprocs]
    
    set element_size [lindex $typeinfo 2]
    
    #
    # We have to work out the whole address.
    #
    set delta [expr $element_size* \
                ($other_index+$bound*$local_index)]
    
    return \
        "$node {addc [lindex $typeinfo 0]; indirect; addc $delta}"
}

The da_address routine begins by checking if TotalView is requesting extra places where this data object might live--this occurs when your program is updating a value in a distributed array. Since some elements of the array could be replicated, TotalView attempts to update them everywhere they may reside. In this example, however, the array data is not replicated. Consequently, the routine returns a null string for all replication values except the first.

For the first (and only) replicated element, da_address performs the same calculations as the Fortran local_address() and iwho() procedures. These routines:

• Compute the index into the local array and the node index for an element.
• Compute the element's full offset, including the address in the undistributed dimension.
• Return the node index and an addressing expression that allows TotalView to locate the data object.

The procedure uses the gSizeof_Element, gDa_Nprocs, and gArray_Bound global variables set up by earlier callbacks.