6.1. Collective Communication: reduce function¶
The type of reduction of many values down to one can be done with different types of operators on the set of values computed by each process.
6.1.1. Reduce all values using sum and max¶
In this example, every process computes the square of (id+1). Then all those values are summed together and also the maximum function is applied.
Program file: 12reduction.py
Example usage:
python run.py ./12reduction.py N
Here the N signifies the number of processes to start up in mpi.
run.py executes this program within mpirun using the number of processes given.
Exercises:
Run, using N = from 1 through 8 processes.
Try replacing MPI.MAX with MPI.MIN(minimum) and/or replacing MPI.SUM with MPI.PROD (product). Then save and run the code again.
6.1.1.1. Explore the code¶
Find the place in this code where the data computed on each process is being reduced to one value. Match the prints to the output you observe when you run it.
from mpi4py import MPI
def main():
comm = MPI.COMM_WORLD
id = comm.Get_rank() #number of the process running the code
numProcesses = comm.Get_size() #total number of processes running
myHostName = MPI.Get_processor_name() #machine name running the code
square = (id+1) * (id+1)
if numProcesses > 1 :
#initiate and complete the reductions
sum = comm.reduce(square, op=MPI.SUM)
max = comm.reduce(square, op=MPI.MAX)
else :
sum = square
max = square
if id == 0: # conductor/root process will print result
print("The sum of the squares is {}".format(sum))
print("The max of the squares is {}".format(max))
########## Run the main function
main()
6.1.2. Reduction on a list of values¶
We can try reduction with lists of values, but the behavior matches Python semantics regarding lists.
Program file: 13reductionList.py
Example usage:
python run.py ./13reductionList.py N
Here the N signifies the number of processes to start up in mpi.
run.py executes this program within mpirun using the number of processes given.
Exercises:
Run, using N = from 1 through 4 processes.
Uncomment the two lines of runnable code that are commented in the main() function. Observe the new results and explain why the MPI.SUM (using the + operator underneath) behaves the way it does on lists, and what the new function called sumListByElements is doing instead.
6.1.2.1. Explore the code¶
In this code, try to explain what the function called sumListByElements does. If you are unfamiliar with the zip function, look up what it does.
from mpi4py import MPI
# Exercise: Can you explain what this function returns,
# given two lists as input?
def sumListByElements(x,y):
return [a+b for a, b in zip(x, y)]
def main():
comm = MPI.COMM_WORLD
id = comm.Get_rank() #number of the process running the code
numProcesses = comm.Get_size() #total number of processes running
myHostName = MPI.Get_processor_name() #machine name running the code
srcList = [1*id, 2*id, 3*id, 4*id, 5*id]
destListMax = comm.reduce(srcList, op=MPI.MAX)
destListSum = comm.reduce(srcList, op=MPI.SUM)
#destListSumByElement = comm.reduce(srcList, op=sumListByElements)
if id == 0: # conductor/root process will print result
print("The resulting reduce max list is {}".format(destListMax))
print("The resulting reduce sum list is {}".format(destListSum))
#print("The resulting reduce sum list is {}".format(destListSumByElement))
########## Run the main function
main()
Note
There are two ways in Python that you might want to sum a set of lists from each process: 1) concatenating the elements together, or 2) summing the element at each location from each process and placing the sum in that location in a new list. In the latter case, the new list is the same length as the original lists on each process.