3.1. Parallel for Loop Program Structure: equal chunks

Program file: 02parallelLoopEqualChunks.py

Example usage:

python run.py ./02parallelLoopEqualChunks.py N

Here the N signifies the number of processes to start up in mpi.

run.py executes this program within mpirun using the number of processes given.

Exercises:

• Run, using these numbers of processes, N: 1, 2, 4, and 8 (i.e., vary the last argument to run.py).

• Change REPS to 16, save, rerun, varying N again.

• Explain how this pattern divides the iterations of the loop among the processes.

Q-1: Which of the following is the correct assignment of loop iterations to processes for this code, when REPS is 8 and numProcesses is 4?


• The processes are assigned chunks in order of their process number.
• Each process gets assigned to an equal-sized chunk of the repetitions in order.
• In this code, blocks, or chunks of more than one repetition are assigned consecutively.

3.1.1. Explore the code

In the code below, notice the use of the variable called REPS. This is designed to be the total amount or work, or repetitions, that the for loop is accomplishing. This particular code is designed so that if those repetitions do not divide equally by the number of processes, then the program will stop with a warning message printed by the conductor process.

Remember that because this is still also a SPMD program, all processes execute the code in the part of the if statement that evaluates to True. Each process has its own id, and we can determine how many processes there are, so we can choose where in the overall number of REPs of the loop each process will execute.

from mpi4py import MPI

def main():
    comm = MPI.COMM_WORLD
    id = comm.Get_rank()            #number of the process running the code
    numProcesses = comm.Get_size()  #total number of processes running
    myHostName = MPI.Get_processor_name()  #machine name running the code

    REPS = 8

    if ((REPS % numProcesses) == 0 and numProcesses <= REPS):
        # How much of the loop should a process work on?
        chunkSize = int(REPS / numProcesses)
        start = id * chunkSize
        stop = start + chunkSize

        # do the work within the range set aside for this process
        for i in range(start, stop):
            print("On {}: Process {} is performing iteration {}"\
            .format(myHostName, id, i))

    else:
        # cannot break into equal chunks; one process reports the error
        if id == 0 :
            print("Please run with number of processes divisible by \
and less than or equal to {}.".format(REPS))

########## Run the main function
main()

You have attempted of activities on this page