CHAPTER 2: A Simple Parallel Program

2.1 The SPMD Patternlet

The first directory we will explore is the patternlets directory. A patternlet is a small program that succinctly illustrates common patterns in parallel programming. The first patternlet we will study is Single Program, Multiple Data (SPMD). Let’s explore this application by traveling to the spmd directory under the patternlets folder:

cd patternlets/spmd
ls

Typing ls should reveal the files:

• Makefile

• spmd2.c

Let’s start by examining spmd2.c, a C program that uses OpenMP pragmas to make it easy to run a portion of the program on multiple threads:

/* spmd2.c
* ... illustrates the SPMD pattern in OpenMP,
* using the commandline arguments
* to control the number of threads.
*
* Joel Adams, Calvin College, November 2009.
*
* Usage: ./spmd2 [numThreads]
*
* Exercise:
* - Compile & run with no commandline args
* - Rerun with different commandline args,
* until you see a problem with thread ids
* - Fix the race condition
* (if necessary, compare to 02.spmd)
*/

#include <stdio.h>
#include <omp.h>
#include <stdlib.h>

int main(int argc, char** argv) {
    int id, numThreads;

    printf("\n");
    if (argc > 1) {
        omp_set_num_threads( atoi(argv[1]) );
    }

    #pragma omp parallel
    {
        id = omp_get_thread_num();
        numThreads = omp_get_num_threads();
        printf("Hello from thread %d of %d\n", id, numThreads);
    }

    printf("\n");
    return 0;
}

The omp parallel pragma on line 30 directs that the block of code within the curly braces be run on separate threads. Prior to Line 30, the program is run serially. When line 30 executes, OpenMP generates a a team of threads (known as forking). Each thread is assigned its own id and runs separate copies of the code between the curly braces. At the end of the pragma’s scope (line 35), OpenMP combines all the threads together to a single-threaded process (known as joining). Conceptually, the process looks like the following:

The code in main() up until line 30 is run in one thread on one core; the fork that occurs between lines 31 and 35 is shown in the middle of the diagram.

The last couple of lines of code (lines 37-28) are run as a single-threaded process after all the threads have joined back together.

2.2 Running the Program

To run the code, we must first compile it. Use the make command in the spmd directory to compile the code:

make

The make command compiles the code in spmd2.c according to the commands in the Makefile, producing an executable called spmd2. To run the spmd2 program on 4 threads, use the following command:

./spmd2 4

Q-1: Try running the program a few times with 4 threads (the up-arrow key will help make this easier). Something should be amiss. Can you figure it out?

• Everything looks ok. No errors here!
• Not quite. Try running the program a few more times. What do you see?
• The threads are printing in a different order. This should not happen.
• This is actually expected behavior. There is no guarantee when a thread will finish (see the diagram above)
• Sometimes the same thread id appears more than once (e.g. multiple 0s). This should not happen.
• Correct. Each thread is given its own unique id; we should not see multiple copies of the same id showing up. Something is seriously wrong with our program!

2.3 Race Conditions

The following video will help you understand what is going on:

Try and answer the following question:

Q-3: What is a race condition?

• It is the smallest set of instructions that must execute sequentailly to ensure correctness.
• Not quite.
• It is a mechanism that helps protect a resource.
• Nope. Consider watching the video again!
• It is something that arises when two or more threads attempt to modify a shared variable
• Correct. Race conditions occur when multiple threads attempt to update the same memory location.

2.4 Fixing the code

In this case, the race condition can be avoided by ensuring that each thread has its own private copy of the id and numThreads variables.

Edit the spmd2.c file in the spmd directory (the full path is shown below):

CSinParallel/RaspberryPiBasics/patternlets/spmd/spmd2.c

The steps to fix the code are:

1. comment out line 23 by adding a // at the very beginning of the line

2. fix lines 32 and 33 to be full variable declarations (see listing below)

Your changed code should look like this:

/* spmd2.c
* ... illustrates the SPMD pattern in OpenMP,
* using the commandline arguments
* to control the number of threads.
*
* Joel Adams, Calvin College, November 2009.
*
* Usage: ./spmd2 [numThreads]
*
* Exercise:
* - Compile & run with no commandline args
* - Rerun with different commandline args,
* until you see a problem with thread ids
* - Fix the race condition
* (if necessary, compare to 02.spmd)
*/

#include <stdio.h>
#include <omp.h>
#include <stdlib.h>

int main(int argc, char** argv) {
    //int id, numThreads;

    printf("\n");
    if (argc > 1) {
        omp_set_num_threads( atoi(argv[1]) );
    }

    #pragma omp parallel
    {
        int id = omp_get_thread_num();
        int numThreads = omp_get_num_threads();
        printf("Hello from thread %d of %d\n", id, numThreads);
    }

    printf("\n");
    return 0;
}

You can recompile and re-run the code by re-using the make command and invoking the spmd2 executable with 4 threads:

make
./spmd2 4

Q-4: Try recompiling the code and running it a few times with 4 threads. What do you observe?

• Sometimes the same thread id appears more than once (e.g. multiple 0s).
• This should not happen, if you fixed the code correctly. Ensure that your updated code matches what is above and recompile using 'make'
• Everything looks ok (thread messages print in random order, but every thread has a unique id). No errors here!
• Great! You just fixed a race condition in the code!

You have attempted of activities on this page