Chapter 4: Message Passing Parallel Patternlets

As we introduced in Chapter 1, Parallel programs contain patterns: code that recurs over and over again in solutions to many problems. The following examples show very simple examples of small portions of these patterns that can be combined to solve a problem. These C code examples use the Message Passing Interface (MPI) library, which is suitable for use on either a single multiprocessor machine or a cluster of machines.

In Chapter 1 we introduced these most common Strategy Patterns:

And these councurrent hoardware patterns:

In these diagrams, the patterns with a gray background are found exclusively in MPI. The white boxes are patterns that are found in both, as are the boxes that are half orange and half gray. We will demonstrate examples of those patterns that pertain to MPI in this chapter. Note that in MPI, the basic unit of execution is a process, so inherent in every MPI example is the process creation and destruction pattern found in the Concurrent Hardware Patterns under Execution Units in the diagram above. In addition, the CPU architecture used by MPI is MIMD.

All of the code examples in this chapter were originally written by Joel Adams of Calvin University.

If you are working on these for the first time, you may want to visit them in order. If you are returning to review a particular patternlet or the pattern category from the diagrams, you can go to that category linked below.

• Program Structure: SPMD, Conductor-Worker, and Parallel Loops
  • 00. Single Program, Multiple Data
  • 01. The Conductor-Worker Implementation Strategy Pattern
  • 02. Data Decomposition: on equal-sized chunks using the parallel for loop pattern
  • 03. Data Decomposition: on chunks of size 1 using parallel for loop
• Coordination Patterns in MPI: Basic message passing
  • 04. Message passing deadlock, using Send-Receive of a single value
  • 05. Message passing 1, using Send-Receive of a single value
  • 06. Message passing 2, using Send-Receive of an array of values
  • 07. Message passing 3, using Send-Receive with conductor-worker pattern
• Collective Communication Patterns in MPI: Broadcast
  • 08. Broadcast: a special form of message passing
  • 09. Broadcast: incorporating user input
  • 10. Broadcast: send receive equivalent
  • 11. Broadcast: send data array to all processes
• Collective Communication Patterns
  • 12. Collective Communication: Reduction on individual value
  • 13. Collective Communication: Reduction on array of values
  • 14. Collective communication: Scatter for data decomposition
  • 15. Collective communication: Gather for message-passing data decomposition
• Barrier Synchronization and Timing MPI Code
  • 16. The Barrier Synchronization Pattern
  • 17. Timing code using the Barrier Coordination Pattern
  • 18. Timing code using the Reduction pattern

You have attempted of activities on this page