CHAPTER 2: Message Passing

Brief review of what message passing means and what it is used for. Talk about process and inter-process communication, and how this is the main way communication occurs in distributed memory architectures, but is general enough to also work on shared memory architectures.

Give some practical “real world” examples of inter-process communication that students may have interacted with: instant messaging, web searches, etc.

Ensure that students understand that interprocess communication can occur on a single machine or enable communication between multiple machines. Define terms such as node, core, cluster compute, peer-to-peer, and process.

Re-introduce the jigsaw puzzle analogy and discuss the distributed memory/process communication use case.

• 2.1 First Steps - Programming with Message Passing
  • The SPMD Pattern
  • The Master-Worker Pattern
• 2.2 Point to Point Communication
  • Send and Receive Hello World
  • Integration - First Attempt
  • Exercise - Populate an Array
  • Ring of passed messages
• 2.3 Deadlock
  • Deadlock Example
  • Visualizing Deadlock
  • Fixing Deadlock
• 2.4 Other Communication Patterns
  • Broadcast
  • Scatter and Gather
  • Reduction
  • Returning to the Array Example
• 2.5 Real World Example - Forest Fire Simulation
  • The parallel MPI version
  • Try some other cases to observe how it scales

You have attempted of activities on this page