Communication - Message Passing
Communication is fundamental to coordinating and integrating the tasks.
Communication is needed for:
task assignment - sending tasks to processing sites
data transfer/exchange - sending data between processing sites
- often occurs before or in conjunction with data exchange
- also used to deal with maintaining integrity of shared data
elements and prevent race conditions
- a barrier is a synchronization mechanism which forces all
processes to wait for the last process