MPI Broadcast

One interesting answer I saw on Stack Overflow.

Someone asked how he can receive the msg send by `MPI_Bcast`.

The answer is

This is a common source of confusion for people new to MPI. You don’t use MPI_Recv() to receive data sent by a broadcast; you use MPI_Bcast().

For MPI collective communications, everyone has to particpate; everyone has to call the Bcast, or the Allreduce, or what have you. (That’s why the Bcast routine has a parameter that specifies the “root”, or who is doing the sending; if only the sender called bcast, you wouldn’t need this.) Everyone calls the broadcast, including the receivers; the receviers don’t just post a receive.

The reason for this is that the collective operations can involve everyone in the communication, so that you state what you want to happen (everyone gets one processes’ data) rather than how it happens (eg, root processor loops over all other ranks and does a send), so that there is scope for optimizing the communication patterns (eg, a tree-based hierarchical communication that takes log(P) steps rather than P steps for P processes).

Good point for how to understand collective operations.