First MPI Program

First MPI Program: Hello World

The usual first program is a "hello world" one, which in C takes the stripped down form: 
            #include 
            #include 
            int main(int argc, char **argv) {
                printf("Hello world from process %d\n", myrank);
                return 0;}
Of course life cannot be so simple in the parallel programming world, so here is the first (and wrong) multiprocessor version: 
            #include <stdio.h>
            #include <stdlib.h>
            #include "mpi.h"
            int main(int argc, char **argv) {
                int myrank, p;
                MPI_Init(&argc, &argv);
                MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
                MPI_Comm_size(MPI_COMM_WORLD, &p);
                printf("Hello world from process %d\n", myrank);
                MPI_Finalize();
                return 0;}
First of all, the program has to be compiled with all the MPI libraries - a makefile will be provided for that. Secondly, it cannot be run simply by typing the executable name. For example, here is what can happened on a cluster when that is done: 
            [odin04.cs.indiana.edu:84] howdy
            [0]  Aborting program ! Could not create p4 procgroup. Possible missing file
            or program started without mpirun.
Instead, it must be run through a script, typically named mpirun which starts up a certain number of processes, all running the same program. For example, to run 4 processes of the above program, execute 
           mpirun -np 4 howdy
When the job runs, what happens is:
  1. Four processes are started, and some kind of communication method is established between them. For a network of workstations this can be opening up sockets (TCP/IP) between the processes, while for a shared memory machine it may involve setting aside some part of shared memory for buffering messages.
  2. Every process has its own data. So each of the four processes has its own copy of the variables myrank and p.
  3. Everybody gets a different value of "myrank", which will be between 0 and 3.
Side note: MPI has a standard start-up mechanism, mpi-exec. Unfortunately mpirun is now embedded in the folklore, and mpi-exec can seem arcane in comparison. So most people continue to provide/use mpirun which is a simplified wrapper for mpi-exec.

Typical output from the program looks like: 

            [odin61.cs.indiana.edu:102] mpirun -np 4 howdy
            Hello world from process 0
            Hello world from process 2
            Hello world from process 3
            Hello world from process 1
Note that the output is not necessarily in rank order - but why should anyone think that the one with rank 1 will print before the one with rank 2? The above program is actually "wrong" in that there is no guarantee of what printf() does with multiple processes. Does each get connected to the terminal? Do they each get their own file descriptor? It is possible (and does occur) that one output line will appear in the middle of another. Following is a safer way to do the howdy program:

Second version of Hello World

            #include <stdio.h>
            #include <stdlib.h>
            #include "mpi.h"
            int main(int argc, char **argv) {
                int myrank;
                int src, dest, p;
                int tag = 1;
                char msg[256];
                MPI_Status status;
            
                MPI_Init(&argc, &argv);
                MPI_Comm_rank(MPI_COMM_WORLD, &myrank);
                MPI_Comm_size(MPI_COMM_WORLD, &p);
             
                if (myrank != 0) {
                  sprintf(msg, "Hello world from process %d\n", myrank);
                  dest = 0;
                  MPI_Send(msg, strlen(msg)+1, MPI_CHAR, dest, tag, MPI_COMM_WORLD);
                }
                else {
                  printf("Hello world from process %d\n", myrank);
                  for (src = 1; src < p; src++) {
                    MPI_Recv(msg, 256, MPI_CHAR, src, tag, MPI_COMM_WORLD, &status);
                    printf("%s", msg);}
                }
                MPI_Finalize();
                return 0;}
What this program does is
  1. Each process gets its rank as before
  2. Every process except the one with rank 0 copies its greeting into a character array buffer called msg. Keep in mind that there are p different arrays called msg - one for each process.
  3. Each process other than the one with rank 0 sends a message to the one with rank 0 (by specifying dest = 0).
  4. The process with rank 0 does not bundle up such a message. Instead, it prints its own greeting, then one-by-one receives the messages from the other processes and prints them out. It receives them into its character array called msg.
This shows the basics of how to get a MIMD code from a SPMD model: run the exact same program, but there is a a branch depending on the process's rank. Now exactly one process is doing I/O, and the messages are guaranteed to be printed in rank order.

  • Next page: Using MPI for This Course
  • Class home page: B673