OpenMP, MPI and pthreads

OpenMP and MPI are complementary.  OpenMP is used to optimize performance of a program running on a single machine with multiple CPUs, i.e an SMP.  MPI uses a message passing mechanism to divide a big problem into smaller problems. Each small problem runs on a separate machine, in parallel with the other.  It then uses a MapReduce paradigm to collect the result and solve the big problem. The advantages of the MapReduce paradigm will be explained separately in another post.

To combine them to solve a big problem, first use MPI to divide the problem into parts which can be solved on a single machine. Now to solve this single-machine problem optimally, use OpenMP to efficiently allocate the work among the multiple CPUs on that machine. This will result in a multithreaded program.

Why should we use OpenMP over pthreads? Both of them use threads to divide the work, right? Well, OpenMP enables you to specify a program at a higher abstraction than pthreads. You need not explicitly write the synchronization part of the code which is the trickiest part in multi-threaded computing.

All three, OpenMP, MPI, and pthreads are portable while internally taking advantage of the underlying OS and hardware architecture.

Of course, there are cons here. MPI is extremely difficult to learn and code and involves many low level implementation details. You may not be able to entirely substitute pthreads with MPI.  MPI parallelizes only loops; any producer-consumer kind of situation has to be explicitly dealt with using pthreads.


Tags: , , , , ,

5 Responses to “OpenMP, MPI and pthreads”

  1. D. L. Kumar Says:

    Very nice info and useful information. One question, In the last paragraph ur intent is to compare MPI vs Pthreads or OpenMP vs Pthreads?

    • clustercomputing Says:

      Frankly, I’ve no idea what MPI is, it’s all bullshit that I put to interviewers. I am mainly looking at your code to see how it was done and giving my own interpretations about it. So in the last para, I think I was comparing only OpenMP and pthreads.

  2. Anusha Says:

    Oye!!!!!! Do you even know whom you are inviting to comment on such technical stuff???
    Nice to see you online 🙂

    • clustercomputing Says:

      Oye Sodabuddi in English Literature, do you know whose friend request you haven’t accepted in orkut? That’s why the invitation to the blog. The invitation to the non-technical blog follows, just searching for a name.

  3. Srikanth Venkateswaran Says:

    Obviously, somebody else had this idea before me, I just didn’t know who. Now I do; for more details, read

    Experiences in Tuning Performance of Hybrid MPI/OpenMP Applications on Quad-core Systems at

    They got 34% improvement for their chosen scientific app over a pure-MPI app, which isn’t very encouraging, considering they has infinite time, are PhDs, and the app was chosen.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: