This example assumes you've started a cluster of N engines (4 in this example) as part of an MPI world.
For the simplest possible way to start 4 engines that belong to the same MPI world, you can run this in a terminal or antoher notebook:
ipcluster start --engines=MPI -n 4
Note: to run the above in a notebook, use a new notebook and prepend the command with
!, but do not run
it in this notebook, as this command will block until you shut down the cluster. To stop the cluster, use
the 'Interrupt' button on the left, which is the equivalent of sending
Ctrl-C to the kernel.
Once the cluster is running, we can connect to it and open a view into it:
from IPython.parallel import Client c = Client() view = c[:]
Let's define a simple function that gets the MPI rank from each engine.
@view.remote(block=True) def mpi_rank(): from mpi4py import MPI comm = MPI.COMM_WORLD return comm.Get_rank()
[3, 0, 2, 1]
For interactive convenience, we load the parallel magic extensions and make this view the active one for the automatic parallelism magics.
This is not necessary and in production codes likely won't be used, as the engines will load their own MPI codes separately. But it makes it easy to illustrate everything from within a single notebook here.
%load_ext parallelmagic view.activate()
Use the autopx magic to make the rest of this cell execute on the engines instead of locally
view.block = True
With autopx enabled, the next cell will actually execute entirely on each engine:
from mpi4py import MPI comm = MPI.COMM_WORLD size = comm.Get_size() rank = comm.Get_rank() if rank == 0: data = [(i+1)**2 for i in range(size)] else: data = None data = comm.scatter(data, root=0) assert data == (rank+1)**2, 'data=%s, rank=%s' % (data, rank)
Though the assertion at the end of the previous block validated the code, we can now
pull the 'data' variable from all the nodes for local inspection.
First, don't forget to toggle off
autopx mode so code runs again in the notebook:
[16, 1, 9, 4]