This is one of the 100 recipes of the IPython Cookbook, the definitive guide to high-performance scientific computing and data science in Python.
For this recipe, you need a MPI installation and the mpi4py package.
!ipython profile create --parallel --profile=mpi
~/.ipython/profile_mpi/ipcluster_config.py
and add the line c.IPClusterEngines.engine_launcher_class = 'MPI'
.ipcluster start -n 4 --engines MPI --profile=mpi
in a terminal.import numpy as np
from IPython.parallel import Client
c = Client(profile='mpi')
view = c[:]
view.scatter('a', np.arange(16., dtype='float'))
allreduce
function. Every node makes the same computation and returns the same result.%%px
from mpi4py import MPI
import numpy as np
print(MPI.COMM_WORLD.allreduce(np.sum(a), op=MPI.SUM))
You'll find all the explanations, figures, references, and much more in the book (to be released later this summer).
IPython Cookbook, by Cyrille Rossant, Packt Publishing, 2014 (500 pages).