parallel

The parallel module provides tools aimed at parallel computing. At this point all parallel solutions use the fork system call and are supported on limited platforms, notably excluding Windows. On unsupported platforms parallel features will disable and a warning is printed.

nutils.parallel.shempty(shape, dtype=<class 'float'>)

create uninitialized array in shared memory

nutils.parallel.shzeros(shape, dtype=<class 'float'>)

create zero-initialized array in shared memory

nutils.parallel.pariter(iterable, nprocs)

iterate in parallel

Fork into nprocs subprocesses, then yield items from iterable such that all processes receive a nonoverlapping subset of the total. It is up to the user to prepare shared memory and/or locks for inter-process communication. The following creates a data vector containing the first four quadratics:

data = shzeros(shape=[4], dtype=int)
for i in pariter(range(4), 2):
  data[i] = i**2
data

As a safety measure nested pariters are blocked by setting the global procid variable; all secundary pariters will be treated like normal serial iterators.

Parameters:
  • iterable (collections.abc.Iterable) – The collection of items to be distributed over processors
  • nprocs (int) – Maximum number of processers to use
Yields:

Items from iterable, distributed over at most nprocs processors.

nutils.parallel.parmap(func, iterable, nprocs, shape=(), dtype=<class 'float'>)

parallel equivalent to builtin map function

Produces an array of func(item) values for all items in iterable. Because of shared memory restrictions func must yield numpy arrays of predetermined shape and type.

Parameters:
  • func (callable) – Takes item from iterable, returns numpy array of shape and dtype
  • iterable (collections.abc.Iterable) – Collection of items
  • nprocs (int) – Maximum number of processers to use
  • shape (tuple) – Return shape of func, defaults to scalar
  • dtype (tuple) – Return dtype of func, defaults to float
Returns:

Return type:

Array of shape len(iterable),+shape and dtype dtype