Astrophysics (Index)About

task-based parallelism

(breaking a computation into tasks that can be run in parallel)

Task-based parallelism (TBP) is the breaking of a computation into small pieces (tasks), then assigning individual tasks to processors (CPUs), each processor carrying out its own tasks independently. It is a technique for organizing computations for efficient handling by supercomputers that have many processors, and especially desirable for lengthy calculations, e.g., requiring days or weeks of computation: given the limited number of supercomputers available to perform scientific calculations, effective use of their many processors results in more science accomplished.

The tasks invariably require data from tasks carried out earlier in the computation (often from more than one), and produce data required by later tasks that combine (or summarize) the data. Thus the tasks of a computation or simulation are not completely free to run at any time in the computation: some ordering is necessary. The challenge is organizing the computation into tasks without too much management overhead (such management of the tasks and their data-needs also requires computation) and such that data is delivered to the processor where it is needed with minimal idle CPU awaiting that data. A computation is well suited to the technique if tasks can be conceived that are limited in calculation time (i.e., not many orders-of-magnitude difference; not including loops that vary greatly in number of iterations) and such that the data needed for each task is manageably small, i.e., from a few earlier tasks rather than all or a huge number. Given this, tractable scheduling has a good chance of keeping the CPUs busy and completing the computation efficiently. This of such value that programmers go to great lengths devising ways to organize computations appropriately and effectively.

Further reading: