task-based parallelism

Task-based parallelism (TBP) is the breaking of a computation into small pieces (tasks), then assigning individual tasks to processors (CPUs), each processor carrying out its own tasks independently. It is a strategy for organizing computation for efficient handling by supercomputers that have many processors, and especially desirable for lengthy calculations, e.g., requiring days or weeks of computation: given the limited number of supercomputers available to perform scientific calculations, effective use of their many processors leads to more scientific accomplishment.

Such computation tasks invariably require data from tasks carried out earlier in the computation (often from more than one), and produce data required by later tasks that combine (or summarize) the data. Thus the tasks of a computation or simulation are not completely free to run at any time during the computation: some ordering is necessary. The challenge is organizing the computation into tasks without too much management-overhead (such management of the tasks and their data-needs also requires computation) and such that data is delivered to the processor where it is needed with minimal idle CPU waiting for that data. A computation is well suited to the technique if tasks can be conceived that are limited in calculation time (i.e., not many orders-of-magnitude difference; not including loops that vary greatly in number of iterations) and such that the data needed for each task is manageably small, i.e., from a few earlier tasks rather than all or a huge number. Given this, tractable scheduling has a good chance of keeping many CPUs busy and completing the computation efficiently. This is of such value that programmers go to great lengths devising effective ways to organize computations.