Parallelizing tasks

The compiler does not automatically parallelize code outside a loop. However, you can use tasking directives and pragmas to instruct the compiler to parallelize this type of code.

The begin_tasks directive and pragma tells the compiler to begin parallelizing a series of tasks.

The next_task directive and pragma marks the end of a task and the start of the next task.

The end_tasks directive and pragma marks the end of a series of tasks to be parallelized and prevents execution from continuing until all tasks have completed.

The sections of code delimited by these directives are referred to as a task list. Within a task list, the compiler does not check for data dependences, perform variable privatization, or perform parallelization analysis. You must manually synchronize any dependences between tasks and manually privatize data as necessary.

The forms of these directives and pragmas are shown in Table 9-8 “Forms of task parallelization directives and pragmas”.

Table 9-8 Forms of task parallelization directives and pragmas

Language Form

Fortran

C$DIR BEGIN_TASKS[(attribute-list)]

C$DIR NEXT_TASK

C$DIR END_TASKS

#pragma _CNX begin_tasks[(attribute-list)]

#pragma _CNX next_task

#pragma _CNX end_tasks

where

attribute-list can contain one of the case-insensitive attributes noted in Table 9-9 “Attributes for task parallelization”.

The optional attribute-list can contain one of the following attribute combinations, with m being an integer constant.

Table 9-9 Attributes for task parallelization

Attribute	Description
`dist`	Instructs the compiler to distribute the tasks across the currently threads, instead of spawning new threads. Use with other valid attributes to `begin_tasks` inside a `parallel`/`end_parallel` region. `begin_tasks` and `parallel`/`end_parallel` must appear inside the same function.
`ordered`	Causes the tasks to be initiated in their lexical order. That is, the first task in the sequence begins to run on its respective thread before the second and so on. In the absence of the `ordered` argument, the starting order is indeterminate. While this argument ensures an ordered starting sequence, it does not provide any synchronization between tasks, and does not guarantee any particular ending order. You can manually synchronize the tasks, if necessary, as described in Chapter 12 “Parallel synchronization”.
`max_threads =` `m`	Restricts execution of the specified loop to no more than `m` threads if specified alone or with the `threads` attribute. `m` must be an integer constant. `max_threads` `=` m is useful when you know the maximum number of threads on which your task runs efficiently. Can include any combination of thread-parallel, ordered or unordered execution.
`dist,` `ordered`	Causes ordered invocation of each task across threads, as specified in the attribute list to the `parallel` directive.
`dist, max_threads =` `m`	Causes thread-parallelism on no more than `m` existing threads.
`ordered, max_threads =` `m`	Causes ordered parallelism on no more than m threads.
`dist, ordered, max_threads =` `m`	Causes ordered thread-parallelism on no more than `m` existing threads.




	NOTE: Do not use tasking directives or pragmas unless you have verified that dependences do not exist. You may insert your own synchronization code in the code delimited by the tasking directives or pragmas. The compiler will not performs dependence checking or synchronization on the code in these regions. Synchronization is discussed in Chapter 12 “Parallel synchronization”.

Parallelizing tasks

The following Fortran example shows how to insert tasking directives into a section of code containing three tasks that can be run in parallel:

C$DIR BEGIN_TASKS

parallel task 1

C$DIR NEXT_TASK

parallel task 2

C$DIR NEXT_TASK

parallel task 3

C$DIR END_TASKS

The example above specifies thread-parallelism by default. The compiler transforms the code into a parallel loop and creates machine code equivalent to the following Fortran code:

C$DIR LOOP_PARALLEL

DO 40 I = 1,3

GOTO (10,20,30)I

10 parallel task 1

GOTO 40

20 parallel task 2

GOTO 40

30 parallel task 3

GOTO 40

40 CONTINUE

If there are more tasks than available threads, some threads execute multiple tasks. If there are more threads than tasks, some threads do not execute tasks.

In this example, the END_TASKS directive and pragma acts as a barrier. All parallel tasks must complete before the code following END_TASKS can execute.

Parallelizing tasks

The following C example illustrates how to use these directives to specify simple task-parallelization:

#pragma _CNX begin_tasks, task_private(i)
for(i=0;i<n-1;i++)
  a[i] = a[i+1] + b[i];
#pragma _CNX next_task
tsub(x,y);
#pragma _CNX next_task
for(i=0;i<500;i++)
  c[i*2] = d[i];
#pragma _CNX end_tasks

In this example, one thread executes the for loop, another thread executes the tsub(x,y) function call, and a third thread assigns the elements of the array d to every other element of c. These threads execute in parallel, but their starting and ending orders are indeterminate.

The tasks are thread-parallelized. This means that there is no room for nested parallelization within the individual parallel tasks of this example, so the forward LCD on the for I loop is inconsequential. There is no way for the loop to run but serially.

The loop induction variable i must be manually privatized here because it is used to control loops in two different tasks. If i were not private, both tasks would modify it, causing wrong answers. The task_private directive and pragma is described in detail in the section “task_private”.

Task parallelism can become even more involved, as described in Chapter 12 “Parallel synchronization”.

Parallelizing tasks

Technical documentation

» Table of Contents

» Glossary