Parallel execution

Parallel execution

When a traditional single-threaded application program is running, the statements making up the program are executed one at a time, in sequential order. Program control structures (loops and branches) repeat statements and may branch to alternative sections of code, but the important point is that, at any given instant, only one statement and one routine is being executed. This is true even of different instances of a program being run by two users at the same time (for example, a text editor). As each process is assigned a scheduled slice of CPU time, the statements are executed in the order maintained for that invocation of the program.

Drivers, however, are part of the kernel and must be ready to run as needed at the request of many processes. A driver may receive a request to write data to a disk while waiting for a previous request to complete. The driver code must be designed specifically to respond to numerous requests without being able to create a separate executable image for each request (as a text editor does). The driver does not create a new version of itself (and its data structures) for each process, so it must anticipate and handle contention problems that result from overlapping I/O requests.

Parallel computing

Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). There are several different forms of parallel computing: bit-level-, instruction-level-, data-, and task parallelism. Parallelism has been employed for many years, mainly in high-performance computing, but interest in it has grown lately due to the physical constraints preventing frequency scaling. As power consumption by computers has become a concern in recent years, parallel computing has become the dominant paradigm in computer architecture, mainly in the form of multicore processors.

Parallel computers can be roughly classified according to the level at which the hardware supports parallelism with multi-core and multi-processor computers having multiple processing elements within a single machine, while clusters, MPPs, and grids use multiple computers to work on the same task. Specialized parallel computer architectures are sometimes used alongside traditional processors, for accelerating specific tasks.

Parallel computer programs are more difficult to write than sequential ones, because concurrency introduces several new classes of potential software bugs, of which race conditions are the most common. Communication and synchronization between the different subtasks is typically one of the greatest barriers to getting good parallel program performance. The speed-up of a program as a result of parallelization is given by Amdahl's law.