I can only do n things at a time

Multiprocessing systems are systems that use multiple central processing units (CPUs) to run several tasks simultaneously.  A multiprocessing system can be symmetric, where all CPUs are of the same type, or asymmetric, where there are different types of CPUs,

Parallel processing is where an application performs several operations at the same time.  There are two varieties of parallel processing:  Single-instruction-multiple-data (SIMD) systems run a single program and distribute the individual operations among several arithmetic/logic units (ALUs), or use a pipelined ALU, which breaks operations into smaller steps and sends operations along this pipeline while other operations are being processed further along the pipeline; operations can be distributed among several pipelined ALUs.  Multiple-instruction-multiple-data (MIMD) systems run the program independently on several CPUs.  Systems that cannot do parallel processing are called single-instruction-single-data (SISD) systems.

The bottleneck in most massively parallel applications is in sharing data between processors.  There are two potential choke points:  reading from and writing to a shared database, and transmitting intermediate results among several processors.  Parallel processor collections are classified by their interconnection topology:  bus, tree, ring, star, hypercube (n-dimensional cube), or n-dimensional mesh.

Asymmetric multiprocessing systems containing commercial processors often use both CPUs and Graphics Processing Units (GPUs).  GPUs are optimized for matrix operations, which can take advantage of pipelies.