P573: Basic Architecture Ideas for High Performance

Basic Architecture Ideas for High Performance

Getting good performance from codes, or just understanding bad performance when it occurs, requires some understanding of basic computer architecture. Computer architecture can be a deep subject which goes into the details of machine organization and its interaction with hardware. Fortunately, there are only three general ideas required for scientific computing, which have held throughout the history of electronic computers:

Data locality
Pipelining
Parallelism

Those concepts are the basis of all high performance architecture. Here is a rough outline of what we need:

Generalities
Memory systems
1. CPU speed vs. memory access speed
2. Memory banks
3. Memory hierarchies
Cache model and examples
Pipelining
1. Instruction pipelining
2. Loop unrolling
3. Flop and memory pipes
Enhancing data locality, pipelining in codes.

Memory banks were not covered in class, but are mentioned here because of their importance for vector machines ... which may be making a comeback.

General Concepts

A computer's architecture is a high level framework for the components making up a computer system and their interconnection. Important features for scientific computing consist of the memory system, the bus structure, internal CPU design, and I/O systems. For parallel machines this is extended to include the interconnection network (topology) for the processors. You need to understand enough architecture to

Know how to map particular algorithms to a machine.
Develop implementations of algorithms suitable for the machine.

Scientific computations involve large amounts of data. Consider climate modeling, a modern application given latitude, longitude, elevation, and time, we want to model temperature, pressure, humidity, and velocity of air as time goes on. We might also propogate chemical species since those have an effect on weather.

Discretize this by laying down a 1 km x 1 km mesh, with 11 mesh points in vertical direction. Gives ~ 5x10⁹ mesh points. Output for a single time value can require ~ 4.5 x 10¹⁰ doubles. Note that the data can be arranged in large multidimensional array.

The primary bottleneck in scientific computing is in moving data, not operating on data. We will analyze this and see how it works by using some small kernel operations operating on vectors and arrays, e.g.,

dotproduct: alpha = x^Ty
daxpy or vector update: y = y + alpha x
matrix-matrix multiplication: C = C + A*B.

These operations are more important than might appear at first view - they really account for a large fraction of machine cycles in computational science and engineering. The pseudo-code used for examples assumes:

Arrays are referenced using round brackets, e.g., v(3) or A(7,11)
Indexing starts from 1, not zero
Loops are indicated by the keyword for, and
```
            for k = 1:n
                alpha = alpha + x(k)*y(k)
            end for
   
```
means to loop over the value of k starting at k=1 and going to k=n.
Loops are given by first and last indices inclusively, unlike some languages that use similar notation to indicate indices that go from 1 to n-1. [Such languages are obscene and should be banned in polite society.]
In general the "Householder convention" will be used: lower case Greek letters for scalars, lower case Roman letters for vectors, and upper case Roman letters for matrices. Because some browsers aren't up to snuff, Greek letters will be used sparingly or spelled out. E.g., alpha = x(3)*A(2,11)

The first architectural aspect to consider is effective use of a memory hierarchy.

Next page: Memory Hierarchy

Started: 03 Jan 1996
Updated: Wed 05 Sep 2007 to fix some malapropisms
Modified: Wed 16 Sep 2009, 07:32 PM re-organization
Last Modified: Wed 16 Sep 2009, 07:33 PM