Coherent memory space

Coherent memory layout

Memory physically resides in memory blocks, with each block controlled by a single EMAC.

	NOTE: The term memory block is synonymous with memory board. There is a difference, however, in that a block is considered a logical entity and a board a physical entity. The EPAC maps logical memory blocks to physical memory boards.

Each block has four banks of memory. Coherent memory is further divided into memory lines 32bytes in size.

Memory blocks are implemented with memory DIMMs, up to 16 DIMMs per block (one block per EMAC). Each DIMM can have one or two rows of SDRAM chips, constructed with either 16-Mbit or 64-Mbit SDRAMs.

Figure 2-3 “Coherent memory space layout” illustrates the layout for the coherent memory space.

Figure 2-3 Coherent memory space layout

Addressing a byte of memory

Figure 2-4 “Conceptual layout of physical memory of a fully populated system” represents a fully populated system with 16 Gbytes of physical memory. It also shows how a byte of memory is addressed.

	NOTE: Figure 2-4 “Conceptual layout of physical memory of a fully populated system” represents only the concept of how memory is configured in the system. It does not depict the physical implementation.

The eight memory boards (MB) are at the top of the drawing. Each board has 16 DIMMS and each DIMM is loaded with memory chips on both sides. Each memory board has four banks comprised of four DIMMs in the vertical direction. Also, each board has eight rows along the horizontal direction. The top and bottom of each DIMM in the horizontal direction are part of two separate and adjacent rows. For example, Row 0 consists of the memory mounted on the bottom of each of the four DIMMs located on the bottom of the memory board in the horizontal direction. If the memory chips were 64-MBit SDRAM, each board would contain two GBytes of memory.

A pair of rows per bank (that is, rows 0 and 1) is sufficient to maintain maximum memory bandwidth. Additional rows add memory capacity, not additional bandwidth.

Figure 2-4 Conceptual layout of physical memory of a fully populated system

As shown in the physical address in Figure 2-2 “Coherent memory space address formats” and the conceptual memory layout in Figure 2-4 “Conceptual layout of physical memory of a fully populated system”, a byte of memory is accessed as follows:

Each row contains 512 Mbytes of physical memory with 16-Mbit SDRAMs or 2 Gbyte with 64-Mbit SDRAMs. Within a row, 32 subpartitions exist, one for each memory board-bank combination (eight memory boards with four banks per board). Each subpartition is 64 Mbytes in size. If 16-Mbit SDRAMs are installed into a row of memory, then only the first 16 Mbytes of each subpartition of a row is accessible. Otherwise, with 64-Mbit SDRAMs, the entire 64 Mbyte subpartition is accessible.

Memory interleaving

Memory interleaving distributes consecutive lines of memory across as many banks as possible. It is supported across four, eight, 16, and 32 banks of memory as follows:

As noted earlier, the term memory block is synonymous with memory board. When generating the interleave, the EPAC maps logical memory blocks to physical memory boards. Therefore, describing memory interleave should be done in terms of memory blocks.

Normal memory interleave supports only pairs of memory blocks. Three pairs of memory blocks are interleaved with 16-way interleave.

Noninterleaved memory accesses are different from interleaved. They are divided into eight rows.

Memory interleave generation

Coherent interleave is performed on all memory references, except in the single memory block mode. To optimize memory bandwidth, memory blocks are installed on pairs of memory boards, even and odd. There are a maximum of four even-odd pairs of memory blocks in a system.

Figure 2-5 “40-bit coherent memory address generation” shows the mechanism for interleave. The 40-bit physical address provides the basis from which the memory blocks and memory bank are selected.

The Memory Board Configuration register supplies the map used to translate a memory block to the associated memory board where the memory line resides. See the section “EPAC Memory Board Configuration register” for more information.

Figure 2-5 40-bit coherent memory address generation

The EMAC online field of the System Configuration register checks that a valid VR value is specified in the memory address. An invalid VR value results in an HPMC.

Bank interleaved memory pattern

Single memory board interleave mode forces contiguous memory lines to reside in the same memory block. Within the memory block, memory lines interleave across the four memory banks. Figure 2-6 “Single memory block interleave pattern” shows the interleave pattern for this mode. The pattern is the same independent of the number of memory blocks in the system. The pattern shown is for a 4096-byte page of memory (128 memory lines per page).

Figure 2-6 Single memory block interleave pattern

Block and bank interleave memory pattern

In multiple memory board interleave modes, memory lines are interleaved across the largest power-of-two memory banks. There are up to eight memory blocks, resulting in 32-way memory line interleaving. The minimum system configuration has two memory blocks, resulting in eight-way memory line interleaving. With both four and six memory blocks in the system, the interleave is 16-way. Figure 2-7 “Memory line interleave pattern with four memory blocks ” shows the interleave pattern for a system with four memory blocks. The pattern shown is for 4096-byte pages of memory (128 memory lines per page).

Figure 2-7 Memory line interleave pattern with four memory blocks