The V-Class server uses CSRs in the EPACs and EMACs and memory
addresses along with the data mover to transfer messages and data.
These CSRs and addresses include:
The implementation uses memory structures for message reception,
message completion queues, and the BTT for I/O data copy
transfers. Memory structures are preallocated regions of memory.
The actual data resides in memory before and after transfer.
Messaging and data copy CSRs |
 |
CSRs in both the EPACs and EMACs control messaging and data
copy. This section specifies the addresses used to access each CSR
of the messaging and data copy hardware.
EPAC Operation Context registers
Each EPAC has two Operation Context registers, one for each
processor. The operation context is applied to other CSRs in two
ways. One is by arming a register, and the other is by indicating
that the armed register was triggered, that is, it performed a specific
function. Figure 4-2 “EPAC CSR Operation Context register
definition” shows the
format of the EPAC CSR Operation Context register.
The bits of the CSR Operation Context register are defined
as follows:
Triggered bit
(bit 62)—Indicates that a CSR operation executed when the
Armed bit was set. The Triggered bit is cleared by software and
is set by hardware.
Armed bit (bit 63)—Set
by software to arm the functionality of specific EPAC processor
CSRs. The EPAC CSRs armed by this bit are:
Data Mover Input Command register
Fetch and Increment address
Fetch and Decrement address
Noncoherent Write address
Coherent Increment address
Each of these CSRs is discussed in this chapter. The Armed
bit is set by software and is cleared by either hardware or software.
Table 4-1 “CSR
Operation Context register transitions when the operation is issued” shows the Armed
and Triggered bit transitions that the hardware controls when software
writes to one of the operation addresses.
Table 4-1 CSR
Operation Context register transitions when the operation is issued
| Present
value | Next value |
|---|
| Triggered | Armed | Triggered | Armed |
|---|
| 0 | 0 | 0 | 0 |
| 0 | 1 | 1 | 0 |
| 1 | 0 | 0 | 0 |
| 1 | 1 | 1 | 1 |
Table 4-2 “CSR
Operation Context register transitions with TLB invalidate” shows the Armed
and Triggered bit transitions that hardware controls when a TLB
invalidate transaction is detected.
Table 4-2 CSR
Operation Context register transitions with TLB invalidate
| Present
value | Next value |
|---|
| Triggered | Armed | Triggered | Armed |
|---|
| 0 | 0 | 0 | 0 |
| 0 | 1 | 0 | 0 |
| 1 | 0 | 1 | 0 |
| 1 | 1 | 1 | 1 |
EPAC Operation Address registers
Each EPAC has two Operation Address registers, one for each
processor. The register stores the address used for CSR operations.
The format of the EPAC Operation Address register is shown
in Figure 4-3 “EPAC Operation Address register definition”. The field of the
register is read by a read access.
The Operation address field (bits 24:63) designates the address
for CSR operations and also the Coherent Increment address.
EPAC Input Command registers
Each EPAC has two input command registers that set the modes
and lengths of messaging and data copy operations.
The Input Command register can be written when the Ready bit
of the CSR is zero and the CSR Operation Context register Armed
bit is a one. There are no restrictions for reading this register.
The format of the Input Command register is shown in Figure 4-4 “EPAC Input Command register format”.
The bits and fields of the Input Command register are defined
as follows:
Ready bit (bit
23)—Indicates that the input registers are ready to perform
an operation. Normally, this bit is set by software and cleared
by hardware. It should be set by software when the input registers
are completely set up for an operation. Hardware clears it when
the messaging and copy state machine has accessed all required information
from the input registers for the operation. The Ready bit is written
by a CSR write access. A CSR read will read the current value. Reset
clears the bit.
Status index field (bits 24:25)—Indicates
part of the status in the Operation Completion status queue. Reset
clears the field.
TLB purge abort enable bit
(bit 26)—Enables an operation to be aborted if a TLB purge
transaction is detected prior to or during the operation. In system
operation, software sets and clears the bit. The operation aborts
prior to starting if the TLB purge seen and TLB purge abort enable
bits are set at the time the messaging and copy state machine starts
the operation. Completion status for an aborted operation is written
to the appropriate status queue. The TLB Purge Abort Enable bit
is written by a CSR write access and read by a CSR read. Reset clears
the bit.
TLB purge seen bit (bit 27)—Indicates
that a TLB purge transaction was detected by an EPAC. The bit is
set by hardware and cleared by software. It is written by a CSR
write. A CSR access reads the current value. Reset clears the bit.
Source BTT enable bit (bit
28)—Indicates the Source Physical Page Frame register contains
the address of the BTT used for accessing the source memory region
of the operation. The bit is written by a CSR write and read by
a CSR read.
Destination BTT enable bit
(bit 29)—Indicates the Destination Physical Page Frame
register contains the address of the BTT used for accessing the
destination memory region of the operation. The bit is written by
a CSR write and read by a CSR read.
Messaging operation bit (bit
30)—Forces the messaging and copy state machine to use
the messaging mechanism to determine the destination address rather
than the destination address of the input register. The bit is written
by a CSR write and read by a CSR read.
Bzero operation bit (bit 31)—Forces
the messaging and copy state machine to clear the destination memory
region rather than copy the source to destination memory region.
The bit is written by a CSR write and read by a CSR read.
Gather operation field (bits
32:33)—Specifies the stride used for a gather operation.
Currently, this field is disabled and set to zero.
Input interrupt enable bit
(bit 34)—Enables an interrupt to the associated processor
when the Input Command register is available for reprogramming by
software. The most significant five bits of the interrupt number
that is sent is specified by this field. The least significant bit
of the interrupt number sent is zero. The bit is written by a CSR
write and read by a CSR read.
Completion interrupt enable
field (bits 35:36)—Enables an interrupt to the associated
processor when the messaging and copy state machine completes the
operation. The field also determines whether an interrupt is sent
when the operation completes with an error if it is sent independently
from the status of the operation. The field is written by a CSR
write and read by a CSR read.
Interrupt number field (bits
37:41)—Specifies the most significant five bits of the
interrupt numbers to be sent to the processor that initiated the
request. An interrupt is sent when either of two events occur:
When the messaging and copy state
machine has completed accessing the input registers
When the messaging and copy state machine completes
the operation
The least significant bit of the interrupt number is a zero
for the first event and a one for the second. The bit is written
by a CSR write and read by a CSR read.
Length-1 field (bits 42:63)—Specifies
the length of the messaging and copy operation. Messaging operations
ignore the least significant 5 bits, forcing the length to be an
integer number of memory lines (32-byte increments). Copies, however,
can be any byte length. A value of zero in the field copies one
byte (one memory line for messaging), and a value of all ones in
the field will clear four Mbytes of memory. The bit is written by
a CSR write and read by a CSR read.
EPAC Source and Destination Physical Page Frame registers
There are two Source/Destination Physical Page Frame
registers on each EPAC to specify the source and destination of
messaging and data copy operations.
The registers can be written to only when the Input Command
CSR Ready bit is zero and the CSR Operation Context register Armed
bit is a one. The register can be read at any time.
The format of the Physical Page Frame register is shown in
Figure 4-5 “EPAC Physical Page Frame register
definition”.
The Physical page frame field (bits 24:51) indicates the physical
page frame of a 40-bit PA-8200 address. If a BTT is being used,
the field specifies the address of the BTT. Otherwise, the field
specifies the source or destination page for the copy operation.
For messaging operations, the Destination Physical Page Frame
register must be programmed with Node ID = 000 and VR to
that of the destination EMAC receiving the message. The Node ID
and VR information are written in the normal physical address field
positions.
EPAC Source and Destination Offset registers
There are two Source/Destination Offset registers
on each EPAC to specify the offset for the source and destination
of a message or copy operation.
The registers can be written to only when the Input Command
CSR Ready bit is zero and the CSR Operation Context register Armed
bit is a one. The register can be read at any time.
The format of the Offset register is shown in Figure 4-6 “EPAC Source and Destination Offset
register definition”.
The BTT/Page offset field (bits 42:63)
is used in one of two ways:
When a BTT is being used, the
most significant 10 bits specify the index into the BTT and the
least significant 12 bits specify the offset into the selected BTE
memory page.
When a BTT is not being used, the field is used
as the offset into a page of memory. The offset within a page can
be up to four Mbytes in size for support of larger page sizes.
For messaging operations, the Destination Offset register
need not be programmed.
EPAC Operation Status Queue registers
There are two Operation Status Queue registers on each EPAC,
one for each of the two processors attached to the EPAC. Status
can not be inserted in a status queue in the same order the processor
sets up the input registers.
The format of the processor 0/1 Status registers
is shown in Figure 4-7 “Operation Status Queue register definition”.
The bits and fields of the Operation Status Queue register
are defined as follows:
Valid bit (bit
0)—Indicates that the Status Queue has valid messaging
and copy state machine completion status. The bit is set when the
state machine has completed and writes status into the queue. The
bit is cleared when the status is read, and no other valid status
remains in the status queue. A CSR read access reads the value,
and a CSR write has no effect. Reset clears the bit.
Overflow bit (bit 1)—Indicates
that a status queue overflow occurred resulting in the loss of status
information. The bit is set when a status queue is full and the
messaging and copy state machine has completed an operation and
its status is destined for that queue. The bit is cleared when the
status register is read. A CSR write does not effect the value of
the bit. Reset clears the bit.
Status index field (bits 2:3)—Indicates
the status index. The two bits are a direct copy of Input Command
register Status Index field just before the operation was started.
Completion status (bits 4:7)—Indicates
the messaging and copy state machine completion status.
Table 4-3 Completion
status field values
| Field values | Completion status |
|---|
| 0 | Operation completed successfully |
| 1 | Data mover detected error |
| 2 | Source memory transaction error |
| 3 | Destination memory transaction error |
| 4 | Source BTE transaction error |
| 5 | Destination BTE transaction error |
| 6 | Message allocate transaction error |
| 7 | Message completion transaction error |
| 8-F | Reserved |
Detecting controller (bits
8:9) and Detecting Xbar port (bits 10:13) fields—Specify
which controller or crossbar port detected the error. This informations
is obtained directly from a transaction error response.
Error code field (bits 14:18)—Specifies
the type of error that caused the operation to fail.
Table 4-4 Error code
values
| Field value | Error code |
|---|
| 0 | TLB purge aborted operation |
| 1 | Insufficient queue space for message |
| 2 | Insufficient memory for message |
| 3 | Message reception disabled |
| 4 | Source BTE translation invalid |
| 5 | Destination BTE translation invalid |
| 6 | Transaction timed out |
| 7-1F | Reserved |
Completion length-1 field (bits
42:63)—Indicates the amount remaining to copy when the
operation finished. The field is only valid if the operation was
aborted with the detection of a TLB purge. The field contains the
value of minus one when the operation completed successfully and
zero or greater if the operation was aborted. The value restarts
an operation when it aborted due to a TLB Purge being detected.
A CSR read access reads the value, and a CSR write has no effect.
EMAC Message Reception Area Configuration register
There is one Message Reception Area Configuration register
on each EMAC to specify the base address for the region of memory
used to receive messages.
The format of the Configuration registers is shown in Figure 4-8 “EMAC Message Reception Area Configuration
register definition”.
The Size field (bits 62:63) specifies
the size of the Message Reception Area. Table 4-5 “Message Reception Area size
options” shows the possible sizes for the Message
Reception Area.
Table 4-5 Message Reception Area size
options
| Field value | Size |
|---|
| 0 | 32 Kbyte |
| 1 | 256 Kbyte |
| 2 | 2 Mbyte |
| 3 | 16 Mbyte |
EMAC Message Reception Area Offset registers
There are two Message Reception Area Offset registers on each
EMAC:
Message Reception Area Available Offset
register—Specifies the region of the message reception
area available for new messages.
Message Reception Area Occupied Offset register—Specifies
the region presently occupied by messages.
One register specifies the offset into the message reception
area where the received messages start and the other specifies where
occupied memory starts.
The format of the Message Reception Area Offset registers
is shown in Figure 4-9 “EMAC Message Reception Area Offset
register definition”.
Offset field (bits 39:58) specifies an
offset into the message reception area. The register is normally
read and written by hardware (to allocate space for new messages).
It is read by hardware to check if sufficient available area exists
for a new message and written by software to free memory consumed
by previously received messages.
Depending on the Size field of the register, some of the most
significant bits of the offset field are not used and must be set
to zero when written by software. Table 4-6 “Offset
bits used for each size option” shows the bits for each possible size
of the message reception area.
Table 4-6 Offset
bits used for each size option
| Size option | Bits used as offset |
|---|
| 32 Kbyte | 10-bits (49:58) |
| 256 Kbyte | 13-bits (46:58) |
| 2 Mbyte | 16-bits (43:58) |
| 16 Mbyte | 19-bits (40:58) |
The message reception area is full when the Message Reception
Area Available Offset is equal to the Message Reception Area Occupied
Offset in the bits specified in Table 4-6 “Offset
bits used for each size option” and the single bit more significant to
that specified in the table is different. Bit 39 of the Offset field
is never used as an offset to the Message Reception Area, but rather
is only used to determine the full status of the Message Reception
Area when the size is 16 Mbytes.
EMAC Message Completion Queue Configuration register
Each EMAC has one Message Completion Queue Configuration register
that specifies the base address for a region of memory used to write
the message completion status.
Figure 4-10 “EMAC Message Completion Queue Configuration
register definition” shows the format
of the register. All fields of the register are read by a read access
and written by a write access.
The bits and fields of the Message Completion Queue Configuration
register are defined as follows:
Row (bits 30:32)
and Page (bits 38:49) fields—Specify the Message Completion
Queue base address. The VB is not part of the base address, because
the hardware uses all banks on the EMAC with specified Row and Page
values as message completion queue area memory.
Interrupt processor field (bits
53:56)—Specifies which of the 16 processors to interrupt
when message completion status is placed in the message completion
queue.
Interrupt number field (bits
57:62)—Specifies the interrupt number used to interrupt
the destination processor when message completion status is placed
in the message completion queue.
Queue enable bit (bit 63)—Enables
receiving messages to the associated message reception area. The
bit is cleared by reset.
EMAC Message Completion Queue Offset registers
Each EMAC has three Message Completion Queue Offset registers:
Message Completion Queue Reserve Offset—Specifies
the offset into the message completion queue memory area where space
has been reserved for message completion status.
Message Completion Queue Write Offset—Specifies
the offset where received message status is written.
Message Completion Queue Read Offset—Specifies
the offset where message completion status is read.
Software must initialize these registers by writing a zero
value, but, thereafter, only hardware needs to read or write the
registers.
Figure 4-11 “EMAC Message Completion Queue Offset
register definition” shows the format
of the three Message Completion Queue Offset registers.
The Offset field (bits 49:60) specifies
an offset into the message completion queue memory area. The most
significant bit of the field (bit 49) is not part of the offset,
but determines the full or empty status of the queue.
The Message Completion Queue is full when bits 50:60 of the
Message Completion Queue Read Offset register are equal to bits
50:60 of the Message Completion Queue Write Offset register and
bit 49 of each register is different. The queue is empty when bits
49:60 of each offset register have the same value.
EMAC Message Allocation address
Each EMAC has a message allocation address. This address is
special in that it does not have registers associated with it, but
rather manipulates other CSRs when accessed. The operation performed
is to check that space exists in the message reception area and
message completion queue, and, if it does exist, to allocate space
in the reception area and reserve an entry in the message completion
queue.
The following functionality is performed by an access to this
address:
Checking that the Message Reception
Area has been enabled to receive a message.
This is performed by checking the Queue Enable bit of the
Message Completion Queue Configuration register.
Checking that an entry exists in the Message Completion
Queue.
The information required for the check is the Message Completion
Queue Reserved Offset and Message Completion Queue Read Offset registers.
The check that is made is that the comparison of the two offsets
does not result in queue full.
Checking that space exists in the message reception
area.
The information needed for this check is the length of the
message, the Message Reception Area Available Memory Offset register,
and the Message Reception Area Occupied Memory Offset register.
The check which is made is that the occupied offset less the available
offset is greater than the length of the message.
Returning status of the unsuccessful allocation
attempt if any of the above checks fail.
Incrementing the Message Reception Area Available
Offset register by the length of the message.
Incrementing the Message Completion Queue Reserved
Offset register by one, indicating one less entry available.
EMAC Message Completion Enqueue address
Each EMAC has a Message Completion Enqueue address that is
special in that it does not have registers associated with it, but
rather other CSRs are manipulated when the address is written to.
The operation performed is writing the completion status to a memory-based
message completion queue.
The message completion queue should not be full, because any
previous access to the Message Allocation register address will
have reserved space in the queue for the completion status.
The following functionality is performed by a write to this
address:
Writing of the completion status to
the memory-based message completion queue.
The memory address to be written is formed by the Row and
Page fields of the Message Completion Queue Configuration register
and the Offset field of the Message Completion Queue Write Offset
register. The data to be written is contained in the write request
packet.
Incrementing by one the Offset field of the Message
Completion Queue Write Offset register.
If the Message Completion Queue was empty prior to accessing
the Message Completion Enqueue address, the processor specified
by the Message Completion Queue Configuration register is interrupted.
Figure 4-12 “EMAC Message Completion Enqueue definition” shows the format
for the request data sent with a write to a Message Completion Enqueue
address.
The bits and fields of the response data returned from a read
to the address are defined as follows:
Completion status
field (bits 1:2)—Specifies the completion status of a received
message. Table 4-7 “Message
Completion Status field values” shows the possible
completion status field values.
Table 4-7 Message
Completion Status field values
| Field value | Completion status |
|---|
| 0 | Message received successfully |
| 1 | Message aborted |
| 2, 3 | Reserved |
For completion status values 0 and 1, the space for the message
was allocated in the Message Reception Area and the memory must
be free.
Length-1 field (bits 10:26)—Specifies
the length of the allocated memory in memory lines (32-byte increments)
for the message. A zero value specifies one memory line (32 bytes)
and a value of all ones specifies 131,072 memory lines (4 Mbytes).
Start Offset field bits 40:58)—Specifies
the offset into the memory reception area to the start of the message.
EMAC Message Completion Dequeue address
Each EMAC has a Message Completion Dequeue address that is
special: it does not have registers associated, and it manipulates
other CSRs when the address is read. The operation performed is
reading the completion status from a memory based message completion
queue.
The functionality performed by a read to this address is listed
below:
Returning a response with the valid
bit as zero if the Message Completion Queue is empty
Reading the completion status from the memory-based
Message Completion Queue
The memory address read is formed by using the Row, and Page
fields of the Message Completion Queue Configuration register and
the Offset field of the Message Completion Queue Read Offset register.
The data that is read is returned in the response packet.
Incrementing by one the Offset field of the Message
Completion Queue Read Offset register
Figure 4-13 “EMAC Message Completion Dequeue definition” shows the format
for the response data returned from a read to a Message Completion
Dequeue address.
The bits and fields of the response data returned from a read
to the address are defined as follows:
Valid bit (bit
0)—Indicates the empty status of the Message Completion
Queue at the time of the read access.
Completion status field (bits
1:2)—Specifies the completion status of a received message.
Table 4-8 “Message
Completion Status field values” shows the possible completion
status field values.
Table 4-8 Message
Completion Status field values
| Field value | Completion status |
|---|
| 0 | Message received successfully |
| 1 | Message aborted |
| 2, 3 | Reserved |
For completion status values 0 and 1, the space for the message
allocated in the Message Reception Area and memory must be freed.
Length-1 field (bits 10:26)—Specifies
the length of the allocated memory in memory lines (32-byte increments)
for the message. A zero value specifies one memory line (32 bytes)
and a value of all ones specifies 131,072 memory lines (4 Mbytes).
Start offset field (bits 40:58)—Specifies
the offset into the memory reception area to the start of the message.
Memory structures |
|
This section describes the memory structures used by the messaging
and data copy hardware. The three data structures are:
Block Translation Table (BTT)
The Message Reception area is a preallocated region of memory
to which messages can be written. The memory is controlled by hardware
that enqueues messages as they are received.
All accesses to message reception areas are through coherent
memory accesses. A processor can copy a message out from the Message
Reception area directly or by using the data copy hardware.
Message Completion Queue area
The Message Completion Queue area holds message completion
status until software is ready to process a received message.
The size of the Message Completion Queue area is fixed at
16 Kbytes. Each entry is 8 bytes in size, resulting in
2048 entries per queue. The Message completion queue area resides
in memory that is physically connected to the EMAC.
Figure 4-14 “Message Completion Queue and entry
definition” shows the format
for a Message Completion Queue and one of its entries.
The fields of the message status entry are as follows:
Completion status
field (bits 0:1)—Specifies the completion status of a received
message. Table 4-9 “Message
Completion Status field values” shows the possible
completion status field values.
Table 4-9 Message
Completion Status field values
| Field value | Completion Status |
|---|
| 0 | Message received successfully |
| 1 | Message aborted |
| 2, 3 | Reserved |
For completion status values 0 and 1, the space for the message
allocated in the Message Reception Area and the memory must be freed.
Length-1 field (bits 10:26)—Specifies
allocated memory in number of memory lines (32-byte increments)
for the message. A value of zero specifies one memory line (32 bytes),
and a value of all ones specifies 131,072 memory lines (4 Mbytes).
Start offset field (bits 40:58)—Specifies
the offset into the Memory Reception Area to the start of the message.
Block Translation Table definition
The BTT provides the I/O system a means to translate
from a peripheral's address space to physical memory. It
specifies a mapping of contiguous addresses to pages of physical
memory. The table is limited to a single page of memory, with each
entry being a word (four bytes) in size.
Each entry in the table is called a Block Translation Entry
(BTE), and it specifies the page frame for a page of physical memory.
A page is 4096 bytes. The BTT specifies a maximum address space
of four Mbytes.
Figure 4-15 “Block Translation Table and Entry
definition” shows the format
for a BTT and one of its entries.
The bits and fields of the BTE are as follows:
Valid bit (bit
0)—Indicates a valid entry. If the messaging and copy state
machine needs to use an entry without the Valid
bit set, the operation is aborted, with completion status indicating
the problem.
Read/Write bit (bit 1)—Ignored
by the messaging and copy state machine.
Physical page frame field (bits
4:31)—Indicates the page frame for either the source or
destination of the operation.
Printable version
