diff options
Diffstat (limited to 'doc/user/mp.t')
| -rw-r--r-- | doc/user/mp.t | 628 |
1 files changed, 628 insertions, 0 deletions
diff --git a/doc/user/mp.t b/doc/user/mp.t new file mode 100644 index 0000000000..05a62978c4 --- /dev/null +++ b/doc/user/mp.t @@ -0,0 +1,628 @@ +@c +@c COPYRIGHT (c) 1996. +@c On-Line Applications Research Corporation (OAR). +@c All rights reserved. +@c + +@ifinfo +@node Multiprocessing Manager, Multiprocessing Manager Introduction, Configuring a System Sizing the RTEMS RAM Workspace, Top +@end ifinfo +@chapter Multiprocessing Manager +@ifinfo +@menu +* Multiprocessing Manager Introduction:: +* Multiprocessing Manager Background:: +* Multiprocessing Manager Multiprocessor Communications Interface Layer:: +* Multiprocessing Manager Operations:: +* Multiprocessing Manager Directives:: +@end menu +@end ifinfo + +@ifinfo +@node Multiprocessing Manager Introduction, Multiprocessing Manager Background, Multiprocessing Manager, Multiprocessing Manager +@end ifinfo +@section Introduction + +In multiprocessor real-time systems, new +requirements, such as sharing data and global resources between +processors, are introduced. This requires an efficient and +reliable communications vehicle which allows all processors to +communicate with each other as necessary. In addition, the +ramifications of multiple processors affect each and every +characteristic of a real-time system, almost always making them +more complicated. + +RTEMS addresses these issues by providing simple and +flexible real-time multiprocessing capabilities. The executive +easily lends itself to both tightly-coupled and loosely-coupled +configurations of the target system hardware. In addition, +RTEMS supports systems composed of both homogeneous and +heterogeneous mixtures of processors and target boards. + +A major design goal of the RTEMS executive was to +transcend the physical boundaries of the target hardware +configuration. This goal is achieved by presenting the +application software with a logical view of the target system +where the boundaries between processor nodes are transparent. +As a result, the application developer may designate objects +such as tasks, queues, events, signals, semaphores, and memory +blocks as global objects. These global objects may then be +accessed by any task regardless of the physical location of the +object and the accessing task. RTEMS automatically determines +that the object being accessed resides on another processor and +performs the actions required to access the desired object. +Simply stated, RTEMS allows the entire system, both hardware and +software, to be viewed logically as a single system. + +@ifinfo +@node Multiprocessing Manager Background, Nodes, Multiprocessing Manager Introduction, Multiprocessing Manager +@end ifinfo +@section Background +@ifinfo +@menu +* Nodes:: +* Global Objects:: +* Global Object Table:: +* Remote Operations:: +* Proxies:: +* Multiprocessor Configuration Table:: +@end menu +@end ifinfo + +RTEMS makes no assumptions regarding the connection +media or topology of a multiprocessor system. The tasks which +compose a particular application can be spread among as many +processors as needed to satisfy the application's timing +requirements. The application tasks can interact using a subset +of the RTEMS directives as if they were on the same processor. +These directives allow application tasks to exchange data, +communicate, and synchronize regardless of which processor they +reside upon. + +The RTEMS multiprocessor execution model is multiple +instruction streams with multiple data streams (MIMD). This +execution model has each of the processors executing code +independent of the other processors. Because of this +parallelism, the application designer can more easily guarantee +deterministic behavior. + +By supporting heterogeneous environments, RTEMS +allows the systems designer to select the most efficient +processor for each subsystem of the application. Configuring +RTEMS for a heterogeneous environment is no more difficult than +for a homogeneous one. In keeping with RTEMS philosophy of +providing transparent physical node boundaries, the minimal +heterogeneous processing required is isolated in the MPCI layer. + +@ifinfo +@node Nodes, Global Objects, Multiprocessing Manager Background, Multiprocessing Manager Background +@end ifinfo +@subsection Nodes + +A processor in a RTEMS system is referred to as a +node. Each node is assigned a unique non-zero node number by +the application designer. RTEMS assumes that node numbers are +assigned consecutively from one to maximum_nodes. The node +number, node, and the maximum number of nodes, maximum_nodes, in +a system are found in the Multiprocessor Configuration Table. +The maximum_nodes field and the number of global objects, +maximum_global_objects, is required to be the same on all nodes +in a system. + +The node number is used by RTEMS to identify each +node when performing remote operations. Thus, the +Multiprocessor Communications Interface Layer (MPCI) must be +able to route messages based on the node number. + +@ifinfo +@node Global Objects, Global Object Table, Nodes, Multiprocessing Manager Background +@end ifinfo +@subsection Global Objects + +All RTEMS objects which are created with the GLOBAL +attribute will be known on all other nodes. Global objects can +be referenced from any node in the system, although certain +directive specific restrictions (e.g. one cannot delete a remote +object) may apply. A task does not have to be global to perform +operations involving remote objects. The maximum number of +global objects is the system is user configurable and can be +found in the maximum_global_objects field in the Multiprocessor +Configuration Table. The distribution of tasks to processors is +performed during the application design phase. Dynamic task +relocation is not supported by RTEMS. + +@ifinfo +@node Global Object Table, Remote Operations, Global Objects, Multiprocessing Manager Background +@end ifinfo +@subsection Global Object Table + +RTEMS maintains two tables containing object +information on every node in a multiprocessor system: a local +object table and a global object table. The local object table +on each node is unique and contains information for all objects +created on this node whether those objects are local or global. +The global object table contains information regarding all +global objects in the system and, consequently, is the same on +every node. + +Since each node must maintain an identical copy of +the global object table, the maximum number of entries in each +copy of the table must be the same. The maximum number of +entries in each copy is determined by the +maximum_global_objects parameter in the Multiprocessor +Configuration Table. This parameter, as well as the +maximum_nodes parameter, is required to be the same on all +nodes. To maintain consistency among the table copies, every +node in the system must be informed of the creation or deletion +of a global object. + +@ifinfo +@node Remote Operations, Proxies, Global Object Table, Multiprocessing Manager Background +@end ifinfo +@subsection Remote Operations + +When an application performs an operation on a remote +global object, RTEMS must generate a Remote Request (RQ) message +and send it to the appropriate node. After completing the +requested operation, the remote node will build a Remote +Response (RR) message and send it to the originating node. +Messages generated as a side-effect of a directive (such as +deleting a global task) are known as Remote Processes (RP) and +do not require the receiving node to respond. + +Other than taking slightly longer to execute +directives on remote objects, the application is unaware of the +location of the objects it acts upon. The exact amount of +overhead required for a remote operation is dependent on the +media connecting the nodes and, to a lesser degree, on the +efficiency of the user-provided MPCI routines. + +The following shows the typical transaction sequence +during a remote application: + +@enumerate + +@item The application issues a directive accessing a +remote global object. + +@item RTEMS determines the node on which the object +resides. + +@item RTEMS calls the user-provided MPCI routine +GET_PACKET to obtain a packet in which to build a RQ message. + +@item After building a message packet, RTEMS calls the +user-provided MPCI routine SEND_PACKET to transmit the packet to +the node on which the object resides (referred to as the +destination node). + +@item The calling task is blocked until the RR message +arrives, and control of the processor is transferred to another +task. + +@item The MPCI layer on the destination node senses the +arrival of a packet (commonly in an ISR), and calls the +multiprocessing_announce directive. This directive readies the +Multiprocessing Server. + +@item The Multiprocessing Server calls the user-provided +MPCI routine RECEIVE_PACKET, performs the requested operation, +builds an RR message, and returns it to the originating node. + +@item The MPCI layer on the originating node senses the +arrival of a packet (typically via an interrupt), and calls the +RTEMS multiprocessing_announce directive. This directive +readies the Multiprocessing Server. + +@item The Multiprocessing Server calls the user-provided +MPCI routine RECEIVE_PACKET, readies the original requesting +task, and blocks until another packet arrives. Control is +transferred to the original task which then completes processing +of the directive. + +@end enumerate + +If an uncorrectable error occurs in the user-provided +MPCI layer, the fatal error handler should be invoked. RTEMS +assumes the reliable transmission and reception of messages by +the MPCI and makes no attempt to detect or correct errors. + +@ifinfo +@node Proxies, Multiprocessor Configuration Table, Remote Operations, Multiprocessing Manager Background +@end ifinfo +@subsection Proxies + +A proxy is an RTEMS data structure which resides on a +remote node and is used to represent a task which must block as +part of a remote operation. This action can occur as part of the +semaphore_obtain and message_queue_receive directives. If the +object were local, the task's control block would be available +for modification to indicate it was blocking on a message queue +or semaphore. However, the task's control block resides only on +the same node as the task. As a result, the remote node must +allocate a proxy to represent the task until it can be readied. + +The maximum number of proxies is defined in the +Multiprocessor Configuration Table. Each node in a +multiprocessor system may require a different number of proxies +to be configured. The distribution of proxy control blocks is +application dependent and is different from the distribution of +tasks. + +@ifinfo +@node Multiprocessor Configuration Table, Multiprocessing Manager Multiprocessor Communications Interface Layer, Proxies, Multiprocessing Manager Background +@end ifinfo +@subsection Multiprocessor Configuration Table + +The Multiprocessor Configuration Table contains +information needed by RTEMS when used in a multiprocessor +system. This table is discussed in detail in the section +Multiprocessor Configuration Table of the Configuring a System +chapter. + +@ifinfo +@node Multiprocessing Manager Multiprocessor Communications Interface Layer, INITIALIZATION, Multiprocessor Configuration Table, Multiprocessing Manager +@end ifinfo +@section Multiprocessor Communications Interface Layer +@ifinfo +@menu +* INITIALIZATION:: +* GET_PACKET:: +* RETURN_PACKET:: +* RECEIVE_PACKET:: +* SEND_PACKET:: +* Supporting Heterogeneous Environments:: +@end menu +@end ifinfo + +The Multiprocessor Communications Interface Layer +(MPCI) is a set of user-provided procedures which enable the +nodes in a multiprocessor system to communicate with one +another. These routines are invoked by RTEMS at various times +in the preparation and processing of remote requests. +Interrupts are enabled when an MPCI procedure is invoked. It is +assumed that if the execution mode and/or interrupt level are +altered by the MPCI layer, that they will be restored prior to +returning to RTEMS. + +The MPCI layer is responsible for managing a pool of +buffers called packets and for sending these packets between +system nodes. Packet buffers contain the messages sent between +the nodes. Typically, the MPCI layer will encapsulate the +packet within an envelope which contains the information needed +by the MPCI layer. The number of packets available is dependent +on the MPCI layer implementation. + +The entry points to the routines in the user's MPCI +layer should be placed in the Multiprocessor Communications +Interface Table. The user must provide entry points for each of +the following table entries in a multiprocessor system: + +@itemize @bullet +@item initialization initialize the MPCI +@item get_packet obtain a packet buffer +@item return_packet return a packet buffer +@item send_packet send a packet to another node +@item receive_packet called to get an arrived packet +@end itemize + +A packet is sent by RTEMS in each of the following +situations: + +@itemize @bullet +@item an RQ is generated on an originating node; +@item an RR is generated on a destination node; +@item a global object is created; +@item a global object is deleted; +@item a local task blocked on a remote object is deleted; +@item during system initialization to check for system consistency. +@end itemize + +If the target hardware supports it, the arrival of a +packet at a node may generate an interrupt. Otherwise, the +real-time clock ISR can check for the arrival of a packet. In +any case, the multiprocessing_announce directive must be called +to announce the arrival of a packet. After exiting the ISR, +control will be passed to the Multiprocessing Server to process +the packet. The Multiprocessing Server will call the get_packet +entry to obtain a packet buffer and the receive_entry entry to +copy the message into the buffer obtained. + +@ifinfo +@node INITIALIZATION, GET_PACKET, Multiprocessing Manager Multiprocessor Communications Interface Layer, Multiprocessing Manager Multiprocessor Communications Interface Layer +@end ifinfo +@subsection INITIALIZATION + +The INITIALIZATION component of the user-provided +MPCI layer is called as part of the initialize_executive +directive to initialize the MPCI layer and associated hardware. +It is invoked immediately after all of the device drivers have +been initialized. This component should be adhere to the +following prototype: + +@example +@group +rtems_mpci_entry user_mpci_initialization( + rtems_configuration_table *configuration +); +@end group +@end example + +where configuration is the address of the user's +Configuration Table. Operations on global objects cannot be +performed until this component is invoked. The INITIALIZATION +component is invoked only once in the life of any system. If +the MPCI layer cannot be successfully initialized, the fatal +error manager should be invoked by this routine. + +One of the primary functions of the MPCI layer is to +provide the executive with packet buffers. The INITIALIZATION +routine must create and initialize a pool of packet buffers. +There must be enough packet buffers so RTEMS can obtain one +whenever needed. + +@ifinfo +@node GET_PACKET, RETURN_PACKET, INITIALIZATION, Multiprocessing Manager Multiprocessor Communications Interface Layer +@end ifinfo +@subsection GET_PACKET + +The GET_PACKET component of the user-provided MPCI +layer is called when RTEMS must obtain a packet buffer to send +or broadcast a message. This component should be adhere to the +following prototype: + +@example +@group +rtems_mpci_entry user_mpci_get_packet( + rtems_packet_prefix **packet +); +@end group +@end example + +where packet is the address of a pointer to a packet. +This routine always succeeds and, upon return, packet will +contain the address of a packet. If for any reason, a packet +cannot be successfully obtained, then the fatal error manager +should be invoked. + +RTEMS has been optimized to avoid the need for +obtaining a packet each time a message is sent or broadcast. +For example, RTEMS sends response messages (RR) back to the +originator in the same packet in which the request message (RQ) +arrived. + +@ifinfo +@node RETURN_PACKET, RECEIVE_PACKET, GET_PACKET, Multiprocessing Manager Multiprocessor Communications Interface Layer +@end ifinfo +@subsection RETURN_PACKET + +The RETURN_PACKET component of the user-provided MPCI +layer is called when RTEMS needs to release a packet to the free +packet buffer pool. This component should be adhere to the +following prototype: + +@example +@group +rtems_mpci_entry user_mpci_return_packet( + rtems_packet_prefix *packet +); +@end group +@end example + +where packet is the address of a packet. If the +packet cannot be successfully returned, the fatal error manager +should be invoked. + +@ifinfo +@node RECEIVE_PACKET, SEND_PACKET, RETURN_PACKET, Multiprocessing Manager Multiprocessor Communications Interface Layer +@end ifinfo +@subsection RECEIVE_PACKET + +The RECEIVE_PACKET component of the user-provided +MPCI layer is called when RTEMS needs to obtain a packet which +has previously arrived. This component should be adhere to the +following prototype: + +@example +@group +rtems_mpci_entry user_mpci_receive_packet( + rtems_packet_prefix **packet +); +@end group +@end example + +where packet is a pointer to the address of a packet +to place the message from another node. If a message is +available, then packet will contain the address of the message +from another node. If no messages are available, this entry +packet should contain NULL. + +@ifinfo +@node SEND_PACKET, Supporting Heterogeneous Environments, RECEIVE_PACKET, Multiprocessing Manager Multiprocessor Communications Interface Layer +@end ifinfo +@subsection SEND_PACKET + +The SEND_PACKET component of the user-provided MPCI +layer is called when RTEMS needs to send a packet containing a +message to another node. This component should be adhere to the +following prototype: + +@example +@group +rtems_mpci_entry user_mpci_send_packet( + rtems_unsigned32 node, + rtems_packet_prefix **packet +); +@end group +@end example + +where node is the node number of the destination and packet is the +address of a packet which containing a message. If the packet cannot +be successfully sent, the fatal error manager should be invoked. + +If node is set to zero, the packet is to be +broadcasted to all other nodes in the system. Although some +MPCI layers will be built upon hardware which support a +broadcast mechanism, others may be required to generate a copy +of the packet for each node in the system. + +Many MPCI layers use the packet_length field of the MP_packet_prefix +of the packet to avoid sending unnecessary data. This is especially +useful if the media connecting the nodes is relatively slow. + +The to_convert field of the MP_packet_prefix portion of the packet indicates +how much of the packet (in unsigned32's) may require conversion in a +heterogeneous system. + +@ifinfo +@node Supporting Heterogeneous Environments, Multiprocessing Manager Operations, SEND_PACKET, Multiprocessing Manager Multiprocessor Communications Interface Layer +@end ifinfo +@subsection Supporting Heterogeneous Environments + +Developing an MPCI layer for a heterogeneous system +requires a thorough understanding of the differences between the +processors which comprise the system. One difficult problem is +the varying data representation schemes used by different +processor types. The most pervasive data representation problem +is the order of the bytes which compose a data entity. +Processors which place the least significant byte at the +smallest address are classified as little endian processors. +Little endian byte-ordering is shown below: + + +@example +@group ++---------------+----------------+---------------+----------------+ +| | | | | +| Byte 3 | Byte 2 | Byte 1 | Byte 0 | +| | | | | ++---------------+----------------+---------------+----------------+ +@end group +@end example + +Conversely, processors which place the most +significant byte at the smallest address are classified as big +endian processors. Big endian byte-ordering is shown below: + +@example +@group ++---------------+----------------+---------------+----------------+ +| | | | | +| Byte 0 | Byte 1 | Byte 2 | Byte 3 | +| | | | | ++---------------+----------------+---------------+----------------+ +@end group +@end example + +Unfortunately, sharing a data structure between big +endian and little endian processors requires translation into a +common endian format. An application designer typically chooses +the common endian format to minimize conversion overhead. + +Another issue in the design of shared data structures +is the alignment of data structure elements. Alignment is both +processor and compiler implementation dependent. For example, +some processors allow data elements to begin on any address +boundary, while others impose restrictions. Common restrictions +are that data elements must begin on either an even address or +on a long word boundary. Violation of these restrictions may +cause an exception or impose a performance penalty. + +Other issues which commonly impact the design of +shared data structures include the representation of floating +point numbers, bit fields, decimal data, and character strings. +In addition, the representation method for negative integers +could be one's or two's complement. These factors combine to +increase the complexity of designing and manipulating data +structures shared between processors. + +RTEMS addressed these issues in the design of the +packets used to communicate between nodes. The RTEMS packet +format is designed to allow the MPCI layer to perform all +necessary conversion without burdening the developer with the +details of the RTEMS packet format. As a result, the MPCI layer +must be aware of the following: + +@itemize @bullet +@item All packets must begin on a four byte boundary. + +@item Packets are composed of both RTEMS and application data. +All RTEMS data is treated as thirty-two (32) bit unsigned +quantities and is in the first MINIMUM_UNSIGNED32S_TO_CONVERT +thirty-two (32) quantities of the packet. + +@item The RTEMS data component of the packet must be in native +endian format. Endian conversion may be performed by either the +sending or receiving MPCI layer. + +@item RTEMS makes no assumptions regarding the application +data component of the packet. +@end itemize + +@ifinfo +@node Multiprocessing Manager Operations, Announcing a Packet, Supporting Heterogeneous Environments, Multiprocessing Manager +@end ifinfo +@section Operations +@ifinfo +@menu +* Announcing a Packet:: +@end menu +@end ifinfo + +@ifinfo +@node Announcing a Packet, Multiprocessing Manager Directives, Multiprocessing Manager Operations, Multiprocessing Manager Operations +@end ifinfo +@subsection Announcing a Packet + +The multiprocessing_announce directive is called by +the MPCI layer to inform RTEMS that a packet has arrived from +another node. This directive can be called from an interrupt +service routine or from within a polling routine. + +@ifinfo +@node Multiprocessing Manager Directives, MULTIPROCESSING_ANNOUNCE - Announce the arrival of a packet, Announcing a Packet, Multiprocessing Manager +@end ifinfo +@section Directives +@ifinfo +@menu +* MULTIPROCESSING_ANNOUNCE - Announce the arrival of a packet:: +@end menu +@end ifinfo + +This section details the additional directives +required to support RTEMS in a multiprocessor configuration. A +subsection is dedicated to each of this manager's directives and +describes the calling sequence, related constants, usage, and +status codes. + +@page +@ifinfo +@node MULTIPROCESSING_ANNOUNCE - Announce the arrival of a packet, Directive Status Codes, Multiprocessing Manager Directives, Multiprocessing Manager Directives +@end ifinfo +@subsection MULTIPROCESSING_ANNOUNCE - Announce the arrival of a packet + +@subheading CALLING SEQUENCE: + +@example +void rtems_multiprocessing_announce( void ); +@end example + +@subheading DIRECTIVE STATUS CODES: + +NONE + +@subheading DESCRIPTION: + +This directive informs RTEMS that a multiprocessing +communications packet has arrived from another node. This +directive is called by the user-provided MPCI, and is only used +in multiprocessor configurations. + +@subheading NOTES: + +This directive is typically called from an ISR. + +This directive will almost certainly cause the +calling task to be preempted. + +This directive does not generate activity on remote nodes. |