From 86b48fb09e78d756cce2329213c0a5b48662cc52 Mon Sep 17 00:00:00 2001 From: Sebastian Huber Date: Wed, 21 Apr 2021 09:34:24 +0200 Subject: c-user: Split up multiprocessing manager This makes it easier to automatically generate parts of the module documentation in the future. Update #3993. --- c-user/index.rst | 2 +- c-user/multiprocessing.rst | 508 -------------------------------- c-user/multiprocessing/background.rst | 419 ++++++++++++++++++++++++++ c-user/multiprocessing/directives.rst | 44 +++ c-user/multiprocessing/index.rst | 17 ++ c-user/multiprocessing/introduction.rst | 35 +++ c-user/multiprocessing/operations.rst | 13 + 7 files changed, 529 insertions(+), 509 deletions(-) delete mode 100644 c-user/multiprocessing.rst create mode 100644 c-user/multiprocessing/background.rst create mode 100644 c-user/multiprocessing/directives.rst create mode 100644 c-user/multiprocessing/index.rst create mode 100644 c-user/multiprocessing/introduction.rst create mode 100644 c-user/multiprocessing/operations.rst diff --git a/c-user/index.rst b/c-user/index.rst index 267d56c..5cd87af 100644 --- a/c-user/index.rst +++ b/c-user/index.rst @@ -49,7 +49,7 @@ RTEMS Classic API Guide (|version|). user-extensions/index config/index self_contained_objects - multiprocessing + multiprocessing/index symmetric_multiprocessing_services pci_library stack_bounds_checker diff --git a/c-user/multiprocessing.rst b/c-user/multiprocessing.rst deleted file mode 100644 index f207106..0000000 --- a/c-user/multiprocessing.rst +++ /dev/null @@ -1,508 +0,0 @@ -.. SPDX-License-Identifier: CC-BY-SA-4.0 - -.. Copyright (C) 1988, 2008 On-Line Applications Research Corporation (OAR) - -.. index:: multiprocessing - -Multiprocessing Manager -*********************** - -Introduction -============ - -In multiprocessor real-time systems, new requirements, such as sharing data and -global resources between processors, are introduced. This requires an -efficient and reliable communications vehicle which allows all processors to -communicate with each other as necessary. In addition, the ramifications of -multiple processors affect each and every characteristic of a real-time system, -almost always making them more complicated. - -RTEMS addresses these issues by providing simple and flexible real-time -multiprocessing capabilities. The executive easily lends itself to both -tightly-coupled and loosely-coupled configurations of the target system -hardware. In addition, RTEMS supports systems composed of both homogeneous and -heterogeneous mixtures of processors and target boards. - -A major design goal of the RTEMS executive was to transcend the physical -boundaries of the target hardware configuration. This goal is achieved by -presenting the application software with a logical view of the target system -where the boundaries between processor nodes are transparent. As a result, the -application developer may designate objects such as tasks, queues, events, -signals, semaphores, and memory blocks as global objects. These global objects -may then be accessed by any task regardless of the physical location of the -object and the accessing task. RTEMS automatically determines that the object -being accessed resides on another processor and performs the actions required -to access the desired object. Simply stated, RTEMS allows the entire system, -both hardware and software, to be viewed logically as a single system. - -The directives provided by the Manager are: - -- rtems_multiprocessing_announce_ - A multiprocessing communications packet has - arrived - -.. index:: multiprocessing topologies - -Background -========== - -RTEMS makes no assumptions regarding the connection media or topology of a -multiprocessor system. The tasks which compose a particular application can be -spread among as many processors as needed to satisfy the application's timing -requirements. The application tasks can interact using a subset of the RTEMS -directives as if they were on the same processor. These directives allow -application tasks to exchange data, communicate, and synchronize regardless of -which processor they reside upon. - -The RTEMS multiprocessor execution model is multiple instruction streams with -multiple data streams (MIMD). This execution model has each of the processors -executing code independent of the other processors. Because of this -parallelism, the application designer can more easily guarantee deterministic -behavior. - -By supporting heterogeneous environments, RTEMS allows the systems designer to -select the most efficient processor for each subsystem of the application. -Configuring RTEMS for a heterogeneous environment is no more difficult than for -a homogeneous one. In keeping with RTEMS philosophy of providing transparent -physical node boundaries, the minimal heterogeneous processing required is -isolated in the MPCI layer. - -.. index:: nodes, definition - -Nodes ------ - -A processor in a RTEMS system is referred to as a node. Each node is assigned -a unique non-zero node number by the application designer. RTEMS assumes that -node numbers are assigned consecutively from one to the ``maximum_nodes`` -configuration parameter. The node number, node, and the maximum number of -nodes, ``maximum_nodes``, in a system are found in the Multiprocessor -Configuration Table. The ``maximum_nodes`` field and the number of global -objects, ``maximum_global_objects``, is required to be the same on all nodes in -a system. - -The node number is used by RTEMS to identify each node when performing remote -operations. Thus, the Multiprocessor Communications Interface Layer (MPCI) -must be able to route messages based on the node number. - -.. index:: global objects, definition - -Global Objects --------------- - -All RTEMS objects which are created with the GLOBAL attribute will be known on -all other nodes. Global objects can be referenced from any node in the system, -although certain directive specific restrictions (e.g. one cannot delete a -remote object) may apply. A task does not have to be global to perform -operations involving remote objects. The maximum number of global objects is -the system is user configurable and can be found in the maximum_global_objects -field in the Multiprocessor Configuration Table. The distribution of tasks to -processors is performed during the application design phase. Dynamic task -relocation is not supported by RTEMS. - -.. index:: global objects table - -Global Object Table -------------------- - -RTEMS maintains two tables containing object information on every node in a -multiprocessor system: a local object table and a global object table. The -local object table on each node is unique and contains information for all -objects created on this node whether those objects are local or global. The -global object table contains information regarding all global objects in the -system and, consequently, is the same on every node. - -Since each node must maintain an identical copy of the global object table, the -maximum number of entries in each copy of the table must be the same. The -maximum number of entries in each copy is determined by the -maximum_global_objects parameter in the Multiprocessor Configuration Table. -This parameter, as well as the maximum_nodes parameter, is required to be the -same on all nodes. To maintain consistency among the table copies, every node -in the system must be informed of the creation or deletion of a global object. - -.. index:: MPCI and remote operations - -Remote Operations ------------------ - -When an application performs an operation on a remote global object, RTEMS must -generate a Remote Request (RQ) message and send it to the appropriate node. -After completing the requested operation, the remote node will build a Remote -Response (RR) message and send it to the originating node. Messages generated -as a side-effect of a directive (such as deleting a global task) are known as -Remote Processes (RP) and do not require the receiving node to respond. - -Other than taking slightly longer to execute directives on remote objects, the -application is unaware of the location of the objects it acts upon. The exact -amount of overhead required for a remote operation is dependent on the media -connecting the nodes and, to a lesser degree, on the efficiency of the -user-provided MPCI routines. - -The following shows the typical transaction sequence during a remote -application: - -#. The application issues a directive accessing a remote global object. - -#. RTEMS determines the node on which the object resides. - -#. RTEMS calls the user-provided MPCI routine ``GET_PACKET`` to obtain a packet - in which to build a RQ message. - -#. After building a message packet, RTEMS calls the user-provided MPCI routine - ``SEND_PACKET`` to transmit the packet to the node on which the object - resides (referred to as the destination node). - -#. The calling task is blocked until the RR message arrives, and control of the - processor is transferred to another task. - -#. The MPCI layer on the destination node senses the arrival of a packet - (commonly in an ISR), and calls the ``rtems_multiprocessing_announce`` - directive. This directive readies the Multiprocessing Server. - -#. The Multiprocessing Server calls the user-provided MPCI routine - ``RECEIVE_PACKET``, performs the requested operation, builds an RR message, - and returns it to the originating node. - -#. The MPCI layer on the originating node senses the arrival of a packet - (typically via an interrupt), and calls the RTEMS - ``rtems_multiprocessing_announce`` directive. This directive readies the - Multiprocessing Server. - -#. The Multiprocessing Server calls the user-provided MPCI routine - ``RECEIVE_PACKET``, readies the original requesting task, and blocks until - another packet arrives. Control is transferred to the original task which - then completes processing of the directive. - -If an uncorrectable error occurs in the user-provided MPCI layer, the fatal -error handler should be invoked. RTEMS assumes the reliable transmission and -reception of messages by the MPCI and makes no attempt to detect or correct -errors. - -.. index:: proxy, definition - -.. _MPCIProxies: - -Proxies -------- - -A proxy is an RTEMS data structure which resides on a remote node and is used -to represent a task which must block as part of a remote operation. This action -can occur as part of the ``rtems_semaphore_obtain`` and -``rtems_message_queue_receive`` directives. If the object were local, the -task's control block would be available for modification to indicate it was -blocking on a message queue or semaphore. However, the task's control block -resides only on the same node as the task. As a result, the remote node must -allocate a proxy to represent the task until it can be readied. - -The maximum number of proxies is defined in the Multiprocessor Configuration -Table. Each node in a multiprocessor system may require a different number of -proxies to be configured. The distribution of proxy control blocks is -application dependent and is different from the distribution of tasks. - -Multiprocessor Configuration Table ----------------------------------- - -The Multiprocessor Configuration Table contains information needed by RTEMS -when used in a multiprocessor system. This table is discussed in detail in the -section Multiprocessor Configuration Table of the Configuring a System chapter. - -Multiprocessor Communications Interface Layer -============================================= - -The Multiprocessor Communications Interface Layer (MPCI) is a set of -user-provided procedures which enable the nodes in a multiprocessor system to -communicate with one another. These routines are invoked by RTEMS at various -times in the preparation and processing of remote requests. Interrupts are -enabled when an MPCI procedure is invoked. It is assumed that if the execution -mode and/or interrupt level are altered by the MPCI layer, that they will be -restored prior to returning to RTEMS. - -.. index:: MPCI, definition - -The MPCI layer is responsible for managing a pool of buffers called packets and -for sending these packets between system nodes. Packet buffers contain the -messages sent between the nodes. Typically, the MPCI layer will encapsulate -the packet within an envelope which contains the information needed by the MPCI -layer. The number of packets available is dependent on the MPCI layer -implementation. - -.. index:: MPCI entry points - -The entry points to the routines in the user's MPCI layer should be placed in -the Multiprocessor Communications Interface Table. The user must provide entry -points for each of the following table entries in a multiprocessor system: - -.. list-table:: - :class: rtems-table - - * - initialization - - initialize the MPCI - * - get_packet - - obtain a packet buffer - * - return_packet - - return a packet buffer - * - send_packet - - send a packet to another node - * - receive_packet - - called to get an arrived packet - -A packet is sent by RTEMS in each of the following situations: - -- an RQ is generated on an originating node; - -- an RR is generated on a destination node; - -- a global object is created; - -- a global object is deleted; - -- a local task blocked on a remote object is deleted; - -- during system initialization to check for system consistency. - -If the target hardware supports it, the arrival of a packet at a node may -generate an interrupt. Otherwise, the real-time clock ISR can check for the -arrival of a packet. In any case, the ``rtems_multiprocessing_announce`` -directive must be called to announce the arrival of a packet. After exiting -the ISR, control will be passed to the Multiprocessing Server to process the -packet. The Multiprocessing Server will call the get_packet entry to obtain a -packet buffer and the receive_entry entry to copy the message into the buffer -obtained. - -INITIALIZATION --------------- - -The INITIALIZATION component of the user-provided MPCI layer is called as part -of the ``rtems_initialize_executive`` directive to initialize the MPCI layer -and associated hardware. It is invoked immediately after all of the device -drivers have been initialized. This component should be adhere to the -following prototype: - -.. index:: rtems_mpci_entry - -.. code-block:: c - - rtems_mpci_entry user_mpci_initialization( void ); - -Operations on global objects cannot be performed until this component is -invoked. The INITIALIZATION component is invoked only once in the life of any -system. If the MPCI layer cannot be successfully initialized, the fatal error -manager should be invoked by this routine. - -One of the primary functions of the MPCI layer is to provide the executive with -packet buffers. The INITIALIZATION routine must create and initialize a pool -of packet buffers. There must be enough packet buffers so RTEMS can obtain one -whenever needed. - -GET_PACKET ----------- - -The GET_PACKET component of the user-provided MPCI layer is called when RTEMS -must obtain a packet buffer to send or broadcast a message. This component -should be adhere to the following prototype: - -.. code-block:: c - - rtems_mpci_entry user_mpci_get_packet( - rtems_packet_prefix **packet - ); - -where packet is the address of a pointer to a packet. This routine always -succeeds and, upon return, packet will contain the address of a packet. If for -any reason, a packet cannot be successfully obtained, then the fatal error -manager should be invoked. - -RTEMS has been optimized to avoid the need for obtaining a packet each time a -message is sent or broadcast. For example, RTEMS sends response messages (RR) -back to the originator in the same packet in which the request message (RQ) -arrived. - -RETURN_PACKET -------------- - -The RETURN_PACKET component of the user-provided MPCI layer is called when -RTEMS needs to release a packet to the free packet buffer pool. This component -should be adhere to the following prototype: - -.. code-block:: c - - rtems_mpci_entry user_mpci_return_packet( - rtems_packet_prefix *packet - ); - -where packet is the address of a packet. If the packet cannot be successfully -returned, the fatal error manager should be invoked. - -RECEIVE_PACKET --------------- - -The RECEIVE_PACKET component of the user-provided MPCI layer is called when -RTEMS needs to obtain a packet which has previously arrived. This component -should be adhere to the following prototype: - -.. code-block:: c - - rtems_mpci_entry user_mpci_receive_packet( - rtems_packet_prefix **packet - ); - -where packet is a pointer to the address of a packet to place the message from -another node. If a message is available, then packet will contain the address -of the message from another node. If no messages are available, this entry -packet should contain NULL. - -SEND_PACKET ------------ - -The SEND_PACKET component of the user-provided MPCI layer is called when RTEMS -needs to send a packet containing a message to another node. This component -should be adhere to the following prototype: - -.. code-block:: c - - rtems_mpci_entry user_mpci_send_packet( - uint32_t node, - rtems_packet_prefix **packet - ); - -where node is the node number of the destination and packet is the address of a -packet which containing a message. If the packet cannot be successfully sent, -the fatal error manager should be invoked. - -If node is set to zero, the packet is to be broadcasted to all other nodes in -the system. Although some MPCI layers will be built upon hardware which -support a broadcast mechanism, others may be required to generate a copy of the -packet for each node in the system. - -.. COMMENT: XXX packet_prefix structure needs to be defined in this document - -Many MPCI layers use the ``packet_length`` field of the ``rtems_packet_prefix`` -portion of the packet to avoid sending unnecessary data. This is especially -useful if the media connecting the nodes is relatively slow. - -The ``to_convert`` field of the ``rtems_packet_prefix`` portion of the packet -indicates how much of the packet in 32-bit units may require conversion in a -heterogeneous system. - -.. index:: heterogeneous multiprocessing - -Supporting Heterogeneous Environments -------------------------------------- - -Developing an MPCI layer for a heterogeneous system requires a thorough -understanding of the differences between the processors which comprise the -system. One difficult problem is the varying data representation schemes used -by different processor types. The most pervasive data representation problem -is the order of the bytes which compose a data entity. Processors which place -the least significant byte at the smallest address are classified as little -endian processors. Little endian byte-ordering is shown below: - -.. code-block:: c - - +---------------+----------------+---------------+----------------+ - | | | | | - | Byte 3 | Byte 2 | Byte 1 | Byte 0 | - | | | | | - +---------------+----------------+---------------+----------------+ - -Conversely, processors which place the most significant byte at the smallest -address are classified as big endian processors. Big endian byte-ordering is -shown below: - -.. code-block:: c - - +---------------+----------------+---------------+----------------+ - | | | | | - | Byte 0 | Byte 1 | Byte 2 | Byte 3 | - | | | | | - +---------------+----------------+---------------+----------------+ - -Unfortunately, sharing a data structure between big endian and little endian -processors requires translation into a common endian format. An application -designer typically chooses the common endian format to minimize conversion -overhead. - -Another issue in the design of shared data structures is the alignment of data -structure elements. Alignment is both processor and compiler implementation -dependent. For example, some processors allow data elements to begin on any -address boundary, while others impose restrictions. Common restrictions are -that data elements must begin on either an even address or on a long word -boundary. Violation of these restrictions may cause an exception or impose a -performance penalty. - -Other issues which commonly impact the design of shared data structures include -the representation of floating point numbers, bit fields, decimal data, and -character strings. In addition, the representation method for negative -integers could be one's or two's complement. These factors combine to increase -the complexity of designing and manipulating data structures shared between -processors. - -RTEMS addressed these issues in the design of the packets used to communicate -between nodes. The RTEMS packet format is designed to allow the MPCI layer to -perform all necessary conversion without burdening the developer with the -details of the RTEMS packet format. As a result, the MPCI layer must be aware -of the following: - -- All packets must begin on a four byte boundary. - -- Packets are composed of both RTEMS and application data. All RTEMS data is - treated as 32-bit unsigned quantities and is in the first ``to_convert`` - 32-bit quantities of the packet. The ``to_convert`` field is part of the - ``rtems_packet_prefix`` portion of the packet. - -- The RTEMS data component of the packet must be in native endian format. - Endian conversion may be performed by either the sending or receiving MPCI - layer. - -- RTEMS makes no assumptions regarding the application data component of the - packet. - -Operations -========== - -Announcing a Packet -------------------- - -The ``rtems_multiprocessing_announce`` directive is called by the MPCI layer to -inform RTEMS that a packet has arrived from another node. This directive can -be called from an interrupt service routine or from within a polling routine. - -Directives -========== - -This section details the additional directives required to support RTEMS in a -multiprocessor configuration. A subsection is dedicated to each of this -manager's directives and describes the calling sequence, related constants, -usage, and status codes. - -.. raw:: latex - - \clearpage - -.. index:: announce arrival of package -.. index:: rtems_multiprocessing_announce - -.. _rtems_multiprocessing_announce: - -MULTIPROCESSING_ANNOUNCE - Announce the arrival of a packet ------------------------------------------------------------ - -CALLING SEQUENCE: - .. code-block:: c - - void rtems_multiprocessing_announce( void ); - -DIRECTIVE STATUS CODES: - NONE - -DESCRIPTION: - This directive informs RTEMS that a multiprocessing communications packet - has arrived from another node. This directive is called by the - user-provided MPCI, and is only used in multiprocessor configurations. - -NOTES: - This directive is typically called from an ISR. - - This directive will almost certainly cause the calling task to be - preempted. - - This directive does not generate activity on remote nodes. diff --git a/c-user/multiprocessing/background.rst b/c-user/multiprocessing/background.rst new file mode 100644 index 0000000..2aaae6e --- /dev/null +++ b/c-user/multiprocessing/background.rst @@ -0,0 +1,419 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. Copyright (C) 1988, 2008 On-Line Applications Research Corporation (OAR) + +.. index:: multiprocessing topologies + +Background +========== + +RTEMS makes no assumptions regarding the connection media or topology of a +multiprocessor system. The tasks which compose a particular application can be +spread among as many processors as needed to satisfy the application's timing +requirements. The application tasks can interact using a subset of the RTEMS +directives as if they were on the same processor. These directives allow +application tasks to exchange data, communicate, and synchronize regardless of +which processor they reside upon. + +The RTEMS multiprocessor execution model is multiple instruction streams with +multiple data streams (MIMD). This execution model has each of the processors +executing code independent of the other processors. Because of this +parallelism, the application designer can more easily guarantee deterministic +behavior. + +By supporting heterogeneous environments, RTEMS allows the systems designer to +select the most efficient processor for each subsystem of the application. +Configuring RTEMS for a heterogeneous environment is no more difficult than for +a homogeneous one. In keeping with RTEMS philosophy of providing transparent +physical node boundaries, the minimal heterogeneous processing required is +isolated in the MPCI layer. + +.. index:: nodes, definition + +Nodes +----- + +A processor in a RTEMS system is referred to as a node. Each node is assigned +a unique non-zero node number by the application designer. RTEMS assumes that +node numbers are assigned consecutively from one to the ``maximum_nodes`` +configuration parameter. The node number, node, and the maximum number of +nodes, ``maximum_nodes``, in a system are found in the Multiprocessor +Configuration Table. The ``maximum_nodes`` field and the number of global +objects, ``maximum_global_objects``, is required to be the same on all nodes in +a system. + +The node number is used by RTEMS to identify each node when performing remote +operations. Thus, the Multiprocessor Communications Interface Layer (MPCI) +must be able to route messages based on the node number. + +.. index:: global objects, definition + +Global Objects +-------------- + +All RTEMS objects which are created with the GLOBAL attribute will be known on +all other nodes. Global objects can be referenced from any node in the system, +although certain directive specific restrictions (e.g. one cannot delete a +remote object) may apply. A task does not have to be global to perform +operations involving remote objects. The maximum number of global objects is +the system is user configurable and can be found in the maximum_global_objects +field in the Multiprocessor Configuration Table. The distribution of tasks to +processors is performed during the application design phase. Dynamic task +relocation is not supported by RTEMS. + +.. index:: global objects table + +Global Object Table +------------------- + +RTEMS maintains two tables containing object information on every node in a +multiprocessor system: a local object table and a global object table. The +local object table on each node is unique and contains information for all +objects created on this node whether those objects are local or global. The +global object table contains information regarding all global objects in the +system and, consequently, is the same on every node. + +Since each node must maintain an identical copy of the global object table, the +maximum number of entries in each copy of the table must be the same. The +maximum number of entries in each copy is determined by the +maximum_global_objects parameter in the Multiprocessor Configuration Table. +This parameter, as well as the maximum_nodes parameter, is required to be the +same on all nodes. To maintain consistency among the table copies, every node +in the system must be informed of the creation or deletion of a global object. + +.. index:: MPCI and remote operations + +Remote Operations +----------------- + +When an application performs an operation on a remote global object, RTEMS must +generate a Remote Request (RQ) message and send it to the appropriate node. +After completing the requested operation, the remote node will build a Remote +Response (RR) message and send it to the originating node. Messages generated +as a side-effect of a directive (such as deleting a global task) are known as +Remote Processes (RP) and do not require the receiving node to respond. + +Other than taking slightly longer to execute directives on remote objects, the +application is unaware of the location of the objects it acts upon. The exact +amount of overhead required for a remote operation is dependent on the media +connecting the nodes and, to a lesser degree, on the efficiency of the +user-provided MPCI routines. + +The following shows the typical transaction sequence during a remote +application: + +#. The application issues a directive accessing a remote global object. + +#. RTEMS determines the node on which the object resides. + +#. RTEMS calls the user-provided MPCI routine ``GET_PACKET`` to obtain a packet + in which to build a RQ message. + +#. After building a message packet, RTEMS calls the user-provided MPCI routine + ``SEND_PACKET`` to transmit the packet to the node on which the object + resides (referred to as the destination node). + +#. The calling task is blocked until the RR message arrives, and control of the + processor is transferred to another task. + +#. The MPCI layer on the destination node senses the arrival of a packet + (commonly in an ISR), and calls the ``rtems_multiprocessing_announce`` + directive. This directive readies the Multiprocessing Server. + +#. The Multiprocessing Server calls the user-provided MPCI routine + ``RECEIVE_PACKET``, performs the requested operation, builds an RR message, + and returns it to the originating node. + +#. The MPCI layer on the originating node senses the arrival of a packet + (typically via an interrupt), and calls the RTEMS + ``rtems_multiprocessing_announce`` directive. This directive readies the + Multiprocessing Server. + +#. The Multiprocessing Server calls the user-provided MPCI routine + ``RECEIVE_PACKET``, readies the original requesting task, and blocks until + another packet arrives. Control is transferred to the original task which + then completes processing of the directive. + +If an uncorrectable error occurs in the user-provided MPCI layer, the fatal +error handler should be invoked. RTEMS assumes the reliable transmission and +reception of messages by the MPCI and makes no attempt to detect or correct +errors. + +.. index:: proxy, definition + +.. _MPCIProxies: + +Proxies +------- + +A proxy is an RTEMS data structure which resides on a remote node and is used +to represent a task which must block as part of a remote operation. This action +can occur as part of the ``rtems_semaphore_obtain`` and +``rtems_message_queue_receive`` directives. If the object were local, the +task's control block would be available for modification to indicate it was +blocking on a message queue or semaphore. However, the task's control block +resides only on the same node as the task. As a result, the remote node must +allocate a proxy to represent the task until it can be readied. + +The maximum number of proxies is defined in the Multiprocessor Configuration +Table. Each node in a multiprocessor system may require a different number of +proxies to be configured. The distribution of proxy control blocks is +application dependent and is different from the distribution of tasks. + +Multiprocessor Configuration Table +---------------------------------- + +The Multiprocessor Configuration Table contains information needed by RTEMS +when used in a multiprocessor system. This table is discussed in detail in the +section Multiprocessor Configuration Table of the Configuring a System chapter. + +Multiprocessor Communications Interface Layer +============================================= + +The Multiprocessor Communications Interface Layer (MPCI) is a set of +user-provided procedures which enable the nodes in a multiprocessor system to +communicate with one another. These routines are invoked by RTEMS at various +times in the preparation and processing of remote requests. Interrupts are +enabled when an MPCI procedure is invoked. It is assumed that if the execution +mode and/or interrupt level are altered by the MPCI layer, that they will be +restored prior to returning to RTEMS. + +.. index:: MPCI, definition + +The MPCI layer is responsible for managing a pool of buffers called packets and +for sending these packets between system nodes. Packet buffers contain the +messages sent between the nodes. Typically, the MPCI layer will encapsulate +the packet within an envelope which contains the information needed by the MPCI +layer. The number of packets available is dependent on the MPCI layer +implementation. + +.. index:: MPCI entry points + +The entry points to the routines in the user's MPCI layer should be placed in +the Multiprocessor Communications Interface Table. The user must provide entry +points for each of the following table entries in a multiprocessor system: + +.. list-table:: + :class: rtems-table + + * - initialization + - initialize the MPCI + * - get_packet + - obtain a packet buffer + * - return_packet + - return a packet buffer + * - send_packet + - send a packet to another node + * - receive_packet + - called to get an arrived packet + +A packet is sent by RTEMS in each of the following situations: + +- an RQ is generated on an originating node; + +- an RR is generated on a destination node; + +- a global object is created; + +- a global object is deleted; + +- a local task blocked on a remote object is deleted; + +- during system initialization to check for system consistency. + +If the target hardware supports it, the arrival of a packet at a node may +generate an interrupt. Otherwise, the real-time clock ISR can check for the +arrival of a packet. In any case, the ``rtems_multiprocessing_announce`` +directive must be called to announce the arrival of a packet. After exiting +the ISR, control will be passed to the Multiprocessing Server to process the +packet. The Multiprocessing Server will call the get_packet entry to obtain a +packet buffer and the receive_entry entry to copy the message into the buffer +obtained. + +INITIALIZATION +-------------- + +The INITIALIZATION component of the user-provided MPCI layer is called as part +of the ``rtems_initialize_executive`` directive to initialize the MPCI layer +and associated hardware. It is invoked immediately after all of the device +drivers have been initialized. This component should be adhere to the +following prototype: + +.. index:: rtems_mpci_entry + +.. code-block:: c + + rtems_mpci_entry user_mpci_initialization( void ); + +Operations on global objects cannot be performed until this component is +invoked. The INITIALIZATION component is invoked only once in the life of any +system. If the MPCI layer cannot be successfully initialized, the fatal error +manager should be invoked by this routine. + +One of the primary functions of the MPCI layer is to provide the executive with +packet buffers. The INITIALIZATION routine must create and initialize a pool +of packet buffers. There must be enough packet buffers so RTEMS can obtain one +whenever needed. + +GET_PACKET +---------- + +The GET_PACKET component of the user-provided MPCI layer is called when RTEMS +must obtain a packet buffer to send or broadcast a message. This component +should be adhere to the following prototype: + +.. code-block:: c + + rtems_mpci_entry user_mpci_get_packet( + rtems_packet_prefix **packet + ); + +where packet is the address of a pointer to a packet. This routine always +succeeds and, upon return, packet will contain the address of a packet. If for +any reason, a packet cannot be successfully obtained, then the fatal error +manager should be invoked. + +RTEMS has been optimized to avoid the need for obtaining a packet each time a +message is sent or broadcast. For example, RTEMS sends response messages (RR) +back to the originator in the same packet in which the request message (RQ) +arrived. + +RETURN_PACKET +------------- + +The RETURN_PACKET component of the user-provided MPCI layer is called when +RTEMS needs to release a packet to the free packet buffer pool. This component +should be adhere to the following prototype: + +.. code-block:: c + + rtems_mpci_entry user_mpci_return_packet( + rtems_packet_prefix *packet + ); + +where packet is the address of a packet. If the packet cannot be successfully +returned, the fatal error manager should be invoked. + +RECEIVE_PACKET +-------------- + +The RECEIVE_PACKET component of the user-provided MPCI layer is called when +RTEMS needs to obtain a packet which has previously arrived. This component +should be adhere to the following prototype: + +.. code-block:: c + + rtems_mpci_entry user_mpci_receive_packet( + rtems_packet_prefix **packet + ); + +where packet is a pointer to the address of a packet to place the message from +another node. If a message is available, then packet will contain the address +of the message from another node. If no messages are available, this entry +packet should contain NULL. + +SEND_PACKET +----------- + +The SEND_PACKET component of the user-provided MPCI layer is called when RTEMS +needs to send a packet containing a message to another node. This component +should be adhere to the following prototype: + +.. code-block:: c + + rtems_mpci_entry user_mpci_send_packet( + uint32_t node, + rtems_packet_prefix **packet + ); + +where node is the node number of the destination and packet is the address of a +packet which containing a message. If the packet cannot be successfully sent, +the fatal error manager should be invoked. + +If node is set to zero, the packet is to be broadcasted to all other nodes in +the system. Although some MPCI layers will be built upon hardware which +support a broadcast mechanism, others may be required to generate a copy of the +packet for each node in the system. + +.. COMMENT: XXX packet_prefix structure needs to be defined in this document + +Many MPCI layers use the ``packet_length`` field of the ``rtems_packet_prefix`` +portion of the packet to avoid sending unnecessary data. This is especially +useful if the media connecting the nodes is relatively slow. + +The ``to_convert`` field of the ``rtems_packet_prefix`` portion of the packet +indicates how much of the packet in 32-bit units may require conversion in a +heterogeneous system. + +.. index:: heterogeneous multiprocessing + +Supporting Heterogeneous Environments +------------------------------------- + +Developing an MPCI layer for a heterogeneous system requires a thorough +understanding of the differences between the processors which comprise the +system. One difficult problem is the varying data representation schemes used +by different processor types. The most pervasive data representation problem +is the order of the bytes which compose a data entity. Processors which place +the least significant byte at the smallest address are classified as little +endian processors. Little endian byte-ordering is shown below: + +.. code-block:: c + + +---------------+----------------+---------------+----------------+ + | | | | | + | Byte 3 | Byte 2 | Byte 1 | Byte 0 | + | | | | | + +---------------+----------------+---------------+----------------+ + +Conversely, processors which place the most significant byte at the smallest +address are classified as big endian processors. Big endian byte-ordering is +shown below: + +.. code-block:: c + + +---------------+----------------+---------------+----------------+ + | | | | | + | Byte 0 | Byte 1 | Byte 2 | Byte 3 | + | | | | | + +---------------+----------------+---------------+----------------+ + +Unfortunately, sharing a data structure between big endian and little endian +processors requires translation into a common endian format. An application +designer typically chooses the common endian format to minimize conversion +overhead. + +Another issue in the design of shared data structures is the alignment of data +structure elements. Alignment is both processor and compiler implementation +dependent. For example, some processors allow data elements to begin on any +address boundary, while others impose restrictions. Common restrictions are +that data elements must begin on either an even address or on a long word +boundary. Violation of these restrictions may cause an exception or impose a +performance penalty. + +Other issues which commonly impact the design of shared data structures include +the representation of floating point numbers, bit fields, decimal data, and +character strings. In addition, the representation method for negative +integers could be one's or two's complement. These factors combine to increase +the complexity of designing and manipulating data structures shared between +processors. + +RTEMS addressed these issues in the design of the packets used to communicate +between nodes. The RTEMS packet format is designed to allow the MPCI layer to +perform all necessary conversion without burdening the developer with the +details of the RTEMS packet format. As a result, the MPCI layer must be aware +of the following: + +- All packets must begin on a four byte boundary. + +- Packets are composed of both RTEMS and application data. All RTEMS data is + treated as 32-bit unsigned quantities and is in the first ``to_convert`` + 32-bit quantities of the packet. The ``to_convert`` field is part of the + ``rtems_packet_prefix`` portion of the packet. + +- The RTEMS data component of the packet must be in native endian format. + Endian conversion may be performed by either the sending or receiving MPCI + layer. + +- RTEMS makes no assumptions regarding the application data component of the + packet. diff --git a/c-user/multiprocessing/directives.rst b/c-user/multiprocessing/directives.rst new file mode 100644 index 0000000..afa3244 --- /dev/null +++ b/c-user/multiprocessing/directives.rst @@ -0,0 +1,44 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. Copyright (C) 1988, 2008 On-Line Applications Research Corporation (OAR) + +Directives +========== + +This section details the additional directives required to support RTEMS in a +multiprocessor configuration. A subsection is dedicated to each of this +manager's directives and describes the calling sequence, related constants, +usage, and status codes. + +.. raw:: latex + + \clearpage + +.. index:: announce arrival of package +.. index:: rtems_multiprocessing_announce + +.. _rtems_multiprocessing_announce: + +MULTIPROCESSING_ANNOUNCE - Announce the arrival of a packet +----------------------------------------------------------- + +CALLING SEQUENCE: + .. code-block:: c + + void rtems_multiprocessing_announce( void ); + +DIRECTIVE STATUS CODES: + NONE + +DESCRIPTION: + This directive informs RTEMS that a multiprocessing communications packet + has arrived from another node. This directive is called by the + user-provided MPCI, and is only used in multiprocessor configurations. + +NOTES: + This directive is typically called from an ISR. + + This directive will almost certainly cause the calling task to be + preempted. + + This directive does not generate activity on remote nodes. diff --git a/c-user/multiprocessing/index.rst b/c-user/multiprocessing/index.rst new file mode 100644 index 0000000..4ec4433 --- /dev/null +++ b/c-user/multiprocessing/index.rst @@ -0,0 +1,17 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. Copyright (C) 2021 embedded brains GmbH (http://www.embedded-brains.de) + +.. index:: multiprocessing + +.. _RTEMSAPIClassicMP: + +Multiprocessing Manager +*********************** + +.. toctree:: + + introduction + background + operations + directives diff --git a/c-user/multiprocessing/introduction.rst b/c-user/multiprocessing/introduction.rst new file mode 100644 index 0000000..3a5edd5 --- /dev/null +++ b/c-user/multiprocessing/introduction.rst @@ -0,0 +1,35 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. Copyright (C) 1988, 2008 On-Line Applications Research Corporation (OAR) + +Introduction +============ + +In multiprocessor real-time systems, new requirements, such as sharing data and +global resources between processors, are introduced. This requires an +efficient and reliable communications vehicle which allows all processors to +communicate with each other as necessary. In addition, the ramifications of +multiple processors affect each and every characteristic of a real-time system, +almost always making them more complicated. + +RTEMS addresses these issues by providing simple and flexible real-time +multiprocessing capabilities. The executive easily lends itself to both +tightly-coupled and loosely-coupled configurations of the target system +hardware. In addition, RTEMS supports systems composed of both homogeneous and +heterogeneous mixtures of processors and target boards. + +A major design goal of the RTEMS executive was to transcend the physical +boundaries of the target hardware configuration. This goal is achieved by +presenting the application software with a logical view of the target system +where the boundaries between processor nodes are transparent. As a result, the +application developer may designate objects such as tasks, queues, events, +signals, semaphores, and memory blocks as global objects. These global objects +may then be accessed by any task regardless of the physical location of the +object and the accessing task. RTEMS automatically determines that the object +being accessed resides on another processor and performs the actions required +to access the desired object. Simply stated, RTEMS allows the entire system, +both hardware and software, to be viewed logically as a single system. + +The directives provided by the Manager are: + +- :ref:`rtems_multiprocessing_announce` diff --git a/c-user/multiprocessing/operations.rst b/c-user/multiprocessing/operations.rst new file mode 100644 index 0000000..08dcded --- /dev/null +++ b/c-user/multiprocessing/operations.rst @@ -0,0 +1,13 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +.. Copyright (C) 1988, 2008 On-Line Applications Research Corporation (OAR) + +Operations +========== + +Announcing a Packet +------------------- + +The ``rtems_multiprocessing_announce`` directive is called by the MPCI layer to +inform RTEMS that a packet has arrived from another node. This directive can +be called from an interrupt service routine or from within a polling routine. -- cgit v1.2.3