eng: Add BSP build system chapter

Update #3818.

2 files changed, 490 insertions, 0 deletions

diff --git a/eng/build-system.rst b/eng/build-system.rst
new file mode 100644
index 0000000..3ec76de
--- /dev/null
+++ b/eng/build-system.rst
@@ -0,0 +1,489 @@
+.. SPDX-License-Identifier: CC-BY-SA-4.0
+
+.. Copyright (C) 2019, 2020 embedded brains GmbH (http://www.embedded-brains.de)
+
+.. _BSPBuildSystem:
+
+BSP Build System
+****************
+
+The purpose of the build system is to produce and install artefacts from the
+RTEMS sources such as static libraries, start files, linker command files,
+configuration header files, header files, test programs, package description
+files, and third-party build system support files for a specific BSP in a user
+controlled configuration.
+
+Goals
+=====
+
+The system should meet the following goals:
+
+* The install location of artefacts should be the same as in previous build
+  systems
+
+* Easy build configuration of BSPs through configuration options
+
+* Enable the BSP build configuration to be placed in a version control system
+  (e.g. no local paths)
+
+* Fast builds (also on Windows)
+
+* Easy to maintain, e.g. add/remove a BSP, add/change/remove configuration
+  options, add/remove a library, add/remove an object to/from a library,
+  add/remove tests
+
+* Reusable build specifications (e.g. generate documentation for BSP options for
+  the user manual)
+
+* Validation of built artefacts (e.g. ensure that the objects are built as
+  specified using the DWARF debug information)
+
+* Support building of BSPs external to the project
+
+* Customization of the build (e.g. build only a subset of the RTEMS functions)
+
+* Support alternative compilers such as clang instead of GCC
+
+* Ability to unit test the build system
+
+* Version control system friendly change sets (e.g. most changes are line based
+  in text files)
+
+Configurable things which depend on the host computer environment such as paths
+to tools are intended to be passed as command line options to the ``waf``
+command line tool.  Configurable things which define what is built and how the
+artefacts are configured are intended to be placed in configuration files that
+can be configuration controlled.  The ``waf`` build script file called
+``wscript`` should know nothing about the layout of the sources.  What is built
+and how it is built should be completely defined by the user configuration and
+the build specification items.
+
+Overview
+========
+
+For an overview of the build system, see the *BSP Build System* chapter of the
+`RTEMS User Manual <https://docs.rtems.org/branches/master/user/bld/>`_.
+
+Commands
+========
+
+This section explains how the :ref:`SpecTypeBuildItemType` items determine what
+the following ``waf`` commands do.
+
+BSP List
+--------
+
+In the ``./waf bsp_list`` command, the BSP list is generated from the
+:ref:`SpecTypeBuildBSPItemType` items.
+
+BSP Defaults
+------------
+
+In the ``./waf bsp_defaults`` command, the BSP defaults are generated from the
+:ref:`SpecTypeBuildBSPItemType` and :ref:`SpecTypeBuildOptionItemType` items.
+Build specification items contribute to the command through the
+``do_defaults()`` method in the ``wscript``.
+
+Configure
+---------
+
+In the ``./waf configure`` command, the build specification items contribute to
+the command through the ``prepare_configure()`` and ``do_configure()`` methods
+in the ``wscript``.
+
+Build, Clean, and Install
+-------------------------
+
+In the ``./waf``, ``./waf clean``, and ``./waf install`` commands, the build
+specification items contribute to the commands through the ``prepare_build()``
+and ``do_build()`` methods in the ``wscript``.
+
+UID Naming Conventions
+======================
+
+Use the following patterns for :ref:`UID names <ReqEngIdent>`:
+
+abi
+    Use the name ``abi`` for the GCC-specific ABI flags item of a BSP family.
+    Each BSP family should have exactly one :ref:`SpecTypeBuildOptionItemType`
+    item which defines the GCC-specific ABI flags for all base BSPs of the
+    family.  For an architecture named *arch* and a BSP family named *family*,
+    the file path is ``spec/build/bsps/arch/family/abi.yml``.
+
+abiclang
+    Use the name ``abiclang`` for the clang-specific ABI flags item of a BSP
+    family.  Each BSP family may have at most one
+    :ref:`SpecTypeBuildOptionItemType` item which defines the clang-specific
+    ABI flags for all base BSPs of the family.  For an architecture named
+    *arch* and a BSP family named *family*, the file path is
+    ``spec/build/bsps/arch/family/abiclang.yml``.
+
+bsp*
+    Use the prefix ``bsp`` for base BSPs.
+
+cfg*
+    Use the prefix ``cfg`` for ``config.h`` header options.
+
+grp*
+    Use the prefix ``grp`` for groups.
+
+lib*
+    Use the prefix ``lib`` for libraries.
+
+linkcmds*
+    Use the prefix ``linkcmds`` for linker command files.
+
+obj*
+    Use the prefix ``obj`` for objects.  Use
+
+    * ``objmpci`` for objects which are enabled by ``RTEMS_MULTIPROCESSING``,
+
+    * ``objnet`` for objects which are enabled by ``RTEMS_NETWORKING``,
+
+    * ``objnetnosmp`` for objects which are enabled by ``RTEMS_NETWORKING`` and
+      not ``RTEMS_SMP``, and
+
+    * ``objsmp`` for objects which are enabled by ``RTEMS_SMP``.
+
+opt*
+    Use the prefix ``opt`` for options.  Use 
+
+    * ``optclock*`` for options which have something to do with the clock
+      driver,
+
+    * ``optconsole*`` for options which have something to do with the console
+      driver,
+
+    * ``optirq*`` for options which have something to do with interrupt
+      processing,
+
+    * ``optmem*`` for options which have something to do with the memory
+      configuration, map, settings, etc., and
+
+    * ``optosc*`` for options which have something to do with oscillators.
+
+start
+    Use the name ``start`` for BSP start file items.  Each architecture or BSP
+    family should have a :ref:`SpecTypeBuildStartFileItemType` item which
+    builds the start file of a BSP.  For an architecture named *arch* and a BSP
+    family named *family*, the file path is ``spec/build/bsps/arch/start.yml``
+    or ``spec/build/bsps/arch/family/start.yml``.  It is preferable to have a
+    shared start file for the architecture instead of a start file for each BSP
+    family.
+
+tst*
+    Use the prefix ``tst`` for test states.
+
+.. _BuildSpecItems:
+
+Build Specification Items
+=========================
+
+Specification items of refinements of the :ref:`SpecTypeBuildItemType` are used
+by the ``wscript`` to determine what it should do.  The ``wscript`` does not
+provide default values.  All values are defined by specification items.  The
+entry point to what is built are the :ref:`SpecTypeBuildBSPItemType` items and
+the top-level :ref:`SpecTypeBuildGroupItemType` item.  The user configuration
+defines which BSPs are built.  The top-level group defaults to ``/grp`` and can
+be changed by the ``--rtems-top-level`` command line option given to the ``waf
+configure`` command.
+
+The top-level group is a trade-off between the specification complexity and a
+strict dependency definition.  Dependencies are normally explicit though the
+item links.  However, using only explicit dependencies would make the
+specification quite complex, see :numref:`BuildExplicitDeps`.  The top-level
+group and explicit :ref:`SpecTypeBuildBSPItemType` items reduce the
+specification complexity since they use a priori knowledge of global build
+dependencies, see :numref:`BuildOrderDeps` for an example.  This approach makes
+the build system easier, but less general.
+
+.. _BuildExplicitDeps:
+
+.. figure:: ../images/eng/bld-deps.*
+    :width: 100%
+    :figclass: align-center
+
+    This example shows how build item dependencies are specified explicitly
+    by item links.  In this example, a user wants to build a group of tests.
+    Each test program has a dependency on the standard RTEMS libraries.  The
+    first issue is that the ``librtemsbsp.a`` needs dependencies to all base
+    BSP variants (more than 100).  The dependencies are the values of the
+    ``links`` attribute in the library item files.  External BSPs would have
+    to modify the library item files.  This is quite undesirable.  The
+    second issue is that the source files of the ``librtemscpu.a`` need a
+    dependency to the ABI compiler flags specified by each BSP.  The third
+    issue is that each BSP has to define its own ``bspopts.h`` configuration
+    header item.
+
+.. _BuildOrderDeps:
+
+.. figure:: ../images/eng/bld-deps2.*
+    :width: 50%
+    :figclass: align-center
+
+    This example shows how build item dependencies are specified by dedicated
+    BSP items, a top-level group, and ordered item links.  The BSP is
+    configured after the top-level group item and built before the top-level
+    group item (defined by ``wscript`` source code).  The library group is
+    configured and built before the test group as specified by the item link
+    order in the top-level group.  The BSP options are processed before the
+    results are written to the configuration header ``bspopts.h`` as defined by
+    the BSP item link order.
+
+.. _BuildHowTo:
+
+How-To
+======
+
+This section presents how to do common maintenance tasks in the build system.
+
+Find the Right Item
+-------------------
+
+You find all build specification items in the RTEMS sources under the
+``spec/build`` directory.  You can use the ``grep`` command line tool to search
+in the build specification items.
+
+Create a BSP Architecture
+-------------------------
+
+Let *arch* be the name of the architecture.  You can add the new BSP
+architecture with:
+
+.. code-block:: none
+
+    $ mkdir spec/build/bsps/arch
+
+For a new architecture try to use a shared start file which can be used by all
+BSPs of the architecture.  Add a :ref:`SpecTypeBuildStartFileItemType` item for
+it:
+
+.. code-block:: none
+
+    $ vi spec/build/bsps/arch/start.yml
+
+Create a BSP Family
+-------------------
+
+Let *family* be the BSP family name and *arch* the name of the architecture.
+You can add the new BSP family with:
+
+.. code-block:: none
+
+    $ mkdir spec/build/bsps/arch/family
+
+Add a :ref:`SpecTypeBuildOptionItemType` item for the ABI flags of the BSP family:
+
+.. code-block:: none
+
+    $ vi spec/build/bsps/arch/family/abi.yml
+
+Define the ABI flags for each base BSP of the family.  The ``${ABI_FLAGS}`` are
+used for the ``${ASFLAGS}``, ``${CFLAGS}``, ``${CXXFLAGS}``, and ``${LDFLAGS}``
+build environment variables.  Please have a look at the following example which
+shows the GCC-specific ABI flags item of the ``sparc/leon3`` BSP family:
+
+.. code-block:: yaml
+
+    SPDX-License-Identifier: CC-BY-SA-4.0 OR BSD-2-Clause
+    actions:
+    - get-string: null
+    - split: null
+    - env-append: null
+    build-type: option
+    copyrights:
+    - Copyright (C) 2020 embedded brains GmbH (http://www.embedded-brains.de)
+    default:
+    - -mcpu=leon3
+    default-by-variant:
+    - value:
+      - -mcpu=leon3
+      - -mfix-ut700
+      variants:
+      - sparc/ut700
+    - value:
+      - -mcpu=leon
+      - -mfix-ut699
+      variants:
+      - sparc/ut699
+    - value:
+      - -mcpu=leon3
+      - -mfix-gr712rc
+      variants:
+      - sparc/gr712rc
+    description: |
+      ABI flags
+    enabled-by: gcc
+    links: []
+    name: ABI_FLAGS
+    type: build
+
+If the architecture has no shared start file, then add a
+:ref:`SpecTypeBuildStartFileItemType` item for the new BSP family:
+
+.. code-block:: none
+
+    $ vi spec/build/bsps/arch/family/start.yml
+
+Add a Base BSP to a BSP Family
+------------------------------
+
+.. _BuildBSPFamilyOneBSP:
+
+.. figure:: ../images/eng/bld-bsp.*
+    :width: 40%
+    :figclass: align-center
+
+    This example shows a BSP family named *family* in the architecture *arch*
+    which consists of only one base BSP named *xyz*.  The BSP sources and
+    installation information is contained in the
+    ``spec:/build/bsps/arch/family/bspxyz`` BSP item.  The items linked by the
+    BSP item are shown using relative UIDs.
+
+.. _BuildBSPFamilyManyBSPs:
+
+.. figure:: ../images/eng/bld-bsp2.*
+    :width: 50%
+    :figclass: align-center
+
+    This example shows a BSP family named *family* in the architecture *arch*
+    which consists of three base BSPs named *rst*, *uvw*, and *xyz*.  The BSP
+    sources and installation information is contained in the *obj* objects
+    item.  The group *grp* defines the main BSP constituents.  The base BSP
+    items ``spec:/build/bsps/arch/family/bsprst``,
+    ``spec:/build/bsps/arch/family/bspuvw``, and
+    ``spec:/build/bsps/arch/family/bspxyz`` just define the name of the base
+    BSP and set a link to the group item.  The base BSP names can be used for
+    example in the ``default-by-variant`` attribute of
+    :ref:`SpecTypeBuildOptionItemType` items.  The items linked by the BSP
+    items are shown using relative UIDs.
+
+Let *family* be the BSP family name, *arch* the name of the architecture, and
+*new* the name of the new base BSP.  You can add the new base BSP with:
+
+.. code-block:: none
+
+    $ vi spec/build/bsps/arch/family/bspnew.yml
+
+Define the attributes of your new base BSP according to
+:ref:`SpecTypeBuildBSPItemType`.
+
+In case the BSP family has no group, then create a group if it is likely that
+the BSP family will contain more than one base BSP (see
+:ref:`BuildHowToBSPFamilyGroup`).
+
+If the BSP family has a group, then link the new base BSP to the group with:
+
+.. code-block:: none
+
+    $ vi spec/build/bsps/arch/familiy/grp.yml
+
+Add a link using a relative UID to the new base BSP item:
+
+.. code-block:: yaml
+
+   links:
+   - role: build-dependency
+     uid: bspnew
+
+Add a BSP Option
+----------------
+
+Let *family* be the BSP family name, *arch* the name of the architecture, and
+*new* the name of the new BSP option.  You can add the new BSP option with:
+
+.. code-block:: none
+
+    $ vi spec/build/bsps/arch/family/optnew.yml
+
+Define the attributes of your new BSP option according to
+:ref:`SpecTypeBuildOptionItemType`.  Link the option item to the corresponding
+group or BSP item using a relative UID:
+
+.. code-block:: yaml
+
+   links:
+   - role: build-dependency
+     uid: optnew
+
+.. _BuildHowToBSPFamilyGroup:
+
+Extend a BSP Family with a Group
+--------------------------------
+
+Let *family* be the BSP family name and *arch* the name of the architecture.  If
+you want to have more than one base BSP in a BSP family, then you have to use a
+group item (see :ref:`SpecTypeBuildGroupItemType`).  Add the group item named *grp* to the
+family with:
+
+.. code-block:: none
+
+    $ vi spec/build/bsps/arch/family/grp.yml
+
+Define the attributes of your new group according to
+:ref:`SpecTypeBuildGroupItemType` and move the links of the existing base BSP
+item to the group item.  Add a link to *obj*.
+
+Add an objects item named *obj* to the family with:
+
+.. code-block:: none
+
+    $ vi spec/build/bsps/arch/family/obj.yml
+
+Define the attributes of your new objects item according to
+:ref:`SpecTypeBuildObjectsItemType` and move the ``cflags``, ``cppflags``,
+``includes``, ``install`` and ``source`` attribute values of the
+existing base BSP item to the objects item.
+
+Add a Test Program
+------------------
+
+Let *collection* be the name of a test program collection and *new* the name of
+the new test program.  You can add the new test program with:
+
+.. code-block:: none
+
+    $ vi spec/build/testsuites/collection/new.yml
+
+Define the attributes of your new test program according to
+:ref:`SpecTypeBuildTestProgramItemType`.
+
+Edit corresponding group item of the test program collection:
+
+.. code-block:: none
+
+    $ vi spec/build/testsuites/collection/grp.yml
+
+Add a link to the new test program item using a relative UID:
+
+.. code-block:: yaml
+
+   links:
+   - role: build-dependency
+     uid: new
+
+Add a Library
+-------------
+
+Let *new* be the name of the new library.  You can add the new library with:
+
+.. code-block:: none
+
+    $ vi spec/build/cpukit/libnew.yml
+
+Define the attributes of your new library according to
+:ref:`SpecTypeBuildLibraryItemType`.
+
+Edit corresponding group item:
+
+.. code-block:: none
+
+    $ vi spec/build/cpukit/grp.yml
+
+Add a link to the new library item using a relative UID:
+
+.. code-block:: yaml
+
+   links:
+   - role: build-dependency
+     uid: libnew
diff --git a/eng/index.rst b/eng/index.rst
index f6b02ec..d5d6f85 100644
--- a/eng/index.rst
+++ b/eng/index.rst
@@ -31,6 +31,7 @@ RTEMS Software Engineering (|version|)
     management
     test-plan
     test-framework
+    build-system
     release-mgmt
     users-manuals
     license-requirements