diff options
author | Chris Johns <chrisj@rtems.org> | 2018-05-20 08:32:42 +1200 |
---|---|---|
committer | Chris Johns <chrisj@rtems.org> | 2018-11-03 16:36:32 +1100 |
commit | 8b67c9135cb78abc394d43e449ccc6387c73f014 (patch) | |
tree | 6d4a21905a43cc4f476a1d6013baf151ce39ade5 /user | |
parent | 21c1a4492a6c437d057c3af605278e9ba5edcac0 (diff) |
user: Add RTEMS executable and test documentation.
Diffstat (limited to 'user')
-rw-r--r-- | user/exe/debugging.rst | 119 | ||||
-rw-r--r-- | user/exe/executables.rst | 102 | ||||
-rw-r--r-- | user/exe/execution.rst | 49 | ||||
-rw-r--r-- | user/exe/index.rst | 19 | ||||
-rw-r--r-- | user/exe/initialization.rst | 130 | ||||
-rw-r--r-- | user/index.rst | 6 | ||||
-rw-r--r-- | user/test/create.rst | 7 | ||||
-rw-r--r-- | user/test/index.rst | 10 | ||||
-rw-r--r-- | user/test/running.rst | 26 | ||||
-rw-r--r-- | user/testing/configuration.rst | 312 | ||||
-rw-r--r-- | user/testing/consoles.rst | 66 | ||||
-rw-r--r-- | user/testing/gdb-jtag.rst | 28 | ||||
-rw-r--r-- | user/testing/index.rst | 46 | ||||
-rw-r--r-- | user/testing/simulation.rst | 27 | ||||
-rw-r--r-- | user/testing/tests.rst | 210 | ||||
-rw-r--r-- | user/testing/tftp.rst | 257 | ||||
-rw-r--r-- | user/tools/tester.rst | 4 |
17 files changed, 1371 insertions, 47 deletions
diff --git a/user/exe/debugging.rst b/user/exe/debugging.rst new file mode 100644 index 0000000..3d1aa38 --- /dev/null +++ b/user/exe/debugging.rst @@ -0,0 +1,119 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +Debugging +========= +.. index:: Debugging + +An RTEMS executable is debugged by loading the code, data and read-only data +into a target with a debugger connected. The debugger running on a host +computer accesses the ELF file reading the debug information it contains. + +The executable being debugged needs to be built with the compiler and linker +debug options enabled. Debug information makes the ELF executable file large +but it does not change the binary footprint of the executable when resident in +the target. Target boot loaders and file conversion tools extract the binary +code, data and read-only data to create the file embedded on the target. + +An ELF executable built with debug information contains DWARF debug +information. DWARF is a detailed description of the executable a debugger uses +to locate functions, find data, understand the type and structure of a +variable, and know how much entry code every call has. The debugger uses this +information to set breaks points, step functions, step instructions, view the +data and much more. + +We recommend the compiler and linker debug options are always enabled. An ELF +file with debug information can be used to investigate a crash report from a +production system if the production ELF image is archived. The RTEMS tools +chain provides tools that can take an address from a crash dump and find the +corresponding instruction and source line. The extra size the debug information +adds does not effect the target footprint and the extra size on a host is small +compared to the benefits it brings. + +A desktop or server operating system's kernel hosts the executable being +debugged handling the interaction with the executable and the debugger. The +debugger knows how to communicate to the kernel to get the information it +needs. Debugging an embedded executable needs an extra piece called an agent to +connect the target to the debugger. The agent provides a standard remote interface to +the debugger and an agent specific connection to the target. + +.. _fig-exe-debug: + +.. figure:: ../../images/user/exe-debug.png + :width: 80% + :alt: Embedded Executable Debugging + :figclass: align-center + + Embedded Executable Debugging + +The RTEMS tool chain provides the GNU debugger GDB. GDB has a remote protocol +that can run over networks using TCP and UDP protocols. The GDB remote protocol +is available in a number of open source and commercial debugging +solutions. Network debugging using the remote protocol helps setting up a +laboratory, the targets can be remote from the developers desktop allowing for +better control of the target hardware while avoiding the need to plug devices +in to an expensive desktop or server machine. + +The following are some examples of GDB and GDB server environments RTEMS +supports. + +.. index:: QEMU + +QEMU contains a debugging agent for the target being simulated. A QEMU command +line option enables a GDB server and the simulator manages the interaction with +the target processor and it's memory and caches. + +.. _fig-exe-debug-qemu: + +.. figure:: ../../images/user/exe-debug-qemu.png + :width: 70% + :alt: QEMU Executable Debugging + :figclass: align-center + + QEMU Executable Debugging + +.. index:: OpenOCD +.. index:: JTAG + +OpenOCD is a JTAG debugging package that interfaces to a wide of JTAG +pods. JTAG is a low level high speed serial interface modern processors provide +as a means of controlling the core processing logic. The features available depend on +the architecture and processor. Typical functions include: + +#. Processor control and register access + +#. System level register access to allow SOC initialization + +#. General address space access + +#. Cache and MMU control + +#. Break and watch points + +.. _fig-exe-debug-qemu: + +.. figure:: ../../images/user/exe-debug-jtag.png + :width: 70% + :alt: OpenOCD JTAG Executable Debugging + :figclass: align-center + + OpenOCD JTAG Executable Debugging + +.. index:: libdebugger + +The RTEMS kernel has a debugging agent called ``libdebugger``. This is a +software based agent that runs within RTEMS using network services to provide a +remote GDB protocol interface. A growing number of architectures are +supported. The RTEMS debugging agent is for application development providing +thread aware stop model debug experience. + +.. _fig-exe-debug-libdebugger: + +.. figure:: ../../images/user/exe-debug-libdebugger.png + :width: 70% + :alt: Libdebugger Executable Debugging + :figclass: align-center + + Libdebugger Executable Debugging diff --git a/user/exe/executables.rst b/user/exe/executables.rst new file mode 100644 index 0000000..d911006 --- /dev/null +++ b/user/exe/executables.rst @@ -0,0 +1,102 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +RTEMS Executable +================ +.. index:: RTEMS Executable + +Running executables is the most important part of working with RTEMS, it is +after all how you run your application and use the RTEMS kernel services. + +An RTEMS executable is embedded in a target and executing an embedded +executable has challenges not faced when executing software on a desktop or +server computer. A desktop or server operating system kernel provides all the +support needed to bring an executable's code and data into a process's address +space passing control to it and cleaning up when it exits. An embedded target +has to provide similar functionality to execute an embedded executable. + +An RTEMS Source Builder (RSB) built RTEMS tool chain is used to create RTEMS +executables. The tool chain executable creates a fixed position statically +linked Extendable Loader Format (ELF) file that contains the RTEMS kernel, +standard libraries, 3rd party libraries and application code. RTEMS executes in +a single address space which means it does not support the ``fork`` or ``exec`` +system calls so statically linking all the code is the easiest and best way to +create an executable. + +An RTEMS application is constructed vertically with the RTEMS kernel, BSP +support code and drivers close to the hardware, above which sit the RTEMS +Application Programming Interfaces (API) for control of threads, mutex and +other resources an application may use. Middle-ware services like networking, +interpreted languages, and protocol stacks sit between the RTEMS APIs and the +application components. The software built into an executable can be see as a +vertical software stack. + +.. _fig-exe-vert-stack: + +.. figure:: ../../images/user/exe-vert-stack.png + :width: 35% + :alt: Vertical Software Stack + :figclass: align-center + + Vertical Software Stack + +Building an Application +======================= +.. index:: Building an Application + +RTEMS views any code it is running and using it's interfaces as an +application. RTEMS conforms to a number of international standards such as +POSIX and can build and run portable code written in languages such as C, C++ +and Ada. + +Applications are built from source into ELF object files, 3rd party packages +can be built as libraries or they can be imported as source into an application +code base. The application, 3rd party packages, RTEMS and standard libraries +are linked to create the RTEMS executable. The executable is transferred to the +target and a bootloader loads it from the non-volatile storage into RAM or the +code is executed in place in the non-volatile storage. The target hardware +defines what happens. + +.. _fig-exe-app: + +.. figure:: ../../images/user/exe-app.png + :width: 90% + :alt: Building an Application + :figclass: align-center + + Building an Application + +The standard and 3rd party libraries are a collection of object files built +using the same set of tools the application source is compiled with. The +package collects it's object files into an archive or library. + +RTEMS does not provide a standard application build system. The RTEMS ecosystem +provides support so a range of build systems can be used. Applications can be +built with ``make``, ``autotools``, ``cmake``, ``waf`` and more. User should +select a build system that meets their project, system, corporate or personal +needs. + +Machine Flags and ABI +--------------------- +.. index:: Machine flags +.. index:: Application Binary Interface +.. index:: ABI + + +All code in an RTEMS executable must be built with the same machine flags. The +machine flags control the instruction set and application binary interface +(ABI) the compiler generates. As the executable is statically linked all code +must use the same instruction set the hardware is configured to support and all +code must conform to the same ABI. Any variation can result in unpredictable +behavior such as crashes, failures or lock ups. It is recommend an executable +is built with the same or equivalent tool set. Mixing of tool set versions can +also result in undefined behavior. The RTEMS tool ``rtems-execinfo`` can audit +an RTEMS executable and list the machine flags and compilers used. + +RTEMS by default does not support instruction emulation for unsupported +instructions. RTEMS applications are normally built from source so binary +compatibility is not as important as performance. Instruction emulation is +costly to execute and rebuilding the executable with the correct instruction +set only needs to be done once. diff --git a/user/exe/execution.rst b/user/exe/execution.rst new file mode 100644 index 0000000..029bd24 --- /dev/null +++ b/user/exe/execution.rst @@ -0,0 +1,49 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +Target Execution +================ +.. index:: Target Execution + +Fixed position statically linked executables have a fixed address in a target's +address space. The location in the address space for code, data and read-only +data is fixed. The BSP defines the memory map and it is set by the BSP +developer based on the target's hardware requirements and it's bootloader. + +Targets typically contains a bootloader that is executed after the target's +processor exits reset. A bootloader is specific to a target's processor and +hardware configuration and is responsible for the low level initialization of +the hardware resources needed to load and execute an operating system's +kernel. In the case of RTEMS this is the RTEMS executable. + +Bootloaders vary in size, complexity and functionality. Some architectures have +a number of bootloader stages and others have only minimal support. An example +of a high end system is Xilinx's Zynq processor with three stages. First a mask +ROM in the System On Chip (SOC) executes after reset loading a first stage +bootloader (FSBL) from an SD card, QSPI flash or NAND flash depending on +signals connected to the device. The FSBL loads a second stage bootloader +(SSBL) such as U-Boot and this loads the kernel. U-Boot can be configured to +load a kernel from a range of media and file system formats as well as over a +network using a number of protocols. This structure provides flexibility at the +system level to support development environments such as a workshop or +laboratory through to tightly control production configurations. + +Bootloaders often have custom formats for the executable image they load. The +formats can be simple to keep the bootloader simple or complex to support +check-sums, encryption or redundancy in case an image becomes corrupted. A +bootloader often provides a host tool that creates the required file from the +RTEMS executable's ELF file. + +If RTEMS is to run from RAM the bootloader reads the image and loads the code, +initialized data and read-only data into the RAM and then jumps to a known +entry point. If the code is executed from non-volatile storage the process to +write the image into that storage will have extracted the various binary parts +and written those to the correct location. + +The important point to note is the binary parts of the executable are somehow +loaded into the target's address space ready to execute. The way this done may +vary but the out come is always the same, the binary code, data and read-only +data is resident in the processor's address space at the BSP defined +addresses. diff --git a/user/exe/index.rst b/user/exe/index.rst new file mode 100644 index 0000000..c40b773 --- /dev/null +++ b/user/exe/index.rst @@ -0,0 +1,19 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +Executables +*********** +.. index:: Executable +.. index:: Embedded executable + +This section discusses what an RTEMS executable is and what happens when you +execute it in a target. The section discusses how an application executable is +created, what happens when an executable is loaded and run as well as +debugging an execiutable. + +.. include:: executables.rst +.. include:: execution.rst +.. include:: initialization.rst +.. include:: debugging.rst diff --git a/user/exe/initialization.rst b/user/exe/initialization.rst new file mode 100644 index 0000000..9ecee16 --- /dev/null +++ b/user/exe/initialization.rst @@ -0,0 +1,130 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +BSP Initialization +================== +.. index:: BSP Initialization + +The bootloader jumps or calls the RTEMS executable's entry point, normally a +fixed address. The BSP entry point or start up code performs: + +#. Low level processor specific initialization that such as setting control + registers so the processor is operating in a mode RTEMS is built for + +#. Cache flushing, clearing and invalidation + +#. Memory management unit (MMU) set up if required + +#. Clear the uninitialized data section + +#. Process a command line if supported by the bootloader + +#. Call ``bootcard`` which disabled interrupts, saves away a command line if + the BSP supports it then call the RTEMS kernel early initialize entry point + ``rtems_initialize_executive``. This call never returns. + +Further BSP initialization happens as part of RTEMS kernel's System +Initialization process. The following handlers are declared and if provided are +placed at the beginning of the initialization handler list. The BSP can +provides: + +``bsp_work_area_initialize`` + This function determines the amount of memory that can be given to RTEMS for + the workspace and the C library heap which ``malloc`` uses. The call + typically uses the ``bsp_work_area_initialize_default`` to perform actually + perform the initialization. + +``bsp_start`` + This function is specialized for each architecture and even for some BSPs. It + performs the low level initialization RTEMS needs so it can run on the + architecture and BSP. + +``bsp_predriver_hook`` + This function can be used to initialize hardware drivers depend on such as + configuring an interrupt controller. The default version is empty and does + nothing. + +BSPs all perform similar operations with common functionality and the RTEMS +kernel provides common code that can be shared between BSPs. The use of the +common code is encouraged for all new BSPs. + +RTEMS Initialization +==================== +.. index:: RTEMS Initialization + +The RTEMS kernel initialization is: + +#. Invoke the registered system initialization handlers + +#. Set the system state to **up** + +#. If the kernel supports SMP request multitasking start. All online cores are + transferred to the **ready to start multitasking** state. + +#. Start threaded multitasking. RTEMS starts multitasking by getting the first + thread to run and dispatching it. + +C++ static object constructors are called in the context of the first running +thread before the thread body is entered. + +System Initialization Handlers +------------------------------ + +RTEMS supports the automatic registration of services used in +applications. This method of initialization automatically configures RTEMS with +only the services used in an application. There is no manual configuration of +services used and no updating of initialization function tables. + +RTEMS uses specialized sections in the ELF executable to perform this task. The +system is based on the `FreeBSD SYSINT Framework +<https://www.freebsd.org/doc/en/books/arch-handbook/sysinit.html>`_. Ordered +initialization is performed before multitasking is started. + +The RTEMS Tool ``rtems-exeinfo`` can provide some detail about the registered +handlers. The following shows the initialization handlers for the *hello world* +sample application in the RTEMS kernel's testsuite:: + + $ rtems-exeinfo --init arm-rtems5/c/xilinx_zynq_zedboard/testsuites/samples/hello.exe + RTEMS Executable Info 5.5416cfa39dd6 + rtems-exeinfo --init arm-rtems5/c/xilinx_zynq_zedboard/testsuites/samples/hello.exe + exe: arm-rtems5/c/xilinx_zynq_zedboard/testsuites/samples/hello.exe + + Compilation: + Producers: 2 + | GNU AS 2.31.1: 14 objects + | GNU C11 7.3.0 20180125 (RTEMS 5, RSB e55769c64cf1a201588565a5662deafe3f1ccdcc, Newlib 103b055035fea328f8bc7826801760fb1c055683): 284 objects + Common flags: 4 + | -march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard + + Init sections: 2 + .init_array + 0x001047c1 frame_dummy + .rtemsroset + 0x00104c05 bsp_work_area_initialize + 0x00104c41 bsp_start + 0x0010eb45 zynq_debug_console_init + 0x0010ec19 rtems_counter_sysinit + 0x0010b779 _User_extensions_Handler_initialization + 0x0010c66d rtems_initialize_data_structures + 0x00107751 _RTEMS_tasks_Manager_initialization + 0x0010d4f5 _POSIX_Keys_Manager_initialization + 0x0010dd09 _Thread_Create_idle + 0x0010cf01 rtems_libio_init + 0x001053a5 rtems_filesystem_initialize + 0x0010546d _Console_simple_Initialize + 0x0010c715 _IO_Initialize_all_drivers + 0x001076d5 _RTEMS_tasks_Initialize_user_tasks_body + 0x0010cfa9 rtems_libio_post_driver + +The section ``.rtemsroset`` lists the handlers called in order. The handlers +can be split into the BSP initialization handlers that start the BSP: + +- ``bsp_work_area_initialize`` +- ``bsp_start`` +- ``zynq_debug_console_init`` +- ``rtems_counter_sysinit`` + +And the remainder are handlers for services used by the application. The list +varies based on the services the application uses. diff --git a/user/index.rst b/user/index.rst index 48870a1..0f60166 100644 --- a/user/index.rst +++ b/user/index.rst @@ -51,10 +51,12 @@ to the Community Project hosted at http://www.rtems.org/. hardware/index bsps/index - tools/index - + exe/index + testing/index tracing/index + tools/index + support/index glossary/index diff --git a/user/test/create.rst b/user/test/create.rst deleted file mode 100644 index b86453a..0000000 --- a/user/test/create.rst +++ /dev/null @@ -1,7 +0,0 @@ -.. comment SPDX-License-Identifier: CC-BY-SA-4.0 - -Creating A Test -=============== -.. index:: Creating a Test - -XXX: How to create a test. diff --git a/user/test/index.rst b/user/test/index.rst deleted file mode 100644 index 7ae43fe..0000000 --- a/user/test/index.rst +++ /dev/null @@ -1,10 +0,0 @@ -.. comment SPDX-License-Identifier: CC-BY-SA-4.0 - -Test Suite -********** - -XXX: All about the test suite. - - -.. include:: running.rst -.. include:: create.rst diff --git a/user/test/running.rst b/user/test/running.rst deleted file mode 100644 index 5184ae1..0000000 --- a/user/test/running.rst +++ /dev/null @@ -1,26 +0,0 @@ -.. comment SPDX-License-Identifier: CC-BY-SA-4.0 - -Running -======== -.. index:: Running a Test - -XXX: How to run tests via waf. - - -Host ----- - -XXX: Host-based tests - - -Simulation ----------- -.. index:: Test Simulation - -XXX: Simulator - - - -Hardware --------- -XXX: Running on real hardware. diff --git a/user/testing/configuration.rst b/user/testing/configuration.rst new file mode 100644 index 0000000..a58b373 --- /dev/null +++ b/user/testing/configuration.rst @@ -0,0 +1,312 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +Tester Configuration +-------------------- + +The RTEMS Tester and RTEMS Run are controlled by configuration data and +scripts. The user specifies a BSP on the command line using the ``--rtems-bsp`` +option as well as optionally specifying a user configuration file using +``--user-config``. + +The Figure :ref:`fig-tester-config-1` shows the various sources of +configuration data and their format. The ``ini`` files are the standard INI +format, the ``mc`` are the internal RTEMS Toolkit's Macro format, and ``cfg`` +is the RTEMS Toolkit's Configuration script format, the same format used by the +RTEMS Source Builder. + +.. _fig-tester-config-1: + +.. figure:: ../../images/user/test-cfg-1.png + :width: 50% + :alt: RTEMS Tester and Run Configuration Files + :figclass: align-center + + RTEMS Tester and Run Configuration Files + +Configuration data is held in a macro database keyed on the macro name. Macros +can be expanded in configuration scripts using the syntax ``%{name}``. The +macro database is layered using maps. The defaults and values created when a +configure script runs live the in the ``global`` map. Values read from the BSP +and User INI configuration files are loaded into maps based on the BSP +name. This lets a single User configuration file contain specialized +configuration values for a number of BSPs and the tester and run commands +select the values based on the selected BSP. Macros are expanded using the BSP +map first giving those values the highest priority. User defined values are +loaded after the BSP configuration values overwriting them letting a user +speckles a BSP's default configuration for their local needs. + +Figure :ref:`fig-tester-config-2` shows the configuration loading and script +execution order. + +.. _fig-tester-config-2: + +.. figure:: ../../images/user/test-cfg-2.png + :width: 50% + :alt: RTEMS Tester and Run Configuration Load and Execute Sequence + :figclass: align-center + + RTEMS Tester and Run Configuration Load and Execute Sequence + +Defaults +^^^^^^^^ + +The RTEMS Tester and RTEMS Run are primed using defaults from the file +``rtems/testing/testing.mc``. All default settings can be overridden in a BSP or +User configuration file. + +.. index:: BSP configuration, User configuration +BSP and User Configuration +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The BSP and User configuration files are INI format files. The BSP +configuration file has to have an INI section that is the name of the BSP +passed on the command line. The section has the following mandatory values: + +.. index:: bsp +``bsp`` + The name of the BSP. The BSP name is used to create a macro map to hold the + BSP's configuration data. Typically this is the same as the BSP name used on + the command line. + +.. index:: arch +``arch`` + The name of the BSP architecture. This is need for the GDB configuration + scripts where the architecture specific GDB needs to run. It is mandatory so + the *arch/bsp* standard RTEMS BSP string can be used. + +.. index:: tester +``tester`` + The tester or run configuration script. This is the name of the configuration + script the RTEMS Tester or RTEMS Run executes as a back end. The ``tester`` + value is typically of the form ``%{_rtscripts}/<script>`` where ``<script>`` + is name of the back end script to be run. + +Target commands support expansion of specific tags to provide a convenient way +for users to customize a local test environment. The parameters expanded are: + +.. index:: @ARCH@ +``@ARCH@`` + The BSP architecture. + +.. index:: @BSP@ +``@BSP@`` + The BSP's name set by the ``bsp`` value. + +.. index:: @EXE@ +``@EXE@`` + The executable name as an absolute path + +.. index:: @FEXE@ +``@FEXE@`` + The filtered executable if a ``target_exe_filter`` is provided else the + executable's file name. + +The following are optional and depend on the back end being used and the local +target hardware set up: + +.. index:: jobs +``jobs`` + The jobs value sets the number of jobs that can be run at once. This setting + only effects the RTEMS Tester. The tester can run up to the ``jobs`` value of + tests concurrently. If the tester back end is a simulator running a job on + each available core lowers the total test time. Overloading a machine with + too many simulators running in parallel can slow down each simulation and + test timeouts may be recorded. + +.. index:: bsp_tty_dev +``bsp_tty_dev`` + The BSP's tty device. This can be a real device on the host machine the + executable is being run from or it can be a telnet server and port defined + using the stand host format. See :ref:`tester-consoles` for details. + +.. index:: target_pretest_command +``target_pretest_command`` + The pre-test command is a host shell command that is called before each test + runs. It can be used to construct a suitable environment or image needed by a + simulator or target. The RTEMS executate being run is provided as an argument + and the bootloader specific format is the output. + + .. index:: target_posttest_command +``target_posttest_command`` + The post-test command is a host shell command that is called after each test + has finished. It can be used to destroy any environment or image created by + the pre-test command. + +.. index:: target_exe_filter +``target_exe_filter`` + The target executable filter transforms the executable name into a filtered + executable name. This filter lets the tester or run command track the name of + any generated file a pre-test command may generate. The syntax is a simplified + ``sed`` regular expression. The first character is a delimiter and there must + be 2 sections therefore 3 delimiter. The first section is a Python regular + expression and the second section is plain text that replaces anywhere the + regular expression matches. For example ``/\.exe/.exe.img/`` will search for + ``.exe`` in the executable name and replace it with ``.exe.img``. Note, there + is no need to escape the text in the second part, it is just plain test. + +.. index:: test_restarts +``test_restarts`` + The number of restarts before the test is considered ``invalid``. Currently + not used. + +.. index:: target_reset_regex +``target_reset_regex`` + The target reset regular expression. This is a `Python regular expression + <https://docs.python.org/2/library/re.html#regular-expression-syntax>`_ used + to filter the console input. If a match is made something has happened during + the boot process that requires a reset. The ``target_reset_command`` is + issued to perform the reset. Typically this field looks for boot loader error + messages that indicate the boot process as failed. + +.. index:: target_start_regex +``target_start_regex`` + + The target start regular expression. This is a Python regular expression to + filter the console input to asynchronously detect if a target has reset. If a + board crashes running a test or at any point reset this filter detects the + restart and ends the test with a suitable result. + +.. index:: target_on_command +``target_on_command`` + The target on command is a host shell command that is called before the first + test. This command powers on a target. Targets should be left powered off + when not running tests or the target may request TFTP downloads that are for + another target interfering with those test results. We recommend you + implement this command as a target off command, a pause, then a target on + command. + +.. index:: target_off_command +``target_off_command`` + The target off command is a host shell command that is called after the last + test powering off the target. + +.. index:: target_reset_command +``target_reset_command`` + The target reset command is a host shell command that is called when the + target needs to be reset. This command can power cycle the target or toggle a + reset signal connected to the target. If you are power cycling a target make + sure you have a suitable pause to let the target completely power down. + +.. _tester-config-scripts: + +Configuration Scripts +^^^^^^^^^^^^^^^^^^^^^ + +Configuration scripts are provided for each supported RTEMS Tester and RTEMS +Run back end and console management. The scripts are in the standard RTEMS +Toolkit Configuration Script format. Please refer to the RTEMS Source Builder +documentation for the basic scripting syntax and usage. + +The RTEMS Tester and RTEMS Run specializes the standard configuration syntax +providing a directive for the console and each supported back end. The +supported directives are: + +- ``%console`` +- ``%execute`` +- ``%gdb`` +- ``%tftp`` + +.. _tester-config-console: +.. index:: Console, %console + +Console +~~~~~~~ + +The ``%console`` configures the console used to access the target's +console. The console can be a process's ``stdout``, a termios tty on Unix and +MacOS and Telnet on all hosts. The directive accepts: + +``stdio`` + The standard output stream from the executing processing. + +``tty <dev> <settings>`` + The name of the ``tty`` to open and use. The ``tty`` device or ``<dev>`` can + be a *termio* device and the ``<settings>`` are standard termios values. + + The Python termios document provides details of the settings that can be + controlled. The settings are a single string where prefix the value with + ``~`` negates the setting. Setting are: + + - ``B115200`` (an example buadrate) + - ``BRKINT`` + - ``IGNBRK`` + - ``IGNCR`` + - ``ICANON`` + - ``ISIG`` + - ``IEXTEN`` + - ``ECHO`` + - ``CLOCAL`` + - ``CRTSCTS`` + - ``VMIN=<value>`` + - ``VTIME=<value`` + +A example in a configuration script is:: + + %define bsp_tty_dev /dev/ttyUSB2 + %define bsp_tty_settings B115200,~BRKINT,IGNBRK,IGNCR,~ICANON,~ISIG,~IEXTEN,~ECHO,CLOCAL,~CRTSCTS,VMIN=1,VTIME=2 + +A example BSP or User configuration file is:: + + [bsp-special] + bsp = example-bsp + bsp_tty_dev = /dev/ttyUSB2 + bsp_tty_settings = B115200,~BRKINT,IGNBRK,IGNCR,~ICANON,~ISIG,~IEXTEN,~ECHO,CLOCAL,~CRTSCTS,VMIN=1,VTIME=2 + +The console directive is managed in the ``%{_rtscripts}/console.cfg`` +configuration script. If the ``%{console_stdio}`` is defined the console will +be ``stdio`` else the console will be the BSP console or ``%{bsp_tty_dev}``. + +Telnet can be combined with the ``ser2net`` daemon to remotely access a +target's physical serial UART interface. + +.. _tester-config-execute: +.. index:: Execute, %execute + +Execute +~~~~~~~ + +The ``%execute`` directive executes a command for each rest. The execute forks +the command and arguments supplied to the execute directive and captures the +``stdout`` stream as the console. If the console directive is set to ``stdout`` +the sub-processes ``stdout`` stream is used as the console. + +The RTEMS Tester will run parallel tests as jobs. + +An example is:: + + %execute %{run_cmd} %{run_opts} %{test_executable} %{test_executable_opts} + +.. _tester-config-gdb: +.. index:: GDB, %gdb + +GDB +~~~ + +The ``%gdb`` directive executes GDB in the machine interface mode give the +RTEMS Tester and RTEMS Run commands control. The console is taken from +GDB if it is ``stdout``. + +The RTEMS Tester will run parallel tests as jobs. + +An example is:: + + %gdb %{gdb_cmd} %{test_executable} %{gdb_script} + +.. _tester-config-tftp: +.. index:: TFTP, %tftp + +TFTP +~~~~ + +The ``%tftp`` directive starts a TFTP session on a specified port sending the +test executable to the target over a networking using the TFTP protocol. + +The RTEMS Tester will run only one test at a time. There is just one physical +board running the test. + +An example is:: + + %tftp %{test_executable} %{tftp_port} diff --git a/user/testing/consoles.rst b/user/testing/consoles.rst new file mode 100644 index 0000000..9fa84ee --- /dev/null +++ b/user/testing/consoles.rst @@ -0,0 +1,66 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +.. _tester-consoles: + +Consoles +-------- + +The RTEMS Tester uses the target's console output to determine the state of a +test. Console interfaces vary depending on the testing mode, the BSP, and the +target hardware. + +Consoles for simulator work best if mapped to the simulator's ``stdout`` +interface. The RTEMS Tester can capture and process the ``stdout`` data from a +simulator while it is running. + +Target hardware console interfaces can vary. The most universal and stable +interface target hardware is a UART interface. There are a number of physical +interfaces for UART data these days. They are: + +#. RS232 + +#. TTL + +#. USB + +RS232 is still present on a number of targets. The best solution is to use a +RS232 to USB pod and convert the port to USB. + +TTL is common on a number of boards where cost is important. A console +interface is typically a development tool and removing the extra devices need +to convert the signal to RS232 or directly to USB is not needed on production +builds of the target. There is a standard header pin out for TTL UART consoles +and you can purchase low cost cables with the header and a built in UART to USB +converter. The cables come is different voltage levels so make sure you check +and use the correct voltage level. + +The USB interface on a target is typcially a slave or OTG interface and all you +need to a standard USB cable. + +We recommend a low cost and low power device to be a terminal server. A +Raspberry Pi or similar low cost computer running Linux can be set up quickly +and with a powered USB hub and can support a number of USB UART ports. A USB +hub with a high power port is recommended that can suppy the Raspberry Pi. + +The open source daemon ``ser2net`` is easy to configure to map the USB UART +ports to the Telnet protocol. There is no need for security because a typical +test environment is part of a lab network that should be partitioned off from +an enginnering or corportate network and not directly connected to the +internet. + +A test set up like this lets you place a terminal server close to your target +hardware providing you with the flexibility to select where you run the RTEMS +Tester. It could be your desktop or an expensive fast host machine in a server +rack. None of this equipment needs to directly interface to the target +hardware. + +The RTEMS Tester directly supports the telnet protcol as a console and can +interface to the ``ser1net`` server. The telnet console will poll the server +waiting for the remote port to connect. If the terminal server ``ser2net`` does +not have a ``tty`` device it will not listen on the port assigned to that +``tty``. A USB ``tty`` can come and go depending on the power state of the +hardware and the target hardware's design and this can cause timing issues if +the target hardware is power cycled as part of a reset process. diff --git a/user/testing/gdb-jtag.rst b/user/testing/gdb-jtag.rst new file mode 100644 index 0000000..367d1a2 --- /dev/null +++ b/user/testing/gdb-jtag.rst @@ -0,0 +1,28 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +GDB and JTAG +------------ +.. index:: GDB, JTAG, Testing + +GDB with JTAG provides a low level way to runs tests on hardware with limited +resources. The RTEMS Tester runs and controls an instance of GDB per test and +GDB connects via the GDB remote protocol to a GDB server that interfaces to the +JTAG port of a target. + +.. _fig-tester-gdb-jtag: + +.. figure:: ../../images/user/test-gdb-jtag.png + :width: 35% + :alt: RTEMS Tester using GDB and JTAG + :figclass: align-center + + RTEMS Tester using GDB and JTAG + +The :ref:`fig-tester-gdb-jtag` figure shows the structure of RTEMS Testing +using GDB and JTAG. The executables are built and the ``rtems-test`` command is +run from the top of the build directory. The RTEMS Tester executes the BSP +architecture's GDB and expects the user to provide a ``gdb-script`` to connect +t the JTAG GDB server. diff --git a/user/testing/index.rst b/user/testing/index.rst new file mode 100644 index 0000000..f3a6846 --- /dev/null +++ b/user/testing/index.rst @@ -0,0 +1,46 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +Testing +******* + +RTEMS developers run test executables when adding new features or testing a bug +fix. All tests are run to make sure changes do not introduce regressions. Users +can run the RTEMS tests to be certain the build of the kernel they have is +functioning. + +The section describes using and configuring the RTEMS Tester and RTEMS Run +tools, the types of laboratory set ups supported and how to add your BSP to the +framework. The tools command line interfaces are detailed in +:ref:`rtems-tester-command`. + +An RTEMS Test is an RTEMS executable where the application code is a +test. Tests in RTEMS print banners to the console to indicate the configuration +of the test and if it has start and finished. + +The RTEMS Tools Project provides the RTEMS Tester and RTEMS Run tools. The +RTEMS Tester command is ``rtems-test`` and the RTEMS Run command is +``rtems-run``. These commands manage the complexity of running embedded +executables. The commands provide a consistent command line interface to a +testing framework that supports the various run time and testing scenarios we +encounter such as simulators, GDB and executing directly on target hardware. + +The RTEMS kernel code contains an extensive set of tests to exercise and test +the RTEMS kernel. The tests check functionality, provide coverage testing and +make sure the kernel is operating as intended on your target system. The +testsuite has support to make creating a test simple and uniform. + +The tests are built by adding ``--enable-tests`` to the RTEMS build +configuration command line. There are over 600 tests and building them does +extend the RTEMS kernel's build time and use more disk space but it worth +building and running them. The RTEMS test executables have the ``.exe`` file +extension. + +.. include:: tests.rst +.. include:: configuration.rst +.. include:: consoles.rst +.. include:: simulation.rst +.. include:: gdb-jtag.rst +.. include:: tftp.rst diff --git a/user/testing/simulation.rst b/user/testing/simulation.rst new file mode 100644 index 0000000..865673f --- /dev/null +++ b/user/testing/simulation.rst @@ -0,0 +1,27 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +Simulation +---------- +.. index:: Simulation, Testing + +Simulation is a important regression and development tool for RTEMS. Developers +use simulation to work on core parts of RTEMS as it provides excellent +debugging supporting. Simulation run via the RTEMS Tester allows a test to run +on each core of your testing host machine lower the time to run all tests. + +.. _fig-tester-simulation: + +.. figure:: ../../images/user/test-simulation.png + :width: 30% + :alt: RTEMS Tester Simulation + :figclass: align-center + + RTEMS Tester Simulation + +The :ref:`fig-tester-simulation` figure shows the structure of RTEMS Testing +using simulation. The executables are built and the ``rtems-test`` command is +run from the top of the build directory. The RTEMS Tester executes the +BSP specific simulator for each test capturing the output diff --git a/user/testing/tests.rst b/user/testing/tests.rst new file mode 100644 index 0000000..345e9c0 --- /dev/null +++ b/user/testing/tests.rst @@ -0,0 +1,210 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +Test Banners +------------ + +All test output banners or strings are embedded in each test and the test +outputs them to the BSP's console as it executes. The RTEMS Tester captures the +BSP's console and uses this information to manage the state of the executing +test. The banner strings are: + +.. _test-banner-begin: +.. index:: test begin, TEST BEGIN + +``*** BEGIN TEST <name> ***`` + The test has loaded, RTEMS has initialized and the test specific code is + about to start executing. The ``<name>`` field is the name of the test. The + test name is internal to the test and may not match the name of the + executable. The test name is informative and not used by the RTEMS Tester. + +.. _test-banner-end: +.. index:: test end, TEST END + +``*** END TEST <name> ***`` + The test has finished without error and has passed. The ``<name>`` field is + the name of the test. See the :ref:`Test Begin Banner <test-banner-begin>` + for details about the name. + +.. index:: test banner version, TEST VERSION + +``*** TEST VERSION: <version>`` + The test prints the RTEMS version return by the RTEMS Version API as + ``<version>``. All tests must match the first test's version or the Wrong + Version error count is incremented. + +.. _test-banner-state: +.. index:: test state, TEST STATE + +``*** TEST STATE: <state>`` + The test is tagged in the RTEMS sources with a special ``<state>`` for this + BSP. See :ref:`Test States <test-states>` for the list of possible + states. The state banner lets the RTEMS Tester categorize and manage the + test. For example a user input test typically needing user interaction may + never complete producing an *invalid* test result. A user input test is + terminated to avoid extended delays in a long test run. + +.. _test-banner-build: +.. index:: test build, TEST BUILD + +``*** TEST BUILD: <build>`` + The test prints the RTEMS build as a space separated series of labels as + ``<build>``. The build labels are created from the configuration settings in + the Super Score header file ``rtems/score/cputops.h``. All tests must match + the first test's build or the Wrong Build error count is incremented. + +.. _test-banner-tools: +.. index:: test tools, TEST TOOLS + +``*** TEST TOOLS: <version>`` + The test prints the RTEMS tools version returned the GGC internal macro + ``_VERSION_`` as ``<version>``. All tests must match the first test's tools + version string or the Wrong Tools error count is incremented. + +.. _test-states: +.. index:: Test states + +Test States +----------- + +The tests states are: + +.. index:: test state passed +``passed`` + The test start and end banners have been sent to the console. + +.. index:: test state failure +``failure`` + The test start banner has been sent to the console and no end banner has been + seen when a target restart is detected. + +.. index:: test state expected-fail +``excepted-fail`` + The test is tagged as ``expected-fail`` in the RTEMS sources for this BSP and + outputs the banner ``*** TEST STATE: EXPECTED_FAIL``. The test is known not + to pass on this BSP. The RTEMS Tester will let the test run as far as it + can and if the test passes it is recorded as a pass in the test results + otherwise it is recorded as *expected-fail*. + +.. index:: test state indeterminate +``indeterminate`` + The test is tagged as ``indeterminate`` in the RTEMS sources for this BSP and + outputs the banner ``*** TEST STATE: INDETERMINATE``. The test may or may not + pass so the result is not able to be determined. The RTEMS Tester will let + the test run as far as it can and record the result as indeterminate. + +.. index:: test state user-input +``user-input`` + The test is tagged as ``user-input`` in the RTEMS sources and outputs the + banner ``*** TEST STATE: USER_INPUT``. The RTEMS Tester will reset the target + if the target's configuration provides a target reset command. + +.. index:: test state benchmark +``benchmark`` + The test is tagged as ``benchmark`` in the RTEMS sources and outputs the + banner ``*** TEST STATE: BENCHMARK``. Benchmarks can take a while to run and + performance is not regression tested in RTEMS. The RTEMS Tester will reset + the target if the target's configuration provides a target reset command. + +.. index:: test state timeout +``timeout`` + The test start banner has been sent to the console and no end banner is seen + within the *timeout* period and the target has not restart. A default + *timeout* can be set in a target configuration, a user configuration or + provide on the RTEMS Tester's command line using the ``--timeout`` option. + +.. index:: test state invalid +``invalid`` + The test did not output a start banner and the RTEMS Tester has detected the + target has restarted. This means the executable did not load correctly, the + RTEMS kernel did not initialize or the RTEMS kernel configuration failed for + this BSP. + +Expected Test States +^^^^^^^^^^^^^^^^^^^^ + +A test's expected state is set in the RTEMS kernel's testsuite. The default for +a tested is to ``pass``. If a test is known to fail it can have it's state set +to ``expected-fail``. Setting tests that are known to fail to ``expected-fail`` +lets everyone know a failure is not to be countered and consider a regression. + +Expected test states are list in test configuration files that end with the +file extension ``.tcfg``. The testsuite supports global test configurations in +the ``testsuite/testdata`` directory. Global test states are applied to all +BSPs. BSPs can provide a test configuration that applies to just that BSP. + +The test configuration file format is:: + + state: test test test + +where ``test test test`` is a list of tests the state applies too. The ``state`` is one +of: + +``include`` + The test list is the name of a test configuration file to include + +``exclude`` + The tests listed are not build. This can happen if a BSP cannot support a + test. For example it does not have enough memory. + +``expected-fail`` + The tests listed are set to expected fail. The test will fail on the BSP + being built. + +``user-input`` + The tests listed require user input to run and are not supported by automatic + testers. + +``indeterminate`` + The tests listed may pass or may not, the result is not reliable. + +``benchmark`` + The tests listed are benchmarks. Benchmarks are flagged and not left to + run to completion because they may take too long. + + + +Test Builds +----------- + +The test reports the build of RTEMS being tested. The build are: + +.. index:: build default +``default`` + The build is the default. No RTEMS configure options have been used. + +.. index:: build posix +``posix`` + The build includes the POSIX API. The RTEMS configure option + ``--enable-posix`` has been used. The ``cpuopts.h`` define ``RTEMS_POSIX`` + has defined and it true. + +.. index:: build smp +``smp`` + The build is an SMP kernel. The RTEMS configure option ``--enable-smp`` has + been used. The ``cpuopts.h`` define ``RTEMS_SMP`` has defined and it true. + +.. index:: build mp +``mp`` + The build is an MP kernel. The RTEMS configure option + ``--enable-multiprocessing`` has been used. The ``cpuopts.h`` define + ``RTEMS_MULTIPROCESSING`` has defined and it true. + +.. index:: build paravirt +``paravirt`` + The build is a paravirtualization kernel. The ``cpuopts.h`` define + ``RTEMS_PARAVIRT`` has defined and it true. + +.. index:: build debug +``debug`` + The build includes kernel debugging support. The RTEMS configure option + ``--enable-debug`` has been used. The ``cpuopts.h`` define ``RTEMS_DEBUG`` + has defined and it true. + +.. index:: build profiling +``profiling`` + The build include profiling support. The RTEMS configure option + ``--enable-profiling`` has been used. The ``cpuopts.h`` define + ``RTEMS_PROFILING`` has defined and it true. diff --git a/user/testing/tftp.rst b/user/testing/tftp.rst new file mode 100644 index 0000000..ee2d3c6 --- /dev/null +++ b/user/testing/tftp.rst @@ -0,0 +1,257 @@ +.. comment SPDX-License-Identifier: CC-BY-SA-4.0 + +.. comment: Copyright (c) 2018 Chris Johns <chrisj@rtems.org> +.. comment: All rights reserved. + +TFTP and U-Boot +--------------- + +.. index:: TFTP, U-Boot, Testing + +TFTP and U-Boot provides a simple way to test RTEMS on a network capable +target. The RTEMS Tester starts a TFTP server for each test and the target's +boot monitor, in this case U-Boot request a file, any file, which the TFTP +server supplies. U-Boot loads the executable and boots it using a standard +U-Boot script. + +.. _fig-tester-tftp-u-boot: + +.. figure:: ../../images/user/test-tftp.png + :width: 35% + :alt: RTEMS Tester using TFTP and U-Boot + :figclass: align-center + + RTEMS Tester using TFTP and U-Boot. + +The :ref:`fig-tester-tftp-u-boot` figure shows the structure and control flow +of the RTEMS Tester using TFTP and U-boot. The executables are built and the +``rtems-test`` command is run from the top of the build directory. + +This test mode can only support a single test job running at once. You cannot +add more test target hardware and run the tests in parallel. + +Target Hardware +^^^^^^^^^^^^^^^ + +The RTEMS Tester TFTP and U-Boot method of testing requires: + +#. A target with network interface. + +#. U-Boot, iPXE or similar boot loader with network driver support for your + target hardware and support for the TFTP protocol. + +#. Network power of IO switch. + +#. Network DHCP server. + +#. Console interface cable that matches your target's console UART interface. + +#. Telnet terminal server. See :ref:`tester-consoles`. + +The network power or IO switch is a device that can control power or an IO pin +over a network connection using a script-able protocol such as Telnet or +curl. This device can be used with the target control commands. + +U-Boot Set Up +~~~~~~~~~~~~~ + +Obtain a working image of the U-Boot boot loader for your target. We suggest +you follow the instructions for you target. + +Configure U-Boot to network boot using the TFTP protocol. This is U-Boot script +for a Zedboard:: + + loadaddr=0x02000000 + uenvcmd=echo Booting RTEMS Zed from net; set autoload no; dhcp; set serverip 10.10.5.2; tftpboot zed/rtems.img; bootm; reset; + +The load address variable ``loadaddr`` is specific to the Zedboard and can be +found in the various examples scripts on the internet. The script then sets +U-Boot environment variable ``autoload`` to ``no`` causing DHCP to only request +a DHCP lease from the DHCP server. The script sets the ``serverip`` to the host +that will be running the RTEMS Tester then issues a TFTP request. The file name +can be anything because the RTEMS Tester ignores it sending the executable +image under test. Finally the script boots the download executable and if that +fails the catch all ``reset`` resets the board and starts the boot process +over. + +Test the target boots and U-Boot runs and obtains a valid DHCP lease. Manually +connect the console's telnet port. + +BSP Configuration +^^^^^^^^^^^^^^^^^ + +The BSP's configuration file must contain the standard fields: + +- ``bsp`` +- ``arch`` +- ``jobs`` - Must be set to ``1``. +- ``tester`` - Set to ``%{_rtscripts}/tftp.cfg`` + +For example the Zedboard's configuration is:: + + [xilinx_zynq_zedboard] + bsp = xilinx_zynq_zedboard + arch = arm + jobs = 1 + tester = %{_rtscripts}/tftp.cfg + +The TFTP configuration supports the following field's: + +``bsp_tty_dev`` + The target's tty console. For telnet this is a host and port pair written in + the standard networking format, for example ``serserver:12345``. + +``test_restarts`` + The number of restarts before the test is considered ``invalid``. + +``target_reset_regex`` + The target reset regular expression. This is a `Python regular expression + <https://docs.python.org/2/library/re.html#regular-expression-syntax>`_ used + to filter the console input. If a match is made something has happened during + the boot process that requires a reset. The ``target_reset_command`` + is issued to perform the reset. This field is typically looks for boot loader + error messages that indicate the boot process as failed. + +``target_start_regex`` + The target start regular expression. This also a Python regular expression to + filter the console input to detect if a target has reset. If a board crashes + running a test or at any point in time and reset this filter detects this as + happened and end the test with a suitable result. + +``target_on_command`` + The target on command is a host shell command that is called before the first + test. This command powers on a target. Targets should be left powered off + when not running tests or the target may request TFTP downloads that are for + another target interfering with those test results. We recommend you + implement this command as a target off command, a pause, then a target on + command. + +``target_off_command`` + The target off command is a host shell command that is called after the last + test powering off the target. + +``target_reset_command`` + The target reset command is a host shell command that is called when the + target needs to be reset. This command can power cycle the target or toggle a + reset signal connected to the target. If you are power cycling a target make + sure you have a suitable pause to let the target completely power down. + +``target_pretest_command`` + The target pretest command is a host shell comment that is called before the + test is run + +The commands in the listed fields can include parameters that are +substituted. The parameters are: + +``@ARCH@`` + The BSP architecture + +``@BSP@`` + The BSP's name + +``@EXE@`` + The executable name. + +``@FEXE@`` + The +. The + ``@ARCH`` is the + +substituted + +Some of these field are normally provided by a user's configuration. To do this +use:: + + requires = bsp_tty_dev, target_on_command, target_off_command, target_reset_command + +The ``requires`` value requires the user provide these settings in their +configuration file. + +The Zedboard's configuration file is:: + + [xilinx_zynq_zedboard] + bsp = xilinx_zynq_zedboard + arch = arm + jobs = 1 + tester = %{_rtscripts}/tftp.cfg + test_restarts = 3 + target_reset_regex = ^No ethernet found.*|^BOOTP broadcast 6.*|^.+complete\.+ TIMEOUT.* + target_start_regex = ^U-Boot SPL .* + requires = target_on_command, target_off_command, target_reset_command, bsp_tty_dev + +The ``target_start_regex`` searches for U-Boot's first console message. This +indicate the board can restarted. + +The ``target_reset_regex`` checks if no ethernet interface is found. This can +happen if U-Boot cannot detect the PHY device. It also checks if too many DHCP +requests happen and finally a check is made for any timeouts reported by +U-Boot. + +An example of a user configuration for the Zedboard is:: + + [xilinx_zynq_zedboard] + bsp_tty_dev = selserver:12345 + target_pretest_command = zynq-mkimg @EXE@ + target_exe_filter = /\.exe/.exe.img/ + target_on_command = power-ctl toggle-on 1 4 + target_off_command = power-ctl off 1 + target_reset_command = power-ctl toggle-on 1 3 + +TFTP Sequences +^^^^^^^^^^^^^^ + +Running a large number of tests on real hardware exposes a range of issues and +RTEMS Tester is designed to be tolerant of failures in booting or loading that +can happen, for example a hardware design. These sequence diagrams document +some of the sequences that can occur when errors happen. + +The simplest sequence is running a test. The target is powered on, the test is +loaded and executed and a pass or fail is determined: + +.. _fig-tester-tftp-seq-1: + +.. figure:: ../../images/user/test-tftp-seq-1.png + :width: 90% + :alt: Test Pass and Fail Sequence + :figclass: align-center + + Test Pass and Fail Sequences + +The target start filter triggers if a start condition is detected. This can +happen if the board crashes or resets with no output. If this happens +repeatedly the test result is invalid: + +.. _fig-tester-tftp-seq-2: + +.. figure:: ../../images/user/test-tftp-seq-2.png + :width: 80% + :alt: Target Start Filter Trigger + :figclass: align-center + + Target Start Filter Trigger + +The reset filter triggers if an error condition is found such as the bootloader +not being able to load the test executable. If the filter triggers the +``target_reset_command`` is run: + +.. _fig-tester-tftp-seq-3: + +.. figure:: ../../images/user/test-tftp-seq-3.png + :width: 50% + :alt: Target Reset Filter Trigger + :figclass: align-center + + Target Reset Filter Trigger + +If the RTEMS Tester does not detect a test has started it can restart the test +by resetting the target. The reset command can toggle an IO pin connected to +reset, request a JTAG pod issue a reset or turn the power off and on: + +.. _fig-tester-tftp-seq-4: + +.. figure:: ../../images/user/test-tftp-seq-4.png + :width: 60% + :alt: Target Timeout + :figclass: align-center + + Target Timeout diff --git a/user/tools/tester.rst b/user/tools/tester.rst index 0c07415..417daa5 100644 --- a/user/tools/tester.rst +++ b/user/tools/tester.rst @@ -5,8 +5,8 @@ .. _rtems-tester-command: -RTEMS Tester and Run Commands -============================= +RTEMS Tester and Run +==================== .. index:: Tools, rtems-test, rtems-run |