rt: Update the documentation.

1 files changed, 152 insertions, 76 deletions

diff --git a/doc/rtems-tester.txt b/doc/rtems-tester.txt
index 58e4c2c..286d6ee 100644
--- a/doc/rtems-tester.txt
+++ b/doc/rtems-tester.txt
@@ -9,7 +9,7 @@ RTEMS Tester
 image:images/rtemswhitebg.jpg["RTEMS",width="30%"]
 
 Chris Johns <chrisj@rtems.org>
-1.0, Janurary 2014
+1.0, June 2014
 
 RTEMS Tester
 ------------
@@ -32,7 +32,7 @@ the test result. The RTEMS Tester provides a framework to do this.
 Running all the RTEMS tests on your target is very important. It provides you
 with a tracable record your RTEMS version and its tools and working at the
 level the RTEMS development team expect when releasing RTEMS. Being able to
-easly run the tests and verify the results is critical is maintiaining a high
+easly run the tests and verify the results is critical in maintiaining a high
 standard.
 
 The RTEMS Tester contains:
@@ -46,6 +46,9 @@ The RTEMS Tester contains:
 IMPORTANT: If you have a problem please see the <<_bugs,reporting bugs>>
            section.
 
+This document is for users of the RTEMS Tester as well as those wanting to
+develop, extend and investigate the RTEMS Tester.
+
 License
 -------
 
@@ -75,15 +78,11 @@ $ mkdir -p development/rtems/test
 $ cd development/rtems/test
 -------------------------------------------------------------
 
-First fetch the RTEMS tester. During the development stages of this tool it is
-provided as a tarball. Point your browser to
-http://www.rtems.org/ftp/pub/rtems/people/chrisj/rtems-tester and download the
-latest version. The versions are dated and number if more than one release is
-made on a single day. Unpack the tarball and enter the directory:
+First fetch the RTEMS tester from the RTEMS Tools repository::
 
 -------------------------------------------------------------
-$ tar jxf rtems-test-20140117-1.tar.bz2
-$ cd rtems-test
+$ git git://git.rtems.org/rtems-tools.git rtems-tools.git
+$ cd rtems-tools.git/tester
 -------------------------------------------------------------
 
 Available BSPs
@@ -93,30 +92,41 @@ You can list the available BSP's with:
 
 -------------------------------------------------------------
 $ ./rtems-test --list-bsps
+  arm920
+  beagleboardxm
+  jmr3904-run
+  jmr3904
   mcf5235
+  psim-run
+  psim
   realview_pbx_a9_qemu
   sis-run
   sis
+  xilinx_zynq_a9_qemu
+  xilinx_zynq_a9_qemu_smp
   xilinx_zynq_zc706
+  xilinx_zynq_zc706_qemu
 -------------------------------------------------------------
 
 [TIP]
 =============================================================
-The list is growing all the time and if your BSP is not support we
-encourage you to add it and submit the configuration back to the project.
+
+The list is growing all the time and if your BSP is not supported we encourage
+you to add it and submit the configuration back to the project.
+
 =============================================================
 
 Some of the BSPs may appear more than once in the list. These are aliased BSP
 configuration's that may use a different backend. An example is the SPARC
 Instruction Simulator (SIS) BSP. There is the 'sis' BSP which uses the GDB
 backend and the 'sis-run' which uses the command line version of the SIS
-simulator. We will show how to use +rtems-test+ with the SIS BSP because it is
-easy to build an use.
+simulator. We will show how to use +rtems-test+ conmand with the SIS BSP
+because it is easy to build an use.
 
 Building RTEMS Tests
 ~~~~~~~~~~~~~~~~~~~~
 
-Building RTEMS with a configuration command line something similar to:
+Build RTEMS with a configuration command line something similar to:
 
 [NOTE]
 =============================================================
@@ -133,15 +143,68 @@ $ git clone git://git.rtems.org/rtems.git rtems.git
 $ cd rtems.git
 $ ./bootstrap -c && ./bootstrap -p && ./bootstrap
 $ cd ..
-$ mkdir sis
-$ ../rtems.git/configure --target=sparc-rtems4.11 \
-                  --enable-tests --enable-rtemsbsp=sis
-$ make
+$ mkdir -p builds/sis
+$ cd builds/sis
+$ ../../rtems.git/configure --target=sparc-rtems4.11 \
+                    --enable-tests --enable-rtemsbsp=sis
+$ make <1>
+-------------------------------------------------------------
+<1> Add the -j option with the number of cores to run the build parallel where
+it can.
+
+Building all the tests takes time and it uses more disk so be patient. When
+finished all the tests will be built ready to run. Before running all the tests
+it is a good idea to run the +hello+ test. The +hello+ test is an RTEMS version
+of the classic "Hello World" example and running it shows you have a working
+tool chain and build of RTEMS ready to run the tests. Using the run command:
+
+-------------------------------------------------------------
+$ sparc-rtems4.11-run <1> sparc-rtems4.11/c/sis/testsuites/samples/hello/hello.exe
+
+*** BEGIN OF TEST HELLO WORLD ***
+Hello World
+*** END OF TEST HELLO WORLD ***
+-------------------------------------------------------------
+<1> The run command is the GDB simulator without the GDB part.
+
+Running the example using GDB:
+
+-------------------------------------------------------------
+$ sparc-rtems4.11-gdb sparc-rtems4.11/c/sis/testsuites/samples/hello/hello.exe
+GNU gdb (GDB) 7.7
+Copyright (C) 2014 Free Software Foundation, Inc.
+License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
+and "show warranty" for details.
+This GDB was configured as "--host=x86_64-apple-darwin13.2.0
+--target=sparc-rtems4.11".
+Type "show configuration" for configuration details.
+For bug reporting instructions, please see:
+<http://www.gnu.org/software/gdb/bugs/>.
+Find the GDB manual and other documentation resources online at:
+<http://www.gnu.org/software/gdb/documentation/>.
+For help, type "help".
+Type "apropos word" to search for commands related to "word"...
+Reading symbols from
+sparc-rtems4.11/c/sis/testsuites/samples/hello/hello.exe...done.
+(gdb) target sim
+Connected to the simulator.
+(gdb) load
+(gdb) r <1>
+Starting program: sparc-rtems4.11/c/sis/testsuites/samples/hello/hello.exe
+
+
+*** BEGIN OF TEST HELLO WORLD ***
+Hello World
+*** END OF TEST HELLO WORLD ***
+[Inferior 1 (process 42000) exited normally]
+(gdb) q
 -------------------------------------------------------------
+<1> To debug set break points before issuing the GDB run command.
 
-It will build all the tests. There are currently 471 separate tests and you can
-run them all with a single RTEMS Tester command. Building all the tests takes
-time and it uses more disk so be patient.
+There are currently close to 500 separate tests and you can run them all with a
+single RTEMS Tester command.
 
 TIP: Use a recent RSB version to build a SPARC tool chain. A recent patch
 fixes issues with the GDB simulator and it improves the test results. It is
@@ -158,70 +221,82 @@ executable is a path to a directory the directories under that path are
 searched for any file with a +.exe+ extension. This is the detault extension
 for RTEMS executables built within RTEMS.
 
-To run the SIS tests we have built run this command:
+To run the SIS tests enter the following command from the top of the SIS BSP
+build tree:
 
 -------------------------------------------------------------
-./rtems-test --log=log_sis_run <1> \
-             --rtems-bsp=sis-run <2> \
-             --rtems-tools=$HOME/development/rtems/4.11 <3> \
-                 $HOME/development/rtems/kernel/sis <4>
+$ ~/development/rtems/test/rtems-tools.git/tester/rtems-test <1> \
+             --log=log_sis_run <2> \
+             --rtems-bsp=sis-run <3> \
+             --rtems-tools=$HOME/development/rtems/4.11 <4> \
+                 sparc-rtems4.11/c/sis/testsuites <5>
 RTEMS Testing - Tester, v0.2.0
-[  5/471] p:0   f:0   t:0   i:0   | sparc/sis: fsdosfswrite01.exe
-[  2/471] p:0   f:0   t:0   i:0   | sparc/sis: fsdosfsformat01.exe
-[  4/471] p:0   f:0   t:0   i:0   | sparc/sis: fsdosfssync01.exe
-[  6/471] p:0   f:0   t:0   i:0   | sparc/sis: fsfseeko01.exe
-[  3/471] p:0   f:0   t:0   i:0   | sparc/sis: fsdosfsname01.exe
-[  1/471] p:0   f:0   t:0   i:0   | sparc/sis: fsbdpart01.exe
-[  7/471] p:0   f:0   t:0   i:0   | sparc/sis: fsimfsgeneric01.exe
-[  8/471] p:0   f:0   t:0   i:0   | sparc/sis: fsnofs01.exe
-[  9/471] p:1   f:0   t:0   i:0   | sparc/sis: fsrfsbitmap01.exe
-[ 10/471] p:4   f:0   t:0   i:0   | sparc/sis: imfs_fserror.exe
-[ 11/471] p:4   f:0   t:0   i:0   | sparc/sis: imfs_fslink.exe
-[ 12/471] p:4   f:0   t:0   i:0   | sparc/sis: imfs_fspatheval.exe
-  ................................................. <5>
-[465/471] p:453 f:4   t:0   i:0   | sparc/sis: tm26.exe
-[466/471] p:454 f:4   t:0   i:0   | sparc/sis: tm27.exe
-[467/471] p:455 f:4   t:0   i:0   | sparc/sis: tm28.exe
-[468/471] p:456 f:4   t:0   i:0   | sparc/sis: tm29.exe
-[469/471] p:457 f:4   t:0   i:0   | sparc/sis: tm30.exe
-[470/471] p:458 f:4   t:0   i:0   | sparc/sis: tmck.exe
-[471/471] p:459 f:4   t:0   i:0   | sparc/sis: tmoverhd.exe
-Passed:   466 <6>
-Failed:     4
-Timeouts:   1
+[  1/500] p:0   f:0   t:0   i:0   | sparc/sis: fsbdpart01.exe
+[  2/500] p:0   f:0   t:0   i:0   | sparc/sis: fsdosfsformat01.exe
+[  5/500] p:0   f:0   t:0   i:0   | sparc/sis: fsdosfswrite01.exe
+[  4/500] p:0   f:0   t:0   i:0   | sparc/sis: fsdosfssync01.exe
+[  7/500] p:0   f:0   t:0   i:0   | sparc/sis: fsimfsgeneric01.exe
+[  3/500] p:0   f:0   t:0   i:0   | sparc/sis: fsdosfsname01.exe
+[  8/500] p:0   f:0   t:0   i:0   | sparc/sis: fsnofs01.exe
+[  6/500] p:0   f:0   t:0   i:0   | sparc/sis: fsfseeko01.exe
+[  9/500] p:5   f:0   t:0   i:0   | sparc/sis: fsrfsbitmap01.exe
+[ 10/500] p:5   f:0   t:0   i:0   | sparc/sis: imfs_fserror.exe
+[ 12/500] p:5   f:0   t:0   i:0   | sparc/sis: imfs_fspatheval.exe
+  ................................................. <6>
+[492/500] p:482 f:0   t:2   i:0   | sparc/sis: tm25.exe
+[493/500] p:483 f:0   t:2   i:0   | sparc/sis: tm26.exe
+[495/500] p:485 f:0   t:2   i:0   | sparc/sis: tm28.exe
+[494/500] p:485 f:0   t:2   i:0   | sparc/sis: tm27.exe
+[496/500] p:487 f:0   t:2   i:0   | sparc/sis: tm29.exe
+[497/500] p:487 f:0   t:2   i:0   | sparc/sis: tm30.exe
+[498/500] p:489 f:0   t:2   i:0   | sparc/sis: tmck.exe
+[499/500] p:490 f:0   t:2   i:0   | sparc/sis: tmcontext01.exe
+[500/500] p:490 f:0   t:2   i:0   | sparc/sis: tmoverhd.exe
+Passed:   495 <7>
+Failed:     0
+Timeouts:   5
 Invalid:    0
-Total:    471
-Testing time: 0:05:29.240084 <7>
+Total:    500
+Average test time: 0:00:00.608249 <8>
+Testing time     : 0:05:04.124696
 -------------------------------------------------------------
-<1> The +--log+ option sends the output to a log file.
-<2> Select the SIS BSP and use the simulator run command.
-<3> Path to the RTEMS tools so the simulator run command can be found.
-<4> Path to the SIS BSP built with all tests.
-<5> The output has been shortened so it fits nicely here.
-<6> The test results. It shows passes, fails, timeouts, and invalid results.
-<7> The total time taken to run all the tests.
-
-This BSP requires the +--rtems-tools+ option because the SIS simulator is the
-+sparc-rtems4.11-run+ command that is part of the RTEMS tools. Not every BSP
+<1> The RTEMS Tester's test command. In this example we are using an absolute
+path.
+<2> The +--log+ option sends the output to a log file. By default only failed
+tests log the complete output.
+<3> Select the SIS BSP and use GDB.
+<4> Path to the RTEMS tools so GDB can be found.
+<5> Path to the SIS BSP built with all tests to run. If you add subdirectories
+to the path specific tests can be run.
+<6> The output has been shortened so it fits nicely here.
+<7> The test results. It shows passes, fails, timeouts, and invalid results. In
+this run 495 tests passed and 5 tests timedout. The timeouts are probability
+due the tests not having enough execute time to complete. The default timeout
+is 180 seconds and some of the interrupt tests need longer. The amount of time
+depends on the performance of your host CPU running the simulations.
+<8> The average time per test and the total time taken to run all the tests.
+
+This BSP requires the +--rtems-tools+ option because the SPARC GDB is the
++sparc-rtems4.11-gdb+ command that is part of the RTEMS tools. Not every BSP
 will require this option so you will need to check the specifics of the BSP
 configration to determine if it is needed.
 
 The output you see is each test starting to run. The +rtems-test+ command can
-run multiple SIS simulators in parallel so you will see a number start quickly
-and then tests start as others finish. The output shown here is from an 8 core
-processor so the first 8 are started in parallel and the status shows the order
-they actually started which is not 1 to 7.
+run multiple SIS GDB simulations in parallel so you will see a number start
+quickly and then tests start as others finish. The output shown here is from an
+8 core processor so the first 8 are started in parallel and the status shows
+the order they actually started which is not 1 to 8.
 
 The test start line shows the current status of the tests. The status reported
 is when the test starts and not the result of that test. A fail, timeout or
 invalid count changing means a test running before this test started failed,
-not the starting test. The status here has 453 tests pass and 4 tests fail when
-test tm26.exe start:
+not the starting test. The status here has 495 tests pass and no failures and 5
+timeouts.:
 
 -------------------------------------------------------------
-[465/471]<1>  p:453<2> f:4<3>  t:0<4>  i:0<5>  | sparc/sis:<6> tm26.exe<7>
+[295/500]<1> p:287<2> f:0<3>  t:1<4>  i:0<5> | sparc/sis:<6> sp26.exe<7>
 -------------------------------------------------------------
-<1> The test number, in this case test 465 of 471 tests.
+<1> The test number, in this case test 295 of 500 tests.
 <2> Passed test count.
 <3> Failied test count.
 <4> Timeout test count.
@@ -231,15 +306,15 @@ test tm26.exe start:
 
 The test log records all the tests and results. The reporting mode by default
 only provides the output history if a test fails, timeouts, or is invalid. The
-time taken by each test is also records.
+time taken by each test is also recorded.
 
 The tests must complete in a specified time or the test is marked as timed
 out. The default timeout is 3 minutes and can be globally changed using the
 +--timeout+ command line option. The time required to complete a test can
 vary. When simulators are run in parallel the time taken depends on the
 specifics of the host machine being used. A test per core is the most stable
-even though more tests can be run than available cores. If your machine needs
-longer or you are using a VM you may need to length the time out.
+method even though more tests can be run than available cores. If your machine
+needs longer or you are using a VM you may need to lengthen the time out.
 
 Test Status
 ~~~~~~~~~~~
@@ -294,7 +369,7 @@ No output is written to the log.
 
 The output is tagged so you can determine where it comes from. The following is
 the complete output for the In Memory File System test +imfs_fslink.exe+
-running on a Coldfire MCF5235 using a GDB and a BDM pod:
+running on a Coldfire MCF5235 using GDB and a BDM pod:
 
 -------------------------------------------------------------
 [ 11/472] p:9   f:0   t:0   i:1   | m68k/mcf5235: imfs_fslink.exe
@@ -368,11 +443,12 @@ Running Tests in Parallel
 The RTEMS Tester supports parallel execution of tests by default. This only
 makes sense if the test backend can run in parallel without resulting in
 resource contention. Simulators are an example of backends that can run in
-parallel.
+parallel. A hardware debug tool like a BDM or JTAG pod can only a single test
+at once to the tests need to be run one at a time.
 
 The test framework manages the test jobs and orders the output in the report
-long in test order. Output is held for completed tests until the next test to
-be reported has finished.
+log in test order. Output is held for completed tests until the next test to be
+reported has finished.
 
 Command Line Help
 ~~~~~~~~~~~~~~~~~