New Configurations documentation.

1 files changed, 333 insertions, 1 deletions

diff --git a/doc/source-builder.txt b/doc/source-builder.txt
index 6e0b4bc..622f0fc 100644
--- a/doc/source-builder.txt
+++ b/doc/source-builder.txt
@@ -613,6 +613,17 @@ pays to run the command with the +--jobs=none+ option to get a log that is
 correctly sequenced. If searching the log file seach for +error:+ and the error
 should be just above it.
 
+[[_contributing]]
+Contributing
+------------
+
+The RSB is open source and open to contributions. These can be bug fixes, new
+features or new configurations. Please break patches down into changes to the
+core Python code, configuration changes or new configurations.
+
+Pleease email me patches via git so I can maintain your commit messages so you
+are acknowledged as the contributor.
+
 Project Sets
 ------------
 
@@ -1189,6 +1200,328 @@ and given a +-2+ revision number. Any dependent scripts referencing the earlier
 revision number will not be effected by the change. This locks down a specific
 configuration over time.
 
+Personal Configurations
+~~~~~~~~~~~~~~~~~~~~~~~
+
+The RSB supports personal configurations. You can view the RTEMS support in the
++rtems+ directory as a private configuration tree that resides within the RSB
+source. There is also the +bare+ set of configurations. You can create your own
+configurations away from the RSB source tree yet use all that the RSB provides.
+
+To create a private configuration change to a suitable directory:
+
+-------------------------------------------------------------
+$ cd ~/work
+$ mkdir test
+$ cd test
+$ mkdir config
+-------------------------------------------------------------
+
+and create a +config+ directory. Here you can add a new configuration or build
+set file. The section 'Adding New Configurations' details how to add a new
+confguration.
+
+New Configurations
+~~~~~~~~~~~~~~~~~~
+
+This section describes how to add a new configuration to the RSB. We will add a
+configuration to build the Device Tree Compiler. The Device Tree Compiler or
+DTC is part of the Flattened Device Tree project and compiles Device Tree
+Source (DTS) files into Device Tree Blobs (DTB). DTB files can be loaded by
+operating systems and used to locate the various resources such as base
+addresses of devices or interrupt numbers allocated to devices. The Device Tree
+Compiler source code can be downloaded from http://www.jdl.com/software. The
+DTC is supported in the RSB and you can find the configuration files under the
++bare/config+ tree. I suggest you have a brief look over these files.
+
+Layering by Including
+^^^^^^^^^^^^^^^^^^^^^
+
+Configurations can be layered using the +%include+ directive. The user invokes
+the outter layers which include inner layers until all the required
+configuration is present and the package can be built. The outter layers can
+provide high level details such as the version and the release and the inner
+layers provide generic configuration details that do not change from one
+release to another. Macro variables are used to provide the specific
+configuration details.
+
+Configuration File Numbering
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Configuration files have a number at the end. This is a release number for that
+configuraiton and it gives us the ability to track a specific configuration for
+a specific version. For example lets say the developers of the DTC package
+change the build system from a single makefile to autoconf and automake between
+version 1.3.0 and version 1.4.0. The configuration file used to build the
+package would change have to change. If we did not number the configuration
+files the ability to build 1.1.0, 1.2.0 or 1.3.0 would be lost if we update a
+common configuration file to build an autoconf and automake version. For
+version 1.2.0 the same build script can be used so we can share the same
+configuration file between version 1.1.0 and version 1.2.0. An update to any
+previous release lets us still build the package.
+
+Common Configuration Scripts
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Common configuration scripts that are independent of version, platform and
+architecture are useful to everyone. These live in the Source Builder's
+configuration directory. Currrently there are scripts to build binutils, expat,
+DTC, GCC, GDB and libusb. These files contain the recipes to build these
+package without the specific details of the versions or patches being
+built. They expect to be wrapped by a confguration file that ties the package
+to a specific version and optionally specific patches.
+
+DTC Example
+^^^^^^^^^^^
+
+We will be building the DTC for your host rather than a package for RTEMS. We
+will create a file called +source-builder/config/dtc-1-1.cfg+. This is a common
+script that can be used to build a specific version using a general recipe. The
+file name is 'dtc-1-1.cfg' where the 'cfg' extension indicates this is a
+configuration file. The first *1* says this is for the major release 1 of the
+package and the last *1* is the build configuration version.
+
+The file starts with some comments that detail the configuration. If there is
+anything unsual about the configuration it is a good idea to add something in
+the comments here. The comments are followed by a check for the release. In
+this case if a release is not provided a default of 1 is used.
+
+-------------------------------------------------------------
+#
+# DTC 1.x.x Version 1.
+#
+# This configuration file configure's, make's and install's DTC.
+#
+
+%if %{release} == %{nil}
+%define release 1
+%endif
+-------------------------------------------------------------
+
+The next section defines some information about the package. It does not effect
+the build and is used to annotate the reports. It is recommended this
+information is kept updated and accurate.
+
+-------------------------------------------------------------
+Name:      dtc-%{dtc_version}-%{_host}-%{release}
+Summary:   Device Tree Compiler v%{dtc_version} for target %{_target} on host %{_host}
+Version:   %{dtc_version}
+Release:   %{release}
+URL: 	   http://www.jdl.com/software/
+BuildRoot: %{_tmppath}/%{name}-root-%(%{__id_u} -n)
+-------------------------------------------------------------
+
+The next section defines the source and any patches. In this case there is a
+single source package and it can be downloaded using the HTTP protocol. The RSB
+knows this is GZip'ped tar file. If more than one package package is needed add
+them increasing the index. The +gcc-4.8-1.cfg+ configuration contains examples
+of more than one source package as well as conditionally including source
+packages based on the outter configuration options.
+
+-------------------------------------------------------------
+#
+# Source
+#
+Source0: http://www.jdl.com/software/dtc-v%{dtc_version}.tgz
+-------------------------------------------------------------
+
+The remainder of the script is broken in to the various phases of a build. They
+are:
+
+. Preperation
+. Bulding
+. Installing, and
+. Cleaning
+
+Preperation is the unpacking of the source, applying any patches as well as any
+package specific set ups. This part of the script is a standard Unix shell
+script. Becareful with the the use of '%' and '$'. The RSB uses '%' while the
+shell scripts use '$'.
+
+A standard pattern you will observe is the saving of the build's top
+directory. This is used instead of changing into a subdirectory and then
+changing to the parent when finished. Some hosts will change in a subdirectory
+that is a link however changing to the parent does not change back to the
+parent of the link rather it changes to the parent of the target of the link
+and that is something the RSB nor you can track easily. The RSB configuration
+script's are a collection of various subtle issues so please ask if you are
+unsure why something is being done a particular way.
+
+The preparation phase will often include a number of conditional patch
+commands. Outter layers can provide patches as needed while being able to use a
+common recipe. Users can override the standard build and supply a custom patch
+for testing using the user macro command line interface.
+
+-------------------------------------------------------------
+#
+# Prepare the source code.
+#
+%prep
+  build_top=$(pwd)
+
+  %setup -q -n dtc-v%{dtc_version}
+  %{?patch0:%patch0 -p1}
+  %{?patch1:%patch1 -p1}
+  %{?patch2:%patch2 -p1}
+  %{?patch3:%patch3 -p1}
+  %{?patch4:%patch4 -p1}
+  %{?patch5:%patch5 -p1}
+  %{?patch6:%patch6 -p1}
+  %{?patch7:%patch7 -p1}
+
+  cd ${build_top}
+-------------------------------------------------------------
+
+The configuration file 'gcc-common-1.cfg' is a complex example of source
+preparation. It contains a number of source packages and patches and it
+combines these into a single source tree for building. It uses links to map
+source into the GCC source tree so GCC can be built using the _single source
+tree_ method. It also shows how to fetch source code from version
+control. Newlib is taken directly from its CVS repository.
+
+Next is the building phase and for the DTC example this is simply a matter of
+running +make+. Note the use of the RSB macros for commands. In the case of
+'%{\__make}' it maps to the correct make for your host. In the case of BSD
+systems we need to use the GNU make and not the GNU make.
+
+If your package requires a configuration stage you need to run this before the
+make stage. Again the GCC common confguration file provides a detailed example.
+
+-------------------------------------------------------------
+%build
+  build_top=$(pwd)
+
+  cd dtc-v%{dtc_version}
+
+  %{build_build_flags}
+
+  %{__make} PREFIX=%{_prefix}
+
+  cd ${build_top}
+-------------------------------------------------------------
+
+You can invoke make with the macro '%{?_smp_flags}' as a command line
+argument. This macro is controlled by the '--jobs' command line option and the
+host CPU detection support in the RSB. If you are on a multicore host you can
+increase the build speed using this macro. It also lets you disabled bulding on
+multicores to aid debugging when tetsing.
+
+Next is the install phase. This phase is a little more complex because you may
+be building a tar file and the end result of the build is never actually
+installed into the prefix on the build host and you may not even have
+permissions to perform a real install. Most packages install to the +prefix+
+and the prefix is typically supplied via the command to the RSB or the
+package's default is used. The default can vary depending on the host's
+operating system. To install to a path that is not the prefix the +DESTDIR+
+make variable is used. Most packages should honour the +DISTDIR+ make variables
+and you can typically specify it on the command line to make when invoking the
+install target. This results in the package being installed to a loction that
+is not the prefix but one you can control. The RSB provides a shell variable
+called +SB_BUILD_ROOT+ you can use. In a build set where you are building a
+number of packages you can collect all the built packages in a single tree that
+is captured in the tar file.
+
+Also note the use of the macro +%{\__rmdir}+. The use of these macros allow the
+RSB to vary specific commands based on the host. This can help on hosts like
+Windows where bugs can effect the standard commands such as 'rm'. There are
+many many macros to help you. You can find these listed in the +defaults.mc+
+file and in the trace output. If you are new to creating and editing
+configurations learning these can take a little time.
+
+-------------------------------------------------------------
+%install
+  build_top=$(pwd)
+
+  %{__rmdir} -rf $SB_BUILD_ROOT
+
+  cd dtc-v%{dtc_version}
+  %{__make} DESTDIR=$SB_BUILD_ROOT PREFIX=%{_prefix} install
+
+  cd ${build_top}
+-------------------------------------------------------------
+
+Finally there is an optional clean section. The RSB will run this section if
++--no-clean+ has not been provided on the command line. The RSB does clean up
+for you.
+
+Once we have the configuration files we can execute the build using the
+`sb-builder` command. The command will perform the build and create a tar file
+in the +tar+ directory.
+
+-------------------------------------------------------------
+$  ../source-builder/sb-builder --prefix=/usr/local \
+     --log=log_dtc devel/dtc-1.2.0
+RTEMS Source Builder, Package Builder v0.2.0
+config: devel/dtc-1.2.0
+package: dtc-1.2.0-x86_64-freebsd9.1-1
+download: http://www.jdl.com/software/dtc-v1.2.0.tgz -> sources/dtc-v1.2.0.tgz
+building: dtc-1.2.0-x86_64-freebsd9.1-1
+$ ls tar
+dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2
+-------------------------------------------------------------
+
+If you want to have the package installed automatically you need to create a
+build set. A build set can build one or more packages from their configurations
+at once to create a single package. For example the GNU tools is typically seen
+as binutils, GCC and GDB and a build set will build each of these packages and
+create a single build set tar file or install the tools on the host into the
+prefix path.
+
+The DTC build set file is called +dtc.bset+ and contains:
+
+-------------------------------------------------------------
+#
+# Build the DTC.
+#
+
+%define release 1
+
+devel/dtc-1.2.0.cfg
+-------------------------------------------------------------
+
+To build this you can use something similar to:
+
+-------------------------------------------------------------
+$ ../source-builder/sb-set-builder --prefix=/usr/local --log=log_dtc \
+   --trace --bset-tar-file --no-install dtc
+RTEMS Source Builder - Set Builder, v0.2.0
+Build Set: dtc
+config: devel/dtc-1.2.0.cfg
+package: dtc-1.2.0-x86_64-freebsd9.1-1
+building: dtc-1.2.0-x86_64-freebsd9.1-1
+tarball: tar/x86_64-freebsd9.1-dtc-set.tar.bz2
+cleaning: dtc-1.2.0-x86_64-freebsd9.1-1
+Build Set: Time 0:00:02.865758
+$ ls tar
+dtc-1.2.0-x86_64-freebsd9.1-1.tar.bz2   x86_64-freebsd9.1-dtc-set.tar.bz2
+-------------------------------------------------------------
+
+The build is for a FreeBSD host and the prefix is for user installed
+packages. In this example I cannot let the source builder perform the install
+because I never run the RSB with root priviledges so a build set or bset tar
+file is created. This can then be installed using root privildges.
+
+The command also supplies the --trace option. The output in the log file will
+contian all the macros.
+
+Debugging
+^^^^^^^^^
+
+New configuration files require debugging. There are two types of
+debugging. The first is debugging RSB script bugs. The +--dry-run+ option is
+used here. Suppling this option will result in most of the RSB processing to be
+performed and suitable output placed in the log file. This with the +--trace+
+option should help you resolve any issues.
+
+The second type of bug to fix are related to the execution of one of
+phases. These are usually a mix of shell script bugs or package set up or
+configuration bugs. Here you can use any normal shell script type debug
+technique such as +set -x+ to output the commands or +echo+
+statements. Debugging package related issues may require you start a build with
+teh RSB and supply +--no-clean+ option and then locate the build directories
+and change directory into them and manually run commands until to figure what
+the package requires.
+
 Snapshot Testing
 ~~~~~~~~~~~~~~~~
 
@@ -1843,7 +2176,6 @@ The +%bconf_without+ macro provides a way to test if the user has passed a
 specific option on the command line with the +--without-<label>+ option. This
 option is only available with the +sb-builder+ command.
 
-
 Commands
 --------