path: root/rtems-rtl.txt

RTEMS Runtime Link Editor (RTL)
===============================
Chris Johns <chrisj@rtems.org>
1.0, Decemeber 2011
:doctype: book
:toc:
:icons:   
:numbered:

image:images/rtemswhitebg.jpg["RTEMS",width="40%"]

Background
----------

This project adds support for the 'dlopen', 'dlclose' and 'dlsym' calls as
defined in the POSIX and related standards to the RTEMS operating system.

This test harness contains a new runtime linker as well as the Android and
NetBSD RTLD code. This support will go away so check earlier snapshots to find
that support. Neither the Android or NetBSD code is being directly used in this
project and only serves as a reference.

The project built and tested the NetBSD code to look into using dynamic ELF
object files as a means of runtime loading code into RTEMS. Dynamic object
files are used by ELF based operating systems to share libraries and support
this function well. This model of dynamic loading does not map to RTEMS with
the same benefits. Simply put, RTEMS is not a multi-process environment nor
does it use memory protection or partitioned memory systems to create a virtual
process environment and this is where the benefits of using dynamic object
file's appear. Shared libraries allow code to be shared amongst a number of
processes operating in different virtual address spaces and while these
processes may share code they are kept separate for security reasons. Dynamic
ELF files provide the following characteristics to support sharing:

. Procedure Lookup Tables (PLT)
. Global Offset Tables (GOT)
. Position Independent Code (PIC)
. Whole file memory footprint

The PLT and GOT tables along with PIC code provide a small contained way to
locate a dynamic object file without the dynamic object file requiring
relocation fixups all through itself. This is important on shared library
system as it avoids the need for each process having a private copy of the
shared library code which defeats the purpose of sharing increasing the memory
usage across the system. On a shared library system the whole object file is
mapped into the process's address space and demand loading manages the physical
memory used by the process. If a process modifies a part of the shared library
the 'copy on write' or COW features of the operating system takes a copy of
that page of memory and makes it private to that process. With a shared library
made up of dynamic object files This happens when the process writes to a data
variable that is part of the shared library or the code is modified. To
minimise possible changes execution within the shared library is realtive using
position independent code (PIC) and calls externally are made using the PLT
tables which are altered to match the shared code's address mapping. The PTL
tables are small relative to the size of a shared library.

The downside of this design is the performance overhead the indirect procedure
and data references incur plus the need to load the whole object file image
into memory. On a multi-process system with virtual memory demand paging these
negatives are minor compared to the positives, how-ever on RTEMS, a single
process system without virtual memory there are no positives only negatives.

The NetBSD code was used to attempt to get relocatable code into a dynamic ELF
file. The tools for RTEMS would not do this. Relocatable code can be found in
relocatable type ELF file. The NetBSD loader checks for any ELF file that is
not dynamic returning an error. This means this code base could not be used
without major changes. The NetBSD code does support a large number of
architectures and has relocation support.

The Android code also only supports dynamic modules and its code base only
supports a few targets and this support is not as cleanly implemented as
NetBSD.

<<<<
RTEMS Runtime Link Editor
-------------------------

The RTL code contained in this project is new code developed for RTEMS using
the features of RTEMS to give a small tight implementation. It supports the
loading of relocatable object files, invoking of contructor and destructor
functions, global symbol relocation and loading directly from archive format
files. The header files provide documentation about the functions provided and
the main.c file shows the command added to support development.

The target code is supported by a host RTEMS specific linker called
'rtems-ld'. The RTEMS linker is contained in a separate package.

Build System
~~~~~~~~~~~~

The package uses the waf build system. You can find waf here:

  http://waf.googlecode.com/

Waf is a fast full featured build environment that removes the need for make,
make files or any other traditional build system infrastructure. For details
please refer to the WAF-README file. It can be used to build any RTEMS project.

Follow the download instructions for waf and install into your path. If you are
on Windows you will need to download a recent 2.7 or later version of Python
and add to the global path.

Testing
-------

The development is using the HEAD branch of RTEMS and the BSP is the Sparc
Instruction Simulator. Other architectures are support and if you have tools
and BSPs installed. The build scripts will attempt to detect them and build for
them. If there is no support for a specific architecture just copy the empty
NIOS2 file to the missing architecture.

To build:

. Get the latest tools.
. Check out RTEMS from CVS and run a bootstrap.
. Build using:
+
-------------------------------------------------------------
$RTEMS_SRC/configure --target=sparc-rtems4.11 \ <1>
                     --prefix=$RTEMS_TEST_PREFIX/build/4.11 \ <2>
                     --enable-rtemsbsp=sis \
                     --enable-tests=samples \
                     --enable-cxx \
                     --enable-maintainer-mode \
                     --enable-rtems-debug
-------------------------------------------------------------

<1> $RTEMS_SRC is the path to the RTEMS source code.
<2> $RTEMS_TEST_PREFIX is the path you install your test RTEMS builds.

. Configure this package change directory to the source code and then:
+
-------------------------------------------------------------
waf configure --rtems=/opt/work/rtems/build/4.11 \
              --rtems-tools=/opt/work/rtems/4.11,/opt/work/rtems/nios/nios2-rtems-11.0
-------------------------------------------------------------
+
You can also use:
+
-------------------------------------------------------------
waf --help 
-------------------------------------------------------------
+
to get a list of options. The RTEMS specific ones are:
+
--rtems=RTEMS_PATH:: Path to an installed RTEMS BSPs.
+
--rtems-tools=RTEMS_TOOLS:: Path to RTEMS tools. This options avoids the need
to have the tools in your shell path.
+
--rtems-version=RTEMS_VERSION:: RTEMS version (default 4.11). This is only
needed if you are not using the default.
+
--rtems-archs=RTEMS_ARCHS:: List of RTEMS architectures to build. By default
all architectures detected when configuring are built.
+
--rtems-bsps=RTEMS_BSPS:: List of BSPs to build. This lets you specify a
specific set of BSPs to build for. By default all BSPs are built.
+
--show-commands:: Print the commands as strings. This will display the full
command used. This can help when debugging tool related issues.

. Build the RTL test application do:
+
-------------------------------------------------------------
waf
-------------------------------------------------------------

. Run in the GDB SIS simulator:
+
-------------------------------------------------------------
sparc-rtems4.11-run build/sparc-rtems4.11-sis/rtld
-------------------------------------------------------------

RTEMS Linker
------------

The loading of relocatable object files allows the memory map to be determined
at run time giving us the maximum flexibility because the user does not need to
manage the memory. This approach how-ever requires loading be managed at the
object file level and this means there will be unresolved symbols and these
symbols can be from any library referenced by the project.

The target link editor can resolve these symbols as it loads object files if
given a list of object files. A linker typically generates this list by loading
the symbol tables of the referenced librarys and then selecting the object
files referenced. The target does not have access to the librarys nor does it
typically have the memory and performance needed to manage large symbol tables.

To resolve this an RTEMS linker has been developed. This is a C++ host only
program that can read and access the ELF files, load the symbols tables and
resolve the external symbols. It has a typical linker command line and outputs
a list of dependent object files or an archive file containting all referenced
object files. You can provide the linker with the base image and only the
object files not present are included. The base image is the static RTEMS image
you boot on your target hardware.

This approach over comes the "library dependence" problem present in embedded
systems that use cross-compilation and standards linkers. If you have a
statically linked base image containing code from a library and you wish to
create a run time loadable image, either statically or incrementally linked
using code from the same library you do not know if code from the library is
present or not. If you feed the linker the base image's symbols and library the
linker will generate errors for duplicant symbols in the base image from the
same library. If you do not provide the library the linker will generate
unresolved symbol errors. We need a special linker that can handle this and
understand the role of the base image and the libraries present.

Run Loadable Models
~~~~~~~~~~~~~~~~~~~

There are a number of possible application models users may wish to use. A
project may decide to vary the model used depending on the phase the project is
in.

Script::

Load the application from the object files in the application and the
libraries.  The target needs to have access to all libraries and this typically
means networked file system (NFS) access to the development host containing the
libraries being used. With a suitable initialisation mounting the network disk
and libraries paths being set an application can be loaded using a script of
files. The approach gives the developer flexiblity to update and play with
various parts of the code as they go.

Single Image::

The RTEMS linker creates a single image of the application by pulling in the
object files from the libraries used by the application. The linker can be
given the base image and will link against the symbols it contains. This means
the base image and an application can use the same library and the application
will only contain the code from the library it needs. If at the point of
loading the object file has been loaded from another application the one
included in this application will not be loaded.

For those users who need a determinisitic layout the loader should load the
same object files in the same place each time. This is deterministic.

Status
------

The current status is loading is:

. Loading of an object file as a file or within a archive works on the Sparc,
i386 and m68k architectures. Other architectures are present how-ever testing
may be limited. The NIOS file is an empty template to allow the package to
build.

. Shell commands +dlopen+ ('dlo'), +dlclose+ ('dlc'), +dlsearch+ ('dls'), and
execute (+dlx+) provide a way to test the loader.

. Shell access to the loader is provided via the 'rtl' command. The command
provdes:
+
. status:: The status of the loader.
+
. list:: List of loaded object modules.
+
. sym:: Symbol table access.
+
. obj:: Object file details.

. The RTEMS linker and loader need a simple way to have a single application
archive loaded and run. This could be a script in a special section of the
archive, an elf file or something else.

Development Tasks
~~~~~~~~~~~~~~~~~

. Add the ability to hold unresolved externals an object module may have once
loaded. We need the ability to load object modules with unresolved externals
because object modules may depend on eac other. 

This is complex issue due to the memory management issues it creates. Holding
the unresolved externals and then releasing them later will fragment the
heap. The proper solution is to add a memory manager and allocator to the
loader. This would be used for all symbol allocations.

The other solution is to not allow loading of object modules with unresolved
symbols and to force application that have them to incrementally link. The
issue is here is 3rd party libraries and that makes this solution less than
ideal.

. Add an application loader. An application needs to be formally
defined. Currently the RTEMS linker creates a standard library archive. A
application format needs some extra information. This could be as simple as
something that indicates this is an RTEMS application, through to the entry
point and a load order for the object modules in the archive to avoid
unresolved externals.

. Place the code into the RTEMS source tree.

Development Cut-n-Pastes for Chris
----------------------------------

Note: the '--' is required to abort getopt processing in sis-gdb

-------------------------------------------------------------
waf configure --rtems=/Users/chris/Development/rtems/build/4.11 \
--rtems-tools=/Users/chris/Development/rtems/4.11,/Users/chris/Development/rtems/nios/gnu/nios2-rtems-11.0
~/Development/rtems/build/testing/sim-scripts/sis-gdb -- \
--annotate=3 /Users/chris/Development/rtems/build/rtl/sparc/rtld
-------------------------------------------------------------

-------------------------------------------------------------
sparc-rtems4.11-readelf -all x-long-name-to-create-gnu-extension-in-archive.o | less
~/Development/rtems/build/testing/sim-scripts/sis -l 10 \
$(find sparc-rtems4.11/c/sis/testsuites/psxtests -name \*.exe) > results-no-oe-align
/Users/chris/Development/rtems/build/newlib/newlib-1.18.0/newlib/libc
-------------------------------------------------------------