summaryrefslogtreecommitdiffstats
path: root/user/tools/bsp-builder.rst
blob: 0c1d3308aa3fc02a0136aab727ebabc728ddcd03 (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
.. SPDX-License-Identifier: CC-BY-SA-4.0

.. Copyright (C) 2017 Chris Johns <chrisj@rtems.org>

RTEMS BSP Builder
=================

.. index:: Tools, rtems-bsp-builder

The RTEMS BSP Builder is an RTEMS developer tool to build RTEMS in ways users
do not to test changes to RTEMS. RTEMS has large number of architectures, board
support packages and configuration options. This tool provides a standard way
to test a change.

Developer Workflows
-------------------

There are a number of RTEMS developers each with a different view or expertise
in RTEMS. Developers can work in the generic areas such as scheduling, file
systems or the shell, or users can become developers adding a new BSP, or even
a new port to a new architecture. A common approach for all these developers is
to select a BSP and to work with that BSP. Developers working in a generic
areas of RTEMS tend to select a BSP that has good simulator support with good
debugging such as QEMU, while developers of a new BSP or a new port tend to
work on target hardware. This type of development does not check the other
architectures, BSP, and build options and a change may change the number of
warnings or introduce build errors. It is important for the RTEMS project to
have developers fix these issues before pushing the changes to the master
repository to avoid breaking the code for other developers. It is best for a
developer to resolve as many issues as they work on changes because comming
back to a problem often proves difficult.

The RTEMS BSP Builder forms part of a developers workflow where patches are
tested before being pushed to the repository.

Build Characteristics
---------------------

Build characteristic are the various parts of a build that can varied changing
what is built. RTEMS can vary builds based on:

#. Architecture

#. Board Support Package (BSP)

#. Build options

#. BSP Options

The BSP Builder provides a template of builds to try and reduce the possble
combinations to something manageable. It is not realistic to build all possible
combinations on a single machine in reasonible time.

The RTEMS BSP Builder specifies it builds in terms of:

#. Profiles

#. Architectures

#. BSPs

#. Builds

The RTEMS BSP Builder builds are created by user options that vary these parameters.

Profiles
^^^^^^^^

A profile is named collection of architectures and board support packages. When
the RTEMS BSP Builder is asked to build a specific profile it builds the BSPs
in the specified architectures.

The default configuration provides standard profiles for :ref:`Tiers`. They are:

#. ``tier-1`` (default)

#. ``tier-2``

#. ``tier-3``

#. ``tier-4``

The ``everythings`` profile allows all BSPs to be built.

Builds
^^^^^^

A build is a list of builds or a build set and each BSP in a profile,
architecture of BSP is built with.

The default configuration provides standard builds based around the commonly
varied configure options.

The builds are:

#. ``all`` (default)

#. ``tests``

#. ``standard``, also ``no-tests``

#. ``debug``

#. ``profiling``

#. ``smp``

#. ``smp-debug``

#. ``posix``

#. ``no-posix``

#. ``posix-debug``

#. ``posix-profiling``

#. ``network``

#. ``no-network``

#. ``network-debug``

#. ``smp-network``

#. ``smp-network-debug``

All Build
~~~~~~~~~

The ``all`` build is:

- ``debug``
- ``profiling``
- ``smp``
- ``smp-debug``
- ``posix``
- ``no-posix``
- ``posix-debug``
- ``posix-profiling``
- ``network``
- ``no-network``
- ``network-debug``
- ``smp-network``
- ``smp-network-debug``

A build maps to specific configuration options. The mappings are:

 +-----------------------+-----------------------------------------------------+
 | ``debug``             | ``config:base``, ``config:debug``                   |
 +-----------------------+-----------------------------------------------------+
 | ``profiling``         | ``config:base``, ``config:profiling``               |
 +-----------------------+-----------------------------------------------------+
 | ``smp``               | ``config:base``, ``config:smp``                     |
 +-----------------------+-----------------------------------------------------+
 | ``smp-debug``         | ``config:base``, ``config:smp``, ``config:debug``   |
 +-----------------------+-----------------------------------------------------+
 | ``posix``             | ``config:base``, ``config:posix``                   |
 +-----------------------+-----------------------------------------------------+
 | ``no-posix``          | ``config:base``, ``config:no-posix``                |
 +-----------------------+-----------------------------------------------------+
 | ``posix-debug``       | ``config:base``, ``config:posix``, ``config:debug`` |
 +-----------------------+-----------------------------------------------------+
 | ``posix-profiling``   | ``config:base``, ``config:posix``,                  |
 |                       | ``config:profiling``                                |
 +-----------------------+-----------------------------------------------------+
 | ``network``           | ``config:base``, ``config:network``                 |
 +-----------------------+-----------------------------------------------------+
 | ``no-network``        | ``config:base``, ``config:no-network``              |
 +-----------------------+-----------------------------------------------------+
 | ``network-debug``     | ``config:base``, ``config:network``,                |
 |                       | ``config:debug``                                    |
 +-----------------------+-----------------------------------------------------+
 | ``smp-network``       | ``config:base``, ``config:smp``, ``config:network`` |
 +-----------------------+-----------------------------------------------------+
 | ``smp-network-debug`` | ``config:base``, ``config:smp``,                    |
 |                       | ``config:network``, ``config:debug``                |
 +-----------------------+-----------------------------------------------------+

Build Configurations
--------------------

Build configurations are ``configure`` options. These are mapped to the various
builds. The configurations are:

 +------------------+----------------------------------------------------------+
 | ``base``         | ``--target=@ARCH@-rtems@RTEMS_VERSION@``                 |
 |                  | ``--enable-rtemsbsp=@BSP@``                              |
 |                  | ``--prefix=@PREFIX@``                                    |
 +------------------+----------------------------------------------------------+
 | ``tests``        | ``--enable-tests``                                       |
 +------------------+----------------------------------------------------------+
 | ``debug``        | ``--enable-debug``                                       |
 +------------------+----------------------------------------------------------+
 | ``no-debug``     | ``--disable-debug``                                      |
 +------------------+----------------------------------------------------------+
 | ``profiling``    | ``--enable-profiling``                                   |
 +------------------+----------------------------------------------------------+
 | ``no-profiling`` | ``--disable-profiling``                                  |
 +------------------+----------------------------------------------------------+
 | ``smp``          | ``--enable-smp``                                         |
 +------------------+----------------------------------------------------------+
 | ``no-smp``       | ``--disable-smp``                                        |
 +------------------+----------------------------------------------------------+
 | ``posix``        | ``--enable-posix``                                       |
 +------------------+----------------------------------------------------------+
 | ``no-posix``     | ``--disable-posix``                                      |
 +------------------+----------------------------------------------------------+
 | ``network``      | ``--enable-networking``                                  |
 +------------------+----------------------------------------------------------+
 | ``no-network``   | ``--disable-networking``                                 |
 +------------------+----------------------------------------------------------+

Performance
-----------

The RTEMS BSP Builder is designed to extract the maximum performance from your
hardware when building RTEMS. The RTEMS build system is based on ``autoconf``,
``automake`` and GNU ``make``. Building consists of two phases:

#. Configuring

#. Building

The Configuring phase and the start of the Build phase runs autoconf's
``configure`` scripts. These execute as a single linear process and are not run
in parallel even if you specify more than one job to ``make``. The configure
part of a build is approximately 30% of the total time and higher if building
the tests. Performing a single build at a time will not fully utilized a
multi-core machine because of the large amount of time the system is idle.

The RTEMS BSP Builder can run more than one build in parallel. A build can also
request ``make`` run its build with more than one job. The ``--jobs`` option
lets a user specify the number of build jobs to run at once and the number of
``make`` jobs each build runs with. Together these options can fully load a
system and can overload a machine.

Tuning the best ratio of buld jobs to make jobs requires running some builds
and observing the system's performance. If the build job count is too low the
system will show idle periods and if you have too many build jobs with too many
make jobs the system will have too many processing running and the operating
system's overheads in administting too processes at once lowers the overall
performance.

A fast eight core machine where the operating system shows sixteen cores can
support a build option of ``--jobs=5/10``. The machine will be fully loaded the
average build time is around 18 seconds.

The type of build selected effects the optimum jobs option. For example
building the tests changes the percentage of time spent configuring copmared to
bulding so the make jobs parameter becomes a dominant factor. Lowering the make
jobs value avoids having too many active processes running at once.

Command
-------

:program:`rtems-bsp-builder` [options]

.. option:: -?

   Display a compact help.

.. option:: -h, --help

   Display the full help.

.. option:: --prefix

   Prefix to pass to configure when building a BSP.

.. option:: --rtems-tools

   The path the RTEMS tools such as the C compiler. This option avoid polluting
   your path. This path is to the tool's prefix used to build and install the
   tools and not exact path to an executable.

.. option:: --rtems

   The path the RTEMS source tree to build.

.. option:: --build-path

   The path to build the BSP and place the build output. This can be any path
   and away from your current directory or the RTEMS source code. The storage
   does not need to be fast like an SSD.

.. option:: --log

   The log file.

.. option:: --config-report

   Print a configuration report and exit.

.. option:: --warnings-report

   Create a warnings report once all builds have finished.

.. option:: --stop-on-error

   Stop the build on an error. The default is to build all the builds for a
   profile.

.. option:: --no-clean

   Do not remove the build once finished. This option lets you inspect the
   built output. The amount of output can be large and disks can fill with this
   option.

.. option:: --profiles

   Build the comma separated list of profiles. The default is ``tier-1``.

.. option:: --arch

   A comma separated list of architecures to build using the selected build.

.. option:: --bsp

   A comma separated list of BSPs to build where a BSP is of the format
   ``arch/bsp`` using the selected build.

.. option:: --build

   The build to be used. The default is ``all``. See ``--config-report`` for a
   list of vlaid builds.

.. option:: --jobs

   The jobs options where the format is ``build-jobs/make-jobs``. The default
   is ``1/num-cores`` where ``num-cores`` is the operating system reported
   number of cores.

.. option:: --dry-run

   Do not do the actual builds just show what would be built.

Examples
^^^^^^^^

The following is a *tier-1* profile build of *all* on a machine where all the
source and tools are located on fast SSD disks and the build happens on a
spinning disk mounted under `build`. The build uses a development source tree
that is bootstrapped and ready to build. The source can have local patches that
need to be regression tested:

.. code-block:: shell

  $ /opt/rtems/5/bin/rtems-bsp-builder --build-path=/build/rtems \
            --rtems-tools=/opt/work/rtems/5 \
            --rtems=/opt/work/chris/rtems/kernel/rtems.git \
            --profiles=tier-1 \
            --jobs=5/10
  RTEMS Tools Project - RTEMS Kernel BSP Builder, 5.not_released
  Profile(s): tier-1
  Cleaning: bsp-builds
  [  1/655] arm/altcycv_devkit (debug)                         Start
  [  1/655] arm/altcycv_devkit (debug)                         Creating: bsp-builds/arm/altcycv_devkit.debug
  [  2/655] arm/altcycv_devkit (no-posix)                      Start
  [  2/655] arm/altcycv_devkit (no-posix)                      Creating: bsp-builds/arm/altcycv_devkit.no-posix
  [  3/655] arm/altcycv_devkit (posix)                         Start
  [  1/655] arm/altcycv_devkit (debug)                         Configuring
  [  3/655] arm/altcycv_devkit (posix)                         Creating: bsp-builds/arm/altcycv_devkit.posix
  [  2/655] arm/altcycv_devkit (no-posix)                      Configuring
  [  4/655] arm/altcycv_devkit (posix-debug)                   Start
  [  1/655] arm/altcycv_devkit (debug)                         Building
  [  3/655] arm/altcycv_devkit (posix)                         Configuring
  [  4/655] arm/altcycv_devkit (posix-debug)                   Creating: bsp-builds/arm/altcycv_devkit.posix-debug
  [  2/655] arm/altcycv_devkit (no-posix)                      Building
  [  5/655] arm/altcycv_devkit (posix-profiling)               Start
  [  4/655] arm/altcycv_devkit (posix-debug)                   Configuring
  [  3/655] arm/altcycv_devkit (posix)                         Building
   ....
  [654/655] sparc/ngmp (posix-profiling)                       PASS
  [654/655] sparc/ngmp (posix-profiling)                       Warnings:0  exes:0  objs:0  libs:0
  [654/655] sparc/ngmp (posix-profiling)                       Finished (duration:0:01:49.002189)
  [654/655] sparc/ngmp (posix-profiling)                       Status: Pass:  655  Fail:    0 (configure:0 build:0)
  [655/655] sparc/ngmp (profiling)                             PASS
  [655/655] sparc/ngmp (profiling)                             Warnings:0  exes:0  objs:0  libs:0
  [655/655] sparc/ngmp (profiling)                             Finished (duration:0:01:260.002098)
  [655/655] sparc/ngmp (profiling)                             Status: Pass:  655  Fail:    0 (configure:0 build:0)
  [651/655] sparc/ngmp (no-posix)                              Cleaning: bsp-builds/sparc/ngmp.no-posix
  [652/655] sparc/ngmp (posix)                                 Cleaning: bsp-builds/sparc/ngmp.posix
  [653/655] sparc/ngmp (posix-debug)                           Cleaning: bsp-builds/sparc/ngmp.posix-debug
  [654/655] sparc/ngmp (posix-profiling)                       Cleaning: bsp-builds/sparc/ngmp.posix-profiling
  [655/655] sparc/ngmp (profiling)                             Cleaning: bsp-builds/sparc/ngmp.profiling
  Total: Warnings:31689  exes:6291  objs:793839  libs:37897
  Failures:
   No failure(s)
  Average BSP Build Time: 0:00:18.165000
  Total Time 3:41:48.075006
  Passes: 655   Failures: 0

To build a couple of BSPs you are interested in with tests:

.. code-block:: shell

  $ /opt/rtems/5/bin/rtems-bsp-builder --build-path=/build/rtems \
            --rtems-tools=/opt/work/rtems/5 \
            --rtems=/opt/work/chris/rtems/kernel/rtems.git \
            ----log=lpc-log \
            --bsp=arm/lpc2362,arm/lpc23xx_tli800 \
            --build=tests \
            --jobs=5/12
  RTEMS Tools Project - RTEMS Kernel BSP Builder, 5.not_released
  BSPS(s): arm/lpc2362, arm/lpc23xx_tli800
  Cleaning: bsp-builds
  [1/2] arm/lpc2362 (tests)        Start
  [1/2] arm/lpc2362 (tests)        Creating: bsp-builds/arm/lpc2362.tests
  [2/2] arm/lpc23xx_tli800 (tests) Start
  [2/2] arm/lpc23xx_tli800 (tests) Creating: bsp-builds/arm/lpc23xx_tli800.tests
  [1/2] arm/lpc2362 (tests)        Configuring
  [2/2] arm/lpc23xx_tli800 (tests) Configuring
  [1/2] arm/lpc2362 (tests)        Building
  [2/2] arm/lpc23xx_tli800 (tests) Building
  [1/2] arm/lpc2362 (tests)        FAIL
  [1/2] arm/lpc2362 (tests)        Warnings:74  exes:58  objs:1645  libs:74
  [1/2] arm/lpc2362 (tests)        Finished (duration:0:01:31.708252)
  [1/2] arm/lpc2362 (tests)        Status: Pass:    0  Fail:    2 (configure:0 build:2)
  [2/2] arm/lpc23xx_tli800 (tests) FAIL
  [2/2] arm/lpc23xx_tli800 (tests) Warnings:74  exes:51  objs:1632  libs:74
  [2/2] arm/lpc23xx_tli800 (tests) Finished (duration:0:01:31.747582)
  [2/2] arm/lpc23xx_tli800 (tests) Status: Pass:    0  Fail:    2 (configure:0 build:2)
  [1/2] arm/lpc2362 (tests)        Cleaning: bsp-builds/arm/lpc2362.tests
  [2/2] arm/lpc23xx_tli800 (tests) Cleaning: bsp-builds/arm/lpc23xx_tli800.tests
  Total: Warnings:74  exes:109  objs:3277  libs:148
  Failures:
     1 tests arm/lpc2362 build:
        configure: /opt/work/chris/rtems/kernel/rtems.git/configure --target\
        =arm-rtems5 --enable-rtemsbsp=lpc2362 --prefix=/opt/rtems/5\
        --enable-tests
       error: ld/collect2:0 error: math.exe section '.rodata' will not fit
              in region 'ROM_INT'; region 'ROM_INT' overflowed by 7284 bytes

     2 tests arm/lpc23xx_tli800 build:
        configure: /opt/work/chris/rtems/kernel/rtems.git/configure --target\
        =arm-rtems5 --enable-rtemsbsp=lpc23xx_tli800\
        --prefix=/opt/rtems/5 --enable-tests
       error: ld/collect2:0 error: math.exe section '.text' will not fit in
              region 'ROM_INT'; region 'ROM_INT' overflowed by 13972 bytes

  Average BSP Build Time: 0:00:46.658257
  Total Time 0:01:33.316514
  Passes: 0   Failures: 2

The summary report printed shows both BSP builds failed with the error detail
shown. In this case both are linker related errors where the test do not fit
into the target's available resources.