path: root/libbsd.txt

RTEMS BSD Library Guide
=======================
:toc:
:icons:
:numbered:
:website: http://www.rtems.org/

== Getting Started

== Issues and TODO

* PCI support on x86 uses a quick and dirty hack, see pci_reserve_map().

* Priority queues are broken with clustered scheduling.

* Per-CPU data should be enabled once the new stack is ready for SMP.

* Per-CPU NETISR(9) should be enabled onece the new stack is ready for SMP.

* Multiple routing tables are not supported.  Every FIB value is set to zero
  (= BSD_DEFAULT_FIB).

* Process identifiers are not supported.  Every PID value is set to zero
  (= BSD_DEFAULT_PID).

* User credentials are not supported.  The following functions allow the
  operation for everyone
  - prison_equal_ip4(),
  - chgsbsize(),
  - cr_cansee(),
  - cr_canseesocket() and
  - cr_canseeinpcb().

* A basic USB functionality test that is known to work on Qemu is desirable.

* Adapt generic IRQ PIC interface code to Simple Vectored Interrupt Model
  so that those architectures can use new TCP/IP and USB code.

* freebsd-userspace/rtems/include/sys/syslog.h is a copy from the old
  RTEMS TCP/IP stack. For some reason, the __printflike markers do not
  compile in this environment. We may want to use the FreeBSD syslog.h
  and get this addressed.

* in_cksum implementations for architectures not supported by FreeBSD.
  This will require figuring out where to put implementations that do
  not originate from FreeBSD and are populated via the script.

* MAC support functions are not thread-safe ("freebsd/lib/libc/posix1e/mac.c").

* IFCONFIG(8): IEEE80211 support is disabled.  This module depends on a XML
  parser and mmap().

* get_cyclecount(): The implementation is a security problem.

* What to do with the priority parameter present in the FreeBSD synchronization
  primitives and the thread creation functions?

* TASKQUEUE(9): Support spin mutexes.

* ZONE(9): Review allocator lock usage in rtems-bsd-chunk.c.

* KQUEUE(2): Choose proper lock for global kqueue list.

* TIMEOUT(9): Maybe use special task instead of timer server to call
  callout_tick().

* sysctl_handle_opaque(): Implement reliable snapshots.

* PING6(8): What to do with SIGALARM?

* <sys/param.h>: Update Newlib to use a MSIZE of 256.

* BPF(4): Add support for zero-copy buffers.

* UNIX(4): Fix race conditions in the area of socket object and file node
  destruction.  Add support for file descriptor transmission via control
  messages.

* PRINTF(9): Add support for log(), the %D format specifier is missing in the
  normal printf() family.

* Why is the interrupt server used?  The BSD interrupt handlers can block on
synchronization primitives like mutexes.  This is in contrast to RTEMS
interrupt service routines.  The BSPs using the generic interrupt support must
implement the `bsp_interrupt_vector_enable()` and
`bsp_interrupt_vector_disable()` routines.  They normally enable/disable a
particular interrupt source at the interrupt controller.  This can be used to
implement the interrupt server.  The interrupt server is a task that wakes-up
in case an associated interrupt happens.  The interrupt source is disabled in
a generic interrupt handler that wakes-up the interrupt server task.   Once the
postponed interrupt processing is performed in the interrupt server the
interrupt source is enabled again.

* Convert all BSP linkcmds to use a linkcmds.base so the sections are
easier to insert.

* NIC Device Drivers
- Only common PCI NIC drivers have been included in the initial set. These
do not include any system on chip or ISA drivers.
- PCI configuration probe does not appear to happen to determine if a
NIC is in I/O or memory space. We have worked around this by using a
static hint to tell the fxp driver the correct mode. But this needs to
be addressed.
- The ISA drivers require more BSD infrastructure to be addressed. This was
outside the scope of the initial porting effort.

== FreeBSD Source

You should be able to rely on FreebSD manual pages and documentation
for details on the code itself.

== BSD Library Source

== Initialization of the BSD Library

The initialization of the BSD library is based on the FreeBSD SYSINIT(9)
infrastructure.  The key to initializing a system is to ensure that the desired
device drivers are explicitly pulled into the linked application.  This plus
linking against the BSD library (`libbsd.a`) will pull in the necessary FreeBSD
infrastructure.

The FreeBSD kernel is not a library like the RTEMS kernel.  It is a bunch of
object files linked together.  If we have a library, then creating the
executable is simple.  We begin with a start symbol and recursively resolve all
references.  With a bunch of object files linked together we need a different
mechanism.  Most object files don't know each other.  Lets say we have a driver
module.  The rest of the system has no references to this driver module.  The
driver module needs a way to tell the rest of the system: Hey, kernel I am
here, please use my services!

This registration of independent components is performed by SYSINIT(9) and
specializations:

http://www.freebsd.org/cgi/man.cgi?query=SYSINIT

The SYSINIT(9) uses some global data structures that are placed in a certain
section.  In the linker command file we need this:

-------------------------------------------------------------------------------
.rtemsroset : {
        KEEP (*(SORT(.rtemsroset.*)))
}

.rtemsrwset : {
        KEEP (*(SORT(.rtemsrwset.*)))
}
-------------------------------------------------------------------------------

This results for example in this executable layout:

-------------------------------------------------------------------------------
[...]
 *(SORT(.rtemsroset.*))
 .rtemsroset.bsd.modmetadata_set.begin
                0x000000000025fe00        0x0 libbsd.a(rtems-bsd-init.o)
                0x000000000025fe00                _bsd__start_set_modmetadata_set
 .rtemsroset.bsd.modmetadata_set.content
                0x000000000025fe00        0x8 libbsd.a(rtems-bsd-nexus.o)
 .rtemsroset.bsd.modmetadata_set.content
                0x000000000025fe08        0x4 libbsd.a(kern_module.o)
[...]
 .rtemsroset.bsd.modmetadata_set.content
                0x000000000025fe68        0x4 libbsd.a(mii.o)
 .rtemsroset.bsd.modmetadata_set.content
                0x000000000025fe6c        0x4 libbsd.a(mii_bitbang.o)
 .rtemsroset.bsd.modmetadata_set.end
                0x000000000025fe70        0x0 libbsd.a(rtems-bsd-init.o)
                0x000000000025fe70                _bsd__stop_set_modmetadata_set
[...]
.rtemsrwset     0x000000000030bad0      0x290
 *(SORT(.rtemsrwset.*))
 .rtemsrwset.bsd.sysinit_set.begin
                0x000000000030bad0        0x0 libbsd.a(rtems-bsd-init.o)
                0x000000000030bad0                _bsd__start_set_sysinit_set
 .rtemsrwset.bsd.sysinit_set.content
                0x000000000030bad0        0x4 libbsd.a(rtems-bsd-nexus.o)
 .rtemsrwset.bsd.sysinit_set.content
                0x000000000030bad4        0x8 libbsd.a(rtems-bsd-thread.o)
 .rtemsrwset.bsd.sysinit_set.content
                0x000000000030badc        0x4 libbsd.a(init_main.o)
[...]
 .rtemsrwset.bsd.sysinit_set.content
                0x000000000030bd54        0x4 libbsd.a(frag6.o)
 .rtemsrwset.bsd.sysinit_set.content
                0x000000000030bd58        0x8 libbsd.a(uipc_accf.o)
 .rtemsrwset.bsd.sysinit_set.end
                0x000000000030bd60        0x0 libbsd.a(rtems-bsd-init.o)
                0x000000000030bd60                _bsd__stop_set_sysinit_set
[...]
-------------------------------------------------------------------------------

Here you can see, that some global data structures are collected into
continuous memory areas.  This memory area can be identified by start and stop
symbols.  This constructs a table of uniform items.

The low level FreeBSD code calls at some time during the initialization the
mi_startup() function (machine independent startup).  This function will sort
the SYSINIT(9) set and call handler functions which perform further
initialization.  The last step is the scheduler invocation.

The SYSINIT(9) routines are run in mi_startup() which is called by
rtems_bsd_initialize().

This is also explained in "The Design and Implementation of the FreeBSD
Operating System" section 14.3 "Kernel Initialization".

In RTEMS we have a library and not a bunch of object files.  Thus we need a way
to pull-in the desired services out of the libbsd.  Here the
`rtems-bsd-sysinit.h` comes into play.  The SYSINIT(9) macros have been
modified and extended for RTEMS in `<sys/kernel.h>`:

-------------------------------------------------------------------------------
#ifndef __rtems__
#define	C_SYSINIT(uniquifier, subsystem, order, func, ident)	\
	static struct sysinit uniquifier ## _sys_init = {	\
		subsystem,					\
		order,						\
		func,						\
		(ident)						\
	};							\
	DATA_SET(sysinit_set,uniquifier ## _sys_init)
#else /* __rtems__ */
#define	SYSINIT_ENTRY_NAME(uniquifier)				\
	_bsd_ ## uniquifier ## _sys_init
#define	SYSINIT_REFERENCE_NAME(uniquifier)			\
	_bsd_ ## uniquifier ## _sys_init_ref
#define	C_SYSINIT(uniquifier, subsystem, order, func, ident)	\
	struct sysinit SYSINIT_ENTRY_NAME(uniquifier) = {	\
		subsystem,					\
		order,						\
		func,						\
		(ident)						\
	};							\
	RWDATA_SET(sysinit_set,SYSINIT_ENTRY_NAME(uniquifier))
#define	SYSINIT_REFERENCE(uniquifier)				\
	extern struct sysinit SYSINIT_ENTRY_NAME(uniquifier);	\
	static struct sysinit const * const			\
	SYSINIT_REFERENCE_NAME(uniquifier) __used		\
	= &SYSINIT_ENTRY_NAME(uniquifier)
#define	SYSINIT_MODULE_REFERENCE(mod)				\
	SYSINIT_REFERENCE(mod ## module)
#define	SYSINIT_DRIVER_REFERENCE(driver, bus)			\
	SYSINIT_MODULE_REFERENCE(driver ## _ ## bus)
#define	SYSINIT_DOMAIN_REFERENCE(dom)				\
	SYSINIT_REFERENCE(domain_add_ ## dom)
#endif /* __rtems__ */
-------------------------------------------------------------------------------

Here you see that the SYSINIT(9) entries are no longer static.  The
\*_REFERENCE() macros will create references to the corresponding modules which
are later resolved by the linker.  The application has to provide an object
file with references to all required FreeBSD modules.

The FreeBSD device model is quite elaborated (with follow-ups):

http://www.freebsd.org/cgi/man.cgi?query=driver

The devices form a tree with the Nexus device at a high-level.  This Nexus
device is architecture specific in FreeBSD.  In RTEMS we have our own Nexus
device, see `rtemsbsd/bsp/bsp-bsd-nexus-devices.c`.

=== SYSCTL_NODE Example

During development, we had an undefined reference to
_bsd_sysctl__net_children that we had trouble tracking down. Thanks to
Chris Johns, we located it. He explained how to read SYSCTL_NODE
definitions. This line from freebsd/netinet/in_proto.c is attempting
to add the "inet" node to the parent node "_net".

----
SYSCTL_NODE(_net,      PF_INET,         inet,   CTLFLAG_RW, 0,
        "Internet Family");
----

Our problem was that we could not find where _bsd_sysctl__net_children
was defined. Chris suggested that when in doubt compile with -save-temps
and look at the preprocessed .i files. But he did not need that. He
explained that this the symbol name _bsd_sysctl__net_children was
automatically generated by a SYSCTL_NODE as follows:

* _bsd_ - added by RTEMS modifications to SYSCTL_NODE macro
* sysctl_ - boilerplace added by SYSCTL_NODE macro
* "" - empty string for parent node
* net - name of SYSCTL_NODE
* children - added by SYSCTL macros

This was all generated by a support macro declaring the node as this:

----
struct sysctl_oid_list SYSCTL_NODE_CHILDREN(parent, name);
----

Given this information, we located this SYSCTL_NODE declaration in
kern/kern_mib.c

----
SYSCTL_NODE(, CTL_KERN,   kern,   CTLFLAG_RW, 0,
        "High kernel, proc, limits &c");
----

=== devfs (Device file system) ===

There is a minimal implementation based on IMFS. The mount point is fixed to
"/dev". Note that the devfs is only used by the cdev subsystem. cdev has been
adapted so that the full path (including the leading "/dev") is given to devfs.
This saves some copy operations.

devfs_create() first creates the full path and then creates an IMFS generic node
for the device.

TBD: remove empty paths on devfs_destroy().

== Notes by File ==

altq_subr.c - Arbitrary choices were made in this file that RTEMS would
not support tsc frequency change.  Additionally, the clock frequency
for machclk_freq is always measured for RTEMS.

conf.h - In order to add make_dev and destroy_dev, variables in the cdev
structure that were not being used were conditionally compiled out. The
capability of supporting children did not appear to be needed and was
not implemented in the rtems version of these routines.

== PF (Firewall) ==

It is possible to use PF as a firewall. See
[https://www.freebsd.org/doc/handbook/firewalls-pf.html] for details on the
range of functions and for how to configure the firewall.

The following is necessary to use PF on RTEMS:

- You have to provide a +/etc/pf.os+ file. The firewall can use it for passive
  OS fingerprinting. If you don't want to use this feature, the file may contain
  nothing except a line of comment (for example "# empty").

- If some filters use protocol names (like tcp or udp) you have to provide a
  +/etc/protocols+ file.

- If some filters use service names (like ssh or http) you have to provide a
  +/etc/services+ file.

- Create a rule file (normally +/etc/pf.conf+). See the FreeBSD manual for the
  syntax.

- Load the rule file using the pfctl command and enable pf. Please note that the
  pfctl command needs a lot of stack. You should use at least
  RTEMS_MINIMUM_STACK_SIZE + 8192 Bytes of stack. An example initialisation can
  look like follows:

----
	int exit_code;
	char *params[] = {
		"pfctl",
		"-f",
		"/etc/pf.conf",
		"-e",
		NULL
	};

	exit_code = rtems_bsd_command_pfctl(ARGC(params), params);
	assert(exit_code == EXIT_SUCCSESS);
----

=== Known restrictions ===

- Currently PF on RTEMS always uses the configuration for memory restricted
  systems (on FreeBSD that means systems with less than 100 MB RAM). This is
  fixed in +pfctl_init_options()+.

== Wireless Network (WLAN) ==

The libbsd provides a basic support for WLAN. Note that currently this support
is still in an early state. The WLAN support is _not_ enabled in the default
buildset. You have to configure libbsd with the
`--buildset=buildset/everything.ini` to enable that feature.

The following gives a rough overview over the necessary steps to connect to an
encrypted network with an RTL8188EU based WiFi dongle:

- Reference all necessary module for your BSP. For some BSPs this is already
  done in the nexus-devices.h:

----
	SYSINIT_MODULE_REFERENCE(wlan_ratectl_none);
	SYSINIT_MODULE_REFERENCE(wlan_sta);
	SYSINIT_MODULE_REFERENCE(wlan_amrr);
	SYSINIT_MODULE_REFERENCE(wlan_wep);
	SYSINIT_MODULE_REFERENCE(wlan_tkip);
	SYSINIT_MODULE_REFERENCE(wlan_ccmp);
	SYSINIT_DRIVER_REFERENCE(rtwn_usb, uhub);
	SYSINIT_REFERENCE(rtwn_rtl8188eufw);
----

- Create your wlan device using ifconfig:
  +ifconfig wlan0 create wlandev rtwn0 up+

- Start a wpa_supplicant instance for that device:
  + wpa_supplicant_fork -Dbsd -iwlan0 -c/media/mmcsd-0-0/wpa_supplicant.conf+

Note that the wpa_supplicant will only be active till the device goes down. A
workaround is to just restart it every time it exits.

=== Known restrictions ===

- The network interface (e.g. wlan0) is currently not automatically created. It
  would be nice, if some service would create it as soon as for example a USB
  device is connected. In FreeBSD the names are assigned via rc.conf with lines
  like +wlans_rtwn0="wlan0"+.

- wpa_supplicant hast to be started after the device is created. It has to be
  restarted every time the connection goes down. Instead of this behaviour,
  there should be some service that starts and restarts wpa_supplicant
  automatically if a interface is ready. Probably the dhcpcd hooks could be used
  for that.

- The current wpa_supplicant implementation is protected with a lock so it can't
  be started more than one time. If multiple interface should be used, all have
  to be handled by that single instance. That makes it hard to add interfaces
  dynamically. wpa_supplicant should be reviewed thoroughly whether multiple
  instances could be started in parallel.

- The control interface of wpa_supplicant most likely doesn't work. The wpa_cli
  application is not ported.

== IPSec ==

The IPSec support is optional in libbsd. It is disabled in the default build
set. Please make sure to use a build set with +netipsec = on+.

To use IPSec the following configuration is necessary:

----
SYSINIT_MODULE_REFERENCE(if_gif);
SYSINIT_MODULE_REFERENCE(cryptodev);
RTEMS_BSD_RC_CONF_SYSINT(rc_conf_ipsec)
RTEMS_BSD_DEFINE_NEXUS_DEVICE(cryptosoft, 0, 0, NULL);
----

Alternatively you can use the `RTEMS_BSD_CONFIG_IPSEC` which also includes the
rc.conf support for ipsec. It's still necessary to include a crypto device in
your config (`cryptosoft` in the above sample).

The necessary initialization steps for a IPSec connection are similar to the
steps on a FreeBSD-System. The example assumes the following setup:

- RTEMS external IP: 192.168.10.1/24
- RTEMS internal IP: 10.10.1.1/24
- remote external IP: 192.168.10.10/24
- remote internal IP: 172.24.0.1/24
- shared key: "mysecretkey"

With this the following steps are necessary:

- Create a gif0 device:

----
SHLL [/] #  ifconfig gif0 create
----

- Configure the gif0 device:

----
SHLL [/] # ifconfig gif0 10.10.1.1 172.24.0.1
SHLL [/] # ifconfig gif0 tunnel 192.168.10.1 192.168.10.10
----

- Add a route to the remote net via the remote IP:

----
SHLL [/] # route add 172.24.0.0/24 172.24.0.1
----

- Call `setkey` with a correct rule set:

----
SHLL [/] # cat /etc/setkey.conf
flush;
spdflush;
spdadd  10.10.1.0/24 172.24.0.0/24 any -P out ipsec esp/tunnel/192.168.10.1-192.168.10.10/use;
spdadd 172.24.0.0/24  10.10.1.0/24 any -P in  ipsec esp/tunnel/192.168.10.10-192.168.10.1/use;
SHLL [/] # setkey -f /etc/setkey.conf
----

- Start a ike-daemon (racoon) with a correct configuration.
----
SHLL [/] # cat /etc/racoon.conf
path	pre_shared_key "/etc/racoon_psk.txt";
log	info;

padding # options are not to be changed
{
	maximum_length			20;
	randomize			off;
	strict_check			off;
	exclusive_tail			off;
}

listen  # address [port] that racoon will listen on
{
	isakmp				192.168.10.1[500];
}

remote 192.168.10.10 [500]
{
	exchange_mode			main;
	my_identifier			address 192.168.10.1;
	peers_identifier		address 192.168.10.10;
	proposal_check			obey;
	
	proposal {
		encryption_algorithm	3des;
		hash_algorithm		md5;
		authentication_method	pre_shared_key;
		lifetime		time 3600 sec;
		dh_group		2;
	}
}

sainfo (address 10.10.1.0/24 any address 172.24.0.0/24 any)
{
	pfs_group			2;
	lifetime			time 28800 sec;
	encryption_algorithm		3des;
	authentication_algorithm	hmac_md5;
	compression_algorithm		deflate;
}
SHLL [/] # cat /etc/racoon_psk.txt
192.168.10.10	mysecretkey
SHLL [/] # racoon -F -f /etc/racoon.conf
----

All commands can be called via the respective API functions. For racoon there is
a `rtems_bsd_racoon_daemon()` function that forks of racoon as a task.

Alternatively IPSec can also be configured via rc.conf entries:

----
cloned_interfaces="gif0"
ifconfig_gif0="10.10.1.1 172.24.0.1 tunnel 192.168.10.1 192.168.10.10"
ike_enable="YES"
ike_program="racoon"
ike_flags="-F -f /etc/racoon.conf"
ike_priority="250"

ipsec_enable="YES"
ipsec_file="/etc/setkey.conf"
----

ATTENTION: It is possible that the first packets slip through the tunnel without
encryption (true for FreeBSD as well as RTEMS). You might want to set up a
firewall rule to prevent that.

== Problems to report to FreeBSD ==

The MMAP_NOT_AVAILABLE define is inverted on its usage.  When it is
defined the mmap method is called. Additionally, it is not used
thoroughly. It is not used in the unmap portion of the source.
The file rec_open.c uses the define MMAP_NOT_AVAILABLE to wrap
the call to mmap and file rec_close.c uses the munmap method.