From 89e72a80c9557ef0be57985ceb578fd7b74eff1b Mon Sep 17 00:00:00 2001 From: Joel Sherrill Date: Mon, 10 Mar 2014 11:35:16 -0500 Subject: smp.t: Add Background and Operation Sections --- doc/user/smp.t | 207 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 200 insertions(+), 7 deletions(-) (limited to 'doc/user/smp.t') diff --git a/doc/user/smp.t b/doc/user/smp.t index 240613ddea..449987a6e1 100644 --- a/doc/user/smp.t +++ b/doc/user/smp.t @@ -1,7 +1,7 @@ @c @c COPYRIGHT (c) 2014. @c On-Line Applications Research Corporation (OAR). -@c All rights reserved. +@c All rights reserved. @c @chapter Symmetric Multiprocessing Services @@ -9,7 +9,7 @@ @section Introduction This chapter describes the services related to Symmetric Multiprocessing -provided by RTEMS. +provided by RTEMS. The application level services currently provided are: @@ -19,10 +19,203 @@ The application level services currently provided are: @item @code{rtems_task_set_affinity} - Set Task Affinity @end itemize +@c +@c +@c @section Background +@subsection Uniprocessor versus SMP Parallelism + +Uniprocessor systems have long been used in embedded systems. In this hardware +model, there are some system execution characteristics which have long been +taken for granted: + +@itemize @bullet +@item one task executes at a time +@item hardware events result in interrupts +@end itemize + +There is no true parallelism. Even when interrupts appear to occur +at the same time, they are processed in largely a serial fashion. +This is true even when the interupt service routines are allowed to +nest. From a tasking viewpoint, it is the responsibility of the real-time +operatimg system to simulate parallelism by switching between tasks. +These task switches occur in response to hardware interrupt events and explicit +application events such as blocking for a resource or delaying. + +With symmetric multiprocessing, the presence of multiple processors +allows for true concurrency and provides for cost-effective performance +improvements. Uniprocessors tend to increase performance by increasing +clock speed and complexity. This tends to lead to hot, power hungry +microprocessors which are poorly suited for many embedded applications. + +The true concurrency is in sharp contrast to the single task and +interrupt model of uniprocessor systems. This results in a fundamental +change to uniprocessor system characteristics listed above. Developers +are faced with a different set of characteristics which, in turn, break +some existing assumptions and result in new challenges. In an SMP system +with N processors, these are the new execution characteristics. + +@itemize @bullet +@item N tasks execute in parallel +@item hardware events result in interrupts +@end itemize + +There is true parallelism with a task executing on each processor and +the possibility of interrupts occurring on each processor. Thus in contrast +to their being one task and one interrupt to consider on a uniprocessor, +there are N tasks and potentially N simultaneous interrupts to consider +on an SMP system. + +This increase in hardware complexity and presence of true parallelism +results in the application developer needing to be even more cautious +about mutual exclusion and shared data access than in a uniprocessor +embedded system. Race conditions that never or rarely happened when an +application executed on a uniprocessor system, become much more likely +due to multiple threads executing in parallel. On a uniprocessor system, +these race conditions would only happen when a task switch occurred at +just the wrong moment. Now there are N-1 tasks executing in parallel +all the time and this results in many more opportunities for small +windows in critical sections to be hit. + +@subsection Task Affinity + +@cindex task affinity +@cindex thread affinity + +RTEMS provides services to manipulate the affinity of a task. Affinity +is used to specify the subset of processors in an SMP system on which +a particular task can execute. + +By default, tasks have an affinity which allows them to execute on any +available processor. + +Task affinity is a possible feature to be supported by SMP-aware +schedulers. However, only a subset of the available schedulers support +affinity. Although the behavior is scheduler specific, if the scheduler +does not support affinity, it is likely to ignore all attempts to set +affinity. + +@subsection Critical Section Techniques and SMP + +As discussed earlier, SMP systems have opportunities for true parallelism +which was not possible on uniprocessor systems. Consequently, multiple +techniques that provided adequate critical sections on uniprocessor +systems are unsafe on SMP systems. In this section, some of these +unsafe techniques will be discussed. + +In general, applications must use proper operating system provided mutual +exclusion mechanisms to ensure correct behavior. This primarily means +the use of binary semaphores or mutexes to implement critical sections. + +@subsubsection Disable Interrupts + +Again on a uniprocessor system, there is only a single processor which +logically executes a single task and takes interrupts. On an SMP system, +each processor may take an interrupt. When the application disables +interrupts, it generally does so by altering a processor register to +mask interrupts and later to re-enable them. On a uniprocessor system, +changing this in the single processor is sufficient. However, on an SMP +system, this register in @strong{ALL} processors must be changed. There +are no comparable capabilities in an SMP system to disable all interrupts +across all processors. + +@subsubsection Highest Priority Task Assumption + +On a uniprocessor system, it is safe to assume that when the highest +priority task in an application executes, it will execute without being +preempted until it voluntarily blocks. Interrupts may occur while it is +executing, but there will be no context switch to another task unless +the highest priority task voluntarily initiates it. + +Given the assumption that no other tasks will have their execution +interleaved with the highest priority task, it is possible for this +task to be constructed such that it does not need to acquire a binary +semaphore or mutex for protected access to shared data. + +In an SMP system, it cannot be assumed there will never be a single task +executing. It should be assumed that every processor is executing another +application task. Further, those tasks will be ones which would not have +been executed in a uniprocessor configuration and should be assumed to +have data synchronization conflicts with what was formerly the highest +priority task which executed without conflict. + +@subsubsection Disable Preemption + +On a uniprocessor system, disabling preemption in a task is very similar +to making the highest priority task assumption. While preemption is +disabled, no task context switches will occur unless the task initiates +them voluntarily. And, just as with the highest priority task assumption, +there are N-1 processors also running tasks. Thus the assumption that no +other tasks will run while the task has preemption disabled is violated. + +@subsection Task Unique Data and SMP + +Per task variables are a service commonly provided by real-time operating +systems for application use. They work by allowing the application +to specify a location in memory (typically a @code{void *}) which is +logically added to the context of a task. On each task switch, the +location in memory is stored and each task can have a unique value in +the same memory location. This memory location is directly accessed as a +variable in a program. + +This works well in a uniprocessor environment because there is one task +executing and one memory location containing a task-specific value. But +it is fundamentally broken on an SMP system because there are always N +tasks executing. With only one location in memory, N-1 tasks will not +have the correct value. + +This paradigm for providing task unique data values is fundamentally +broken on SMP systems. + +@subsubsection Classic API Per Task Variables + +The Classic API provides three directives to support per task variables. These are: + +@itemize @bullet +@item @code{@value{DIRPREFIX}task_variable_add} - Associate per task variable +@item @code{@value{DIRPREFIX}task_variable_get} - Obtain value of a a per task variable +@item @code{@value{DIRPREFIX}task_variable_delete} - Remove per task variable +@end itemize + +As task variables are unsafe for use on SMP systems, the use of these +services should be eliminated in all software that is to be used in +an SMP environment. It is recommended that the application developer +consider the use of POSIX Keys or Thread Local Storage (TLS). POSIX Keys +are not enabled in all RTEMS configurations. + +@b{STATUS}: As of March 2014, some support services in the +@code{rtems/cpukit} use per task variables. When these uses are +eliminated, the per task variable directives will be disabled when +building RTEMS in SMP configuration. + +@c +@c +@c @section Operations +@subsection Setting Affinity to a Single Processor + +In many embedded applications targeting SMP systems, it is common to lock individual tasks to specific cores. In this way, one can designate a core for I/O tasks, another for computation, etc.. The following illustrates the code sequence necessary to assign a task an affinity for processor zero (0). + +@example +rtems_status_code sc; +cpu_set_t set; + +CPU_EMPTY( &set ); +CPU_SET( 0, &set ); + +sc = rtems_task_set_affinity(rtems_task_self(), sizeof(set), &set); +assert(sc == RTEMS_SUCCESSFUL); +@end example + +It is important to note that the @code{cpu_set_t} is not validated until the +@code{@value{DIRPREFIX}task_set_affinity} call is made. At that point, +it is validated against the current system configuration. + +@c +@c +@c @section Directives This section details the symmetric multiprocessing services. A subsection @@ -76,7 +269,7 @@ NONE rtems_status_code rtems_task_get_affinity( rtems_id id, size_t cpusetsize, - cpu_set_t *cpuset + cpu_set_t *cpuset ); @end example @end ifset @@ -88,7 +281,7 @@ rtems_status_code rtems_task_get_affinity( @table @b @item RTEMS_UNSATISFIED -The cpuset pointer argument is invalid. +The @code{cpuset} pointer argument is invalid. @item RTEMS_UNSATISFIED The @code{cpusetsize} does not match the value of @code{affinitysetsize} @@ -124,7 +317,7 @@ NONE rtems_status_code rtems_task_set_affinity( rtems_id id, size_t cpusetsize, - cpu_set_t *cpuset + cpu_set_t *cpuset ); @end example @end ifset @@ -136,7 +329,7 @@ rtems_status_code rtems_task_set_affinity( @table @b @item RTEMS_UNSATISFIED -The cpuset pointer argument is invalid. +The @code{cpuset} pointer argument is invalid. @item RTEMS_UNSATISFIED The @code{cpusetsize} does not match the value of @code{affinitysetsize} @@ -157,7 +350,7 @@ The @code{id} is invalid. The @code{rtems_task_set_affinity} routine is used to set the @code{affinity.set} field of the thread control object associated -with @code{id}. This value controls the cpuset that the task can +with @code{id}. This value controls the @code{cpuset} that the task can execute on. @subheading NOTES: -- cgit v1.2.3