blueDonkey.org

This Web

All Webs

blueDonkey.org

Other Sites

The VxWorks Kernel

• The VxWorks Scheduler
  • Priority Pre-Emptive Scheduling
  • Round-Robin Mode or Time Slicing
  • Locking the Scheduler
    • Simple Mutual Exclusion
  • Changing the Scheduling Algorithm
• Wind Kernel Objects
  • VxWorks AE Enhancements
    • Object Naming
    • Object Ownership & Resource Reclamation
• Tasks
  • Basics
    • What is a Task?
    • What Does Context Switching Entail?
    • How to Measure Context Switch Times
  • Task Stacks
    • Setting the Stack Size
    • Detecting Stack Overflow
    • Stack Overflow Protection (VxWorks AE)
  • Task Entry Functions
    • Entry Function Name
    • Argument Types
    • Passing argc/argv-Like Parameters
  • Using taskInit()
  • Restarting a Task
  • VxWorks AE Differences
• Semaphores
• Basics
  • Binary Semaphores
  • Counting Semaphores
  • Mutual Exclusion Semaphores
  • Flushing Semaphores
• Message Queues
  • Basics
  • Named Message Queues
  • Almost Zero Copy Message Queues
• Watchdogs
  • Basics
  • Watchdogs vs POSIX Timers
• Interrupts
  • Interrupts and VxWorks
  • Interrupts and Scheduling
  • Interrupt Numbers and Interrupt Vectors
  • Interrupt Handler Rules
  • Interrupt Priorities
  • Non-Maskable Interrupts
    • Using NMIs for Data Transfer
    • Signalling Tasks from an NMI
• Exceptions & Signals
  • Adding a Custom Exception Handler
  • Adding a Signal Handler
  • Sending Signals
  • Clean Task Exit Code
  • Clean Task Restart Code
  • Signals and System Calls

The VxWorks Scheduler

Priority Pre-Emptive Scheduling

The base scheduling algorithm that VxWorks provides is called priority pre-emptive. This basically means that the highest priority ready task will always run. In its default mode, even if other tasks at the same priority level are ready, the running task will continue until it either blocks voluntarily, or a higher priority task becomes ready. This can be changed (see the section about round-robin mode below).

In VxWorks, the priority values range from 0, the highest priority, to 255, the lowest. How you assign priority to your tasks will depend on a number of factors. Here are a few things to consider:

• Any tasks that are responsible for meeting strict deadlines, or have very strict periodic requirements should be assigned a high enough priority to make sure that they will meet these deadlines. Minimize the number of tasks running with higher priority, and if possible calculate the worst case latency and/or interruption due to higher priority tasks.

• Tasks that perform intensive computational work should normally be a lower priority than tasks that need to respond to external events. If such a task also has a deadline though, it might need to be assigned a higher priority, permanently or just temporarily as needed. Or it might make use of the task locking facility to prevent the scheduler changing to another ready task, even if it is a higher priority one. See taskLock() and taskUnlock() in the VxWorks Reference Manual for more information on locking the scheduler.

• While there are times when adjusting the priority of tasks dynamically can be useful, it is rarely a good idea. The one clear exception to this guideline is the use of the priority inheritance mechanism associated with mutual exclusion semaphores. In that case, the kernel will automatically adjust the priority of a task holding a protected resource to the same level as the highest pending task.

• In general, application tasks should have a lower priority than any of the VxWorks kernel tasks. In particular, be careful when assigning priorities to tasks that make use of networking APIs - they should always be lower priority than the network stack's worker task, tNetTask.

• Perhaps the most important rule of all: never use task priority to achieve synchronization. If you need to make a number of tasks run in a specific sequence, do so using semaphores or message queues, not priority.

Round-Robin Mode or Time Slicing

In addition to the basic priority pre-emptive scheduling, VxWorks offers an option to enable round-robin mode, or time slicing between tasks within a single priority level. Irrespective of this setting, if a high priority tasks hogs the CPU, the lower priority tasks will never get any CPU time.

To enable the round-robin feature, use the kernelTimeSlice() function, passing it the number of system clock ticks to be allocated to each task:

kernelTimeSlice (sysClkRateGet () / 10);    /* 1/10 second per quantum */

To disable the time slicing feature, simply set the quantum to zero:

kernelTimeSlice (0);                        /* Disable time slicing */

Locking the Scheduler

The VxWorks API includes a pair of routines to lock the scheduler on the currently running task, and unlock it again. Those routines are taskLock() and taskUnlock() respectively. Locking a task will prevent any higher priority ones that become ready from running, so it should be used with some caution. Also notice that if a task locks the scheduler, and then blocks voluntarily for any reason (e.g. tries to take a semaphore), it will lose the CPU and the lock will be temporarily removed. When that task regains control, the lock will automatically be re-imposed. This behaviour can be useful, but unless you were wanting to exploit it by design, you should make sure that your task does not block while the scheduler is locked.

The following paragraphs look at some uses for task locking.

Simple Mutual Exclusion

Task locking can provide simple, light-weight mutual exclusion. The overhead of a call to taskLock() will be lower than a corresponding mutex semaphore call, but the impact of the call is much broader: locking the scheduler will prevent any other task from running, even ones totally unrelated to the critical region being protected. Still, there are some cases where it might be useful.

taskLock();       // Enter critical region
...               // "Short" critical region here (should not block)
taskUnlock();     // Exit critical region

Note that while the scope of task locking is much broader than that of a mutex semaphore, it still cannot be used to protect a critical region that is shared with an ISR. For that, only intLock() can be used.

Here's a slightly more advanced code fragment that implements a multiple reader/single writer type mutual exclusion system.

//
// Constants to pass to the lock() routine
//
#define READER 1
#define WRITER 2

//
// Locals for this example; would be better in a dynamically
// allocated structure in a real system.
//
LOCAL SEM_ID mutex;              // mutex semaphore
LOCAL SEM_ID signalRd;           // binary semaphore, created empty
LOCAL SEM_ID signalWr;           // binary semaphore, created empty
LOCAL int    writersPending = 0; // count of the number of writers waiting
LOCAL int    locked = 0;         // resource locked 
LOCAL int    readers = 0;        // number of readers

//

// Lock routine
//

STATUS lock (int mode)
{
        switch (mode)
        {
                case READER:
                {
                        retryRd:
                        semTake (mutex, WAIT_FOREVER);
                        
                        if ((locked != 0) || (writersPending > 0))
                        {
                                // The resource is not available,
                                // so wait for it
                                
                                taskLock ();
                                semGive (mutex);
                                semTake (signalRd, WAIT_FOREVER);
                                taskUnlock ();
                                
                                goto retryRd;
                        }
                        readers++;
                        semGive (mutex);
                        return OK;
                }
                case WRITER:
                {
                        retryWr:
                        semTake (mutex, WAIT_FOREVER);
                        
                        if ((locked != 0) ||
                                (writersPending > 0) ||
                                (readers > 0))
                        {
                                // Resource not available for writing
                                
                                writersPending++;     // block new readers
                                taskLock ();
                                semGive (mutex);
                                semTake (signalWr, WAIT_FOREVER);
                                taskUnlock ();
                                goto retryWr;
                        }
                        
                        locked = taskIdSelf ();
                        writersPending--;
                        return OK;
                }
                default:
                // Unknown lock mode
                return ERROR;
        }    
}

STATUS unlock ()
{
        semTake (mutex, WAIT_FOREVER);
        
        if (locked == 0)
        {
                // Must be a reader getting out of the
                // critical section
                
                if (readers > 0)
                readers--;
                
                //
                // If the readers count reaches zero and a writer is
                // pending, then signal the writer now that it can go
                //
                if ((writerPending) && (readers == 0))
                {
                        taskLock ();
                        semGive (mutex);
                        semFlush (signalWr);
                        taskUnlock ();
                        
                        return OK;
                }
                
                semGive (mutex);
                return OK;
        }
        else if (locked == taskIdSelf())
        {
                // The writer is leaving the critical region
                
                if (writersPending > 0)
                {
                        //
                        // Writers are pending, so signal them; we
                        // give writers priority here
                        //
                        semFlush (signalWr);
                }
                else
                {
                        //
                        // There are no writers waiting, so signal
                        // any readers.
                        //
                        semFlush (signalRd);
                }
                
                semGive (mutex);
                return OK;
        }
        else
        {
                // Trying to unlock when we didn't hold the lock
                return ERROR;
        }
}

The lock() routine locks the scheduler to make sure the caller blocks on one of the signal semaphores before any other task gets to run. This is important since the signal semaphores are really only being used as pend queues.

Note that this code example is far from a perfect implementation of the multi-reader/single writer semaphore system. In particular, it is possible for the readers to call unlock() repeatedly, or if they did not have the resource locked in the first place.

You will also need to make sure that the semaphores are initialised somewhere. The mutex can be created with whatever combination of flags you would normally use (I tend to go for SEM_Q_PRIORITY, SEM_DELETE_SAFE and SEM_INVERSION_SAFE). The binary semaphores can be created with either queuing strategy since nothing will ever be giving them, so it will make no difference. The initial state though must be SEM_EMPTY.

Changing the Scheduling Algorithm

TBD

Wind Kernel Objects

• What are they
• Where to find information on the classes of object

VxWorks AE Enhancements

Object Naming

All objects in VxWorks AE can be named. Some must be (e.g. protection domains and tasks), for others the name is optional (e.g. semaphores and message queues). Each class of object has its own namespace, so while it would not be allowed to have two semaphores with the same name, it is allowed to have a semaphore and a message queue sharing a name. That said, it might make more sense to use a convention when naming objects so that they are easier to keep track of.

One possible convention would be something like this:

[application group]/[object class]/[name]

An alternative would be to switch the first two parts of that name:

[object class]/[application group]/[name]

Some examples using these conventions are:

1. XYZ/sem/data_mutex
   XYZ/mq/data_queue
   XYZ/mq/cmd_queue
   system/sem/sync_sem
    
2. sem/XYZ/data_mutex
   mq/XYZ/data_queue
   mq/XYZ/cmd_queue
   sem/system/sync_sem

Setting or Changing an Object's Name:

To set the name of an object, like a semaphore or message queue, that does not have a name parameter for its create method, use the objNameSet() function. This function can also be used to change the name of an existing object, or even remove the name of an object (assuming its class permits anonymous objects). Here are some code examples:

/* Set the name on a message queue */

objNameSet ((OBJ_ID) msgQId, "mq/app/message_queue");

/* Change the name of a semaphore */

objNameSet ((OBJ_ID) semId, "sem/app/renamed_sem");

/* Make a watchdog anonymous again */

objNameSet ((OBJ_ID) wdogId, NULL);

Looking up an Object's ID:

Since all objects are named, it is possible to obtain an ID for a given object from its name and class. This can be useful when using a semaphore or message queue to communicate between two domains since they cannot access the object ID using a shared global variable as was often used in VxWorks 5.x coding. Here's an example of looking up the ID of a message queue using its name:

#include "types/vxWind.h"
#include "msgQLib.h"

MSG_Q_ID msgQIDGet (char * queueName)
{
        return (MSG_Q_ID) objNameToId (windMsgQClass, queueName);
}

Object Ownership & Resource Reclamation

The other big change in the object management that VxWorks AE introduced was object ownership and its associated resource reclamation system. The system allows arbitrary hierarchies of ownership to be created, if that makes sense, but by default uses a model similar to the simpler scheme that is common in the process models of desktop operating systems.

How it Works:

Each object has one and only one parent. The one exception to this rule is the kernel protection domain, which, as the root of the ownership hierarchy has no owner. Each object can own any number of other objects. The only rules are:

• There can be no loops in the graph (i.e. it must be a well-formed tree)

• Tasks may only be owned by their home protection domain, or by another task in that domain.

• The kernel protection domain must always be the root of the ownership tree

By default, when an object is created it will be owned by the home domain of the task that created it. So, if task A in domain D creates a new semaphore, that semaphore will be owned by domain D. The VX_TASK_OBJ_OWNER is an option that can be passed to taskSpawn() or taskCreate() to change this behaviour. If task A had be spawned with this option, then the semaphore would be owned directly by task A instead. This option allows the focus of resource reclamation to be shifted from the domain down to the individual task(s) within it.

You can examine the current ownership hierarchy using the objShowAll() function. When typed at a shell prompt without any parameters, this routine will display information about the kernel domain, and then the complete object ownership hierarchy. Using the Tornado IDE's Inspector tool it is possible to view this ownership tree in a more graphical form, and navigate around it simply by clicking. The Inspector is also capable of manipulating these objects (e.g. giving a semaphore); obviously, this feature should be used with great care.

Changing the Ownership of an Object:

To change the ownership of an object, the objOwnerSet() function should be used.

/* Claim ownership of the semaphore for this task */

objOwnerSet ((OBJ_ID) semId, (OBJ_ID) taskIdSelf());

One thing that might be useful in some cases is to give the ownership of an object to the kernel. This will prevent it from ever being deleted as part of a resource reclamation operation, which can be useful if the object is being shared between several domains. The following example shows how to do this:

/* Give the message queue to the kernel */

objOwnerSet ((OBJ_ID) semId, (OBJ_ID) pdIdKernelGet());

Limitations:

The object ownership scheme in VxWorks AE was designed not as a security measure, but rather as a system to support resource reclamation. As such, not being the owner of an object does not prevent a task from acquiring the ID or using the object. Of course, the memory that holds the object's data structure will be protected since it resides in the kernel, but obtaining the ID (e.g. by looking it up from the object's name) will allow any task to use the object via its API.

Resource Reclamation:

When any object is deleted via its delete or destroy method, before the deletion completes an objects that are owned by the object will also be deleted. This is a recursive process that will result in every object in the tree owned by the original one being deleted. There are two exceptions to this, both of which will result in the object being orphaned and its ownership being transferred to the kernel domain:

1. Any object that has been marked as exempt from resource reclamation (this is an internal feature; there is no API to set this flag on given object)
    
2. A special case where the object being deleted indirectly causes the deletion of the current task's home domain. Since it is not possible to delete that domain and keep the task running, the domain will not be deleted. Instead it will be marked as an orphan and its ownership assigned to the kernel domain.

The kernel domain can never be deleted.

Tasks

Basics

What is a Task?

A task is the active element of a VxWorks system. They are the only objects that are scheduled. It is important to separate a task from any code that it may execute during its life. Code is totally passive until executed by a task. As long as it is written correctly (i.e. it follows some simple rules regarding re-entrancy), a piece of code may be executed by more than one task.

Tasks can be thought of as running concurrently, though obviously with only a single CPU this is never truly the case. It is a useful way of thinking about them though when designing a system where tasks must share resources, or must synchronise their activities. In both situations each task must make use of synchronisation primitives, such as semaphores or message queues, to ensure that the overall behaviour is correct irrespective of the order in which the tasks really gain CPU time.

Comparing VxWorks to other operating systems, a task can be thought of as a thread. The real difference between VxWorks and those other operating systems is that VxWorks has only one process in which all the threads (tasks) run.

Looking at VxWorks AE, the same analog of task and thread holds true, but now there is also an analog of the process: the protection domain. Unlike, for example, Unix processes though, the VxWorks AE protection domain is never scheduled. It remains a passive container for tasks, other objects and, of course, memory.

What Does Context Switching Entail?

Context switching is the mechanism by which a multi-tasking operating system switches control between two threads of execution, thereby giving the illusion of concurrency. Of course, the scheduling algorithm, which is the means of deciding when to switch, and which thread to switch to, will have a dramatic effect on how well that illusion is maintained. A desktop operating system has very different needs to an RTOS in this regard. A RTOS scheduling algorithm, such as the priority pre-emptive one used by VxWorks, is more concerned with predictability (determinism) than a general purpose OS, which is generally more concerned with allocating a fair share of resources to each user/process.

For VxWorks, context switching is simply a matter of saving the current state of one task in its TCB, and then restoring the state of the newly selected task. The state in this case is simply the content of the CPU's registers, including the stack pointer and program counter. The context switch time is a function of the number of registers that need to be saved and restored. For this reason, RISC processors, which tend to have many more registers than their CISC counterparts, tend to have longer context switch times. Additionally, extra registers, such as floating point registers also add to this time. It is for that reason that VxWorks tasks can choose whether or not the floating point registers need to be saved. A task that makes no use of floating point arithmetic, does not need to save or restore these registers, and will therefore have a faster context switch time. The VX_FP_TASK option, passed to taskSpawn(), controls whether these registers are saved and restored.

Note: Some versions of the GNU compiler for PowerPC contain an optimisation that allows the compiler to make use of floating point registers for non-floating point operations. The known cases where this occurs are:

1. When working with long long types;
2. When copying structures that are exactly 8 bytes in length.

Unless you are certain that the compiler you are using does not have this optimisation, it is safest to spawn all tasks with the VX_FP_TASK option on PowerPC systems.

How to Measure Context Switch Times

TBD

Task Stacks

Setting the Stack Size

Selecting the size of the task stacks is one of the most important decisions that the system designer needs to make. Setting it too small can lead to some very odd, and difficult to debug crashes; setting it too large will waste memory. There is no simple rule for working out the required size. If you have the ability to analyse the complete calling tree for your application, including any possible recursions or loops in the graph, then it is theoretically possible to calculate the stack requirements for each task.

More practically, use an empirical system to determine a good setting for each task. In the past, I have assigned a large amount of memory to each task stack (I used 1 megabyte because I had enough RAM to do that, but even 100KB is probably more than enough for most tasks). Run the application for a representative amount of time, try as many operations as possible or run an extreme condition test case (if you have such a thing). Once you are happy that you've run the application long enough for it to hit its maximum usage, use checkStack() to see the actual maximum usage for each task. (Make sure that none of your tasks are spawned with the VX_NO_STACK_FILL option as this is the mechanism that checkStack() uses to determine stack usage.)

Take those maximum values, add a safety margin (say 10%) and use these values for your task stack sizes. I would also recommend that you define them all in a single header file so that they cen be easily updated during development should one or more of them need to be changed.

Note: On some versions of VxWorks, with certain architectures, the interrupt handlers will make use of the interrupted task's stack. If your system is one of these, make sure that you add enough additional space to the stack to allow for the worst case nested interrupt handlers too. The architecture supplement that came with your system should tell you whether you need to consider this.

Detecting Stack Overflow

There are a number of ways of detecting stack overflows:

1. The checkStack() utility will warn you if any of your tasks have exceeded their stack allocation.
    
2. If the output of i from the shell shows garbled task names, this is an indication that the task has overrun its allocated stack memory (the task's name is stored at the base of the stack in VxWorks 5.x).
    
3. You experience random crashes. One of the first thing to check if you get unexpected or random crashes is the stack usage. One task overflowing its stack can easily corrupt critical data structures of another task, causing a crash the next time the affected task is scheduled.
    

One thing to note here is that the checkStack() facility makes use of the stack filling option. When tasks are spawned in VxWorks, by default the stack is filled with a special value (0xeeeeeeee). While this has no impact on the performance of the task once it is running, it will add some time to the actual creation of the task. Since spawning of tasks is something that should never be considered deterministic, this should not be a problem for most people, but if system start-up time is critical in your application, then disabling the stack filling in your deployed system could help improve that initial boot time. Use a macro for your task options so that you can easily re-instate the stack filling for debugging purposes should you ever need to.

Stack Overflow Protection (VxWorks AE)

If you have VxWorks AE, non-kernel tasks are protected by one or more guard pages. If a task tries to enter one of these guard pages, it will be suspended with an MMU exception, which should state in parentheses after the exception name that it was caused by stack overflow.

This protection is not perfect. If the task tries to carve more than the size of the guard region in one chunk it will avoid detection. The default number of guard pages is just one, 4KB on most systems, but can be extended by means of the TASK_EXTRA_GUARD_PAGES configuration parameter.

Task Entry Functions

Entry Function Name

The entry point function for your task can be almost anything except main(). The reason for avoiding main() is that the GNU compiler tends to generate some additional code for process initialisation when it sees that function name. Since VxWorks is not a process model operating system, this code is at best not necessary; in the worst case it will cause undefined symbol errors at load time.

Bear in mind that since the code for all your tasks will live in a single namespace, each task's entry point will need a unique name.

Argument Types

Each task can have up to ten arguments, but there are some restrictions of the types of those arguments. The function prototype for taskSpawn() specifies these arguments as integers, which are 32-bit values on VxWorks systems. That means that you can essentially pass any type of parameter that can be cast directly to a 32-bit value without loss of information. That includes smaller integer types, pointers and single precision floating point values on most systems. Passing floating point values from the shell can be difficult though, so it is best avoided if possible.

Passing argc/argv-Like Parameters

For those porting Unix-style applications that expect to have argc and argv style arguments, this is simple to achieve. Either pass the count and string manually, or simply use a wrapper like this:

#define MAX_ARGS    64

STATUS wrapper (
        FUNCPTR func,
        char *  args
)
{
        char *  last;
        char *  ptr;
        int     argc = 0;
        char *  argv[MAX_ARGS];
        
        ptr = strtok_r (args, " ", &last);
        
        while ((ptr) && (argc < MAX_ARGS))
        {
                argv[argc] = ptr;
                argc++;
                ptr = strtok_r (NULL, " ", &last);
        }
        
        /* Check for too many arguments */
        
        if (ptr != NULL)
        {
                return ERROR;
        } else {
                return func (argc, argv);
        }
} 

To use this code, simply spawn the wrapper as follows:

taskSpawn (TASK_NAME,
        PRIORITY,
        TASK_OPTS,
        STACK_SIZE,
        (FUNCPTR) wrapper,          /* wrapper function */
        (FUNCPTR) taskEntry,        /* real entry point */
        "arg1 arg2 arg3 arg4",      /* arg string */
        0,0,0,0,0,0,0,0);           /* unused args */

There are some caveats to the wrapper code, most notable of which is that it will modify the argument string passed into it. This will work correctly on VxWorks 5.x systems, but on a VxWorks AE system might cause problems if the string being passed is a constant since it will be marked as read-only in that case, and protected by the MMU (the -fwritable-strings compiler option can be used to disable this behaviour, but that is not recommended).

Also, restarting the task will not be possible since the argument string has been modified. It is simple to create a version of this that does not suffer from these limitations by making a private copy of the string to modify. That is left as an exercise for the reader.

Using taskInit()

There are really two reasons for using taskInit() instead of taskSpawn():

1. To create a dormant task (i.e. one that will never be scheduled)
    
2. To control the location of the stack memory

The real problem with taskInit() is that you have to allocate the task stack & TCB memory always, making it less desirable for those simply looking to start a dormant task. VxWorks AE introduced a new routine, taskCreate() (not to be confused with the unpublished internal routine taskCreat() that is present in VxWorks 5.x systems). This has the same API as taskSpawn(), but will create a dormant task. VxWorks AE does not support user-allocation of the stack memory; taskInit() cannot be used outside of the kernel domain.

The code below shows an implementation of a routine called taskCreate() that shows how to use taskInit().

int taskCreate (
        char *  name,
        int     priority,
        int     options,
        int     stackSize,
        FUNCPTR entry,
        int     arg1,
        int     arg2,
        int     arg3,
        int     arg4,
        int     arg5,
        int     arg6,
        int     arg7,
        int     arg8,
        int     arg9,
        int     arg10
)
{
        int     ret;
        char  * memArea;
        char  * pStackBase;
        
        /* initialize TCB and stack space */
        
        memArea = (char *) malloc (STACK_ROUND_UP(stackSize) +
                sizeof (WIND_TCB) + 16);
        
        if (memArea == NULL)
        {
                return ERROR;
        }
        
        /*
        * Calculate the base of the stack. Direction 
        * of stack growth depends on architecture.
        */
        
        #if (_STACK_DIR == _STACK_GROWS_DOWN)
        pStackBase = (char *)(memArea + STACK_SIZE + sizeof (WIND_TCB));
        #else
        pStackBase = memArea + STACK_ROUND_UP (sizeof (WIND_TCB) + 16);
        #endif
        
        /* initialize task */
        
        ret = taskInit ((WIND_TCB *) (memArea + 16),
                name,
                priority,
                options,
                pStackBase,
                stackSize,
                entry,
                arg1, arg2, arg3, arg4, arg5,
                arg6, arg7, arg8, arg9, arg10);
        
        if (ret == ERROR)
        {
                return ERROR;
        }
        else
        {
                return (int) memArea;
        }
}

It is possible to achieve the same effect without using taskInit(). The code here shows another implementation of taskCreate().

typedef struct {
        int     arg1,
        int     arg2,
        int     arg3,
        int     arg4,
        int     arg5,
        int     arg6,
        int     arg7,
        int     arg8,
        int     arg9,
        int     arg10
} TASK_ARGS;

static int wrapper (
        FUNCPTR     entry,
        TASK_ARGS * args
)
{
        int ret;
        
        /* Block now until somebody activates us */
        
        taskSuspend (0);
        
        /* Call the main task entry function */
        
        ret = (*entry)(args->arg1, args->arg2, args->arg3,
                args->arg4, args->arg5, args->arg6,
                args->arg7, args->arg8, args->arg9, 
                args->arg10);
        
        /* Free the memory allocated for the task's arguments */
        
        free (args);
        
        /* Return whatever the task's entry point returned to us */
        
        return ret;
}

int taskCreate (
        char *  name,
        int     priority,
        int     options,
        int     stackSize,
        FUNCPTR entry,
        int     arg1,
        int     arg2,
        int     arg3,
        int     arg4,
        int     arg5,
        int     arg6,
        int     arg7,
        int     arg8,
        int     arg9,
        int     arg10
)
{
        TASK_ARGS * args = (TASK_ARGS*) malloc (sizeof (TASK_ARGS));
        
        if (args == NULL)
        {
                return ERROR;
        }
        
        args->arg1  = arg1;
        args->arg2  = arg2;
        args->arg3  = arg3;
        args->arg4  = arg4;
        args->arg5  = arg5;
        args->arg6  = arg6;
        args->arg7  = arg7;
        args->arg8  = arg8;
        args->arg9  = arg9;
        args->arg10 = arg10;
        
        return taskSpawn (name, priority, options, stackSize,
                entry, args, 0, 0, 0, 0, 0, 0, 0, 0, 0); 
}

Restarting a Task

While VxWorks provides a function that allows you to restart a task, taskRestart(), I would urge you to think very carefully about using it - there are some potentially very serious side effects of this function. Perhaps the worst is that it makes little effort to deal with resources that might have been held by the task at the time of the restart. Since it actually deletes the old task, areas protected by a delete-safe mutex should be safe - the taskRestart() will pend until the mutex is unlocked (given).

If you do wish to make use of this function, I would strongly recommend that the following guidelines are observed:

1. Steps are taken to track the state of any file descriptors, semaphores, message queues etc that are used by the task. The initialisation code for the task should only create these if do not already exist.
2. Avoid using it in a way that would have one task restart another. This makes it harder for a task to protect itself against restarts at inappropriate times (e.g. mid way through writing some state information to a file).
3. Make use of mutex semaphores with the delete safety option and taskSafe()/taskUnsafe() functions to protect areas where the task cannot be restarted safely.

Perhaps a nicer way of handling a restart is to use the setjmp() and longjmp() and functions. Once the task's initialisation is completed, a checkpoint is created using the setjmp() function. Any time a restart is signalled (and it could be done using a signal so as to allow remote tasks or even interrupt handlers to restart it), a call is made to longjmp() to restore execution to the checkpoint. Even in this case though, it is important for the task to block the delivery of the signal during critical regions of the code.

The bottom line here is that there is no way safe way to restart a task that was not designed to be restartable. No matter which mechanism you use, as the designer of the system you must ensure that the critical regions are appropriately guarded, and that the current state of the task can be accurately discovered during the start sequence code.

VxWorks AE Differences

• Separation of stack & TCB
• Two stacks
• Priority banding
• Execution mode
• taskCreate() not taskInit()
• Stack overflow protection

Semaphores

Basics

There are three types of semaphore available in the VxWorks world. Each is optimised for a specific function. This section looks at each of the three types in turn, and explains when to use them and when not to. While there are different create routines for each of the three types, they all share the same interface functions.

/* Creation functions */
SEM_ID semBCreate(int options, SEM_B_STATE initialState);
SEM_ID semCCreate(int options, int initialCount);
SEM_ID semMCreate(int options);

/* General use functions */
STATUS semTake(SEM_ID semID, int timeout);
STATUS semGive(SEM_ID semID);
STATUS semFlush(SEM_ID semID);

/* Deletion */
STATUS semDelete(SEM_ID semID);

Binary Semaphores

As the name implies, binary semaphores have just two states: full or empty. When full, a call to semTake() will return immediately and a call to semGive() will have no effect (note: semGive() never blocks). When the semaphore is empty, a call to semTake() will block either until the semaphore is given by another task, or the optional timeout expires. Use WAIT_FOREVER for the timeout value to block until the semaphore is available, and NO_WAIT to return immediately, even if the semaphore is not available. When using a timeout, be sure to check the status return, and if it is ERROR check the value of errno to ensure that the reason was a timeout.

Binary semaphores are optimised for signalling operations between tasks, or from an ISR to task level. They are the fastest of the three semaphore, and are frequently used within VxWorks ISRs. While they could also be used for mutual exclusion protection, this would be sub-optimal and should be avoided. The mutex semaphore type is designed for ensuring mutual exclusion and has some extra features to protect against misuse.

Counting Semaphores

Counting semaphores are the logical extension of the binary semaphore. Now, rather than just two states, the semaphore is a counter. As long as the count is greater than zero a call to semTake() will return immediately (decrementing the count first of course). Calls to semGive() increment the count. The timeouts work the same way as in the binary semaphore case.

Use a counting semaphore when there are a fixed number of resources to be allocated, e.g. in a driver that wants to limit the number of simultaneous opens, or to count events, e.g. in a producer-consumer type application when the producer can generate multiple blocks of data to be consumed.

Mutual Exclusion Semaphores

Mutual exclusion semaphores, or mutex semaphores for short, are designed to protect a critical region of code against simultaneous execution by multiple tasks. They are essentially binary semaphores in that they have two states, locked and unlocked, but they also have some additional features designed to further protect against buggy code getting into a critical region.

The key differences between mutex semaphores and binary semaphores are as follows:

• Recursive
• Priority inversion protection
• Delete safety

Flushing Semaphores

Message Queues

Basics

Named Message Queues

• Pipes
• VxWorks AE named objects

Almost Zero Copy Message Queues

• Add code example

Watchdogs

Basics

• Creating a timer
• Starting a timer
• Canceling a timer
• Restrictions on the handler routine

Watchdogs vs POSIX Timers

Interrupts

Interrupts and VxWorks

VxWorks provides some infrastructure code that allows users to attach C handlers to interrupts and not have to worry about saving and restoring CPU context etc. There are a number of different schemes that are found, depending on the CPU and/or BSP:

• Interrupts are vectored in the CPU (i.e. there are individual interrupt vectors for each interrupt source, and the CPU determines which to call based on the source).

• Interrupts are multiplexed onto a single CPU interrupt line, and must be de-multiplexed by software.

• A combination of the two (i.e. some of the interrupt sources can share a single vector)

In the first two cases, the interrupt controller driver provides an implementation for the function intConnect() that will perform whatever actions are necessary to attach the specified C function to the specified interrupt source. Depending on the architecture, this might involve allocating some memory to build a wrapper for the C function, or it could be as simple as just storing the function's address and argument into a table.

The third case is the more complex one. In this environment there are often two routines for connecting interrupts. The standard intConnect() function is used to attach a de-multiplexor to the first level interrupt vector. That facility will then provide a second routine, typically xxxIntConnect(), to inform the de-multiplexor code which routine(s) to call.

An example of this is the PCI interrupt support, which provides the function pciIntConnect() to attach handlers to the PCI interrupt handler chain. The PCI interrupt support code creates a linked list of handlers to call. Each of the routines attached to a single PCI interrupt is called in turn, so it is important that handlers can quickly decide whether the device that they service has caused an interrupt, and if not return.

Interrupt handlers are typically wrapped by calls to two other routines. The C equivalent of the sequence, which is actually hand-coded architecture-dependent assembler, would look something like this:

// Save context
// Setup C environment for handler
// Re-enable interrupts on the CPU
intEnt();

// User's interrupt handler function
handler();

// Invoke scheduler if necessary,
// otherwise restore saved context
intExit();

Interrupts and Scheduling

The intExit() function shown in the previous section is responsible for deciding whether to invoke the scheduler, or simply restore the saved context and return. There are a number of tests that are performed to make this decision. The pseudo-code below shows the sequence that a typical implementation of intExit() might follow:

void intExit()
{
        // Lock interrupts
        
        if (nested)
        {
                // Restore context; the scheduler will be invoked,
                // if necessary by the first interrupt handler's
                // intExit() call when it terminates
                
                return;
        }
        
        if (kernelState)
        {
                // Restore context; anything that would have changed
                // the scheduling decision during the handler's execution
                // would have been deferred to the work queue. It will
                // be executed when the system exits from kernel state.   
                
                return;
        }
        
        if (current task == head of ready queue)
        {
                // Restore context; interrupt handler did not change
                // the highest priority ready task, so drop back to
                // the interrupted task directly
                
                return;
        }
        
        if (scheduler locked && current task ready)
        {
                // Restore context; current task has locked the
                // scheduler, so we cannot switch tasks yet. The
                // switch will happen when the running task
                // releases the lock on the scheduler.
                
                return;
        }
        
        // Save context of interrupted task to its TCB
        
        // Unlock interrupts
        
        reschedule();    // NEVER RETURNS
}

As can be seen from the sequence above, there is a special fast exit path for nested interrupts. Since intEnt() will typically re-enable interrupts at the CPU level at least, it is possible for one interrupt handler to be interrupted by another. Interrupt priority schemes can be used to manage this (see Interrupt Priorities below).

Also, it should be noted that even when the kernel is in a critical region (denoted by kernelState being true in the sequence above), it is still possible to handle interrupts. When an interrupt does arrive while the kernel is in a critical region, any actions it performs that affect kernel objects (e.g. giving a semaphore or sending a message) will be deferred to the work queue. Interrupt handlers that do not exit cleanly, and repeatedly call kernel routines, are one of the more common causes of perhaps the most infamous VxWorks panic message: workQPanic: Kernel work queue overflow.

Interrupt Numbers and Interrupt Vectors

One area of confusion in VxWorks is the meanings of interrupt number and interrupt vector. This is especially true for those that have worked on systems where there is only a single interrupt vector in the CPU, and a de-multiplexor in software handles distribution of incoming interrupts since in these systems the value of the vector and the number are often the same (the vector in these cases is often just the index into an array of function pointers).

An interrupt vector traditionally is an address in memory where either the handler's address is stored, or a jump/branch instruction to the handler is stored. The block of memory, the location of which is normally defined by the CPU, though it may be configurable, is called the vector table. The interrupt number is simply an index into the vector table. Interrupt numbers are sometimes referred to as interrupt levels.

VxWorks defines two macros that convert between these two values:

Macro	Description
INUM_TO_IVEC	Converts an interrupt number to an interrupt vector.
IVEC_TO_INUM	Converts an interrupt vector back to an interrupt number.

Routines such as intConnect() take an interrupt vector as their parameter. Make sure to use the INUM_TO_IVEC macro to convert interrupt numbers/levels to the appropriate vector when connecting the handler function.

Interrupt Handler Rules

There are some basic rules that must be followed when writing ISRs to ensure that the system will continue to work correctly:

• Do not call any routine that could block. For example, semTake(), malloc(), memPartFree(), read(), msgQReceive() and printf(). All of those routines might cause the calling task to block. Since an ISR is not a task, it cannot block.

• Do not spend a long time in an ISR. If the work you need to do will take some time (e.g. copying a block of data from a device buffer to memory), then defer the work to task level by means of a semaphore or message queue.

• It is on the list above, but printf() deserves special mention since it is commonly used for debugging. Do not use it in an ISR. Instead, use logMsg() which is safe for use from an ISR.

• Do not use any routine that operates on the current task - since an ISR is not a task, there is technically no current task. Many taskLib functions can be called (though not all), but do not use the shorthand form of specifying zero for the task ID. Also note that there is no safe way to obtain the ID of the interrupted task. The kernel's idea of the current task may in fact be incorrect if the interrupt occurred while the scheduler was running (which is possible).

The rules are all pretty much common sense, and the VxWorks Programmer's Guide contains a list of the functions that are safe to use in an ISR. Keep to that list of functions, and keep your ISRs as short as possible to avoid problems.

Interrupt Priorities

On most systems interrupts are assigned priorities in hardware. This may be simply assigned based on the interrupt pin they are connected to on the CPU or interrupt controller, i.e. hard-wired, or it may be something that is programmable via a PIC.

When using a prioritised interrupt system, each time an interrupt occurs those of lower priority are automatically locked out. Higher priority interrupts can still arrive, interrupting the current ISR. The reality may not be that simple, and there could be options enabling different schemes. Some common variations are:

• Options controlling the handling of priorities with cascaded interrupt controllers (common in x86 and PowerPC systems)

• Options for changing the time at which the end of interrupt signal is sent to the controller.

• Options for changing the relative priority of the system clock interrupt.

The last of those is worthy of a little more explanation. The system clock is normally connected to a regular interrupt source such as a timer or decrementer. As with any interrupt, it is assigned a priority, but unlike other sources it is not always connected to the interrupt controller. Depending on the architecture it might be assigned the highest priority (e.g. x86 systems) or the lowest (e.g. PowerPC).

Having the clock as the highest priority ensures a regular clock beat, useful in periodic systems. On the other hand, if the clock ISR ever needs to perform a lot of work, such as when a large number of timers expire on the same clock tick, this can result in increased interrupt latency for other devices. If your system is very dependent on interrupt latency, make sure that the clock's interrupt is either a lower priority than your critical one, that interrupts are re-enabled during clock tick processing, or that you keep your timer usage to an absolute minimum (in this context timer also includes timeouts on things like semaphores and message queues).

Non-Maskable Interrupts

Non-maskable interrupts (NMIs), and on some architectures fast interrupts, are not handled by VxWorks? by design. This is to make sure that they are available for application usage with the absolute minimum of overhead. That independence from the operating system though comes with some downsides, perhaps the largest being that you cannot make any operating system calls at all. Of course, you can still make use of C library routines and the like.

Using NMIs for Data Transfer

What can they be used for if they cannot interface with the kernel? Well, operations like transferring data to or from a device on demand (e.g. feeding audio data to DAC) are simple enough to set up. Configure a ring of buffers to hold the data. Fill a buffer and start the device running. When it is ready for more data, the device generates an NMI and the handler simply loads the next block from the ring. Back in the VxWorks space, additional buffers can be filled, and the link word set once the buffer is ready allowing the NMI handler to move to that buffer. If it ever stalls, the VxWorks space can simply restart it.

This is a simple way to use NMIs that takes advantage of their low latency to keep data flowing to a device, while at the same time keeping within the rules of NMIs under VxWorks.

Signalling Tasks from an NMI

This is somewhat harder to do since none of the kernel primitives can be called directly from the NMI handler. Here's two possible solutions:

1. The cleanest mechanism is to have the NMI handler cause a normal external interrupt to be generated. This can often be done using a timer or some other device that is able to generate hardware interrupts on request. Note: you cannot generate a software exception to do this as you don't know the state of the kernel at the time of the NMI; generating an exception while the kernel was in a critical region could cause a kernel panic and reboot the system.
    
2. If latency is not important, then simply set a flag somewhere in memory and have a VxWorks task polling that flag periodically. Make sure that the flag is declared as a volatile to prevent the compiler optimising away the accesses to the flag.

Exceptions & Signals

Adding a Custom Exception Handler

• Exception hooks
• Changing the exception vector

Adding a Signal Handler

• Registering to receive exception-related signals
• Registering for other signals
• Pointer to POSIX section for queued signals (?)

Sending Signals

• Using kill() to send signals to a task
• Using raise() to signal the calling task
• Using sigqueue() to queue a POSIX signal

Clean Task Exit Code

• Code example

Clean Task Restart Code

• Code example

Signals and System Calls

• Effects of interrupting a system call with a signal
• Differences between VxWorks and POSIX systems

-- JohnGordon - 06 Jul 2003