blueDonkey.org

This Web

All Webs

blueDonkey.org

Other Sites

The I/O System

• Device Names & File Descriptors
• Device Drivers
  • Character Mode
  • Block Mode
  • Layered Drivers
    • SIO
    • USB
• Pseudo-Devices
  • VxWorks Pipes
    • What is in a Pipe
    • Pipes vs Message Queues
    • Pipes & ISRs
    • Changing the Pipe's Queue Options
    • Deleting a Pipe Device
  • Pseudo TTYs
  • Implementing Unix-style Pipes
• I/O Without Drivers
• Bus Information
  • The Local Bus
  • PCI
    • Windows & Address Translation
    • Accessing Configuration Registers
    • Auto-configuration vs Manual Configuration
    • PC BIOS Configuration
    • Endian Issues
  • VME
  • USB
• VxWorks AE and the I/O System
  • Location of Device Drivers
  • Creating New Device Instances

Device Names & File Descriptors

• How the device name space works
• File descriptors & the device table
• ANSI FILE* objects
• VxWorks AE differences

Device Drivers

Character Mode

• Example driver template
• Porting from other platforms
• Use for non-character oriented devices (e.g. framebuffers)

Block Mode

• How to implement one

Layered Drivers

SIO

USB

Pseudo-Devices

VxWorks Pipes

What is in a Pipe

A pipe in VxWorks is basically a light-weight wrapper around a message queue. The creation routine for a pipe, pipeDevCreate(), even takes a maximum number of messages in the pipe, and a maximum message size. So, a pipe is providing an I/O system style interface (i.e. a file descriptor) to the VxWorks message queue.

Pipes vs Message Queues

If a pipe is just a wrapper around a message queue, why not just use the message queue API directly? Having a file descriptor interface gives you a few things that the native message queue API does not have:

1. The ability to use the queue in a select() call
2. The ability to use standard read() and write() calls, or redirect standard I/O onto it
3. Portability to other platforms
4. Independence from the underlying transport (you could make it a socket to a remote board and it would still work).

At the same time, using the pipeDrv wrapper does hide some of the features of message queues:

1. Message priority
2. Task queuing order (tasks waiting on a pipe are usually FIFO queued - see below)

Since they are implemented as a wrapper around the message queue, they are also slightly slower than the native message queue API. For larger messages this additional overhead will probably be insignificant compared to the cost of copying the data.

Pipes & ISRs

The pipe driver is the only I/O system driver that allows use of its write method from interrupt level. Unlike a normal write() call though, it will always be non-blocking. If the underlying queue is full, then the message will be discarded (and the call will return ERROR allowing the ISR to at least record the loss).

Changing the Pipe's Queue Options

By default, the message queue underlying a pipe device will be created with the MSG_Q_FIFO option. This can be changed to MSG_Q_PRIORITY by changing the value of the global variable pipeMsgQOptions before creating the pipe device with pipeDevCreate(). This change affect all pipes created after it has been made.

It is not possible to change the priority of an individual message, nor the blocking behaviour of the pipe (the pipe driver does not support non-blocking I/O).

Deleting a Pipe Device

In VxWorks 5.x there is no pipeDevDelete() function (there is such a function in VxWorks AE). Worse still, the data structure that represents the pipe device in memory is private to the code, so there is no easy way to implement this function. A little research of the pipe device structure though reveals that it looks something like this:

struct pipe_device {
        DEV_HDR          deviceHeader;   /* I/O system device header */
        MSG_Q            messageQueue;   /* The underlying message queue */
        SEL_WAKEUP_LIST  selectList;     /* List of tasks selecting on this pipe */
        int              numberOpen;     /* Number of opens */
} PIPE_DEVICE;

From that it is possible to write a simple routine that will delete a pipe device by name:

STATUS pipeDevDelete (char * name)
{
        PIPE_DEVICE *  device;
        char *         tail = NULL;
        
        // Get the device ID from the name
        
        pipeDevice = iosDevFind (name, &tail);
        
        if (pipeDevice == NULL) {
                return ERROR;
        }
        
        // Error if it is still in use    
        
        if (pipeDevice->numberOpen > 0) {
                return ERROR;
        }
        
        /* -- 1 -- */
        
        // Delete from the I/O system now
        
        iosDevDelete ((DEV_HDR *) pipeDevice);
        
        /* -- 2 -- */
        
        // Terminate the message queue without freeing the
        // object's memory (the memory is part of the device
        // structure)
        
        msgQDestroy (&pipeDevice->messageQueue, FALSE);
        
        // Free the device's memory
        
        free ((char *) pipeDevice);
        
        return OK;
}

There is no need to do anything with the selection list since we already made sure that there were no users of the pipe by checking the number of opens. If you wish to allow deletion of a pipe that is in use, then you will need to clean up the selection list too (see the section in the programmer's guide about writing select() support in a driver for more information on this).

The simple routine above also has a race condition in it: between the places marked 1 and 2 it is possible that another task might open the pipe, defeating the test. Since pipeDevDelete() is able to block, neither task locking nor interrupt locking would guarantee we were safe from this (though they would reduce the probability). A better solution would be to lock the I/O system semaphore, but this is not visible outside of iosLib (the semaphore as well as the lock & unlock methods are statics). User beware (or ask Wind River for the source to both pipeDrv and iosLib so you can do this properly).

Note that we are using an unpublished function in here to destroy the message queue without releasing the memory. This is because the pipe device uses an embedded message queue structure rather than a message queue ID. Like tasks, message queues have a msgQInit() function which initialises a queue in memory you provide. Unlike tasks, this interface, and the corresponding destroy routine are not published. This means that these routines might change in the future. Of course, the structure of the pipe device might also change, so use this code with care, and only if you really need to delete the pipe. You might consider implementing your own pipe (like one of the ones below) if you need to be certain of portability to newer versions of VxWorks.

Pseudo TTYs

Implementing Unix-style Pipes

• Add example code

I/O Without Drivers

• Direct access to I/O space
• Address translation issues

Bus Information

The Local Bus

The local bus is the bus the CPU, and normally the memory, is connected to. Memory in this context also includes ROM or flash memory. Microcontroller devices may also have a range of on-chip I/O devices on this bus, whereas the higher end processor-only CPUs will probably have a host bridge to a PCI bus in most cases. I/O devices are then connected to the PCI bus, or to another bus bridged off of the PCI bus (e.g. an ISA bus).

When working with the local bus, there is no need to translate CPU physical addresses to bus addresses; they are the same thing. In most VxWorks systems, since the CPU's virtual address is the same as the physical address, this also imlpies that there is no need to translate the virtual addresses used by the software either.

In systems where an MMU or MPU has been set up to translate virtual addresses, or, as with MIPS and SH systems, the kernel is running in a space that is always translated, then an additional virtual to physical translation step may be necessary when working with DMA type devices. There are two macros for converting virtual to physical addresses intended for use in driver code:

• CACHE_DRV_VIRT_TO_PHYS
• CACHE_DMA_VIRT_TO_PHYS

These seem to be used interchangeably in VxWorks drivers (though each driver seems to stick to one of them). The template END driver, and the USB stubs use CACHE_DRV_VIRT_TO_PHYS. These routines will also work on VxWorks AE systems.

There are also corresponding macros for the reverse translation (i.e. physical to virtual):

• CACHE_DRV_PHYS_TO_VIRT
• CACHE_DMA_PHYS_TO_VIRT

Use of these should be avoided where possible though. On systems where the translation is not a simple linear relationship (such as VxWorks AE), this reverse translation can be a time consuming and non-deterministic operation. Most drivers can be written in such a way that the virtual address is cached somewhere so that this reverse translation is unnecessary.

PCI

Windows & Address Translation

While on x86 systems the PCI bus is essentially the CPU's local bus, on other architectures this is not normally the case. Between the CPU's local bus and the PCI bus there is a host bridge device. This bridge usually has three distinct areas of operation:

1. Allowing the CPU to access PCI address spaces. This usually involves opening a number of small windows onto the full address space of the PCI bus, each with a different PCI transaction type (e.g. a PCI memory window and a PCI I/O window). It is usually impossible to see the whole PCI bus at once from the CPU since it has two 4GB address spaces (memory & I/O space are separate address spaces in the PCI world), and a third special address space for configuration (see below).
    
2. Allowing PCI bus masters (e.g. DMA hardware) to access the main memory (which is usually on the CPU's local bus for performance reasons).
    
3. Allowing the CPU to generate PCI configuration transactions. This is normally achieved by using a special window.

The diagram below shows how all of this fits together:

Figure 4.1 - CPU/PCI bus mappings

These windows are defined by means of macros in the BSP. For some host bridges these values can be changed, for others their values are hardcoded in the bridge chip and cannot be changed. In either case, the addresses are usually encoded in some macros in the BSP. The table below shows some of the more commonly used macros in BSPs.

Macro	Description
CPU_PCI_ISA_IO_ADRS	The start of ISA I/O space, as seen from the CPU.
CPU_PCI_ISA_MEM_ADRS	The start of ISA memory space, as seen from the CPU.
CPU_PCI_IO_ADRS	The start of PCI I/O space, as seen from the CPU.
CPU_PCI_MEM_ADRS	The start of PCI memory space, as seen from the CPU.
PCI_CNFG_ADRS or CPU_PCI_CNFG_ADRS	The start of PCI config space, as seen from the CPU.
PCI_IO_ADRS	The base of PCI I/O space, from the PCI perspective.
PCI_MEM_ADRS	Base of PCI memory space, from the PCI perspective.
PCI2DRAM_BASE_ADRS	Start address of system memory as seen from PCI bus.

Unfortuntely, these macros are certainly not used in all BSPs that support PCI, and unlike the support for VME, there are no defined functions for translating between PCI and local bus addresses that are standard across all BSPs.

Accessing Configuration Registers

In order to be able to access the configuration registers of a PCI device, it is necessary to locate the device. Devices are located using a triple made up of a bus number, a device number and a function number. In order to find those three numbers there are two search functions:

Routine Name	Description
pciFindDevice()	Locate a device based on its PCI vendor ID and PCI device ID.
pciFindClass()	Locate a device based on its PCI class code.

Use the pciFindDevice() function when you want to locate a specific device (this is the most common case for drivers as they tend to be device specific). Use the pciFindClass() function when you do not care about the exact device manufacturer or part number, and instead are interested in locating any device of a specific class.

int bus;
int dev;
int func;

/* Find first instance of device */

if (pciFindDevice (VENDOR, DEVICE, 0, &bus, &dev, &func) != OK)
{
        printf ("Failed to find any matching devices\n");
        return;
}

/* Find first instance of device in specific class */

if (pciFindClass (CLASS_CODE, 0, &bus, &dev, &func) != OK)
{
        printf ("Failed to find any matching devices\n");
        return;
}

Auto-configuration vs Manual Configuration

PC BIOS Configuration

Intel x86/Pentium systems generally come with a BIOS that takes care of all the PCI configuration prior to booting the operating system. This makes the operating system's involvement easier in some ways, but a little more complex in others.

Where it is simpler is that all the PCI devices are allocated address information as requested in their configuration registers, and are enabled. Where it is more complex is that the BSP cannot know what addresses will be allocated. That means that any entries in the system's MMU tables to allow access to the PCI devices will need to be established dynamically at initialisation time.

Endian Issues

One thing to be very careful of when working with PCI bus devices is the endianess of values crossing the bridge. PCI is a little endian bus, and all devices connected to it will be little endian too. As a result, it may be necessary for software to swap the byte order of values being written to PCI device registers, or values being stored in memory that will be read and processed by a PCI device (e.g. DMA chain descriptors).

VxWorks BSPs for big-endian CPU systems that also have PCI support provide routines for reading and writing to PCI devices:

Routine	Description
sysPciInByte()	Read a byte from PCI configuration space (no byte order conversion is necessary when reading just one byte).
sysPciInLong()	Read a 32-bit value from PCI configuration space, converting it to big-endian byte order.
sysPciInWord()	Read a 16-bit value from PCI configuration space, converting it to big-endian byte order.
sysPciOutByte()	Write a byte to PCI configuration space.
sysPciOutLong()	Write a 32-bit value to PCI configuration space, converting it to little-endian byte order.
sysPciOutWord()	Write a 16-bit value to PCI configuration space, converting it to little-endian byte order.
sysPciRead32()	Read a 32-bit value from PCI I/O or memory space, and convert to big-endian byte order.
sysPciWrite32()	Write a 32-bit value to PCI I/O or memory space, converting it to little-endian byte order.

VME

• Types of VME transfer
• Windows & address translation
• Using 64-bit transfers

USB

• Types of transfer
• VxWorks USB stack basics
• Writing isochronous transfer drivers

VxWorks AE and the I/O System

Location of Device Drivers

• Why I/O system drivers need to be in the kernel
• How to implement a direct I/O driver outside the kernel

Creating New Device Instances

• Why creating them from outside the kernel can be problematic

-- JohnGordon - 29 Jul 2003

Attachment:	Action:	Size:	Date:	Who:	Comment:
figure-4-1.draw	action	6584	25 Apr 2003 - 18:49	JohnGordon	TWikiDraw? draw file
figure-4-1.gif	action	5830	25 Apr 2003 - 18:49	JohnGordon	TWikiDraw? GIF file