Assignment updates

16/11/21 The syntax of comments inside the code snippets was changed to not use the one line version.

1   Assignment

A common task in real-time systems is to handle asynchronous events. In a computer, such events are usually represented by hardware interrupts. The most important parameter of real-time computer systems is so called interrupt latency i.e. the time from when the event physically occurs to when an application task can handle it. The delay depends on system load, but in a real-time operating system it should be bounded.

In this task, you will write a simple program to measure this delay by using a hardware timer called Tripple Timer Counters (TTC). The timer is basically a counter register, which the hardware automatically increments with certain rate. We will use the timer in so called interval mode. In this mode, when the incrementing register reaches some pre-configured value (called interval), the counter is reset to zero and an interrupt request (IRQ) is generated.

The following timing diagram shows various situations that can occur when an interrupt is handled in a multitasking operating system and how a hardware timer can be used to measure the corresponding latencies:

images/timer.png

In the diagram, the first and the third IRQ is handled without delays: The interrupt service routine (ISR) is executed right after the IRQ arrives and the service task takes control immediately after the ISR finishes (because the semaphore was released in the ISR). For the second IRQ, the ISR cannot be executed right after the IRQ because there was some other part of the operating system running (perhaps another ISR), which disabled the interrupts globally. This leads to delaying the execution of both the ISR and the service task. The fourth IRQ illustrates the case where some task disabled preemption. The ISR is executed right after the IRQ, however when the semaphore is released, the service thread is not executed immediately because of the disabled preemption. This leads to delaying of the service thread execution.

Since the IRQs are generated by the timer, we can easily measure the corresponding latencies by reading the timer's counter.

1.1   Solution guidelines

  1. This project must be compiled as Downloadable Kernel Module, since we need direct access to registers of the hardware timer.

  2. As a base for your code use our config.h (also shown on the bottom of this page).

  3. Entry point function is called CreateTasks. It will setup the registers and spawn the required tasks.

  4. You need to write a simple low-level device driver (in function void timer_isr(void) for the hardware timer. See the hints below for source code examples. The driver will contain the Interrupt Service Routine (as seen in the diagram). This routine will read and store the timer value (ISR Time Stamp) and release the semaphore.

  5. Service thread (task named tService), is awaiting the semaphore to be unlocked. After successful semTake, timer value (Task Time Stamp) is read and stored as well.

  6. The monitoring thread (task named tMonitor) will output the statistics from both measurements (the most interesting statistics is maximum; why?) and data for drawing two separate histograms. Each histogram bin will show how many times the corresponding time value has been measured. The update period of the monitoring threads will be 1 second.

    The output should be composed of three rows: latencies of bins (in µs), histogram of interrupt->ISR latency and histogram of interrupt->Service latency. Values are delimited by comma ',', and '\n' at the end of the row.

  1. (Lab only) What happens with maximums when you load the system by flooding board's network interface with the following command?

    ping -s 64000 -i 0.2 <board_IP_addr>
    

1.2   Lab submission guidelines

In order to submit this assignment during the lab, please prepare:

  1. Histogram of both measurements (interrupt->ISR and interrupt->Service).
    • Plot them both into one figure (but they should be easily distinguishable).
    • Use 1us bins on x-axis.
    • Set y-axis to log scale. Set boundaries from 0.1 so that the '1' are also visible.

2   Hints

2.1   Architecture-dependency

Since we need to access the hardware timer, we must to find the addresses of its registers in Zynq-7000 Technical Reference Manual (TRM). The TTC timer is described in Section 8.5 and its registers in Appendix B.32.

Some information about the registers is provided in VxWorks Zynq7k board support package (BSP), available in xlnx_zynq7k.h. To make your work easier, we extracted the missing information in the form of constant definitions for you:

#include <xlnx_zynq7k.h>

/* register offsets (see TRM B.32) */
#define ZYNQ_TIMER_CLOCK_CTRL       (0x00)
#define ZYNQ_TIMER_COUNTER_CTRL     (0x0c)
#define ZYNQ_TIMER_COUNTER_VAL      (0x18)
#define ZYNQ_TIMER_INTERVAL         (0x24)
#define ZYNQ_TIMER_INTERRUPT        (0x54)
#define ZYNQ_TIMER_INTERRUPT_EN     (0x60)
/* register bit definitions */
#define INTERRUPT_EN_IV             (0x01)
#define CTRL_EN                     (0<<0)
#define CTRL_DIS                    (1<<0)
#define CTRL_INT                    (1<<1)
#define CLOCK_PRESCALE              (0x01 << 1)
#define CLOCK_PRESCALE_EN           (0x1)
/* register access */
#define TTC0_TIMER2_CLOCK_CTRL      (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_CLOCK_CTRL)))
#define TTC0_TIMER2_COUNTER_CTRL    (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_COUNTER_CTRL)))
#define TTC0_TIMER2_COUNTER_VAL     (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_COUNTER_VAL)))
#define TTC0_TIMER2_INTERVAL        (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_INTERVAL)))
#define TTC0_TIMER2_INTERRUPT       (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_INTERRUPT)))
#define TTC0_TIMER2_INTERRUPT_EN    (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_INTERRUPT_EN)))

Please note that the file xlnx_zynq7k.h is not in the include search directories by default. We need to configure compiler to add extra path for include searching. To do this, select Project → Properties → Build Properties → Paths and add the line:

-I$(WIND_BASE)/target/config/xlnx_zynq7k

2.2   Timer driver

To write the timer driver, you will need to configure it via its registers and write an interrupt handler.

We will use the following macros defined in the last section:

  • TTC0_TIMER2_CLOCK_CTRL is the clock configuration register. Because the timer counter is only 16 bit wide we will have to set the prescaler to make the counter overflow less frequently:
    • CLOCK_PRESCALE is the value of the prescaler. We will divide the input clock of 111.111111 MHz by 4.
    • CLOCK_PRESCALE_EN bit enables the prescaler.
  • TTC0_TIMER2_COUNTER_CTRL is the counter configuration register. The values we will use are:
    • CTRL_DIS to disable counter.
    • CTRL_EN to enable counter, i.e. not disable it.
    • CTRL_INT to set Interval mode.
  • TTC0_TIMER2_COUNTER_VAL is the register with the current value of the counter.
  • TTC0_TIMER2_INTERVAL is the maximum value that the counter will count to. Together with the prescaler, this value can be used to set IRQ frequency.
  • TTC0_TIMER2_INTERRUPT_EN register allows enabling different types of interrupts.
  • TTC0_TIMER2_INTERRUPT registers shows, which interrupts are pending.

Wind API provides interrupt connection routines in the intLib – architecture-independent interrupt subroutine library. See the documentation for intConnect(...), intEnable(...) etc. The library identifies the interrupt vectors through a constant called INUM. This constant is passed to the functions as an argument defining the interrupt vector to work with. Our timer's IRQ is identified with INT_LVL_TTC0_2 macro. You can use the following code snippets to configure the timer and its interrupt handler.

#include <intLib.h>
#include <iv.h>
#include <semLib.h>
#include <taskLib.h>
#include <xlnx_zynq7k.h>

/* timer init (see TRM 8.5.5) */
TTC0_TIMER2_COUNTER_CTRL = CTRL_DIS;
TTC0_TIMER2_CLOCK_CTRL = CLOCK_PRESCALE | CLOCK_PRESCALE_EN;
TTC0_TIMER2_INTERVAL = TIM_MAX; /* See "Choosing the timer period" below */
TTC0_TIMER2_INTERRUPT_EN = INTERRUPT_EN_IV;
TTC0_TIMER2_COUNTER_CTRL = CTRL_INT | CTRL_EN;

/* interrupt init */
intConnect(INT_VEC_TTC0_2, timer_isr, 0);
intEnable(INT_LVL_TTC0_2);

/* uninit all */
TTC0_TIMER2_COUNTER_CTRL = CTRL_DIS;
intDisable(INT_LVL_TTC0_2);
intDisconnect(INT_VEC_TTC0_2, timer_isr, 0);

2.3   Interrupt handler

When writing the interrupt handler (also called interrupt service routine (ISR)) keep in mind that after the TTC generates the interrupt, and you read and store the timer value, you need to acknowledge the interrupt request to stop the hardware generating it. This is done by clearing (setting to zero) the corresponding bits in the interrupt register. The register has so called clear-on-read property, so to clear the bit reading the register as shown below is sufficient:

/* read ISR register to clear interrupt (see TRM B.32) */
TTC0_TIMER2_INTERRUPT;

Note that if you do not reset the interrupt, the processor appear as halted, because once you return from the interrupt handler, it will be immediately called again.

2.4   Choosing the timer period

One reaming question is what period to choose for our timer-generated IRQs. The proper value depends on the characteristic periodicity of system load, processor speed etc. Best results are obtained when the period is approximately two times the maximum expected latency. This ensures that we will cover all intervals when IRQs are disabled. A good value for our hardware is about 300 µs.

Since we set the prescaler to 4, the timer increments every 1/111111111*4 seconds, which equals to 36 nanoseconds.

2.5   Config header

/* config.h
 * PSR 6schedlat assignment
 * DO NOT MODIFY
 */

#include <xlnx_zynq7k.h>

/* Register offsets (see TRM B.32) */
#define ZYNQ_TIMER_CLOCK_CTRL       (0x00)
#define ZYNQ_TIMER_COUNTER_CTRL     (0x0c)
#define ZYNQ_TIMER_COUNTER_VAL      (0x18)
#define ZYNQ_TIMER_INTERVAL         (0x24)
#define ZYNQ_TIMER_INTERRUPT        (0x54)
#define ZYNQ_TIMER_INTERRUPT_EN     (0x60)
/* Register bit definitions */
#define INTERRUPT_EN_IV             (0x01)
#define CTRL_EN                     (0<<0)
#define CTRL_DIS                    (1<<0)
#define CTRL_INT                    (1<<1)
#define CLOCK_PRESCALE              (0x01 << 1)
#define CLOCK_PRESCALE_EN           (0x1)
/* Register access */
#define TTC0_TIMER2_CLOCK_CTRL      (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_CLOCK_CTRL)))
#define TTC0_TIMER2_COUNTER_CTRL    (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_COUNTER_CTRL)))
#define TTC0_TIMER2_COUNTER_VAL     (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_COUNTER_VAL)))
#define TTC0_TIMER2_INTERVAL        (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_INTERVAL)))
#define TTC0_TIMER2_INTERRUPT       (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_INTERRUPT)))
#define TTC0_TIMER2_INTERRUPT_EN    (*((volatile UINT32 *)(ZYNQ7K_TTC0_TIMER2_BASE + ZYNQ_TIMER_INTERRUPT_EN)))

/* sysClkRateSet(CLOCK_RATE) */
#define CLOCK_RATE 1000

/* Semaphore that is passed between `timer_isr` and Service task. */
SEM_ID isr_semaphore;

/*
 * timer_isr()
 *
 *  This function is attached to timer as an interrupt handler.
 *  On every call, this function reads and stores the timer value
 *  and releases `isr_semaphore`.
 */
void timer_isr(void);

/*
 * ServiceTask()
 *  : (int) o -- "optional" argument, you can use it freely if you want to
 *
 *  This function is spawned as a Service task with name `tService`.
 *
 *  Service task is running in a loop, waiting for `isr_semaphore`
 *  to be unlocked. After successfully taking this semaphore, timer
 *  value is read and stored.
 */
void ServiceTask(int o);

/*
 * MonitorTask()
 *  : (int) o -- "optional" argument, you can use it freely if you want to
 *
 *  This function is spawned as a Monitor task with name `tMonitor`.
 *
 *  Monitor task is running in a loop, printing out measurement data
 *  from both `timer_isr` and Service task. After printing, task is
 *  suspended for 1 second.
 *
 *  Data format:
 *  The functions prints out three rows every time:
 *   1) First row is the x-axis, values in microseconds, ordered in a strictly
 *      increasing sequence.
 *   2) Second row is histogram for "interrupt--timer_isr" latency. The sequence
 *      has the same size as first row (number of elements) and the values
 *      correspond to the x-axis (number of occurences of given latency).
 *   3) Third row is histogram for "interrupt--ServiceTask" latency. The rest is
 *      same as the second row.
 *   Printed out values are delimited by comma ',' only. Rows are ended with '\n'.
 */
void MonitorTask(int o);

/*
 * CreateTasks()
 *  : (int) seconds -- how many seconds should the measurement last
 *                     0 = INFINITE
 *
 *  Entry point function.
 *
 *  This function makes a measurement of interrupt latency.
 *  First, it prepares registers of the hardware and links
 *  `timer_isr` function as an interrupt handler.
 *  Then, Service and Monitor tasks are spawned.
 *
 *  Whole application (that is, after running this function),
 *  should print to stdout 'Measurement started' message. Then,
 *  every 1 second measurement data are printed. At the end,
 *  after `seconds` seconds have passed, the 'Measurement finished'
 *  message is printed and application is terminated (all tasks deleted,
 *  and interrupt is disabled + disconnected).
 *
 *  Note:
 *  See Monitor task for more information about the measurement data
 *  format.
 *
 *  Example:
 *      -> CreateTasks(1)
 *      Measurement started
 *      0,1,2,3,4,5,6,7
 *      59,984,34,0,45,3,784,35
 *      0,0,0,0,0,13,1159,772
 *      Measurement finished
 *
 *      -> CreateTasks(2)
 *      Measurement started
 *      0,1,2,3,4,5,6,7
 *      59,984,34,0,45,3,784,35
 *      0,0,0,0,0,13,1159,772
 *      0,1,2,3,4,5,6,7
 *      147,1751,134,5,85,13,997,80
 *      0,0,0,0,0,30,1909,1273
 *      Measurement finished
 */
void CreateTasks(int measurements);