Define Per-CPU Scheduling State for P-EDF

In this step, we define the scheduler state on each processor that is required to implement P-EDF.

Headers

In preparation, we need to pull in a couple of definitions. Add the following headers to sched_demo.c.

   1 #include <linux/percpu.h>
   2 
   3 #include <litmus/litmus.h>
   4 #include <litmus/rt_domain.h>
   5 #include <litmus/edf_common.h>
   6 

The header litmus/litmus.h contains many types and macros central to LITMUS^RT, litmus/rt_domain.h contains declarations for "real-time domains", which contain ready-made release and ready queues, and litmus/edf_common.h defines EDF priority functions and misc. helper functions.

The Linux header linux/percpu.h is required to make CPU-local allocations.

Per-Processor State

To implement, we need a ready queue and a release queue on each processor, for which we will use the rt_domain_t abstraction. Further, we need to know which task is currently scheduled. For convenience, we'll also keep track of the CPU ID in the local state (this will make preemptions easier in a later step). This leads to the following definition of struct demo_cpu_state, which will encompass all local state of the DEMO scheduler.

   1 struct demo_cpu_state {
   2         rt_domain_t     local_queues;
   3         int             cpu;
   4 
   5         struct task_struct* scheduled;
   6 };
   7 
   8 static DEFINE_PER_CPU(struct demo_cpu_state, demo_cpu_state);
   9 
  10 #define cpu_state_for(cpu_id)   (&per_cpu(demo_cpu_state, cpu_id))
  11 #define local_cpu_state()       (&__get_cpu_var(demo_cpu_state))
  12 

Note that, in Line 8, we statically allocate the state required for the plugin using Linux's per-processor allocation macro DEFINE_PER_CPU(). To make the later code more readable, we also define two accessor macros, cpu_state_for() in Line 10 and local_cpu_state() in Line 11, to wrap Linux's per-cpu data structure API.

Plugin Initialization

To make sure that the per-processor state is properly initialized, we are next going to add the plugin activation callback. The activate_plugin() method of a plugin object is called when the plugin is selected by the user (with setsched in liblitmus) and is a good place to implement plugin initialization tasks.

In the DEMO plugin, we are simply going to initialize the ready queue and the release queue using edf_domain_init() and initialize the two fields cpu and scheduled.

   1 static long demo_activate_plugin(void)
   2 {
   3         int cpu;
   4         struct demo_cpu_state *state;
   5 
   6         for_each_online_cpu(cpu) {
   7                 TRACE("Initializing CPU%d...\n", cpu);
   8 
   9                 state = cpu_state_for(cpu);
  10 
  11                 state->cpu = cpu;
  12                 state->scheduled = NULL;
  13                 edf_domain_init(&state->local_queues, NULL, NULL);
  14         }
  15 
  16         return 0;
  17 }

The updated plugin definition now looks as follows (note that demo_activate_plugin is hooked up to .activate_plugin).

   1 static struct sched_plugin demo_plugin = {
   2         .plugin_name            = "DEMO",
   3         .schedule               = demo_schedule,
   4         .admit_task             = demo_admit_task,
   5         .activate_plugin        = demo_activate_plugin,
   6 };

Testing

Boot the modified kernel in a VM and start recording the TRACE() log before switching to the DEMO plugin. You should be able to see the per-CPU initialization being carried out.

[root@litmus-rt ~]# cat /dev/litmus/log > debug.txt &                                                                                                                                                           
[1] 1459
[root@litmus-rt ~]# setsched DEMO                                                                                                                                                                               
[root@litmus-rt ~]# cat debug.txt 
1 P2 [demo_activate_plugin@litmus/sched_demo.c:48]: Initializing CPU0...
2 P2 [demo_activate_plugin@litmus/sched_demo.c:48]: Initializing CPU1...
3 P2 [demo_activate_plugin@litmus/sched_demo.c:48]: Initializing CPU2...
4 P2 [demo_activate_plugin@litmus/sched_demo.c:48]: Initializing CPU3...
5 P2 [vprintk@kernel/printk.c:883]: <6>Switching to LITMUS^RT plugin DEMO.

In the next step, we are finally going to add scheduling logic.