Issue
After going through available links and code, what I understood about task scheduling on other cores (secondary cores) in SMP is as follow:
System tick handler is always executed on Primary core. Jiffies are incremented on this core.
Part of scheduler code, which handles time slice for threads running on all cores, is executed on the primary core when a time slice of a thread running on a secondary core is over. The scheduler picks a new task for the secondary core and generates a re-scheduling Interrupt (IPI) for the other core to execute the new task.
The in-depth logic is definitely more complex than what I have summarized in above, but at the top level, is this the way SMP works in Linux? Please let me know if this conclusion is correct.
Solution
- System tick handler is always executed on Primary core. Jiffies are incremented on this core.
Yes, this is correct. Only one CPU is responsible for actually updating jiffies. This CPU is tick_do_timer_cpu (which is an int variable holding the CPU number).
It's worth noting that although there is only one CPU updating jiffies, that CPU can change over time depending on the kernel configuration: the special value TICK_DO_TIMER_NONE can be assigned to tick_do_timer_cpu when that CPU goes to sleep (see the code here), and the next CPU that ticks will become the new CPU responsible for jiffies updates.
- Part of scheduler code, which handles time slice for threads running on all cores, is executed on the primary core when a time slice of a thread running on a secondary core is over.
No, this is not true. It would be pretty bad to let a single CPU handle scheduling for all CPUs in the system. Nowadays we have processors with a lot of CPU cores, so leaving the job to a single CPU could easily choke it.
Each CPU keeps track of the tasks running on it and has its own runqueue. When the scheduler interrupt occurs for a CPU, the scheduler is invoked and it will decide what to do according to the current state of the tasks in its runqueue. Tasks can also be moved from one CPU to another when needed for load balancing purposes.
Answering the other questions in your comment:
If each core has its own process book keeping, then it implies each core should get regular ticks. So is it like the timer interrupt is routed to all the cores, for regular ticks?
Yes, indeed that's the case (excluding special situations that can only happen in "tickless" kernels, see links below).
If that's the case, which core will increment the jiffies counter?
As said above, the one defined by tick_do_timer_cpu.
When process is created, to which CPU is it assigned, and how is it decided?
That depends on another part of the scheduler code, which is responsible for this exact purpose. Load balancing is taken into account when creating a new task, and in particular the wake_up_new_task() function (among other things) is responsible for choosing which runqueue the new task should be put in.
See also these two other answers of mine, which may help clarify some doubts:
- Understanding the
scheduler_tick()function of Linux Kernel - How are jiffies incremented in a tickless kernel?
Answered By - Marco Bonelli Answer Checked By - Gilberto Lyons (WPSolving Admin)