Monday, January 3, 2022

[SOLVED] Accurate calculation of CPU usage given in percentage in Linux?

Issue

It's a question which has been asked many times, however there is no well supported answer I could find.

Many people suggest the use of top command, but if you run top once (because you have a script for example collecting Cpu usage every 1 second) it will always give the same Cpu usage result (example 1, example 2).

A more accurate way to calculate CPU usage, is by reading the values from /proc/stat, but most of the answers use only the first 4 fields from /proc/stat to calculate it (one example here).

/proc/stat/ has 10 fields per CPU core as of Linux kernel 2.6.33!

I also found this Accurately Calculating CPU Utilization in Linux using /proc/stat question which is pointing out the same issue, -that most other questions only take into consideration 4 out of the many fields- but still the answer given here starts with "I think" (not certain), and except that, it is only concerned about the first 7 fields (out of 10 in /proc/stat/)

This perl script uses all of the fields to calculate the CPU usage, which again I do not think is correct after some further investigation.

After taking a quick look into the kernel code here, it looks like, for example, guest_nice and guest fields are always increasing together with nice and user (so they should not be included in the cpu usage calculation, since they are included in nice and user fields already)

/*
 * Account guest cpu time to a process.
 * @p: the process that the cpu time gets accounted to
 * @cputime: the cpu time spent in virtual machine since the last update
 * @cputime_scaled: cputime scaled by cpu frequency
 */
static void account_guest_time(struct task_struct *p, cputime_t cputime,
                   cputime_t cputime_scaled)
{
    u64 *cpustat = kcpustat_this_cpu->cpustat;

    /* Add guest time to process. */
    p->utime += cputime;
    p->utimescaled += cputime_scaled;
    account_group_user_time(p, cputime);
    p->gtime += cputime;

    /* Add guest time to cpustat. */
    if (task_nice(p) > 0) {
        cpustat[CPUTIME_NICE] += (__force u64) cputime;
        cpustat[CPUTIME_GUEST_NICE] += (__force u64) cputime;
    } else {
        cpustat[CPUTIME_USER] += (__force u64) cputime;
        cpustat[CPUTIME_GUEST] += (__force u64) cputime;
    }
}

So to sum up, what is an accurate way to calculate the CPU usage in Linux and which fields should be considered in the calculations and how (which fields are attributed to the idle time and which fields to non-idle time)?


Solution

According the htop source code, my assumptions looks like they are valid:

(see static inline double LinuxProcessList_scanCPUTime(LinuxProcessList* this) function at LinuxProcessList.c)

// Guest time is already accounted in usertime
usertime = usertime - guest;                             # As you see here, it subtracts guest from user time
nicetime = nicetime - guestnice;                         # and guest_nice from nice time
// Fields existing on kernels >= 2.6
// (and RHEL's patched kernel 2.4...)
unsigned long long int idlealltime = idletime + ioWait;  # ioWait is added in the idleTime
unsigned long long int systemalltime = systemtime + irq + softIrq;
unsigned long long int virtalltime = guest + guestnice;
unsigned long long int totaltime = usertime + nicetime + systemalltime + idlealltime + steal + virtalltime;

And so, from fields listed in the first line of /proc/stat: (see section 1.8 at documentation)

     user    nice   system  idle      iowait irq   softirq  steal  guest  guest_nice
cpu  74608   2520   24433   1117073   6176   4054  0        0      0      0

Algorithmically, we can calculate the CPU usage percentage like:

PrevIdle = previdle + previowait
Idle = idle + iowait

PrevNonIdle = prevuser + prevnice + prevsystem + previrq + prevsoftirq + prevsteal
NonIdle = user + nice + system + irq + softirq + steal

PrevTotal = PrevIdle + PrevNonIdle
Total = Idle + NonIdle

# differentiate: actual value minus the previous one
totald = Total - PrevTotal
idled = Idle - PrevIdle

CPU_Percentage = (totald - idled)/totald


Answered By - Vangelis Tasoulas
at