Accurate calculation of CPU usage given in percentage in Linux?

It's a question which has been asked many times, however there is no well supported answer I could find.

Many people suggest the use of top command, but if you run top once (because you have a script for example collecting Cpu usage every 1 second) it will always give the same Cpu usage result (example 1, example 2).

A more accurate way to calculate CPU usage, is by reading the values from /proc/stat, but most of the answers use only the first 4 fields from /proc/stat to calculate it (one example here).

/proc/stat/ has 10 fields per CPU core as of Linux kernel 2.6.33!

I also found this Accurately Calculating CPU Utilization in Linux using /proc/stat question which is pointing out the same issue, -that most other questions only take into consideration 4 out of the many fields- but still the answer given here starts with "I think" (not certain), and except that, it is only concerned about the first 7 fields (out of 10 in /proc/stat/)

This perl script uses all of the fields to calculate the CPU usage, which again I do not think is correct after some further investigation.

After taking a quick look into the kernel code here, it looks like, for example, guest_nice and guest fields are always increasing together with nice and user (so they should not be included in the cpu usage calculation, since they are included in nice and user fields already)

/*
* Account guest cpu time to a process.
* @p: the process that the cpu time gets accounted to
* @cputime: the cpu time spent in virtual machine since the last update
* @cputime_scaled: cputime scaled by cpu frequency
*/
static void account_guest_time(struct task_struct *p, cputime_t cputime,
cputime_t cputime_scaled)
{
u64 *cpustat = kcpustat_this_cpu->cpustat;


/* Add guest time to process. */
p->utime += cputime;
p->utimescaled += cputime_scaled;
account_group_user_time(p, cputime);
p->gtime += cputime;


/* Add guest time to cpustat. */
if (task_nice(p) > 0) {
cpustat[CPUTIME_NICE] += (__force u64) cputime;
cpustat[CPUTIME_GUEST_NICE] += (__force u64) cputime;
} else {
cpustat[CPUTIME_USER] += (__force u64) cputime;
cpustat[CPUTIME_GUEST] += (__force u64) cputime;
}
}

So to sum up, what is an accurate way to calculate the CPU usage in Linux and which fields should be considered in the calculations and how (which fields are attributed to the idle time and which fields to non-idle time)?

103731 次浏览
小开
最佳答案

According the htop source code, my assumptions looks like they are valid:

(see static inline double LinuxProcessList_scanCPUTime(LinuxProcessList* this) function at LinuxProcessList.c)

// Guest time is already accounted in usertime
usertime = usertime - guest;                             # As you see here, it subtracts guest from user time
nicetime = nicetime - guestnice;                         # and guest_nice from nice time
// Fields existing on kernels >= 2.6
// (and RHEL's patched kernel 2.4...)
unsigned long long int idlealltime = idletime + ioWait;  # ioWait is added in the idleTime
unsigned long long int systemalltime = systemtime + irq + softIrq;
unsigned long long int virtalltime = guest + guestnice;
unsigned long long int totaltime = usertime + nicetime + systemalltime + idlealltime + steal + virtalltime;

And so, from fields listed in the first line of /proc/stat: (see section 1.8 at documentation)

     user    nice   system  idle      iowait irq   softirq  steal  guest  guest_nice
cpu  74608   2520   24433   1117073   6176   4054  0        0      0      0

Algorithmically, we can calculate the CPU usage percentage like:

PrevIdle = previdle + previowait
Idle = idle + iowait


PrevNonIdle = prevuser + prevnice + prevsystem + previrq + prevsoftirq + prevsteal
NonIdle = user + nice + system + irq + softirq + steal


PrevTotal = PrevIdle + PrevNonIdle
Total = Idle + NonIdle


# differentiate: actual value minus the previous one
totald = Total - PrevTotal
idled = Idle - PrevIdle


CPU_Percentage = (totald - idled)/totald
小开

Hey i was also researching for the topic and found this thread really helpful. I used Vangelis Tasoulas formula to write a small python script for this. Attached is my Python code for the issue. It loads the cpu usage per cpu_id every second. Maybe its helps others as well. Also comments/suggestions are welcome :-)

#!/usr/bin/python
# -*- coding: utf-8 -*-


'''
Created on 04.12.2014


@author: plagtag
'''
from time import sleep
import sys


class GetCpuLoad(object):
'''
classdocs
'''




def __init__(self, percentage=True, sleeptime = 1):
'''
@parent class: GetCpuLoad
@date: 04.12.2014
@author: plagtag
@info:
@param:
@return: CPU load in percentage
'''
self.percentage = percentage
self.cpustat = '/proc/stat'
self.sep = ' '
self.sleeptime = sleeptime


def getcputime(self):
'''
http://stackoverflow.com/questions/23367857/accurate-calculation-of-cpu-usage-given-in-percentage-in-linux
read in cpu information from file
The meanings of the columns are as follows, from left to right:
0cpuid: number of cpu
1user: normal processes executing in user mode
2nice: niced processes executing in user mode
3system: processes executing in kernel mode
4idle: twiddling thumbs
5iowait: waiting for I/O to complete
6irq: servicing interrupts
7softirq: servicing softirqs


#the formulas from htop
user    nice   system  idle      iowait irq   softirq  steal  guest  guest_nice
cpu  74608   2520   24433   1117073   6176   4054  0        0      0      0




Idle=idle+iowait
NonIdle=user+nice+system+irq+softirq+steal
Total=Idle+NonIdle # first line of file for all cpus


CPU_Percentage=((Total-PrevTotal)-(Idle-PrevIdle))/(Total-PrevTotal)
'''
cpu_infos = {} #collect here the information
with open(self.cpustat,'r') as f_stat:
lines = [line.split(self.sep) for content in f_stat.readlines() for line in content.split('\n') if line.startswith('cpu')]


#compute for every cpu
for cpu_line in lines:
if '' in cpu_line: cpu_line.remove('')#remove empty elements
cpu_line = [cpu_line[0]]+[float(i) for i in cpu_line[1:]]#type casting
cpu_id,user,nice,system,idle,iowait,irq,softrig,steal,guest,guest_nice = cpu_line


Idle=idle+iowait
NonIdle=user+nice+system+irq+softrig+steal


Total=Idle+NonIdle
#update dictionionary
cpu_infos.update({cpu_id:{'total':Total,'idle':Idle}})
return cpu_infos


def getcpuload(self):
'''
CPU_Percentage=((Total-PrevTotal)-(Idle-PrevIdle))/(Total-PrevTotal)


'''
start = self.getcputime()
#wait a second
sleep(self.sleeptime)
stop = self.getcputime()


cpu_load = {}


for cpu in start:
Total = stop[cpu]['total']
PrevTotal = start[cpu]['total']


Idle = stop[cpu]['idle']
PrevIdle = start[cpu]['idle']
CPU_Percentage=((Total-PrevTotal)-(Idle-PrevIdle))/(Total-PrevTotal)*100
cpu_load.update({cpu: CPU_Percentage})
return cpu_load




if __name__=='__main__':
x = GetCpuLoad()
while True:
try:
data = x.getcpuload()
print data
except KeyboardInterrupt:


sys.exit("Finished")
小开

The following is a bash script which is based on Vangelis's answer. It produces output like this:

total 49.1803
cpu0 14.2857
cpu1 100
cpu2 28.5714
cpu3 100
cpu4 30
cpu5 25

Create a file called get_cpu_usage.sh

Run it using the following command: bash get_cpu_usage.sh 0.2

The argument is the number of seconds to measure. In this case it's 200 milliseconds.

The contents are:

#!/bin/sh


sleepDurationSeconds=$1


previousDate=$(date +%s%N | cut -b1-13)
previousStats=$(cat /proc/stat)


sleep $sleepDurationSeconds


currentDate=$(date +%s%N | cut -b1-13)
currentStats=$(cat /proc/stat)


cpus=$(echo "$currentStats" | grep -P 'cpu' | awk -F " " '{print $1}')


for cpu in $cpus
do
currentLine=$(echo "$currentStats" | grep "$cpu ")
user=$(echo "$currentLine" | awk -F " " '{print $2}')
nice=$(echo "$currentLine" | awk -F " " '{print $3}')
system=$(echo "$currentLine" | awk -F " " '{print $4}')
idle=$(echo "$currentLine" | awk -F " " '{print $5}')
iowait=$(echo "$currentLine" | awk -F " " '{print $6}')
irq=$(echo "$currentLine" | awk -F " " '{print $7}')
softirq=$(echo "$currentLine" | awk -F " " '{print $8}')
steal=$(echo "$currentLine" | awk -F " " '{print $9}')
guest=$(echo "$currentLine" | awk -F " " '{print $10}')
guest_nice=$(echo "$currentLine" | awk -F " " '{print $11}')


previousLine=$(echo "$previousStats" | grep "$cpu ")
prevuser=$(echo "$previousLine" | awk -F " " '{print $2}')
prevnice=$(echo "$previousLine" | awk -F " " '{print $3}')
prevsystem=$(echo "$previousLine" | awk -F " " '{print $4}')
previdle=$(echo "$previousLine" | awk -F " " '{print $5}')
previowait=$(echo "$previousLine" | awk -F " " '{print $6}')
previrq=$(echo "$previousLine" | awk -F " " '{print $7}')
prevsoftirq=$(echo "$previousLine" | awk -F " " '{print $8}')
prevsteal=$(echo "$previousLine" | awk -F " " '{print $9}')
prevguest=$(echo "$previousLine" | awk -F " " '{print $10}')
prevguest_nice=$(echo "$previousLine" | awk -F " " '{print $11}')


PrevIdle=$((previdle + previowait))
Idle=$((idle + iowait))


PrevNonIdle=$((prevuser + prevnice + prevsystem + previrq + prevsoftirq + prevsteal))
NonIdle=$((user + nice + system + irq + softirq + steal))


PrevTotal=$((PrevIdle + PrevNonIdle))
Total=$((Idle + NonIdle))


totald=$((Total - PrevTotal))
idled=$((Idle - PrevIdle))


CPU_Percentage=$(awk "BEGIN {print ($totald - $idled)/$totald*100}")


if [[ "$cpu" == "cpu" ]]; then
echo "total "$CPU_Percentage
else
echo $cpu" "$CPU_Percentage
fi
done
小开

idnt.net has a good description for how to use the /proc/stat cpu data, include a bash-script for extracting cpu and description of the lines. I just wanted to link it here, since I found it valuable.

小开

I was also looking for the same. Here is my ruby program based on the Vangelis Tasoulas's answer:

#!/usr/bin/env ruby
$VERBOSE = true


prev_file = IO.readlines(::File.join('', 'proc', 'stat')).select { |line| line.start_with?('cpu') }
Kernel.sleep(0.05)
file = IO.readlines(::File.join('', 'proc', 'stat')).select { |line| line.start_with?('cpu') }


file.size.times do |i|
data, prev_data = file[i].split.map(&:to_f), prev_file[i].split.map(&:to_f)


%w(user nice sys idle iowait irq softirq steal).each_with_index do |el, index|
eval "@#{el}, @prev_#{el} = #{data[index + 1]}, #{prev_data[index + 1]}"
end


previdle, idle = @prev_idle + @prev_iowait, @idle + @iowait
totald = idle + (@user + @nice + @sys + @irq + @softirq + @steal) -
(previdle + (@prev_user + @prev_nice + @prev_sys + @prev_irq + @prev_softirq + @prev_steal))


puts "CPU #{i}: #{((totald - (idle - previdle)) / totald * 100).round(2)} %"
end
小开

The following is a bash script builds on Fidel's answer and arberg's link.

I wanted to lower the usage of cat awk grep and date calls and spend less cpu usage trying to figure out the cpu usage.

Output:

total: 4%
cpu0: 10%
cpu1: 5%
cpu2: 1%
cpu3: 1%

create a bash script using this:

#!/bin/bash


# Paramiter one used to set time in sec between reads
sleepDurationSeconds=$1


# read cpu stats to arrays
readarray -t previousStats < <( awk '/^cpu /{flag=1}/^intr/{flag=0}flag' /proc/stat )
sleep $sleepDurationSeconds
readarray -t currentStats < <( awk '/^cpu /{flag=1}/^intr/{flag=0}flag' /proc/stat )


# loop through the arrays
for i in "${!previousStats[@]}"; do
# Break up arrays 1 line sting into an array element for each item in string
previousStat_elemant_array=(${previousStats[i]})
currentStat_elemant_array=(${currentStats[i]})


# Get all columns from user to steal
previousStat_colums="${previousStat_elemant_array[@]:1:7}"
currentStat_colums="${currentStat_elemant_array[@]:1:7}"


# Replace the column seperator (space) with +
previous_cpu_sum=$((${previousStat_colums// /+}))
current_cpu_sum=$((${currentStat_colums// /+}))


# Get the delta between two reads
cpu_delta=$((current_cpu_sum - previous_cpu_sum))


# Get the idle time Delta
cpu_idle=$((currentStat_elemant_array[4]- previousStat_elemant_array[4]))


# Calc time spent working
cpu_used=$((cpu_delta - cpu_idle))


# Calc percentage
cpu_usage=$((100 * cpu_used / cpu_delta))


# Get cpu used for calc cpu percentage used
cpu_used_for_calc="${currentStat_elemant_array[0]}"


if [[ "$cpu_used_for_calc" == "cpu" ]]; then
echo "total: "$cpu_usage"%"
else
echo $cpu_used_for_calc": "$cpu_usage"%"
fi


done

提交答案