Any where from the Run object to find out how much CPU the job is consuming?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Any where from the Run object to find out how much CPU the job is consuming?

Michael Carter
Basically the use case.

We've got a few jobs doing:

loop {
  retry(10) {
     withCredetials {
         ... some code. that takes about 10ms. or less
     }
  }
}

It's eating up 100% CPU on the master.   I suspect we have other(s)/cases of people doing things like this and I'm trying to add a way to detect the high CPU usage via my plugin.    

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/b79274ed-e0d3-4503-aa4d-a245e15def6en%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Any where from the Run object to find out how much CPU the job is consuming?

Jesse Glick-4
Run `jstack`? Not sure what this plugin is or how it relates to diagnosing high CPU usage.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2jOwOCn9iMA0oO34jT%2BHZcehJWMD_41y%3D%2BGCpx24y09w%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Any where from the Run object to find out how much CPU the job is consuming?

Michael Carter
Guess I should have added more context.  

I have users adding this type of code to their pipeline scripts.

loop {
  retry(10) {
     withCredetials {
         ... some code. that takes about 10ms. or less
     }
  }
}

Which of course causes a lot of CPU usage on the master due to the frequent lookups to the identical username/password.    What I'm looking is there a way from inside Jenkins https://javadoc.jenkins-ci.org/hudson/model/Run.html to get at how much CPU/Memory the Pipeline Job is used during it's run?

I want to write a Jenkins Plugin (Like an Action Badge or something) to report on it to screen and save to an external audit DB.  But I first must get the resource usage from inside Jenkins somehow.    

Hence the question can I get to it somehow from https://javadoc.jenkins-ci.org/hudson/model/Run.html or do I have to hook into the onStarted/onCompleted triggers with some sort of monitor?
On Tuesday, March 30, 2021 at 3:38:03 PM UTC-4 Jesse wrote:
Run `jstack`? Not sure what this plugin is or how it relates to diagnosing high CPU usage.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/f469a2b6-18d5-4351-bd4a-19b642479647n%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Any where from the Run object to find out how much CPU the job is consuming?

Jesse Glick-4
This is not really possible. There is https://github.com/jenkinsci/workflow-cps-plugin/blob/f522e783e917bc0aa0513a29684285b710163905/src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java#L357-L387 (which is not an API you may use from another plugin) but the `run` timing measures only CPS VM thread usage, not aggregate time used by steps invoked from that thread, and certainly not memory usage however one might define that (a complex topic).

You might have better success detecting and warning about dumb idioms. Not necessarily with a plugin.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1JOgdn_rz8OsCMVXqTLf2kxOjQ%3DoGxyXvBCat3txSRUQ%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Any where from the Run object to find out how much CPU the job is consuming?

Basil Crow
In reply to this post by Michael Carter
On Tue, Mar 30, 2021 at 1:21 PM Michael Carter
<[hidden email]> wrote:
>
> Which of course causes a lot of CPU usage on the master due to the frequent lookups to the identical username/password.

I suppose you have already concluded that the high CPU usage is caused
by the abovementioned credential lookups. But in case you (or anyone
else) are still investigating other possible causes of high CPU usage,
I have had great success with generating flame graphs from Jenkins
controller Java processes using
https://github.com/jvm-profiling-tools/async-profiler. It's pretty
fascinating to look at the flame graph for a minute or so of execution
of even a healthy Jenkins controller: you get a sense of what your
workload typically looks like and therefore develop a baseline against
which to analyze pathological performance later on. It's even more
fascinating to look at the flame graph for Jenkins initialization,
especially on a busy controller.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjqdKGBMbPoEugZaU6fq4txqDqJa4MZSdcv2oSdkx6%2Bx8Q%40mail.gmail.com.