Jenkins queue stuck because of simple queue-prioritizing plugin

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Jenkins queue stuck because of simple queue-prioritizing plugin

George Cimpoies

Hi!

 

I have developed a quite straightforward Jenkins plugin (packaged as .hpi) to prioritize (push to the front of the queue) some builds that arrive in the queue based on certain conditions.

 

You can find the source code below in CustomSorter.java where I basically just override the default comparator and load it at Jenkins initialization.

The comparator then takes the decision between lhs and rhs based on my custom criteria.

 

After I load the plugin (for 8-12 hours) everything works fine & as expected (builds that meet my criteria get pushed to the front of the queue), but then for no apparent reason the queue gets stuck and thus very long since no jobs are executed even if all(or most of) the executors are free.

 

I have absolutely no idea what might be causing the issue and how I could track down the root cause(I’ve added logging to each comparison so that I have the result of each comparison in the Jenkins logs).

 

I’ll leave the .hpi and .java file below.


P.S. I've also opened a bug here https://issues.jenkins-ci.org/browse/JENKINS-51600.


Link to the plugin .hpi

https://drive.google.com/file/d/1lN8Bw0jqCGk-OpSCKMH6QvksGSKr7-Oi/view?usp=sharing

 

Thanks in advance,

George



--
George Cimpoies| Software Engineer - Cluj | ULLINK
1 Victor Deleu Street, 2nd & 3rd Floor | Cluj-Napoca, Cluj 400112 | [hidden email] |

The information contained in or attached to this email is strictly confidential. If you are not the intended recipient, please notify us immediately by telephone and return the message to us.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CABE7CDeiBbrZODevzZTjN95HA%3DotTks1Xgs9%3D8eaNQdjJ6vgZw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

CustomSorter.java (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins queue stuck because of simple queue-prioritizing plugin

Daniel Beck

> On 30. May 2018, at 11:23, George Cimpoies <[hidden email]> wrote:
>
> I have absolutely no idea what might be causing the issue and how I could track down the root cause(I’ve added logging to each comparison so that I have the result of each comparison in the Jenkins logs).
>

Try https://wiki.jenkins.io/display/JENKINS/Obtaining+a+thread+dump

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/4409C1B3-05C4-4637-8014-826453C75A91%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins queue stuck because of simple queue-prioritizing plugin

George Cimpoies
Obtaining a thread dump would be useful only after the problem is reproduced, but since I can't reproduce it on demand (since it's unpredictable) I'm not sure how that would help.

On Wed, May 30, 2018 at 2:14 PM, Daniel Beck <[hidden email]> wrote:

> On 30. May 2018, at 11:23, George Cimpoies <[hidden email]> wrote:
>
> I have absolutely no idea what might be causing the issue and how I could track down the root cause(I’ve added logging to each comparison so that I have the result of each comparison in the Jenkins logs).
>

Try https://wiki.jenkins.io/display/JENKINS/Obtaining+a+thread+dump

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/4409C1B3-05C4-4637-8014-826453C75A91%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.



--
George Cimpoies| Implementation Engineer - Cluj | ULLINK | T: +40 364 413 815
1 Victor Deleu Street, 2nd & 3rd Floor | Cluj-Napoca, Cluj 400112 | [hidden email] |

The information contained in or attached to this email is strictly confidential. If you are not the intended recipient, please notify us immediately by telephone and return the message to us.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CABE7CDexwqtp0Mv7ANQoP49Jg0_tR8CQ6KbuxCi8G3y6_DG-GA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Jenkins queue stuck because of simple queue-prioritizing plugin

Daniel Beck

> On 30. May 2018, at 13:20, George Cimpoies <[hidden email]> wrote:
>
> Obtaining a thread dump would be useful only after the problem is reproduced, but since I can't reproduce it on demand (since it's unpredictable) I'm not sure how that would help.

Once the problem happens, this might tell you why the queue is stuck (assuming threads are busy somewhere). Of course, needs the problem to occur to have potentially useful output.


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/65C33716-ACFB-4E37-9F7D-CC742501A28F%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.