Pipeline equivalent of the elastic axis plugin

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Pipeline equivalent of the elastic axis plugin

Sébastien Hinderer-2
Dear all,

In our legacy matrix jobs we use the elastic axis plugin quite a lot.

This plugin creates a matrix axis from a label: all the nodes with the
given label will be on the axis and thus the given job can be run on all
these nodes.

However, the elastic axis plugin does not provide any pipeline step.
Is there another known way to achieve this in a pipeline, please?

Many thanks in advance for any hint,

Sébastien.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20200817062618.GA170986%40om.localdomain.
Reply | Threaded
Open this post in threaded view
|

Re: Pipeline equivalent of the elastic axis plugin

Mark Waite-2


On Mon, Aug 17, 2020 at 12:26 AM Sébastien Hinderer <[hidden email]> wrote:
Dear all,

In our legacy matrix jobs we use the elastic axis plugin quite a lot.

This plugin creates a matrix axis from a label: all the nodes with the
given label will be on the axis and thus the given job can be run on all
these nodes.

However, the elastic axis plugin does not provide any pipeline step.
Is there another known way to achieve this in a pipeline, please?

Many thanks in advance for any hint,


I really like the elastic axis plugin for many of the things I need to do.  
Unfortunately, it doesn't handle offline agents as well as I'd like.

I use the following pipeline code to replace a job that was using
elastic axis and was blocking because some agents were offline:

def nodes = nodesByLabel label: 'windows || linux || openbsd || freebsd'
nodes = nodes.sort()

Map tasks = [:]

for (int i = 0; i < nodes.size(); i++) {
    def label = nodes[i]
    def stageName = 'Clean ' + label.replaceAll('(-jagent)|(-ssh)|(-mwaite)|(.markwaite.net)|(-beemarkwaite)|(-jenkins)', '')
    tasks[label] = {
        node(label) {
            stage(stageName) {
                checkout scm
                withAnt(installation: 'ant-latest', jdk: 'jdk8') {
                    if (isUnix()) {
                        sh 'ant clean'
                    } else {
                        bat 'ant clean'
                    }
                }
            }
        }
    }
}

timeout(time: 13, unit: 'MINUTES') {
    parallel(tasks)
}

I don't know if that will help in your case, but it was a first
step to meet my needs.  I have many jobs that need the
same transformation.

Sébastien.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20200817062618.GA170986%40om.localdomain.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtFmGVTnMHsts8Pbc%2Bh_bVFi3pAOUrtx0AxbdskSdALk6Q%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: Pipeline equivalent of the elastic axis plugin

Sébastien Hinderer-2
Dear Mark,

Many thakns for your contribution! I think I did find a plugin that was
able to "ignore" the off-line nodes when running a job, that was really
convenient. I can probably find it again if that heps you, just let me
know.

Many thanks for your help on the topic of the message. To be honest, I
find it uqite sad and disappointing that the solutions have to be that
complex and convoluted, but this is not at all a criticism agains your
code, on the countrary I'm just too happy that there is at least one
solution to the problem. It's just that I hoped it would be a simpler
one, but really thanks for sharing it and congratulations for having
come up with it in the first place!

Best wishes,

Sébastien.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20200817140821.GB174062%40om.localdomain.
Reply | Threaded
Open this post in threaded view
|

Re: Pipeline equivalent of the elastic axis plugin

Mark Waite-2


On Mon, Aug 17, 2020 at 8:08 AM Sébastien Hinderer <[hidden email]> wrote:
Dear Mark,

Many thakns for your contribution! I think I did find a plugin that was
able to "ignore" the off-line nodes when running a job, that was really
convenient. I can probably find it again if that heps you, just let me
know.

Many thanks for your help on the topic of the message. To be honest, I
find it uqite sad and disappointing that the solutions have to be that
complex and convoluted, but this is not at all a criticism agains your
code, on the countrary I'm just too happy that there is at least one
solution to the problem. It's just that I hoped it would be a simpler
one, but really thanks for sharing it and congratulations for having
come up with it in the first place!


Sure, all the best in your exploring to find a solution that meets your needs.

The pipeline code lists the online agents that match a label expression, then
iterates over that list of agents to create a node for each online agent.  I  like
the concept very much, since it allows me the option of more advanced selection
conditions than the elastic axis plugin could reasonably provide.

I agree that it is not as simple as the elastic axis plugin.  In my case, the 
simplicity of the elastic axis plugin leaves me with jobs that are always blocked 
because my agents are rarely 100% all online at the same time.

Mark Waite
 
Best wishes,

Sébastien.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20200817140821.GB174062%40om.localdomain.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtFuAR%3DUWB3C2Q28xXHUXPWoFBhY83SkPTyXxD9dgbiTQw%40mail.gmail.com.