Gracefully ignoring unavailable agents

[Mark Raynsford]

Hello!

Let's say I'm using a declarative pipeline setup like this:

~~
#!groovy

pipeline {

  agent none

  stages {
    stage('Build') {
      parallel {
        stage('Build Linux') {
          stages {
            stage('Build:linux:openjdk-11-hotspot') {
              agent {
                label 'linux'
              }
              ...
            }
            stage('Build:linux:openjdk-15-hotspot') {
              agent {
                label 'linux'
              }
              ...
            }
          }
        }

        stage('Build Windows') {
          stages {
            stage('Build:windows:openjdk-11-hotspot') {
              agent {
                label 'windows'
              }
              ...
            }
            stage('Build:windows:openjdk-15-hotspot') {
              agent {
                label 'windows'
              }
              ...
            }
          }
        }
      }
    }
  }
}
~~

As you can see, I use labels to select agents to run builds on specific
operating systems and with a fixed set of specific JDK versions. This
all works fine. However: With my current hardware setup, the Windows
node is _not_ going to be online 24/7 (the Linux node is always
available).

Is there a way I can adapt the above pipeline to basically say "If
there's a Windows node online, run the stages, otherwise trivially
succeed"? Right now, if no Windows node is available, the pipeline will
hang until either one becomes available or the entire job times out.

--
Mark Raynsford | https://www.io7m.com

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20201231212628.447a21b5%40sunflower.int.arc7.info.

[Mark Waite-2]

On Thu, Dec 31, 2020 at 2:27 PM 'Mark Raynsford' via Jenkins Users <[hidden email]> wrote:

Hello!

Let's say I'm using a declarative pipeline setup like this:

~~
#!groovy

pipeline {

  agent none

  stages {
    stage('Build') {
      parallel {
        stage('Build Linux') {
          stages {
            stage('Build:linux:openjdk-11-hotspot') {
              agent {
                label 'linux'
              }
              ...
            }
            stage('Build:linux:openjdk-15-hotspot') {
              agent {
                label 'linux'
              }
              ...
            }
          }
        }

        stage('Build Windows') {
          stages {
            stage('Build:windows:openjdk-11-hotspot') {
              agent {
                label 'windows'
              }
              ...
            }
            stage('Build:windows:openjdk-15-hotspot') {
              agent {
                label 'windows'
              }
              ...
            }
          }
        }
      }
    }
  }
}
~~

As you can see, I use labels to select agents to run builds on specific
operating systems and with a fixed set of specific JDK versions. This
all works fine. However: With my current hardware setup, the Windows
node is _not_ going to be online 24/7 (the Linux node is always
available).

Is there a way I can adapt the above pipeline to basically say "If
there's a Windows node online, run the stages, otherwise trivially
succeed"? Right now, if no Windows node is available, the pipeline will
hang until either one becomes available or the entire job times out.

--
Mark Raynsford | https://www.io7m.com

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20201231212628.447a21b5%40sunflower.int.arc7.info.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtF93ptAXbZzCXtBTFuVhCz2P62TgNPvFu8H8E%3DKfAXB8Q%40mail.gmail.com.

[Mark Waite-2]

Apologies for the empty message that preceded this one.  Fat fingers on an unfamiliar keyboard.

I've been using the pipeline step 'nodesByLabel' in scripted pipelines to select label based subsets of agents that are currently available.  I'm not sure that the same technique can be used with declarative pipeline.

On Sat, Jan 2, 2021 at 10:30 AM Mark Waite <[hidden email]> wrote:

On Thu, Dec 31, 2020 at 2:27 PM 'Mark Raynsford' via Jenkins Users <[hidden email]> wrote:

Hello!

Let's say I'm using a declarative pipeline setup like this:

~~
#!groovy

pipeline {

  agent none

  stages {
    stage('Build') {
      parallel {
        stage('Build Linux') {
          stages {
            stage('Build:linux:openjdk-11-hotspot') {
              agent {
                label 'linux'
              }
              ...
            }
            stage('Build:linux:openjdk-15-hotspot') {
              agent {
                label 'linux'
              }
              ...
            }
          }
        }

        stage('Build Windows') {
          stages {
            stage('Build:windows:openjdk-11-hotspot') {
              agent {
                label 'windows'
              }
              ...
            }
            stage('Build:windows:openjdk-15-hotspot') {
              agent {
                label 'windows'
              }
              ...
            }
          }
        }
      }
    }
  }
}
~~

As you can see, I use labels to select agents to run builds on specific
operating systems and with a fixed set of specific JDK versions. This
all works fine. However: With my current hardware setup, the Windows
node is _not_ going to be online 24/7 (the Linux node is always
available).

Is there a way I can adapt the above pipeline to basically say "If
there's a Windows node online, run the stages, otherwise trivially
succeed"? Right now, if no Windows node is available, the pipeline will
hang until either one becomes available or the entire job times out.

--
Mark Raynsford | https://www.io7m.com

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20201231212628.447a21b5%40sunflower.int.arc7.info.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAO49JtEdA-UxmdmRsSLKo1eNvHxJ%2BGq%2BSzpucQ%3DtepLFy2kZiw%40mail.gmail.com.

[Stefan Thurnherr]

As mentioned by Mark, nodesByLabel will give you control over what agents are currently available. But you will need a script {} section in an initialization stage and probably set an env variable that you then check in the when {} section of the windows stages. So not purely declarative anymore.

Another idea (in case you are not aware of this yet) would be to set a very short stage-specific timeout for the windows stages only. The windows stages will then show as ABORTED, but since the Linux stages are run in parallel they should still go ahead and run.

Stage timeout documentation: https://www.jenkins.io/doc/book/pipeline/syntax/#stage-options

On Thu, Dec 31, 2020 at 10:26 PM 'Mark Raynsford' via Jenkins Users <[hidden email]> wrote:

Hello!

Let's say I'm using a declarative pipeline setup like this:

~~
#!groovy

pipeline {

  agent none

  stages {
    stage('Build') {
      parallel {
        stage('Build Linux') {
          stages {
            stage('Build:linux:openjdk-11-hotspot') {
              agent {
                label 'linux'
              }
              ...
            }
            stage('Build:linux:openjdk-15-hotspot') {
              agent {
                label 'linux'
              }
              ...
            }
          }
        }

        stage('Build Windows') {
          stages {
            stage('Build:windows:openjdk-11-hotspot') {
              agent {
                label 'windows'
              }
              ...
            }
            stage('Build:windows:openjdk-15-hotspot') {
              agent {
                label 'windows'
              }
              ...
            }
          }
        }
      }
    }
  }
}
~~

As you can see, I use labels to select agents to run builds on specific
operating systems and with a fixed set of specific JDK versions. This
all works fine. However: With my current hardware setup, the Windows
node is _not_ going to be online 24/7 (the Linux node is always
available).

Is there a way I can adapt the above pipeline to basically say "If
there's a Windows node online, run the stages, otherwise trivially
succeed"? Right now, if no Windows node is available, the pipeline will
hang until either one becomes available or the entire job times out.

--
Mark Raynsford | https://www.io7m.com

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20201231212628.447a21b5%40sunflower.int.arc7.info.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CABwQARvWz2xLyzzP83kf9W8sfU_wfkpZnC09EJas5fgsrpA9Bw%40mail.gmail.com.

[Mark Raynsford]

On 2021-01-04T22:33:55 +0100
ST <[hidden email]> wrote:

> As mentioned by Mark, nodesByLabel will give you control over what agents
> are currently available. But you will need a script {} section in an
> initialization stage and probably set an env variable that you then check
> in the when {} section of the windows stages. So not purely declarative
> anymore.
>
> Another idea (in case you are not aware of this yet) would be to set a very
> short *stage-specific timeout* for the windows stages only. The windows
> stages will then show as ABORTED, but since the Linux stages are run in
> parallel they should still go ahead and run.
> Stage timeout documentation:
> https://www.jenkins.io/doc/book/pipeline/syntax/#stage-options

Thanks, both of you!

I think I might go another way: Throw hardware at the problem and
arrange to have a Windows node that's actually on 24/7.

--
Mark Raynsford | https://www.io7m.com

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/20210105213434.6ea0e6fa%40sunflower.int.arc7.info.