Releasing with declarative pipeline

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Releasing with declarative pipeline

Ari Maniatis
I'm starting to experiment with Jenkinsfile configuration and its mostly coming together. Probably the hardest thing is sorting out which instructions are for the old script syntax and which for the new declarative syntax. Probably a few years from now stackoverflow will have more correct answers than wrong ones.

Anyhow, I'm stuck on understanding how we are supposed to perform a release. Today we use the release plugin so there is a 'release' button directly in the UI which performs different (additional) gradle steps after the build is successful. In addition, there is a field to type in a version number for this release.

All that works well, so something similar would be great. I'm also open to other ideas like committing a tag in svn, but I can't see how that would work either since I'm only tracking one branch in jenkins. Checking out every branch would be horrendous.

Thanks for any pointers.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0abf55b1-a2b5-41d4-8eb7-b974120f66f6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Ari Maniatis
OK, I've made a small amount of progress. I've learned that:

1. The release plugin doesn't work with pipeline
2. The promoted build plugin doesn't work with pipeline
3. Some users are using the "input" command to cause the build to hang for a few days, and then be able to make a release. This mostly sucks.
4. No one else seems to have solved this problem either.

Now I could make a separate pipeline just for releases, but that seems clunky. What I'm trying now is a parameterised build to allow me to tick a checkbox for "release" and trigger optional stages within the pipeline.

On Thursday, 28 December 2017 17:30:08 UTC+11, Ari Maniatis wrote:
I'm starting to experiment with Jenkinsfile configuration and its mostly coming together. Probably the hardest thing is sorting out which instructions are for the old script syntax and which for the new declarative syntax. Probably a few years from now stackoverflow will have more correct answers than wrong ones.

Anyhow, I'm stuck on understanding how we are supposed to perform a release. Today we use the release plugin so there is a 'release' button directly in the UI which performs different (additional) gradle steps after the build is successful. In addition, there is a field to type in a version number for this release.

All that works well, so something similar would be great. I'm also open to other ideas like committing a tag in svn, but I can't see how that would work either since I'm only tracking one branch in jenkins. Checking out every branch would be horrendous.

Thanks for any pointers.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/191b31af-380f-4bb9-ad42-b96554f8104d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Victor Martinez
Hi Ari,

AFAIK, the release plugin is supported with pipelines: https://issues.jenkins-ci.org/browse/JENKINS-40765 (release 2.7+). Regarding the promoted build plugin, I'm afraid it's still an open topic: https://issues.jenkins-ci.org/browse/JENKINS-36089

Input command is the one I used in the past, you need to be sure to run the input step outside of the node section to avoid to allocate an executor to that particular step (https://stackoverflow.com/questions/37831386/jenkins-pipeline-input-step-blocks-executor).

Cheers

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/7d4c0139-d608-41dd-b4dd-3cb0a8268ccc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Ari Maniatis
Thanks Victor,

Do you have any pointers or examples of the docs for declaration configuration of the release plugin?

Ari

On Thursday, 4 January 2018 01:23:34 UTC+11, Victor Martinez wrote:
Hi Ari,

AFAIK, the release plugin is supported with pipelines: <a href="https://issues.jenkins-ci.org/browse/JENKINS-40765" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fissues.jenkins-ci.org%2Fbrowse%2FJENKINS-40765\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHvg-4lyUm_zyBHm1AxqlDNIMk6WA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fissues.jenkins-ci.org%2Fbrowse%2FJENKINS-40765\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHvg-4lyUm_zyBHm1AxqlDNIMk6WA&#39;;return true;">https://issues.jenkins-ci.org/browse/JENKINS-40765 (release 2.7+). Regarding the promoted build plugin, I'm afraid it's still an open topic: <a href="https://issues.jenkins-ci.org/browse/JENKINS-36089" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fissues.jenkins-ci.org%2Fbrowse%2FJENKINS-36089\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEvtwWxpK_pcIUMTbI5d5whZ7wgwQ&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fissues.jenkins-ci.org%2Fbrowse%2FJENKINS-36089\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEvtwWxpK_pcIUMTbI5d5whZ7wgwQ&#39;;return true;">https://issues.jenkins-ci.org/browse/JENKINS-36089

Input command is the one I used in the past, you need to be sure to run the input step outside of the node section to avoid to allocate an executor to that particular step (<a href="https://stackoverflow.com/questions/37831386/jenkins-pipeline-input-step-blocks-executor" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fstackoverflow.com%2Fquestions%2F37831386%2Fjenkins-pipeline-input-step-blocks-executor\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEqJEPTHsIS0lnR99wL_mcTL-Ndyg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fstackoverflow.com%2Fquestions%2F37831386%2Fjenkins-pipeline-input-step-blocks-executor\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEqJEPTHsIS0lnR99wL_mcTL-Ndyg&#39;;return true;">https://stackoverflow.com/questions/37831386/jenkins-pipeline-input-step-blocks-executor).

Cheers

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b0bcfc46-6fde-41e6-be48-a2f637655521%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Victor Martinez
I've not used that plugin in the past, but there are a couple of examples in the below test:
- https://github.com/jenkinsci/release-plugin/blob/master/src/test/java/hudson/plugins/release/pipeline/ReleaseStepTest.java


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/8bda5313-74fa-48b4-954b-f1693f9e9983%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Ari Maniatis
I’m not really sure how to read that. I tried

pipeline {
  release { }
}

But got an error. Maybe it needs to contain steps? Or stages?

Ari

On 4 January 2018 at 1:54:02am, Victor Martinez ([hidden email]) wrote:

I've not used that plugin in the past, but there are a couple of examples in the below test:


--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/4pUPjGuum44/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/8bda5313-74fa-48b4-954b-f1693f9e9983%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAJHfvuCG0%2BG2TyF2YfzLY1ycWKyaJgf5A4f2Seuo%2BX%3DVZnDrbA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Victor Martinez
I just ran a raw example and it basically triggers another job with some arguments.

stage("Build") {
  node {
      sh 'echo mvn'
  }
}
stage("Release") {
    release 'archive'
}


Where archive is just a basic job with some basic hello world in the release section

It looks like a kind of similar step like the build DSL one 



--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/311879d1-265f-41be-92ef-9e7623903cf6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Ari Maniatis
Ah OK. That doesn't seem terribly useful since the other job then cannot be a pipeline because it doesn't support a release stage.

What's really needed here is simpler: the ability to put a button in the Jenkins UI with an arbitrary name (like "release") and which triggers the build while injecting some parameter. Maybe parameterised builds are the closest solution.

Ari


On Thursday, 4 January 2018 03:19:12 UTC+11, Victor Martinez wrote:
I just ran a raw example and it basically triggers another job with some arguments.

stage("Build") {
  node {
      sh 'echo mvn'
  }
}
stage("Release") {
    release 'archive'
}


Where archive is just a basic job with some basic hello world in the release section

It looks like a kind of similar step like the<a href="https://jenkins.io/doc/pipeline/steps/pipeline-build-step/" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fjenkins.io%2Fdoc%2Fpipeline%2Fsteps%2Fpipeline-build-step%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEWT0X8m7x3_MTNPkNTTn-4lGctHw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fjenkins.io%2Fdoc%2Fpipeline%2Fsteps%2Fpipeline-build-step%2F\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNEWT0X8m7x3_MTNPkNTTn-4lGctHw&#39;;return true;"> build DSL one 



--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/063acf54-f8a9-4bf1-a878-b72dccea4182%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Steven Foster
In reply to this post by Ari Maniatis
I have been using the multibranch project's tag build functionality to achieve this. An environment variable is provided when running a tag build, so the pipeline can have a stage which checks for the presence of that variable.
Using multibranch to build a single branch seems a little counter intuitive, but the branch api provides convenience in other areas and allows other branches to be added later with ease.

The release process becomes something like:
Push a tag -> Multibranch project detects tag and creates a job for that tag -> Manually run that tag job to perform a release (somewhat like a button on a build but a little more clear and structured imo)
 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/4e7bcd25-4c80-4972-aa5b-3a34132340d0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Steven Foster
In reply to this post by Ari Maniatis
I am using Git though, I'm not sure about the current availability of this functionality for SVN

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/4a0b2796-34c3-477a-8c8c-5324b9fdc77b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Ari Maniatis
In reply to this post by Steven Foster
I think that's a great idea. The main downsides I see are:

* the multibranch plugin (which I use already) is pretty flaky. I find it will sometimes reset all the branches to be enabled even though I try to disable old branches to get them out of the view
* if you create a lot of branches you end up with a huge number of workspaces and disk space which never goes away. In svn the solution to that is to create an 'archived' folder and move old branches/tags there. You can't do that in git though since it is more rigid in how it handles tags/branches.
* The UI gets cluttered in Jenkins

But I love the idea of a tag driving the release process and the version naming is embedded in the tag name itself.

On Thursday, 4 January 2018 16:11:14 UTC+11, Steven Foster wrote:
I have been using the multibranch project's tag build functionality to achieve this. An environment variable is provided when running a tag build, so the pipeline can have a stage which checks for the presence of that variable.
Using multibranch to build a single branch seems a little counter intuitive, but the branch api provides convenience in other areas and allows other branches to be added later with ease.

The release process becomes something like:
Push a tag -> Multibranch project detects tag and creates a job for that tag -> Manually run that tag job to perform a release (somewhat like a button on a build but a little more clear and structured imo)
 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/13b94c91-3864-4a1e-acd8-f450bfb87078%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

stephenconnolly

On Thu 4 Jan 2018 at 06:57, Ari Maniatis <[hidden email]> wrote:
I think that's a great idea. The main downsides I see are:

* the multibranch plugin (which I use already) is pretty flaky. I find it will sometimes reset all the branches to be enabled even though I try to disable old branches to get them out of the view

You shouldn’t be manually disabling branches. The disabled state should be controlled by the multibranch folder.

Or could you explain what you are doing?

(The enabled state should be updated to its “correct” value during a full scan)

There should be no flakey (but there may be a bug whereby you can disable the branch by the UI... which we should fix)


* if you create a lot of branches you end up with a huge number of workspaces and disk space which never goes away. In svn the solution to that is to create an 'archived' folder and move old branches/tags there. You can't do that in git though since it is more rigid in how it handles tags/branches.

Well one thing I do is have my pipeline empty the workspace at the end. This does mean a slower checkout, but I branch a lot, so I get a slower checkout anyway on a new branch.


* The UI gets cluttered in Jenkins

But I love the idea of a tag driving the release process and the version naming is embedded in the tag name itself.

Tags should be on a separate tab, so the UI shouldn’t be so bad (granted BlueOcean doesn’t use the SCMCategory information yet and has hard-coded its tabs - against my advice - so yes the BO UI would be cluttered)

You can write a SCMFilter that only discovers “recent” tags - would work best for annotated tags - so that would limit the clutter to only those tags created (or worst case with last commit) say in the last week (or whatever you configure)



On Thursday, 4 January 2018 16:11:14 UTC+11, Steven Foster wrote:
I have been using the multibranch project's tag build functionality to achieve this. An environment variable is provided when running a tag build, so the pipeline can have a stage which checks for the presence of that variable.
Using multibranch to build a single branch seems a little counter intuitive, but the branch api provides convenience in other areas and allows other branches to be added later with ease.

The release process becomes something like:
Push a tag -> Multibranch project detects tag and creates a job for that tag -> Manually run that tag job to perform a release (somewhat like a button on a build but a little more clear and structured imo)
 

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/13b94c91-3864-4a1e-acd8-f450bfb87078%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Sent from my phone

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CA%2BnPnMxYg%3D3ENSuTHTenARv%3D3JkZyQ1R3O8vw67Vd0HGmHjuZw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Releasing with declarative pipeline

Ari Maniatis
On 4 January 2018 at 7:19:53pm, Stephen Connolly ([hidden email]) wrote:
* the multibranch plugin (which I use already) is pretty flaky. I find it will sometimes reset all the branches to be enabled even though I try to disable old branches to get them out of the view

You shouldn’t be manually disabling branches. The disabled state should be controlled by the multibranch folder.

Or could you explain what you are doing?


I’m using the old deprecated multibranch plugin (not the pipeline version) and as we finish with each branch I press the “disable project” button for that job. Which works until some upgrade of the plugin enables them all again. I don’t know how to control the disabled state in any other way. But perhaps that’s a feature of the pipeline version of this plugin.


(The enabled state should be updated to its “correct” value during a full scan)

There should be no flakey (but there may be a bug whereby you can disable the branch by the UI... which we should fix)

I’ve always been able to press ‘disable project’ per branch and thought that was supposed to work like that.


* if you create a lot of branches you end up with a huge number of workspaces and disk space which never goes away. In svn the solution to that is to create an 'archived' folder and move old branches/tags there. You can't do that in git though since it is more rigid in how it handles tags/branches.

Well one thing I do is have my pipeline empty the workspace at the end. This does mean a slower checkout, but I branch a lot, so I get a slower checkout anyway on a new branch.


* The UI gets cluttered in Jenkins

But I love the idea of a tag driving the release process and the version naming is embedded in the tag name itself.

Tags should be on a separate tab, so the UI shouldn’t be so bad (granted BlueOcean doesn’t use the SCMCategory information yet and has hard-coded its tabs - against my advice - so yes the BO UI would be cluttered)


I have to say that I find Blue Ocean very hard to understand. Fixed width, non-responsive CSS.  Headings like branches and pull requests when you have none. Very little clarity about what things are links and what aren’t. Projects listed in random order. No sense of where in the page hierarchy you are.


You can write a SCMFilter that only discovers “recent” tags - would work best for annotated tags - so that would limit the clutter to only those tags created (or worst case with last commit) say in the last week (or whatever you configure)


That’s really interesting.


Probably the biggest deal breaker of the tag-to-release approach for me is that on one project I’ve got a rather unusual structure with multiple projects inside one SCM root. So I have 10 jenkinsfiles in different folders for each project I want to build separately. It works great for what we want, but it means that I can’t easily tag each project separately.

We still name our releases with different versions per project and we don’t want to release and tag them all at once. So either we restructure our code completely, or we have some trick with naming tags like projectA-10.2.3 to identify which one should be released.


Thanks for your ideas. I think Jenkins could do more to make releasing a really first class concept in the UI and pipeline. But there are still pieces missing (eg. being able to set a build to keep forever from the pipeline).

Ari


--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAJHfvuD_Gm%3D2cpoH1Uu0%3D2-O2JFQauGiGzctTSvmuGcJBFkWLw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.