Recommending for cleaning up old builds when multibranch pipeline is in use?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Recommending for cleaning up old builds when multibranch pipeline is in use?

Mark Stosberg
For cleaning up old builds, the "Discard Old Builds" plugin is recommended:

https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin

However, the instructions for reference a "post build" step which doesn't exist in the Configure interface for Multibranch Pipeline projects. 

What's the recommended way to clean-up builds from multi-branch pipeline projects? Ideally I'd like different pruning policies for different branches. 

Thanks,

     Mark

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Recommending for cleaning up old builds when multibranch pipeline is in use?

Baptiste MATHUS
I don't think there's something per branch, but you probably want to either search JIRA and/or file a new feature request about it.

You shouldn't need a special plugin, this is a standard feature. Here's the way to do it with Declarative Pipeline: 

pipeline {
    options {
        buildDiscarder(logRotator(numToKeepStr: '20'))
    }
...
}


HTH

2017-02-28 20:24 GMT+01:00 Mark Stosberg <[hidden email]>:
For cleaning up old builds, the "Discard Old Builds" plugin is recommended:


However, the instructions for reference a "post build" step which doesn't exist in the Configure interface for Multibranch Pipeline projects. 

What's the recommended way to clean-up builds from multi-branch pipeline projects? Ideally I'd like different pruning policies for different branches. 

Thanks,

     Mark

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS6mM4%2B_hVif-X5ED6WA8G7TdmLfpU4h3kH0ff77zQoFJA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Recommending for cleaning up old builds when multibranch pipeline is in use?

Mark Stosberg
Thanks for reply, and for providing both the Pipeline syntax formats. This is the first time I noticed that there are two distinct Pipeline syntaxes.

I'm trying out the Scripted Pipeline syntax now.  

I tried to find documentation on the difference between 'numToKeepStr' and 'artifactNumToKeepStr' but couldn't. I set them both the same number in hopes that will have the desired effect of saving a particular number of builds. 

      Mark

On Wednesday, March 1, 2017 at 4:08:53 AM UTC-5, Baptiste Mathus wrote:
I don't think there's something per branch, but you probably want to either search JIRA and/or file a new feature request about it.

You shouldn't need a special plugin, this is a standard feature. Here's the way to do it with Declarative Pipeline: 

pipeline {
    options {
        buildDiscarder(logRotator(numToKeepStr: '20'))
    }
...
}

Or with scripted Pipeline: <a href="https://github.com/jenkinsci/jenkins/blob/master/Jenkinsfile#L17-L19" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2Fmaster%2FJenkinsfile%23L17-L19\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0cMkaTLm-Jbs6LSsCGmS5h6xRxg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2Fmaster%2FJenkinsfile%23L17-L19\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0cMkaTLm-Jbs6LSsCGmS5h6xRxg&#39;;return true;">https://github.com/jenkinsci/jenkins/blob/master/Jenkinsfile#L17-L19

HTH

2017-02-28 20:24 GMT+01:00 Mark Stosberg <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="wHnKQjEKAgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">ma...@...>:
For cleaning up old builds, the "Discard Old Builds" plugin is recommended:

<a href="https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins-ci.org%2Fdisplay%2FJENKINS%2FDiscard%2BOld%2BBuild%2Bplugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGimvt3IGWHcv4Jg6iYZTaZhVVrqg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins-ci.org%2Fdisplay%2FJENKINS%2FDiscard%2BOld%2BBuild%2Bplugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGimvt3IGWHcv4Jg6iYZTaZhVVrqg&#39;;return true;">https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin

However, the instructions for reference a "post build" step which doesn't exist in the Configure interface for Multibranch Pipeline projects. 

What's the recommended way to clean-up builds from multi-branch pipeline projects? Ideally I'd like different pruning policies for different branches. 

Thanks,

     Mark

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="wHnKQjEKAgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/ec6ee4ea-0deb-45f4-ba64-b5ad4f33ff8a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Recommending for cleaning up old builds when multibranch pipeline is in use?

Baptiste MATHUS
<a href="http://javadoc.jenkins.io/hudson/tasks/LogRotator.html#LogRotator(java.lang.String, java.lang.String, java.lang.String, java.lang.String)">http://javadoc.jenkins.io/hudson/tasks/LogRotator.html#LogRotator(java.lang.String, java.lang.String, java.lang.String, java.lang.String) could be helpful.



2017-03-01 20:30 GMT+01:00 Mark Stosberg <[hidden email]>:
Thanks for reply, and for providing both the Pipeline syntax formats. This is the first time I noticed that there are two distinct Pipeline syntaxes.

I'm trying out the Scripted Pipeline syntax now.  

I tried to find documentation on the difference between 'numToKeepStr' and 'artifactNumToKeepStr' but couldn't. I set them both the same number in hopes that will have the desired effect of saving a particular number of builds. 

      Mark

On Wednesday, March 1, 2017 at 4:08:53 AM UTC-5, Baptiste Mathus wrote:
I don't think there's something per branch, but you probably want to either search JIRA and/or file a new feature request about it.

You shouldn't need a special plugin, this is a standard feature. Here's the way to do it with Declarative Pipeline: 

pipeline {
    options {
        buildDiscarder(logRotator(numToKeepStr: '20'))
    }
...
}


HTH

2017-02-28 20:24 GMT+01:00 Mark Stosberg <[hidden email]>:
For cleaning up old builds, the "Discard Old Builds" plugin is recommended:


However, the instructions for reference a "post build" step which doesn't exist in the Configure interface for Multibranch Pipeline projects. 

What's the recommended way to clean-up builds from multi-branch pipeline projects? Ideally I'd like different pruning policies for different branches. 

Thanks,

     Mark

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/ec6ee4ea-0deb-45f4-ba64-b5ad4f33ff8a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CANWgJS75FWLkkbet2FvMKuQkUpdPat%2BbkP19RjXVPvyYim815g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Recommending for cleaning up old builds when multibranch pipeline is in use?

Mark Stosberg
In reply to this post by Mark Stosberg
On Wednesday, March 1, 2017 at 4:08:53 AM UTC-5, Baptiste Mathus wrote:
I don't think there's something per branch, but you probably want to either search JIRA and/or file a new feature request about it.

You shouldn't need a special plugin, this is a standard feature. Here's the way to do it with Declarative Pipeline: 

pipeline {
    options {
        buildDiscarder(logRotator(numToKeepStr: '20'))
    }
...
}

Or with scripted Pipeline: <a href="https://github.com/jenkinsci/jenkins/blob/master/Jenkinsfile#L17-L19" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2Fmaster%2FJenkinsfile%23L17-L19\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0cMkaTLm-Jbs6LSsCGmS5h6xRxg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2Fmaster%2FJenkinsfile%23L17-L19\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0cMkaTLm-Jbs6LSsCGmS5h6xRxg&#39;;return true;">https://github.com/jenkinsci/jenkins/blob/master/Jenkinsfile#L17-L19


I'm stil not seeing any signs that the buildDiscarder plugin is working for me. At the top of my Jenkinsfile, I have this:

properties([[
 $class: 'jenkins.model.BuildDiscarderProperty',
  strategy: [$class: 'LogRotator', numToKeepStr: '50', artifactNumToKeepStr: '50']
]])

That's just above my  "node {}" block. 

From reading related source code at:
https://github.com/jenkinsci/jenkins/blob/08def67a18eee51de9f3f99bc2a792fee1c160e0/core/src/main/java/hudson/tasks/LogRotator.java

I can see that this function should generate some logging, but I can't find the log entries indicating this is running in either the web-based Jenkins log page, or under /var/log/jenkins/*

The effective-but-imperfect workaround I've found is to just manually delete only builds:

    find "/var/lib/jenkins/jobs/My Pipeline/branches/" -mindepth 3 -maxdepth 3 -type d -ctime +15 | xargs -d '\n' rm -rf

That doesn't have the safety nets of preserving any builds that are last-successful, promoted, etc. 

Some questions:

  * Should the logging I see in the source code *always* be happening or there something I need to do activate that logging?
  * Should properties() call work outside a node block?
  * Are there any serious drawbacks to just manually deleting old build myself? This built-in log rotation has proven especially time-consuming for what seems like it should be a simple feature to setup. 

Thanks,

      Mark

2017-02-28 20:24 GMT+01:00 Mark Stosberg <[hidden email]>:
For cleaning up old builds, the "Discard Old Builds" plugin is recommended:

<a href="https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins-ci.org%2Fdisplay%2FJENKINS%2FDiscard%2BOld%2BBuild%2Bplugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGimvt3IGWHcv4Jg6iYZTaZhVVrqg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins-ci.org%2Fdisplay%2FJENKINS%2FDiscard%2BOld%2BBuild%2Bplugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGimvt3IGWHcv4Jg6iYZTaZhVVrqg&#39;;return true;">https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin

However, the instructions for reference a "post build" step which doesn't exist in the Configure interface for Multibranch Pipeline projects. 

What's the recommended way to clean-up builds from multi-branch pipeline projects? Ideally I'd like different pruning policies for different branches. 

Thanks,

     Mark

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/67e52c4d-8131-4ef7-8446-8cee1f31dce3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Recommending for cleaning up old builds when multibranch pipeline is in use?

Kevin Brotcke
I also couldn't get the buildDiscarder working, nothing seems to be happening. In our case we create a lot of long lasting branches which filled up our Jenins server disk space pretty quickly. I wrote a quick Groovy script to delete old builds across all branches but I'd prefer a more integrated solution:

https://gist.github.com/kevin-brotcke/8f65f07e7314ae811df242abc70b2ef2


On Friday, May 5, 2017 at 1:06:04 PM UTC-7, Mark Stosberg wrote:
On Wednesday, March 1, 2017 at 4:08:53 AM UTC-5, Baptiste Mathus wrote:
I don't think there's something per branch, but you probably want to either search JIRA and/or file a new feature request about it.

You shouldn't need a special plugin, this is a standard feature. Here's the way to do it with Declarative Pipeline: 

pipeline {
    options {
        buildDiscarder(logRotator(numToKeepStr: '20'))
    }
...
}

Or with scripted Pipeline: <a href="https://github.com/jenkinsci/jenkins/blob/master/Jenkinsfile#L17-L19" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2Fmaster%2FJenkinsfile%23L17-L19\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0cMkaTLm-Jbs6LSsCGmS5h6xRxg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2Fmaster%2FJenkinsfile%23L17-L19\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0cMkaTLm-Jbs6LSsCGmS5h6xRxg&#39;;return true;">https://github.com/jenkinsci/jenkins/blob/master/Jenkinsfile#L17-L19


I'm stil not seeing any signs that the buildDiscarder plugin is working for me. At the top of my Jenkinsfile, I have this:

properties([[
 $class: 'jenkins.model.BuildDiscarderProperty',
  strategy: [$class: 'LogRotator', numToKeepStr: '50', artifactNumToKeepStr: '50']
]])

That's just above my  "node {}" block. 

From reading related source code at:
<a href="https://github.com/jenkinsci/jenkins/blob/08def67a18eee51de9f3f99bc2a792fee1c160e0/core/src/main/java/hudson/tasks/LogRotator.java" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2F08def67a18eee51de9f3f99bc2a792fee1c160e0%2Fcore%2Fsrc%2Fmain%2Fjava%2Fhudson%2Ftasks%2FLogRotator.java\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHecF-afBg4pZAaCF7VwECvcbda1Q&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2F08def67a18eee51de9f3f99bc2a792fee1c160e0%2Fcore%2Fsrc%2Fmain%2Fjava%2Fhudson%2Ftasks%2FLogRotator.java\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHecF-afBg4pZAaCF7VwECvcbda1Q&#39;;return true;">https://github.com/jenkinsci/jenkins/blob/08def67a18eee51de9f3f99bc2a792fee1c160e0/core/src/main/java/hudson/tasks/LogRotator.java

I can see that this function should generate some logging, but I can't find the log entries indicating this is running in either the web-based Jenkins log page, or under /var/log/jenkins/*

The effective-but-imperfect workaround I've found is to just manually delete only builds:

    find "/var/lib/jenkins/jobs/My Pipeline/branches/" -mindepth 3 -maxdepth 3 -type d -ctime +15 | xargs -d '\n' rm -rf

That doesn't have the safety nets of preserving any builds that are last-successful, promoted, etc. 

Some questions:

  * Should the logging I see in the source code *always* be happening or there something I need to do activate that logging?
  * Should properties() call work outside a node block?
  * Are there any serious drawbacks to just manually deleting old build myself? This built-in log rotation has proven especially time-consuming for what seems like it should be a simple feature to setup. 

Thanks,

      Mark

2017-02-28 20:24 GMT+01:00 Mark Stosberg <[hidden email]>:
For cleaning up old builds, the "Discard Old Builds" plugin is recommended:

<a href="https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins-ci.org%2Fdisplay%2FJENKINS%2FDiscard%2BOld%2BBuild%2Bplugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGimvt3IGWHcv4Jg6iYZTaZhVVrqg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins-ci.org%2Fdisplay%2FJENKINS%2FDiscard%2BOld%2BBuild%2Bplugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGimvt3IGWHcv4Jg6iYZTaZhVVrqg&#39;;return true;">https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin

However, the instructions for reference a "post build" step which doesn't exist in the Configure interface for Multibranch Pipeline projects. 

What's the recommended way to clean-up builds from multi-branch pipeline projects? Ideally I'd like different pruning policies for different branches. 

Thanks,

     Mark

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b6dfc6d2-0d82-4bc3-a356-a710cfc01a6d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Recommending for cleaning up old builds when multibranch pipeline is in use?

Mark Waite-2


On Tuesday, November 28, 2017 at 4:50:48 PM UTC-7, Kevin Brotcke wrote:
I also couldn't get the buildDiscarder working, nothing seems to be happening. In our case we create a lot of long lasting branches which filled up our Jenins server disk space pretty quickly. I wrote a quick Groovy script to delete old builds across all branches but I'd prefer a more integrated solution:

<a href="https://gist.github.com/kevin-brotcke/8f65f07e7314ae811df242abc70b2ef2" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgist.github.com%2Fkevin-brotcke%2F8f65f07e7314ae811df242abc70b2ef2\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFGUhbm0XbbO8JN0qRgFHxO_CJGQg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgist.github.com%2Fkevin-brotcke%2F8f65f07e7314ae811df242abc70b2ef2\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFGUhbm0XbbO8JN0qRgFHxO_CJGQg&#39;;return true;">https://gist.github.com/kevin-brotcke/8f65f07e7314ae811df242abc70b2ef2


From scripted pipeline, I use properties([[$class: 'BuildDiscarderProperty', strategy: [$class: 'LogRotator', numToKeepStr: '7']]]) and it works great.  I haven't tried it with declarative pipeline, but would be surprised if it did not work just as well there.

Mark Waite

 

On Friday, May 5, 2017 at 1:06:04 PM UTC-7, Mark Stosberg wrote:
On Wednesday, March 1, 2017 at 4:08:53 AM UTC-5, Baptiste Mathus wrote:
I don't think there's something per branch, but you probably want to either search JIRA and/or file a new feature request about it.

You shouldn't need a special plugin, this is a standard feature. Here's the way to do it with Declarative Pipeline: 

pipeline {
    options {
        buildDiscarder(logRotator(numToKeepStr: '20'))
    }
...
}

Or with scripted Pipeline: <a href="https://github.com/jenkinsci/jenkins/blob/master/Jenkinsfile#L17-L19" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2Fmaster%2FJenkinsfile%23L17-L19\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0cMkaTLm-Jbs6LSsCGmS5h6xRxg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2Fmaster%2FJenkinsfile%23L17-L19\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNG0cMkaTLm-Jbs6LSsCGmS5h6xRxg&#39;;return true;">https://github.com/jenkinsci/jenkins/blob/master/Jenkinsfile#L17-L19


I'm stil not seeing any signs that the buildDiscarder plugin is working for me. At the top of my Jenkinsfile, I have this:

properties([[
 $class: 'jenkins.model.BuildDiscarderProperty',
  strategy: [$class: 'LogRotator', numToKeepStr: '50', artifactNumToKeepStr: '50']
]])

That's just above my  "node {}" block. 

From reading related source code at:
<a href="https://github.com/jenkinsci/jenkins/blob/08def67a18eee51de9f3f99bc2a792fee1c160e0/core/src/main/java/hudson/tasks/LogRotator.java" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2F08def67a18eee51de9f3f99bc2a792fee1c160e0%2Fcore%2Fsrc%2Fmain%2Fjava%2Fhudson%2Ftasks%2FLogRotator.java\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHecF-afBg4pZAaCF7VwECvcbda1Q&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fjenkins%2Fblob%2F08def67a18eee51de9f3f99bc2a792fee1c160e0%2Fcore%2Fsrc%2Fmain%2Fjava%2Fhudson%2Ftasks%2FLogRotator.java\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHecF-afBg4pZAaCF7VwECvcbda1Q&#39;;return true;">https://github.com/jenkinsci/jenkins/blob/08def67a18eee51de9f3f99bc2a792fee1c160e0/core/src/main/java/hudson/tasks/LogRotator.java

I can see that this function should generate some logging, but I can't find the log entries indicating this is running in either the web-based Jenkins log page, or under /var/log/jenkins/*

The effective-but-imperfect workaround I've found is to just manually delete only builds:

    find "/var/lib/jenkins/jobs/My Pipeline/branches/" -mindepth 3 -maxdepth 3 -type d -ctime +15 | xargs -d '\n' rm -rf

That doesn't have the safety nets of preserving any builds that are last-successful, promoted, etc. 

Some questions:

  * Should the logging I see in the source code *always* be happening or there something I need to do activate that logging?
  * Should properties() call work outside a node block?
  * Are there any serious drawbacks to just manually deleting old build myself? This built-in log rotation has proven especially time-consuming for what seems like it should be a simple feature to setup. 

Thanks,

      Mark

2017-02-28 20:24 GMT+01:00 Mark Stosberg <[hidden email]>:
For cleaning up old builds, the "Discard Old Builds" plugin is recommended:

<a href="https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins-ci.org%2Fdisplay%2FJENKINS%2FDiscard%2BOld%2BBuild%2Bplugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGimvt3IGWHcv4Jg6iYZTaZhVVrqg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwiki.jenkins-ci.org%2Fdisplay%2FJENKINS%2FDiscard%2BOld%2BBuild%2Bplugin\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGimvt3IGWHcv4Jg6iYZTaZhVVrqg&#39;;return true;">https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin

However, the instructions for reference a "post build" step which doesn't exist in the Configure interface for Multibranch Pipeline projects. 

What's the recommended way to clean-up builds from multi-branch pipeline projects? Ideally I'd like different pruning policies for different branches. 

Thanks,

     Mark

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/265b88d6-3c72-4c0f-8c59-de35dabcafd5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Recommending for cleaning up old builds when multibranch pipeline is in use?

Kevin Brotcke
I can confirm deleting old builds PER branch is working for declarative syntax by using the option:
options {
  buildDiscarder(logRotator(numToKeepStr:'10'))
}

But this doesn't really solve my problem. I want to keep max N builds across ALL branches in a multibranch pipeline to maintain a more predictable disk usage rate. That's what my Groovy script accomplishes.


On Tue, Nov 28, 2017 at 4:53 PM, Mark Waite <[hidden email]> wrote:


On Tuesday, November 28, 2017 at 4:50:48 PM UTC-7, Kevin Brotcke wrote:
I also couldn't get the buildDiscarder working, nothing seems to be happening. In our case we create a lot of long lasting branches which filled up our Jenins server disk space pretty quickly. I wrote a quick Groovy script to delete old builds across all branches but I'd prefer a more integrated solution:



From scripted pipeline, I use properties([[$class: 'BuildDiscarderProperty', strategy: [$class: 'LogRotator', numToKeepStr: '7']]]) and it works great.  I haven't tried it with declarative pipeline, but would be surprised if it did not work just as well there.

Mark Waite

 

On Friday, May 5, 2017 at 1:06:04 PM UTC-7, Mark Stosberg wrote:
On Wednesday, March 1, 2017 at 4:08:53 AM UTC-5, Baptiste Mathus wrote:
I don't think there's something per branch, but you probably want to either search JIRA and/or file a new feature request about it.

You shouldn't need a special plugin, this is a standard feature. Here's the way to do it with Declarative Pipeline: 

pipeline {
    options {
        buildDiscarder(logRotator(numToKeepStr: '20'))
    }
...
}



I'm stil not seeing any signs that the buildDiscarder plugin is working for me. At the top of my Jenkinsfile, I have this:

properties([[
 $class: 'jenkins.model.BuildDiscarderProperty',
  strategy: [$class: 'LogRotator', numToKeepStr: '50', artifactNumToKeepStr: '50']
]])

That's just above my  "node {}" block. 

From reading related source code at:

I can see that this function should generate some logging, but I can't find the log entries indicating this is running in either the web-based Jenkins log page, or under /var/log/jenkins/*

The effective-but-imperfect workaround I've found is to just manually delete only builds:

    find "/var/lib/jenkins/jobs/My Pipeline/branches/" -mindepth 3 -maxdepth 3 -type d -ctime +15 | xargs -d '\n' rm -rf

That doesn't have the safety nets of preserving any builds that are last-successful, promoted, etc. 

Some questions:

  * Should the logging I see in the source code *always* be happening or there something I need to do activate that logging?
  * Should properties() call work outside a node block?
  * Are there any serious drawbacks to just manually deleting old build myself? This built-in log rotation has proven especially time-consuming for what seems like it should be a simple feature to setup. 

Thanks,

      Mark

2017-02-28 20:24 GMT+01:00 Mark Stosberg <[hidden email]>:
For cleaning up old builds, the "Discard Old Builds" plugin is recommended:


However, the instructions for reference a "post build" step which doesn't exist in the Configure interface for Multibranch Pipeline projects. 

What's the recommended way to clean-up builds from multi-branch pipeline projects? Ideally I'd like different pruning policies for different branches. 

Thanks,

     Mark

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1d9e0343-bbce-474f-a4fe-7623a4f39aaf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/_06kZ20xpEE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/265b88d6-3c72-4c0f-8c59-de35dabcafd5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAKge%3Dw-VgEYsReQ9LpLYnFEO8YvZ0B3xyGRmg%2BrRev4%2BysgXsQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.