Triggering builds from changes in sub-directories using Github Branch Source

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Triggering builds from changes in sub-directories using Github Branch Source

Jonathan Ballet

Hello,

In one of our Git repository, we have a directory hierarchy similar to this:

    dev/
        Jenkinsfile
        folder-1/Jenkinsfile
        folder-2/Jenkinsfile

    test/
        Jenkinsfile
        folder-3/Jenkinsfile
        folder-4/Jenkinsfile

There are other files as well, in each sub-directories.

I'm looking for a way to have 6 builds out of this repository:

* 1 build runs `dev/Jenkinsfile` when any file changes in the `dev/` directory and children
* 1 build runs `dev/folder-1/Jenkinsfile` when any file changes in the `dev/folder-1/` directory
* 1 build runs `dev/folder-2/Jenkinsfile` when any file changes in the `dev/folder-2/` directory
* etc.

Also, I'd like to have these builds scopped by branches, we are using the
Multibranch Pipeline (+ GitHub Branches Source) build type for that.

Is there a known solution that combines all the above? (sub-directories, pipelines, webhooks triggered by Git(Hub) with branches support)

Thanks!

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/12b8fbde-39c8-412e-b00e-6263a0df6a6en%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Triggering builds from changes in sub-directories using Github Branch Source

Gianluca
Hi,
yes ... we do.

We have various Jenkinsfile in sub-directories and different pipelines.

This is how we did it:
- we have one Multibranch project for each Jenkinsfile and in the configuration we specify which Jenkinsfile is used for the jobs
- we have a custom check that we use on the "when" block to avoid that a pipeline is been built when a push only affects other directories

// There is a bug in the changeset Jenkins command: https://issues.jenkins-ci.org/browse/JENKINS-44849
// ---
// It returns true if the list of changed files in the PR matches the grep pattern passed
// NOTE: it will always returns true in case the build is running on a branch and not on a PR
// because we always want to run a full pipelines on master, release and feature branches
def areFilesChanged(grepPattern) {
   
if (!env.CHANGE_TARGET) return true
    sh(
       
// NOTE: the triple-dot are necessary to get the same diff result shown by GitHub PRs
        script: "git diff origin/${env.CHANGE_TARGET}...HEAD --name-only | grep '${grepPattern}'",
       
returnStatus: true
    ) == 0
}

and that's it.

Cheers,
Gianluca.


On Tuesday, 11 August 2020 14:46:40 UTC+1, Jonathan Ballet wrote:

Hello,

In one of our Git repository, we have a directory hierarchy similar to this:

    dev/
        Jenkinsfile
        folder-1/Jenkinsfile
        folder-2/Jenkinsfile

    test/
        Jenkinsfile
        folder-3/Jenkinsfile
        folder-4/Jenkinsfile

There are other files as well, in each sub-directories.

I'm looking for a way to have 6 builds out of this repository:

* 1 build runs `dev/Jenkinsfile` when any file changes in the `dev/` directory and children
* 1 build runs `dev/folder-1/Jenkinsfile` when any file changes in the `dev/folder-1/` directory
* 1 build runs `dev/folder-2/Jenkinsfile` when any file changes in the `dev/folder-2/` directory
* etc.

Also, I'd like to have these builds scopped by branches, we are using the
Multibranch Pipeline (+ GitHub Branches Source) build type for that.

Is there a known solution that combines all the above? (sub-directories, pipelines, webhooks triggered by Git(Hub) with branches support)

Thanks!

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/ed2852e4-d205-4719-a769-9224800a163bo%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Triggering builds from changes in sub-directories using Github Branch Source

Jonathan Ballet
On Tuesday, 11 August 2020 at 15:55:17 UTC+2 Gianluca wrote:
Hi,
yes ... we do.

We have various Jenkinsfile in sub-directories and different pipelines.

This is how we did it:
- we have one Multibranch project for each Jenkinsfile and in the configuration we specify which Jenkinsfile is used for the jobs
- we have a custom check that we use on the "when" block to avoid that a pipeline is been built when a push only affects other directories

// There is a bug in the changeset Jenkins command: https://issues.jenkins-ci.org/browse/JENKINS-44849
// ---
// It returns true if the list of changed files in the PR matches the grep pattern passed
// NOTE: it will always returns true in case the build is running on a branch and not on a PR
// because we always want to run a full pipelines on master, release and feature branches
def areFilesChanged(grepPattern) {
   
if (!env.CHANGE_TARGET) return true
    sh(
       
// NOTE: the triple-dot are necessary to get the same diff result shown by GitHub PRs
        script: "git diff origin/${env.CHANGE_TARGET}...HEAD --name-only | grep '${grepPattern}'",
       
returnStatus: true
    ) == 0
}

and that's it.

I see, is this function called within the Jenkinsfile itself, right?
Is your build starting to run, then you are calling that function and the build is interrupted prematurely if the files modified doesn't match your pattern?


On Tuesday, 11 August 2020 14:46:40 UTC+1, Jonathan Ballet wrote:

Hello,

In one of our Git repository, we have a directory hierarchy similar to this:

    dev/
        Jenkinsfile
        folder-1/Jenkinsfile
        folder-2/Jenkinsfile

    test/
        Jenkinsfile
        folder-3/Jenkinsfile
        folder-4/Jenkinsfile

There are other files as well, in each sub-directories.

I'm looking for a way to have 6 builds out of this repository:

* 1 build runs `dev/Jenkinsfile` when any file changes in the `dev/` directory and children
* 1 build runs `dev/folder-1/Jenkinsfile` when any file changes in the `dev/folder-1/` directory
* 1 build runs `dev/folder-2/Jenkinsfile` when any file changes in the `dev/folder-2/` directory
* etc.

Also, I'd like to have these builds scopped by branches, we are using the
Multibranch Pipeline (+ GitHub Branches Source) build type for that.

Is there a known solution that combines all the above? (sub-directories, pipelines, webhooks triggered by Git(Hub) with branches support)

Thanks!

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/1dbab58e-18cf-4369-96a2-549b4e919c47n%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Triggering builds from changes in sub-directories using Github Branch Source

Gianluca


Il giorno lunedì 17 agosto 2020 alle 16:54:26 UTC+1 [hidden email] ha scritto:
On Tuesday, 11 August 2020 at 15:55:17 UTC+2 Gianluca wrote:
Hi,
yes ... we do.

We have various Jenkinsfile in sub-directories and different pipelines.

This is how we did it:
- we have one Multibranch project for each Jenkinsfile and in the configuration we specify which Jenkinsfile is used for the jobs
- we have a custom check that we use on the "when" block to avoid that a pipeline is been built when a push only affects other directories

// There is a bug in the changeset Jenkins command: https://issues.jenkins-ci.org/browse/JENKINS-44849
// ---
// It returns true if the list of changed files in the PR matches the grep pattern passed
// NOTE: it will always returns true in case the build is running on a branch and not on a PR
// because we always want to run a full pipelines on master, release and feature branches
def areFilesChanged(grepPattern) {
   
if (!env.CHANGE_TARGET) return true
    sh(
       
// NOTE: the triple-dot are necessary to get the same diff result shown by GitHub PRs
        script: "git diff origin/${env.CHANGE_TARGET}...HEAD --name-only | grep '${grepPattern}'",
       
returnStatus: true
    ) == 0
}

and that's it.

I see, is this function called within the Jenkinsfile itself, right?
Yes.
 
Is your build starting to run, then you are calling that function and the build is interrupted prematurely if the files modified doesn't match your pattern?
Yes. We do in two different ways depending on which one fits better:
1) Using the "when" option of the stage:
stage("something") {
  when { expression { areFilesChanged("src/python3") } }
...

2) Using an if condition inside a script block
script {
   if ( areFilesChanged("src/rails") ) {
     ... steps
   }






On Tuesday, 11 August 2020 14:46:40 UTC+1, Jonathan Ballet wrote:

Hello,

In one of our Git repository, we have a directory hierarchy similar to this:

    dev/
        Jenkinsfile
        folder-1/Jenkinsfile
        folder-2/Jenkinsfile

    test/
        Jenkinsfile
        folder-3/Jenkinsfile
        folder-4/Jenkinsfile

There are other files as well, in each sub-directories.

I'm looking for a way to have 6 builds out of this repository:

* 1 build runs `dev/Jenkinsfile` when any file changes in the `dev/` directory and children
* 1 build runs `dev/folder-1/Jenkinsfile` when any file changes in the `dev/folder-1/` directory
* 1 build runs `dev/folder-2/Jenkinsfile` when any file changes in the `dev/folder-2/` directory
* etc.

Also, I'd like to have these builds scopped by branches, we are using the
Multibranch Pipeline (+ GitHub Branches Source) build type for that.

Is there a known solution that combines all the above? (sub-directories, pipelines, webhooks triggered by Git(Hub) with branches support)

Thanks!

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/312bb201-5369-4814-8864-df95a0f05127n%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: Triggering builds from changes in sub-directories using Github Branch Source

Jonathan Ballet


On Mon, 17 Aug 2020, at 17:59, Gianluca wrote:


Il giorno lunedì 17 agosto 2020 alle 16:54:26 UTC+1 [hidden email] ha scritto:
On Tuesday, 11 August 2020 at 15:55:17 UTC+2 Gianluca wrote:
Hi,
yes ... we do.

We have various Jenkinsfile in sub-directories and different pipelines.

This is how we did it:
- we have one Multibranch project for each Jenkinsfile and in the configuration we specify which Jenkinsfile is used for the jobs
- we have a custom check that we use on the "when" block to avoid that a pipeline is been built when a push only affects other directories

// There is a bug in the changeset Jenkins command: https://issues.jenkins-ci.org/browse/JENKINS-44849
// ---
// It returns true if the list of changed files in the PR matches the grep pattern passed
// NOTE: it will always returns true in case the build is running on a branch and not on a PR
// because we always want to run a full pipelines on master, release and feature branches
def areFilesChanged(grepPattern) {
   
if (!env.CHANGE_TARGET) return true
    sh(
       
// NOTE: the triple-dot are necessary to get the same diff result shown by GitHub PRs
        script: "git diff origin/${env.CHANGE_TARGET}...HEAD --name-only | grep '${grepPattern}'",
       
returnStatus: true
    ) == 0
}

and that's it.

I see, is this function called within the Jenkinsfile itself, right?
Yes.
 
Is your build starting to run, then you are calling that function and the build is interrupted prematurely if the files modified doesn't match your pattern?
Yes. We do in two different ways depending on which one fits better:
1) Using the "when" option of the stage:
stage("something") {
  when { expression { areFilesChanged("src/python3") } }
...

2) Using an if condition inside a script block
script {
   if ( areFilesChanged("src/rails") ) {
     ... steps
   }



OK I see, thanks for the tips!

I was hoping to find a pre-build solution though - we have been using this kind of conditional builds in the past and it created lot of spurious "short builds", I was wondering if there was another way to do that.







On Tuesday, 11 August 2020 14:46:40 UTC+1, Jonathan Ballet wrote:

Hello,

In one of our Git repository, we have a directory hierarchy similar to this:

    dev/
        Jenkinsfile
        folder-1/Jenkinsfile
        folder-2/Jenkinsfile

    test/
        Jenkinsfile
        folder-3/Jenkinsfile
        folder-4/Jenkinsfile

There are other files as well, in each sub-directories.

I'm looking for a way to have 6 builds out of this repository:

* 1 build runs `dev/Jenkinsfile` when any file changes in the `dev/` directory and children
* 1 build runs `dev/folder-1/Jenkinsfile` when any file changes in the `dev/folder-1/` directory
* 1 build runs `dev/folder-2/Jenkinsfile` when any file changes in the `dev/folder-2/` directory
* etc.

Also, I'd like to have these builds scopped by branches, we are using the
Multibranch Pipeline (+ GitHub Branches Source) build type for that.

Is there a known solution that combines all the above? (sub-directories, pipelines, webhooks triggered by Git(Hub) with branches support)

Thanks!


--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this group and all its topics, send an email to [hidden email].

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/904639eb-74da-4fab-9ec5-597f676b46b4%40beta.fastmail.com.