Configuration that works for PRs and master

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Configuration that works for PRs and master

Hiesgen, Raphael
Hello fellow Jenkins users,

I'll start with my goal, shortly describe what we have and than what I've tried to achieve said goal.



# Goal

Create a Jenkins configuration that is triggered when a PR is opened or manually to build the master branch for several OS / compiler configurations.


# What we have

We currently make a distinction between the 'all' and 'pr' builds. The 'all' job triggers builds for gcc, clang, msvc. These are separate items/jobs that each have their own build matrix and configuration.

In contrast, the 'pr' configuration has its own build matrix and as a result can only specify one build configuration which limits it to builds with one compiler since we can't use the same cmake generator on Windows and Linux. It also can't call the same builds as the 'all' job because I haven't found a way to pass the branch and commit to pull into triggered jobs (potentially reusing the jobs of the 'all' item).


# What we tried

1. Find a way to pass branch / PR information to the compiler / OS specific stages. The build step to 'trigger/call builds on other projects' has an option to 'Pass-through Git Commit that was built' but I have no clue what it passes on and how I could integrate that in the next stage.

2. Let the 'all' and 'pr' jobs create an artifact that includes the code to build in the subsequent jobs and pass them on.

3. Move to a Pipeline job with a Jenkinsfile. My first problem here is that i don't know how to distinguish between the PR trigger and the build for the master branch to get the right code. Secondly, the parallel keyword has changed recently? Previously you could generate a map with several configuration and then call parallel and pass the map. That does not seem to work anymore, is there a new way to do this?
(In general, writing a Jenkisfiles seems to require manual build scripts where other (traditional?) Jenkins jobs offer build steps such as cmake and ctest.)



Overall, I imagine that this is not an uncommon workflow. I've searched a lot for similar problems but without success. Any help is appreciated! Maybe I overlooked something obvious?

Thanks,
Raphael

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/F618CCAE-43C7-4836-AAA0-11D9E087A666%40haw-hamburg.de.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Configuration that works for PRs and master

Cuong Tran
If you use Jenkinsfile, you can use BRANCH_NAME environment to detect whether it's from a PR or not. What you describes is pretty straight forward to do with Jenkins file.

Check out:
https://jenkins.io/doc/book/pipeline/syntax/#when


On Tuesday, January 16, 2018 at 7:52:57 AM UTC-8, Hiesgen, Raphael wrote:
Hello fellow Jenkins users,

I'll start with my goal, shortly describe what we have and than what I've tried to achieve said goal.



# Goal

Create a Jenkins configuration that is triggered when a PR is opened or manually to build the master branch for several OS / compiler configurations.


# What we have

We currently make a distinction between the 'all' and 'pr' builds. The 'all' job triggers builds for gcc, clang, msvc. These are separate items/jobs that each have their own build matrix and configuration.

In contrast, the 'pr' configuration has its own build matrix and as a result can only specify one build configuration which limits it to builds with one compiler since we can't use the same cmake generator on Windows and Linux. It also can't call the same builds as the 'all' job because I haven't found a way to pass the branch and commit to pull into triggered jobs (potentially reusing the jobs of the 'all' item).


# What we tried

1. Find a way to pass branch / PR information to the compiler / OS specific stages. The build step to 'trigger/call builds on other projects' has an option to 'Pass-through Git Commit that was built' but I have no clue what it passes on and how I could integrate that in the next stage.

2. Let the 'all' and 'pr' jobs create an artifact that includes the code to build in the subsequent jobs and pass them on.

3. Move to a Pipeline job with a Jenkinsfile. My first problem here is that i don't know how to distinguish between the PR trigger and the build for the master branch to get the right code. Secondly, the parallel keyword has changed recently? Previously you could generate a map with several configuration and then call parallel and pass the map. That does not seem to work anymore, is there a new way to do this?
(In general, writing a Jenkisfiles seems to require manual build scripts where other (traditional?) Jenkins jobs offer build steps such as cmake and ctest.)



Overall, I imagine that this is not an uncommon workflow. I've searched a lot for similar problems but without success. Any help is appreciated! Maybe I overlooked something obvious?

Thanks,
Raphael

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e29afa8b-a65a-4f98-bd9a-9037595fd547%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Configuration that works for PRs and master

Hiesgen, Raphael
Thanks for the pointer! So, the recommended way is my third option: Use a Jenkinsfile? I got a few follow up questions.

> If you use Jenkinsfile, you can use BRANCH_NAME environment to detect whether it's from a PR or not.  What you describes is pretty straight forward to do with Jenkins file.  
>
> Check out:
>      https://jenkins.io/doc/book/pipeline/syntax/#when

1. env.BRANCH_NAME always returns null for me. Using a when statement (`when { branch 'master' } }`) never evaluates to true for me, but I don't know what my branch is to debug this. I expected it to be master.

2. Since I have at least 8 different tags, I have to write 16+ stages if I can only use when. Is there a way to write a function to generate stages? Most of them use the same steps besides the label of their agents, in addition to the when differentiation.

Regards,
Raphael

> On Tuesday, January 16, 2018 at 7:52:57 AM UTC-8, Hiesgen, Raphael wrote:
> Hello fellow Jenkins users,
>
> I'll start with my goal, shortly describe what we have and than what I've tried to achieve said goal.
>
>
>
> # Goal
>
> Create a Jenkins configuration that is triggered when a PR is opened or manually to build the master branch for several OS / compiler configurations.
>
>
> # What we have
>
> We currently make a distinction between the 'all' and 'pr' builds. The 'all' job triggers builds for gcc, clang, msvc. These are separate items/jobs that each have their own build matrix and configuration.
>
> In contrast, the 'pr' configuration has its own build matrix and as a result can only specify one build configuration which limits it to builds with one compiler since we can't use the same cmake generator on Windows and Linux. It also can't call the same builds as the 'all' job because I haven't found a way to pass the branch and commit to pull into triggered jobs (potentially reusing the jobs of the 'all' item).
>
>
> # What we tried
>
> 1. Find a way to pass branch / PR information to the compiler / OS specific stages. The build step to 'trigger/call builds on other projects' has an option to 'Pass-through Git Commit that was built' but I have no clue what it passes on and how I could integrate that in the next stage.
>
> 2. Let the 'all' and 'pr' jobs create an artifact that includes the code to build in the subsequent jobs and pass them on.
>
> 3. Move to a Pipeline job with a Jenkinsfile. My first problem here is that i don't know how to distinguish between the PR trigger and the build for the master branch to get the right code. Secondly, the parallel keyword has changed recently? Previously you could generate a map with several configuration and then call parallel and pass the map. That does not seem to work anymore, is there a new way to do this?
> (In general, writing a Jenkisfiles seems to require manual build scripts where other (traditional?) Jenkins jobs offer build steps such as cmake and ctest.)
>
>
>
> Overall, I imagine that this is not an uncommon workflow. I've searched a lot for similar problems but without success. Any help is appreciated! Maybe I overlooked something obvious?
>
> Thanks,
> Raphael
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e29afa8b-a65a-4f98-bd9a-9037595fd547%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/B7CC039A-6670-4D60-B68D-524EAA21D79F%40haw-hamburg.de.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Configuration that works for PRs and master

Steven Foster
env.BRANCH_NAME is only populated when your job is from a Multibranch Project type (which is probably a good project type to use for this kind of building)

On Wednesday, January 17, 2018 at 10:45:53 AM UTC, Hiesgen, Raphael wrote:
Thanks for the pointer! So, the recommended way is my third option: Use a Jenkinsfile? I got a few follow up questions.

> If you use Jenkinsfile, you can use BRANCH_NAME environment to detect whether it's from a PR or not.  What you describes is pretty straight forward to do with Jenkins file.  
>
> Check out:
>      <a href="https://jenkins.io/doc/book/pipeline/syntax/#when" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fjenkins.io%2Fdoc%2Fbook%2Fpipeline%2Fsyntax%2F%23when\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFco5PSosOD3Kz-Rodzl2QyCfD1dg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fjenkins.io%2Fdoc%2Fbook%2Fpipeline%2Fsyntax%2F%23when\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFco5PSosOD3Kz-Rodzl2QyCfD1dg&#39;;return true;">https://jenkins.io/doc/book/pipeline/syntax/#when

1. env.BRANCH_NAME always returns null for me. Using a when statement (`when { branch 'master' } }`) never evaluates to true for me, but I don't know what my branch is to debug this. I expected it to be master.

2. Since I have at least 8 different tags, I have to write 16+ stages if I can only use when. Is there a way to write a function to generate stages? Most of them use the same steps besides the label of their agents, in addition to the when differentiation.

Regards,
Raphael

> On Tuesday, January 16, 2018 at 7:52:57 AM UTC-8, Hiesgen, Raphael wrote:
> Hello fellow Jenkins users,
>
> I'll start with my goal, shortly describe what we have and than what I've tried to achieve said goal.
>
>
>
> # Goal
>
> Create a Jenkins configuration that is triggered when a PR is opened or manually to build the master branch for several OS / compiler configurations.
>
>
> # What we have
>
> We currently make a distinction between the 'all' and 'pr' builds. The 'all' job triggers builds for gcc, clang, msvc. These are separate items/jobs that each have their own build matrix and configuration.
>
> In contrast, the 'pr' configuration has its own build matrix and as a result can only specify one build configuration which limits it to builds with one compiler since we can't use the same cmake generator on Windows and Linux. It also can't call the same builds as the 'all' job because I haven't found a way to pass the branch and commit to pull into triggered jobs (potentially reusing the jobs of the 'all' item).
>
>
> # What we tried
>
> 1. Find a way to pass branch / PR information to the compiler / OS specific stages. The build step to 'trigger/call builds on other projects' has an option to 'Pass-through Git Commit that was built' but I have no clue what it passes on and how I could integrate that in the next stage.
>
> 2. Let the 'all' and 'pr' jobs create an artifact that includes the code to build in the subsequent jobs and pass them on.
>
> 3. Move to a Pipeline job with a Jenkinsfile. My first problem here is that i don't know how to distinguish between the PR trigger and the build for the master branch to get the right code. Secondly, the parallel keyword has changed recently? Previously you could generate a map with several configuration and then call parallel and pass the map. That does not seem to work anymore, is there a new way to do this?
> (In general, writing a Jenkisfiles seems to require manual build scripts where other (traditional?) Jenkins jobs offer build steps such as cmake and ctest.)
>
>
>
> Overall, I imagine that this is not an uncommon workflow. I've searched a lot for similar problems but without success. Any help is appreciated! Maybe I overlooked something obvious?
>
> Thanks,
> Raphael
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="STlIKNFeAQAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkinsci-use...@googlegroups.com.
> To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-users/e29afa8b-a65a-4f98-bd9a-9037595fd547%40googlegroups.com" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/e29afa8b-a65a-4f98-bd9a-9037595fd547%40googlegroups.com&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-users/e29afa8b-a65a-4f98-bd9a-9037595fd547%40googlegroups.com&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-users/e29afa8b-a65a-4f98-bd9a-9037595fd547%40googlegroups.com.
> For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/754f7e7a-0be1-4a8d-adb9-e15ddbeb04c2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Configuration that works for PRs and master

Hiesgen, Raphael
Thanks, that makes things a bit easier! Got the branch stuff now, although checkout scm just pull the right things, I guess.

Still looking for an easier approach to the parallelization. But, more importantly, unlike the other 'pipeline' project, this one doesn't have a configuration section for the GitHub Pull Request Builder and as a result I cannot specify a trigger phrase or limit the users that can trigger a built. Step by step.

> On 17. Jan 2018, at 11:59 AM, Steven Foster <[hidden email]> wrote:
>
> env.BRANCH_NAME is only populated when your job is from a Multibranch Project type (which is probably a good project type to use for this kind of building)
>
> On Wednesday, January 17, 2018 at 10:45:53 AM UTC, Hiesgen, Raphael wrote:
> Thanks for the pointer! So, the recommended way is my third option: Use a Jenkinsfile? I got a few follow up questions.
>
> > If you use Jenkinsfile, you can use BRANCH_NAME environment to detect whether it's from a PR or not.  What you describes is pretty straight forward to do with Jenkins file.  
> >
> > Check out:
> >      https://jenkins.io/doc/book/pipeline/syntax/#when 
>
> 1. env.BRANCH_NAME always returns null for me. Using a when statement (`when { branch 'master' } }`) never evaluates to true for me, but I don't know what my branch is to debug this. I expected it to be master.
>
> 2. Since I have at least 8 different tags, I have to write 16+ stages if I can only use when. Is there a way to write a function to generate stages? Most of them use the same steps besides the label of their agents, in addition to the when differentiation.
>
> Regards,
> Raphael
>
> > On Tuesday, January 16, 2018 at 7:52:57 AM UTC-8, Hiesgen, Raphael wrote:
> > Hello fellow Jenkins users,
> >
> > I'll start with my goal, shortly describe what we have and than what I've tried to achieve said goal.
> >
> >
> >
> > # Goal
> >
> > Create a Jenkins configuration that is triggered when a PR is opened or manually to build the master branch for several OS / compiler configurations.
> >
> >
> > # What we have
> >
> > We currently make a distinction between the 'all' and 'pr' builds. The 'all' job triggers builds for gcc, clang, msvc. These are separate items/jobs that each have their own build matrix and configuration.
> >
> > In contrast, the 'pr' configuration has its own build matrix and as a result can only specify one build configuration which limits it to builds with one compiler since we can't use the same cmake generator on Windows and Linux. It also can't call the same builds as the 'all' job because I haven't found a way to pass the branch and commit to pull into triggered jobs (potentially reusing the jobs of the 'all' item).
> >
> >
> > # What we tried
> >
> > 1. Find a way to pass branch / PR information to the compiler / OS specific stages. The build step to 'trigger/call builds on other projects' has an option to 'Pass-through Git Commit that was built' but I have no clue what it passes on and how I could integrate that in the next stage.
> >
> > 2. Let the 'all' and 'pr' jobs create an artifact that includes the code to build in the subsequent jobs and pass them on.
> >
> > 3. Move to a Pipeline job with a Jenkinsfile. My first problem here is that i don't know how to distinguish between the PR trigger and the build for the master branch to get the right code. Secondly, the parallel keyword has changed recently? Previously you could generate a map with several configuration and then call parallel and pass the map. That does not seem to work anymore, is there a new way to do this?
> > (In general, writing a Jenkisfiles seems to require manual build scripts where other (traditional?) Jenkins jobs offer build steps such as cmake and ctest.)
> >
> >
> >
> > Overall, I imagine that this is not an uncommon workflow. I've searched a lot for similar problems but without success. Any help is appreciated! Maybe I overlooked something obvious?
> >
> > Thanks,
> > Raphael
> >
> > --
> > You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> > To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e29afa8b-a65a-4f98-bd9a-9037595fd547%40googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/754f7e7a-0be1-4a8d-adb9-e15ddbeb04c2%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/6171D679-2BFA-4E99-A886-2D64F6391681%40haw-hamburg.de.
For more options, visit https://groups.google.com/d/optout.