Customize node creation from within a pipeline definition

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Customize node creation from within a pipeline definition

Jonathan Ballet
Hi,

I'm looking how possible it would be to allow the customization of a new Jenkins worker based on the definition of a node within a Jenkinsfile file.

Considering a node provisioner which is able to create new nodes from Docker images for example, I would like to be able to do something similar to this:

node("my-node-provisioner", docker_image="internal/jenkins-worker:v12", memory_limit="4GB") {
   
// my steps here
}

That would allow the developers to customize the context in which their job is running in, within the available node provisioner possibilities, while keeping all the versioning on how the job needs to be executed within the Jenkinsfile of the project.

Some more context about what I'd like to do: I'm using the Nomad plugin to schedule our Jenkins jobs in the pool of machines we are managing via the Nomad scheduler. Nodes ("slave templates" in the Nomad plugin terminology) can be configured from Jenkins main configuration screen, but it's not that convenient. See my RFC here for even more details: https://github.com/jenkinsci/nomad-plugin/issues/20

It's my first time digging inside Jenkins extensions and I don't really know where I should start to look for. Any pointers would be appreciated! I'm especially interested to know if it's something which has been done before, and if it's desirable or not.

Thanks!

 Jonathan

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/3bfab27d-79ab-427b-b67b-ad69a6830001%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Customize node creation from within a pipeline definition

Jesse Glick-4
The argument to `node` is a label expression, and the `Cloud` SPI
allows interpretation of labels, so you could probably write something
like

node('nomad && nomaddocker:internal/jenkins-worker:v12 &&
nomadmemlimit:4Gb') {…}

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr166Uu4%2BjQn%2By5_Yy8%3DzuvBor_ADLqidr0K_0%2BzyWG7Tg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Customize node creation from within a pipeline definition

Carlos Sanchez

The kubernetes plugin does that, so you can check for inspiration
https://github.com/jenkinsci/kubernetes-plugin/blob/master/README.md#pipeline-support

It would be nice to have a common syntax across container provisioners


On Tue, Jul 25, 2017, 14:55 Jesse Glick <[hidden email]> wrote:
The argument to `node` is a label expression, and the `Cloud` SPI
allows interpretation of labels, so you could probably write something
like

node('nomad && nomaddocker:internal/jenkins-worker:v12 &&
nomadmemlimit:4Gb') {…}

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr166Uu4%2BjQn%2By5_Yy8%3DzuvBor_ADLqidr0K_0%2BzyWG7Tg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CALHFn6MuxFMor-DnOtaBS%3DBE60-88tZ%3D6EBedsHfBbTjpMyr5A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Customize node creation from within a pipeline definition

Jonathan Ballet
In reply to this post by Jesse Glick-4
Le mardi 25 juillet 2017 14:54:56 UTC+2, Jesse Glick a écrit :
The argument to `node` is a label expression, and the `Cloud` SPI
allows interpretation of labels, so you could probably write something
like

node('nomad && nomaddocker:internal/jenkins-worker:v12 &&
nomadmemlimit:4Gb') {…}

Right, I see exactly where it's being done. It looks a bit hackish from the outside, but that could indeed be a possible solution :)

Do you know other plugins which are doing something like this? I'd like to know which kind of syntax or what uses they are making of these labels!

Thanks for the reply,

 Jonathan

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/f5b57f1e-ccec-4f34-9441-bcf40efc1ec9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Customize node creation from within a pipeline definition

Jonathan Ballet
In reply to this post by Carlos Sanchez
Le mardi 25 juillet 2017 15:05:01 UTC+2, Carlos Sanchez a écrit :

The kubernetes plugin does that, so you can check for inspiration
<a href="https://github.com/jenkinsci/kubernetes-plugin/blob/master/README.md#pipeline-support" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fkubernetes-plugin%2Fblob%2Fmaster%2FREADME.md%23pipeline-support\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFtYufT1VmHJtVbQpqPACJovwh_xg&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fjenkinsci%2Fkubernetes-plugin%2Fblob%2Fmaster%2FREADME.md%23pipeline-support\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFtYufT1VmHJtVbQpqPACJovwh_xg&#39;;return true;">https://github.com/jenkinsci/kubernetes-plugin/blob/master/README.md#pipeline-support

It would be nice to have a common syntax across container provisioners


 That really looks like what I wanted to implement! I didn't thought of wrapping the node() call inside something which would define the environment to build in, that's definitively interesting, I'll try to play with that I guess.... :)

As for the common syntax, I'm just looking now at what you guys did for Kubernetes, I really like that example: https://github.com/jenkinsci/kubernetes-plugin/blob/master/examples/declarative.groovy but I haven't done any Jenkins plugin before, so there's still lot of work coming in :)

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/90d963e5-0f7e-4d58-8365-1347a0eccfa4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Customize node creation from within a pipeline definition

Jesse Glick-4
In reply to this post by Carlos Sanchez
On Tue, Jul 25, 2017 at 9:04 AM, Carlos Sanchez <[hidden email]> wrote:
> The kubernetes plugin does that

I think it is doing something different than what I suggested; useful
only if you need to orchestrate several containers. Would be overkill
for the use case Jonathan was discussing.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr25t%2BfFd8Y1ykV%3DWVCTvQ-YU0dPzH_z5FEvav263PK36w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.