Is there Python support for ci.jenkins.io plugin builds?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Is there Python support for ci.jenkins.io plugin builds?

Chris Kilding
My plugin uses an external dependency, Moto (the AWS mock server) for its integration tests.

Moto is written in Python, installed with Pip, in a virtualenv. It can be used either directly (where we must do the pip install ourselves) or through the Java wrapper Localstack (which does little more than run those same commands for us).

It appears that ci.jenkins.io agents have *some* kind of Python toolchain installed, but it’s limited: there is no ‘pip’ and maybe no ‘virtualenv’ either.

Do any other Jenkins plugins use Python or Python dependencies, and if so how are Python/Pip/Virtualenv set up in their builds?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/a3dece82-07e7-406d-984c-7fa37ec49f1e%40www.fastmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Is there Python support for ci.jenkins.io plugin builds?

R. Tyler Croy
(replies inline)

On Wed, 15 May 2019, Chris Kilding wrote:

> My plugin uses an external dependency, Moto (the AWS mock server) for its
> integration tests.
>
> Moto is written in Python, installed with Pip, in a virtualenv. It can be used
> either directly (where we must do the pip install ourselves) or through the
> Java wrapper Localstack (which does little more than run those same commands
> for us).
>
> It appears that ci.jenkins.io agents have *some* kind of Python toolchain
> installed, but it’s limited: there is no ‘pip’ and maybe no ‘virtualenv’
> either.
>
> Do any other Jenkins plugins use Python or Python dependencies, and if so how
> are Python/Pip/Virtualenv set up in their builds?


The fact that there is Python is incidental more than anything. We will not be
installing any system packages on agents however, so I suggest working out how
to use Docker containers to meet your needs, as those are definitely supported
on machines with the 'docker' label.

Cheers.

--
GitHub:  https://github.com/rtyler

GPG Key ID: 0F2298A980EE31ACCA0A7825E5C92681BEF6CEA2

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20190515163618.jgyeizkbqejmhbpg%40grape.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Is there Python support for ci.jenkins.io plugin builds?

Chris Kilding
Yep, that sounds reasonable. (I had suspected as much - there would be no end to the possible system packages that plugin developers might want.)

Do the ‘docker’-labelled Jenkins agents support the usual ‘dockerfile’ directive to let the build define a Docker image ad-hoc? Or do all images have to be pre-packed and hosted on a registry?

Chris

On Wed, 15 May 2019, at 5:36 PM, R. Tyler Croy wrote:
(replies inline)

On Wed, 15 May 2019, Chris Kilding wrote:

> My plugin uses an external dependency, Moto (the AWS mock server) for its
> integration tests.

> Moto is written in Python, installed with Pip, in a virtualenv. It can be used
> either directly (where we must do the pip install ourselves) or through the
> Java wrapper Localstack (which does little more than run those same commands
> for us).

> It appears that ci.jenkins.io agents have *some* kind of Python toolchain
> installed, but it’s limited: there is no ‘pip’ and maybe no ‘virtualenv’
> either.

> Do any other Jenkins plugins use Python or Python dependencies, and if so how
> are Python/Pip/Virtualenv set up in their builds?


The fact that there is Python is incidental more than anything. We will not be
installing any system packages on agents however, so I suggest working out how
to use Docker containers to meet your needs, as those are definitely supported
on machines with the 'docker' label.

Cheers.

--
GitHub:  https://github.com/rtyler

GPG Key ID: 0F2298A980EE31ACCA0A7825E5C92681BEF6CEA2

-- 
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/20190515163618.jgyeizkbqejmhbpg%40grape.
For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/85c23396-ad07-4791-8a65-ef4795f8c852%40www.fastmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Is there Python support for ci.jenkins.io plugin builds?

R. Tyler Croy-2
(replies inline)

On Wed, 15 May 2019, Chris Kilding wrote:

> Yep, that sounds reasonable. (I had suspected as much - there would be no end
> to the possible system packages that plugin developers might want.)
>
> Do the ‘docker’-labelled Jenkins agents support the usual ‘dockerfile’
> directive to let the build define a Docker image ad-hoc? Or do all images have
> to be pre-packed and hosted on a registry?


The 'docker' labeled agents are all full VMs and the pipelines do have full
access to the Docker socket, so Dockerfiles should be fine.



--
GitHub:  https://github.com/rtyler

GPG Key ID: 0F2298A980EE31ACCA0A7825E5C92681BEF6CEA2

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/WOA6p30q5mgycsLk5VZWSBiSey1OiqQD4rFxG5C_2j1RKEGXDJYRZKpdymzN5Sl2z8BnWWsD7k9XBQvtlLx6PX8FVVDESCa8G4-N2vd-Xng%3D%40brokenco.de.
For more options, visit https://groups.google.com/d/optout.

signature.asc (891 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there Python support for ci.jenkins.io plugin builds?

Chris Kilding
I was able to substitute in the fabric8 Docker Maven Plugin to run Moto. It seems to work fine on the Linux build agent, but the Windows agent doesn’t like it (no Docker host or docker.sock available.)

Is there another way to get Docker support on the Windows agent, or is this ‘in the works’ for the future?

Chris

On Wed, 15 May 2019, at 6:45 PM, R. Tyler Croy wrote:
(replies inline)

On Wed, 15 May 2019, Chris Kilding wrote:

> Yep, that sounds reasonable. (I had suspected as much - there would be no end
> to the possible system packages that plugin developers might want.)

> Do the ‘docker’-labelled Jenkins agents support the usual ‘dockerfile’
> directive to let the build define a Docker image ad-hoc? Or do all images have
> to be pre-packed and hosted on a registry?


The 'docker' labeled agents are all full VMs and the pipelines do have full
access to the Docker socket, so Dockerfiles should be fine.



--
GitHub:  https://github.com/rtyler

GPG Key ID: 0F2298A980EE31ACCA0A7825E5C92681BEF6CEA2

-- 
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/WOA6p30q5mgycsLk5VZWSBiSey1OiqQD4rFxG5C_2j1RKEGXDJYRZKpdymzN5Sl2z8BnWWsD7k9XBQvtlLx6PX8FVVDESCa8G4-N2vd-Xng%3D%40brokenco.de.
For more options, visit https://groups.google.com/d/optout.


Attachments:
  • signature.asc

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/0b556d65-974f-40c3-a9d7-700f59717543%40www.fastmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Is there Python support for ci.jenkins.io plugin builds?

R. Tyler Croy-2
(replies inline)

On Thu, 16 May 2019, Chris Kilding wrote:

> I was able to substitute in the fabric8 Docker Maven Plugin to run Moto. It
> seems to work fine on the Linux build agent, but the Windows agent doesn’t like
> it (no Docker host or docker.sock available.)
>
> Is there another way to get Docker support on the Windows agent, or is this ‘in
> the works’ for the future?


Neither unfortunately. There is an infrastructure ticket which I don't have any
plans to get to: https://issues.jenkins.io/browse/INFRA-1400




--
GitHub:  https://github.com/rtyler

GPG Key ID: 0F2298A980EE31ACCA0A7825E5C92681BEF6CEA2

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/h8ekP_4C4XNiCREsuXDB-OSeihblF-Vlx7Cgil7rQLyirqSDJZdeg5ToeRNSbFT9023KLpj-0EJ78Xwp7CCENla3BkgJzxJFzaEfwzQ82LM%3D%40brokenco.de.
For more options, visit https://groups.google.com/d/optout.

signature.asc (891 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Is there Python support for ci.jenkins.io plugin builds?

Joseph P
In reply to this post by Chris Kilding
We use testcontainers in JCasC with great success and we are able to run those tests on Windows (if only they weren't so unstable)

We use assumeTrue to disable tests when docker is not available.

See here: https://github.com/jenkinsci/configuration-as-code-plugin/blob/791fccccee41018fbe9e00c32409aadc92eb30df/plugin/src/test/java/io/jenkins/plugins/casc/vault/VaultSecretSourceTest.java#L66

On Thursday, May 16, 2019 at 2:23:55 PM UTC+2, Chris Kilding wrote:
I was able to substitute in the fabric8 Docker Maven Plugin to run Moto. It seems to work fine on the Linux build agent, but the Windows agent doesn’t like it (no Docker host or docker.sock available.)

Is there another way to get Docker support on the Windows agent, or is this ‘in the works’ for the future?

Chris

On Wed, 15 May 2019, at 6:45 PM, R. Tyler Croy wrote:
(replies inline)

On Wed, 15 May 2019, Chris Kilding wrote:

> Yep, that sounds reasonable. (I had suspected as much - there would be no end
> to the possible system packages that plugin developers might want.)

> Do the ‘docker’-labelled Jenkins agents support the usual ‘dockerfile’
> directive to let the build define a Docker image ad-hoc? Or do all images have
> to be pre-packed and hosted on a registry?


The 'docker' labeled agents are all full VMs and the pipelines do have full
access to the Docker socket, so Dockerfiles should be fine.



--
GitHub:  <a href="https://github.com/rtyler" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frtyler\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFtsnCUZ085B8982iTQ2KFqz5gYhw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frtyler\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFtsnCUZ085B8982iTQ2KFqz5gYhw&#39;;return true;">https://github.com/rtyler

GPG Key ID: 0F2298A980EE31ACCA0A7825E5C92681BEF6CEA2

-- 
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="1WqD9dbTAgAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">jenkin...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/jenkinsci-dev/WOA6p30q5mgycsLk5VZWSBiSey1OiqQD4rFxG5C_2j1RKEGXDJYRZKpdymzN5Sl2z8BnWWsD7k9XBQvtlLx6PX8FVVDESCa8G4-N2vd-Xng%3D%40brokenco.de" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-dev/WOA6p30q5mgycsLk5VZWSBiSey1OiqQD4rFxG5C_2j1RKEGXDJYRZKpdymzN5Sl2z8BnWWsD7k9XBQvtlLx6PX8FVVDESCa8G4-N2vd-Xng%3D%40brokenco.de&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/jenkinsci-dev/WOA6p30q5mgycsLk5VZWSBiSey1OiqQD4rFxG5C_2j1RKEGXDJYRZKpdymzN5Sl2z8BnWWsD7k9XBQvtlLx6PX8FVVDESCa8G4-N2vd-Xng%3D%40brokenco.de&#39;;return true;">https://groups.google.com/d/msgid/jenkinsci-dev/WOA6p30q5mgycsLk5VZWSBiSey1OiqQD4rFxG5C_2j1RKEGXDJYRZKpdymzN5Sl2z8BnWWsD7k9XBQvtlLx6PX8FVVDESCa8G4-N2vd-Xng%3D%40brokenco.de.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.


Attachments:
  • signature.asc

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/d4ed2144-0377-4fa0-9a8b-0578bbc01bb6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.